qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results/nemotron3-nano-30b.server.log

995 lines
95 KiB
Plaintext
Raw Permalink Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.067.074 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.067.078 I device_info:
0.00.067.097 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.067.102 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.067.111 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.067.168 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.067.172 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.067.213 I srv init: running without SSL
0.00.067.273 I srv init: using 23 threads for HTTP server
0.00.067.597 I srv start: binding port with default address family
0.00.068.799 I srv llama_server: loading model
0.00.068.806 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/Nemotron-3-Nano-30B-A3B-Q5_K_M.gguf'
0.00.068.854 I common_init_result: fitting params to device memory ...
0.00.068.856 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.19.264.606 W llama_context: n_ctx_seq (8192) < n_ctx_train (1048576) -- the full capacity of the model will not be utilized
0.19.368.958 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.19.556.763 I srv load_model: initializing slots, n_slots = 4
0.19.588.681 W srv load_model: speculative decoding will use checkpoints
0.19.588.700 W common_speculative_init: no implementations specified for speculative decoding
0.19.588.700 I slot load_model: id 0 | task -1 | new slot, n_ctx = 8192
0.19.588.704 I slot load_model: id 1 | task -1 | new slot, n_ctx = 8192
0.19.588.705 I slot load_model: id 2 | task -1 | new slot, n_ctx = 8192
0.19.588.705 I slot load_model: id 3 | task -1 | new slot, n_ctx = 8192
0.19.588.761 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.19.588.761 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.19.588.761 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.19.588.761 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.19.588.779 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.19.623.813 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
<think></think>Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
<think>
'
0.19.652.621 I srv init: init: chat template, thinking = 1
0.19.653.243 I srv llama_server: model loaded
0.19.653.248 I srv llama_server: server is listening on http://127.0.0.1:8090
0.19.653.253 I srv update_slots: all slots are idle
0.19.993.510 I srv operator(): Chat format: peg-native
0.19.993.799 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.19.993.802 I srv get_availabl: updating prompt cache
0.19.993.808 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.19.993.813 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.19.993.814 I srv get_availabl: prompt cache update took 0.01 ms
0.19.993.917 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.19.993.919 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.19.993.920 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.19.993.935 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.19.993.935 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.19.993.941 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.19.993.941 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.20.755.401 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 46, pos_max = 46, n_tokens = 47, size = 47.618 MiB)
0.21.510.004 I slot print_timing: id 3 | task 0 | prompt eval time = 919.18 ms / 76 tokens ( 12.09 ms per token, 82.68 tokens per second)
0.21.510.007 I slot print_timing: id 3 | task 0 | eval time = 596.85 ms / 50 tokens ( 11.94 ms per token, 83.77 tokens per second)
0.21.510.008 I slot print_timing: id 3 | task 0 | total time = 1516.04 ms / 126 tokens
0.21.510.012 I slot print_timing: id 3 | task 0 | graphs reused = 49
0.21.510.051 I slot release: id 3 | task 0 | stop processing: n_tokens = 125, truncated = 0
0.21.510.055 I srv update_slots: all slots are idle
0.21.541.116 I srv operator(): Chat format: peg-native
0.21.541.350 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.746 (> 0.100 thold), f_keep = 0.400
0.21.541.352 I srv get_availabl: updating prompt cache
0.21.541.376 W srv prompt_save: - saving prompt with length 125, total state size = 48.352 MiB (draft: 0.000 MiB)
0.21.630.294 I srv load: - looking for better prompt, base f_keep = 0.400, sim = 0.746
0.21.630.300 I srv update: - cache state: 1 prompts, 95.969 MiB (limits: 8192.000 MiB, 8192 tokens, 10670 est)
0.21.630.301 I srv update: - prompt 0x5607578c8c20: 125 tokens, checkpoints: 1, 95.969 MiB
0.21.630.302 I srv get_availabl: prompt cache update took 88.95 ms
0.21.630.402 I slot launch_slot_: id 3 | task 53 | processing task, is_child = 0
0.21.630.403 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.21.630.404 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.21.630.410 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.21.630.410 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.21.630.420 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.21.630.421 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.21.630.438 I slot update_slots: id 3 | task 53 | Checking checkpoint with [46, 46] against 50...
0.21.642.429 W slot update_slots: id 3 | task 53 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.22.578.013 I slot print_timing: id 3 | task 53 | prompt eval time = 114.42 ms / 20 tokens ( 5.72 ms per token, 174.79 tokens per second)
0.22.578.017 I slot print_timing: id 3 | task 53 | eval time = 833.15 ms / 70 tokens ( 11.90 ms per token, 84.02 tokens per second)
0.22.578.017 I slot print_timing: id 3 | task 53 | total time = 947.57 ms / 90 tokens
0.22.578.018 I slot print_timing: id 3 | task 53 | graphs reused = 117
0.22.578.044 I slot release: id 3 | task 53 | stop processing: n_tokens = 136, truncated = 0
0.22.578.052 I srv update_slots: all slots are idle
0.22.608.470 I srv operator(): Chat format: peg-native
0.22.608.710 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.368
0.22.608.713 I srv get_availabl: updating prompt cache
0.22.608.730 W srv prompt_save: - saving prompt with length 136, total state size = 48.416 MiB (draft: 0.000 MiB)
0.22.687.042 I srv load: - looking for better prompt, base f_keep = 0.368, sim = 0.714
0.22.687.045 I srv load: - found better prompt with f_keep = 0.424, sim = 0.757
0.22.710.201 I srv update: - cache state: 1 prompts, 96.034 MiB (limits: 8192.000 MiB, 8192 tokens, 11601 est)
0.22.710.204 I srv update: - prompt 0x5607592b97f0: 136 tokens, checkpoints: 1, 96.034 MiB
0.22.710.205 I srv get_availabl: prompt cache update took 101.49 ms
0.22.710.301 I slot launch_slot_: id 3 | task 125 | processing task, is_child = 0
0.22.710.303 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.22.710.303 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.22.710.309 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.22.710.309 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.22.710.314 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.22.710.314 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.22.710.327 I slot update_slots: id 3 | task 125 | Checking checkpoint with [46, 46] against 53...
0.22.753.522 W slot update_slots: id 3 | task 125 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.23.543.059 I slot print_timing: id 3 | task 125 | prompt eval time = 168.02 ms / 23 tokens ( 7.31 ms per token, 136.89 tokens per second)
0.23.543.063 I slot print_timing: id 3 | task 125 | eval time = 664.71 ms / 56 tokens ( 11.87 ms per token, 84.25 tokens per second)
0.23.543.064 I slot print_timing: id 3 | task 125 | total time = 832.73 ms / 79 tokens
0.23.543.064 I slot print_timing: id 3 | task 125 | graphs reused = 171
0.23.543.092 I slot release: id 3 | task 125 | stop processing: n_tokens = 125, truncated = 0
0.23.543.100 I srv update_slots: all slots are idle
0.23.573.193 I srv operator(): Chat format: peg-native
0.23.573.428 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.739 (> 0.100 thold), f_keep = 0.408
0.23.573.428 I srv get_availabl: updating prompt cache
0.23.573.449 W srv prompt_save: - saving prompt with length 125, total state size = 48.352 MiB (draft: 0.000 MiB)
0.23.650.251 I srv load: - looking for better prompt, base f_keep = 0.408, sim = 0.739
0.23.650.256 I srv update: - cache state: 2 prompts, 192.004 MiB (limits: 8192.000 MiB, 8192 tokens, 11135 est)
0.23.650.257 I srv update: - prompt 0x5607592b97f0: 136 tokens, checkpoints: 1, 96.034 MiB
0.23.650.258 I srv update: - prompt 0x56075f59ada0: 125 tokens, checkpoints: 1, 95.969 MiB
0.23.650.259 I srv get_availabl: prompt cache update took 76.83 ms
0.23.650.348 I slot launch_slot_: id 3 | task 183 | processing task, is_child = 0
0.23.650.351 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.23.650.352 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.23.650.360 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.23.650.360 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.23.650.365 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.23.650.366 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.23.650.381 I slot update_slots: id 3 | task 183 | Checking checkpoint with [46, 46] against 51...
0.23.662.288 W slot update_slots: id 3 | task 183 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.24.366.009 I slot print_timing: id 3 | task 183 | prompt eval time = 134.91 ms / 22 tokens ( 6.13 ms per token, 163.08 tokens per second)
0.24.366.012 I slot print_timing: id 3 | task 183 | eval time = 580.72 ms / 49 tokens ( 11.85 ms per token, 84.38 tokens per second)
0.24.366.013 I slot print_timing: id 3 | task 183 | total time = 715.62 ms / 71 tokens
0.24.366.013 I slot print_timing: id 3 | task 183 | graphs reused = 218
0.24.366.039 I slot release: id 3 | task 183 | stop processing: n_tokens = 117, truncated = 0
0.24.366.046 I srv update_slots: all slots are idle
0.24.396.159 I srv operator(): Chat format: peg-native
0.24.396.394 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.718 (> 0.100 thold), f_keep = 0.436
0.24.396.397 I srv get_availabl: updating prompt cache
0.24.396.414 W srv prompt_save: - saving prompt with length 117, total state size = 48.305 MiB (draft: 0.000 MiB)
0.24.456.637 I srv load: - looking for better prompt, base f_keep = 0.436, sim = 0.718
0.24.456.643 I srv update: - cache state: 3 prompts, 287.926 MiB (limits: 8192.000 MiB, 8192 tokens, 10754 est)
0.24.456.644 I srv update: - prompt 0x5607592b97f0: 136 tokens, checkpoints: 1, 96.034 MiB
0.24.456.645 I srv update: - prompt 0x56075f59ada0: 125 tokens, checkpoints: 1, 95.969 MiB
0.24.456.645 I srv update: - prompt 0x5607593e13b0: 117 tokens, checkpoints: 1, 95.923 MiB
0.24.456.647 I srv get_availabl: prompt cache update took 60.25 ms
0.24.456.739 I slot launch_slot_: id 3 | task 234 | processing task, is_child = 0
0.24.456.741 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.24.456.742 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.24.456.748 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.24.456.749 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.24.456.753 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.24.456.754 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.24.456.765 I slot update_slots: id 3 | task 234 | Checking checkpoint with [46, 46] against 51...
0.24.468.613 W slot update_slots: id 3 | task 234 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.24.623.512 I slot print_timing: id 3 | task 234 | prompt eval time = 137.68 ms / 24 tokens ( 5.74 ms per token, 174.31 tokens per second)
0.24.623.515 I slot print_timing: id 3 | task 234 | eval time = 29.06 ms / 3 tokens ( 9.69 ms per token, 103.24 tokens per second)
0.24.623.516 I slot print_timing: id 3 | task 234 | total time = 166.74 ms / 27 tokens
0.24.623.517 I slot print_timing: id 3 | task 234 | graphs reused = 219
0.24.623.548 I slot release: id 3 | task 234 | stop processing: n_tokens = 73, truncated = 0
0.24.623.558 I srv update_slots: all slots are idle
0.24.653.808 I srv operator(): Chat format: peg-native
0.24.654.056 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.725 (> 0.100 thold), f_keep = 0.685
0.24.654.105 I slot launch_slot_: id 3 | task 239 | processing task, is_child = 0
0.24.654.107 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.24.654.108 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.24.654.114 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.24.654.114 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.24.654.119 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.24.654.119 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.24.654.128 I slot update_slots: id 3 | task 239 | Checking checkpoint with [46, 46] against 50...
0.24.666.140 W slot update_slots: id 3 | task 239 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.24.926.039 I slot print_timing: id 3 | task 239 | prompt eval time = 134.74 ms / 22 tokens ( 6.12 ms per token, 163.28 tokens per second)
0.24.926.043 I slot print_timing: id 3 | task 239 | eval time = 137.17 ms / 12 tokens ( 11.43 ms per token, 87.48 tokens per second)
0.24.926.043 I slot print_timing: id 3 | task 239 | total time = 271.91 ms / 34 tokens
0.24.926.044 I slot print_timing: id 3 | task 239 | graphs reused = 229
0.24.926.070 I slot release: id 3 | task 239 | stop processing: n_tokens = 80, truncated = 0
0.24.926.079 I srv update_slots: all slots are idle
0.24.956.770 I srv operator(): Chat format: peg-native
0.24.957.017 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.625
0.24.957.066 I slot launch_slot_: id 3 | task 253 | processing task, is_child = 0
0.24.957.068 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.24.957.068 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.24.957.074 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.24.957.075 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.24.957.079 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.24.957.079 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.24.957.087 I slot update_slots: id 3 | task 253 | Checking checkpoint with [46, 46] against 50...
0.24.969.112 W slot update_slots: id 3 | task 253 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.25.455.045 I slot print_timing: id 3 | task 253 | prompt eval time = 144.95 ms / 28 tokens ( 5.18 ms per token, 193.18 tokens per second)
0.25.455.048 I slot print_timing: id 3 | task 253 | eval time = 353.01 ms / 30 tokens ( 11.77 ms per token, 84.98 tokens per second)
0.25.455.049 I slot print_timing: id 3 | task 253 | total time = 497.95 ms / 58 tokens
0.25.455.050 I slot print_timing: id 3 | task 253 | graphs reused = 257
0.25.455.077 I slot release: id 3 | task 253 | stop processing: n_tokens = 104, truncated = 0
0.25.455.087 I srv update_slots: all slots are idle
0.25.485.216 I srv operator(): Chat format: peg-native
0.25.485.456 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.785 (> 0.100 thold), f_keep = 0.490
0.25.485.458 I srv get_availabl: updating prompt cache
0.25.485.472 W srv prompt_save: - saving prompt with length 104, total state size = 48.228 MiB (draft: 0.000 MiB)
0.25.571.441 I srv load: - looking for better prompt, base f_keep = 0.490, sim = 0.785
0.25.571.448 I srv update: - cache state: 4 prompts, 383.772 MiB (limits: 8192.000 MiB, 8192 tokens, 10288 est)
0.25.571.449 I srv update: - prompt 0x5607592b97f0: 136 tokens, checkpoints: 1, 96.034 MiB
0.25.571.450 I srv update: - prompt 0x56075f59ada0: 125 tokens, checkpoints: 1, 95.969 MiB
0.25.571.450 I srv update: - prompt 0x5607593e13b0: 117 tokens, checkpoints: 1, 95.923 MiB
0.25.571.451 I srv update: - prompt 0x56075f480ce0: 104 tokens, checkpoints: 1, 95.846 MiB
0.25.571.452 I srv get_availabl: prompt cache update took 85.99 ms
0.25.571.556 I slot launch_slot_: id 3 | task 285 | processing task, is_child = 0
0.25.571.557 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.25.571.557 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.25.571.564 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.25.571.564 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.25.571.569 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.25.571.569 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.25.571.580 I slot update_slots: id 3 | task 285 | Checking checkpoint with [46, 46] against 51...
0.25.583.358 W slot update_slots: id 3 | task 285 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.26.693.366 I slot print_timing: id 3 | task 285 | prompt eval time = 109.24 ms / 18 tokens ( 6.07 ms per token, 164.77 tokens per second)
0.26.693.370 I slot print_timing: id 3 | task 285 | eval time = 1012.54 ms / 85 tokens ( 11.91 ms per token, 83.95 tokens per second)
0.26.693.371 I slot print_timing: id 3 | task 285 | total time = 1121.78 ms / 103 tokens
0.26.693.371 I slot print_timing: id 3 | task 285 | graphs reused = 340
0.26.693.400 I slot release: id 3 | task 285 | stop processing: n_tokens = 149, truncated = 0
0.26.693.407 I srv update_slots: all slots are idle
0.26.724.096 I srv operator(): Chat format: peg-native
0.26.724.331 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.336
0.26.724.334 I srv get_availabl: updating prompt cache
0.26.724.352 W srv prompt_save: - saving prompt with length 149, total state size = 48.493 MiB (draft: 0.000 MiB)
0.26.799.336 I srv load: - looking for better prompt, base f_keep = 0.336, sim = 0.667
0.26.799.340 I srv load: - found better prompt with f_keep = 0.375, sim = 0.680
0.26.824.342 I srv update: - cache state: 4 prompts, 383.849 MiB (limits: 8192.000 MiB, 8192 tokens, 10564 est)
0.26.824.345 I srv update: - prompt 0x56075f59ada0: 125 tokens, checkpoints: 1, 95.969 MiB
0.26.824.345 I srv update: - prompt 0x5607593e13b0: 117 tokens, checkpoints: 1, 95.923 MiB
0.26.824.346 I srv update: - prompt 0x56075f480ce0: 104 tokens, checkpoints: 1, 95.846 MiB
0.26.824.346 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.26.824.348 I srv get_availabl: prompt cache update took 100.01 ms
0.26.824.449 I slot launch_slot_: id 3 | task 372 | processing task, is_child = 0
0.26.824.451 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.26.824.452 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.26.824.458 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.26.824.458 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.26.824.463 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.26.824.463 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.26.824.477 I slot update_slots: id 3 | task 372 | Checking checkpoint with [46, 46] against 51...
0.26.863.795 W slot update_slots: id 3 | task 372 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.27.386.386 I slot print_timing: id 3 | task 372 | prompt eval time = 173.38 ms / 28 tokens ( 6.19 ms per token, 161.49 tokens per second)
0.27.386.389 I slot print_timing: id 3 | task 372 | eval time = 388.52 ms / 33 tokens ( 11.77 ms per token, 84.94 tokens per second)
0.27.386.390 I slot print_timing: id 3 | task 372 | total time = 561.90 ms / 61 tokens
0.27.386.390 I slot print_timing: id 3 | task 372 | graphs reused = 371
0.27.386.418 I slot release: id 3 | task 372 | stop processing: n_tokens = 107, truncated = 0
0.27.386.425 I srv update_slots: all slots are idle
0.27.416.563 I srv operator(): Chat format: peg-native
0.27.416.800 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.467
0.27.416.802 I srv get_availabl: updating prompt cache
0.27.416.824 W srv prompt_save: - saving prompt with length 107, total state size = 48.246 MiB (draft: 0.000 MiB)
0.27.475.841 I srv load: - looking for better prompt, base f_keep = 0.467, sim = 0.714
0.27.475.845 I srv load: - found better prompt with f_keep = 0.490, sim = 0.729
0.27.499.886 I srv update: - cache state: 4 prompts, 383.866 MiB (limits: 8192.000 MiB, 8192 tokens, 10627 est)
0.27.499.889 I srv update: - prompt 0x56075f59ada0: 125 tokens, checkpoints: 1, 95.969 MiB
0.27.499.890 I srv update: - prompt 0x5607593e13b0: 117 tokens, checkpoints: 1, 95.923 MiB
0.27.499.890 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.27.499.891 I srv update: - prompt 0x5607592d1090: 107 tokens, checkpoints: 1, 95.864 MiB
0.27.499.892 I srv get_availabl: prompt cache update took 83.09 ms
0.27.500.053 I slot launch_slot_: id 3 | task 407 | processing task, is_child = 0
0.27.500.057 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.27.500.058 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.27.500.065 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.27.500.066 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.27.500.071 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.27.500.072 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.27.500.088 I slot update_slots: id 3 | task 407 | Checking checkpoint with [46, 46] against 51...
0.27.538.789 W slot update_slots: id 3 | task 407 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.28.319.397 I slot print_timing: id 3 | task 407 | prompt eval time = 166.04 ms / 23 tokens ( 7.22 ms per token, 138.52 tokens per second)
0.28.319.401 I slot print_timing: id 3 | task 407 | eval time = 653.26 ms / 55 tokens ( 11.88 ms per token, 84.19 tokens per second)
0.28.319.401 I slot print_timing: id 3 | task 407 | total time = 819.30 ms / 78 tokens
0.28.319.402 I slot print_timing: id 3 | task 407 | graphs reused = 424
0.28.319.430 I slot release: id 3 | task 407 | stop processing: n_tokens = 124, truncated = 0
0.28.319.438 I srv update_slots: all slots are idle
0.28.349.679 I srv operator(): Chat format: peg-native
0.28.349.917 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.403
0.28.349.920 I srv get_availabl: updating prompt cache
0.28.349.940 W srv prompt_save: - saving prompt with length 124, total state size = 48.346 MiB (draft: 0.000 MiB)
0.28.432.758 I srv load: - looking for better prompt, base f_keep = 0.403, sim = 0.667
0.28.432.764 I srv update: - cache state: 5 prompts, 479.830 MiB (limits: 8192.000 MiB, 8192 tokens, 10619 est)
0.28.432.765 I srv update: - prompt 0x56075f59ada0: 125 tokens, checkpoints: 1, 95.969 MiB
0.28.432.766 I srv update: - prompt 0x5607593e13b0: 117 tokens, checkpoints: 1, 95.923 MiB
0.28.432.766 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.28.432.767 I srv update: - prompt 0x5607592d1090: 107 tokens, checkpoints: 1, 95.864 MiB
0.28.432.767 I srv update: - prompt 0x560759284e40: 124 tokens, checkpoints: 1, 95.964 MiB
0.28.432.768 I srv get_availabl: prompt cache update took 82.85 ms
0.28.432.866 I slot launch_slot_: id 3 | task 464 | processing task, is_child = 0
0.28.432.866 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.28.432.867 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.28.432.874 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.28.432.874 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.28.432.879 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.28.432.879 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.28.432.893 I slot update_slots: id 3 | task 464 | Checking checkpoint with [46, 46] against 50...
0.28.444.901 W slot update_slots: id 3 | task 464 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.28.932.398 I slot print_timing: id 3 | task 464 | prompt eval time = 146.47 ms / 28 tokens ( 5.23 ms per token, 191.17 tokens per second)
0.28.932.402 I slot print_timing: id 3 | task 464 | eval time = 353.03 ms / 30 tokens ( 11.77 ms per token, 84.98 tokens per second)
0.28.932.403 I slot print_timing: id 3 | task 464 | total time = 499.50 ms / 58 tokens
0.28.932.403 I slot print_timing: id 3 | task 464 | graphs reused = 452
0.28.932.431 I slot release: id 3 | task 464 | stop processing: n_tokens = 104, truncated = 0
0.28.932.439 I srv update_slots: all slots are idle
0.28.962.555 I srv operator(): Chat format: peg-native
0.28.962.796 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.481
0.28.962.798 I srv get_availabl: updating prompt cache
0.28.962.815 W srv prompt_save: - saving prompt with length 104, total state size = 48.228 MiB (draft: 0.000 MiB)
0.29.032.628 I srv load: - looking for better prompt, base f_keep = 0.481, sim = 0.714
0.29.032.635 I srv update: - cache state: 6 prompts, 575.676 MiB (limits: 8192.000 MiB, 8192 tokens, 10331 est)
0.29.032.636 I srv update: - prompt 0x56075f59ada0: 125 tokens, checkpoints: 1, 95.969 MiB
0.29.032.637 I srv update: - prompt 0x5607593e13b0: 117 tokens, checkpoints: 1, 95.923 MiB
0.29.032.637 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.29.032.638 I srv update: - prompt 0x5607592d1090: 107 tokens, checkpoints: 1, 95.864 MiB
0.29.032.638 I srv update: - prompt 0x560759284e40: 124 tokens, checkpoints: 1, 95.964 MiB
0.29.032.639 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.29.032.640 I srv get_availabl: prompt cache update took 69.84 ms
0.29.032.731 I slot launch_slot_: id 3 | task 496 | processing task, is_child = 0
0.29.032.731 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.032.732 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.032.739 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.032.739 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.032.744 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.032.744 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.29.032.756 I slot update_slots: id 3 | task 496 | Checking checkpoint with [46, 46] against 50...
0.29.044.595 W slot update_slots: id 3 | task 496 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.29.581.402 I slot print_timing: id 3 | task 496 | prompt eval time = 135.99 ms / 23 tokens ( 5.91 ms per token, 169.13 tokens per second)
0.29.581.407 I slot print_timing: id 3 | task 496 | eval time = 412.64 ms / 35 tokens ( 11.79 ms per token, 84.82 tokens per second)
0.29.581.407 I slot print_timing: id 3 | task 496 | total time = 548.64 ms / 58 tokens
0.29.581.408 I slot print_timing: id 3 | task 496 | graphs reused = 485
0.29.581.436 I slot release: id 3 | task 496 | stop processing: n_tokens = 104, truncated = 0
0.29.581.447 I srv update_slots: all slots are idle
0.29.611.694 I srv operator(): Chat format: peg-native
0.29.611.910 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.694 (> 0.100 thold), f_keep = 0.481
0.29.611.913 I srv get_availabl: updating prompt cache
0.29.611.932 W srv prompt_save: - saving prompt with length 104, total state size = 48.228 MiB (draft: 0.000 MiB)
0.29.671.577 I srv load: - looking for better prompt, base f_keep = 0.481, sim = 0.694
0.29.671.586 I srv update: - cache state: 7 prompts, 671.522 MiB (limits: 8192.000 MiB, 8192 tokens, 10125 est)
0.29.671.587 I srv update: - prompt 0x56075f59ada0: 125 tokens, checkpoints: 1, 95.969 MiB
0.29.671.587 I srv update: - prompt 0x5607593e13b0: 117 tokens, checkpoints: 1, 95.923 MiB
0.29.671.588 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.29.671.589 I srv update: - prompt 0x5607592d1090: 107 tokens, checkpoints: 1, 95.864 MiB
0.29.671.589 I srv update: - prompt 0x560759284e40: 124 tokens, checkpoints: 1, 95.964 MiB
0.29.671.590 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.29.671.590 I srv update: - prompt 0x56075931b340: 104 tokens, checkpoints: 1, 95.846 MiB
0.29.671.592 I srv get_availabl: prompt cache update took 59.68 ms
0.29.671.693 I slot launch_slot_: id 3 | task 533 | processing task, is_child = 0
0.29.671.695 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.671.696 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.671.702 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.671.703 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.671.707 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.671.708 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.29.671.722 I slot update_slots: id 3 | task 533 | Checking checkpoint with [46, 46] against 50...
0.29.683.615 W slot update_slots: id 3 | task 533 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.29.853.050 I slot print_timing: id 3 | task 533 | prompt eval time = 140.06 ms / 25 tokens ( 5.60 ms per token, 178.50 tokens per second)
0.29.853.053 I slot print_timing: id 3 | task 533 | eval time = 41.27 ms / 4 tokens ( 10.32 ms per token, 96.93 tokens per second)
0.29.853.054 I slot print_timing: id 3 | task 533 | total time = 181.32 ms / 29 tokens
0.29.853.054 I slot print_timing: id 3 | task 533 | graphs reused = 487
0.29.853.082 I slot release: id 3 | task 533 | stop processing: n_tokens = 75, truncated = 0
0.29.853.089 I srv update_slots: all slots are idle
0.29.883.959 I srv operator(): Chat format: peg-native
0.29.884.229 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.704 (> 0.100 thold), f_keep = 0.667
0.29.884.306 I slot launch_slot_: id 3 | task 539 | processing task, is_child = 0
0.29.884.307 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.884.308 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.884.314 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.884.315 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.884.319 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.884.320 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.29.884.328 I slot update_slots: id 3 | task 539 | Checking checkpoint with [46, 46] against 50...
0.29.896.384 W slot update_slots: id 3 | task 539 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.30.472.067 I slot print_timing: id 3 | task 539 | prompt eval time = 138.77 ms / 24 tokens ( 5.78 ms per token, 172.94 tokens per second)
0.30.472.071 I slot print_timing: id 3 | task 539 | eval time = 448.96 ms / 38 tokens ( 11.81 ms per token, 84.64 tokens per second)
0.30.472.071 I slot print_timing: id 3 | task 539 | total time = 587.73 ms / 62 tokens
0.30.472.072 I slot print_timing: id 3 | task 539 | graphs reused = 523
0.30.472.099 I slot release: id 3 | task 539 | stop processing: n_tokens = 108, truncated = 0
0.30.472.106 I srv update_slots: all slots are idle
0.30.502.499 I srv operator(): Chat format: peg-native
0.30.502.738 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.685 (> 0.100 thold), f_keep = 0.463
0.30.502.740 I srv get_availabl: updating prompt cache
0.30.502.758 W srv prompt_save: - saving prompt with length 108, total state size = 48.252 MiB (draft: 0.000 MiB)
0.30.568.889 I srv load: - looking for better prompt, base f_keep = 0.463, sim = 0.685
0.30.568.894 I srv load: - found better prompt with f_keep = 0.490, sim = 0.699
0.30.592.799 I srv update: - cache state: 7 prompts, 671.546 MiB (limits: 8192.000 MiB, 8192 tokens, 10173 est)
0.30.592.802 I srv update: - prompt 0x56075f59ada0: 125 tokens, checkpoints: 1, 95.969 MiB
0.30.592.803 I srv update: - prompt 0x5607593e13b0: 117 tokens, checkpoints: 1, 95.923 MiB
0.30.592.803 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.30.592.804 I srv update: - prompt 0x5607592d1090: 107 tokens, checkpoints: 1, 95.864 MiB
0.30.592.804 I srv update: - prompt 0x560759284e40: 124 tokens, checkpoints: 1, 95.964 MiB
0.30.592.805 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.30.592.805 I srv update: - prompt 0x56075f480ce0: 108 tokens, checkpoints: 1, 95.870 MiB
0.30.592.806 I srv get_availabl: prompt cache update took 90.06 ms
0.30.592.904 I slot launch_slot_: id 3 | task 579 | processing task, is_child = 0
0.30.592.908 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.30.592.909 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.30.592.914 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.30.592.914 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.30.592.919 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.30.592.919 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.30.592.932 I slot update_slots: id 3 | task 579 | Checking checkpoint with [46, 46] against 51...
0.30.616.623 W slot update_slots: id 3 | task 579 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.30.980.411 I slot print_timing: id 3 | task 579 | prompt eval time = 154.07 ms / 26 tokens ( 5.93 ms per token, 168.75 tokens per second)
0.30.980.414 I slot print_timing: id 3 | task 579 | eval time = 233.40 ms / 20 tokens ( 11.67 ms per token, 85.69 tokens per second)
0.30.980.414 I slot print_timing: id 3 | task 579 | total time = 387.47 ms / 46 tokens
0.30.980.415 I slot print_timing: id 3 | task 579 | graphs reused = 541
0.30.980.444 I slot release: id 3 | task 579 | stop processing: n_tokens = 92, truncated = 0
0.30.980.451 I srv update_slots: all slots are idle
0.31.010.494 I srv operator(): Chat format: peg-native
0.31.010.731 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.725 (> 0.100 thold), f_keep = 0.543
0.31.010.790 I slot launch_slot_: id 3 | task 601 | processing task, is_child = 0
0.31.010.792 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.31.010.793 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.31.010.800 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.31.010.800 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.31.010.806 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.31.010.806 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.31.010.815 I slot update_slots: id 3 | task 601 | Checking checkpoint with [46, 46] against 50...
0.31.022.957 W slot update_slots: id 3 | task 601 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.31.270.095 I slot print_timing: id 3 | task 601 | prompt eval time = 134.52 ms / 22 tokens ( 6.11 ms per token, 163.54 tokens per second)
0.31.270.100 I slot print_timing: id 3 | task 601 | eval time = 124.75 ms / 11 tokens ( 11.34 ms per token, 88.17 tokens per second)
0.31.270.100 I slot print_timing: id 3 | task 601 | total time = 259.27 ms / 33 tokens
0.31.270.101 I slot print_timing: id 3 | task 601 | graphs reused = 550
0.31.270.129 I slot release: id 3 | task 601 | stop processing: n_tokens = 79, truncated = 0
0.31.270.136 I srv update_slots: all slots are idle
0.31.300.251 I srv operator(): Chat format: peg-native
0.31.300.488 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.685 (> 0.100 thold), f_keep = 0.633
0.31.300.550 I slot launch_slot_: id 3 | task 614 | processing task, is_child = 0
0.31.300.552 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.31.300.553 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.31.300.560 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.31.300.560 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.31.300.566 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.31.300.566 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.31.300.575 I slot update_slots: id 3 | task 614 | Checking checkpoint with [46, 46] against 50...
0.31.312.789 W slot update_slots: id 3 | task 614 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.32.180.438 I slot print_timing: id 3 | task 614 | prompt eval time = 142.01 ms / 26 tokens ( 5.46 ms per token, 183.09 tokens per second)
0.32.180.442 I slot print_timing: id 3 | task 614 | eval time = 737.85 ms / 62 tokens ( 11.90 ms per token, 84.03 tokens per second)
0.32.180.442 I slot print_timing: id 3 | task 614 | total time = 879.86 ms / 88 tokens
0.32.180.443 I slot print_timing: id 3 | task 614 | graphs reused = 610
0.32.180.471 I slot release: id 3 | task 614 | stop processing: n_tokens = 134, truncated = 0
0.32.180.480 I srv update_slots: all slots are idle
0.32.211.262 I srv operator(): Chat format: peg-native
0.32.211.497 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.735 (> 0.100 thold), f_keep = 0.373
0.32.211.499 I srv get_availabl: updating prompt cache
0.32.211.520 W srv prompt_save: - saving prompt with length 134, total state size = 48.405 MiB (draft: 0.000 MiB)
0.32.271.955 I srv load: - looking for better prompt, base f_keep = 0.373, sim = 0.735
0.32.271.961 I srv load: - found better prompt with f_keep = 0.477, sim = 0.750
0.32.296.302 I srv update: - cache state: 7 prompts, 671.704 MiB (limits: 8192.000 MiB, 8192 tokens, 10500 est)
0.32.296.305 I srv update: - prompt 0x56075f59ada0: 125 tokens, checkpoints: 1, 95.969 MiB
0.32.296.305 I srv update: - prompt 0x5607593e13b0: 117 tokens, checkpoints: 1, 95.923 MiB
0.32.296.306 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.32.296.307 I srv update: - prompt 0x560759284e40: 124 tokens, checkpoints: 1, 95.964 MiB
0.32.296.307 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.32.296.308 I srv update: - prompt 0x56075f480ce0: 108 tokens, checkpoints: 1, 95.870 MiB
0.32.296.308 I srv update: - prompt 0x5607578c8c20: 134 tokens, checkpoints: 1, 96.022 MiB
0.32.296.310 I srv get_availabl: prompt cache update took 84.81 ms
0.32.296.403 I slot launch_slot_: id 3 | task 678 | processing task, is_child = 0
0.32.296.404 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.32.296.404 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.32.296.410 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.32.296.410 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.32.296.416 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.32.296.416 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.32.296.430 I slot update_slots: id 3 | task 678 | Checking checkpoint with [46, 46] against 51...
0.32.324.052 W slot update_slots: id 3 | task 678 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.32.592.506 I slot print_timing: id 3 | task 678 | prompt eval time = 146.53 ms / 21 tokens ( 6.98 ms per token, 143.31 tokens per second)
0.32.592.509 I slot print_timing: id 3 | task 678 | eval time = 149.54 ms / 13 tokens ( 11.50 ms per token, 86.93 tokens per second)
0.32.592.510 I slot print_timing: id 3 | task 678 | total time = 296.07 ms / 34 tokens
0.32.592.510 I slot print_timing: id 3 | task 678 | graphs reused = 621
0.32.592.538 I slot release: id 3 | task 678 | stop processing: n_tokens = 80, truncated = 0
0.32.592.545 I srv update_slots: all slots are idle
0.32.623.358 I srv operator(): Chat format: peg-native
0.32.623.590 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.746 (> 0.100 thold), f_keep = 0.625
0.32.623.653 I slot launch_slot_: id 3 | task 693 | processing task, is_child = 0
0.32.623.655 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.32.623.656 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.32.623.663 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.32.623.663 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.32.623.669 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.32.623.669 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.32.623.677 I slot update_slots: id 3 | task 693 | Checking checkpoint with [46, 46] against 50...
0.32.636.004 W slot update_slots: id 3 | task 693 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.32.861.442 I slot print_timing: id 3 | task 693 | prompt eval time = 112.57 ms / 20 tokens ( 5.63 ms per token, 177.67 tokens per second)
0.32.861.446 I slot print_timing: id 3 | task 693 | eval time = 125.19 ms / 11 tokens ( 11.38 ms per token, 87.86 tokens per second)
0.32.861.446 I slot print_timing: id 3 | task 693 | total time = 237.76 ms / 31 tokens
0.32.861.447 I slot print_timing: id 3 | task 693 | graphs reused = 630
0.32.861.473 I slot release: id 3 | task 693 | stop processing: n_tokens = 77, truncated = 0
0.32.861.480 I srv update_slots: all slots are idle
0.32.892.144 I srv operator(): Chat format: peg-native
0.32.892.403 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.773 (> 0.100 thold), f_keep = 0.662
0.32.892.481 I slot launch_slot_: id 3 | task 706 | processing task, is_child = 0
0.32.892.483 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.32.892.484 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.32.892.491 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.32.892.491 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.32.892.496 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.32.892.497 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.32.892.505 I slot update_slots: id 3 | task 706 | Checking checkpoint with [46, 46] against 51...
0.32.904.640 W slot update_slots: id 3 | task 706 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.33.069.094 I slot print_timing: id 3 | task 706 | prompt eval time = 111.07 ms / 19 tokens ( 5.85 ms per token, 171.06 tokens per second)
0.33.069.097 I slot print_timing: id 3 | task 706 | eval time = 65.51 ms / 6 tokens ( 10.92 ms per token, 91.59 tokens per second)
0.33.069.098 I slot print_timing: id 3 | task 706 | total time = 176.58 ms / 25 tokens
0.33.069.098 I slot print_timing: id 3 | task 706 | graphs reused = 634
0.33.069.127 I slot release: id 3 | task 706 | stop processing: n_tokens = 71, truncated = 0
0.33.069.136 I srv update_slots: all slots are idle
0.33.099.315 I srv operator(): Chat format: peg-native
0.33.099.545 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.769 (> 0.100 thold), f_keep = 0.704
0.33.099.599 I slot launch_slot_: id 3 | task 714 | processing task, is_child = 0
0.33.099.600 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.33.099.601 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.33.099.607 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.33.099.607 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.33.099.612 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.33.099.612 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.33.099.620 I slot update_slots: id 3 | task 714 | Checking checkpoint with [46, 46] against 50...
0.33.111.833 W slot update_slots: id 3 | task 714 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.33.286.418 I slot print_timing: id 3 | task 714 | prompt eval time = 109.48 ms / 18 tokens ( 6.08 ms per token, 164.41 tokens per second)
0.33.286.421 I slot print_timing: id 3 | task 714 | eval time = 77.31 ms / 7 tokens ( 11.04 ms per token, 90.54 tokens per second)
0.33.286.422 I slot print_timing: id 3 | task 714 | total time = 186.79 ms / 25 tokens
0.33.286.422 I slot print_timing: id 3 | task 714 | graphs reused = 639
0.33.286.449 I slot release: id 3 | task 714 | stop processing: n_tokens = 71, truncated = 0
0.33.286.456 I srv update_slots: all slots are idle
0.33.316.514 I srv operator(): Chat format: peg-native
0.33.316.751 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.704
0.33.316.814 I slot launch_slot_: id 3 | task 723 | processing task, is_child = 0
0.33.316.816 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.33.316.817 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.33.316.824 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.33.316.824 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.33.316.830 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.33.316.830 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.33.316.839 I slot update_slots: id 3 | task 723 | Checking checkpoint with [46, 46] against 50...
0.33.329.037 W slot update_slots: id 3 | task 723 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.34.165.440 I slot print_timing: id 3 | task 723 | prompt eval time = 134.86 ms / 23 tokens ( 5.86 ms per token, 170.54 tokens per second)
0.34.165.443 I slot print_timing: id 3 | task 723 | eval time = 713.73 ms / 60 tokens ( 11.90 ms per token, 84.07 tokens per second)
0.34.165.444 I slot print_timing: id 3 | task 723 | total time = 848.59 ms / 83 tokens
0.34.165.444 I slot print_timing: id 3 | task 723 | graphs reused = 697
0.34.165.474 I slot release: id 3 | task 723 | stop processing: n_tokens = 129, truncated = 0
0.34.165.483 I srv update_slots: all slots are idle
0.34.195.854 I srv operator(): Chat format: peg-native
0.34.196.099 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.725 (> 0.100 thold), f_keep = 0.388
0.34.196.101 I srv get_availabl: updating prompt cache
0.34.196.118 W srv prompt_save: - saving prompt with length 129, total state size = 48.375 MiB (draft: 0.000 MiB)
0.34.255.173 I srv load: - looking for better prompt, base f_keep = 0.388, sim = 0.725
0.34.255.179 I srv load: - found better prompt with f_keep = 0.424, sim = 0.768
0.34.278.953 I srv update: - cache state: 7 prompts, 671.728 MiB (limits: 8192.000 MiB, 8192 tokens, 10549 est)
0.34.278.956 I srv update: - prompt 0x5607593e13b0: 117 tokens, checkpoints: 1, 95.923 MiB
0.34.278.957 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.34.278.958 I srv update: - prompt 0x560759284e40: 124 tokens, checkpoints: 1, 95.964 MiB
0.34.278.958 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.34.278.959 I srv update: - prompt 0x56075f480ce0: 108 tokens, checkpoints: 1, 95.870 MiB
0.34.278.959 I srv update: - prompt 0x5607578c8c20: 134 tokens, checkpoints: 1, 96.022 MiB
0.34.278.960 I srv update: - prompt 0x5607593b3030: 129 tokens, checkpoints: 1, 95.993 MiB
0.34.278.961 I srv get_availabl: prompt cache update took 82.86 ms
0.34.279.060 I slot launch_slot_: id 3 | task 785 | processing task, is_child = 0
0.34.279.061 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.34.279.062 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.34.279.067 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.34.279.067 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.34.279.072 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.34.279.072 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.34.279.084 I slot update_slots: id 3 | task 785 | Checking checkpoint with [46, 46] against 53...
0.34.301.005 W slot update_slots: id 3 | task 785 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.34.647.447 I slot print_timing: id 3 | task 785 | prompt eval time = 146.84 ms / 22 tokens ( 6.67 ms per token, 149.82 tokens per second)
0.34.647.450 I slot print_timing: id 3 | task 785 | eval time = 221.52 ms / 19 tokens ( 11.66 ms per token, 85.77 tokens per second)
0.34.647.450 I slot print_timing: id 3 | task 785 | total time = 368.36 ms / 41 tokens
0.34.647.451 I slot print_timing: id 3 | task 785 | graphs reused = 714
0.34.647.481 I slot release: id 3 | task 785 | stop processing: n_tokens = 87, truncated = 0
0.34.647.488 I srv update_slots: all slots are idle
0.34.677.441 I srv operator(): Chat format: peg-native
0.34.677.670 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.575
0.34.677.761 I slot launch_slot_: id 3 | task 806 | processing task, is_child = 0
0.34.677.763 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.34.677.764 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.34.677.771 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.34.677.771 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.34.677.776 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.34.677.777 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.34.677.785 I slot update_slots: id 3 | task 806 | Checking checkpoint with [46, 46] against 50...
0.34.689.960 W slot update_slots: id 3 | task 806 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.35.082.440 I slot print_timing: id 3 | task 806 | prompt eval time = 146.83 ms / 28 tokens ( 5.24 ms per token, 190.70 tokens per second)
0.35.082.444 I slot print_timing: id 3 | task 806 | eval time = 257.82 ms / 22 tokens ( 11.72 ms per token, 85.33 tokens per second)
0.35.082.444 I slot print_timing: id 3 | task 806 | total time = 404.65 ms / 50 tokens
0.35.082.445 I slot print_timing: id 3 | task 806 | graphs reused = 734
0.35.082.474 I slot release: id 3 | task 806 | stop processing: n_tokens = 96, truncated = 0
0.35.082.481 I srv update_slots: all slots are idle
0.35.112.696 I srv operator(): Chat format: peg-native
0.35.112.933 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.676 (> 0.100 thold), f_keep = 0.521
0.35.113.009 I slot launch_slot_: id 3 | task 830 | processing task, is_child = 0
0.35.113.011 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.35.113.012 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.35.113.019 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.35.113.020 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.35.113.025 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.35.113.026 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.35.113.035 I slot update_slots: id 3 | task 830 | Checking checkpoint with [46, 46] against 50...
0.35.125.152 W slot update_slots: id 3 | task 830 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.35.491.450 I slot print_timing: id 3 | task 830 | prompt eval time = 145.33 ms / 27 tokens ( 5.38 ms per token, 185.79 tokens per second)
0.35.491.453 I slot print_timing: id 3 | task 830 | eval time = 233.08 ms / 20 tokens ( 11.65 ms per token, 85.81 tokens per second)
0.35.491.453 I slot print_timing: id 3 | task 830 | total time = 378.41 ms / 47 tokens
0.35.491.454 I slot print_timing: id 3 | task 830 | graphs reused = 752
0.35.491.484 I slot release: id 3 | task 830 | stop processing: n_tokens = 93, truncated = 0
0.35.491.492 I srv update_slots: all slots are idle
0.35.522.084 I srv operator(): Chat format: peg-native
0.35.522.338 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.769 (> 0.100 thold), f_keep = 0.538
0.35.522.404 I slot launch_slot_: id 3 | task 852 | processing task, is_child = 0
0.35.522.406 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.35.522.407 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.35.522.415 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.35.522.416 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.35.522.422 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.35.522.423 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.35.522.433 I slot update_slots: id 3 | task 852 | Checking checkpoint with [46, 46] against 50...
0.35.534.586 W slot update_slots: id 3 | task 852 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.35.733.475 I slot print_timing: id 3 | task 852 | prompt eval time = 109.82 ms / 18 tokens ( 6.10 ms per token, 163.91 tokens per second)
0.35.733.479 I slot print_timing: id 3 | task 852 | eval time = 101.22 ms / 9 tokens ( 11.25 ms per token, 88.92 tokens per second)
0.35.733.480 I slot print_timing: id 3 | task 852 | total time = 211.04 ms / 27 tokens
0.35.733.480 I slot print_timing: id 3 | task 852 | graphs reused = 759
0.35.733.510 I slot release: id 3 | task 852 | stop processing: n_tokens = 73, truncated = 0
0.35.733.519 I srv update_slots: all slots are idle
0.35.763.485 I srv operator(): Chat format: peg-native
0.35.763.725 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.725 (> 0.100 thold), f_keep = 0.685
0.35.763.789 I slot launch_slot_: id 3 | task 863 | processing task, is_child = 0
0.35.763.791 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.35.763.791 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.35.763.798 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.35.763.799 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.35.763.804 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.35.763.805 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.35.763.813 I slot update_slots: id 3 | task 863 | Checking checkpoint with [46, 46] against 50...
0.35.775.972 W slot update_slots: id 3 | task 863 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.36.059.461 I slot print_timing: id 3 | task 863 | prompt eval time = 134.60 ms / 22 tokens ( 6.12 ms per token, 163.45 tokens per second)
0.36.059.465 I slot print_timing: id 3 | task 863 | eval time = 161.04 ms / 14 tokens ( 11.50 ms per token, 86.93 tokens per second)
0.36.059.466 I slot print_timing: id 3 | task 863 | total time = 295.64 ms / 36 tokens
0.36.059.466 I slot print_timing: id 3 | task 863 | graphs reused = 771
0.36.059.495 I slot release: id 3 | task 863 | stop processing: n_tokens = 82, truncated = 0
0.36.059.504 I srv update_slots: all slots are idle
0.36.089.432 I srv operator(): Chat format: peg-native
0.36.089.671 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.704 (> 0.100 thold), f_keep = 0.610
0.36.089.721 I slot launch_slot_: id 3 | task 879 | processing task, is_child = 0
0.36.089.722 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.36.089.727 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.36.089.733 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.36.089.734 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.36.089.738 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.36.089.739 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.36.089.746 I slot update_slots: id 3 | task 879 | Checking checkpoint with [46, 46] against 50...
0.36.101.833 W slot update_slots: id 3 | task 879 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.36.391.466 I slot print_timing: id 3 | task 879 | prompt eval time = 140.50 ms / 24 tokens ( 5.85 ms per token, 170.82 tokens per second)
0.36.391.470 I slot print_timing: id 3 | task 879 | eval time = 161.21 ms / 14 tokens ( 11.52 ms per token, 86.84 tokens per second)
0.36.391.471 I slot print_timing: id 3 | task 879 | total time = 301.71 ms / 38 tokens
0.36.391.471 I slot print_timing: id 3 | task 879 | graphs reused = 783
0.36.391.500 I slot release: id 3 | task 879 | stop processing: n_tokens = 84, truncated = 0
0.36.391.510 I srv update_slots: all slots are idle
0.36.421.500 I srv operator(): Chat format: peg-native
0.36.421.739 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.610 (> 0.100 thold), f_keep = 0.595
0.36.421.791 I slot launch_slot_: id 3 | task 895 | processing task, is_child = 0
0.36.421.793 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.36.421.793 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.36.421.800 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.36.421.800 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.36.421.804 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.36.421.805 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.36.421.812 I slot update_slots: id 3 | task 895 | Checking checkpoint with [46, 46] against 50...
0.36.434.002 W slot update_slots: id 3 | task 895 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.37.615.495 I slot print_timing: id 3 | task 895 | prompt eval time = 179.88 ms / 35 tokens ( 5.14 ms per token, 194.58 tokens per second)
0.37.615.499 I slot print_timing: id 3 | task 895 | eval time = 1013.80 ms / 85 tokens ( 11.93 ms per token, 83.84 tokens per second)
0.37.615.499 I slot print_timing: id 3 | task 895 | total time = 1193.68 ms / 120 tokens
0.37.615.500 I slot print_timing: id 3 | task 895 | graphs reused = 866
0.37.615.530 I slot release: id 3 | task 895 | stop processing: n_tokens = 166, truncated = 0
0.37.615.540 I srv update_slots: all slots are idle
0.37.645.553 I srv operator(): Chat format: peg-native
0.37.645.795 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.699 (> 0.100 thold), f_keep = 0.307
0.37.645.797 I srv get_availabl: updating prompt cache
0.37.645.816 W srv prompt_save: - saving prompt with length 166, total state size = 48.592 MiB (draft: 0.000 MiB)
0.37.709.405 I srv load: - looking for better prompt, base f_keep = 0.307, sim = 0.699
0.37.709.413 I srv update: - cache state: 8 prompts, 767.938 MiB (limits: 8192.000 MiB, 8192 tokens, 10998 est)
0.37.709.414 I srv update: - prompt 0x5607593e13b0: 117 tokens, checkpoints: 1, 95.923 MiB
0.37.709.415 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.37.709.416 I srv update: - prompt 0x560759284e40: 124 tokens, checkpoints: 1, 95.964 MiB
0.37.709.416 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.37.709.417 I srv update: - prompt 0x56075f480ce0: 108 tokens, checkpoints: 1, 95.870 MiB
0.37.709.417 I srv update: - prompt 0x5607578c8c20: 134 tokens, checkpoints: 1, 96.022 MiB
0.37.709.418 I srv update: - prompt 0x5607593b3030: 129 tokens, checkpoints: 1, 95.993 MiB
0.37.709.418 I srv update: - prompt 0x560759469c30: 166 tokens, checkpoints: 1, 96.210 MiB
0.37.709.420 I srv get_availabl: prompt cache update took 63.62 ms
0.37.709.522 I slot launch_slot_: id 3 | task 982 | processing task, is_child = 0
0.37.709.523 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.37.709.524 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.37.709.531 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.37.709.531 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.37.709.535 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.37.709.536 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.37.709.548 I slot update_slots: id 3 | task 982 | Checking checkpoint with [46, 46] against 51...
0.37.721.388 W slot update_slots: id 3 | task 982 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.38.471.477 I slot print_timing: id 3 | task 982 | prompt eval time = 144.15 ms / 26 tokens ( 5.54 ms per token, 180.36 tokens per second)
0.38.471.480 I slot print_timing: id 3 | task 982 | eval time = 617.77 ms / 52 tokens ( 11.88 ms per token, 84.17 tokens per second)
0.38.471.481 I slot print_timing: id 3 | task 982 | total time = 761.92 ms / 78 tokens
0.38.471.482 I slot print_timing: id 3 | task 982 | graphs reused = 916
0.38.471.510 I slot release: id 3 | task 982 | stop processing: n_tokens = 124, truncated = 0
0.38.471.519 I srv update_slots: all slots are idle
0.38.501.467 I srv operator(): Chat format: peg-native
0.38.501.709 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.735 (> 0.100 thold), f_keep = 0.403
0.38.501.712 I srv get_availabl: updating prompt cache
0.38.501.734 W srv prompt_save: - saving prompt with length 124, total state size = 48.346 MiB (draft: 0.000 MiB)
0.38.566.528 I srv load: - looking for better prompt, base f_keep = 0.403, sim = 0.735
0.38.566.537 I srv update: - cache state: 9 prompts, 863.902 MiB (limits: 8192.000 MiB, 8192 tokens, 10952 est)
0.38.566.538 I srv update: - prompt 0x5607593e13b0: 117 tokens, checkpoints: 1, 95.923 MiB
0.38.566.539 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.38.566.540 I srv update: - prompt 0x560759284e40: 124 tokens, checkpoints: 1, 95.964 MiB
0.38.566.540 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.38.566.541 I srv update: - prompt 0x56075f480ce0: 108 tokens, checkpoints: 1, 95.870 MiB
0.38.566.541 I srv update: - prompt 0x5607578c8c20: 134 tokens, checkpoints: 1, 96.022 MiB
0.38.566.542 I srv update: - prompt 0x5607593b3030: 129 tokens, checkpoints: 1, 95.993 MiB
0.38.566.542 I srv update: - prompt 0x560759469c30: 166 tokens, checkpoints: 1, 96.210 MiB
0.38.566.543 I srv update: - prompt 0x56075f4a7850: 124 tokens, checkpoints: 1, 95.964 MiB
0.38.566.544 I srv get_availabl: prompt cache update took 64.83 ms
0.38.566.649 I slot launch_slot_: id 3 | task 1036 | processing task, is_child = 0
0.38.566.649 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.566.650 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.38.566.658 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.566.658 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.38.566.663 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.566.663 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.38.566.675 I slot update_slots: id 3 | task 1036 | Checking checkpoint with [46, 46] against 50...
0.38.578.582 W slot update_slots: id 3 | task 1036 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.40.255.491 I slot print_timing: id 3 | task 1036 | prompt eval time = 133.46 ms / 21 tokens ( 6.36 ms per token, 157.35 tokens per second)
0.40.255.494 I slot print_timing: id 3 | task 1036 | eval time = 1555.35 ms / 130 tokens ( 11.96 ms per token, 83.58 tokens per second)
0.40.255.495 I slot print_timing: id 3 | task 1036 | total time = 1688.81 ms / 151 tokens
0.40.255.496 I slot print_timing: id 3 | task 1036 | graphs reused = 1044
0.40.255.525 I slot release: id 3 | task 1036 | stop processing: n_tokens = 197, truncated = 0
0.40.255.534 I srv update_slots: all slots are idle
0.40.285.629 I srv operator(): Chat format: peg-native
0.40.285.872 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.685 (> 0.100 thold), f_keep = 0.254
0.40.285.874 I srv get_availabl: updating prompt cache
0.40.285.895 W srv prompt_save: - saving prompt with length 197, total state size = 48.774 MiB (draft: 0.000 MiB)
0.40.347.993 I srv load: - looking for better prompt, base f_keep = 0.254, sim = 0.685
0.40.348.000 I srv load: - found better prompt with f_keep = 0.436, sim = 0.699
0.40.371.778 I srv update: - cache state: 9 prompts, 864.371 MiB (limits: 8192.000 MiB, 8192 tokens, 11704 est)
0.40.371.781 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.40.371.782 I srv update: - prompt 0x560759284e40: 124 tokens, checkpoints: 1, 95.964 MiB
0.40.371.783 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.40.371.783 I srv update: - prompt 0x56075f480ce0: 108 tokens, checkpoints: 1, 95.870 MiB
0.40.371.784 I srv update: - prompt 0x5607578c8c20: 134 tokens, checkpoints: 1, 96.022 MiB
0.40.371.784 I srv update: - prompt 0x5607593b3030: 129 tokens, checkpoints: 1, 95.993 MiB
0.40.371.785 I srv update: - prompt 0x560759469c30: 166 tokens, checkpoints: 1, 96.210 MiB
0.40.371.785 I srv update: - prompt 0x56075f4a7850: 124 tokens, checkpoints: 1, 95.964 MiB
0.40.371.786 I srv update: - prompt 0x56075f59f250: 197 tokens, checkpoints: 1, 96.392 MiB
0.40.371.787 I srv get_availabl: prompt cache update took 85.91 ms
0.40.371.884 I slot launch_slot_: id 3 | task 1168 | processing task, is_child = 0
0.40.371.885 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.40.371.886 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.40.371.891 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.40.371.892 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.40.371.902 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.40.371.902 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.40.371.921 I slot update_slots: id 3 | task 1168 | Checking checkpoint with [46, 46] against 51...
0.40.387.486 W slot update_slots: id 3 | task 1168 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.41.713.519 I slot print_timing: id 3 | task 1168 | prompt eval time = 145.94 ms / 26 tokens ( 5.61 ms per token, 178.15 tokens per second)
0.41.713.522 I slot print_timing: id 3 | task 1168 | eval time = 1195.65 ms / 100 tokens ( 11.96 ms per token, 83.64 tokens per second)
0.41.713.523 I slot print_timing: id 3 | task 1168 | total time = 1341.59 ms / 126 tokens
0.41.713.524 I slot print_timing: id 3 | task 1168 | graphs reused = 1142
0.41.713.552 I slot release: id 3 | task 1168 | stop processing: n_tokens = 172, truncated = 0
0.41.713.559 I srv update_slots: all slots are idle
0.41.743.674 I srv operator(): Chat format: peg-native
0.41.743.910 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.699 (> 0.100 thold), f_keep = 0.297
0.41.743.912 I srv get_availabl: updating prompt cache
0.41.743.933 W srv prompt_save: - saving prompt with length 172, total state size = 48.628 MiB (draft: 0.000 MiB)
0.41.803.632 I srv load: - looking for better prompt, base f_keep = 0.297, sim = 0.699
0.41.803.638 I srv load: - found better prompt with f_keep = 0.427, sim = 0.726
0.41.827.431 I srv update: - cache state: 9 prompts, 864.653 MiB (limits: 8192.000 MiB, 8192 tokens, 12155 est)
0.41.827.434 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.41.827.435 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.41.827.436 I srv update: - prompt 0x56075f480ce0: 108 tokens, checkpoints: 1, 95.870 MiB
0.41.827.436 I srv update: - prompt 0x5607578c8c20: 134 tokens, checkpoints: 1, 96.022 MiB
0.41.827.437 I srv update: - prompt 0x5607593b3030: 129 tokens, checkpoints: 1, 95.993 MiB
0.41.827.437 I srv update: - prompt 0x560759469c30: 166 tokens, checkpoints: 1, 96.210 MiB
0.41.827.438 I srv update: - prompt 0x56075f4a7850: 124 tokens, checkpoints: 1, 95.964 MiB
0.41.827.438 I srv update: - prompt 0x56075f59f250: 197 tokens, checkpoints: 1, 96.392 MiB
0.41.827.438 I srv update: - prompt 0x5607592c9910: 172 tokens, checkpoints: 1, 96.245 MiB
0.41.827.440 I srv get_availabl: prompt cache update took 83.53 ms
0.41.827.531 I slot launch_slot_: id 3 | task 1270 | processing task, is_child = 0
0.41.827.532 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.41.827.533 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.41.827.538 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.41.827.539 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.41.827.543 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.41.827.543 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.41.827.555 I slot update_slots: id 3 | task 1270 | Checking checkpoint with [46, 46] against 53...
0.41.859.045 W slot update_slots: id 3 | task 1270 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.42.159.476 I slot print_timing: id 3 | task 1270 | prompt eval time = 167.55 ms / 26 tokens ( 6.44 ms per token, 155.18 tokens per second)
0.42.159.479 I slot print_timing: id 3 | task 1270 | eval time = 164.37 ms / 14 tokens ( 11.74 ms per token, 85.17 tokens per second)
0.42.159.480 I slot print_timing: id 3 | task 1270 | total time = 331.92 ms / 40 tokens
0.42.159.481 I slot print_timing: id 3 | task 1270 | graphs reused = 1154
0.42.159.509 I slot release: id 3 | task 1270 | stop processing: n_tokens = 86, truncated = 0
0.42.159.517 I srv update_slots: all slots are idle
0.42.189.604 I srv operator(): Chat format: peg-native
0.42.189.845 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.746 (> 0.100 thold), f_keep = 0.581
0.42.189.907 I slot launch_slot_: id 3 | task 1286 | processing task, is_child = 0
0.42.189.909 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.42.189.909 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.42.189.917 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.42.189.917 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.42.189.923 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.42.189.923 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.42.189.932 I slot update_slots: id 3 | task 1286 | Checking checkpoint with [46, 46] against 50...
0.42.202.138 W slot update_slots: id 3 | task 1286 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.42.766.526 I slot print_timing: id 3 | task 1286 | prompt eval time = 114.38 ms / 20 tokens ( 5.72 ms per token, 174.86 tokens per second)
0.42.766.529 I slot print_timing: id 3 | task 1286 | eval time = 462.21 ms / 39 tokens ( 11.85 ms per token, 84.38 tokens per second)
0.42.766.530 I slot print_timing: id 3 | task 1286 | total time = 576.59 ms / 59 tokens
0.42.766.531 I slot print_timing: id 3 | task 1286 | graphs reused = 1191
0.42.766.559 I slot release: id 3 | task 1286 | stop processing: n_tokens = 105, truncated = 0
0.42.766.566 I srv update_slots: all slots are idle
0.42.796.592 I srv operator(): Chat format: peg-native
0.42.796.835 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.781 (> 0.100 thold), f_keep = 0.476
0.42.796.837 I srv get_availabl: updating prompt cache
0.42.796.855 W srv prompt_save: - saving prompt with length 105, total state size = 48.234 MiB (draft: 0.000 MiB)
0.42.864.367 I srv load: - looking for better prompt, base f_keep = 0.476, sim = 0.781
0.42.864.378 I srv update: - cache state: 10 prompts, 960.505 MiB (limits: 8192.000 MiB, 8192 tokens, 11838 est)
0.42.864.378 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.42.864.379 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.42.864.380 I srv update: - prompt 0x56075f480ce0: 108 tokens, checkpoints: 1, 95.870 MiB
0.42.864.380 I srv update: - prompt 0x5607578c8c20: 134 tokens, checkpoints: 1, 96.022 MiB
0.42.864.381 I srv update: - prompt 0x5607593b3030: 129 tokens, checkpoints: 1, 95.993 MiB
0.42.864.381 I srv update: - prompt 0x560759469c30: 166 tokens, checkpoints: 1, 96.210 MiB
0.42.864.382 I srv update: - prompt 0x56075f4a7850: 124 tokens, checkpoints: 1, 95.964 MiB
0.42.864.382 I srv update: - prompt 0x56075f59f250: 197 tokens, checkpoints: 1, 96.392 MiB
0.42.864.383 I srv update: - prompt 0x5607592c9910: 172 tokens, checkpoints: 1, 96.245 MiB
0.42.864.383 I srv update: - prompt 0x5607593afc00: 105 tokens, checkpoints: 1, 95.852 MiB
0.42.864.385 I srv get_availabl: prompt cache update took 67.55 ms
0.42.864.488 I slot launch_slot_: id 3 | task 1327 | processing task, is_child = 0
0.42.864.488 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.42.864.489 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.42.864.495 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.42.864.496 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.42.864.500 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.42.864.500 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.42.864.512 I slot update_slots: id 3 | task 1327 | Checking checkpoint with [46, 46] against 50...
0.42.876.398 W slot update_slots: id 3 | task 1327 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.43.109.520 I slot print_timing: id 3 | task 1327 | prompt eval time = 106.07 ms / 17 tokens ( 6.24 ms per token, 160.27 tokens per second)
0.43.109.524 I slot print_timing: id 3 | task 1327 | eval time = 138.94 ms / 12 tokens ( 11.58 ms per token, 86.37 tokens per second)
0.43.109.525 I slot print_timing: id 3 | task 1327 | total time = 245.00 ms / 29 tokens
0.43.109.525 I slot print_timing: id 3 | task 1327 | graphs reused = 1201
0.43.109.556 I slot release: id 3 | task 1327 | stop processing: n_tokens = 75, truncated = 0
0.43.109.564 I srv update_slots: all slots are idle
0.43.139.658 I srv operator(): Chat format: peg-native
0.43.139.905 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.667
0.43.139.967 I slot launch_slot_: id 3 | task 1341 | processing task, is_child = 0
0.43.139.969 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.43.139.970 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.43.139.985 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.43.139.985 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.43.139.991 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.43.139.992 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.43.140.001 I slot update_slots: id 3 | task 1341 | Checking checkpoint with [46, 46] against 50...
0.43.152.065 W slot update_slots: id 3 | task 1341 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.44.218.392 I slot print_timing: id 3 | task 1341 | prompt eval time = 138.25 ms / 23 tokens ( 6.01 ms per token, 166.37 tokens per second)
0.44.218.396 I slot print_timing: id 3 | task 1341 | eval time = 940.14 ms / 78 tokens ( 12.05 ms per token, 82.97 tokens per second)
0.44.218.397 I slot print_timing: id 3 | task 1341 | total time = 1078.38 ms / 101 tokens
0.44.218.398 I slot print_timing: id 3 | task 1341 | graphs reused = 1277
0.44.218.431 I slot release: id 3 | task 1341 | stop processing: n_tokens = 147, truncated = 0
0.44.218.440 I srv update_slots: all slots are idle
0.44.248.625 I srv operator(): Chat format: peg-native
0.44.248.863 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.735 (> 0.100 thold), f_keep = 0.340
0.44.248.865 I srv get_availabl: updating prompt cache
0.44.248.895 W srv prompt_save: - saving prompt with length 147, total state size = 48.481 MiB (draft: 0.000 MiB)
0.44.310.873 I srv load: - looking for better prompt, base f_keep = 0.340, sim = 0.735
0.44.310.880 I srv load: - found better prompt with f_keep = 0.396, sim = 0.779
0.44.334.174 I srv update: - cache state: 10 prompts, 960.581 MiB (limits: 8192.000 MiB, 8192 tokens, 11947 est)
0.44.334.177 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.44.334.178 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.44.334.179 I srv update: - prompt 0x56075f480ce0: 108 tokens, checkpoints: 1, 95.870 MiB
0.44.334.180 I srv update: - prompt 0x5607593b3030: 129 tokens, checkpoints: 1, 95.993 MiB
0.44.334.180 I srv update: - prompt 0x560759469c30: 166 tokens, checkpoints: 1, 96.210 MiB
0.44.334.181 I srv update: - prompt 0x56075f4a7850: 124 tokens, checkpoints: 1, 95.964 MiB
0.44.334.181 I srv update: - prompt 0x56075f59f250: 197 tokens, checkpoints: 1, 96.392 MiB
0.44.334.182 I srv update: - prompt 0x5607592c9910: 172 tokens, checkpoints: 1, 96.245 MiB
0.44.334.183 I srv update: - prompt 0x5607593afc00: 105 tokens, checkpoints: 1, 95.852 MiB
0.44.334.184 I srv update: - prompt 0x56075f119d80: 147 tokens, checkpoints: 1, 96.099 MiB
0.44.334.185 I srv get_availabl: prompt cache update took 85.32 ms
0.44.334.330 I slot launch_slot_: id 3 | task 1421 | processing task, is_child = 0
0.44.334.332 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.44.334.332 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.44.334.340 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.44.334.340 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.44.334.345 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.44.334.346 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.44.334.355 I slot update_slots: id 3 | task 1421 | Checking checkpoint with [46, 46] against 53...
0.44.351.722 W slot update_slots: id 3 | task 1421 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.45.135.123 I slot print_timing: id 3 | task 1421 | prompt eval time = 141.11 ms / 21 tokens ( 6.72 ms per token, 148.82 tokens per second)
0.45.135.126 I slot print_timing: id 3 | task 1421 | eval time = 659.66 ms / 55 tokens ( 11.99 ms per token, 83.38 tokens per second)
0.45.135.127 I slot print_timing: id 3 | task 1421 | total time = 800.76 ms / 76 tokens
0.45.135.127 I slot print_timing: id 3 | task 1421 | graphs reused = 1330
0.45.135.155 I slot release: id 3 | task 1421 | stop processing: n_tokens = 122, truncated = 0
0.45.135.161 I srv update_slots: all slots are idle
0.45.165.193 I srv operator(): Chat format: peg-native
0.45.165.445 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.761 (> 0.100 thold), f_keep = 0.418
0.45.165.447 I srv get_availabl: updating prompt cache
0.45.165.466 W srv prompt_save: - saving prompt with length 122, total state size = 48.334 MiB (draft: 0.000 MiB)
0.45.228.163 I srv load: - looking for better prompt, base f_keep = 0.418, sim = 0.761
0.45.228.172 I srv update: - cache state: 11 prompts, 1056.533 MiB (limits: 8192.000 MiB, 8192 tokens, 11808 est)
0.45.228.173 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.45.228.173 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.45.228.174 I srv update: - prompt 0x56075f480ce0: 108 tokens, checkpoints: 1, 95.870 MiB
0.45.228.174 I srv update: - prompt 0x5607593b3030: 129 tokens, checkpoints: 1, 95.993 MiB
0.45.228.175 I srv update: - prompt 0x560759469c30: 166 tokens, checkpoints: 1, 96.210 MiB
0.45.228.175 I srv update: - prompt 0x56075f4a7850: 124 tokens, checkpoints: 1, 95.964 MiB
0.45.228.176 I srv update: - prompt 0x56075f59f250: 197 tokens, checkpoints: 1, 96.392 MiB
0.45.228.180 I srv update: - prompt 0x5607592c9910: 172 tokens, checkpoints: 1, 96.245 MiB
0.45.228.181 I srv update: - prompt 0x5607593afc00: 105 tokens, checkpoints: 1, 95.852 MiB
0.45.228.181 I srv update: - prompt 0x56075f119d80: 147 tokens, checkpoints: 1, 96.099 MiB
0.45.228.182 I srv update: - prompt 0x560759477380: 122 tokens, checkpoints: 1, 95.952 MiB
0.45.228.183 I srv get_availabl: prompt cache update took 62.73 ms
0.45.228.285 I slot launch_slot_: id 3 | task 1478 | processing task, is_child = 0
0.45.228.286 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.45.228.286 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.45.228.292 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.45.228.293 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.45.228.297 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.45.228.297 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.45.228.309 I slot update_slots: id 3 | task 1478 | Checking checkpoint with [46, 46] against 51...
0.45.240.233 W slot update_slots: id 3 | task 1478 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.46.198.991 I slot print_timing: id 3 | task 1478 | prompt eval time = 113.50 ms / 20 tokens ( 5.67 ms per token, 176.21 tokens per second)
0.46.198.995 I slot print_timing: id 3 | task 1478 | eval time = 857.17 ms / 71 tokens ( 12.07 ms per token, 82.83 tokens per second)
0.46.198.995 I slot print_timing: id 3 | task 1478 | total time = 970.67 ms / 91 tokens
0.46.198.996 I slot print_timing: id 3 | task 1478 | graphs reused = 1399
0.46.199.025 I slot release: id 3 | task 1478 | stop processing: n_tokens = 137, truncated = 0
0.46.199.032 I srv update_slots: all slots are idle
0.46.229.053 I srv operator(): Chat format: peg-native
0.46.229.296 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.365
0.46.229.299 I srv get_availabl: updating prompt cache
0.46.229.327 W srv prompt_save: - saving prompt with length 137, total state size = 48.422 MiB (draft: 0.000 MiB)
0.46.296.084 I srv load: - looking for better prompt, base f_keep = 0.365, sim = 0.714
0.46.296.093 I srv load: - found better prompt with f_keep = 0.472, sim = 0.729
0.46.318.726 I srv update: - cache state: 11 prompts, 1056.703 MiB (limits: 8192.000 MiB, 8192 tokens, 12031 est)
0.46.318.728 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.46.318.729 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.46.318.730 I srv update: - prompt 0x5607593b3030: 129 tokens, checkpoints: 1, 95.993 MiB
0.46.318.730 I srv update: - prompt 0x560759469c30: 166 tokens, checkpoints: 1, 96.210 MiB
0.46.318.731 I srv update: - prompt 0x56075f4a7850: 124 tokens, checkpoints: 1, 95.964 MiB
0.46.318.731 I srv update: - prompt 0x56075f59f250: 197 tokens, checkpoints: 1, 96.392 MiB
0.46.318.731 I srv update: - prompt 0x5607592c9910: 172 tokens, checkpoints: 1, 96.245 MiB
0.46.318.732 I srv update: - prompt 0x5607593afc00: 105 tokens, checkpoints: 1, 95.852 MiB
0.46.318.732 I srv update: - prompt 0x56075f119d80: 147 tokens, checkpoints: 1, 96.099 MiB
0.46.318.733 I srv update: - prompt 0x560759477380: 122 tokens, checkpoints: 1, 95.952 MiB
0.46.318.733 I srv update: - prompt 0x5607592927a0: 137 tokens, checkpoints: 1, 96.040 MiB
0.46.318.735 I srv get_availabl: prompt cache update took 89.43 ms
0.46.318.830 I slot launch_slot_: id 3 | task 1551 | processing task, is_child = 0
0.46.318.832 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.46.318.833 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.46.318.839 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.46.318.839 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.46.318.849 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.46.318.849 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.46.318.868 I slot update_slots: id 3 | task 1551 | Checking checkpoint with [46, 46] against 51...
0.46.343.672 W slot update_slots: id 3 | task 1551 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.47.045.535 I slot print_timing: id 3 | task 1551 | prompt eval time = 149.63 ms / 23 tokens ( 6.51 ms per token, 153.71 tokens per second)
0.47.045.539 I slot print_timing: id 3 | task 1551 | eval time = 577.03 ms / 48 tokens ( 12.02 ms per token, 83.18 tokens per second)
0.47.045.539 I slot print_timing: id 3 | task 1551 | total time = 726.66 ms / 71 tokens
0.47.045.540 I slot print_timing: id 3 | task 1551 | graphs reused = 1445
0.47.045.565 I slot release: id 3 | task 1551 | stop processing: n_tokens = 117, truncated = 0
0.47.045.572 I srv update_slots: all slots are idle
0.47.075.552 I srv operator(): Chat format: peg-native
0.47.075.804 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.758 (> 0.100 thold), f_keep = 0.427
0.47.075.805 I srv get_availabl: updating prompt cache
0.47.075.824 W srv prompt_save: - saving prompt with length 117, total state size = 48.305 MiB (draft: 0.000 MiB)
0.47.134.365 I srv load: - looking for better prompt, base f_keep = 0.427, sim = 0.758
0.47.134.376 I srv update: - cache state: 12 prompts, 1152.626 MiB (limits: 8192.000 MiB, 8192 tokens, 11861 est)
0.47.134.377 I srv update: - prompt 0x56075930f5d0: 149 tokens, checkpoints: 1, 96.110 MiB
0.47.134.378 I srv update: - prompt 0x56075f5c0210: 104 tokens, checkpoints: 1, 95.846 MiB
0.47.134.379 I srv update: - prompt 0x5607593b3030: 129 tokens, checkpoints: 1, 95.993 MiB
0.47.134.380 I srv update: - prompt 0x560759469c30: 166 tokens, checkpoints: 1, 96.210 MiB
0.47.134.380 I srv update: - prompt 0x56075f4a7850: 124 tokens, checkpoints: 1, 95.964 MiB
0.47.134.381 I srv update: - prompt 0x56075f59f250: 197 tokens, checkpoints: 1, 96.392 MiB
0.47.134.381 I srv update: - prompt 0x5607592c9910: 172 tokens, checkpoints: 1, 96.245 MiB
0.47.134.382 I srv update: - prompt 0x5607593afc00: 105 tokens, checkpoints: 1, 95.852 MiB
0.47.134.382 I srv update: - prompt 0x56075f119d80: 147 tokens, checkpoints: 1, 96.099 MiB
0.47.134.383 I srv update: - prompt 0x560759477380: 122 tokens, checkpoints: 1, 95.952 MiB
0.47.134.383 I srv update: - prompt 0x5607592927a0: 137 tokens, checkpoints: 1, 96.040 MiB
0.47.134.384 I srv update: - prompt 0x560759403d50: 117 tokens, checkpoints: 1, 95.923 MiB
0.47.134.385 I srv get_availabl: prompt cache update took 58.58 ms
0.47.134.505 I slot launch_slot_: id 3 | task 1601 | processing task, is_child = 0
0.47.134.507 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.47.134.508 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.47.134.514 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.47.134.514 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.47.134.520 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.47.134.520 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.47.134.535 I slot update_slots: id 3 | task 1601 | Checking checkpoint with [46, 46] against 50...
0.47.146.372 W slot update_slots: id 3 | task 1601 | restored context checkpoint (pos_min = 46, pos_max = 46, n_tokens = 47, n_past = 47, size = 47.618 MiB)
0.48.520.424 I slot print_timing: id 3 | task 1601 | prompt eval time = 112.49 ms / 19 tokens ( 5.92 ms per token, 168.91 tokens per second)
0.48.520.428 I slot print_timing: id 3 | task 1601 | eval time = 1273.40 ms / 105 tokens ( 12.13 ms per token, 82.46 tokens per second)
0.48.520.429 I slot print_timing: id 3 | task 1601 | total time = 1385.88 ms / 124 tokens
0.48.520.430 I slot print_timing: id 3 | task 1601 | graphs reused = 1548
0.48.520.457 I slot release: id 3 | task 1601 | stop processing: n_tokens = 170, truncated = 0
0.48.520.466 I srv update_slots: all slots are idle
0.48.520.911 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink