qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results_roles/gpt-oss-20b.server.log

801 lines
79 KiB
Plaintext
Raw Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.065.075 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.065.078 I device_info:
0.00.065.098 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.103 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.110 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.065.168 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.065.171 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.065.209 I srv init: running without SSL
0.00.065.268 I srv init: using 23 threads for HTTP server
0.00.065.592 I srv start: binding port with default address family
0.00.066.824 I srv llama_server: loading model
0.00.066.832 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/openai_gpt-oss-20b-Q5_K_M.gguf'
0.00.066.882 I common_init_result: fitting params to device memory ...
0.00.066.884 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.01.542.424 W load: setting token '<|message|>' (200008) attribute to USER_DEFINED (16), old attributes: 8
0.01.542.429 W load: setting token '<|start|>' (200006) attribute to USER_DEFINED (16), old attributes: 8
0.01.542.429 W load: setting token '<|constrain|>' (200003) attribute to USER_DEFINED (16), old attributes: 8
0.01.543.611 W load: setting token '<|channel|>' (200005) attribute to USER_DEFINED (16), old attributes: 8
0.01.549.095 W load: special_eog_ids contains both '<|return|>' and '<|call|>', or '<|calls|>' and '<|flush|>' tokens, removing '<|end|>' token from EOG list
0.08.777.957 W llama_context: n_ctx_seq (24576) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
0.08.840.601 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.09.011.734 I srv load_model: initializing slots, n_slots = 4
0.09.044.181 W common_speculative_init: no implementations specified for speculative decoding
0.09.044.186 I slot load_model: id 0 | task -1 | new slot, n_ctx = 24576
0.09.044.189 I slot load_model: id 1 | task -1 | new slot, n_ctx = 24576
0.09.044.190 I slot load_model: id 2 | task -1 | new slot, n_ctx = 24576
0.09.044.190 I slot load_model: id 3 | task -1 | new slot, n_ctx = 24576
0.09.044.242 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.09.044.242 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.09.044.243 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.09.044.243 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.09.044.258 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.09.047.886 I init: chat template, example_format: '<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.
Knowledge cutoff: 2024-06
Current date: 2026-07-12
Reasoning: medium
# Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>developer<|message|># Instructions
You are a helpful assistant<|end|><|start|>user<|message|>Hello<|end|><|start|>assistant<|channel|>final<|message|>Hi there<|end|><|start|>user<|message|>How are you?<|end|><|start|>assistant'
0.09.048.451 I srv init: init: chat template, thinking = 1
0.09.048.486 I srv llama_server: model loaded
0.09.048.490 I srv llama_server: server is listening on http://127.0.0.1:8090
0.09.048.495 I srv update_slots: all slots are idle
0.09.970.521 I srv operator(): Chat format: peg-native
0.09.974.434 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.09.974.436 I srv get_availabl: updating prompt cache
0.09.974.441 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.09.974.445 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 24576 tokens, 8589934592 est)
0.09.974.446 I srv get_availabl: prompt cache update took 0.01 ms
0.09.975.348 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.09.975.351 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.09.975.352 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.09.975.371 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.09.975.372 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.09.975.385 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.09.975.385 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.10.662.486 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 103, n_tokens = 104, size = 2.439 MiB)
0.12.418.284 I slot print_timing: id 3 | task 0 | prompt eval time = 793.20 ms / 126 tokens ( 6.30 ms per token, 158.85 tokens per second)
0.12.418.289 I slot print_timing: id 3 | task 0 | eval time = 1649.67 ms / 145 tokens ( 11.38 ms per token, 87.90 tokens per second)
0.12.418.289 I slot print_timing: id 3 | task 0 | total time = 2442.87 ms / 271 tokens
0.12.418.294 I slot print_timing: id 3 | task 0 | graphs reused = 143
0.12.418.349 I slot release: id 3 | task 0 | stop processing: n_tokens = 270, truncated = 0
0.12.418.355 I srv update_slots: all slots are idle
0.12.426.529 I srv operator(): Chat format: peg-native
0.12.430.265 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.829 (> 0.100 thold), f_keep = 0.396
0.12.430.268 I srv get_availabl: updating prompt cache
0.12.430.308 W srv prompt_save: - saving prompt with length 270, total state size = 9.333 MiB (draft: 0.000 MiB)
0.12.435.448 I srv load: - looking for better prompt, base f_keep = 0.396, sim = 0.829
0.12.435.455 I srv update: - cache state: 1 prompts, 11.772 MiB (limits: 8192.000 MiB, 24576 tokens, 187885 est)
0.12.435.456 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
0.12.435.456 I srv get_availabl: prompt cache update took 5.19 ms
0.12.436.381 I slot launch_slot_: id 3 | task 148 | processing task, is_child = 0
0.12.436.383 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.12.436.384 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.12.436.399 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.12.436.400 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.12.436.413 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.12.436.413 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.12.436.433 I slot update_slots: id 3 | task 148 | Checking checkpoint with [0, 103] against 0...
0.12.438.075 W slot update_slots: id 3 | task 148 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 2.439 MiB)
0.13.009.257 I slot print_timing: id 3 | task 148 | prompt eval time = 115.36 ms / 26 tokens ( 4.44 ms per token, 225.39 tokens per second)
0.13.009.260 I slot print_timing: id 3 | task 148 | eval time = 457.46 ms / 41 tokens ( 11.16 ms per token, 89.62 tokens per second)
0.13.009.261 I slot print_timing: id 3 | task 148 | total time = 572.82 ms / 67 tokens
0.13.009.262 I slot print_timing: id 3 | task 148 | graphs reused = 182
0.13.009.290 I slot release: id 3 | task 148 | stop processing: n_tokens = 169, truncated = 0
0.13.009.299 I srv update_slots: all slots are idle
0.13.017.593 I srv operator(): Chat format: peg-native
0.13.021.356 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.718 (> 0.100 thold), f_keep = 0.633
0.13.022.296 I slot launch_slot_: id 3 | task 192 | processing task, is_child = 0
0.13.022.298 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.13.022.299 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.13.022.315 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.13.022.315 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.13.022.328 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.13.022.329 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.13.022.344 I slot update_slots: id 3 | task 192 | Checking checkpoint with [0, 103] against 0...
0.13.024.208 W slot update_slots: id 3 | task 192 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 2.439 MiB)
0.16.208.024 I slot print_timing: id 3 | task 192 | n_decoded = 264, tg = 87.88 t/s, tg_3s = 87.88 t/s
0.19.212.374 I slot print_timing: id 3 | task 192 | n_decoded = 525, tg = 87.38 t/s, tg_3s = 86.87 t/s
0.21.822.289 I slot print_timing: id 3 | task 192 | prompt eval time = 181.63 ms / 46 tokens ( 3.95 ms per token, 253.26 tokens per second)
0.21.822.293 I slot print_timing: id 3 | task 192 | eval time = 8618.31 ms / 750 tokens ( 11.49 ms per token, 87.02 tokens per second)
0.21.822.293 I slot print_timing: id 3 | task 192 | total time = 8799.94 ms / 796 tokens
0.21.822.294 I slot print_timing: id 3 | task 192 | graphs reused = 927
0.21.822.324 I slot release: id 3 | task 192 | stop processing: n_tokens = 898, truncated = 0
0.21.822.334 I srv update_slots: all slots are idle
0.21.830.764 I srv operator(): Chat format: peg-native
0.21.834.505 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.823 (> 0.100 thold), f_keep = 0.119
0.21.834.506 I srv get_availabl: updating prompt cache
0.21.834.564 W srv prompt_save: - saving prompt with length 898, total state size = 24.059 MiB (draft: 0.000 MiB)
0.21.854.538 I srv load: - looking for better prompt, base f_keep = 0.119, sim = 0.823
0.21.854.548 I srv update: - cache state: 2 prompts, 38.270 MiB (limits: 8192.000 MiB, 24576 tokens, 250016 est)
0.21.854.549 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
0.21.854.549 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
0.21.854.551 I srv get_availabl: prompt cache update took 20.04 ms
0.21.855.463 I slot launch_slot_: id 3 | task 945 | processing task, is_child = 0
0.21.855.467 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.21.855.467 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.21.855.489 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.21.855.489 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.21.855.503 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.21.855.504 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.21.855.525 I slot update_slots: id 3 | task 945 | Checking checkpoint with [0, 103] against 0...
0.21.857.304 W slot update_slots: id 3 | task 945 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 2.439 MiB)
0.24.820.089 I slot print_timing: id 3 | task 945 | prompt eval time = 119.33 ms / 27 tokens ( 4.42 ms per token, 226.26 tokens per second)
0.24.820.093 I slot print_timing: id 3 | task 945 | eval time = 2845.23 ms / 248 tokens ( 11.47 ms per token, 87.16 tokens per second)
0.24.820.094 I slot print_timing: id 3 | task 945 | total time = 2964.56 ms / 275 tokens
0.24.820.095 I slot print_timing: id 3 | task 945 | graphs reused = 1172
0.24.820.123 I slot release: id 3 | task 945 | stop processing: n_tokens = 377, truncated = 0
0.24.820.131 I srv update_slots: all slots are idle
0.24.828.285 I srv operator(): Chat format: peg-native
0.24.832.025 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.849 (> 0.100 thold), f_keep = 0.284
0.24.832.027 I srv get_availabl: updating prompt cache
0.24.832.068 W srv prompt_save: - saving prompt with length 377, total state size = 11.842 MiB (draft: 0.000 MiB)
0.24.841.731 I srv load: - looking for better prompt, base f_keep = 0.284, sim = 0.849
0.24.841.739 I srv update: - cache state: 3 prompts, 52.552 MiB (limits: 8192.000 MiB, 24576 tokens, 240841 est)
0.24.841.740 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
0.24.841.741 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
0.24.841.741 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
0.24.841.742 I srv get_availabl: prompt cache update took 9.71 ms
0.24.842.671 I slot launch_slot_: id 3 | task 1196 | processing task, is_child = 0
0.24.842.673 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.24.842.674 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.24.842.692 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.24.842.692 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.24.842.707 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.24.842.707 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.24.842.728 I slot update_slots: id 3 | task 1196 | Checking checkpoint with [0, 103] against 0...
0.24.844.255 W slot update_slots: id 3 | task 1196 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 2.439 MiB)
0.26.124.027 I slot print_timing: id 3 | task 1196 | prompt eval time = 116.52 ms / 23 tokens ( 5.07 ms per token, 197.39 tokens per second)
0.26.124.031 I slot print_timing: id 3 | task 1196 | eval time = 1164.78 ms / 102 tokens ( 11.42 ms per token, 87.57 tokens per second)
0.26.124.032 I slot print_timing: id 3 | task 1196 | total time = 1281.29 ms / 125 tokens
0.26.124.032 I slot print_timing: id 3 | task 1196 | graphs reused = 1272
0.26.124.063 I slot release: id 3 | task 1196 | stop processing: n_tokens = 227, truncated = 0
0.26.124.071 I srv update_slots: all slots are idle
0.26.131.659 I srv operator(): Chat format: peg-native
0.26.135.619 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.738 (> 0.100 thold), f_keep = 0.471
0.26.135.621 I srv get_availabl: updating prompt cache
0.26.135.659 W srv prompt_save: - saving prompt with length 227, total state size = 8.325 MiB (draft: 0.000 MiB)
0.26.144.441 I srv load: - looking for better prompt, base f_keep = 0.471, sim = 0.738
0.26.144.451 I srv update: - cache state: 4 prompts, 63.316 MiB (limits: 8192.000 MiB, 24576 tokens, 229267 est)
0.26.144.451 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
0.26.144.452 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
0.26.144.452 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
0.26.144.453 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
0.26.144.454 I srv get_availabl: prompt cache update took 8.83 ms
0.26.145.387 I slot launch_slot_: id 3 | task 1301 | processing task, is_child = 0
0.26.145.390 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.26.145.390 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.26.145.406 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.26.145.406 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.26.145.419 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.26.145.419 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.26.145.439 I slot update_slots: id 3 | task 1301 | Checking checkpoint with [0, 103] against 0...
0.26.147.160 W slot update_slots: id 3 | task 1301 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 2.439 MiB)
0.29.309.110 I slot print_timing: id 3 | task 1301 | n_decoded = 262, tg = 87.09 t/s, tg_3s = 87.09 t/s
0.29.585.109 I slot print_timing: id 3 | task 1301 | prompt eval time = 155.34 ms / 42 tokens ( 3.70 ms per token, 270.38 tokens per second)
0.29.585.112 I slot print_timing: id 3 | task 1301 | eval time = 3284.33 ms / 286 tokens ( 11.48 ms per token, 87.08 tokens per second)
0.29.585.113 I slot print_timing: id 3 | task 1301 | total time = 3439.66 ms / 328 tokens
0.29.585.113 I slot print_timing: id 3 | task 1301 | graphs reused = 1555
0.29.585.146 I slot release: id 3 | task 1301 | stop processing: n_tokens = 430, truncated = 0
0.29.585.154 I srv update_slots: all slots are idle
0.29.593.594 I srv operator(): Chat format: peg-native
0.29.597.312 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.748 (> 0.100 thold), f_keep = 0.249
0.29.597.314 I srv get_availabl: updating prompt cache
0.29.597.360 W srv prompt_save: - saving prompt with length 430, total state size = 13.085 MiB (draft: 0.000 MiB)
0.29.609.577 I srv load: - looking for better prompt, base f_keep = 0.249, sim = 0.748
0.29.609.586 I srv update: - cache state: 5 prompts, 78.840 MiB (limits: 8192.000 MiB, 24576 tokens, 228803 est)
0.29.609.586 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
0.29.609.587 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
0.29.609.588 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
0.29.609.588 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
0.29.609.589 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
0.29.609.590 I srv get_availabl: prompt cache update took 12.28 ms
0.29.610.512 I slot launch_slot_: id 3 | task 1590 | processing task, is_child = 0
0.29.610.514 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.610.515 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.610.533 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.610.533 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.610.546 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.610.547 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.29.610.566 I slot update_slots: id 3 | task 1590 | Checking checkpoint with [0, 103] against 0...
0.29.612.102 W slot update_slots: id 3 | task 1590 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 2.439 MiB)
0.32.781.095 I slot print_timing: id 3 | task 1590 | n_decoded = 262, tg = 87.08 t/s, tg_3s = 87.08 t/s
0.35.781.576 I slot print_timing: id 3 | task 1590 | n_decoded = 521, tg = 86.70 t/s, tg_3s = 86.32 t/s
0.38.784.260 I slot print_timing: id 3 | task 1590 | n_decoded = 777, tg = 86.22 t/s, tg_3s = 85.26 t/s
0.41.793.314 I slot print_timing: id 3 | task 1590 | n_decoded = 1030, tg = 85.68 t/s, tg_3s = 84.08 t/s
0.44.604.406 I slot print_timing: id 3 | task 1590 | prompt eval time = 161.93 ms / 40 tokens ( 4.05 ms per token, 247.03 tokens per second)
0.44.604.411 I slot print_timing: id 3 | task 1590 | eval time = 14831.91 ms / 1264 tokens ( 11.73 ms per token, 85.22 tokens per second)
0.44.604.411 I slot print_timing: id 3 | task 1590 | total time = 14993.83 ms / 1304 tokens
0.44.604.412 I slot print_timing: id 3 | task 1590 | graphs reused = 2812
0.44.604.446 I slot release: id 3 | task 1590 | stop processing: n_tokens = 1406, truncated = 0
0.44.604.458 I srv update_slots: all slots are idle
0.44.612.737 I srv operator(): Chat format: peg-native
0.44.616.508 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.829 (> 0.100 thold), f_keep = 0.076
0.44.616.510 I srv get_availabl: updating prompt cache
0.44.616.608 W srv prompt_save: - saving prompt with length 1406, total state size = 35.971 MiB (draft: 0.000 MiB)
0.44.675.656 I srv load: - looking for better prompt, base f_keep = 0.076, sim = 0.829
0.44.675.665 I srv update: - cache state: 6 prompts, 117.250 MiB (limits: 8192.000 MiB, 24576 tokens, 252082 est)
0.44.675.665 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
0.44.675.666 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
0.44.675.667 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
0.44.675.667 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
0.44.675.668 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
0.44.675.668 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
0.44.675.669 I srv get_availabl: prompt cache update took 59.16 ms
0.44.676.584 I slot launch_slot_: id 3 | task 2857 | processing task, is_child = 0
0.44.676.587 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.44.676.587 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.44.676.604 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.44.676.604 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.44.676.618 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.44.676.618 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.44.676.638 I slot update_slots: id 3 | task 2857 | Checking checkpoint with [0, 103] against 0...
0.44.678.334 W slot update_slots: id 3 | task 2857 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 2.439 MiB)
0.47.798.538 I slot print_timing: id 3 | task 2857 | n_decoded = 260, tg = 86.51 t/s, tg_3s = 86.51 t/s
0.50.807.275 I slot print_timing: id 3 | task 2857 | n_decoded = 518, tg = 86.13 t/s, tg_3s = 85.75 t/s
0.53.817.339 I slot print_timing: id 3 | task 2857 | n_decoded = 773, tg = 85.66 t/s, tg_3s = 84.72 t/s
0.56.819.478 I slot print_timing: id 3 | task 2857 | n_decoded = 1023, tg = 85.06 t/s, tg_3s = 83.27 t/s
0.59.823.497 I slot print_timing: id 3 | task 2857 | n_decoded = 1281, tg = 85.23 t/s, tg_3s = 85.88 t/s
1.02.832.176 I slot print_timing: id 3 | task 2857 | n_decoded = 1542, tg = 85.48 t/s, tg_3s = 86.75 t/s
1.05.837.246 I slot print_timing: id 3 | task 2857 | n_decoded = 1800, tg = 85.53 t/s, tg_3s = 85.85 t/s
1.08.845.265 I slot print_timing: id 3 | task 2857 | n_decoded = 2055, tg = 85.44 t/s, tg_3s = 84.77 t/s
1.11.853.973 I slot print_timing: id 3 | task 2857 | n_decoded = 2309, tg = 85.33 t/s, tg_3s = 84.42 t/s
1.12.816.970 I slot print_timing: id 3 | task 2857 | prompt eval time = 116.59 ms / 26 tokens ( 4.48 ms per token, 223.01 tokens per second)
1.12.816.973 I slot print_timing: id 3 | task 2857 | eval time = 28023.74 ms / 2390 tokens ( 11.73 ms per token, 85.28 tokens per second)
1.12.816.974 I slot print_timing: id 3 | task 2857 | total time = 28140.33 ms / 2416 tokens
1.12.816.975 I slot print_timing: id 3 | task 2857 | graphs reused = 5191
1.12.817.004 I slot release: id 3 | task 2857 | stop processing: n_tokens = 2518, truncated = 0
1.12.817.021 I srv update_slots: all slots are idle
1.12.825.560 I srv operator(): Chat format: peg-native
1.12.829.572 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.446 (> 0.100 thold), f_keep = 0.028
1.12.829.574 I srv get_availabl: updating prompt cache
1.12.829.669 W srv prompt_save: - saving prompt with length 2518, total state size = 62.046 MiB (draft: 0.000 MiB)
1.12.928.652 I srv load: - looking for better prompt, base f_keep = 0.028, sim = 0.446
1.12.928.662 I srv update: - cache state: 7 prompts, 181.736 MiB (limits: 8192.000 MiB, 24576 tokens, 276138 est)
1.12.928.663 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
1.12.928.663 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
1.12.928.664 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
1.12.928.665 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
1.12.928.665 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
1.12.928.666 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
1.12.928.666 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
1.12.928.667 I srv get_availabl: prompt cache update took 99.09 ms
1.12.929.596 I slot launch_slot_: id 3 | task 5250 | processing task, is_child = 0
1.12.929.598 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.12.929.599 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.12.929.618 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.12.929.619 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.12.929.633 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.12.929.633 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.12.929.654 I slot update_slots: id 3 | task 5250 | Checking checkpoint with [0, 103] against 0...
1.12.929.655 W slot update_slots: id 3 | task 5250 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.12.929.656 W slot update_slots: id 3 | task 5250 | erased invalidated context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_swa = 128, pos_next = 0, size = 2.439 MiB)
1.13.182.515 I slot create_check: id 3 | task 5250 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 121, n_tokens = 122, size = 2.861 MiB)
1.16.329.529 I slot print_timing: id 3 | task 5250 | n_decoded = 266, tg = 88.34 t/s, tg_3s = 88.34 t/s
1.19.339.092 I slot print_timing: id 3 | task 5250 | n_decoded = 528, tg = 87.70 t/s, tg_3s = 87.06 t/s
1.19.644.149 I slot print_timing: id 3 | task 5250 | prompt eval time = 388.89 ms / 157 tokens ( 2.48 ms per token, 403.72 tokens per second)
1.19.644.152 I slot print_timing: id 3 | task 5250 | eval time = 6325.60 ms / 554 tokens ( 11.42 ms per token, 87.58 tokens per second)
1.19.644.153 I slot print_timing: id 3 | task 5250 | total time = 6714.49 ms / 711 tokens
1.19.644.153 I slot print_timing: id 3 | task 5250 | graphs reused = 5741
1.19.644.188 I slot release: id 3 | task 5250 | stop processing: n_tokens = 710, truncated = 0
1.19.644.198 I srv update_slots: all slots are idle
1.19.652.649 I srv operator(): Chat format: peg-native
1.19.656.513 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.691 (> 0.100 thold), f_keep = 0.176
1.19.656.514 I srv get_availabl: updating prompt cache
1.19.656.571 W srv prompt_save: - saving prompt with length 710, total state size = 19.651 MiB (draft: 0.000 MiB)
1.19.674.161 I srv load: - looking for better prompt, base f_keep = 0.176, sim = 0.691
1.19.674.174 I srv update: - cache state: 8 prompts, 204.247 MiB (limits: 8192.000 MiB, 24576 tokens, 274179 est)
1.19.674.176 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
1.19.674.177 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
1.19.674.177 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
1.19.674.178 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
1.19.674.178 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
1.19.674.179 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
1.19.674.179 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
1.19.674.180 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
1.19.674.181 I srv get_availabl: prompt cache update took 17.67 ms
1.19.675.120 I slot launch_slot_: id 3 | task 5807 | processing task, is_child = 0
1.19.675.122 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.19.675.123 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.19.675.143 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.19.675.143 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.19.675.157 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.19.675.157 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.19.675.178 I slot update_slots: id 3 | task 5807 | Checking checkpoint with [0, 121] against 0...
1.19.677.292 W slot update_slots: id 3 | task 5807 | restored context checkpoint (pos_min = 0, pos_max = 121, n_tokens = 122, n_past = 121, size = 2.861 MiB)
1.22.894.025 I slot print_timing: id 3 | task 5807 | n_decoded = 263, tg = 87.61 t/s, tg_3s = 87.61 t/s
1.25.897.404 I slot print_timing: id 3 | task 5807 | n_decoded = 524, tg = 87.26 t/s, tg_3s = 86.90 t/s
1.28.900.986 I slot print_timing: id 3 | task 5807 | n_decoded = 782, tg = 86.80 t/s, tg_3s = 85.90 t/s
1.31.910.022 I slot print_timing: id 3 | task 5807 | n_decoded = 1038, tg = 86.37 t/s, tg_3s = 85.08 t/s
1.34.912.381 I slot print_timing: id 3 | task 5807 | n_decoded = 1292, tg = 86.02 t/s, tg_3s = 84.60 t/s
1.37.913.112 I slot print_timing: id 3 | task 5807 | n_decoded = 1543, tg = 85.62 t/s, tg_3s = 83.65 t/s
1.40.917.534 I slot print_timing: id 3 | task 5807 | n_decoded = 1793, tg = 85.28 t/s, tg_3s = 83.21 t/s
1.43.922.562 I slot print_timing: id 3 | task 5807 | n_decoded = 2043, tg = 85.02 t/s, tg_3s = 83.19 t/s
1.46.923.509 I slot print_timing: id 3 | task 5807 | n_decoded = 2292, tg = 84.79 t/s, tg_3s = 82.97 t/s
1.49.923.621 I slot print_timing: id 3 | task 5807 | n_decoded = 2541, tg = 84.61 t/s, tg_3s = 83.00 t/s
1.52.926.638 I slot print_timing: id 3 | task 5807 | n_decoded = 2793, tg = 84.55 t/s, tg_3s = 83.92 t/s
1.55.928.603 I slot print_timing: id 3 | task 5807 | n_decoded = 3048, tg = 84.58 t/s, tg_3s = 84.94 t/s
1.58.934.437 I slot print_timing: id 3 | task 5807 | n_decoded = 3302, tg = 84.57 t/s, tg_3s = 84.50 t/s
2.01.940.381 I slot print_timing: id 3 | task 5807 | n_decoded = 3552, tg = 84.47 t/s, tg_3s = 83.17 t/s
2.04.946.556 I slot print_timing: id 3 | task 5807 | n_decoded = 3802, tg = 84.39 t/s, tg_3s = 83.16 t/s
2.07.951.587 I slot print_timing: id 3 | task 5807 | n_decoded = 4052, tg = 84.31 t/s, tg_3s = 83.19 t/s
2.10.954.526 I slot print_timing: id 3 | task 5807 | n_decoded = 4302, tg = 84.25 t/s, tg_3s = 83.25 t/s
2.13.962.554 I slot print_timing: id 3 | task 5807 | n_decoded = 4552, tg = 84.19 t/s, tg_3s = 83.11 t/s
2.16.963.515 I slot print_timing: id 3 | task 5807 | n_decoded = 4801, tg = 84.12 t/s, tg_3s = 82.97 t/s
2.19.972.729 I slot print_timing: id 3 | task 5807 | n_decoded = 5051, tg = 84.07 t/s, tg_3s = 83.08 t/s
2.22.978.755 I slot print_timing: id 3 | task 5807 | n_decoded = 5301, tg = 84.03 t/s, tg_3s = 83.17 t/s
2.25.987.869 I slot print_timing: id 3 | task 5807 | n_decoded = 5551, tg = 83.98 t/s, tg_3s = 83.08 t/s
2.28.990.724 I slot print_timing: id 3 | task 5807 | n_decoded = 5798, tg = 83.91 t/s, tg_3s = 82.26 t/s
2.31.999.326 I slot print_timing: id 3 | task 5807 | n_decoded = 6046, tg = 83.85 t/s, tg_3s = 82.43 t/s
2.33.183.775 I slot print_timing: id 3 | task 5807 | prompt eval time = 216.88 ms / 60 tokens ( 3.61 ms per token, 276.65 tokens per second)
2.33.183.779 I slot print_timing: id 3 | task 5807 | eval time = 73291.71 ms / 6144 tokens ( 11.93 ms per token, 83.83 tokens per second)
2.33.183.779 I slot print_timing: id 3 | task 5807 | total time = 73508.59 ms / 6204 tokens
2.33.183.780 I slot print_timing: id 3 | task 5807 | graphs reused = 11859
2.33.183.819 I slot release: id 3 | task 5807 | stop processing: n_tokens = 6324, truncated = 0
2.33.183.847 I srv update_slots: all slots are idle
2.33.193.445 I srv operator(): Chat format: peg-native
2.33.197.479 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.706 (> 0.100 thold), f_keep = 0.020
2.33.197.481 I srv get_availabl: updating prompt cache
2.33.197.724 W srv prompt_save: - saving prompt with length 6324, total state size = 151.293 MiB (draft: 0.000 MiB)
2.33.336.553 I srv load: - looking for better prompt, base f_keep = 0.020, sim = 0.706
2.33.336.564 I srv update: - cache state: 9 prompts, 358.402 MiB (limits: 8192.000 MiB, 24576 tokens, 300798 est)
2.33.336.565 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
2.33.336.566 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
2.33.336.566 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
2.33.336.567 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
2.33.336.568 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
2.33.336.568 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
2.33.336.569 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
2.33.336.569 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
2.33.336.570 I srv update: - prompt 0x6091202847f0: 6324 tokens, checkpoints: 1, 154.154 MiB
2.33.336.571 I srv get_availabl: prompt cache update took 139.09 ms
2.33.337.531 I slot launch_slot_: id 3 | task 11954 | processing task, is_child = 0
2.33.337.533 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.33.337.534 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.33.337.564 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.33.337.565 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.33.337.581 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.33.337.581 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.33.337.605 I slot update_slots: id 3 | task 11954 | Checking checkpoint with [0, 121] against 0...
2.33.339.469 W slot update_slots: id 3 | task 11954 | restored context checkpoint (pos_min = 0, pos_max = 121, n_tokens = 122, n_past = 121, size = 2.861 MiB)
2.36.565.284 I slot print_timing: id 3 | task 11954 | n_decoded = 268, tg = 89.01 t/s, tg_3s = 89.01 t/s
2.39.569.112 I slot print_timing: id 3 | task 11954 | n_decoded = 533, tg = 88.62 t/s, tg_3s = 88.22 t/s
2.42.572.733 I slot print_timing: id 3 | task 11954 | n_decoded = 795, tg = 88.16 t/s, tg_3s = 87.23 t/s
2.45.580.868 I slot print_timing: id 3 | task 11954 | n_decoded = 1056, tg = 87.81 t/s, tg_3s = 86.76 t/s
2.48.586.217 I slot print_timing: id 3 | task 11954 | n_decoded = 1315, tg = 87.48 t/s, tg_3s = 86.18 t/s
2.51.590.492 I slot print_timing: id 3 | task 11954 | n_decoded = 1572, tg = 87.16 t/s, tg_3s = 85.54 t/s
2.54.599.905 I slot print_timing: id 3 | task 11954 | n_decoded = 1826, tg = 86.77 t/s, tg_3s = 84.40 t/s
2.57.363.955 I slot print_timing: id 3 | task 11954 | prompt eval time = 216.95 ms / 56 tokens ( 3.87 ms per token, 258.13 tokens per second)
2.57.363.958 I slot print_timing: id 3 | task 11954 | eval time = 23809.40 ms / 2056 tokens ( 11.58 ms per token, 86.35 tokens per second)
2.57.363.959 I slot print_timing: id 3 | task 11954 | total time = 24026.35 ms / 2112 tokens
2.57.363.959 I slot print_timing: id 3 | task 11954 | graphs reused = 13905
2.57.363.997 I slot release: id 3 | task 11954 | stop processing: n_tokens = 2232, truncated = 0
2.57.364.012 I srv update_slots: all slots are idle
2.57.373.565 I srv operator(): Chat format: peg-native
2.57.377.900 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.668 (> 0.100 thold), f_keep = 0.056
2.57.377.902 I srv get_availabl: updating prompt cache
2.57.378.009 W srv prompt_save: - saving prompt with length 2232, total state size = 55.340 MiB (draft: 0.000 MiB)
2.57.451.682 I srv load: - looking for better prompt, base f_keep = 0.056, sim = 0.668
2.57.451.692 I srv update: - cache state: 10 prompts, 416.603 MiB (limits: 8192.000 MiB, 24576 tokens, 302665 est)
2.57.451.693 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
2.57.451.694 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
2.57.451.695 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
2.57.451.696 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
2.57.451.696 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
2.57.451.697 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
2.57.451.697 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
2.57.451.697 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
2.57.451.698 I srv update: - prompt 0x6091202847f0: 6324 tokens, checkpoints: 1, 154.154 MiB
2.57.451.698 I srv update: - prompt 0x6091202d6680: 2232 tokens, checkpoints: 1, 58.201 MiB
2.57.451.699 I srv get_availabl: prompt cache update took 73.80 ms
2.57.452.555 I slot launch_slot_: id 3 | task 14013 | processing task, is_child = 0
2.57.452.557 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.57.452.558 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.57.452.576 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.57.452.576 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.57.452.590 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.57.452.590 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.57.452.610 I slot update_slots: id 3 | task 14013 | Checking checkpoint with [0, 121] against 0...
2.57.454.517 W slot update_slots: id 3 | task 14013 | restored context checkpoint (pos_min = 0, pos_max = 121, n_tokens = 122, n_past = 121, size = 2.861 MiB)
3.00.692.746 I slot print_timing: id 3 | task 14013 | n_decoded = 263, tg = 87.55 t/s, tg_3s = 87.55 t/s
3.03.700.894 I slot print_timing: id 3 | task 14013 | n_decoded = 523, tg = 86.99 t/s, tg_3s = 86.43 t/s
3.06.710.446 I slot print_timing: id 3 | task 14013 | n_decoded = 781, tg = 86.57 t/s, tg_3s = 85.73 t/s
3.09.711.921 I slot print_timing: id 3 | task 14013 | n_decoded = 1036, tg = 86.17 t/s, tg_3s = 84.96 t/s
3.12.525.939 I slot print_timing: id 3 | task 14013 | prompt eval time = 236.04 ms / 66 tokens ( 3.58 ms per token, 279.61 tokens per second)
3.12.525.942 I slot print_timing: id 3 | task 14013 | eval time = 14837.28 ms / 1272 tokens ( 11.66 ms per token, 85.73 tokens per second)
3.12.525.943 I slot print_timing: id 3 | task 14013 | total time = 15073.33 ms / 1338 tokens
3.12.525.944 I slot print_timing: id 3 | task 14013 | graphs reused = 15170
3.12.525.977 I slot release: id 3 | task 14013 | stop processing: n_tokens = 1458, truncated = 0
3.12.525.988 I srv update_slots: all slots are idle
3.12.534.423 I srv operator(): Chat format: peg-native
3.12.538.286 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.086
3.12.538.289 I srv get_availabl: updating prompt cache
3.12.538.365 W srv prompt_save: - saving prompt with length 1458, total state size = 37.191 MiB (draft: 0.000 MiB)
3.12.590.470 I srv load: - looking for better prompt, base f_keep = 0.086, sim = 0.776
3.12.590.480 I srv update: - cache state: 11 prompts, 456.654 MiB (limits: 8192.000 MiB, 24576 tokens, 302274 est)
3.12.590.481 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
3.12.590.482 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
3.12.590.482 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
3.12.590.483 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
3.12.590.484 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
3.12.590.484 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
3.12.590.485 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
3.12.590.485 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
3.12.590.486 I srv update: - prompt 0x6091202847f0: 6324 tokens, checkpoints: 1, 154.154 MiB
3.12.590.486 I srv update: - prompt 0x6091202d6680: 2232 tokens, checkpoints: 1, 58.201 MiB
3.12.590.487 I srv update: - prompt 0x6091203be2f0: 1458 tokens, checkpoints: 1, 40.052 MiB
3.12.590.488 I srv get_availabl: prompt cache update took 52.20 ms
3.12.591.413 I slot launch_slot_: id 3 | task 15288 | processing task, is_child = 0
3.12.591.415 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.12.591.416 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.12.591.432 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.12.591.433 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.12.591.447 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.12.591.447 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.12.591.467 I slot update_slots: id 3 | task 15288 | Checking checkpoint with [0, 121] against 0...
3.12.593.329 W slot update_slots: id 3 | task 15288 | restored context checkpoint (pos_min = 0, pos_max = 121, n_tokens = 122, n_past = 121, size = 2.861 MiB)
3.15.751.356 I slot print_timing: id 3 | task 15288 | n_decoded = 261, tg = 86.85 t/s, tg_3s = 86.85 t/s
3.18.755.134 I slot print_timing: id 3 | task 15288 | n_decoded = 518, tg = 86.20 t/s, tg_3s = 85.56 t/s
3.20.886.497 I slot print_timing: id 3 | task 15288 | prompt eval time = 154.65 ms / 40 tokens ( 3.87 ms per token, 258.66 tokens per second)
3.20.886.501 I slot print_timing: id 3 | task 15288 | eval time = 8140.38 ms / 698 tokens ( 11.66 ms per token, 85.75 tokens per second)
3.20.886.502 I slot print_timing: id 3 | task 15288 | total time = 8295.02 ms / 738 tokens
3.20.886.503 I slot print_timing: id 3 | task 15288 | graphs reused = 15863
3.20.886.542 I slot release: id 3 | task 15288 | stop processing: n_tokens = 858, truncated = 0
3.20.886.555 I srv update_slots: all slots are idle
3.20.894.933 I srv operator(): Chat format: peg-native
3.20.898.767 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.706 (> 0.100 thold), f_keep = 0.146
3.20.898.770 I srv get_availabl: updating prompt cache
3.20.898.854 W srv prompt_save: - saving prompt with length 858, total state size = 23.121 MiB (draft: 0.000 MiB)
3.20.918.705 I srv load: - looking for better prompt, base f_keep = 0.146, sim = 0.706
3.20.918.720 I srv update: - cache state: 12 prompts, 482.637 MiB (limits: 8192.000 MiB, 24576 tokens, 300565 est)
3.20.918.722 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
3.20.918.723 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
3.20.918.724 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
3.20.918.725 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
3.20.918.725 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
3.20.918.727 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
3.20.918.728 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
3.20.918.729 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
3.20.918.729 I srv update: - prompt 0x6091202847f0: 6324 tokens, checkpoints: 1, 154.154 MiB
3.20.918.730 I srv update: - prompt 0x6091202d6680: 2232 tokens, checkpoints: 1, 58.201 MiB
3.20.918.730 I srv update: - prompt 0x6091203be2f0: 1458 tokens, checkpoints: 1, 40.052 MiB
3.20.918.731 I srv update: - prompt 0x6091274e2a70: 858 tokens, checkpoints: 1, 25.982 MiB
3.20.918.733 I srv get_availabl: prompt cache update took 19.96 ms
3.20.919.694 I slot launch_slot_: id 3 | task 15989 | processing task, is_child = 0
3.20.919.697 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.20.919.697 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.20.919.721 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.20.919.721 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.20.919.738 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.20.919.738 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.20.919.763 I slot update_slots: id 3 | task 15989 | Checking checkpoint with [0, 121] against 0...
3.20.921.641 W slot update_slots: id 3 | task 15989 | restored context checkpoint (pos_min = 0, pos_max = 121, n_tokens = 122, n_past = 121, size = 2.861 MiB)
3.24.136.962 I slot print_timing: id 3 | task 15989 | n_decoded = 261, tg = 86.93 t/s, tg_3s = 86.93 t/s
3.27.139.434 I slot print_timing: id 3 | task 15989 | n_decoded = 521, tg = 86.76 t/s, tg_3s = 86.60 t/s
3.27.496.386 I slot print_timing: id 3 | task 15989 | prompt eval time = 214.71 ms / 56 tokens ( 3.83 ms per token, 260.82 tokens per second)
3.27.496.389 I slot print_timing: id 3 | task 15989 | eval time = 6361.91 ms / 552 tokens ( 11.53 ms per token, 86.77 tokens per second)
3.27.496.389 I slot print_timing: id 3 | task 15989 | total time = 6576.62 ms / 608 tokens
3.27.496.390 I slot print_timing: id 3 | task 15989 | graphs reused = 16411
3.27.496.422 I slot release: id 3 | task 15989 | stop processing: n_tokens = 728, truncated = 0
3.27.496.431 I srv update_slots: all slots are idle
3.27.505.163 I srv operator(): Chat format: peg-native
3.27.508.948 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.374 (> 0.100 thold), f_keep = 0.096
3.27.508.950 I srv get_availabl: updating prompt cache
3.27.509.003 W srv prompt_save: - saving prompt with length 728, total state size = 20.073 MiB (draft: 0.000 MiB)
3.27.525.840 I srv load: - looking for better prompt, base f_keep = 0.096, sim = 0.374
3.27.525.854 I srv update: - cache state: 13 prompts, 505.571 MiB (limits: 8192.000 MiB, 24576 tokens, 298727 est)
3.27.525.855 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
3.27.525.856 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
3.27.525.856 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
3.27.525.857 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
3.27.525.857 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
3.27.525.858 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
3.27.525.858 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
3.27.525.859 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
3.27.525.859 I srv update: - prompt 0x6091202847f0: 6324 tokens, checkpoints: 1, 154.154 MiB
3.27.525.859 I srv update: - prompt 0x6091202d6680: 2232 tokens, checkpoints: 1, 58.201 MiB
3.27.525.860 I srv update: - prompt 0x6091203be2f0: 1458 tokens, checkpoints: 1, 40.052 MiB
3.27.525.860 I srv update: - prompt 0x6091274e2a70: 858 tokens, checkpoints: 1, 25.982 MiB
3.27.525.861 I srv update: - prompt 0x60912027f930: 728 tokens, checkpoints: 1, 22.934 MiB
3.27.525.862 I srv get_availabl: prompt cache update took 16.91 ms
3.27.526.700 I slot launch_slot_: id 3 | task 16544 | processing task, is_child = 0
3.27.526.702 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.27.526.703 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.27.526.722 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.27.526.722 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.27.526.734 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.27.526.735 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.27.526.754 I slot update_slots: id 3 | task 16544 | Checking checkpoint with [0, 121] against 0...
3.27.526.755 W slot update_slots: id 3 | task 16544 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
3.27.526.756 W slot update_slots: id 3 | task 16544 | erased invalidated context checkpoint (pos_min = 0, pos_max = 121, n_tokens = 122, n_swa = 128, pos_next = 0, size = 2.861 MiB)
3.27.774.132 I slot create_check: id 3 | task 16544 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 116, n_tokens = 117, size = 2.744 MiB)
3.30.730.842 I slot print_timing: id 3 | task 16544 | prompt eval time = 425.43 ms / 187 tokens ( 2.28 ms per token, 439.55 tokens per second)
3.30.730.846 I slot print_timing: id 3 | task 16544 | eval time = 2778.65 ms / 244 tokens ( 11.39 ms per token, 87.81 tokens per second)
3.30.730.847 I slot print_timing: id 3 | task 16544 | total time = 3204.08 ms / 431 tokens
3.30.730.847 I slot print_timing: id 3 | task 16544 | graphs reused = 16652
3.30.730.881 I slot release: id 3 | task 16544 | stop processing: n_tokens = 430, truncated = 0
3.30.730.890 I srv update_slots: all slots are idle
3.30.801.205 I srv operator(): Chat format: peg-native
3.30.805.016 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.291
3.30.805.018 I srv get_availabl: updating prompt cache
3.30.805.076 W srv prompt_save: - saving prompt with length 430, total state size = 13.085 MiB (draft: 0.000 MiB)
3.30.817.478 I srv load: - looking for better prompt, base f_keep = 0.291, sim = 0.833
3.30.817.493 I srv update: - cache state: 14 prompts, 521.400 MiB (limits: 8192.000 MiB, 24576 tokens, 296414 est)
3.30.817.494 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
3.30.817.495 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
3.30.817.496 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
3.30.817.496 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
3.30.817.497 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
3.30.817.497 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
3.30.817.498 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
3.30.817.498 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
3.30.817.500 I srv update: - prompt 0x6091202847f0: 6324 tokens, checkpoints: 1, 154.154 MiB
3.30.817.501 I srv update: - prompt 0x6091202d6680: 2232 tokens, checkpoints: 1, 58.201 MiB
3.30.817.501 I srv update: - prompt 0x6091203be2f0: 1458 tokens, checkpoints: 1, 40.052 MiB
3.30.817.502 I srv update: - prompt 0x6091274e2a70: 858 tokens, checkpoints: 1, 25.982 MiB
3.30.817.502 I srv update: - prompt 0x60912027f930: 728 tokens, checkpoints: 1, 22.934 MiB
3.30.817.503 I srv update: - prompt 0x609120118aa0: 430 tokens, checkpoints: 1, 15.829 MiB
3.30.817.504 I srv get_availabl: prompt cache update took 12.48 ms
3.30.818.444 I slot launch_slot_: id 3 | task 16791 | processing task, is_child = 0
3.30.818.446 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.30.818.447 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.30.818.466 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.30.818.467 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.30.818.480 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.30.818.481 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.30.818.501 I slot update_slots: id 3 | task 16791 | Checking checkpoint with [0, 116] against 0...
3.30.820.480 W slot update_slots: id 3 | task 16791 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 2.744 MiB)
3.33.965.061 I slot print_timing: id 3 | task 16791 | n_decoded = 269, tg = 89.55 t/s, tg_3s = 89.55 t/s
3.34.426.869 I slot print_timing: id 3 | task 16791 | prompt eval time = 142.58 ms / 34 tokens ( 4.19 ms per token, 238.46 tokens per second)
3.34.426.872 I slot print_timing: id 3 | task 16791 | eval time = 3465.78 ms / 310 tokens ( 11.18 ms per token, 89.45 tokens per second)
3.34.426.873 I slot print_timing: id 3 | task 16791 | total time = 3608.36 ms / 344 tokens
3.34.426.874 I slot print_timing: id 3 | task 16791 | graphs reused = 16959
3.34.426.907 I slot release: id 3 | task 16791 | stop processing: n_tokens = 459, truncated = 0
3.34.426.917 I srv update_slots: all slots are idle
3.34.481.152 I srv operator(): Chat format: peg-native
3.34.485.048 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.796 (> 0.100 thold), f_keep = 0.272
3.34.485.050 I srv get_availabl: updating prompt cache
3.34.485.103 W srv prompt_save: - saving prompt with length 459, total state size = 13.765 MiB (draft: 0.000 MiB)
3.34.497.894 I srv load: - looking for better prompt, base f_keep = 0.272, sim = 0.796
3.34.497.910 I srv update: - cache state: 15 prompts, 537.909 MiB (limits: 8192.000 MiB, 24576 tokens, 294307 est)
3.34.497.911 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
3.34.497.911 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
3.34.497.912 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
3.34.497.912 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
3.34.497.913 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
3.34.497.913 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
3.34.497.914 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
3.34.497.914 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
3.34.497.915 I srv update: - prompt 0x6091202847f0: 6324 tokens, checkpoints: 1, 154.154 MiB
3.34.497.915 I srv update: - prompt 0x6091202d6680: 2232 tokens, checkpoints: 1, 58.201 MiB
3.34.497.916 I srv update: - prompt 0x6091203be2f0: 1458 tokens, checkpoints: 1, 40.052 MiB
3.34.497.917 I srv update: - prompt 0x6091274e2a70: 858 tokens, checkpoints: 1, 25.982 MiB
3.34.497.917 I srv update: - prompt 0x60912027f930: 728 tokens, checkpoints: 1, 22.934 MiB
3.34.497.917 I srv update: - prompt 0x609120118aa0: 430 tokens, checkpoints: 1, 15.829 MiB
3.34.497.918 I srv update: - prompt 0x60912023bd70: 459 tokens, checkpoints: 1, 16.509 MiB
3.34.497.919 I srv get_availabl: prompt cache update took 12.87 ms
3.34.498.855 I slot launch_slot_: id 3 | task 17104 | processing task, is_child = 0
3.34.498.857 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.34.498.858 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.34.498.877 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.34.498.877 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.34.498.891 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.34.498.891 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.34.498.910 I slot update_slots: id 3 | task 17104 | Checking checkpoint with [0, 116] against 0...
3.34.500.589 W slot update_slots: id 3 | task 17104 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 2.744 MiB)
3.37.313.861 I slot print_timing: id 3 | task 17104 | prompt eval time = 163.13 ms / 41 tokens ( 3.98 ms per token, 251.33 tokens per second)
3.37.313.865 I slot print_timing: id 3 | task 17104 | eval time = 2651.82 ms / 236 tokens ( 11.24 ms per token, 89.00 tokens per second)
3.37.313.865 I slot print_timing: id 3 | task 17104 | total time = 2814.95 ms / 277 tokens
3.37.313.866 I slot print_timing: id 3 | task 17104 | graphs reused = 17192
3.37.313.898 I slot release: id 3 | task 17104 | stop processing: n_tokens = 392, truncated = 0
3.37.313.907 I srv update_slots: all slots are idle
3.37.393.356 I srv operator(): Chat format: peg-native
3.37.397.219 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.672 (> 0.100 thold), f_keep = 0.319
3.37.397.221 I srv get_availabl: updating prompt cache
3.37.397.264 W srv prompt_save: - saving prompt with length 392, total state size = 12.194 MiB (draft: 0.000 MiB)
3.37.408.894 I srv load: - looking for better prompt, base f_keep = 0.319, sim = 0.672
3.37.408.908 I srv update: - cache state: 16 prompts, 552.846 MiB (limits: 8192.000 MiB, 24576 tokens, 292163 est)
3.37.408.909 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
3.37.408.909 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
3.37.408.910 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
3.37.408.910 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
3.37.408.911 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
3.37.408.912 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
3.37.408.912 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
3.37.408.913 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
3.37.408.913 I srv update: - prompt 0x6091202847f0: 6324 tokens, checkpoints: 1, 154.154 MiB
3.37.408.914 I srv update: - prompt 0x6091202d6680: 2232 tokens, checkpoints: 1, 58.201 MiB
3.37.408.914 I srv update: - prompt 0x6091203be2f0: 1458 tokens, checkpoints: 1, 40.052 MiB
3.37.408.915 I srv update: - prompt 0x6091274e2a70: 858 tokens, checkpoints: 1, 25.982 MiB
3.37.408.915 I srv update: - prompt 0x60912027f930: 728 tokens, checkpoints: 1, 22.934 MiB
3.37.408.916 I srv update: - prompt 0x609120118aa0: 430 tokens, checkpoints: 1, 15.829 MiB
3.37.408.916 I srv update: - prompt 0x60912023bd70: 459 tokens, checkpoints: 1, 16.509 MiB
3.37.408.917 I srv update: - prompt 0x609120403390: 392 tokens, checkpoints: 1, 14.938 MiB
3.37.408.918 I srv get_availabl: prompt cache update took 11.70 ms
3.37.409.880 I slot launch_slot_: id 3 | task 17343 | processing task, is_child = 0
3.37.409.882 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.37.409.882 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.37.409.901 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.37.409.901 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.37.409.914 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.37.409.914 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.37.409.934 I slot update_slots: id 3 | task 17343 | Checking checkpoint with [0, 116] against 0...
3.37.411.598 W slot update_slots: id 3 | task 17343 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 2.744 MiB)
3.40.597.875 I slot print_timing: id 3 | task 17343 | n_decoded = 266, tg = 88.61 t/s, tg_3s = 88.61 t/s
3.43.601.367 I slot print_timing: id 3 | task 17343 | n_decoded = 529, tg = 88.09 t/s, tg_3s = 87.56 t/s
3.46.607.306 I slot print_timing: id 3 | task 17343 | n_decoded = 788, tg = 87.45 t/s, tg_3s = 86.16 t/s
3.48.104.583 I slot print_timing: id 3 | task 17343 | prompt eval time = 186.10 ms / 70 tokens ( 2.66 ms per token, 376.13 tokens per second)
3.48.104.587 I slot print_timing: id 3 | task 17343 | eval time = 10508.54 ms / 916 tokens ( 11.47 ms per token, 87.17 tokens per second)
3.48.104.587 I slot print_timing: id 3 | task 17343 | total time = 10694.64 ms / 986 tokens
3.48.104.588 I slot print_timing: id 3 | task 17343 | graphs reused = 18102
3.48.104.625 I slot release: id 3 | task 17343 | stop processing: n_tokens = 1101, truncated = 0
3.48.104.635 I srv update_slots: all slots are idle
3.48.183.857 I srv operator(): Chat format: peg-native
3.48.187.826 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.781 (> 0.100 thold), f_keep = 0.114
3.48.187.828 I srv get_availabl: updating prompt cache
3.48.187.902 W srv prompt_save: - saving prompt with length 1101, total state size = 28.819 MiB (draft: 0.000 MiB)
3.48.217.243 I srv load: - looking for better prompt, base f_keep = 0.114, sim = 0.781
3.48.217.256 I srv update: - cache state: 17 prompts, 584.410 MiB (limits: 8192.000 MiB, 24576 tokens, 291817 est)
3.48.217.257 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
3.48.217.257 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
3.48.217.258 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
3.48.217.259 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
3.48.217.259 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
3.48.217.260 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
3.48.217.260 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
3.48.217.261 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
3.48.217.261 I srv update: - prompt 0x6091202847f0: 6324 tokens, checkpoints: 1, 154.154 MiB
3.48.217.262 I srv update: - prompt 0x6091202d6680: 2232 tokens, checkpoints: 1, 58.201 MiB
3.48.217.262 I srv update: - prompt 0x6091203be2f0: 1458 tokens, checkpoints: 1, 40.052 MiB
3.48.217.263 I srv update: - prompt 0x6091274e2a70: 858 tokens, checkpoints: 1, 25.982 MiB
3.48.217.263 I srv update: - prompt 0x60912027f930: 728 tokens, checkpoints: 1, 22.934 MiB
3.48.217.264 I srv update: - prompt 0x609120118aa0: 430 tokens, checkpoints: 1, 15.829 MiB
3.48.217.264 I srv update: - prompt 0x60912023bd70: 459 tokens, checkpoints: 1, 16.509 MiB
3.48.217.264 I srv update: - prompt 0x609120403390: 392 tokens, checkpoints: 1, 14.938 MiB
3.48.217.265 I srv update: - prompt 0x6091203da560: 1101 tokens, checkpoints: 1, 31.563 MiB
3.48.217.266 I srv get_availabl: prompt cache update took 29.44 ms
3.48.218.222 I slot launch_slot_: id 3 | task 18262 | processing task, is_child = 0
3.48.218.224 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.48.218.225 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.48.218.241 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.48.218.241 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.48.218.255 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.48.218.255 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.48.218.277 I slot update_slots: id 3 | task 18262 | Checking checkpoint with [0, 116] against 0...
3.48.220.183 W slot update_slots: id 3 | task 18262 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 2.744 MiB)
3.51.397.965 I slot print_timing: id 3 | task 18262 | n_decoded = 262, tg = 87.05 t/s, tg_3s = 87.05 t/s
3.51.604.943 I slot print_timing: id 3 | task 18262 | prompt eval time = 169.90 ms / 44 tokens ( 3.86 ms per token, 258.97 tokens per second)
3.51.604.946 I slot print_timing: id 3 | task 18262 | eval time = 3216.76 ms / 280 tokens ( 11.49 ms per token, 87.04 tokens per second)
3.51.604.946 I slot print_timing: id 3 | task 18262 | total time = 3386.66 ms / 324 tokens
3.51.604.947 I slot print_timing: id 3 | task 18262 | graphs reused = 18379
3.51.604.980 I slot release: id 3 | task 18262 | stop processing: n_tokens = 439, truncated = 0
3.51.604.989 I srv update_slots: all slots are idle
3.51.657.344 I srv operator(): Chat format: peg-native
3.51.661.191 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.702 (> 0.100 thold), f_keep = 0.273
3.51.661.193 I srv get_availabl: updating prompt cache
3.51.661.239 W srv prompt_save: - saving prompt with length 439, total state size = 13.296 MiB (draft: 0.000 MiB)
3.51.674.043 I srv load: - looking for better prompt, base f_keep = 0.273, sim = 0.702
3.51.674.058 I srv update: - cache state: 18 prompts, 600.450 MiB (limits: 8192.000 MiB, 24576 tokens, 290011 est)
3.51.674.058 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
3.51.674.059 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
3.51.674.059 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
3.51.674.060 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
3.51.674.061 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
3.51.674.061 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
3.51.674.062 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
3.51.674.062 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
3.51.674.063 I srv update: - prompt 0x6091202847f0: 6324 tokens, checkpoints: 1, 154.154 MiB
3.51.674.063 I srv update: - prompt 0x6091202d6680: 2232 tokens, checkpoints: 1, 58.201 MiB
3.51.674.064 I srv update: - prompt 0x6091203be2f0: 1458 tokens, checkpoints: 1, 40.052 MiB
3.51.674.064 I srv update: - prompt 0x6091274e2a70: 858 tokens, checkpoints: 1, 25.982 MiB
3.51.674.065 I srv update: - prompt 0x60912027f930: 728 tokens, checkpoints: 1, 22.934 MiB
3.51.674.065 I srv update: - prompt 0x609120118aa0: 430 tokens, checkpoints: 1, 15.829 MiB
3.51.674.066 I srv update: - prompt 0x60912023bd70: 459 tokens, checkpoints: 1, 16.509 MiB
3.51.674.066 I srv update: - prompt 0x609120403390: 392 tokens, checkpoints: 1, 14.938 MiB
3.51.674.067 I srv update: - prompt 0x6091203da560: 1101 tokens, checkpoints: 1, 31.563 MiB
3.51.674.067 I srv update: - prompt 0x609120301c40: 439 tokens, checkpoints: 1, 16.040 MiB
3.51.674.068 I srv get_availabl: prompt cache update took 12.87 ms
3.51.675.000 I slot launch_slot_: id 3 | task 18545 | processing task, is_child = 0
3.51.675.003 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.51.675.003 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.51.675.021 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.51.675.022 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.51.675.035 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.51.675.035 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.51.675.057 I slot update_slots: id 3 | task 18545 | Checking checkpoint with [0, 116] against 0...
3.51.676.833 W slot update_slots: id 3 | task 18545 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 2.744 MiB)
3.53.420.206 I slot print_timing: id 3 | task 18545 | prompt eval time = 209.70 ms / 55 tokens ( 3.81 ms per token, 262.28 tokens per second)
3.53.420.209 I slot print_timing: id 3 | task 18545 | eval time = 1535.44 ms / 134 tokens ( 11.46 ms per token, 87.27 tokens per second)
3.53.420.210 I slot print_timing: id 3 | task 18545 | total time = 1745.14 ms / 189 tokens
3.53.420.211 I slot print_timing: id 3 | task 18545 | graphs reused = 18510
3.53.420.243 I slot release: id 3 | task 18545 | stop processing: n_tokens = 304, truncated = 0
3.53.420.251 I srv update_slots: all slots are idle
3.53.428.409 I srv operator(): Chat format: peg-native
3.53.432.450 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.759 (> 0.100 thold), f_keep = 0.395
3.53.432.452 I srv get_availabl: updating prompt cache
3.53.432.492 W srv prompt_save: - saving prompt with length 304, total state size = 10.131 MiB (draft: 0.000 MiB)
3.53.442.559 I srv load: - looking for better prompt, base f_keep = 0.395, sim = 0.759
3.53.442.573 I srv update: - cache state: 19 prompts, 613.324 MiB (limits: 8192.000 MiB, 24576 tokens, 287984 est)
3.53.442.574 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
3.53.442.575 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
3.53.442.575 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
3.53.442.576 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
3.53.442.576 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
3.53.442.577 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
3.53.442.577 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
3.53.442.578 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
3.53.442.578 I srv update: - prompt 0x6091202847f0: 6324 tokens, checkpoints: 1, 154.154 MiB
3.53.442.579 I srv update: - prompt 0x6091202d6680: 2232 tokens, checkpoints: 1, 58.201 MiB
3.53.442.579 I srv update: - prompt 0x6091203be2f0: 1458 tokens, checkpoints: 1, 40.052 MiB
3.53.442.580 I srv update: - prompt 0x6091274e2a70: 858 tokens, checkpoints: 1, 25.982 MiB
3.53.442.580 I srv update: - prompt 0x60912027f930: 728 tokens, checkpoints: 1, 22.934 MiB
3.53.442.581 I srv update: - prompt 0x609120118aa0: 430 tokens, checkpoints: 1, 15.829 MiB
3.53.442.581 I srv update: - prompt 0x60912023bd70: 459 tokens, checkpoints: 1, 16.509 MiB
3.53.442.582 I srv update: - prompt 0x609120403390: 392 tokens, checkpoints: 1, 14.938 MiB
3.53.442.582 I srv update: - prompt 0x6091203da560: 1101 tokens, checkpoints: 1, 31.563 MiB
3.53.442.583 I srv update: - prompt 0x609120301c40: 439 tokens, checkpoints: 1, 16.040 MiB
3.53.442.583 I srv update: - prompt 0x6091275b2a30: 304 tokens, checkpoints: 1, 12.874 MiB
3.53.442.584 I srv get_availabl: prompt cache update took 10.13 ms
3.53.443.515 I slot launch_slot_: id 3 | task 18682 | processing task, is_child = 0
3.53.443.517 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.53.443.517 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.53.443.534 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.53.443.534 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.53.443.547 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.53.443.548 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.53.443.568 I slot update_slots: id 3 | task 18682 | Checking checkpoint with [0, 116] against 0...
3.53.445.236 W slot update_slots: id 3 | task 18682 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 2.744 MiB)
3.56.387.967 I slot print_timing: id 3 | task 18682 | prompt eval time = 169.12 ms / 42 tokens ( 4.03 ms per token, 248.34 tokens per second)
3.56.387.971 I slot print_timing: id 3 | task 18682 | eval time = 2775.27 ms / 242 tokens ( 11.47 ms per token, 87.20 tokens per second)
3.56.387.971 I slot print_timing: id 3 | task 18682 | total time = 2944.39 ms / 284 tokens
3.56.387.972 I slot print_timing: id 3 | task 18682 | graphs reused = 18749
3.56.388.005 I slot release: id 3 | task 18682 | stop processing: n_tokens = 399, truncated = 0
3.56.388.014 I srv update_slots: all slots are idle
3.56.458.011 I srv operator(): Chat format: peg-native
3.56.461.796 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.786 (> 0.100 thold), f_keep = 0.313
3.56.461.798 I srv get_availabl: updating prompt cache
3.56.461.851 W srv prompt_save: - saving prompt with length 399, total state size = 12.358 MiB (draft: 0.000 MiB)
3.56.473.753 I srv load: - looking for better prompt, base f_keep = 0.313, sim = 0.786
3.56.473.769 I srv update: - cache state: 20 prompts, 628.426 MiB (limits: 8192.000 MiB, 24576 tokens, 286264 est)
3.56.473.770 I srv update: - prompt 0x60912021a6a0: 270 tokens, checkpoints: 1, 11.772 MiB
3.56.473.770 I srv update: - prompt 0x609120397320: 898 tokens, checkpoints: 1, 26.498 MiB
3.56.473.771 I srv update: - prompt 0x60911f5b9880: 377 tokens, checkpoints: 1, 14.281 MiB
3.56.473.771 I srv update: - prompt 0x609120374f80: 227 tokens, checkpoints: 1, 10.764 MiB
3.56.473.772 I srv update: - prompt 0x60912749c520: 430 tokens, checkpoints: 1, 15.524 MiB
3.56.473.772 I srv update: - prompt 0x6091274ccbf0: 1406 tokens, checkpoints: 1, 38.410 MiB
3.56.473.773 I srv update: - prompt 0x6091237ac790: 2518 tokens, checkpoints: 1, 64.485 MiB
3.56.473.773 I srv update: - prompt 0x60912044eee0: 710 tokens, checkpoints: 1, 22.512 MiB
3.56.473.774 I srv update: - prompt 0x6091202847f0: 6324 tokens, checkpoints: 1, 154.154 MiB
3.56.473.774 I srv update: - prompt 0x6091202d6680: 2232 tokens, checkpoints: 1, 58.201 MiB
3.56.473.775 I srv update: - prompt 0x6091203be2f0: 1458 tokens, checkpoints: 1, 40.052 MiB
3.56.473.775 I srv update: - prompt 0x6091274e2a70: 858 tokens, checkpoints: 1, 25.982 MiB
3.56.473.776 I srv update: - prompt 0x60912027f930: 728 tokens, checkpoints: 1, 22.934 MiB
3.56.473.776 I srv update: - prompt 0x609120118aa0: 430 tokens, checkpoints: 1, 15.829 MiB
3.56.473.777 I srv update: - prompt 0x60912023bd70: 459 tokens, checkpoints: 1, 16.509 MiB
3.56.473.780 I srv update: - prompt 0x609120403390: 392 tokens, checkpoints: 1, 14.938 MiB
3.56.473.781 I srv update: - prompt 0x6091203da560: 1101 tokens, checkpoints: 1, 31.563 MiB
3.56.473.781 I srv update: - prompt 0x609120301c40: 439 tokens, checkpoints: 1, 16.040 MiB
3.56.473.782 I srv update: - prompt 0x6091275b2a30: 304 tokens, checkpoints: 1, 12.874 MiB
3.56.473.782 I srv update: - prompt 0x60912024ff90: 399 tokens, checkpoints: 1, 15.102 MiB
3.56.473.784 I srv get_availabl: prompt cache update took 11.98 ms
3.56.474.709 I slot launch_slot_: id 3 | task 18927 | processing task, is_child = 0
3.56.474.711 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.56.474.712 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.56.474.731 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.56.474.731 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.56.474.745 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.56.474.745 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.56.474.767 I slot update_slots: id 3 | task 18927 | Checking checkpoint with [0, 116] against 0...
3.56.476.656 W slot update_slots: id 3 | task 18927 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 2.744 MiB)
3.59.648.032 I slot print_timing: id 3 | task 18927 | n_decoded = 261, tg = 86.77 t/s, tg_3s = 86.77 t/s
4.00.613.991 I slot print_timing: id 3 | task 18927 | prompt eval time = 165.45 ms / 43 tokens ( 3.85 ms per token, 259.90 tokens per second)
4.00.613.995 I slot print_timing: id 3 | task 18927 | eval time = 3973.77 ms / 345 tokens ( 11.52 ms per token, 86.82 tokens per second)
4.00.613.996 I slot print_timing: id 3 | task 18927 | total time = 4139.22 ms / 388 tokens
4.00.613.997 I slot print_timing: id 3 | task 18927 | graphs reused = 19091
4.00.614.029 I slot release: id 3 | task 18927 | stop processing: n_tokens = 503, truncated = 0
4.00.614.037 I srv update_slots: all slots are idle
4.00.674.389 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink