qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results_roles/ministral3-14b.server.log

725 lines
70 KiB
Plaintext
Raw Permalink Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.065.115 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.065.119 I device_info:
0.00.065.140 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.144 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.151 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.065.207 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.065.209 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.065.246 I srv init: running without SSL
0.00.065.304 I srv init: using 23 threads for HTTP server
0.00.065.630 I srv start: binding port with default address family
0.00.066.846 I srv llama_server: loading model
0.00.066.854 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/Ministral-3-14B-Reasoning-2512-Q5_K_M.gguf'
0.00.066.906 I common_init_result: fitting params to device memory ...
0.00.066.910 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.05.566.088 W llama_context: setting new yarn_attn_factor = 1.0000 (mscale == 1.0, mscale_all_dim = 1.0)
0.05.566.096 W llama_context: n_ctx_seq (24576) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.05.628.554 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.05.774.586 I srv load_model: initializing slots, n_slots = 4
0.05.822.320 W common_speculative_init: no implementations specified for speculative decoding
0.05.822.326 I slot load_model: id 0 | task -1 | new slot, n_ctx = 24576
0.05.822.330 I slot load_model: id 1 | task -1 | new slot, n_ctx = 24576
0.05.822.330 I slot load_model: id 2 | task -1 | new slot, n_ctx = 24576
0.05.822.331 I slot load_model: id 3 | task -1 | new slot, n_ctx = 24576
0.05.822.368 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.05.822.368 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.05.822.368 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.05.822.369 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.05.822.384 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.05.825.403 I init: chat template, example_format: '[SYSTEM_PROMPT]You are a helpful assistant[/SYSTEM_PROMPT][INST]Hello[/INST]Hi there</s>[INST]How are you?[/INST]'
0.05.825.820 I srv init: init: chat template, thinking = 1
0.05.825.853 I srv llama_server: model loaded
0.05.825.857 I srv llama_server: server is listening on http://127.0.0.1:8090
0.05.825.862 I srv update_slots: all slots are idle
0.09.965.358 I srv operator(): Chat format: peg-native
0.09.965.528 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.09.965.533 I srv get_availabl: updating prompt cache
0.09.965.538 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.09.965.543 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 24576 tokens, 8589934592 est)
0.09.965.544 I srv get_availabl: prompt cache update took 0.01 ms
0.09.965.573 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.09.965.574 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.09.965.575 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.09.965.595 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.09.965.596 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.09.965.612 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.09.965.614 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.12.016.391 I slot print_timing: id 3 | task 0 | prompt eval time = 284.87 ms / 58 tokens ( 4.91 ms per token, 203.60 tokens per second)
0.12.016.394 I slot print_timing: id 3 | task 0 | eval time = 1765.87 ms / 57 tokens ( 30.98 ms per token, 32.28 tokens per second)
0.12.016.395 I slot print_timing: id 3 | task 0 | total time = 2050.74 ms / 115 tokens
0.12.016.399 I slot print_timing: id 3 | task 0 | graphs reused = 0
0.12.016.437 I slot release: id 3 | task 0 | stop processing: n_tokens = 114, truncated = 0
0.12.016.441 I srv update_slots: all slots are idle
0.12.019.041 I srv operator(): Chat format: peg-native
0.12.019.147 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.672 (> 0.100 thold), f_keep = 0.360
0.12.019.149 I srv get_availabl: updating prompt cache
0.12.019.179 W srv prompt_save: - saving prompt with length 114, total state size = 17.815 MiB (draft: 0.000 MiB)
0.12.034.460 I srv load: - looking for better prompt, base f_keep = 0.360, sim = 0.672
0.12.034.466 I srv update: - cache state: 1 prompts, 17.815 MiB (limits: 8192.000 MiB, 24576 tokens, 52422 est)
0.12.034.467 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
0.12.034.468 I srv get_availabl: prompt cache update took 15.32 ms
0.12.034.501 I slot launch_slot_: id 3 | task 58 | processing task, is_child = 0
0.12.034.502 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.12.034.503 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.12.034.518 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.12.034.519 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.12.034.532 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.12.034.532 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.12.280.225 I slot print_timing: id 3 | task 58 | prompt eval time = 214.15 ms / 20 tokens ( 10.71 ms per token, 93.39 tokens per second)
0.12.280.231 I slot print_timing: id 3 | task 58 | eval time = 31.52 ms / 2 tokens ( 15.76 ms per token, 63.44 tokens per second)
0.12.280.232 I slot print_timing: id 3 | task 58 | total time = 245.67 ms / 22 tokens
0.12.280.233 I slot print_timing: id 3 | task 58 | graphs reused = 0
0.12.280.251 I slot release: id 3 | task 58 | stop processing: n_tokens = 62, truncated = 0
0.12.280.257 I srv update_slots: all slots are idle
0.12.282.839 I srv operator(): Chat format: peg-native
0.12.282.942 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.506 (> 0.100 thold), f_keep = 0.661
0.12.282.957 I slot launch_slot_: id 3 | task 61 | processing task, is_child = 0
0.12.282.957 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.12.282.958 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.12.282.972 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.12.282.972 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.12.282.985 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.12.282.985 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.13.628.340 I slot print_timing: id 3 | task 61 | prompt eval time = 176.00 ms / 40 tokens ( 4.40 ms per token, 227.27 tokens per second)
0.13.628.343 I slot print_timing: id 3 | task 61 | eval time = 1169.33 ms / 38 tokens ( 30.77 ms per token, 32.50 tokens per second)
0.13.628.344 I slot print_timing: id 3 | task 61 | total time = 1345.33 ms / 78 tokens
0.13.628.344 I slot print_timing: id 3 | task 61 | graphs reused = 0
0.13.628.366 I slot release: id 3 | task 61 | stop processing: n_tokens = 118, truncated = 0
0.13.628.372 I srv update_slots: all slots are idle
0.13.630.841 I srv operator(): Chat format: peg-native
0.13.630.966 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.651 (> 0.100 thold), f_keep = 0.347
0.13.630.967 I srv get_availabl: updating prompt cache
0.13.630.996 W srv prompt_save: - saving prompt with length 118, total state size = 18.440 MiB (draft: 0.000 MiB)
0.13.647.305 I srv load: - looking for better prompt, base f_keep = 0.347, sim = 0.651
0.13.647.316 I srv update: - cache state: 2 prompts, 36.255 MiB (limits: 8192.000 MiB, 24576 tokens, 52422 est)
0.13.647.317 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
0.13.647.318 I srv update: - prompt 0x585827c6a4c0: 118 tokens, checkpoints: 0, 18.440 MiB
0.13.647.319 I srv get_availabl: prompt cache update took 16.35 ms
0.13.647.350 I slot launch_slot_: id 3 | task 100 | processing task, is_child = 0
0.13.647.351 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.13.647.351 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.13.647.368 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.13.647.368 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.13.647.380 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.13.647.381 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.16.980.260 I slot print_timing: id 3 | task 100 | n_decoded = 100, tg = 32.07 t/s, tg_3s = 32.07 t/s
0.20.008.699 I slot print_timing: id 3 | task 100 | n_decoded = 196, tg = 31.89 t/s, tg_3s = 31.70 t/s
0.22.670.072 I slot print_timing: id 3 | task 100 | prompt eval time = 214.49 ms / 22 tokens ( 9.75 ms per token, 102.57 tokens per second)
0.22.670.075 I slot print_timing: id 3 | task 100 | eval time = 8808.18 ms / 280 tokens ( 31.46 ms per token, 31.79 tokens per second)
0.22.670.076 I slot print_timing: id 3 | task 100 | total time = 9022.66 ms / 302 tokens
0.22.670.077 I slot print_timing: id 3 | task 100 | graphs reused = 0
0.22.670.094 I slot release: id 3 | task 100 | stop processing: n_tokens = 342, truncated = 0
0.22.670.101 I srv update_slots: all slots are idle
0.22.672.573 I srv operator(): Chat format: peg-native
0.22.672.698 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.695 (> 0.100 thold), f_keep = 0.120
0.22.672.700 I srv get_availabl: updating prompt cache
0.22.672.741 W srv prompt_save: - saving prompt with length 342, total state size = 53.442 MiB (draft: 0.000 MiB)
0.22.714.453 I srv load: - looking for better prompt, base f_keep = 0.120, sim = 0.695
0.22.714.462 I srv update: - cache state: 3 prompts, 89.697 MiB (limits: 8192.000 MiB, 24576 tokens, 52423 est)
0.22.714.463 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
0.22.714.463 I srv update: - prompt 0x585827c6a4c0: 118 tokens, checkpoints: 0, 18.440 MiB
0.22.714.464 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
0.22.714.465 I srv get_availabl: prompt cache update took 41.77 ms
0.22.714.503 I slot launch_slot_: id 3 | task 381 | processing task, is_child = 0
0.22.714.503 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.22.714.504 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.22.714.520 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.22.714.521 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.22.714.533 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.22.714.534 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.24.033.081 I slot print_timing: id 3 | task 381 | prompt eval time = 214.72 ms / 18 tokens ( 11.93 ms per token, 83.83 tokens per second)
0.24.033.086 I slot print_timing: id 3 | task 381 | eval time = 1103.80 ms / 36 tokens ( 30.66 ms per token, 32.61 tokens per second)
0.24.033.087 I slot print_timing: id 3 | task 381 | total time = 1318.52 ms / 54 tokens
0.24.033.087 I slot print_timing: id 3 | task 381 | graphs reused = 0
0.24.033.106 I slot release: id 3 | task 381 | stop processing: n_tokens = 94, truncated = 0
0.24.033.113 I srv update_slots: all slots are idle
0.24.035.590 I srv operator(): Chat format: peg-native
0.24.035.698 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.519 (> 0.100 thold), f_keep = 0.436
0.24.035.698 I srv get_availabl: updating prompt cache
0.24.035.723 W srv prompt_save: - saving prompt with length 94, total state size = 14.690 MiB (draft: 0.000 MiB)
0.24.048.877 I srv load: - looking for better prompt, base f_keep = 0.436, sim = 0.519
0.24.048.886 I srv update: - cache state: 4 prompts, 104.386 MiB (limits: 8192.000 MiB, 24576 tokens, 52423 est)
0.24.048.887 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
0.24.048.887 I srv update: - prompt 0x585827c6a4c0: 118 tokens, checkpoints: 0, 18.440 MiB
0.24.048.888 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
0.24.048.888 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
0.24.048.889 I srv get_availabl: prompt cache update took 13.19 ms
0.24.048.921 I slot launch_slot_: id 3 | task 418 | processing task, is_child = 0
0.24.048.922 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.24.048.923 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.24.048.938 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.24.048.938 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.24.048.951 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.24.048.951 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.25.745.180 I slot print_timing: id 3 | task 418 | prompt eval time = 176.17 ms / 38 tokens ( 4.64 ms per token, 215.70 tokens per second)
0.25.745.184 I slot print_timing: id 3 | task 418 | eval time = 1520.03 ms / 49 tokens ( 31.02 ms per token, 32.24 tokens per second)
0.25.745.184 I slot print_timing: id 3 | task 418 | total time = 1696.20 ms / 87 tokens
0.25.745.185 I slot print_timing: id 3 | task 418 | graphs reused = 0
0.25.745.204 I slot release: id 3 | task 418 | stop processing: n_tokens = 127, truncated = 0
0.25.745.210 I srv update_slots: all slots are idle
0.25.747.714 I srv operator(): Chat format: peg-native
0.25.747.839 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.547 (> 0.100 thold), f_keep = 0.323
0.25.747.841 I srv get_availabl: updating prompt cache
0.25.747.872 W srv prompt_save: - saving prompt with length 127, total state size = 19.846 MiB (draft: 0.000 MiB)
0.25.765.198 I srv load: - looking for better prompt, base f_keep = 0.323, sim = 0.547
0.25.765.206 I srv update: - cache state: 5 prompts, 124.233 MiB (limits: 8192.000 MiB, 24576 tokens, 52422 est)
0.25.765.207 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
0.25.765.207 I srv update: - prompt 0x585827c6a4c0: 118 tokens, checkpoints: 0, 18.440 MiB
0.25.765.208 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
0.25.765.208 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
0.25.765.213 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
0.25.765.214 I srv get_availabl: prompt cache update took 17.37 ms
0.25.765.251 I slot launch_slot_: id 3 | task 468 | processing task, is_child = 0
0.25.765.252 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.25.765.253 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.25.765.269 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.25.765.269 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.25.765.282 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.25.765.282 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.27.837.699 I slot print_timing: id 3 | task 468 | prompt eval time = 175.69 ms / 34 tokens ( 5.17 ms per token, 193.53 tokens per second)
0.27.837.706 I slot print_timing: id 3 | task 468 | eval time = 1896.70 ms / 61 tokens ( 31.09 ms per token, 32.16 tokens per second)
0.27.837.707 I slot print_timing: id 3 | task 468 | total time = 2072.38 ms / 95 tokens
0.27.837.707 I slot print_timing: id 3 | task 468 | graphs reused = 0
0.27.837.730 I slot release: id 3 | task 468 | stop processing: n_tokens = 135, truncated = 0
0.27.837.738 I srv update_slots: all slots are idle
0.27.840.126 I srv operator(): Chat format: peg-native
0.27.840.226 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.672 (> 0.100 thold), f_keep = 0.304
0.27.840.229 I srv get_availabl: updating prompt cache
0.27.840.266 W srv prompt_save: - saving prompt with length 135, total state size = 21.096 MiB (draft: 0.000 MiB)
0.27.858.493 I srv load: - looking for better prompt, base f_keep = 0.304, sim = 0.672
0.27.858.500 I srv load: - found better prompt with f_keep = 0.373, sim = 0.721
0.27.868.048 I srv update: - cache state: 5 prompts, 126.889 MiB (limits: 8192.000 MiB, 24576 tokens, 52423 est)
0.27.868.052 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
0.27.868.052 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
0.27.868.053 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
0.27.868.053 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
0.27.868.054 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
0.27.868.055 I srv get_availabl: prompt cache update took 27.82 ms
0.27.868.088 I slot launch_slot_: id 3 | task 530 | processing task, is_child = 0
0.27.868.089 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.27.868.090 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.27.868.105 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.27.868.105 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.27.868.118 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.27.868.118 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.28.650.170 I slot print_timing: id 3 | task 530 | prompt eval time = 214.08 ms / 17 tokens ( 12.59 ms per token, 79.41 tokens per second)
0.28.650.174 I slot print_timing: id 3 | task 530 | eval time = 567.95 ms / 19 tokens ( 29.89 ms per token, 33.45 tokens per second)
0.28.650.175 I slot print_timing: id 3 | task 530 | total time = 782.03 ms / 36 tokens
0.28.650.175 I slot print_timing: id 3 | task 530 | graphs reused = 0
0.28.650.192 I slot release: id 3 | task 530 | stop processing: n_tokens = 79, truncated = 0
0.28.650.199 I srv update_slots: all slots are idle
0.28.652.721 I srv operator(): Chat format: peg-native
0.28.652.816 I slot get_availabl: id 2 | task -1 | selected slot by LRU, t_last = -1
0.28.652.818 I srv get_availabl: updating prompt cache
0.28.652.820 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.28.652.822 I srv update: - cache state: 5 prompts, 126.889 MiB (limits: 8192.000 MiB, 24576 tokens, 52423 est)
0.28.652.823 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
0.28.652.824 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
0.28.652.824 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
0.28.652.825 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
0.28.652.830 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
0.28.652.831 I srv get_availabl: prompt cache update took 0.01 ms
0.28.652.845 I slot launch_slot_: id 2 | task 550 | processing task, is_child = 0
0.28.652.846 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.28.652.846 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.28.652.861 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.28.652.861 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.28.652.874 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.28.652.899 W srv prompt_save: - saving prompt with length 79, total state size = 12.346 MiB (draft: 0.000 MiB)
0.28.664.047 I srv update: - cache state: 6 prompts, 139.235 MiB (limits: 8192.000 MiB, 24576 tokens, 52422 est)
0.28.664.050 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
0.28.664.051 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
0.28.664.052 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
0.28.664.052 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
0.28.664.053 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
0.28.664.053 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
0.28.664.054 I slot prompt_clear: id 3 | task -1 | clearing prompt with 79 tokens
0.32.110.226 I slot print_timing: id 2 | task 550 | n_decoded = 100, tg = 32.00 t/s, tg_3s = 32.00 t/s
0.35.125.236 I slot print_timing: id 2 | task 550 | n_decoded = 195, tg = 31.76 t/s, tg_3s = 31.51 t/s
0.38.153.765 I slot print_timing: id 2 | task 550 | n_decoded = 290, tg = 31.63 t/s, tg_3s = 31.37 t/s
0.41.179.087 I slot print_timing: id 2 | task 550 | n_decoded = 385, tg = 31.57 t/s, tg_3s = 31.40 t/s
0.44.184.889 I slot print_timing: id 2 | task 550 | n_decoded = 479, tg = 31.51 t/s, tg_3s = 31.27 t/s
0.47.192.889 I slot print_timing: id 2 | task 550 | n_decoded = 573, tg = 31.47 t/s, tg_3s = 31.25 t/s
0.50.195.769 I slot print_timing: id 2 | task 550 | n_decoded = 667, tg = 31.45 t/s, tg_3s = 31.30 t/s
0.53.207.037 I slot print_timing: id 2 | task 550 | n_decoded = 761, tg = 31.42 t/s, tg_3s = 31.22 t/s
0.56.214.941 I slot print_timing: id 2 | task 550 | n_decoded = 855, tg = 31.40 t/s, tg_3s = 31.25 t/s
0.59.227.259 I slot print_timing: id 2 | task 550 | n_decoded = 949, tg = 31.38 t/s, tg_3s = 31.21 t/s
1.02.252.317 I slot print_timing: id 2 | task 550 | n_decoded = 1044, tg = 31.38 t/s, tg_3s = 31.40 t/s
1.05.253.349 I slot print_timing: id 2 | task 550 | n_decoded = 1138, tg = 31.38 t/s, tg_3s = 31.32 t/s
1.07.851.126 I slot print_timing: id 2 | task 550 | prompt eval time = 321.28 ms / 90 tokens ( 3.57 ms per token, 280.13 tokens per second)
1.07.851.132 I slot print_timing: id 2 | task 550 | eval time = 38865.74 ms / 1219 tokens ( 31.88 ms per token, 31.36 tokens per second)
1.07.851.132 I slot print_timing: id 2 | task 550 | total time = 39187.02 ms / 1309 tokens
1.07.851.133 I slot print_timing: id 2 | task 550 | graphs reused = 0
1.07.851.153 I slot release: id 2 | task 550 | stop processing: n_tokens = 1308, truncated = 0
1.07.851.163 I srv update_slots: all slots are idle
1.07.853.940 I srv operator(): Chat format: peg-native
1.07.854.038 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.513 (> 0.100 thold), f_keep = 0.045
1.07.854.040 I srv get_availabl: updating prompt cache
1.07.854.103 W srv prompt_save: - saving prompt with length 1308, total state size = 204.391 MiB (draft: 0.000 MiB)
1.08.077.847 I srv load: - looking for better prompt, base f_keep = 0.045, sim = 0.513
1.08.077.855 I srv update: - cache state: 7 prompts, 343.625 MiB (limits: 8192.000 MiB, 24576 tokens, 52423 est)
1.08.077.856 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
1.08.077.856 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
1.08.077.857 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
1.08.077.857 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
1.08.077.858 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
1.08.077.859 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
1.08.077.859 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
1.08.077.860 I srv get_availabl: prompt cache update took 223.82 ms
1.08.077.889 I slot launch_slot_: id 2 | task 1770 | processing task, is_child = 0
1.08.077.889 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.08.077.890 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.08.077.908 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.08.077.908 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.08.077.921 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
1.08.077.921 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
1.11.446.207 I slot print_timing: id 2 | task 1770 | n_decoded = 100, tg = 32.00 t/s, tg_3s = 32.00 t/s
1.14.453.316 I slot print_timing: id 2 | task 1770 | n_decoded = 195, tg = 31.80 t/s, tg_3s = 31.59 t/s
1.17.472.307 I slot print_timing: id 2 | task 1770 | n_decoded = 290, tg = 31.69 t/s, tg_3s = 31.47 t/s
1.20.498.540 I slot print_timing: id 2 | task 1770 | n_decoded = 385, tg = 31.61 t/s, tg_3s = 31.39 t/s
1.23.502.393 I slot print_timing: id 2 | task 1770 | n_decoded = 479, tg = 31.55 t/s, tg_3s = 31.29 t/s
1.26.505.371 I slot print_timing: id 2 | task 1770 | n_decoded = 573, tg = 31.51 t/s, tg_3s = 31.30 t/s
1.28.391.381 I slot print_timing: id 2 | task 1770 | prompt eval time = 242.81 ms / 56 tokens ( 4.34 ms per token, 230.64 tokens per second)
1.28.391.385 I slot print_timing: id 2 | task 1770 | eval time = 20070.63 ms / 632 tokens ( 31.76 ms per token, 31.49 tokens per second)
1.28.391.386 I slot print_timing: id 2 | task 1770 | total time = 20313.43 ms / 688 tokens
1.28.391.386 I slot print_timing: id 2 | task 1770 | graphs reused = 0
1.28.391.410 I slot release: id 2 | task 1770 | stop processing: n_tokens = 746, truncated = 0
1.28.391.420 I srv update_slots: all slots are idle
1.28.393.985 I srv operator(): Chat format: peg-native
1.28.394.083 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.522 (> 0.100 thold), f_keep = 0.079
1.28.394.085 I srv get_availabl: updating prompt cache
1.28.394.126 W srv prompt_save: - saving prompt with length 746, total state size = 116.572 MiB (draft: 0.000 MiB)
1.28.524.102 I srv load: - looking for better prompt, base f_keep = 0.079, sim = 0.522
1.28.524.109 I srv update: - cache state: 8 prompts, 460.197 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
1.28.524.110 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
1.28.524.110 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
1.28.524.111 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
1.28.524.111 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
1.28.524.112 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
1.28.524.113 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
1.28.524.113 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
1.28.524.114 I srv update: - prompt 0x585827c9bb00: 746 tokens, checkpoints: 0, 116.572 MiB
1.28.524.115 I srv get_availabl: prompt cache update took 130.03 ms
1.28.524.140 I slot launch_slot_: id 2 | task 2403 | processing task, is_child = 0
1.28.524.141 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.28.524.141 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.28.524.156 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.28.524.156 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.28.524.169 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
1.28.524.169 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
1.31.904.451 I slot print_timing: id 2 | task 2403 | n_decoded = 100, tg = 31.88 t/s, tg_3s = 31.88 t/s
1.34.924.418 I slot print_timing: id 2 | task 2403 | n_decoded = 195, tg = 31.67 t/s, tg_3s = 31.46 t/s
1.37.950.427 I slot print_timing: id 2 | task 2403 | n_decoded = 290, tg = 31.58 t/s, tg_3s = 31.39 t/s
1.40.950.896 I slot print_timing: id 2 | task 2403 | n_decoded = 384, tg = 31.52 t/s, tg_3s = 31.33 t/s
1.43.964.067 I slot print_timing: id 2 | task 2403 | n_decoded = 478, tg = 31.45 t/s, tg_3s = 31.20 t/s
1.46.964.859 I slot print_timing: id 2 | task 2403 | n_decoded = 572, tg = 31.43 t/s, tg_3s = 31.33 t/s
1.49.976.680 I slot print_timing: id 2 | task 2403 | n_decoded = 667, tg = 31.45 t/s, tg_3s = 31.54 t/s
1.53.001.773 I slot print_timing: id 2 | task 2403 | n_decoded = 762, tg = 31.44 t/s, tg_3s = 31.40 t/s
1.56.026.786 I slot print_timing: id 2 | task 2403 | n_decoded = 857, tg = 31.44 t/s, tg_3s = 31.40 t/s
1.57.750.806 I slot print_timing: id 2 | task 2403 | prompt eval time = 243.02 ms / 54 tokens ( 4.50 ms per token, 222.20 tokens per second)
1.57.750.810 I slot print_timing: id 2 | task 2403 | eval time = 28983.59 ms / 911 tokens ( 31.82 ms per token, 31.43 tokens per second)
1.57.750.810 I slot print_timing: id 2 | task 2403 | total time = 29226.61 ms / 965 tokens
1.57.750.811 I slot print_timing: id 2 | task 2403 | graphs reused = 0
1.57.750.833 I slot release: id 2 | task 2403 | stop processing: n_tokens = 1023, truncated = 0
1.57.750.843 I srv update_slots: all slots are idle
1.57.753.593 I srv operator(): Chat format: peg-native
1.57.753.708 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.480 (> 0.100 thold), f_keep = 0.058
1.57.753.709 I srv get_availabl: updating prompt cache
1.57.753.764 W srv prompt_save: - saving prompt with length 1023, total state size = 159.856 MiB (draft: 0.000 MiB)
1.57.959.099 I srv load: - looking for better prompt, base f_keep = 0.058, sim = 0.480
1.57.959.106 I srv update: - cache state: 9 prompts, 620.054 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
1.57.959.107 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
1.57.959.107 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
1.57.959.108 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
1.57.959.108 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
1.57.959.109 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
1.57.959.109 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
1.57.959.110 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
1.57.959.111 I srv update: - prompt 0x585827c9bb00: 746 tokens, checkpoints: 0, 116.572 MiB
1.57.959.111 I srv update: - prompt 0x585827c7a860: 1023 tokens, checkpoints: 0, 159.856 MiB
1.57.959.112 I srv get_availabl: prompt cache update took 205.40 ms
1.57.959.138 I slot launch_slot_: id 2 | task 3315 | processing task, is_child = 0
1.57.959.139 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.57.959.140 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.57.959.157 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.57.959.157 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.57.959.170 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
1.57.959.170 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
2.01.354.724 I slot print_timing: id 2 | task 3315 | n_decoded = 100, tg = 31.99 t/s, tg_3s = 31.99 t/s
2.04.361.770 I slot print_timing: id 2 | task 3315 | n_decoded = 195, tg = 31.80 t/s, tg_3s = 31.59 t/s
2.07.379.366 I slot print_timing: id 2 | task 3315 | n_decoded = 290, tg = 31.69 t/s, tg_3s = 31.48 t/s
2.10.401.314 I slot print_timing: id 2 | task 3315 | n_decoded = 385, tg = 31.63 t/s, tg_3s = 31.44 t/s
2.13.427.773 I slot print_timing: id 2 | task 3315 | n_decoded = 480, tg = 31.58 t/s, tg_3s = 31.39 t/s
2.16.428.777 I slot print_timing: id 2 | task 3315 | n_decoded = 574, tg = 31.54 t/s, tg_3s = 31.32 t/s
2.19.436.030 I slot print_timing: id 2 | task 3315 | n_decoded = 668, tg = 31.50 t/s, tg_3s = 31.26 t/s
2.22.446.253 I slot print_timing: id 2 | task 3315 | n_decoded = 762, tg = 31.47 t/s, tg_3s = 31.23 t/s
2.25.456.024 I slot print_timing: id 2 | task 3315 | n_decoded = 856, tg = 31.44 t/s, tg_3s = 31.23 t/s
2.28.464.248 I slot print_timing: id 2 | task 3315 | n_decoded = 950, tg = 31.42 t/s, tg_3s = 31.25 t/s
2.31.472.307 I slot print_timing: id 2 | task 3315 | n_decoded = 1044, tg = 31.40 t/s, tg_3s = 31.25 t/s
2.34.485.238 I slot print_timing: id 2 | task 3315 | n_decoded = 1138, tg = 31.39 t/s, tg_3s = 31.20 t/s
2.37.503.112 I slot print_timing: id 2 | task 3315 | n_decoded = 1231, tg = 31.34 t/s, tg_3s = 30.82 t/s
2.40.520.738 I slot print_timing: id 2 | task 3315 | n_decoded = 1324, tg = 31.31 t/s, tg_3s = 30.82 t/s
2.43.524.038 I slot print_timing: id 2 | task 3315 | n_decoded = 1417, tg = 31.28 t/s, tg_3s = 30.97 t/s
2.46.530.373 I slot print_timing: id 2 | task 3315 | n_decoded = 1510, tg = 31.26 t/s, tg_3s = 30.93 t/s
2.49.538.408 I slot print_timing: id 2 | task 3315 | n_decoded = 1603, tg = 31.24 t/s, tg_3s = 30.92 t/s
2.50.577.702 I slot print_timing: id 2 | task 3315 | prompt eval time = 269.81 ms / 64 tokens ( 4.22 ms per token, 237.21 tokens per second)
2.50.577.705 I slot print_timing: id 2 | task 3315 | eval time = 52348.70 ms / 1635 tokens ( 32.02 ms per token, 31.23 tokens per second)
2.50.577.706 I slot print_timing: id 2 | task 3315 | total time = 52618.51 ms / 1699 tokens
2.50.577.706 I slot print_timing: id 2 | task 3315 | graphs reused = 0
2.50.577.729 I slot release: id 2 | task 3315 | stop processing: n_tokens = 1757, truncated = 0
2.50.577.744 I srv update_slots: all slots are idle
2.50.580.382 I srv operator(): Chat format: peg-native
2.50.580.488 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.628 (> 0.100 thold), f_keep = 0.034
2.50.580.490 I srv get_availabl: updating prompt cache
2.50.580.571 W srv prompt_save: - saving prompt with length 1757, total state size = 274.552 MiB (draft: 0.000 MiB)
2.50.880.288 I srv load: - looking for better prompt, base f_keep = 0.034, sim = 0.628
2.50.880.295 I srv update: - cache state: 10 prompts, 894.606 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
2.50.880.296 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
2.50.880.297 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
2.50.880.297 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
2.50.880.298 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
2.50.880.298 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
2.50.880.299 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
2.50.880.300 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
2.50.880.300 I srv update: - prompt 0x585827c9bb00: 746 tokens, checkpoints: 0, 116.572 MiB
2.50.880.301 I srv update: - prompt 0x585827c7a860: 1023 tokens, checkpoints: 0, 159.856 MiB
2.50.880.301 I srv update: - prompt 0x585827cafab0: 1757 tokens, checkpoints: 0, 274.552 MiB
2.50.880.302 I srv get_availabl: prompt cache update took 299.81 ms
2.50.880.334 I slot launch_slot_: id 2 | task 4951 | processing task, is_child = 0
2.50.880.335 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.50.880.335 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.50.880.354 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.50.880.355 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.50.880.368 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
2.50.880.368 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
2.54.176.929 I slot print_timing: id 2 | task 4951 | n_decoded = 100, tg = 32.05 t/s, tg_3s = 32.05 t/s
2.57.185.369 I slot print_timing: id 2 | task 4951 | n_decoded = 195, tg = 31.82 t/s, tg_3s = 31.58 t/s
3.00.201.048 I slot print_timing: id 2 | task 4951 | n_decoded = 290, tg = 31.71 t/s, tg_3s = 31.50 t/s
3.03.223.951 I slot print_timing: id 2 | task 4951 | n_decoded = 385, tg = 31.64 t/s, tg_3s = 31.43 t/s
3.06.253.851 I slot print_timing: id 2 | task 4951 | n_decoded = 480, tg = 31.58 t/s, tg_3s = 31.35 t/s
3.09.258.943 I slot print_timing: id 2 | task 4951 | n_decoded = 574, tg = 31.53 t/s, tg_3s = 31.28 t/s
3.12.271.045 I slot print_timing: id 2 | task 4951 | n_decoded = 668, tg = 31.49 t/s, tg_3s = 31.21 t/s
3.15.283.259 I slot print_timing: id 2 | task 4951 | n_decoded = 762, tg = 31.45 t/s, tg_3s = 31.21 t/s
3.18.291.296 I slot print_timing: id 2 | task 4951 | n_decoded = 856, tg = 31.43 t/s, tg_3s = 31.25 t/s
3.21.299.717 I slot print_timing: id 2 | task 4951 | n_decoded = 950, tg = 31.41 t/s, tg_3s = 31.25 t/s
3.24.310.290 I slot print_timing: id 2 | task 4951 | n_decoded = 1044, tg = 31.39 t/s, tg_3s = 31.22 t/s
3.25.015.246 I slot print_timing: id 2 | task 4951 | prompt eval time = 176.02 ms / 35 tokens ( 5.03 ms per token, 198.84 tokens per second)
3.25.015.250 I slot print_timing: id 2 | task 4951 | eval time = 33958.83 ms / 1066 tokens ( 31.86 ms per token, 31.39 tokens per second)
3.25.015.250 I slot print_timing: id 2 | task 4951 | total time = 34134.85 ms / 1101 tokens
3.25.015.251 I slot print_timing: id 2 | task 4951 | graphs reused = 0
3.25.015.272 I slot release: id 2 | task 4951 | stop processing: n_tokens = 1159, truncated = 0
3.25.015.282 I srv update_slots: all slots are idle
3.25.018.059 I srv operator(): Chat format: peg-native
3.25.018.171 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.541 (> 0.100 thold), f_keep = 0.051
3.25.018.173 I srv get_availabl: updating prompt cache
3.25.018.227 W srv prompt_save: - saving prompt with length 1159, total state size = 181.108 MiB (draft: 0.000 MiB)
3.25.185.741 I srv load: - looking for better prompt, base f_keep = 0.051, sim = 0.541
3.25.185.749 I srv update: - cache state: 11 prompts, 1075.714 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
3.25.185.750 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
3.25.185.750 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
3.25.185.751 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
3.25.185.751 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
3.25.185.752 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
3.25.185.752 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
3.25.185.753 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
3.25.185.754 I srv update: - prompt 0x585827c9bb00: 746 tokens, checkpoints: 0, 116.572 MiB
3.25.185.754 I srv update: - prompt 0x585827c7a860: 1023 tokens, checkpoints: 0, 159.856 MiB
3.25.185.755 I srv update: - prompt 0x585827cafab0: 1757 tokens, checkpoints: 0, 274.552 MiB
3.25.185.761 I srv update: - prompt 0x585827dbaa70: 1159 tokens, checkpoints: 0, 181.108 MiB
3.25.185.762 I srv get_availabl: prompt cache update took 167.59 ms
3.25.185.794 I slot launch_slot_: id 2 | task 6018 | processing task, is_child = 0
3.25.185.795 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.25.185.796 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.25.185.814 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.25.185.814 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.25.185.827 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.25.185.827 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
3.28.560.715 I slot print_timing: id 2 | task 6018 | n_decoded = 100, tg = 31.93 t/s, tg_3s = 31.93 t/s
3.31.591.843 I slot print_timing: id 2 | task 6018 | n_decoded = 196, tg = 31.80 t/s, tg_3s = 31.67 t/s
3.34.607.338 I slot print_timing: id 2 | task 6018 | n_decoded = 291, tg = 31.70 t/s, tg_3s = 31.50 t/s
3.37.621.882 I slot print_timing: id 2 | task 6018 | n_decoded = 386, tg = 31.66 t/s, tg_3s = 31.51 t/s
3.40.651.669 I slot print_timing: id 2 | task 6018 | n_decoded = 481, tg = 31.60 t/s, tg_3s = 31.36 t/s
3.43.655.732 I slot print_timing: id 2 | task 6018 | n_decoded = 575, tg = 31.55 t/s, tg_3s = 31.29 t/s
3.46.663.827 I slot print_timing: id 2 | task 6018 | n_decoded = 669, tg = 31.50 t/s, tg_3s = 31.25 t/s
3.49.671.830 I slot print_timing: id 2 | task 6018 | n_decoded = 763, tg = 31.47 t/s, tg_3s = 31.25 t/s
3.50.887.854 I slot print_timing: id 2 | task 6018 | prompt eval time = 242.58 ms / 50 tokens ( 4.85 ms per token, 206.12 tokens per second)
3.50.887.858 I slot print_timing: id 2 | task 6018 | eval time = 25459.42 ms / 801 tokens ( 31.78 ms per token, 31.46 tokens per second)
3.50.887.859 I slot print_timing: id 2 | task 6018 | total time = 25702.00 ms / 851 tokens
3.50.887.860 I slot print_timing: id 2 | task 6018 | graphs reused = 0
3.50.887.882 I slot release: id 2 | task 6018 | stop processing: n_tokens = 909, truncated = 0
3.50.887.892 I srv update_slots: all slots are idle
3.50.890.466 I srv operator(): Chat format: peg-native
3.50.890.573 I slot get_availabl: id 1 | task -1 | selected slot by LRU, t_last = -1
3.50.890.575 I srv get_availabl: updating prompt cache
3.50.890.577 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
3.50.890.580 I srv update: - cache state: 11 prompts, 1075.714 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
3.50.890.581 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
3.50.890.581 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
3.50.890.583 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
3.50.890.587 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
3.50.890.587 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
3.50.890.588 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
3.50.890.588 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
3.50.890.589 I srv update: - prompt 0x585827c9bb00: 746 tokens, checkpoints: 0, 116.572 MiB
3.50.890.589 I srv update: - prompt 0x585827c7a860: 1023 tokens, checkpoints: 0, 159.856 MiB
3.50.890.590 I srv update: - prompt 0x585827cafab0: 1757 tokens, checkpoints: 0, 274.552 MiB
3.50.890.591 I srv update: - prompt 0x585827dbaa70: 1159 tokens, checkpoints: 0, 181.108 MiB
3.50.890.591 I srv get_availabl: prompt cache update took 0.01 ms
3.50.890.603 I slot launch_slot_: id 1 | task 6820 | processing task, is_child = 0
3.50.890.603 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.50.890.604 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.50.890.620 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.50.890.664 W srv prompt_save: - saving prompt with length 909, total state size = 142.043 MiB (draft: 0.000 MiB)
3.51.036.823 I srv update: - cache state: 12 prompts, 1217.757 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
3.51.036.826 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
3.51.036.827 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
3.51.036.827 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
3.51.036.828 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
3.51.036.829 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
3.51.036.829 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
3.51.036.830 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
3.51.036.830 I srv update: - prompt 0x585827c9bb00: 746 tokens, checkpoints: 0, 116.572 MiB
3.51.036.831 I srv update: - prompt 0x585827c7a860: 1023 tokens, checkpoints: 0, 159.856 MiB
3.51.036.831 I srv update: - prompt 0x585827cafab0: 1757 tokens, checkpoints: 0, 274.552 MiB
3.51.036.832 I srv update: - prompt 0x585827dbaa70: 1159 tokens, checkpoints: 0, 181.108 MiB
3.51.036.832 I srv update: - prompt 0x585827d95730: 909 tokens, checkpoints: 0, 142.043 MiB
3.51.036.833 I slot prompt_clear: id 2 | task -1 | clearing prompt with 909 tokens
3.51.036.958 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.51.036.958 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
3.54.252.285 I slot print_timing: id 1 | task 6820 | prompt eval time = 451.01 ms / 125 tokens ( 3.61 ms per token, 277.15 tokens per second)
3.54.252.289 I slot print_timing: id 1 | task 6820 | eval time = 2764.25 ms / 88 tokens ( 31.41 ms per token, 31.84 tokens per second)
3.54.252.291 I slot print_timing: id 1 | task 6820 | total time = 3215.26 ms / 213 tokens
3.54.252.291 I slot print_timing: id 1 | task 6820 | graphs reused = 0
3.54.252.329 I slot release: id 1 | task 6820 | stop processing: n_tokens = 212, truncated = 0
3.54.252.334 I srv update_slots: all slots are idle
3.54.315.461 I srv operator(): Chat format: peg-native
3.54.315.571 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.706 (> 0.100 thold), f_keep = 0.283
3.54.315.573 I srv get_availabl: updating prompt cache
3.54.315.617 W srv prompt_save: - saving prompt with length 212, total state size = 33.128 MiB (draft: 0.000 MiB)
3.54.342.404 I srv load: - looking for better prompt, base f_keep = 0.283, sim = 0.706
3.54.342.414 I srv update: - cache state: 13 prompts, 1250.885 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
3.54.342.415 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
3.54.342.415 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
3.54.342.416 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
3.54.342.416 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
3.54.342.418 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
3.54.342.418 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
3.54.342.419 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
3.54.342.419 I srv update: - prompt 0x585827c9bb00: 746 tokens, checkpoints: 0, 116.572 MiB
3.54.342.420 I srv update: - prompt 0x585827c7a860: 1023 tokens, checkpoints: 0, 159.856 MiB
3.54.342.420 I srv update: - prompt 0x585827cafab0: 1757 tokens, checkpoints: 0, 274.552 MiB
3.54.342.421 I srv update: - prompt 0x585827dbaa70: 1159 tokens, checkpoints: 0, 181.108 MiB
3.54.342.422 I srv update: - prompt 0x585827d95730: 909 tokens, checkpoints: 0, 142.043 MiB
3.54.342.422 I srv update: - prompt 0x585825be21f0: 212 tokens, checkpoints: 0, 33.128 MiB
3.54.342.423 I srv get_availabl: prompt cache update took 26.85 ms
3.54.342.461 I slot launch_slot_: id 1 | task 6909 | processing task, is_child = 0
3.54.342.462 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.54.342.463 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.54.342.479 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.54.342.480 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.54.342.493 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.54.342.493 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
3.55.560.166 I slot print_timing: id 1 | task 6909 | prompt eval time = 231.46 ms / 25 tokens ( 9.26 ms per token, 108.01 tokens per second)
3.55.560.169 I slot print_timing: id 1 | task 6909 | eval time = 986.19 ms / 32 tokens ( 30.82 ms per token, 32.45 tokens per second)
3.55.560.170 I slot print_timing: id 1 | task 6909 | total time = 1217.65 ms / 57 tokens
3.55.560.170 I slot print_timing: id 1 | task 6909 | graphs reused = 0
3.55.560.192 I slot release: id 1 | task 6909 | stop processing: n_tokens = 116, truncated = 0
3.55.560.200 I srv update_slots: all slots are idle
3.55.827.643 I srv operator(): Chat format: peg-native
3.55.827.766 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.645 (> 0.100 thold), f_keep = 0.517
3.55.827.801 I slot launch_slot_: id 1 | task 6942 | processing task, is_child = 0
3.55.827.804 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.55.827.805 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.55.827.825 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.55.827.825 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.55.827.840 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.55.827.840 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
3.57.467.818 I slot print_timing: id 1 | task 6942 | prompt eval time = 175.96 ms / 33 tokens ( 5.33 ms per token, 187.55 tokens per second)
3.57.467.822 I slot print_timing: id 1 | task 6942 | eval time = 1463.99 ms / 47 tokens ( 31.15 ms per token, 32.10 tokens per second)
3.57.467.822 I slot print_timing: id 1 | task 6942 | total time = 1639.95 ms / 80 tokens
3.57.467.823 I slot print_timing: id 1 | task 6942 | graphs reused = 0
3.57.467.844 I slot release: id 1 | task 6942 | stop processing: n_tokens = 139, truncated = 0
3.57.467.849 I srv update_slots: all slots are idle
3.57.542.029 I srv operator(): Chat format: peg-native
3.57.542.147 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.484 (> 0.100 thold), f_keep = 0.432
3.57.542.150 I srv get_availabl: updating prompt cache
3.57.542.193 W srv prompt_save: - saving prompt with length 139, total state size = 21.721 MiB (draft: 0.000 MiB)
3.57.560.611 I srv load: - looking for better prompt, base f_keep = 0.432, sim = 0.484
3.57.560.622 I srv update: - cache state: 14 prompts, 1272.606 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
3.57.560.623 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
3.57.560.624 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
3.57.560.625 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
3.57.560.625 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
3.57.560.626 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
3.57.560.627 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
3.57.560.627 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
3.57.560.628 I srv update: - prompt 0x585827c9bb00: 746 tokens, checkpoints: 0, 116.572 MiB
3.57.560.628 I srv update: - prompt 0x585827c7a860: 1023 tokens, checkpoints: 0, 159.856 MiB
3.57.560.629 I srv update: - prompt 0x585827cafab0: 1757 tokens, checkpoints: 0, 274.552 MiB
3.57.560.629 I srv update: - prompt 0x585827dbaa70: 1159 tokens, checkpoints: 0, 181.108 MiB
3.57.560.630 I srv update: - prompt 0x585827d95730: 909 tokens, checkpoints: 0, 142.043 MiB
3.57.560.631 I srv update: - prompt 0x585825be21f0: 212 tokens, checkpoints: 0, 33.128 MiB
3.57.560.631 I srv update: - prompt 0x585827ca0c80: 139 tokens, checkpoints: 0, 21.721 MiB
3.57.560.633 I srv get_availabl: prompt cache update took 18.48 ms
3.57.560.666 I slot launch_slot_: id 1 | task 6990 | processing task, is_child = 0
3.57.560.667 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.57.560.667 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.57.560.685 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.57.560.693 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.57.560.708 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.57.560.709 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
4.00.976.666 I slot print_timing: id 1 | task 6990 | n_decoded = 100, tg = 31.79 t/s, tg_3s = 31.79 t/s
4.02.765.765 I slot print_timing: id 1 | task 6990 | prompt eval time = 269.94 ms / 64 tokens ( 4.22 ms per token, 237.09 tokens per second)
4.02.765.770 I slot print_timing: id 1 | task 6990 | eval time = 4935.09 ms / 156 tokens ( 31.64 ms per token, 31.61 tokens per second)
4.02.765.770 I slot print_timing: id 1 | task 6990 | total time = 5205.03 ms / 220 tokens
4.02.765.771 I slot print_timing: id 1 | task 6990 | graphs reused = 0
4.02.765.796 I slot release: id 1 | task 6990 | stop processing: n_tokens = 279, truncated = 0
4.02.765.804 I srv update_slots: all slots are idle
4.02.840.081 I srv operator(): Chat format: peg-native
4.02.840.196 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.612 (> 0.100 thold), f_keep = 0.215
4.02.840.198 I srv get_availabl: updating prompt cache
4.02.840.243 W srv prompt_save: - saving prompt with length 279, total state size = 43.598 MiB (draft: 0.000 MiB)
4.02.874.331 I srv load: - looking for better prompt, base f_keep = 0.215, sim = 0.612
4.02.874.343 I srv update: - cache state: 15 prompts, 1316.204 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
4.02.874.344 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
4.02.874.344 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
4.02.874.345 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
4.02.874.350 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
4.02.874.351 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
4.02.874.352 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
4.02.874.352 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
4.02.874.353 I srv update: - prompt 0x585827c9bb00: 746 tokens, checkpoints: 0, 116.572 MiB
4.02.874.353 I srv update: - prompt 0x585827c7a860: 1023 tokens, checkpoints: 0, 159.856 MiB
4.02.874.354 I srv update: - prompt 0x585827cafab0: 1757 tokens, checkpoints: 0, 274.552 MiB
4.02.874.354 I srv update: - prompt 0x585827dbaa70: 1159 tokens, checkpoints: 0, 181.108 MiB
4.02.874.355 I srv update: - prompt 0x585827d95730: 909 tokens, checkpoints: 0, 142.043 MiB
4.02.874.355 I srv update: - prompt 0x585825be21f0: 212 tokens, checkpoints: 0, 33.128 MiB
4.02.874.356 I srv update: - prompt 0x585827ca0c80: 139 tokens, checkpoints: 0, 21.721 MiB
4.02.874.356 I srv update: - prompt 0x585827cab690: 279 tokens, checkpoints: 0, 43.598 MiB
4.02.874.358 I srv get_availabl: prompt cache update took 34.16 ms
4.02.874.397 I slot launch_slot_: id 1 | task 7147 | processing task, is_child = 0
4.02.874.398 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.02.874.399 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.02.874.415 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.02.874.415 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.02.874.446 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
4.02.874.446 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
4.04.294.592 I slot print_timing: id 1 | task 7147 | prompt eval time = 176.66 ms / 38 tokens ( 4.65 ms per token, 215.10 tokens per second)
4.04.294.596 I slot print_timing: id 1 | task 7147 | eval time = 1243.46 ms / 40 tokens ( 31.09 ms per token, 32.17 tokens per second)
4.04.294.597 I slot print_timing: id 1 | task 7147 | total time = 1420.12 ms / 78 tokens
4.04.294.598 I slot print_timing: id 1 | task 7147 | graphs reused = 0
4.04.294.619 I slot release: id 1 | task 7147 | stop processing: n_tokens = 137, truncated = 0
4.04.294.626 I srv update_slots: all slots are idle
4.04.341.961 I srv operator(): Chat format: peg-native
4.04.342.069 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.524 (> 0.100 thold), f_keep = 0.401
4.04.342.071 I srv get_availabl: updating prompt cache
4.04.342.105 W srv prompt_save: - saving prompt with length 137, total state size = 21.409 MiB (draft: 0.000 MiB)
4.04.360.174 I srv load: - looking for better prompt, base f_keep = 0.401, sim = 0.524
4.04.360.183 I srv update: - cache state: 16 prompts, 1337.613 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
4.04.360.184 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
4.04.360.185 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
4.04.360.185 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
4.04.360.187 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
4.04.360.188 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
4.04.360.188 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
4.04.360.189 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
4.04.360.190 I srv update: - prompt 0x585827c9bb00: 746 tokens, checkpoints: 0, 116.572 MiB
4.04.360.190 I srv update: - prompt 0x585827c7a860: 1023 tokens, checkpoints: 0, 159.856 MiB
4.04.360.191 I srv update: - prompt 0x585827cafab0: 1757 tokens, checkpoints: 0, 274.552 MiB
4.04.360.191 I srv update: - prompt 0x585827dbaa70: 1159 tokens, checkpoints: 0, 181.108 MiB
4.04.360.192 I srv update: - prompt 0x585827d95730: 909 tokens, checkpoints: 0, 142.043 MiB
4.04.360.192 I srv update: - prompt 0x585825be21f0: 212 tokens, checkpoints: 0, 33.128 MiB
4.04.360.193 I srv update: - prompt 0x585827ca0c80: 139 tokens, checkpoints: 0, 21.721 MiB
4.04.360.193 I srv update: - prompt 0x585827cab690: 279 tokens, checkpoints: 0, 43.598 MiB
4.04.360.194 I srv update: - prompt 0x585827607a70: 137 tokens, checkpoints: 0, 21.409 MiB
4.04.360.195 I srv get_availabl: prompt cache update took 18.12 ms
4.04.360.225 I slot launch_slot_: id 1 | task 7188 | processing task, is_child = 0
4.04.360.225 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.04.360.226 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.04.360.243 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.04.360.243 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.04.360.256 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
4.04.360.256 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
4.05.658.638 I slot print_timing: id 1 | task 7188 | prompt eval time = 243.12 ms / 50 tokens ( 4.86 ms per token, 205.66 tokens per second)
4.05.658.642 I slot print_timing: id 1 | task 7188 | eval time = 1055.24 ms / 34 tokens ( 31.04 ms per token, 32.22 tokens per second)
4.05.658.643 I slot print_timing: id 1 | task 7188 | total time = 1298.36 ms / 84 tokens
4.05.658.644 I slot print_timing: id 1 | task 7188 | graphs reused = 0
4.05.658.665 I slot release: id 1 | task 7188 | stop processing: n_tokens = 138, truncated = 0
4.05.658.671 I srv update_slots: all slots are idle
4.05.661.433 I srv operator(): Chat format: peg-native
4.05.661.535 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.591 (> 0.100 thold), f_keep = 0.399
4.05.661.537 I srv get_availabl: updating prompt cache
4.05.661.562 W srv prompt_save: - saving prompt with length 138, total state size = 21.565 MiB (draft: 0.000 MiB)
4.05.679.740 I srv load: - looking for better prompt, base f_keep = 0.399, sim = 0.591
4.05.679.748 I srv load: - found better prompt with f_keep = 0.432, sim = 0.645
4.05.690.744 I srv update: - cache state: 16 prompts, 1337.457 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
4.05.690.747 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
4.05.690.748 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
4.05.690.749 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
4.05.690.749 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
4.05.690.750 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
4.05.690.750 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
4.05.690.751 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
4.05.690.751 I srv update: - prompt 0x585827c9bb00: 746 tokens, checkpoints: 0, 116.572 MiB
4.05.690.752 I srv update: - prompt 0x585827c7a860: 1023 tokens, checkpoints: 0, 159.856 MiB
4.05.690.752 I srv update: - prompt 0x585827cafab0: 1757 tokens, checkpoints: 0, 274.552 MiB
4.05.690.753 I srv update: - prompt 0x585827dbaa70: 1159 tokens, checkpoints: 0, 181.108 MiB
4.05.690.753 I srv update: - prompt 0x585827d95730: 909 tokens, checkpoints: 0, 142.043 MiB
4.05.690.754 I srv update: - prompt 0x585825be21f0: 212 tokens, checkpoints: 0, 33.128 MiB
4.05.690.754 I srv update: - prompt 0x585827cab690: 279 tokens, checkpoints: 0, 43.598 MiB
4.05.690.755 I srv update: - prompt 0x585827607a70: 137 tokens, checkpoints: 0, 21.409 MiB
4.05.690.755 I srv update: - prompt 0x585827dacf00: 138 tokens, checkpoints: 0, 21.565 MiB
4.05.690.757 I srv get_availabl: prompt cache update took 29.22 ms
4.05.690.791 I slot launch_slot_: id 1 | task 7223 | processing task, is_child = 0
4.05.690.792 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.05.690.793 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.05.690.809 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.05.690.809 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.05.690.821 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
4.05.690.822 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
4.09.036.759 I slot print_timing: id 1 | task 7223 | n_decoded = 100, tg = 31.54 t/s, tg_3s = 31.54 t/s
4.09.964.749 I slot print_timing: id 1 | task 7223 | prompt eval time = 175.68 ms / 33 tokens ( 5.32 ms per token, 187.84 tokens per second)
4.09.964.752 I slot print_timing: id 1 | task 7223 | eval time = 4098.22 ms / 129 tokens ( 31.77 ms per token, 31.48 tokens per second)
4.09.964.753 I slot print_timing: id 1 | task 7223 | total time = 4273.90 ms / 162 tokens
4.09.964.754 I slot print_timing: id 1 | task 7223 | graphs reused = 0
4.09.964.773 I slot release: id 1 | task 7223 | stop processing: n_tokens = 221, truncated = 0
4.09.964.780 I srv update_slots: all slots are idle
4.10.028.703 I srv operator(): Chat format: peg-native
4.10.028.818 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.645 (> 0.100 thold), f_keep = 0.271
4.10.028.819 I srv get_availabl: updating prompt cache
4.10.028.848 W srv prompt_save: - saving prompt with length 221, total state size = 34.535 MiB (draft: 0.000 MiB)
4.10.056.571 I srv load: - looking for better prompt, base f_keep = 0.271, sim = 0.645
4.10.056.581 I srv update: - cache state: 17 prompts, 1371.991 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
4.10.056.582 I srv update: - prompt 0x5858266783b0: 114 tokens, checkpoints: 0, 17.815 MiB
4.10.056.594 I srv update: - prompt 0x585827c649f0: 342 tokens, checkpoints: 0, 53.442 MiB
4.10.056.595 I srv update: - prompt 0x585827c8f5e0: 94 tokens, checkpoints: 0, 14.690 MiB
4.10.056.596 I srv update: - prompt 0x585827c64bb0: 127 tokens, checkpoints: 0, 19.846 MiB
4.10.056.596 I srv update: - prompt 0x5858277c4210: 135 tokens, checkpoints: 0, 21.096 MiB
4.10.056.597 I srv update: - prompt 0x5858277c42d0: 79 tokens, checkpoints: 0, 12.346 MiB
4.10.056.598 I srv update: - prompt 0x585827d9ac30: 1308 tokens, checkpoints: 0, 204.391 MiB
4.10.056.598 I srv update: - prompt 0x585827c9bb00: 746 tokens, checkpoints: 0, 116.572 MiB
4.10.056.599 I srv update: - prompt 0x585827c7a860: 1023 tokens, checkpoints: 0, 159.856 MiB
4.10.056.599 I srv update: - prompt 0x585827cafab0: 1757 tokens, checkpoints: 0, 274.552 MiB
4.10.056.600 I srv update: - prompt 0x585827dbaa70: 1159 tokens, checkpoints: 0, 181.108 MiB
4.10.056.600 I srv update: - prompt 0x585827d95730: 909 tokens, checkpoints: 0, 142.043 MiB
4.10.056.601 I srv update: - prompt 0x585825be21f0: 212 tokens, checkpoints: 0, 33.128 MiB
4.10.056.601 I srv update: - prompt 0x585827cab690: 279 tokens, checkpoints: 0, 43.598 MiB
4.10.056.602 I srv update: - prompt 0x585827607a70: 137 tokens, checkpoints: 0, 21.409 MiB
4.10.056.602 I srv update: - prompt 0x585827dacf00: 138 tokens, checkpoints: 0, 21.565 MiB
4.10.056.603 I srv update: - prompt 0x585824b2c190: 221 tokens, checkpoints: 0, 34.535 MiB
4.10.056.604 I srv get_availabl: prompt cache update took 27.78 ms
4.10.056.645 I slot launch_slot_: id 1 | task 7353 | processing task, is_child = 0
4.10.056.646 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.10.056.647 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.10.056.665 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.10.056.665 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.10.056.680 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
4.10.056.681 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
4.11.288.631 I slot print_timing: id 1 | task 7353 | prompt eval time = 176.16 ms / 33 tokens ( 5.34 ms per token, 187.33 tokens per second)
4.11.288.634 I slot print_timing: id 1 | task 7353 | eval time = 1055.75 ms / 34 tokens ( 31.05 ms per token, 32.20 tokens per second)
4.11.288.635 I slot print_timing: id 1 | task 7353 | total time = 1231.91 ms / 67 tokens
4.11.288.635 I slot print_timing: id 1 | task 7353 | graphs reused = 0
4.11.288.656 I slot release: id 1 | task 7353 | stop processing: n_tokens = 126, truncated = 0
4.11.288.662 I srv update_slots: all slots are idle
4.11.348.871 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink