qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results_roles/minimax-m2-iq2.server.log

1815 lines
189 KiB
Plaintext
Raw Permalink Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.207.173 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.207.176 I device_info:
0.00.207.198 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.207.202 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.207.210 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.207.270 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.207.273 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.207.461 I srv init: running without SSL
0.00.207.523 I srv init: using 23 threads for HTTP server
0.00.207.855 I srv start: binding port with default address family
0.00.209.057 I srv llama_server: loading model
0.00.209.065 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/MiniMaxAI_MiniMax-M2-IQ2_S/MiniMaxAI_MiniMax-M2-IQ2_S-00001-of-00002.gguf'
0.00.209.114 I common_init_result: fitting params to device memory ...
0.00.209.116 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.06.572.346 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
0.06.668.375 W llama_model_loader: tensor overrides to CPU are used with mmap enabled - consider using --no-mmap for better performance
0.44.171.200 W llama_context: n_ctx_seq (24576) < n_ctx_train (196608) -- the full capacity of the model will not be utilized
0.44.402.291 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.44.651.026 I srv load_model: initializing slots, n_slots = 4
0.44.773.990 W common_speculative_init: no implementations specified for speculative decoding
0.44.773.996 I slot load_model: id 0 | task -1 | new slot, n_ctx = 24576
0.44.774.001 I slot load_model: id 1 | task -1 | new slot, n_ctx = 24576
0.44.774.002 I slot load_model: id 2 | task -1 | new slot, n_ctx = 24576
0.44.774.002 I slot load_model: id 3 | task -1 | new slot, n_ctx = 24576
0.44.774.115 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.44.774.117 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.44.774.118 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.44.774.118 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.44.774.137 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.44.778.464 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
0.44.792.109 I init: chat template, example_format: ']~!b[]~b]system
You are a helpful assistant[e~[
]~b]user
Hello[e~[
]~b]ai
Hi there[e~[
]~b]user
How are you?[e~[
]~b]ai
<think>
'
0.44.792.114 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
0.44.805.031 I srv init: init: chat template, thinking = 1
0.44.805.067 I srv llama_server: model loaded
0.44.805.072 I srv llama_server: server is listening on http://127.0.0.1:8090
0.44.805.077 I srv update_slots: all slots are idle
0.44.944.997 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
0.44.963.542 I srv operator(): Chat format: peg-native
0.44.969.752 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.44.969.754 I srv get_availabl: updating prompt cache
0.44.969.759 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.44.969.763 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 24576 tokens, 8589934592 est)
0.44.969.765 I srv get_availabl: prompt cache update took 0.01 ms
0.44.971.555 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.44.971.557 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.44.971.558 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.44.971.573 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.44.971.573 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.44.971.586 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.44.971.586 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.52.536.830 I slot print_timing: id 3 | task 0 | n_decoded = 100, tg = 21.76 t/s, tg_3s = 21.76 t/s
0.55.582.650 I slot print_timing: id 3 | task 0 | n_decoded = 166, tg = 21.73 t/s, tg_3s = 21.67 t/s
0.58.583.053 I slot print_timing: id 3 | task 0 | n_decoded = 231, tg = 21.71 t/s, tg_3s = 21.66 t/s
1.01.596.378 I slot print_timing: id 3 | task 0 | n_decoded = 295, tg = 21.60 t/s, tg_3s = 21.24 t/s
1.04.569.137 I slot print_timing: id 3 | task 0 | prompt eval time = 2970.44 ms / 67 tokens ( 44.33 ms per token, 22.56 tokens per second)
1.04.569.145 I slot print_timing: id 3 | task 0 | eval time = 16627.08 ms / 359 tokens ( 46.31 ms per token, 21.59 tokens per second)
1.04.569.146 I slot print_timing: id 3 | task 0 | total time = 19597.51 ms / 426 tokens
1.04.569.151 I slot print_timing: id 3 | task 0 | graphs reused = 357
1.04.569.194 I slot release: id 3 | task 0 | stop processing: n_tokens = 425, truncated = 0
1.04.569.206 I srv update_slots: all slots are idle
1.04.570.216 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
1.04.587.195 I srv operator(): Chat format: peg-native
1.04.592.630 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.629 (> 0.100 thold), f_keep = 0.104
1.04.592.632 I srv get_availabl: updating prompt cache
1.04.592.681 W srv prompt_save: - saving prompt with length 425, total state size = 102.936 MiB (draft: 0.000 MiB)
1.04.665.397 I srv load: - looking for better prompt, base f_keep = 0.104, sim = 0.629
1.04.665.404 I srv update: - cache state: 1 prompts, 102.936 MiB (limits: 8192.000 MiB, 24576 tokens, 33822 est)
1.04.665.405 I srv update: - prompt 0x6354c2edb8c0: 425 tokens, checkpoints: 0, 102.936 MiB
1.04.665.406 I srv get_availabl: prompt cache update took 72.77 ms
1.04.667.042 I slot launch_slot_: id 3 | task 361 | processing task, is_child = 0
1.04.667.045 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.04.667.046 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.04.667.063 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.04.667.063 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.04.667.075 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.04.667.075 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.08.003.902 I slot print_timing: id 3 | task 361 | prompt eval time = 794.52 ms / 26 tokens ( 30.56 ms per token, 32.72 tokens per second)
1.08.003.908 I slot print_timing: id 3 | task 361 | eval time = 2542.27 ms / 56 tokens ( 45.40 ms per token, 22.03 tokens per second)
1.08.003.908 I slot print_timing: id 3 | task 361 | total time = 3336.79 ms / 82 tokens
1.08.003.909 I slot print_timing: id 3 | task 361 | graphs reused = 411
1.08.003.934 I slot release: id 3 | task 361 | stop processing: n_tokens = 125, truncated = 0
1.08.003.947 I srv update_slots: all slots are idle
1.08.004.896 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
1.08.023.047 I srv operator(): Chat format: peg-native
1.08.028.493 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.484 (> 0.100 thold), f_keep = 0.352
1.08.028.495 I srv get_availabl: updating prompt cache
1.08.028.532 W srv prompt_save: - saving prompt with length 125, total state size = 30.276 MiB (draft: 0.000 MiB)
1.08.052.980 I srv load: - looking for better prompt, base f_keep = 0.352, sim = 0.484
1.08.052.987 I srv update: - cache state: 2 prompts, 133.212 MiB (limits: 8192.000 MiB, 24576 tokens, 33822 est)
1.08.052.987 I srv update: - prompt 0x6354c2edb8c0: 425 tokens, checkpoints: 0, 102.936 MiB
1.08.052.988 I srv update: - prompt 0x6354c2eccbe0: 125 tokens, checkpoints: 0, 30.276 MiB
1.08.052.989 I srv get_availabl: prompt cache update took 24.49 ms
1.08.054.572 I slot launch_slot_: id 3 | task 418 | processing task, is_child = 0
1.08.054.574 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.08.054.575 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.08.054.589 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.08.054.590 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.08.054.601 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.08.054.602 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.14.307.646 I slot print_timing: id 3 | task 418 | n_decoded = 100, tg = 21.88 t/s, tg_3s = 21.88 t/s
1.17.323.440 I slot print_timing: id 3 | task 418 | n_decoded = 165, tg = 21.75 t/s, tg_3s = 21.55 t/s
1.20.327.607 I slot print_timing: id 3 | task 418 | n_decoded = 228, tg = 21.53 t/s, tg_3s = 20.97 t/s
1.23.333.058 I slot print_timing: id 3 | task 418 | n_decoded = 292, tg = 21.48 t/s, tg_3s = 21.29 t/s
1.26.334.614 I slot print_timing: id 3 | task 418 | n_decoded = 357, tg = 21.51 t/s, tg_3s = 21.66 t/s
1.29.340.023 I slot print_timing: id 3 | task 418 | n_decoded = 422, tg = 21.53 t/s, tg_3s = 21.63 t/s
1.32.359.866 I slot print_timing: id 3 | task 418 | n_decoded = 487, tg = 21.53 t/s, tg_3s = 21.52 t/s
1.35.374.338 I slot print_timing: id 3 | task 418 | n_decoded = 552, tg = 21.53 t/s, tg_3s = 21.56 t/s
1.38.380.106 I slot print_timing: id 3 | task 418 | n_decoded = 617, tg = 21.54 t/s, tg_3s = 21.63 t/s
1.41.404.579 I slot print_timing: id 3 | task 418 | n_decoded = 682, tg = 21.54 t/s, tg_3s = 21.49 t/s
1.44.412.096 I slot print_timing: id 3 | task 418 | n_decoded = 746, tg = 21.51 t/s, tg_3s = 21.28 t/s
1.47.424.600 I slot print_timing: id 3 | task 418 | n_decoded = 810, tg = 21.49 t/s, tg_3s = 21.24 t/s
1.50.432.642 I slot print_timing: id 3 | task 418 | n_decoded = 874, tg = 21.48 t/s, tg_3s = 21.28 t/s
1.53.451.578 I slot print_timing: id 3 | task 418 | n_decoded = 938, tg = 21.46 t/s, tg_3s = 21.20 t/s
1.56.462.338 I slot print_timing: id 3 | task 418 | n_decoded = 1002, tg = 21.45 t/s, tg_3s = 21.26 t/s
1.59.466.505 I slot print_timing: id 3 | task 418 | n_decoded = 1066, tg = 21.44 t/s, tg_3s = 21.30 t/s
2.02.471.972 I slot print_timing: id 3 | task 418 | n_decoded = 1130, tg = 21.43 t/s, tg_3s = 21.29 t/s
2.05.516.160 I slot print_timing: id 3 | task 418 | n_decoded = 1193, tg = 21.39 t/s, tg_3s = 20.70 t/s
2.08.524.252 I slot print_timing: id 3 | task 418 | n_decoded = 1257, tg = 21.38 t/s, tg_3s = 21.28 t/s
2.11.534.208 I slot print_timing: id 3 | task 418 | n_decoded = 1321, tg = 21.38 t/s, tg_3s = 21.26 t/s
2.14.543.232 I slot print_timing: id 3 | task 418 | n_decoded = 1385, tg = 21.37 t/s, tg_3s = 21.27 t/s
2.17.563.373 I slot print_timing: id 3 | task 418 | n_decoded = 1449, tg = 21.36 t/s, tg_3s = 21.19 t/s
2.20.584.081 I slot print_timing: id 3 | task 418 | n_decoded = 1512, tg = 21.34 t/s, tg_3s = 20.86 t/s
2.23.595.620 I slot print_timing: id 3 | task 418 | n_decoded = 1575, tg = 21.32 t/s, tg_3s = 20.92 t/s
2.26.610.338 I slot print_timing: id 3 | task 418 | n_decoded = 1638, tg = 21.31 t/s, tg_3s = 20.90 t/s
2.29.629.461 I slot print_timing: id 3 | task 418 | n_decoded = 1702, tg = 21.30 t/s, tg_3s = 21.20 t/s
2.32.641.428 I slot print_timing: id 3 | task 418 | n_decoded = 1762, tg = 21.25 t/s, tg_3s = 19.92 t/s
2.35.670.757 I slot print_timing: id 3 | task 418 | n_decoded = 1822, tg = 21.20 t/s, tg_3s = 19.81 t/s
2.38.683.555 I slot print_timing: id 3 | task 418 | n_decoded = 1883, tg = 21.17 t/s, tg_3s = 20.25 t/s
2.41.703.860 I slot print_timing: id 3 | task 418 | n_decoded = 1944, tg = 21.14 t/s, tg_3s = 20.20 t/s
2.44.748.763 I slot print_timing: id 3 | task 418 | n_decoded = 2005, tg = 21.10 t/s, tg_3s = 20.03 t/s
2.47.793.443 I slot print_timing: id 3 | task 418 | n_decoded = 2066, tg = 21.07 t/s, tg_3s = 20.03 t/s
2.50.830.700 I slot print_timing: id 3 | task 418 | n_decoded = 2127, tg = 21.04 t/s, tg_3s = 20.08 t/s
2.53.864.425 I slot print_timing: id 3 | task 418 | n_decoded = 2188, tg = 21.01 t/s, tg_3s = 20.11 t/s
2.56.896.858 I slot print_timing: id 3 | task 418 | n_decoded = 2248, tg = 20.98 t/s, tg_3s = 19.79 t/s
2.59.900.033 I slot print_timing: id 3 | task 418 | n_decoded = 2308, tg = 20.95 t/s, tg_3s = 19.98 t/s
3.02.900.072 I slot print_timing: id 3 | task 418 | n_decoded = 2368, tg = 20.93 t/s, tg_3s = 20.00 t/s
3.05.907.642 I slot print_timing: id 3 | task 418 | n_decoded = 2428, tg = 20.90 t/s, tg_3s = 19.95 t/s
3.08.944.329 I slot print_timing: id 3 | task 418 | n_decoded = 2488, tg = 20.87 t/s, tg_3s = 19.76 t/s
3.11.982.399 I slot print_timing: id 3 | task 418 | n_decoded = 2547, tg = 20.84 t/s, tg_3s = 19.42 t/s
3.15.025.535 I slot print_timing: id 3 | task 418 | n_decoded = 2607, tg = 20.81 t/s, tg_3s = 19.72 t/s
3.18.075.328 I slot print_timing: id 3 | task 418 | n_decoded = 2667, tg = 20.78 t/s, tg_3s = 19.67 t/s
3.21.086.751 I slot print_timing: id 3 | task 418 | n_decoded = 2725, tg = 20.75 t/s, tg_3s = 19.26 t/s
3.24.126.331 I slot print_timing: id 3 | task 418 | n_decoded = 2783, tg = 20.71 t/s, tg_3s = 19.08 t/s
3.27.168.143 I slot print_timing: id 3 | task 418 | n_decoded = 2842, tg = 20.68 t/s, tg_3s = 19.40 t/s
3.30.203.849 I slot print_timing: id 3 | task 418 | n_decoded = 2901, tg = 20.65 t/s, tg_3s = 19.44 t/s
3.33.238.474 I slot print_timing: id 3 | task 418 | n_decoded = 2959, tg = 20.62 t/s, tg_3s = 19.11 t/s
3.36.286.608 I slot print_timing: id 3 | task 418 | n_decoded = 3018, tg = 20.59 t/s, tg_3s = 19.36 t/s
3.39.314.636 I slot print_timing: id 3 | task 418 | n_decoded = 3076, tg = 20.56 t/s, tg_3s = 19.15 t/s
3.42.341.115 I slot print_timing: id 3 | task 418 | n_decoded = 3134, tg = 20.54 t/s, tg_3s = 19.16 t/s
3.45.387.310 I slot print_timing: id 3 | task 418 | n_decoded = 3192, tg = 20.51 t/s, tg_3s = 19.04 t/s
3.48.398.803 I slot print_timing: id 3 | task 418 | n_decoded = 3250, tg = 20.48 t/s, tg_3s = 19.26 t/s
3.51.401.420 I slot print_timing: id 3 | task 418 | n_decoded = 3308, tg = 20.46 t/s, tg_3s = 19.32 t/s
3.54.405.644 I slot print_timing: id 3 | task 418 | n_decoded = 3365, tg = 20.44 t/s, tg_3s = 18.97 t/s
3.57.434.688 I slot print_timing: id 3 | task 418 | n_decoded = 3423, tg = 20.41 t/s, tg_3s = 19.15 t/s
4.00.447.002 I slot print_timing: id 3 | task 418 | n_decoded = 3481, tg = 20.39 t/s, tg_3s = 19.25 t/s
4.03.471.390 I slot print_timing: id 3 | task 418 | n_decoded = 3539, tg = 20.37 t/s, tg_3s = 19.18 t/s
4.06.525.527 I slot print_timing: id 3 | task 418 | n_decoded = 3597, tg = 20.35 t/s, tg_3s = 18.99 t/s
4.09.556.299 I slot print_timing: id 3 | task 418 | n_decoded = 3654, tg = 20.32 t/s, tg_3s = 18.81 t/s
4.12.581.500 I slot print_timing: id 3 | task 418 | n_decoded = 3712, tg = 20.30 t/s, tg_3s = 19.17 t/s
4.15.626.035 I slot print_timing: id 3 | task 418 | n_decoded = 3770, tg = 20.28 t/s, tg_3s = 19.05 t/s
4.18.667.094 I slot print_timing: id 3 | task 418 | n_decoded = 3828, tg = 20.26 t/s, tg_3s = 19.07 t/s
4.21.709.163 I slot print_timing: id 3 | task 418 | n_decoded = 3886, tg = 20.24 t/s, tg_3s = 19.07 t/s
4.24.747.845 I slot print_timing: id 3 | task 418 | n_decoded = 3944, tg = 20.22 t/s, tg_3s = 19.09 t/s
4.27.790.237 I slot print_timing: id 3 | task 418 | n_decoded = 4002, tg = 20.21 t/s, tg_3s = 19.06 t/s
4.30.795.981 I slot print_timing: id 3 | task 418 | n_decoded = 4058, tg = 20.18 t/s, tg_3s = 18.63 t/s
4.33.797.416 I slot print_timing: id 3 | task 418 | n_decoded = 4115, tg = 20.17 t/s, tg_3s = 18.99 t/s
4.36.802.199 I slot print_timing: id 3 | task 418 | n_decoded = 4172, tg = 20.15 t/s, tg_3s = 18.97 t/s
4.39.811.384 I slot print_timing: id 3 | task 418 | n_decoded = 4229, tg = 20.13 t/s, tg_3s = 18.94 t/s
4.42.838.988 I slot print_timing: id 3 | task 418 | n_decoded = 4286, tg = 20.11 t/s, tg_3s = 18.83 t/s
4.45.870.191 I slot print_timing: id 3 | task 418 | n_decoded = 4343, tg = 20.09 t/s, tg_3s = 18.80 t/s
4.48.872.063 I slot print_timing: id 3 | task 418 | n_decoded = 4399, tg = 20.07 t/s, tg_3s = 18.66 t/s
4.51.922.518 I slot print_timing: id 3 | task 418 | n_decoded = 4455, tg = 20.05 t/s, tg_3s = 18.36 t/s
4.54.970.936 I slot print_timing: id 3 | task 418 | n_decoded = 4512, tg = 20.03 t/s, tg_3s = 18.70 t/s
4.58.013.636 I slot print_timing: id 3 | task 418 | n_decoded = 4568, tg = 20.01 t/s, tg_3s = 18.40 t/s
5.01.053.828 I slot print_timing: id 3 | task 418 | n_decoded = 4624, tg = 19.99 t/s, tg_3s = 18.42 t/s
5.04.070.978 I slot print_timing: id 3 | task 418 | n_decoded = 4679, tg = 19.97 t/s, tg_3s = 18.23 t/s
5.07.119.346 I slot print_timing: id 3 | task 418 | n_decoded = 4735, tg = 19.95 t/s, tg_3s = 18.37 t/s
5.10.153.663 I slot print_timing: id 3 | task 418 | n_decoded = 4791, tg = 19.93 t/s, tg_3s = 18.46 t/s
5.13.193.224 I slot print_timing: id 3 | task 418 | n_decoded = 4847, tg = 19.91 t/s, tg_3s = 18.42 t/s
5.16.231.134 I slot print_timing: id 3 | task 418 | n_decoded = 4903, tg = 19.89 t/s, tg_3s = 18.43 t/s
5.19.259.953 I slot print_timing: id 3 | task 418 | n_decoded = 4959, tg = 19.87 t/s, tg_3s = 18.49 t/s
5.22.287.463 I slot print_timing: id 3 | task 418 | n_decoded = 5015, tg = 19.86 t/s, tg_3s = 18.50 t/s
5.25.333.409 I slot print_timing: id 3 | task 418 | n_decoded = 5071, tg = 19.84 t/s, tg_3s = 18.39 t/s
5.28.374.200 I slot print_timing: id 3 | task 418 | n_decoded = 5127, tg = 19.82 t/s, tg_3s = 18.42 t/s
5.31.409.878 I slot print_timing: id 3 | task 418 | n_decoded = 5183, tg = 19.81 t/s, tg_3s = 18.45 t/s
5.34.462.379 I slot print_timing: id 3 | task 418 | n_decoded = 5239, tg = 19.79 t/s, tg_3s = 18.35 t/s
5.37.468.091 I slot print_timing: id 3 | task 418 | n_decoded = 5294, tg = 19.77 t/s, tg_3s = 18.30 t/s
5.40.471.134 I slot print_timing: id 3 | task 418 | n_decoded = 5349, tg = 19.76 t/s, tg_3s = 18.31 t/s
5.43.479.021 I slot print_timing: id 3 | task 418 | n_decoded = 5404, tg = 19.74 t/s, tg_3s = 18.29 t/s
5.46.485.018 I slot print_timing: id 3 | task 418 | n_decoded = 5459, tg = 19.73 t/s, tg_3s = 18.30 t/s
5.49.508.512 I slot print_timing: id 3 | task 418 | n_decoded = 5514, tg = 19.71 t/s, tg_3s = 18.19 t/s
5.52.526.964 I slot print_timing: id 3 | task 418 | n_decoded = 5568, tg = 19.69 t/s, tg_3s = 17.89 t/s
5.55.574.575 I slot print_timing: id 3 | task 418 | n_decoded = 5623, tg = 19.67 t/s, tg_3s = 18.05 t/s
5.58.630.241 I slot print_timing: id 3 | task 418 | n_decoded = 5677, tg = 19.65 t/s, tg_3s = 17.67 t/s
6.01.638.999 I slot print_timing: id 3 | task 418 | n_decoded = 5731, tg = 19.63 t/s, tg_3s = 17.95 t/s
6.04.646.251 I slot print_timing: id 3 | task 418 | n_decoded = 5785, tg = 19.62 t/s, tg_3s = 17.96 t/s
6.07.675.272 I slot print_timing: id 3 | task 418 | n_decoded = 5839, tg = 19.60 t/s, tg_3s = 17.83 t/s
6.10.694.458 I slot print_timing: id 3 | task 418 | n_decoded = 5893, tg = 19.58 t/s, tg_3s = 17.89 t/s
6.13.697.465 I slot print_timing: id 3 | task 418 | n_decoded = 5947, tg = 19.57 t/s, tg_3s = 17.98 t/s
6.16.741.562 I slot print_timing: id 3 | task 418 | n_decoded = 6002, tg = 19.55 t/s, tg_3s = 18.07 t/s
6.19.747.803 I slot print_timing: id 3 | task 418 | n_decoded = 6056, tg = 19.53 t/s, tg_3s = 17.96 t/s
6.22.772.533 I slot print_timing: id 3 | task 418 | n_decoded = 6109, tg = 19.52 t/s, tg_3s = 17.52 t/s
6.25.806.734 I slot print_timing: id 3 | task 418 | n_decoded = 6163, tg = 19.50 t/s, tg_3s = 17.80 t/s
6.28.835.210 I slot print_timing: id 3 | task 418 | n_decoded = 6217, tg = 19.48 t/s, tg_3s = 17.83 t/s
6.31.855.870 I slot print_timing: id 3 | task 418 | n_decoded = 6271, tg = 19.47 t/s, tg_3s = 17.88 t/s
6.34.898.238 I slot print_timing: id 3 | task 418 | n_decoded = 6325, tg = 19.45 t/s, tg_3s = 17.75 t/s
6.37.947.050 I slot print_timing: id 3 | task 418 | n_decoded = 6379, tg = 19.44 t/s, tg_3s = 17.71 t/s
6.41.001.794 I slot print_timing: id 3 | task 418 | n_decoded = 6433, tg = 19.42 t/s, tg_3s = 17.68 t/s
6.44.054.892 I slot print_timing: id 3 | task 418 | n_decoded = 6487, tg = 19.40 t/s, tg_3s = 17.69 t/s
6.47.111.559 I slot print_timing: id 3 | task 418 | n_decoded = 6541, tg = 19.39 t/s, tg_3s = 17.67 t/s
6.50.147.134 I slot print_timing: id 3 | task 418 | n_decoded = 6594, tg = 19.37 t/s, tg_3s = 17.46 t/s
6.53.164.010 I slot print_timing: id 3 | task 418 | n_decoded = 6647, tg = 19.35 t/s, tg_3s = 17.57 t/s
6.56.183.952 I slot print_timing: id 3 | task 418 | n_decoded = 6700, tg = 19.34 t/s, tg_3s = 17.55 t/s
6.59.190.459 I slot print_timing: id 3 | task 418 | n_decoded = 6753, tg = 19.32 t/s, tg_3s = 17.63 t/s
7.02.201.950 I slot print_timing: id 3 | task 418 | n_decoded = 6806, tg = 19.31 t/s, tg_3s = 17.60 t/s
7.05.228.205 I slot print_timing: id 3 | task 418 | n_decoded = 6858, tg = 19.29 t/s, tg_3s = 17.18 t/s
7.08.268.028 I slot print_timing: id 3 | task 418 | n_decoded = 6910, tg = 19.27 t/s, tg_3s = 17.11 t/s
7.11.312.249 I slot print_timing: id 3 | task 418 | n_decoded = 6963, tg = 19.26 t/s, tg_3s = 17.41 t/s
7.14.312.630 I slot print_timing: id 3 | task 418 | n_decoded = 7015, tg = 19.24 t/s, tg_3s = 17.33 t/s
7.17.358.157 I slot print_timing: id 3 | task 418 | n_decoded = 7068, tg = 19.23 t/s, tg_3s = 17.40 t/s
7.20.407.030 I slot print_timing: id 3 | task 418 | n_decoded = 7121, tg = 19.21 t/s, tg_3s = 17.38 t/s
7.23.432.276 I slot print_timing: id 3 | task 418 | n_decoded = 7174, tg = 19.20 t/s, tg_3s = 17.52 t/s
7.26.446.137 I slot print_timing: id 3 | task 418 | n_decoded = 7226, tg = 19.18 t/s, tg_3s = 17.25 t/s
7.29.492.619 I slot print_timing: id 3 | task 418 | n_decoded = 7279, tg = 19.17 t/s, tg_3s = 17.40 t/s
7.32.519.050 I slot print_timing: id 3 | task 418 | n_decoded = 7332, tg = 19.15 t/s, tg_3s = 17.51 t/s
7.35.527.365 I slot print_timing: id 3 | task 418 | n_decoded = 7383, tg = 19.14 t/s, tg_3s = 16.95 t/s
7.38.562.938 I slot print_timing: id 3 | task 418 | n_decoded = 7435, tg = 19.12 t/s, tg_3s = 17.13 t/s
7.41.613.119 I slot print_timing: id 3 | task 418 | n_decoded = 7487, tg = 19.11 t/s, tg_3s = 17.05 t/s
7.44.648.298 I slot print_timing: id 3 | task 418 | n_decoded = 7539, tg = 19.09 t/s, tg_3s = 17.13 t/s
7.47.692.227 I slot print_timing: id 3 | task 418 | n_decoded = 7591, tg = 19.08 t/s, tg_3s = 17.08 t/s
7.50.724.923 I slot print_timing: id 3 | task 418 | n_decoded = 7642, tg = 19.06 t/s, tg_3s = 16.82 t/s
7.53.759.406 I slot print_timing: id 3 | task 418 | n_decoded = 7694, tg = 19.04 t/s, tg_3s = 17.14 t/s
7.56.787.975 I slot print_timing: id 3 | task 418 | n_decoded = 7746, tg = 19.03 t/s, tg_3s = 17.17 t/s
7.59.808.435 I slot print_timing: id 3 | task 418 | n_decoded = 7798, tg = 19.02 t/s, tg_3s = 17.22 t/s
8.02.844.631 I slot print_timing: id 3 | task 418 | n_decoded = 7850, tg = 19.00 t/s, tg_3s = 17.13 t/s
8.05.894.458 I slot print_timing: id 3 | task 418 | n_decoded = 7902, tg = 18.99 t/s, tg_3s = 17.05 t/s
8.08.925.052 I slot print_timing: id 3 | task 418 | n_decoded = 7953, tg = 18.97 t/s, tg_3s = 16.83 t/s
8.11.927.938 I slot print_timing: id 3 | task 418 | n_decoded = 8004, tg = 18.96 t/s, tg_3s = 16.98 t/s
8.14.975.431 I slot print_timing: id 3 | task 418 | n_decoded = 8056, tg = 18.94 t/s, tg_3s = 17.06 t/s
8.17.979.621 I slot print_timing: id 3 | task 418 | n_decoded = 8107, tg = 18.93 t/s, tg_3s = 16.98 t/s
8.21.033.897 I slot print_timing: id 3 | task 418 | n_decoded = 8158, tg = 18.92 t/s, tg_3s = 16.70 t/s
8.23.054.574 I slot print_timing: id 3 | task 418 | prompt eval time = 1683.61 ms / 47 tokens ( 35.82 ms per token, 27.92 tokens per second)
8.23.054.579 I slot print_timing: id 3 | task 418 | eval time = 433316.33 ms / 8192 tokens ( 52.90 ms per token, 18.91 tokens per second)
8.23.054.580 I slot print_timing: id 3 | task 418 | total time = 434999.94 ms / 8239 tokens
8.23.054.580 I slot print_timing: id 3 | task 418 | graphs reused = 8569
8.23.054.609 I slot release: id 3 | task 418 | stop processing: n_tokens = 8282, truncated = 0
8.23.054.649 I srv update_slots: all slots are idle
8.23.056.149 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
8.23.075.311 I srv operator(): Chat format: peg-native
8.23.081.333 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.629 (> 0.100 thold), f_keep = 0.005
8.23.081.336 I srv get_availabl: updating prompt cache
8.23.081.669 W srv prompt_save: - saving prompt with length 8282, total state size = 2005.893 MiB (draft: 0.000 MiB)
8.24.456.030 I srv load: - looking for better prompt, base f_keep = 0.005, sim = 0.629
8.24.456.036 I srv update: - cache state: 3 prompts, 2139.105 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
8.24.456.037 I srv update: - prompt 0x6354c2edb8c0: 425 tokens, checkpoints: 0, 102.936 MiB
8.24.456.038 I srv update: - prompt 0x6354c2eccbe0: 125 tokens, checkpoints: 0, 30.276 MiB
8.24.456.039 I srv update: - prompt 0x6354c2c0b630: 8282 tokens, checkpoints: 0, 2005.893 MiB
8.24.456.040 I srv get_availabl: prompt cache update took 1374.70 ms
8.24.457.827 I slot launch_slot_: id 3 | task 8611 | processing task, is_child = 0
8.24.457.829 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
8.24.457.830 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
8.24.457.860 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
8.24.457.861 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
8.24.457.876 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
8.24.457.877 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
8.29.929.203 I slot print_timing: id 3 | task 8611 | n_decoded = 100, tg = 21.54 t/s, tg_3s = 21.54 t/s
8.32.961.167 I slot print_timing: id 3 | task 8611 | n_decoded = 164, tg = 21.37 t/s, tg_3s = 21.11 t/s
8.36.000.247 I slot print_timing: id 3 | task 8611 | n_decoded = 228, tg = 21.28 t/s, tg_3s = 21.06 t/s
8.39.009.873 I slot print_timing: id 3 | task 8611 | n_decoded = 292, tg = 21.28 t/s, tg_3s = 21.27 t/s
8.42.052.077 I slot print_timing: id 3 | task 8611 | n_decoded = 358, tg = 21.35 t/s, tg_3s = 21.69 t/s
8.45.072.017 I slot print_timing: id 3 | task 8611 | n_decoded = 423, tg = 21.38 t/s, tg_3s = 21.52 t/s
8.48.097.679 I slot print_timing: id 3 | task 8611 | n_decoded = 488, tg = 21.39 t/s, tg_3s = 21.48 t/s
8.51.099.211 I slot print_timing: id 3 | task 8611 | n_decoded = 553, tg = 21.42 t/s, tg_3s = 21.66 t/s
8.52.398.486 I slot print_timing: id 3 | task 8611 | prompt eval time = 829.81 ms / 26 tokens ( 31.92 ms per token, 31.33 tokens per second)
8.52.398.490 I slot print_timing: id 3 | task 8611 | eval time = 27110.77 ms / 581 tokens ( 46.66 ms per token, 21.43 tokens per second)
8.52.398.490 I slot print_timing: id 3 | task 8611 | total time = 27940.57 ms / 607 tokens
8.52.398.492 I slot print_timing: id 3 | task 8611 | graphs reused = 9146
8.52.398.521 I slot release: id 3 | task 8611 | stop processing: n_tokens = 650, truncated = 0
8.52.398.536 I srv update_slots: all slots are idle
8.52.399.427 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
8.52.418.305 I srv operator(): Chat format: peg-native
8.52.424.319 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.657 (> 0.100 thold), f_keep = 0.068
8.52.424.321 I srv get_availabl: updating prompt cache
8.52.424.375 W srv prompt_save: - saving prompt with length 650, total state size = 157.431 MiB (draft: 0.000 MiB)
8.52.537.072 I srv load: - looking for better prompt, base f_keep = 0.068, sim = 0.657
8.52.537.080 I srv update: - cache state: 4 prompts, 2296.536 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
8.52.537.081 I srv update: - prompt 0x6354c2edb8c0: 425 tokens, checkpoints: 0, 102.936 MiB
8.52.537.081 I srv update: - prompt 0x6354c2eccbe0: 125 tokens, checkpoints: 0, 30.276 MiB
8.52.537.082 I srv update: - prompt 0x6354c2c0b630: 8282 tokens, checkpoints: 0, 2005.893 MiB
8.52.537.083 I srv update: - prompt 0x6354c6628780: 650 tokens, checkpoints: 0, 157.431 MiB
8.52.537.084 I srv get_availabl: prompt cache update took 112.76 ms
8.52.538.864 I slot launch_slot_: id 3 | task 9193 | processing task, is_child = 0
8.52.538.866 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
8.52.538.867 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
8.52.538.881 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
8.52.538.881 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
8.52.538.894 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
8.52.538.895 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
8.57.827.134 I slot print_timing: id 3 | task 9193 | n_decoded = 100, tg = 21.79 t/s, tg_3s = 21.79 t/s
9.00.840.372 I slot print_timing: id 3 | task 9193 | n_decoded = 165, tg = 21.70 t/s, tg_3s = 21.57 t/s
9.03.845.510 I slot print_timing: id 3 | task 9193 | n_decoded = 230, tg = 21.68 t/s, tg_3s = 21.63 t/s
9.06.875.947 I slot print_timing: id 3 | task 9193 | n_decoded = 294, tg = 21.56 t/s, tg_3s = 21.12 t/s
9.09.830.295 I slot print_timing: id 3 | task 9193 | prompt eval time = 699.28 ms / 23 tokens ( 30.40 ms per token, 32.89 tokens per second)
9.09.830.300 I slot print_timing: id 3 | task 9193 | eval time = 16592.08 ms / 357 tokens ( 46.48 ms per token, 21.52 tokens per second)
9.09.830.301 I slot print_timing: id 3 | task 9193 | total time = 17291.37 ms / 380 tokens
9.09.830.302 I slot print_timing: id 3 | task 9193 | graphs reused = 9500
9.09.830.328 I slot release: id 3 | task 9193 | stop processing: n_tokens = 423, truncated = 0
9.09.830.343 I srv update_slots: all slots are idle
9.09.831.418 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
9.09.850.555 I srv operator(): Chat format: peg-native
9.09.856.552 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.518 (> 0.100 thold), f_keep = 0.104
9.09.856.554 I srv get_availabl: updating prompt cache
9.09.856.599 W srv prompt_save: - saving prompt with length 423, total state size = 102.452 MiB (draft: 0.000 MiB)
9.09.932.202 I srv load: - looking for better prompt, base f_keep = 0.104, sim = 0.518
9.09.932.209 I srv update: - cache state: 5 prompts, 2398.988 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
9.09.932.210 I srv update: - prompt 0x6354c2edb8c0: 425 tokens, checkpoints: 0, 102.936 MiB
9.09.932.211 I srv update: - prompt 0x6354c2eccbe0: 125 tokens, checkpoints: 0, 30.276 MiB
9.09.932.212 I srv update: - prompt 0x6354c2c0b630: 8282 tokens, checkpoints: 0, 2005.893 MiB
9.09.932.216 I srv update: - prompt 0x6354c6628780: 650 tokens, checkpoints: 0, 157.431 MiB
9.09.932.217 I srv update: - prompt 0x6354c28b1d40: 423 tokens, checkpoints: 0, 102.452 MiB
9.09.932.218 I srv get_availabl: prompt cache update took 75.66 ms
9.09.933.980 I slot launch_slot_: id 3 | task 9551 | processing task, is_child = 0
9.09.933.982 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
9.09.933.983 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
9.09.933.998 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
9.09.933.999 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
9.09.934.011 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
9.09.934.012 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
9.16.203.484 I slot print_timing: id 3 | task 9551 | n_decoded = 100, tg = 21.69 t/s, tg_3s = 21.69 t/s
9.19.239.480 I slot print_timing: id 3 | task 9551 | n_decoded = 166, tg = 21.71 t/s, tg_3s = 21.74 t/s
9.22.244.594 I slot print_timing: id 3 | task 9551 | n_decoded = 231, tg = 21.69 t/s, tg_3s = 21.63 t/s
9.25.272.120 I slot print_timing: id 3 | task 9551 | n_decoded = 296, tg = 21.64 t/s, tg_3s = 21.47 t/s
9.28.316.609 I slot print_timing: id 3 | task 9551 | n_decoded = 361, tg = 21.59 t/s, tg_3s = 21.35 t/s
9.31.317.297 I slot print_timing: id 3 | task 9551 | n_decoded = 426, tg = 21.60 t/s, tg_3s = 21.66 t/s
9.32.410.215 I slot print_timing: id 3 | task 9551 | prompt eval time = 1659.55 ms / 41 tokens ( 40.48 ms per token, 24.71 tokens per second)
9.32.410.219 I slot print_timing: id 3 | task 9551 | eval time = 20816.62 ms / 449 tokens ( 46.36 ms per token, 21.57 tokens per second)
9.32.410.220 I slot print_timing: id 3 | task 9551 | total time = 22476.17 ms / 490 tokens
9.32.410.220 I slot print_timing: id 3 | task 9551 | graphs reused = 9945
9.32.410.251 I slot release: id 3 | task 9551 | stop processing: n_tokens = 533, truncated = 0
9.32.410.265 I srv update_slots: all slots are idle
9.32.411.261 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
9.32.428.794 I srv operator(): Chat format: peg-native
9.32.434.361 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.524 (> 0.100 thold), f_keep = 0.083
9.32.434.363 I srv get_availabl: updating prompt cache
9.32.434.409 W srv prompt_save: - saving prompt with length 533, total state size = 129.093 MiB (draft: 0.000 MiB)
9.32.521.530 I srv load: - looking for better prompt, base f_keep = 0.083, sim = 0.524
9.32.521.537 I srv update: - cache state: 6 prompts, 2528.081 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
9.32.521.538 I srv update: - prompt 0x6354c2edb8c0: 425 tokens, checkpoints: 0, 102.936 MiB
9.32.521.539 I srv update: - prompt 0x6354c2eccbe0: 125 tokens, checkpoints: 0, 30.276 MiB
9.32.521.539 I srv update: - prompt 0x6354c2c0b630: 8282 tokens, checkpoints: 0, 2005.893 MiB
9.32.521.540 I srv update: - prompt 0x6354c6628780: 650 tokens, checkpoints: 0, 157.431 MiB
9.32.521.540 I srv update: - prompt 0x6354c28b1d40: 423 tokens, checkpoints: 0, 102.452 MiB
9.32.521.541 I srv update: - prompt 0x6354c9e4fcf0: 533 tokens, checkpoints: 0, 129.093 MiB
9.32.521.542 I srv get_availabl: prompt cache update took 87.18 ms
9.32.523.197 I slot launch_slot_: id 3 | task 10001 | processing task, is_child = 0
9.32.523.199 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
9.32.523.200 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
9.32.523.213 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
9.32.523.213 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
9.32.523.225 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
9.32.523.225 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
9.38.626.582 I slot print_timing: id 3 | task 10001 | n_decoded = 100, tg = 21.78 t/s, tg_3s = 21.78 t/s
9.41.631.878 I slot print_timing: id 3 | task 10001 | n_decoded = 165, tg = 21.72 t/s, tg_3s = 21.63 t/s
9.44.673.633 I slot print_timing: id 3 | task 10001 | n_decoded = 230, tg = 21.62 t/s, tg_3s = 21.37 t/s
9.47.684.316 I slot print_timing: id 3 | task 10001 | n_decoded = 294, tg = 21.54 t/s, tg_3s = 21.26 t/s
9.50.721.625 I slot print_timing: id 3 | task 10001 | n_decoded = 360, tg = 21.58 t/s, tg_3s = 21.73 t/s
9.53.734.185 I slot print_timing: id 3 | task 10001 | n_decoded = 425, tg = 21.58 t/s, tg_3s = 21.58 t/s
9.56.757.866 I slot print_timing: id 3 | task 10001 | n_decoded = 490, tg = 21.57 t/s, tg_3s = 21.50 t/s
9.59.766.706 I slot print_timing: id 3 | task 10001 | n_decoded = 555, tg = 21.57 t/s, tg_3s = 21.60 t/s
10.02.804.589 I slot print_timing: id 3 | task 10001 | n_decoded = 620, tg = 21.55 t/s, tg_3s = 21.40 t/s
10.05.821.938 I slot print_timing: id 3 | task 10001 | n_decoded = 685, tg = 21.55 t/s, tg_3s = 21.54 t/s
10.08.843.090 I slot print_timing: id 3 | task 10001 | n_decoded = 749, tg = 21.52 t/s, tg_3s = 21.18 t/s
10.11.875.067 I slot print_timing: id 3 | task 10001 | n_decoded = 813, tg = 21.49 t/s, tg_3s = 21.11 t/s
10.14.891.032 I slot print_timing: id 3 | task 10001 | n_decoded = 877, tg = 21.47 t/s, tg_3s = 21.22 t/s
10.17.924.991 I slot print_timing: id 3 | task 10001 | n_decoded = 941, tg = 21.44 t/s, tg_3s = 21.09 t/s
10.20.936.119 I slot print_timing: id 3 | task 10001 | n_decoded = 1005, tg = 21.43 t/s, tg_3s = 21.25 t/s
10.23.940.644 I slot print_timing: id 3 | task 10001 | n_decoded = 1069, tg = 21.42 t/s, tg_3s = 21.30 t/s
10.26.958.053 I slot print_timing: id 3 | task 10001 | n_decoded = 1133, tg = 21.41 t/s, tg_3s = 21.21 t/s
10.29.969.729 I slot print_timing: id 3 | task 10001 | n_decoded = 1197, tg = 21.40 t/s, tg_3s = 21.25 t/s
10.31.438.054 I slot print_timing: id 3 | task 10001 | prompt eval time = 1512.96 ms / 40 tokens ( 37.82 ms per token, 26.44 tokens per second)
10.31.438.058 I slot print_timing: id 3 | task 10001 | eval time = 57401.84 ms / 1228 tokens ( 46.74 ms per token, 21.39 tokens per second)
10.31.438.058 I slot print_timing: id 3 | task 10001 | total time = 58914.80 ms / 1268 tokens
10.31.438.059 I slot print_timing: id 3 | task 10001 | graphs reused = 11166
10.31.438.086 I slot release: id 3 | task 10001 | stop processing: n_tokens = 1311, truncated = 0
10.31.438.105 I srv update_slots: all slots are idle
10.31.439.085 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
10.31.457.611 I srv operator(): Chat format: peg-native
10.31.463.129 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.629 (> 0.100 thold), f_keep = 0.034
10.31.463.131 I srv get_availabl: updating prompt cache
10.31.463.203 W srv prompt_save: - saving prompt with length 1311, total state size = 317.524 MiB (draft: 0.000 MiB)
10.31.669.936 I srv load: - looking for better prompt, base f_keep = 0.034, sim = 0.629
10.31.669.943 I srv update: - cache state: 7 prompts, 2845.605 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
10.31.669.944 I srv update: - prompt 0x6354c2edb8c0: 425 tokens, checkpoints: 0, 102.936 MiB
10.31.669.944 I srv update: - prompt 0x6354c2eccbe0: 125 tokens, checkpoints: 0, 30.276 MiB
10.31.669.945 I srv update: - prompt 0x6354c2c0b630: 8282 tokens, checkpoints: 0, 2005.893 MiB
10.31.669.946 I srv update: - prompt 0x6354c6628780: 650 tokens, checkpoints: 0, 157.431 MiB
10.31.669.946 I srv update: - prompt 0x6354c28b1d40: 423 tokens, checkpoints: 0, 102.452 MiB
10.31.669.947 I srv update: - prompt 0x6354c9e4fcf0: 533 tokens, checkpoints: 0, 129.093 MiB
10.31.669.947 I srv update: - prompt 0x6354c2d32b70: 1311 tokens, checkpoints: 0, 317.524 MiB
10.31.669.948 I srv get_availabl: prompt cache update took 206.82 ms
10.31.671.544 I slot launch_slot_: id 3 | task 11230 | processing task, is_child = 0
10.31.671.546 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
10.31.671.546 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
10.31.671.560 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
10.31.671.560 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
10.31.671.572 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
10.31.671.573 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
10.37.126.277 I slot print_timing: id 3 | task 11230 | n_decoded = 100, tg = 21.63 t/s, tg_3s = 21.63 t/s
10.40.158.760 I slot print_timing: id 3 | task 11230 | n_decoded = 166, tg = 21.68 t/s, tg_3s = 21.76 t/s
10.43.167.527 I slot print_timing: id 3 | task 11230 | n_decoded = 231, tg = 21.66 t/s, tg_3s = 21.60 t/s
10.46.178.948 I slot print_timing: id 3 | task 11230 | n_decoded = 296, tg = 21.64 t/s, tg_3s = 21.58 t/s
10.49.216.538 I slot print_timing: id 3 | task 11230 | n_decoded = 362, tg = 21.66 t/s, tg_3s = 21.73 t/s
10.52.252.581 I slot print_timing: id 3 | task 11230 | n_decoded = 428, tg = 21.67 t/s, tg_3s = 21.74 t/s
10.55.266.891 I slot print_timing: id 3 | task 11230 | n_decoded = 493, tg = 21.66 t/s, tg_3s = 21.56 t/s
10.58.291.139 I slot print_timing: id 3 | task 11230 | n_decoded = 558, tg = 21.64 t/s, tg_3s = 21.49 t/s
11.01.338.828 I slot print_timing: id 3 | task 11230 | n_decoded = 623, tg = 21.61 t/s, tg_3s = 21.33 t/s
11.04.343.637 I slot print_timing: id 3 | task 11230 | n_decoded = 687, tg = 21.58 t/s, tg_3s = 21.30 t/s
11.07.357.004 I slot print_timing: id 3 | task 11230 | n_decoded = 751, tg = 21.55 t/s, tg_3s = 21.24 t/s
11.10.378.881 I slot print_timing: id 3 | task 11230 | n_decoded = 815, tg = 21.52 t/s, tg_3s = 21.18 t/s
11.13.398.219 I slot print_timing: id 3 | task 11230 | n_decoded = 879, tg = 21.49 t/s, tg_3s = 21.20 t/s
11.16.401.906 I slot print_timing: id 3 | task 11230 | n_decoded = 943, tg = 21.48 t/s, tg_3s = 21.31 t/s
11.19.418.270 I slot print_timing: id 3 | task 11230 | n_decoded = 1007, tg = 21.46 t/s, tg_3s = 21.22 t/s
11.22.459.321 I slot print_timing: id 3 | task 11230 | n_decoded = 1071, tg = 21.44 t/s, tg_3s = 21.05 t/s
11.25.467.303 I slot print_timing: id 3 | task 11230 | n_decoded = 1135, tg = 21.43 t/s, tg_3s = 21.28 t/s
11.28.475.382 I slot print_timing: id 3 | task 11230 | n_decoded = 1199, tg = 21.42 t/s, tg_3s = 21.28 t/s
11.31.511.535 I slot print_timing: id 3 | task 11230 | n_decoded = 1263, tg = 21.40 t/s, tg_3s = 21.08 t/s
11.34.522.404 I slot print_timing: id 3 | task 11230 | n_decoded = 1327, tg = 21.40 t/s, tg_3s = 21.26 t/s
11.37.535.300 I slot print_timing: id 3 | task 11230 | n_decoded = 1391, tg = 21.39 t/s, tg_3s = 21.24 t/s
11.40.546.712 I slot print_timing: id 3 | task 11230 | n_decoded = 1455, tg = 21.38 t/s, tg_3s = 21.25 t/s
11.43.583.505 I slot print_timing: id 3 | task 11230 | n_decoded = 1519, tg = 21.37 t/s, tg_3s = 21.07 t/s
11.46.610.648 I slot print_timing: id 3 | task 11230 | n_decoded = 1582, tg = 21.35 t/s, tg_3s = 20.81 t/s
11.49.649.938 I slot print_timing: id 3 | task 11230 | n_decoded = 1646, tg = 21.34 t/s, tg_3s = 21.06 t/s
11.52.670.757 I slot print_timing: id 3 | task 11230 | n_decoded = 1710, tg = 21.33 t/s, tg_3s = 21.19 t/s
11.55.697.814 I slot print_timing: id 3 | task 11230 | n_decoded = 1771, tg = 21.29 t/s, tg_3s = 20.15 t/s
11.58.745.524 I slot print_timing: id 3 | task 11230 | n_decoded = 1833, tg = 21.25 t/s, tg_3s = 20.34 t/s
12.01.760.920 I slot print_timing: id 3 | task 11230 | n_decoded = 1894, tg = 21.22 t/s, tg_3s = 20.23 t/s
12.04.761.951 I slot print_timing: id 3 | task 11230 | n_decoded = 1955, tg = 21.19 t/s, tg_3s = 20.33 t/s
12.07.792.245 I slot print_timing: id 3 | task 11230 | n_decoded = 2016, tg = 21.16 t/s, tg_3s = 20.13 t/s
12.10.824.546 I slot print_timing: id 3 | task 11230 | n_decoded = 2077, tg = 21.12 t/s, tg_3s = 20.12 t/s
12.13.853.236 I slot print_timing: id 3 | task 11230 | n_decoded = 2138, tg = 21.10 t/s, tg_3s = 20.14 t/s
12.16.879.291 I slot print_timing: id 3 | task 11230 | n_decoded = 2199, tg = 21.07 t/s, tg_3s = 20.16 t/s
12.19.926.850 I slot print_timing: id 3 | task 11230 | n_decoded = 2260, tg = 21.04 t/s, tg_3s = 20.02 t/s
12.22.944.954 I slot print_timing: id 3 | task 11230 | n_decoded = 2320, tg = 21.01 t/s, tg_3s = 19.88 t/s
12.25.985.820 I slot print_timing: id 3 | task 11230 | n_decoded = 2380, tg = 20.97 t/s, tg_3s = 19.73 t/s
12.29.014.000 I slot print_timing: id 3 | task 11230 | n_decoded = 2440, tg = 20.94 t/s, tg_3s = 19.81 t/s
12.32.044.558 I slot print_timing: id 3 | task 11230 | n_decoded = 2500, tg = 20.91 t/s, tg_3s = 19.80 t/s
12.35.085.882 I slot print_timing: id 3 | task 11230 | n_decoded = 2560, tg = 20.88 t/s, tg_3s = 19.73 t/s
12.38.125.458 I slot print_timing: id 3 | task 11230 | n_decoded = 2620, tg = 20.86 t/s, tg_3s = 19.74 t/s
12.41.174.977 I slot print_timing: id 3 | task 11230 | n_decoded = 2680, tg = 20.83 t/s, tg_3s = 19.68 t/s
12.44.216.093 I slot print_timing: id 3 | task 11230 | n_decoded = 2740, tg = 20.80 t/s, tg_3s = 19.73 t/s
12.47.265.023 I slot print_timing: id 3 | task 11230 | n_decoded = 2799, tg = 20.77 t/s, tg_3s = 19.35 t/s
12.50.307.171 I slot print_timing: id 3 | task 11230 | n_decoded = 2858, tg = 20.74 t/s, tg_3s = 19.39 t/s
12.53.342.311 I slot print_timing: id 3 | task 11230 | n_decoded = 2917, tg = 20.71 t/s, tg_3s = 19.44 t/s
12.56.357.343 I slot print_timing: id 3 | task 11230 | n_decoded = 2976, tg = 20.69 t/s, tg_3s = 19.57 t/s
12.59.399.497 I slot print_timing: id 3 | task 11230 | n_decoded = 3035, tg = 20.66 t/s, tg_3s = 19.39 t/s
13.02.434.104 I slot print_timing: id 3 | task 11230 | n_decoded = 3094, tg = 20.64 t/s, tg_3s = 19.44 t/s
13.05.478.775 I slot print_timing: id 3 | task 11230 | n_decoded = 3153, tg = 20.61 t/s, tg_3s = 19.38 t/s
13.08.521.506 I slot print_timing: id 3 | task 11230 | n_decoded = 3211, tg = 20.58 t/s, tg_3s = 19.06 t/s
13.11.553.473 I slot print_timing: id 3 | task 11230 | n_decoded = 3269, tg = 20.55 t/s, tg_3s = 19.13 t/s
13.14.605.645 I slot print_timing: id 3 | task 11230 | n_decoded = 3328, tg = 20.53 t/s, tg_3s = 19.33 t/s
13.17.651.414 I slot print_timing: id 3 | task 11230 | n_decoded = 3387, tg = 20.51 t/s, tg_3s = 19.37 t/s
13.20.700.924 I slot print_timing: id 3 | task 11230 | n_decoded = 3446, tg = 20.49 t/s, tg_3s = 19.35 t/s
13.23.715.924 I slot print_timing: id 3 | task 11230 | n_decoded = 3504, tg = 20.47 t/s, tg_3s = 19.24 t/s
13.26.767.010 I slot print_timing: id 3 | task 11230 | n_decoded = 3562, tg = 20.44 t/s, tg_3s = 19.01 t/s
13.29.792.884 I slot print_timing: id 3 | task 11230 | n_decoded = 3620, tg = 20.42 t/s, tg_3s = 19.17 t/s
13.32.815.046 I slot print_timing: id 3 | task 11230 | n_decoded = 3678, tg = 20.40 t/s, tg_3s = 19.19 t/s
13.35.836.728 I slot print_timing: id 3 | task 11230 | n_decoded = 3736, tg = 20.38 t/s, tg_3s = 19.19 t/s
13.38.872.947 I slot print_timing: id 3 | task 11230 | n_decoded = 3794, tg = 20.36 t/s, tg_3s = 19.10 t/s
13.41.906.830 I slot print_timing: id 3 | task 11230 | n_decoded = 3852, tg = 20.34 t/s, tg_3s = 19.12 t/s
13.44.939.451 I slot print_timing: id 3 | task 11230 | n_decoded = 3910, tg = 20.32 t/s, tg_3s = 19.13 t/s
13.47.940.864 I slot print_timing: id 3 | task 11230 | n_decoded = 3967, tg = 20.30 t/s, tg_3s = 18.99 t/s
13.50.982.039 I slot print_timing: id 3 | task 11230 | n_decoded = 4025, tg = 20.28 t/s, tg_3s = 19.07 t/s
13.54.000.656 I slot print_timing: id 3 | task 11230 | n_decoded = 4082, tg = 20.26 t/s, tg_3s = 18.88 t/s
13.57.009.474 I slot print_timing: id 3 | task 11230 | n_decoded = 4139, tg = 20.24 t/s, tg_3s = 18.94 t/s
14.00.023.920 I slot print_timing: id 3 | task 11230 | n_decoded = 4196, tg = 20.22 t/s, tg_3s = 18.91 t/s
14.03.056.281 I slot print_timing: id 3 | task 11230 | n_decoded = 4253, tg = 20.20 t/s, tg_3s = 18.80 t/s
14.06.072.424 I slot print_timing: id 3 | task 11230 | n_decoded = 4309, tg = 20.18 t/s, tg_3s = 18.57 t/s
14.09.100.081 I slot print_timing: id 3 | task 11230 | n_decoded = 4365, tg = 20.15 t/s, tg_3s = 18.50 t/s
14.12.136.849 I slot print_timing: id 3 | task 11230 | n_decoded = 4422, tg = 20.13 t/s, tg_3s = 18.77 t/s
14.15.180.010 I slot print_timing: id 3 | task 11230 | n_decoded = 4479, tg = 20.11 t/s, tg_3s = 18.73 t/s
14.18.221.020 I slot print_timing: id 3 | task 11230 | n_decoded = 4536, tg = 20.10 t/s, tg_3s = 18.74 t/s
14.21.239.463 I slot print_timing: id 3 | task 11230 | n_decoded = 4591, tg = 20.07 t/s, tg_3s = 18.22 t/s
14.24.292.897 I slot print_timing: id 3 | task 11230 | n_decoded = 4646, tg = 20.04 t/s, tg_3s = 18.01 t/s
14.27.329.028 I slot print_timing: id 3 | task 11230 | n_decoded = 4702, tg = 20.02 t/s, tg_3s = 18.44 t/s
14.30.367.537 I slot print_timing: id 3 | task 11230 | n_decoded = 4758, tg = 20.00 t/s, tg_3s = 18.43 t/s
14.33.404.774 I slot print_timing: id 3 | task 11230 | n_decoded = 4814, tg = 19.98 t/s, tg_3s = 18.44 t/s
14.36.425.012 I slot print_timing: id 3 | task 11230 | n_decoded = 4869, tg = 19.96 t/s, tg_3s = 18.21 t/s
14.39.464.646 I slot print_timing: id 3 | task 11230 | n_decoded = 4924, tg = 19.94 t/s, tg_3s = 18.09 t/s
14.42.497.691 I slot print_timing: id 3 | task 11230 | n_decoded = 4979, tg = 19.92 t/s, tg_3s = 18.13 t/s
14.45.540.051 I slot print_timing: id 3 | task 11230 | n_decoded = 5035, tg = 19.90 t/s, tg_3s = 18.41 t/s
14.48.591.703 I slot print_timing: id 3 | task 11230 | n_decoded = 5091, tg = 19.88 t/s, tg_3s = 18.35 t/s
14.51.625.725 I slot print_timing: id 3 | task 11230 | n_decoded = 5146, tg = 19.86 t/s, tg_3s = 18.13 t/s
14.54.650.529 I slot print_timing: id 3 | task 11230 | n_decoded = 5201, tg = 19.84 t/s, tg_3s = 18.18 t/s
14.57.698.427 I slot print_timing: id 3 | task 11230 | n_decoded = 5257, tg = 19.82 t/s, tg_3s = 18.37 t/s
15.00.749.380 I slot print_timing: id 3 | task 11230 | n_decoded = 5313, tg = 19.81 t/s, tg_3s = 18.35 t/s
15.03.757.154 I slot print_timing: id 3 | task 11230 | n_decoded = 5368, tg = 19.79 t/s, tg_3s = 18.29 t/s
15.06.765.504 I slot print_timing: id 3 | task 11230 | n_decoded = 5423, tg = 19.77 t/s, tg_3s = 18.28 t/s
15.09.779.554 I slot print_timing: id 3 | task 11230 | n_decoded = 5478, tg = 19.76 t/s, tg_3s = 18.25 t/s
15.12.787.015 I slot print_timing: id 3 | task 11230 | n_decoded = 5533, tg = 19.74 t/s, tg_3s = 18.29 t/s
15.15.820.983 I slot print_timing: id 3 | task 11230 | n_decoded = 5588, tg = 19.72 t/s, tg_3s = 18.13 t/s
15.18.857.598 I slot print_timing: id 3 | task 11230 | n_decoded = 5643, tg = 19.71 t/s, tg_3s = 18.11 t/s
15.21.879.884 I slot print_timing: id 3 | task 11230 | n_decoded = 5697, tg = 19.69 t/s, tg_3s = 17.87 t/s
15.24.923.011 I slot print_timing: id 3 | task 11230 | n_decoded = 5752, tg = 19.67 t/s, tg_3s = 18.07 t/s
15.27.965.909 I slot print_timing: id 3 | task 11230 | n_decoded = 5807, tg = 19.65 t/s, tg_3s = 18.07 t/s
15.31.020.485 I slot print_timing: id 3 | task 11230 | n_decoded = 5862, tg = 19.64 t/s, tg_3s = 18.01 t/s
15.34.069.492 I slot print_timing: id 3 | task 11230 | n_decoded = 5917, tg = 19.62 t/s, tg_3s = 18.04 t/s
15.37.117.932 I slot print_timing: id 3 | task 11230 | n_decoded = 5972, tg = 19.61 t/s, tg_3s = 18.04 t/s
15.40.165.454 I slot print_timing: id 3 | task 11230 | n_decoded = 6027, tg = 19.59 t/s, tg_3s = 18.05 t/s
15.43.175.319 I slot print_timing: id 3 | task 11230 | n_decoded = 6081, tg = 19.57 t/s, tg_3s = 17.94 t/s
15.46.228.151 I slot print_timing: id 3 | task 11230 | n_decoded = 6135, tg = 19.56 t/s, tg_3s = 17.69 t/s
15.49.245.470 I slot print_timing: id 3 | task 11230 | n_decoded = 6189, tg = 19.54 t/s, tg_3s = 17.90 t/s
15.52.262.354 I slot print_timing: id 3 | task 11230 | n_decoded = 6243, tg = 19.52 t/s, tg_3s = 17.90 t/s
15.55.281.845 I slot print_timing: id 3 | task 11230 | n_decoded = 6296, tg = 19.51 t/s, tg_3s = 17.55 t/s
15.58.334.604 I slot print_timing: id 3 | task 11230 | n_decoded = 6350, tg = 19.49 t/s, tg_3s = 17.69 t/s
16.01.383.456 I slot print_timing: id 3 | task 11230 | n_decoded = 6404, tg = 19.47 t/s, tg_3s = 17.71 t/s
16.04.438.047 I slot print_timing: id 3 | task 11230 | n_decoded = 6458, tg = 19.46 t/s, tg_3s = 17.68 t/s
16.07.454.775 I slot print_timing: id 3 | task 11230 | n_decoded = 6511, tg = 19.44 t/s, tg_3s = 17.57 t/s
16.10.491.443 I slot print_timing: id 3 | task 11230 | n_decoded = 6564, tg = 19.42 t/s, tg_3s = 17.45 t/s
16.13.511.797 I slot print_timing: id 3 | task 11230 | n_decoded = 6617, tg = 19.40 t/s, tg_3s = 17.55 t/s
16.16.524.595 I slot print_timing: id 3 | task 11230 | n_decoded = 6670, tg = 19.39 t/s, tg_3s = 17.59 t/s
16.19.534.998 I slot print_timing: id 3 | task 11230 | n_decoded = 6723, tg = 19.37 t/s, tg_3s = 17.61 t/s
16.22.550.749 I slot print_timing: id 3 | task 11230 | n_decoded = 6776, tg = 19.36 t/s, tg_3s = 17.57 t/s
16.25.557.963 I slot print_timing: id 3 | task 11230 | n_decoded = 6829, tg = 19.34 t/s, tg_3s = 17.62 t/s
16.28.585.309 I slot print_timing: id 3 | task 11230 | n_decoded = 6882, tg = 19.33 t/s, tg_3s = 17.51 t/s
16.31.634.376 I slot print_timing: id 3 | task 11230 | n_decoded = 6935, tg = 19.31 t/s, tg_3s = 17.38 t/s
16.34.678.083 I slot print_timing: id 3 | task 11230 | n_decoded = 6988, tg = 19.29 t/s, tg_3s = 17.41 t/s
16.37.713.058 I slot print_timing: id 3 | task 11230 | n_decoded = 7041, tg = 19.28 t/s, tg_3s = 17.46 t/s
16.40.740.196 I slot print_timing: id 3 | task 11230 | n_decoded = 7094, tg = 19.26 t/s, tg_3s = 17.51 t/s
16.43.782.193 I slot print_timing: id 3 | task 11230 | n_decoded = 7147, tg = 19.25 t/s, tg_3s = 17.42 t/s
16.46.804.650 I slot print_timing: id 3 | task 11230 | n_decoded = 7200, tg = 19.24 t/s, tg_3s = 17.54 t/s
16.49.825.485 I slot print_timing: id 3 | task 11230 | n_decoded = 7253, tg = 19.22 t/s, tg_3s = 17.54 t/s
16.52.880.791 I slot print_timing: id 3 | task 11230 | n_decoded = 7306, tg = 19.21 t/s, tg_3s = 17.35 t/s
16.55.937.206 I slot print_timing: id 3 | task 11230 | n_decoded = 7359, tg = 19.19 t/s, tg_3s = 17.34 t/s
16.58.941.562 I slot print_timing: id 3 | task 11230 | n_decoded = 7410, tg = 19.18 t/s, tg_3s = 16.98 t/s
17.01.973.286 I slot print_timing: id 3 | task 11230 | n_decoded = 7462, tg = 19.16 t/s, tg_3s = 17.15 t/s
17.05.005.020 I slot print_timing: id 3 | task 11230 | n_decoded = 7514, tg = 19.14 t/s, tg_3s = 17.15 t/s
17.08.036.618 I slot print_timing: id 3 | task 11230 | n_decoded = 7566, tg = 19.13 t/s, tg_3s = 17.15 t/s
17.11.078.925 I slot print_timing: id 3 | task 11230 | n_decoded = 7618, tg = 19.11 t/s, tg_3s = 17.09 t/s
17.14.108.793 I slot print_timing: id 3 | task 11230 | n_decoded = 7670, tg = 19.10 t/s, tg_3s = 17.16 t/s
17.17.134.437 I slot print_timing: id 3 | task 11230 | n_decoded = 7722, tg = 19.08 t/s, tg_3s = 17.19 t/s
17.20.147.970 I slot print_timing: id 3 | task 11230 | n_decoded = 7773, tg = 19.07 t/s, tg_3s = 16.92 t/s
17.23.200.446 I slot print_timing: id 3 | task 11230 | n_decoded = 7824, tg = 19.05 t/s, tg_3s = 16.71 t/s
17.26.210.865 I slot print_timing: id 3 | task 11230 | n_decoded = 7875, tg = 19.04 t/s, tg_3s = 16.94 t/s
17.29.261.461 I slot print_timing: id 3 | task 11230 | n_decoded = 7927, tg = 19.02 t/s, tg_3s = 17.05 t/s
17.32.314.206 I slot print_timing: id 3 | task 11230 | n_decoded = 7979, tg = 19.01 t/s, tg_3s = 17.03 t/s
17.35.329.591 I slot print_timing: id 3 | task 11230 | n_decoded = 8030, tg = 18.99 t/s, tg_3s = 16.91 t/s
17.38.382.783 I slot print_timing: id 3 | task 11230 | n_decoded = 8082, tg = 18.98 t/s, tg_3s = 17.03 t/s
17.41.418.669 I slot print_timing: id 3 | task 11230 | n_decoded = 8133, tg = 18.96 t/s, tg_3s = 16.80 t/s
17.44.437.832 I slot print_timing: id 3 | task 11230 | n_decoded = 8184, tg = 18.95 t/s, tg_3s = 16.89 t/s
17.44.912.089 I slot print_timing: id 3 | task 11230 | prompt eval time = 831.41 ms / 26 tokens ( 31.98 ms per token, 31.27 tokens per second)
17.44.912.093 I slot print_timing: id 3 | task 11230 | eval time = 432409.07 ms / 8192 tokens ( 52.78 ms per token, 18.95 tokens per second)
17.44.912.094 I slot print_timing: id 3 | task 11230 | total time = 433240.48 ms / 8218 tokens
17.44.912.095 I slot print_timing: id 3 | task 11230 | graphs reused = 19324
17.44.912.120 I slot release: id 3 | task 11230 | stop processing: n_tokens = 8261, truncated = 0
17.44.912.161 I srv update_slots: all slots are idle
17.44.913.835 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
17.44.931.554 I srv operator(): Chat format: peg-native
17.44.937.109 I slot get_availabl: id 2 | task -1 | selected slot by LRU, t_last = -1
17.44.937.111 I srv get_availabl: updating prompt cache
17.44.937.113 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
17.44.937.117 I srv update: - cache state: 7 prompts, 2845.605 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
17.44.937.119 I srv update: - prompt 0x6354c2edb8c0: 425 tokens, checkpoints: 0, 102.936 MiB
17.44.937.120 I srv update: - prompt 0x6354c2eccbe0: 125 tokens, checkpoints: 0, 30.276 MiB
17.44.937.121 I srv update: - prompt 0x6354c2c0b630: 8282 tokens, checkpoints: 0, 2005.893 MiB
17.44.937.121 I srv update: - prompt 0x6354c6628780: 650 tokens, checkpoints: 0, 157.431 MiB
17.44.937.122 I srv update: - prompt 0x6354c28b1d40: 423 tokens, checkpoints: 0, 102.452 MiB
17.44.937.122 I srv update: - prompt 0x6354c9e4fcf0: 533 tokens, checkpoints: 0, 129.093 MiB
17.44.937.123 I srv update: - prompt 0x6354c2d32b70: 1311 tokens, checkpoints: 0, 317.524 MiB
17.44.937.124 I srv get_availabl: prompt cache update took 0.01 ms
17.44.939.105 I slot launch_slot_: id 2 | task 19423 | processing task, is_child = 0
17.44.939.107 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
17.44.939.107 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
17.44.939.135 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
17.44.939.135 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
17.44.939.152 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
17.44.939.404 W srv prompt_save: - saving prompt with length 8261, total state size = 2000.807 MiB (draft: 0.000 MiB)
17.46.245.760 I srv update: - cache state: 8 prompts, 4846.412 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
17.46.245.764 I srv update: - prompt 0x6354c2edb8c0: 425 tokens, checkpoints: 0, 102.936 MiB
17.46.245.765 I srv update: - prompt 0x6354c2eccbe0: 125 tokens, checkpoints: 0, 30.276 MiB
17.46.245.765 I srv update: - prompt 0x6354c2c0b630: 8282 tokens, checkpoints: 0, 2005.893 MiB
17.46.245.766 I srv update: - prompt 0x6354c6628780: 650 tokens, checkpoints: 0, 157.431 MiB
17.46.245.766 I srv update: - prompt 0x6354c28b1d40: 423 tokens, checkpoints: 0, 102.452 MiB
17.46.245.767 I srv update: - prompt 0x6354c9e4fcf0: 533 tokens, checkpoints: 0, 129.093 MiB
17.46.245.768 I srv update: - prompt 0x6354c2d32b70: 1311 tokens, checkpoints: 0, 317.524 MiB
17.46.245.768 I srv update: - prompt 0x6354c2d09130: 8261 tokens, checkpoints: 0, 2000.807 MiB
17.46.245.769 I slot prompt_clear: id 3 | task -1 | clearing prompt with 8261 tokens
17.54.406.983 I slot print_timing: id 2 | task 19423 | n_decoded = 100, tg = 21.70 t/s, tg_3s = 21.70 t/s
17.57.445.006 I slot print_timing: id 2 | task 19423 | n_decoded = 165, tg = 21.58 t/s, tg_3s = 21.40 t/s
18.00.481.418 I slot print_timing: id 2 | task 19423 | n_decoded = 231, tg = 21.62 t/s, tg_3s = 21.74 t/s
18.03.510.852 I slot print_timing: id 2 | task 19423 | n_decoded = 296, tg = 21.59 t/s, tg_3s = 21.46 t/s
18.06.526.384 I slot print_timing: id 2 | task 19423 | n_decoded = 361, tg = 21.58 t/s, tg_3s = 21.56 t/s
18.09.531.498 I slot print_timing: id 2 | task 19423 | n_decoded = 426, tg = 21.59 t/s, tg_3s = 21.63 t/s
18.12.542.560 I slot print_timing: id 2 | task 19423 | n_decoded = 491, tg = 21.59 t/s, tg_3s = 21.59 t/s
18.15.551.348 I slot print_timing: id 2 | task 19423 | n_decoded = 556, tg = 21.59 t/s, tg_3s = 21.60 t/s
18.18.578.512 I slot print_timing: id 2 | task 19423 | n_decoded = 621, tg = 21.58 t/s, tg_3s = 21.47 t/s
18.21.609.573 I slot print_timing: id 2 | task 19423 | n_decoded = 686, tg = 21.56 t/s, tg_3s = 21.44 t/s
18.24.619.587 I slot print_timing: id 2 | task 19423 | n_decoded = 750, tg = 21.54 t/s, tg_3s = 21.26 t/s
18.27.626.881 I slot print_timing: id 2 | task 19423 | n_decoded = 813, tg = 21.49 t/s, tg_3s = 20.95 t/s
18.30.627.174 I slot print_timing: id 2 | task 19423 | n_decoded = 875, tg = 21.43 t/s, tg_3s = 20.66 t/s
18.33.667.807 I slot print_timing: id 2 | task 19423 | n_decoded = 939, tg = 21.40 t/s, tg_3s = 21.05 t/s
18.36.676.838 I slot print_timing: id 2 | task 19423 | n_decoded = 1003, tg = 21.40 t/s, tg_3s = 21.27 t/s
18.39.690.038 I slot print_timing: id 2 | task 19423 | n_decoded = 1067, tg = 21.39 t/s, tg_3s = 21.24 t/s
18.42.690.417 I slot print_timing: id 2 | task 19423 | n_decoded = 1131, tg = 21.38 t/s, tg_3s = 21.33 t/s
18.45.703.723 I slot print_timing: id 2 | task 19423 | n_decoded = 1195, tg = 21.38 t/s, tg_3s = 21.24 t/s
18.48.707.452 I slot print_timing: id 2 | task 19423 | n_decoded = 1258, tg = 21.36 t/s, tg_3s = 20.97 t/s
18.51.723.493 I slot print_timing: id 2 | task 19423 | n_decoded = 1322, tg = 21.35 t/s, tg_3s = 21.22 t/s
18.54.734.152 I slot print_timing: id 2 | task 19423 | n_decoded = 1386, tg = 21.34 t/s, tg_3s = 21.26 t/s
18.57.753.632 I slot print_timing: id 2 | task 19423 | n_decoded = 1450, tg = 21.34 t/s, tg_3s = 21.20 t/s
19.00.776.641 I slot print_timing: id 2 | task 19423 | n_decoded = 1514, tg = 21.33 t/s, tg_3s = 21.17 t/s
19.03.788.659 I slot print_timing: id 2 | task 19423 | n_decoded = 1578, tg = 21.33 t/s, tg_3s = 21.25 t/s
19.06.819.670 I slot print_timing: id 2 | task 19423 | n_decoded = 1641, tg = 21.31 t/s, tg_3s = 20.79 t/s
19.09.867.788 I slot print_timing: id 2 | task 19423 | n_decoded = 1704, tg = 21.28 t/s, tg_3s = 20.67 t/s
19.12.871.895 I slot print_timing: id 2 | task 19423 | n_decoded = 1765, tg = 21.25 t/s, tg_3s = 20.31 t/s
19.15.889.910 I slot print_timing: id 2 | task 19423 | n_decoded = 1826, tg = 21.21 t/s, tg_3s = 20.21 t/s
19.18.894.731 I slot print_timing: id 2 | task 19423 | n_decoded = 1887, tg = 21.18 t/s, tg_3s = 20.30 t/s
19.21.916.672 I slot print_timing: id 2 | task 19423 | n_decoded = 1948, tg = 21.15 t/s, tg_3s = 20.19 t/s
19.24.958.555 I slot print_timing: id 2 | task 19423 | n_decoded = 2008, tg = 21.10 t/s, tg_3s = 19.72 t/s
19.27.963.639 I slot print_timing: id 2 | task 19423 | n_decoded = 2068, tg = 21.07 t/s, tg_3s = 19.97 t/s
19.30.990.255 I slot print_timing: id 2 | task 19423 | n_decoded = 2129, tg = 21.04 t/s, tg_3s = 20.15 t/s
19.34.022.970 I slot print_timing: id 2 | task 19423 | n_decoded = 2190, tg = 21.01 t/s, tg_3s = 20.11 t/s
19.37.028.925 I slot print_timing: id 2 | task 19423 | n_decoded = 2250, tg = 20.98 t/s, tg_3s = 19.96 t/s
19.40.029.012 I slot print_timing: id 2 | task 19423 | n_decoded = 2310, tg = 20.96 t/s, tg_3s = 20.00 t/s
19.43.034.465 I slot print_timing: id 2 | task 19423 | n_decoded = 2370, tg = 20.93 t/s, tg_3s = 19.96 t/s
19.46.035.203 I slot print_timing: id 2 | task 19423 | n_decoded = 2430, tg = 20.91 t/s, tg_3s = 20.00 t/s
19.49.081.967 I slot print_timing: id 2 | task 19423 | n_decoded = 2490, tg = 20.87 t/s, tg_3s = 19.69 t/s
19.52.085.005 I slot print_timing: id 2 | task 19423 | n_decoded = 2548, tg = 20.84 t/s, tg_3s = 19.31 t/s
19.55.086.070 I slot print_timing: id 2 | task 19423 | n_decoded = 2607, tg = 20.81 t/s, tg_3s = 19.66 t/s
19.58.088.076 I slot print_timing: id 2 | task 19423 | n_decoded = 2666, tg = 20.78 t/s, tg_3s = 19.65 t/s
20.01.095.193 I slot print_timing: id 2 | task 19423 | n_decoded = 2725, tg = 20.75 t/s, tg_3s = 19.62 t/s
20.04.125.983 I slot print_timing: id 2 | task 19423 | n_decoded = 2784, tg = 20.73 t/s, tg_3s = 19.47 t/s
20.07.152.937 I slot print_timing: id 2 | task 19423 | n_decoded = 2843, tg = 20.70 t/s, tg_3s = 19.49 t/s
20.10.174.015 I slot print_timing: id 2 | task 19423 | n_decoded = 2902, tg = 20.67 t/s, tg_3s = 19.53 t/s
20.13.185.927 I slot print_timing: id 2 | task 19423 | n_decoded = 2961, tg = 20.65 t/s, tg_3s = 19.59 t/s
20.16.233.916 I slot print_timing: id 2 | task 19423 | n_decoded = 3020, tg = 20.62 t/s, tg_3s = 19.36 t/s
20.19.278.395 I slot print_timing: id 2 | task 19423 | n_decoded = 3079, tg = 20.60 t/s, tg_3s = 19.38 t/s
20.22.289.005 I slot print_timing: id 2 | task 19423 | n_decoded = 3136, tg = 20.57 t/s, tg_3s = 18.93 t/s
20.25.331.798 I slot print_timing: id 2 | task 19423 | n_decoded = 3195, tg = 20.54 t/s, tg_3s = 19.39 t/s
20.28.381.507 I slot print_timing: id 2 | task 19423 | n_decoded = 3254, tg = 20.52 t/s, tg_3s = 19.35 t/s
20.31.423.198 I slot print_timing: id 2 | task 19423 | n_decoded = 3313, tg = 20.50 t/s, tg_3s = 19.40 t/s
20.34.463.724 I slot print_timing: id 2 | task 19423 | n_decoded = 3372, tg = 20.48 t/s, tg_3s = 19.40 t/s
20.37.505.046 I slot print_timing: id 2 | task 19423 | n_decoded = 3431, tg = 20.46 t/s, tg_3s = 19.40 t/s
20.40.520.505 I slot print_timing: id 2 | task 19423 | n_decoded = 3489, tg = 20.44 t/s, tg_3s = 19.23 t/s
20.43.557.185 I slot print_timing: id 2 | task 19423 | n_decoded = 3547, tg = 20.41 t/s, tg_3s = 19.10 t/s
20.46.581.225 I slot print_timing: id 2 | task 19423 | n_decoded = 3605, tg = 20.39 t/s, tg_3s = 19.18 t/s
20.49.599.302 I slot print_timing: id 2 | task 19423 | n_decoded = 3663, tg = 20.37 t/s, tg_3s = 19.22 t/s
20.52.619.160 I slot print_timing: id 2 | task 19423 | n_decoded = 3721, tg = 20.35 t/s, tg_3s = 19.21 t/s
20.55.644.055 I slot print_timing: id 2 | task 19423 | n_decoded = 3778, tg = 20.33 t/s, tg_3s = 18.84 t/s
20.58.686.059 I slot print_timing: id 2 | task 19423 | n_decoded = 3836, tg = 20.31 t/s, tg_3s = 19.07 t/s
21.01.725.250 I slot print_timing: id 2 | task 19423 | n_decoded = 3894, tg = 20.29 t/s, tg_3s = 19.08 t/s
21.04.768.755 I slot print_timing: id 2 | task 19423 | n_decoded = 3952, tg = 20.27 t/s, tg_3s = 19.06 t/s
21.07.793.544 I slot print_timing: id 2 | task 19423 | n_decoded = 4009, tg = 20.25 t/s, tg_3s = 18.84 t/s
21.10.824.511 I slot print_timing: id 2 | task 19423 | n_decoded = 4066, tg = 20.23 t/s, tg_3s = 18.81 t/s
21.13.840.709 I slot print_timing: id 2 | task 19423 | n_decoded = 4123, tg = 20.21 t/s, tg_3s = 18.90 t/s
21.16.891.557 I slot print_timing: id 2 | task 19423 | n_decoded = 4181, tg = 20.19 t/s, tg_3s = 19.01 t/s
21.19.907.591 I slot print_timing: id 2 | task 19423 | n_decoded = 4238, tg = 20.17 t/s, tg_3s = 18.90 t/s
21.22.913.895 I slot print_timing: id 2 | task 19423 | n_decoded = 4294, tg = 20.15 t/s, tg_3s = 18.63 t/s
21.25.924.437 I slot print_timing: id 2 | task 19423 | n_decoded = 4350, tg = 20.13 t/s, tg_3s = 18.60 t/s
21.28.928.595 I slot print_timing: id 2 | task 19423 | n_decoded = 4405, tg = 20.10 t/s, tg_3s = 18.31 t/s
21.31.963.051 I slot print_timing: id 2 | task 19423 | n_decoded = 4462, tg = 20.08 t/s, tg_3s = 18.78 t/s
21.35.012.224 I slot print_timing: id 2 | task 19423 | n_decoded = 4519, tg = 20.07 t/s, tg_3s = 18.69 t/s
21.38.060.406 I slot print_timing: id 2 | task 19423 | n_decoded = 4575, tg = 20.04 t/s, tg_3s = 18.37 t/s
21.41.061.705 I slot print_timing: id 2 | task 19423 | n_decoded = 4630, tg = 20.02 t/s, tg_3s = 18.33 t/s
21.44.079.946 I slot print_timing: id 2 | task 19423 | n_decoded = 4686, tg = 20.00 t/s, tg_3s = 18.55 t/s
21.47.119.570 I slot print_timing: id 2 | task 19423 | n_decoded = 4742, tg = 19.98 t/s, tg_3s = 18.42 t/s
21.50.157.945 I slot print_timing: id 2 | task 19423 | n_decoded = 4798, tg = 19.96 t/s, tg_3s = 18.43 t/s
21.53.191.365 I slot print_timing: id 2 | task 19423 | n_decoded = 4854, tg = 19.94 t/s, tg_3s = 18.46 t/s
21.56.229.067 I slot print_timing: id 2 | task 19423 | n_decoded = 4910, tg = 19.92 t/s, tg_3s = 18.43 t/s
21.59.267.619 I slot print_timing: id 2 | task 19423 | n_decoded = 4966, tg = 19.91 t/s, tg_3s = 18.43 t/s
22.02.319.474 I slot print_timing: id 2 | task 19423 | n_decoded = 5021, tg = 19.88 t/s, tg_3s = 18.02 t/s
22.05.368.559 I slot print_timing: id 2 | task 19423 | n_decoded = 5077, tg = 19.87 t/s, tg_3s = 18.37 t/s
22.08.406.343 I slot print_timing: id 2 | task 19423 | n_decoded = 5133, tg = 19.85 t/s, tg_3s = 18.43 t/s
22.11.445.348 I slot print_timing: id 2 | task 19423 | n_decoded = 5189, tg = 19.83 t/s, tg_3s = 18.43 t/s
22.14.481.388 I slot print_timing: id 2 | task 19423 | n_decoded = 5245, tg = 19.82 t/s, tg_3s = 18.45 t/s
22.17.495.504 I slot print_timing: id 2 | task 19423 | n_decoded = 5300, tg = 19.80 t/s, tg_3s = 18.25 t/s
22.20.502.335 I slot print_timing: id 2 | task 19423 | n_decoded = 5354, tg = 19.78 t/s, tg_3s = 17.96 t/s
22.23.506.204 I slot print_timing: id 2 | task 19423 | n_decoded = 5408, tg = 19.76 t/s, tg_3s = 17.98 t/s
22.26.512.698 I slot print_timing: id 2 | task 19423 | n_decoded = 5463, tg = 19.74 t/s, tg_3s = 18.29 t/s
22.29.513.935 I slot print_timing: id 2 | task 19423 | n_decoded = 5518, tg = 19.73 t/s, tg_3s = 18.33 t/s
22.32.554.666 I slot print_timing: id 2 | task 19423 | n_decoded = 5573, tg = 19.71 t/s, tg_3s = 18.09 t/s
22.35.560.796 I slot print_timing: id 2 | task 19423 | n_decoded = 5627, tg = 19.69 t/s, tg_3s = 17.96 t/s
22.38.571.518 I slot print_timing: id 2 | task 19423 | n_decoded = 5681, tg = 19.67 t/s, tg_3s = 17.94 t/s
22.41.600.562 I slot print_timing: id 2 | task 19423 | n_decoded = 5735, tg = 19.65 t/s, tg_3s = 17.83 t/s
22.44.652.651 I slot print_timing: id 2 | task 19423 | n_decoded = 5789, tg = 19.63 t/s, tg_3s = 17.69 t/s
22.47.670.127 I slot print_timing: id 2 | task 19423 | n_decoded = 5843, tg = 19.62 t/s, tg_3s = 17.90 t/s
22.50.700.594 I slot print_timing: id 2 | task 19423 | n_decoded = 5897, tg = 19.60 t/s, tg_3s = 17.82 t/s
22.53.728.259 I slot print_timing: id 2 | task 19423 | n_decoded = 5951, tg = 19.58 t/s, tg_3s = 17.84 t/s
22.56.733.349 I slot print_timing: id 2 | task 19423 | n_decoded = 6005, tg = 19.56 t/s, tg_3s = 17.97 t/s
22.59.770.535 I slot print_timing: id 2 | task 19423 | n_decoded = 6059, tg = 19.55 t/s, tg_3s = 17.78 t/s
23.02.816.722 I slot print_timing: id 2 | task 19423 | n_decoded = 6113, tg = 19.53 t/s, tg_3s = 17.73 t/s
23.05.871.694 I slot print_timing: id 2 | task 19423 | n_decoded = 6167, tg = 19.51 t/s, tg_3s = 17.68 t/s
23.08.889.549 I slot print_timing: id 2 | task 19423 | n_decoded = 6221, tg = 19.50 t/s, tg_3s = 17.89 t/s
23.11.934.972 I slot print_timing: id 2 | task 19423 | n_decoded = 6275, tg = 19.48 t/s, tg_3s = 17.73 t/s
23.14.987.855 I slot print_timing: id 2 | task 19423 | n_decoded = 6328, tg = 19.46 t/s, tg_3s = 17.36 t/s
23.18.015.431 I slot print_timing: id 2 | task 19423 | n_decoded = 6381, tg = 19.44 t/s, tg_3s = 17.51 t/s
23.21.065.886 I slot print_timing: id 2 | task 19423 | n_decoded = 6435, tg = 19.43 t/s, tg_3s = 17.70 t/s
23.24.120.069 I slot print_timing: id 2 | task 19423 | n_decoded = 6489, tg = 19.41 t/s, tg_3s = 17.68 t/s
23.27.171.907 I slot print_timing: id 2 | task 19423 | n_decoded = 6543, tg = 19.39 t/s, tg_3s = 17.69 t/s
23.30.197.977 I slot print_timing: id 2 | task 19423 | n_decoded = 6596, tg = 19.38 t/s, tg_3s = 17.51 t/s
23.33.214.421 I slot print_timing: id 2 | task 19423 | n_decoded = 6649, tg = 19.36 t/s, tg_3s = 17.57 t/s
23.36.239.429 I slot print_timing: id 2 | task 19423 | n_decoded = 6702, tg = 19.35 t/s, tg_3s = 17.52 t/s
23.39.253.867 I slot print_timing: id 2 | task 19423 | n_decoded = 6755, tg = 19.33 t/s, tg_3s = 17.58 t/s
23.42.291.280 I slot print_timing: id 2 | task 19423 | n_decoded = 6807, tg = 19.31 t/s, tg_3s = 17.12 t/s
23.45.329.067 I slot print_timing: id 2 | task 19423 | n_decoded = 6859, tg = 19.29 t/s, tg_3s = 17.12 t/s
23.48.375.320 I slot print_timing: id 2 | task 19423 | n_decoded = 6912, tg = 19.28 t/s, tg_3s = 17.40 t/s
23.51.431.370 I slot print_timing: id 2 | task 19423 | n_decoded = 6965, tg = 19.26 t/s, tg_3s = 17.34 t/s
23.54.466.255 I slot print_timing: id 2 | task 19423 | n_decoded = 7017, tg = 19.24 t/s, tg_3s = 17.13 t/s
23.57.514.230 I slot print_timing: id 2 | task 19423 | n_decoded = 7070, tg = 19.23 t/s, tg_3s = 17.39 t/s
24.00.547.270 I slot print_timing: id 2 | task 19423 | n_decoded = 7123, tg = 19.21 t/s, tg_3s = 17.47 t/s
24.03.567.405 I slot print_timing: id 2 | task 19423 | n_decoded = 7176, tg = 19.20 t/s, tg_3s = 17.55 t/s
24.06.585.764 I slot print_timing: id 2 | task 19423 | n_decoded = 7229, tg = 19.19 t/s, tg_3s = 17.56 t/s
24.09.605.035 I slot print_timing: id 2 | task 19423 | n_decoded = 7282, tg = 19.17 t/s, tg_3s = 17.55 t/s
24.12.661.246 I slot print_timing: id 2 | task 19423 | n_decoded = 7335, tg = 19.16 t/s, tg_3s = 17.34 t/s
24.15.700.857 I slot print_timing: id 2 | task 19423 | n_decoded = 7387, tg = 19.14 t/s, tg_3s = 17.11 t/s
24.18.737.010 I slot print_timing: id 2 | task 19423 | n_decoded = 7439, tg = 19.13 t/s, tg_3s = 17.13 t/s
24.21.766.246 I slot print_timing: id 2 | task 19423 | n_decoded = 7491, tg = 19.11 t/s, tg_3s = 17.17 t/s
24.24.795.201 I slot print_timing: id 2 | task 19423 | n_decoded = 7543, tg = 19.10 t/s, tg_3s = 17.17 t/s
24.27.835.051 I slot print_timing: id 2 | task 19423 | n_decoded = 7595, tg = 19.08 t/s, tg_3s = 17.11 t/s
24.30.857.193 I slot print_timing: id 2 | task 19423 | n_decoded = 7647, tg = 19.07 t/s, tg_3s = 17.21 t/s
24.33.900.391 I slot print_timing: id 2 | task 19423 | n_decoded = 7699, tg = 19.05 t/s, tg_3s = 17.09 t/s
24.36.951.307 I slot print_timing: id 2 | task 19423 | n_decoded = 7751, tg = 19.04 t/s, tg_3s = 17.04 t/s
24.40.002.803 I slot print_timing: id 2 | task 19423 | n_decoded = 7803, tg = 19.02 t/s, tg_3s = 17.04 t/s
24.43.055.005 I slot print_timing: id 2 | task 19423 | n_decoded = 7855, tg = 19.01 t/s, tg_3s = 17.04 t/s
24.46.101.433 I slot print_timing: id 2 | task 19423 | n_decoded = 7907, tg = 18.99 t/s, tg_3s = 17.07 t/s
24.49.148.061 I slot print_timing: id 2 | task 19423 | n_decoded = 7959, tg = 18.98 t/s, tg_3s = 17.07 t/s
24.52.150.272 I slot print_timing: id 2 | task 19423 | n_decoded = 8010, tg = 18.97 t/s, tg_3s = 16.99 t/s
24.55.206.980 I slot print_timing: id 2 | task 19423 | n_decoded = 8062, tg = 18.95 t/s, tg_3s = 17.01 t/s
24.58.230.050 I slot print_timing: id 2 | task 19423 | n_decoded = 8113, tg = 18.94 t/s, tg_3s = 16.87 t/s
25.01.267.676 I slot print_timing: id 2 | task 19423 | n_decoded = 8164, tg = 18.92 t/s, tg_3s = 16.79 t/s
25.02.922.007 I slot print_timing: id 2 | task 19423 | prompt eval time = 3551.57 ms / 99 tokens ( 35.87 ms per token, 27.88 tokens per second)
25.02.922.012 I slot print_timing: id 2 | task 19423 | eval time = 433123.46 ms / 8192 tokens ( 52.87 ms per token, 18.91 tokens per second)
25.02.922.012 I slot print_timing: id 2 | task 19423 | total time = 436675.02 ms / 8291 tokens
25.02.922.013 I slot print_timing: id 2 | task 19423 | graphs reused = 27482
25.02.922.042 I slot release: id 2 | task 19423 | stop processing: n_tokens = 8290, truncated = 0
25.02.922.085 I srv update_slots: all slots are idle
25.02.923.833 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
25.02.941.793 I srv operator(): Chat format: peg-native
25.02.947.347 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.508 (> 0.100 thold), f_keep = 0.008
25.02.947.349 I srv get_availabl: updating prompt cache
25.02.947.591 W srv prompt_save: - saving prompt with length 8290, total state size = 2007.831 MiB (draft: 0.000 MiB)
25.04.153.344 I srv load: - looking for better prompt, base f_keep = 0.008, sim = 0.508
25.04.153.351 I srv update: - cache state: 9 prompts, 6854.243 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
25.04.153.352 I srv update: - prompt 0x6354c2edb8c0: 425 tokens, checkpoints: 0, 102.936 MiB
25.04.153.353 I srv update: - prompt 0x6354c2eccbe0: 125 tokens, checkpoints: 0, 30.276 MiB
25.04.153.353 I srv update: - prompt 0x6354c2c0b630: 8282 tokens, checkpoints: 0, 2005.893 MiB
25.04.153.354 I srv update: - prompt 0x6354c6628780: 650 tokens, checkpoints: 0, 157.431 MiB
25.04.153.354 I srv update: - prompt 0x6354c28b1d40: 423 tokens, checkpoints: 0, 102.452 MiB
25.04.153.355 I srv update: - prompt 0x6354c9e4fcf0: 533 tokens, checkpoints: 0, 129.093 MiB
25.04.153.355 I srv update: - prompt 0x6354c2d32b70: 1311 tokens, checkpoints: 0, 317.524 MiB
25.04.153.356 I srv update: - prompt 0x6354c2d09130: 8261 tokens, checkpoints: 0, 2000.807 MiB
25.04.153.357 I srv update: - prompt 0x6354c2ebdcd0: 8290 tokens, checkpoints: 0, 2007.831 MiB
25.04.153.359 I srv get_availabl: prompt cache update took 1206.01 ms
25.04.154.961 I slot launch_slot_: id 2 | task 27617 | processing task, is_child = 0
25.04.154.963 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
25.04.154.964 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
25.04.154.992 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
25.04.154.992 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
25.04.155.007 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
25.04.155.008 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
25.11.019.298 I slot print_timing: id 2 | task 27617 | n_decoded = 100, tg = 21.54 t/s, tg_3s = 21.54 t/s
25.14.056.945 I slot print_timing: id 2 | task 27617 | n_decoded = 165, tg = 21.48 t/s, tg_3s = 21.40 t/s
25.17.072.874 I slot print_timing: id 2 | task 27617 | n_decoded = 230, tg = 21.50 t/s, tg_3s = 21.55 t/s
25.20.080.557 I slot print_timing: id 2 | task 27617 | n_decoded = 294, tg = 21.45 t/s, tg_3s = 21.28 t/s
25.23.118.179 I slot print_timing: id 2 | task 27617 | n_decoded = 360, tg = 21.50 t/s, tg_3s = 21.73 t/s
25.26.140.700 I slot print_timing: id 2 | task 27617 | n_decoded = 425, tg = 21.50 t/s, tg_3s = 21.51 t/s
25.29.162.401 I slot print_timing: id 2 | task 27617 | n_decoded = 490, tg = 21.50 t/s, tg_3s = 21.51 t/s
25.31.101.525 I slot print_timing: id 2 | task 27617 | prompt eval time = 2221.51 ms / 61 tokens ( 36.42 ms per token, 27.46 tokens per second)
25.31.101.529 I slot print_timing: id 2 | task 27617 | eval time = 24724.97 ms / 532 tokens ( 46.48 ms per token, 21.52 tokens per second)
25.31.101.530 I slot print_timing: id 2 | task 27617 | total time = 26946.48 ms / 593 tokens
25.31.101.531 I slot print_timing: id 2 | task 27617 | graphs reused = 28010
25.31.101.563 I slot release: id 2 | task 27617 | stop processing: n_tokens = 655, truncated = 0
25.31.101.578 I srv update_slots: all slots are idle
25.31.102.563 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
25.31.121.017 I srv operator(): Chat format: peg-native
25.31.126.640 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.529 (> 0.100 thold), f_keep = 0.096
25.31.126.642 I srv get_availabl: updating prompt cache
25.31.126.695 W srv prompt_save: - saving prompt with length 655, total state size = 158.642 MiB (draft: 0.000 MiB)
25.31.234.695 I srv load: - looking for better prompt, base f_keep = 0.096, sim = 0.529
25.31.234.703 I srv update: - cache state: 10 prompts, 7012.885 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
25.31.234.703 I srv update: - prompt 0x6354c2edb8c0: 425 tokens, checkpoints: 0, 102.936 MiB
25.31.234.704 I srv update: - prompt 0x6354c2eccbe0: 125 tokens, checkpoints: 0, 30.276 MiB
25.31.234.705 I srv update: - prompt 0x6354c2c0b630: 8282 tokens, checkpoints: 0, 2005.893 MiB
25.31.234.705 I srv update: - prompt 0x6354c6628780: 650 tokens, checkpoints: 0, 157.431 MiB
25.31.234.706 I srv update: - prompt 0x6354c28b1d40: 423 tokens, checkpoints: 0, 102.452 MiB
25.31.234.707 I srv update: - prompt 0x6354c9e4fcf0: 533 tokens, checkpoints: 0, 129.093 MiB
25.31.234.708 I srv update: - prompt 0x6354c2d32b70: 1311 tokens, checkpoints: 0, 317.524 MiB
25.31.234.708 I srv update: - prompt 0x6354c2d09130: 8261 tokens, checkpoints: 0, 2000.807 MiB
25.31.234.709 I srv update: - prompt 0x6354c2ebdcd0: 8290 tokens, checkpoints: 0, 2007.831 MiB
25.31.234.712 I srv update: - prompt 0x6354c2cffe70: 655 tokens, checkpoints: 0, 158.642 MiB
25.31.234.713 I srv get_availabl: prompt cache update took 108.07 ms
25.31.236.359 I slot launch_slot_: id 2 | task 28150 | processing task, is_child = 0
25.31.236.361 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
25.31.236.362 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
25.31.236.376 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
25.31.236.376 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
25.31.236.392 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
25.31.236.392 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
25.37.751.067 I slot print_timing: id 2 | task 28150 | n_decoded = 100, tg = 21.89 t/s, tg_3s = 21.89 t/s
25.40.756.399 I slot print_timing: id 2 | task 28150 | n_decoded = 165, tg = 21.78 t/s, tg_3s = 21.63 t/s
25.43.801.364 I slot print_timing: id 2 | task 28150 | n_decoded = 231, tg = 21.75 t/s, tg_3s = 21.68 t/s
25.46.803.861 I slot print_timing: id 2 | task 28150 | n_decoded = 296, tg = 21.73 t/s, tg_3s = 21.65 t/s
25.49.830.474 I slot print_timing: id 2 | task 28150 | n_decoded = 361, tg = 21.68 t/s, tg_3s = 21.48 t/s
25.52.854.511 I slot print_timing: id 2 | task 28150 | n_decoded = 426, tg = 21.65 t/s, tg_3s = 21.49 t/s
25.55.891.422 I slot print_timing: id 2 | task 28150 | n_decoded = 491, tg = 21.62 t/s, tg_3s = 21.40 t/s
25.58.933.985 I slot print_timing: id 2 | task 28150 | n_decoded = 555, tg = 21.55 t/s, tg_3s = 21.03 t/s
26.01.945.599 I slot print_timing: id 2 | task 28150 | n_decoded = 619, tg = 21.52 t/s, tg_3s = 21.25 t/s
26.04.962.664 I slot print_timing: id 2 | task 28150 | n_decoded = 683, tg = 21.49 t/s, tg_3s = 21.21 t/s
26.05.056.751 I slot print_timing: id 2 | task 28150 | prompt eval time = 1945.74 ms / 56 tokens ( 34.75 ms per token, 28.78 tokens per second)
26.05.056.755 I slot print_timing: id 2 | task 28150 | eval time = 31874.57 ms / 685 tokens ( 46.53 ms per token, 21.49 tokens per second)
26.05.056.756 I slot print_timing: id 2 | task 28150 | total time = 33820.32 ms / 741 tokens
26.05.056.756 I slot print_timing: id 2 | task 28150 | graphs reused = 28690
26.05.056.792 I slot release: id 2 | task 28150 | stop processing: n_tokens = 803, truncated = 0
26.05.056.812 I srv update_slots: all slots are idle
26.05.057.706 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
26.05.076.750 I srv operator(): Chat format: peg-native
26.05.082.901 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.488 (> 0.100 thold), f_keep = 0.078
26.05.082.903 I srv get_availabl: updating prompt cache
26.05.082.981 W srv prompt_save: - saving prompt with length 803, total state size = 194.487 MiB (draft: 0.000 MiB)
26.05.216.979 I srv load: - looking for better prompt, base f_keep = 0.078, sim = 0.488
26.05.216.988 I srv update: - cache state: 11 prompts, 7207.372 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
26.05.216.990 I srv update: - prompt 0x6354c2edb8c0: 425 tokens, checkpoints: 0, 102.936 MiB
26.05.216.993 I srv update: - prompt 0x6354c2eccbe0: 125 tokens, checkpoints: 0, 30.276 MiB
26.05.216.994 I srv update: - prompt 0x6354c2c0b630: 8282 tokens, checkpoints: 0, 2005.893 MiB
26.05.216.994 I srv update: - prompt 0x6354c6628780: 650 tokens, checkpoints: 0, 157.431 MiB
26.05.216.995 I srv update: - prompt 0x6354c28b1d40: 423 tokens, checkpoints: 0, 102.452 MiB
26.05.216.996 I srv update: - prompt 0x6354c9e4fcf0: 533 tokens, checkpoints: 0, 129.093 MiB
26.05.216.999 I srv update: - prompt 0x6354c2d32b70: 1311 tokens, checkpoints: 0, 317.524 MiB
26.05.217.000 I srv update: - prompt 0x6354c2d09130: 8261 tokens, checkpoints: 0, 2000.807 MiB
26.05.217.001 I srv update: - prompt 0x6354c2ebdcd0: 8290 tokens, checkpoints: 0, 2007.831 MiB
26.05.217.001 I srv update: - prompt 0x6354c2cffe70: 655 tokens, checkpoints: 0, 158.642 MiB
26.05.217.002 I srv update: - prompt 0x6354ca9ab8c0: 803 tokens, checkpoints: 0, 194.487 MiB
26.05.217.005 I srv get_availabl: prompt cache update took 134.10 ms
26.05.218.785 I slot launch_slot_: id 2 | task 28836 | processing task, is_child = 0
26.05.218.788 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
26.05.218.788 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
26.05.218.805 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
26.05.218.805 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
26.05.218.823 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
26.05.218.825 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
26.11.550.517 I slot print_timing: id 2 | task 28836 | n_decoded = 100, tg = 21.89 t/s, tg_3s = 21.89 t/s
26.14.595.694 I slot print_timing: id 2 | task 28836 | n_decoded = 165, tg = 21.67 t/s, tg_3s = 21.35 t/s
26.17.639.069 I slot print_timing: id 2 | task 28836 | n_decoded = 231, tg = 21.68 t/s, tg_3s = 21.69 t/s
26.20.644.673 I slot print_timing: id 2 | task 28836 | n_decoded = 296, tg = 21.66 t/s, tg_3s = 21.63 t/s
26.23.650.708 I slot print_timing: id 2 | task 28836 | n_decoded = 361, tg = 21.66 t/s, tg_3s = 21.62 t/s
26.26.694.090 I slot print_timing: id 2 | task 28836 | n_decoded = 426, tg = 21.61 t/s, tg_3s = 21.36 t/s
26.29.739.531 I slot print_timing: id 2 | task 28836 | n_decoded = 491, tg = 21.58 t/s, tg_3s = 21.34 t/s
26.32.743.200 I slot print_timing: id 2 | task 28836 | n_decoded = 556, tg = 21.58 t/s, tg_3s = 21.64 t/s
26.35.764.851 I slot print_timing: id 2 | task 28836 | n_decoded = 621, tg = 21.58 t/s, tg_3s = 21.51 t/s
26.38.794.469 I slot print_timing: id 2 | task 28836 | n_decoded = 685, tg = 21.53 t/s, tg_3s = 21.12 t/s
26.41.826.892 I slot print_timing: id 2 | task 28836 | n_decoded = 749, tg = 21.50 t/s, tg_3s = 21.11 t/s
26.44.844.026 I slot print_timing: id 2 | task 28836 | n_decoded = 813, tg = 21.47 t/s, tg_3s = 21.21 t/s
26.47.870.659 I slot print_timing: id 2 | task 28836 | n_decoded = 876, tg = 21.42 t/s, tg_3s = 20.82 t/s
26.50.883.073 I slot print_timing: id 2 | task 28836 | n_decoded = 939, tg = 21.39 t/s, tg_3s = 20.91 t/s
26.53.901.567 I slot print_timing: id 2 | task 28836 | n_decoded = 1003, tg = 21.38 t/s, tg_3s = 21.20 t/s
26.56.939.471 I slot print_timing: id 2 | task 28836 | n_decoded = 1066, tg = 21.34 t/s, tg_3s = 20.74 t/s
26.59.940.120 I slot print_timing: id 2 | task 28836 | n_decoded = 1128, tg = 21.30 t/s, tg_3s = 20.66 t/s
27.02.955.645 I slot print_timing: id 2 | task 28836 | n_decoded = 1192, tg = 21.30 t/s, tg_3s = 21.22 t/s
27.05.966.745 I slot print_timing: id 2 | task 28836 | n_decoded = 1256, tg = 21.29 t/s, tg_3s = 21.25 t/s
27.08.988.655 I slot print_timing: id 2 | task 28836 | n_decoded = 1320, tg = 21.29 t/s, tg_3s = 21.18 t/s
27.12.011.812 I slot print_timing: id 2 | task 28836 | n_decoded = 1384, tg = 21.28 t/s, tg_3s = 21.17 t/s
27.15.037.694 I slot print_timing: id 2 | task 28836 | n_decoded = 1448, tg = 21.28 t/s, tg_3s = 21.15 t/s
27.18.059.054 I slot print_timing: id 2 | task 28836 | n_decoded = 1510, tg = 21.24 t/s, tg_3s = 20.52 t/s
27.21.097.050 I slot print_timing: id 2 | task 28836 | n_decoded = 1572, tg = 21.21 t/s, tg_3s = 20.41 t/s
27.24.123.558 I slot print_timing: id 2 | task 28836 | n_decoded = 1636, tg = 21.21 t/s, tg_3s = 21.15 t/s
27.27.133.820 I slot print_timing: id 2 | task 28836 | n_decoded = 1698, tg = 21.18 t/s, tg_3s = 20.60 t/s
27.30.136.975 I slot print_timing: id 2 | task 28836 | n_decoded = 1757, tg = 21.13 t/s, tg_3s = 19.65 t/s
27.33.158.119 I slot print_timing: id 2 | task 28836 | n_decoded = 1818, tg = 21.10 t/s, tg_3s = 20.19 t/s
27.36.165.050 I slot print_timing: id 2 | task 28836 | n_decoded = 1879, tg = 21.07 t/s, tg_3s = 20.29 t/s
27.39.179.386 I slot print_timing: id 2 | task 28836 | n_decoded = 1939, tg = 21.03 t/s, tg_3s = 19.90 t/s
27.42.228.249 I slot print_timing: id 2 | task 28836 | n_decoded = 2000, tg = 21.00 t/s, tg_3s = 20.01 t/s
27.45.263.619 I slot print_timing: id 2 | task 28836 | n_decoded = 2061, tg = 20.97 t/s, tg_3s = 20.10 t/s
27.48.303.136 I slot print_timing: id 2 | task 28836 | n_decoded = 2122, tg = 20.94 t/s, tg_3s = 20.07 t/s
27.51.348.619 I slot print_timing: id 2 | task 28836 | n_decoded = 2183, tg = 20.92 t/s, tg_3s = 20.03 t/s
27.54.396.726 I slot print_timing: id 2 | task 28836 | n_decoded = 2244, tg = 20.89 t/s, tg_3s = 20.01 t/s
27.57.402.816 I slot print_timing: id 2 | task 28836 | n_decoded = 2304, tg = 20.87 t/s, tg_3s = 19.96 t/s
28.00.403.234 I slot print_timing: id 2 | task 28836 | n_decoded = 2364, tg = 20.84 t/s, tg_3s = 20.00 t/s
28.03.440.613 I slot print_timing: id 2 | task 28836 | n_decoded = 2424, tg = 20.81 t/s, tg_3s = 19.75 t/s
28.06.451.257 I slot print_timing: id 2 | task 28836 | n_decoded = 2482, tg = 20.78 t/s, tg_3s = 19.26 t/s
28.09.459.285 I slot print_timing: id 2 | task 28836 | n_decoded = 2541, tg = 20.75 t/s, tg_3s = 19.61 t/s
28.12.468.129 I slot print_timing: id 2 | task 28836 | n_decoded = 2600, tg = 20.72 t/s, tg_3s = 19.61 t/s
28.15.519.239 I slot print_timing: id 2 | task 28836 | n_decoded = 2659, tg = 20.69 t/s, tg_3s = 19.34 t/s
28.18.535.408 I slot print_timing: id 2 | task 28836 | n_decoded = 2717, tg = 20.65 t/s, tg_3s = 19.23 t/s
28.21.543.299 I slot print_timing: id 2 | task 28836 | n_decoded = 2775, tg = 20.62 t/s, tg_3s = 19.28 t/s
28.24.557.328 I slot print_timing: id 2 | task 28836 | n_decoded = 2834, tg = 20.60 t/s, tg_3s = 19.58 t/s
28.27.605.342 I slot print_timing: id 2 | task 28836 | n_decoded = 2893, tg = 20.57 t/s, tg_3s = 19.36 t/s
28.30.632.411 I slot print_timing: id 2 | task 28836 | n_decoded = 2952, tg = 20.55 t/s, tg_3s = 19.49 t/s
28.33.676.002 I slot print_timing: id 2 | task 28836 | n_decoded = 3011, tg = 20.53 t/s, tg_3s = 19.38 t/s
28.36.720.826 I slot print_timing: id 2 | task 28836 | n_decoded = 3070, tg = 20.50 t/s, tg_3s = 19.38 t/s
28.39.757.546 I slot print_timing: id 2 | task 28836 | n_decoded = 3129, tg = 20.48 t/s, tg_3s = 19.43 t/s
28.42.806.523 I slot print_timing: id 2 | task 28836 | n_decoded = 3188, tg = 20.46 t/s, tg_3s = 19.35 t/s
28.45.852.802 I slot print_timing: id 2 | task 28836 | n_decoded = 3245, tg = 20.43 t/s, tg_3s = 18.71 t/s
28.48.864.333 I slot print_timing: id 2 | task 28836 | n_decoded = 3302, tg = 20.40 t/s, tg_3s = 18.93 t/s
28.51.907.313 I slot print_timing: id 2 | task 28836 | n_decoded = 3361, tg = 20.38 t/s, tg_3s = 19.39 t/s
28.54.925.717 I slot print_timing: id 2 | task 28836 | n_decoded = 3419, tg = 20.36 t/s, tg_3s = 19.22 t/s
28.57.941.609 I slot print_timing: id 2 | task 28836 | n_decoded = 3476, tg = 20.33 t/s, tg_3s = 18.90 t/s
29.00.952.512 I slot print_timing: id 2 | task 28836 | n_decoded = 3533, tg = 20.31 t/s, tg_3s = 18.93 t/s
29.03.968.836 I slot print_timing: id 2 | task 28836 | n_decoded = 3590, tg = 20.28 t/s, tg_3s = 18.90 t/s
29.07.001.243 I slot print_timing: id 2 | task 28836 | n_decoded = 3648, tg = 20.26 t/s, tg_3s = 19.13 t/s
29.10.024.351 I slot print_timing: id 2 | task 28836 | n_decoded = 3705, tg = 20.24 t/s, tg_3s = 18.85 t/s
29.13.064.864 I slot print_timing: id 2 | task 28836 | n_decoded = 3762, tg = 20.22 t/s, tg_3s = 18.75 t/s
29.16.100.607 I slot print_timing: id 2 | task 28836 | n_decoded = 3819, tg = 20.19 t/s, tg_3s = 18.78 t/s
29.19.118.286 I slot print_timing: id 2 | task 28836 | n_decoded = 3876, tg = 20.17 t/s, tg_3s = 18.89 t/s
29.22.129.812 I slot print_timing: id 2 | task 28836 | n_decoded = 3933, tg = 20.15 t/s, tg_3s = 18.93 t/s
29.25.167.929 I slot print_timing: id 2 | task 28836 | n_decoded = 3990, tg = 20.13 t/s, tg_3s = 18.76 t/s
29.28.186.800 I slot print_timing: id 2 | task 28836 | n_decoded = 4047, tg = 20.11 t/s, tg_3s = 18.88 t/s
29.31.205.785 I slot print_timing: id 2 | task 28836 | n_decoded = 4104, tg = 20.10 t/s, tg_3s = 18.88 t/s
29.34.212.696 I slot print_timing: id 2 | task 28836 | n_decoded = 4161, tg = 20.08 t/s, tg_3s = 18.96 t/s
29.37.224.609 I slot print_timing: id 2 | task 28836 | n_decoded = 4218, tg = 20.06 t/s, tg_3s = 18.92 t/s
29.40.262.242 I slot print_timing: id 2 | task 28836 | n_decoded = 4275, tg = 20.04 t/s, tg_3s = 18.76 t/s
29.43.295.136 I slot print_timing: id 2 | task 28836 | n_decoded = 4332, tg = 20.03 t/s, tg_3s = 18.79 t/s
29.46.335.380 I slot print_timing: id 2 | task 28836 | n_decoded = 4389, tg = 20.01 t/s, tg_3s = 18.75 t/s
29.49.370.628 I slot print_timing: id 2 | task 28836 | n_decoded = 4446, tg = 19.99 t/s, tg_3s = 18.78 t/s
29.52.382.364 I slot print_timing: id 2 | task 28836 | n_decoded = 4502, tg = 19.97 t/s, tg_3s = 18.59 t/s
29.55.403.004 I slot print_timing: id 2 | task 28836 | n_decoded = 4558, tg = 19.95 t/s, tg_3s = 18.54 t/s
29.58.429.604 I slot print_timing: id 2 | task 28836 | n_decoded = 4614, tg = 19.94 t/s, tg_3s = 18.50 t/s
30.01.481.695 I slot print_timing: id 2 | task 28836 | n_decoded = 4670, tg = 19.91 t/s, tg_3s = 18.35 t/s
30.04.511.679 I slot print_timing: id 2 | task 28836 | n_decoded = 4725, tg = 19.89 t/s, tg_3s = 18.15 t/s
30.07.542.040 I slot print_timing: id 2 | task 28836 | n_decoded = 4780, tg = 19.87 t/s, tg_3s = 18.15 t/s
30.10.583.947 I slot print_timing: id 2 | task 28836 | n_decoded = 4836, tg = 19.85 t/s, tg_3s = 18.41 t/s
30.13.589.754 I slot print_timing: id 2 | task 28836 | n_decoded = 4891, tg = 19.83 t/s, tg_3s = 18.30 t/s
30.16.627.116 I slot print_timing: id 2 | task 28836 | n_decoded = 4947, tg = 19.82 t/s, tg_3s = 18.44 t/s
30.19.669.399 I slot print_timing: id 2 | task 28836 | n_decoded = 5003, tg = 19.80 t/s, tg_3s = 18.41 t/s
30.22.705.238 I slot print_timing: id 2 | task 28836 | n_decoded = 5059, tg = 19.78 t/s, tg_3s = 18.45 t/s
30.25.743.539 I slot print_timing: id 2 | task 28836 | n_decoded = 5115, tg = 19.77 t/s, tg_3s = 18.43 t/s
30.28.785.094 I slot print_timing: id 2 | task 28836 | n_decoded = 5171, tg = 19.75 t/s, tg_3s = 18.41 t/s
30.31.836.572 I slot print_timing: id 2 | task 28836 | n_decoded = 5227, tg = 19.74 t/s, tg_3s = 18.35 t/s
30.34.867.005 I slot print_timing: id 2 | task 28836 | n_decoded = 5282, tg = 19.72 t/s, tg_3s = 18.15 t/s
30.37.891.425 I slot print_timing: id 2 | task 28836 | n_decoded = 5337, tg = 19.70 t/s, tg_3s = 18.19 t/s
30.40.938.604 I slot print_timing: id 2 | task 28836 | n_decoded = 5392, tg = 19.68 t/s, tg_3s = 18.05 t/s
30.43.975.565 I slot print_timing: id 2 | task 28836 | n_decoded = 5447, tg = 19.66 t/s, tg_3s = 18.11 t/s
30.46.994.942 I slot print_timing: id 2 | task 28836 | n_decoded = 5502, tg = 19.65 t/s, tg_3s = 18.22 t/s
30.50.048.415 I slot print_timing: id 2 | task 28836 | n_decoded = 5557, tg = 19.63 t/s, tg_3s = 18.01 t/s
30.53.095.627 I slot print_timing: id 2 | task 28836 | n_decoded = 5611, tg = 19.61 t/s, tg_3s = 17.72 t/s
30.56.141.208 I slot print_timing: id 2 | task 28836 | n_decoded = 5666, tg = 19.59 t/s, tg_3s = 18.06 t/s
30.59.181.989 I slot print_timing: id 2 | task 28836 | n_decoded = 5720, tg = 19.58 t/s, tg_3s = 17.76 t/s
31.02.190.791 I slot print_timing: id 2 | task 28836 | n_decoded = 5774, tg = 19.56 t/s, tg_3s = 17.95 t/s
31.05.228.885 I slot print_timing: id 2 | task 28836 | n_decoded = 5828, tg = 19.54 t/s, tg_3s = 17.77 t/s
31.08.256.308 I slot print_timing: id 2 | task 28836 | n_decoded = 5882, tg = 19.52 t/s, tg_3s = 17.84 t/s
31.11.286.648 I slot print_timing: id 2 | task 28836 | n_decoded = 5936, tg = 19.51 t/s, tg_3s = 17.82 t/s
31.14.329.406 I slot print_timing: id 2 | task 28836 | n_decoded = 5990, tg = 19.49 t/s, tg_3s = 17.75 t/s
31.17.373.539 I slot print_timing: id 2 | task 28836 | n_decoded = 6044, tg = 19.47 t/s, tg_3s = 17.74 t/s
31.20.412.445 I slot print_timing: id 2 | task 28836 | n_decoded = 6098, tg = 19.46 t/s, tg_3s = 17.77 t/s
31.23.441.772 I slot print_timing: id 2 | task 28836 | n_decoded = 6152, tg = 19.44 t/s, tg_3s = 17.83 t/s
31.26.470.980 I slot print_timing: id 2 | task 28836 | n_decoded = 6206, tg = 19.42 t/s, tg_3s = 17.83 t/s
31.29.489.222 I slot print_timing: id 2 | task 28836 | n_decoded = 6260, tg = 19.41 t/s, tg_3s = 17.89 t/s
31.32.542.707 I slot print_timing: id 2 | task 28836 | n_decoded = 6314, tg = 19.39 t/s, tg_3s = 17.68 t/s
31.35.595.397 I slot print_timing: id 2 | task 28836 | n_decoded = 6368, tg = 19.38 t/s, tg_3s = 17.69 t/s
31.38.648.505 I slot print_timing: id 2 | task 28836 | n_decoded = 6422, tg = 19.36 t/s, tg_3s = 17.69 t/s
31.41.697.747 I slot print_timing: id 2 | task 28836 | n_decoded = 6476, tg = 19.35 t/s, tg_3s = 17.71 t/s
31.44.762.142 I slot print_timing: id 2 | task 28836 | n_decoded = 6530, tg = 19.33 t/s, tg_3s = 17.62 t/s
31.47.782.881 I slot print_timing: id 2 | task 28836 | n_decoded = 6583, tg = 19.32 t/s, tg_3s = 17.55 t/s
31.50.821.803 I slot print_timing: id 2 | task 28836 | n_decoded = 6636, tg = 19.30 t/s, tg_3s = 17.44 t/s
31.53.849.867 I slot print_timing: id 2 | task 28836 | n_decoded = 6689, tg = 19.28 t/s, tg_3s = 17.50 t/s
31.56.885.192 I slot print_timing: id 2 | task 28836 | n_decoded = 6742, tg = 19.27 t/s, tg_3s = 17.46 t/s
31.59.921.049 I slot print_timing: id 2 | task 28836 | n_decoded = 6795, tg = 19.25 t/s, tg_3s = 17.46 t/s
32.02.943.977 I slot print_timing: id 2 | task 28836 | n_decoded = 6848, tg = 19.24 t/s, tg_3s = 17.53 t/s
32.05.966.182 I slot print_timing: id 2 | task 28836 | n_decoded = 6901, tg = 19.22 t/s, tg_3s = 17.54 t/s
32.09.013.735 I slot print_timing: id 2 | task 28836 | n_decoded = 6954, tg = 19.21 t/s, tg_3s = 17.39 t/s
32.12.021.426 I slot print_timing: id 2 | task 28836 | n_decoded = 7006, tg = 19.19 t/s, tg_3s = 17.29 t/s
32.15.057.041 I slot print_timing: id 2 | task 28836 | n_decoded = 7059, tg = 19.18 t/s, tg_3s = 17.46 t/s
32.18.103.036 I slot print_timing: id 2 | task 28836 | n_decoded = 7112, tg = 19.16 t/s, tg_3s = 17.40 t/s
32.21.152.504 I slot print_timing: id 2 | task 28836 | n_decoded = 7165, tg = 19.15 t/s, tg_3s = 17.38 t/s
32.24.178.339 I slot print_timing: id 2 | task 28836 | n_decoded = 7218, tg = 19.14 t/s, tg_3s = 17.52 t/s
32.27.205.318 I slot print_timing: id 2 | task 28836 | n_decoded = 7271, tg = 19.12 t/s, tg_3s = 17.51 t/s
32.30.230.618 I slot print_timing: id 2 | task 28836 | n_decoded = 7323, tg = 19.11 t/s, tg_3s = 17.19 t/s
32.33.279.193 I slot print_timing: id 2 | task 28836 | n_decoded = 7375, tg = 19.09 t/s, tg_3s = 17.06 t/s
32.36.335.209 I slot print_timing: id 2 | task 28836 | n_decoded = 7427, tg = 19.08 t/s, tg_3s = 17.02 t/s
32.39.369.282 I slot print_timing: id 2 | task 28836 | n_decoded = 7479, tg = 19.06 t/s, tg_3s = 17.14 t/s
32.42.403.616 I slot print_timing: id 2 | task 28836 | n_decoded = 7531, tg = 19.05 t/s, tg_3s = 17.14 t/s
32.45.446.070 I slot print_timing: id 2 | task 28836 | n_decoded = 7583, tg = 19.03 t/s, tg_3s = 17.09 t/s
32.48.479.763 I slot print_timing: id 2 | task 28836 | n_decoded = 7635, tg = 19.02 t/s, tg_3s = 17.14 t/s
32.51.501.531 I slot print_timing: id 2 | task 28836 | n_decoded = 7687, tg = 19.00 t/s, tg_3s = 17.21 t/s
32.54.522.244 I slot print_timing: id 2 | task 28836 | n_decoded = 7739, tg = 18.99 t/s, tg_3s = 17.21 t/s
32.57.541.558 I slot print_timing: id 2 | task 28836 | n_decoded = 7791, tg = 18.98 t/s, tg_3s = 17.22 t/s
33.00.590.429 I slot print_timing: id 2 | task 28836 | n_decoded = 7843, tg = 18.96 t/s, tg_3s = 17.06 t/s
33.03.609.556 I slot print_timing: id 2 | task 28836 | n_decoded = 7894, tg = 18.95 t/s, tg_3s = 16.89 t/s
33.06.655.319 I slot print_timing: id 2 | task 28836 | n_decoded = 7946, tg = 18.93 t/s, tg_3s = 17.07 t/s
33.09.664.209 I slot print_timing: id 2 | task 28836 | n_decoded = 7997, tg = 18.92 t/s, tg_3s = 16.95 t/s
33.12.669.540 I slot print_timing: id 2 | task 28836 | n_decoded = 8048, tg = 18.91 t/s, tg_3s = 16.97 t/s
33.15.689.563 I slot print_timing: id 2 | task 28836 | n_decoded = 8099, tg = 18.89 t/s, tg_3s = 16.89 t/s
33.18.742.303 I slot print_timing: id 2 | task 28836 | n_decoded = 8150, tg = 18.88 t/s, tg_3s = 16.71 t/s
33.21.247.660 I slot print_timing: id 2 | task 28836 | prompt eval time = 1763.05 ms / 66 tokens ( 26.71 ms per token, 37.44 tokens per second)
33.21.247.664 I slot print_timing: id 2 | task 28836 | eval time = 434265.75 ms / 8192 tokens ( 53.01 ms per token, 18.86 tokens per second)
33.21.247.664 I slot print_timing: id 2 | task 28836 | total time = 436028.79 ms / 8258 tokens
33.21.247.665 I slot print_timing: id 2 | task 28836 | graphs reused = 36848
33.21.247.706 I slot release: id 2 | task 28836 | stop processing: n_tokens = 8320, truncated = 0
33.21.247.760 I srv update_slots: all slots are idle
33.21.249.414 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
33.21.270.407 I srv operator(): Chat format: peg-native
33.21.276.516 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.612 (> 0.100 thold), f_keep = 0.008
33.21.276.518 I srv get_availabl: updating prompt cache
33.21.276.844 W srv prompt_save: - saving prompt with length 8320, total state size = 2015.097 MiB (draft: 0.000 MiB)
33.22.506.402 I srv load: - looking for better prompt, base f_keep = 0.008, sim = 0.612
33.22.506.410 W srv update: - cache size limit reached, removing oldest entry (size = 102.936 MiB)
33.22.513.846 W srv update: - cache size limit reached, removing oldest entry (size = 30.276 MiB)
33.22.516.371 W srv update: - cache size limit reached, removing oldest entry (size = 2005.893 MiB)
33.22.679.789 I srv update: - cache state: 9 prompts, 7083.363 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
33.22.679.794 I srv update: - prompt 0x6354c6628780: 650 tokens, checkpoints: 0, 157.431 MiB
33.22.679.796 I srv update: - prompt 0x6354c28b1d40: 423 tokens, checkpoints: 0, 102.452 MiB
33.22.679.797 I srv update: - prompt 0x6354c9e4fcf0: 533 tokens, checkpoints: 0, 129.093 MiB
33.22.679.798 I srv update: - prompt 0x6354c2d32b70: 1311 tokens, checkpoints: 0, 317.524 MiB
33.22.679.799 I srv update: - prompt 0x6354c2d09130: 8261 tokens, checkpoints: 0, 2000.807 MiB
33.22.679.800 I srv update: - prompt 0x6354c2ebdcd0: 8290 tokens, checkpoints: 0, 2007.831 MiB
33.22.679.801 I srv update: - prompt 0x6354c2cffe70: 655 tokens, checkpoints: 0, 158.642 MiB
33.22.679.802 I srv update: - prompt 0x6354ca9ab8c0: 803 tokens, checkpoints: 0, 194.487 MiB
33.22.679.803 I srv update: - prompt 0x6354c2a6f0c0: 8320 tokens, checkpoints: 0, 2015.097 MiB
33.22.679.805 I srv get_availabl: prompt cache update took 1403.29 ms
33.22.681.844 I slot launch_slot_: id 2 | task 37029 | processing task, is_child = 0
33.22.681.847 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
33.22.681.848 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
33.22.681.881 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
33.22.681.882 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
33.22.681.901 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
33.22.681.903 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
33.28.845.891 I slot print_timing: id 2 | task 37029 | n_decoded = 100, tg = 21.49 t/s, tg_3s = 21.49 t/s
33.31.848.655 I slot print_timing: id 2 | task 37029 | n_decoded = 165, tg = 21.55 t/s, tg_3s = 21.65 t/s
33.34.889.995 I slot print_timing: id 2 | task 37029 | n_decoded = 231, tg = 21.59 t/s, tg_3s = 21.70 t/s
33.37.898.119 I slot print_timing: id 2 | task 37029 | n_decoded = 296, tg = 21.60 t/s, tg_3s = 21.61 t/s
33.40.937.208 I slot print_timing: id 2 | task 37029 | n_decoded = 362, tg = 21.62 t/s, tg_3s = 21.72 t/s
33.43.963.835 I slot print_timing: id 2 | task 37029 | n_decoded = 427, tg = 21.60 t/s, tg_3s = 21.48 t/s
33.46.970.529 I slot print_timing: id 2 | task 37029 | n_decoded = 492, tg = 21.60 t/s, tg_3s = 21.62 t/s
33.48.495.675 I slot print_timing: id 2 | task 37029 | prompt eval time = 1509.84 ms / 40 tokens ( 37.75 ms per token, 26.49 tokens per second)
33.48.495.680 I slot print_timing: id 2 | task 37029 | eval time = 24303.90 ms / 525 tokens ( 46.29 ms per token, 21.60 tokens per second)
33.48.495.680 I slot print_timing: id 2 | task 37029 | total time = 25813.73 ms / 565 tokens
33.48.495.681 I slot print_timing: id 2 | task 37029 | graphs reused = 37369
33.48.495.711 I slot release: id 2 | task 37029 | stop processing: n_tokens = 627, truncated = 0
33.48.495.726 I srv update_slots: all slots are idle
33.48.496.607 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
33.48.516.976 I srv operator(): Chat format: peg-native
33.48.523.061 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.529 (> 0.100 thold), f_keep = 0.100
33.48.523.063 I srv get_availabl: updating prompt cache
33.48.523.125 W srv prompt_save: - saving prompt with length 627, total state size = 151.860 MiB (draft: 0.000 MiB)
33.48.664.725 I srv load: - looking for better prompt, base f_keep = 0.100, sim = 0.529
33.48.664.734 I srv update: - cache state: 10 prompts, 7235.223 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
33.48.664.735 I srv update: - prompt 0x6354c6628780: 650 tokens, checkpoints: 0, 157.431 MiB
33.48.664.736 I srv update: - prompt 0x6354c28b1d40: 423 tokens, checkpoints: 0, 102.452 MiB
33.48.664.744 I srv update: - prompt 0x6354c9e4fcf0: 533 tokens, checkpoints: 0, 129.093 MiB
33.48.664.745 I srv update: - prompt 0x6354c2d32b70: 1311 tokens, checkpoints: 0, 317.524 MiB
33.48.664.746 I srv update: - prompt 0x6354c2d09130: 8261 tokens, checkpoints: 0, 2000.807 MiB
33.48.664.747 I srv update: - prompt 0x6354c2ebdcd0: 8290 tokens, checkpoints: 0, 2007.831 MiB
33.48.664.747 I srv update: - prompt 0x6354c2cffe70: 655 tokens, checkpoints: 0, 158.642 MiB
33.48.664.748 I srv update: - prompt 0x6354ca9ab8c0: 803 tokens, checkpoints: 0, 194.487 MiB
33.48.664.749 I srv update: - prompt 0x6354c2a6f0c0: 8320 tokens, checkpoints: 0, 2015.097 MiB
33.48.664.749 I srv update: - prompt 0x6354c2a2a820: 627 tokens, checkpoints: 0, 151.860 MiB
33.48.664.750 I srv get_availabl: prompt cache update took 141.69 ms
33.48.666.763 I slot launch_slot_: id 2 | task 37555 | processing task, is_child = 0
33.48.666.765 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
33.48.666.766 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
33.48.666.783 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
33.48.666.784 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
33.48.666.798 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
33.48.666.799 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
33.55.159.322 I slot print_timing: id 2 | task 37555 | n_decoded = 100, tg = 21.91 t/s, tg_3s = 21.91 t/s
33.58.162.317 I slot print_timing: id 2 | task 37555 | n_decoded = 164, tg = 21.67 t/s, tg_3s = 21.31 t/s
34.01.200.565 I slot print_timing: id 2 | task 37555 | n_decoded = 230, tg = 21.69 t/s, tg_3s = 21.72 t/s
34.04.239.946 I slot print_timing: id 2 | task 37555 | n_decoded = 296, tg = 21.69 t/s, tg_3s = 21.71 t/s
34.07.275.450 I slot print_timing: id 2 | task 37555 | n_decoded = 361, tg = 21.64 t/s, tg_3s = 21.41 t/s
34.10.280.167 I slot print_timing: id 2 | task 37555 | n_decoded = 425, tg = 21.59 t/s, tg_3s = 21.30 t/s
34.13.294.494 I slot print_timing: id 2 | task 37555 | n_decoded = 490, tg = 21.59 t/s, tg_3s = 21.56 t/s
34.16.308.819 I slot print_timing: id 2 | task 37555 | n_decoded = 555, tg = 21.58 t/s, tg_3s = 21.56 t/s
34.19.320.860 I slot print_timing: id 2 | task 37555 | n_decoded = 620, tg = 21.58 t/s, tg_3s = 21.58 t/s
34.22.363.228 I slot print_timing: id 2 | task 37555 | n_decoded = 684, tg = 21.53 t/s, tg_3s = 21.04 t/s
34.25.373.162 I slot print_timing: id 2 | task 37555 | n_decoded = 748, tg = 21.51 t/s, tg_3s = 21.26 t/s
34.28.397.934 I slot print_timing: id 2 | task 37555 | n_decoded = 812, tg = 21.48 t/s, tg_3s = 21.16 t/s
34.31.401.190 I slot print_timing: id 2 | task 37555 | n_decoded = 876, tg = 21.47 t/s, tg_3s = 21.31 t/s
34.34.421.426 I slot print_timing: id 2 | task 37555 | n_decoded = 940, tg = 21.45 t/s, tg_3s = 21.19 t/s
34.37.432.171 I slot print_timing: id 2 | task 37555 | n_decoded = 1004, tg = 21.44 t/s, tg_3s = 21.26 t/s
34.40.475.089 I slot print_timing: id 2 | task 37555 | n_decoded = 1068, tg = 21.41 t/s, tg_3s = 21.03 t/s
34.43.484.276 I slot print_timing: id 2 | task 37555 | n_decoded = 1132, tg = 21.40 t/s, tg_3s = 21.27 t/s
34.46.500.418 I slot print_timing: id 2 | task 37555 | n_decoded = 1196, tg = 21.39 t/s, tg_3s = 21.22 t/s
34.49.513.259 I slot print_timing: id 2 | task 37555 | n_decoded = 1260, tg = 21.39 t/s, tg_3s = 21.24 t/s
34.52.516.103 I slot print_timing: id 2 | task 37555 | n_decoded = 1323, tg = 21.37 t/s, tg_3s = 20.98 t/s
34.55.527.296 I slot print_timing: id 2 | task 37555 | n_decoded = 1386, tg = 21.35 t/s, tg_3s = 20.92 t/s
34.58.561.495 I slot print_timing: id 2 | task 37555 | n_decoded = 1450, tg = 21.33 t/s, tg_3s = 21.09 t/s
35.01.574.650 I slot print_timing: id 2 | task 37555 | n_decoded = 1514, tg = 21.33 t/s, tg_3s = 21.24 t/s
35.04.582.418 I slot print_timing: id 2 | task 37555 | n_decoded = 1577, tg = 21.31 t/s, tg_3s = 20.95 t/s
35.07.598.730 I slot print_timing: id 2 | task 37555 | n_decoded = 1640, tg = 21.30 t/s, tg_3s = 20.89 t/s
35.10.640.717 I slot print_timing: id 2 | task 37555 | n_decoded = 1703, tg = 21.28 t/s, tg_3s = 20.71 t/s
35.13.657.199 I slot print_timing: id 2 | task 37555 | n_decoded = 1764, tg = 21.24 t/s, tg_3s = 20.22 t/s
35.16.662.902 I slot print_timing: id 2 | task 37555 | n_decoded = 1825, tg = 21.20 t/s, tg_3s = 20.29 t/s
35.19.663.641 I slot print_timing: id 2 | task 37555 | n_decoded = 1886, tg = 21.17 t/s, tg_3s = 20.33 t/s
35.22.685.422 I slot print_timing: id 2 | task 37555 | n_decoded = 1947, tg = 21.14 t/s, tg_3s = 20.19 t/s
35.25.722.971 I slot print_timing: id 2 | task 37555 | n_decoded = 2008, tg = 21.11 t/s, tg_3s = 20.08 t/s
35.28.755.835 I slot print_timing: id 2 | task 37555 | n_decoded = 2069, tg = 21.08 t/s, tg_3s = 20.11 t/s
35.31.803.148 I slot print_timing: id 2 | task 37555 | n_decoded = 2130, tg = 21.05 t/s, tg_3s = 20.02 t/s
35.34.825.590 I slot print_timing: id 2 | task 37555 | n_decoded = 2189, tg = 21.00 t/s, tg_3s = 19.52 t/s
35.37.874.854 I slot print_timing: id 2 | task 37555 | n_decoded = 2249, tg = 20.96 t/s, tg_3s = 19.68 t/s
35.40.909.329 I slot print_timing: id 2 | task 37555 | n_decoded = 2309, tg = 20.93 t/s, tg_3s = 19.77 t/s
35.43.947.801 I slot print_timing: id 2 | task 37555 | n_decoded = 2369, tg = 20.90 t/s, tg_3s = 19.75 t/s
35.46.951.959 I slot print_timing: id 2 | task 37555 | n_decoded = 2429, tg = 20.88 t/s, tg_3s = 19.97 t/s
35.49.990.389 I slot print_timing: id 2 | task 37555 | n_decoded = 2488, tg = 20.84 t/s, tg_3s = 19.42 t/s
35.53.014.043 I slot print_timing: id 2 | task 37555 | n_decoded = 2547, tg = 20.81 t/s, tg_3s = 19.51 t/s
35.56.020.222 I slot print_timing: id 2 | task 37555 | n_decoded = 2606, tg = 20.78 t/s, tg_3s = 19.63 t/s
35.59.038.726 I slot print_timing: id 2 | task 37555 | n_decoded = 2665, tg = 20.75 t/s, tg_3s = 19.55 t/s
36.02.054.871 I slot print_timing: id 2 | task 37555 | n_decoded = 2724, tg = 20.72 t/s, tg_3s = 19.56 t/s
36.05.075.340 I slot print_timing: id 2 | task 37555 | n_decoded = 2783, tg = 20.69 t/s, tg_3s = 19.53 t/s
36.08.093.060 I slot print_timing: id 2 | task 37555 | n_decoded = 2842, tg = 20.67 t/s, tg_3s = 19.55 t/s
36.11.141.341 I slot print_timing: id 2 | task 37555 | n_decoded = 2901, tg = 20.64 t/s, tg_3s = 19.36 t/s
36.14.175.185 I slot print_timing: id 2 | task 37555 | n_decoded = 2960, tg = 20.62 t/s, tg_3s = 19.45 t/s
36.17.204.484 I slot print_timing: id 2 | task 37555 | n_decoded = 3018, tg = 20.59 t/s, tg_3s = 19.15 t/s
36.20.254.932 I slot print_timing: id 2 | task 37555 | n_decoded = 3077, tg = 20.56 t/s, tg_3s = 19.34 t/s
36.23.267.594 I slot print_timing: id 2 | task 37555 | n_decoded = 3135, tg = 20.53 t/s, tg_3s = 19.25 t/s
36.26.313.263 I slot print_timing: id 2 | task 37555 | n_decoded = 3194, tg = 20.51 t/s, tg_3s = 19.37 t/s
36.29.322.874 I slot print_timing: id 2 | task 37555 | n_decoded = 3252, tg = 20.49 t/s, tg_3s = 19.27 t/s
36.32.371.846 I slot print_timing: id 2 | task 37555 | n_decoded = 3311, tg = 20.47 t/s, tg_3s = 19.35 t/s
36.35.385.930 I slot print_timing: id 2 | task 37555 | n_decoded = 3369, tg = 20.44 t/s, tg_3s = 19.24 t/s
36.38.390.743 I slot print_timing: id 2 | task 37555 | n_decoded = 3427, tg = 20.42 t/s, tg_3s = 19.30 t/s
36.41.418.259 I slot print_timing: id 2 | task 37555 | n_decoded = 3485, tg = 20.40 t/s, tg_3s = 19.16 t/s
36.44.423.056 I slot print_timing: id 2 | task 37555 | n_decoded = 3542, tg = 20.38 t/s, tg_3s = 18.97 t/s
36.47.426.415 I slot print_timing: id 2 | task 37555 | n_decoded = 3599, tg = 20.35 t/s, tg_3s = 18.98 t/s
36.50.435.975 I slot print_timing: id 2 | task 37555 | n_decoded = 3656, tg = 20.33 t/s, tg_3s = 18.94 t/s
36.53.467.244 I slot print_timing: id 2 | task 37555 | n_decoded = 3714, tg = 20.31 t/s, tg_3s = 19.13 t/s
36.56.514.094 I slot print_timing: id 2 | task 37555 | n_decoded = 3772, tg = 20.29 t/s, tg_3s = 19.04 t/s
36.59.546.931 I slot print_timing: id 2 | task 37555 | n_decoded = 3830, tg = 20.27 t/s, tg_3s = 19.12 t/s
37.02.593.406 I slot print_timing: id 2 | task 37555 | n_decoded = 3888, tg = 20.25 t/s, tg_3s = 19.04 t/s
37.05.627.413 I slot print_timing: id 2 | task 37555 | n_decoded = 3946, tg = 20.23 t/s, tg_3s = 19.12 t/s
37.08.628.935 I slot print_timing: id 2 | task 37555 | n_decoded = 4003, tg = 20.21 t/s, tg_3s = 18.99 t/s
37.11.633.116 I slot print_timing: id 2 | task 37555 | n_decoded = 4060, tg = 20.20 t/s, tg_3s = 18.97 t/s
37.14.636.994 I slot print_timing: id 2 | task 37555 | n_decoded = 4117, tg = 20.18 t/s, tg_3s = 18.98 t/s
37.17.652.735 I slot print_timing: id 2 | task 37555 | n_decoded = 4174, tg = 20.16 t/s, tg_3s = 18.90 t/s
37.20.675.080 I slot print_timing: id 2 | task 37555 | n_decoded = 4231, tg = 20.14 t/s, tg_3s = 18.86 t/s
37.23.722.328 I slot print_timing: id 2 | task 37555 | n_decoded = 4288, tg = 20.12 t/s, tg_3s = 18.71 t/s
37.26.730.496 I slot print_timing: id 2 | task 37555 | n_decoded = 4343, tg = 20.09 t/s, tg_3s = 18.28 t/s
37.29.756.514 I slot print_timing: id 2 | task 37555 | n_decoded = 4399, tg = 20.07 t/s, tg_3s = 18.51 t/s
37.32.762.322 I slot print_timing: id 2 | task 37555 | n_decoded = 4455, tg = 20.05 t/s, tg_3s = 18.63 t/s
37.35.807.694 I slot print_timing: id 2 | task 37555 | n_decoded = 4511, tg = 20.03 t/s, tg_3s = 18.39 t/s
37.38.842.240 I slot print_timing: id 2 | task 37555 | n_decoded = 4567, tg = 20.01 t/s, tg_3s = 18.45 t/s
37.41.882.575 I slot print_timing: id 2 | task 37555 | n_decoded = 4623, tg = 19.99 t/s, tg_3s = 18.42 t/s
37.44.899.324 I slot print_timing: id 2 | task 37555 | n_decoded = 4679, tg = 19.97 t/s, tg_3s = 18.56 t/s
37.47.918.979 I slot print_timing: id 2 | task 37555 | n_decoded = 4735, tg = 19.95 t/s, tg_3s = 18.55 t/s
37.50.919.499 I slot print_timing: id 2 | task 37555 | n_decoded = 4789, tg = 19.93 t/s, tg_3s = 18.00 t/s
37.53.969.034 I slot print_timing: id 2 | task 37555 | n_decoded = 4845, tg = 19.91 t/s, tg_3s = 18.36 t/s
37.56.987.865 I slot print_timing: id 2 | task 37555 | n_decoded = 4900, tg = 19.89 t/s, tg_3s = 18.22 t/s
38.00.006.075 I slot print_timing: id 2 | task 37555 | n_decoded = 4955, tg = 19.87 t/s, tg_3s = 18.22 t/s
38.03.059.556 I slot print_timing: id 2 | task 37555 | n_decoded = 5011, tg = 19.85 t/s, tg_3s = 18.34 t/s
38.06.098.376 I slot print_timing: id 2 | task 37555 | n_decoded = 5067, tg = 19.83 t/s, tg_3s = 18.43 t/s
38.09.141.168 I slot print_timing: id 2 | task 37555 | n_decoded = 5123, tg = 19.81 t/s, tg_3s = 18.40 t/s
38.12.182.817 I slot print_timing: id 2 | task 37555 | n_decoded = 5179, tg = 19.80 t/s, tg_3s = 18.41 t/s
38.15.222.323 I slot print_timing: id 2 | task 37555 | n_decoded = 5235, tg = 19.78 t/s, tg_3s = 18.42 t/s
38.18.233.051 I slot print_timing: id 2 | task 37555 | n_decoded = 5290, tg = 19.77 t/s, tg_3s = 18.27 t/s
38.21.237.582 I slot print_timing: id 2 | task 37555 | n_decoded = 5345, tg = 19.75 t/s, tg_3s = 18.31 t/s
38.24.245.121 I slot print_timing: id 2 | task 37555 | n_decoded = 5400, tg = 19.73 t/s, tg_3s = 18.29 t/s
38.27.256.495 I slot print_timing: id 2 | task 37555 | n_decoded = 5455, tg = 19.72 t/s, tg_3s = 18.26 t/s
38.30.271.152 I slot print_timing: id 2 | task 37555 | n_decoded = 5510, tg = 19.70 t/s, tg_3s = 18.24 t/s
38.33.278.977 I slot print_timing: id 2 | task 37555 | n_decoded = 5564, tg = 19.68 t/s, tg_3s = 17.95 t/s
38.36.300.799 I slot print_timing: id 2 | task 37555 | n_decoded = 5618, tg = 19.66 t/s, tg_3s = 17.87 t/s
38.39.342.969 I slot print_timing: id 2 | task 37555 | n_decoded = 5673, tg = 19.65 t/s, tg_3s = 18.08 t/s
38.42.380.914 I slot print_timing: id 2 | task 37555 | n_decoded = 5728, tg = 19.63 t/s, tg_3s = 18.10 t/s
38.45.432.103 I slot print_timing: id 2 | task 37555 | n_decoded = 5783, tg = 19.61 t/s, tg_3s = 18.03 t/s
38.48.486.917 I slot print_timing: id 2 | task 37555 | n_decoded = 5837, tg = 19.59 t/s, tg_3s = 17.68 t/s
38.51.537.961 I slot print_timing: id 2 | task 37555 | n_decoded = 5892, tg = 19.58 t/s, tg_3s = 18.03 t/s
38.54.549.477 I slot print_timing: id 2 | task 37555 | n_decoded = 5946, tg = 19.56 t/s, tg_3s = 17.93 t/s
38.57.576.038 I slot print_timing: id 2 | task 37555 | n_decoded = 6000, tg = 19.55 t/s, tg_3s = 17.84 t/s
39.00.627.991 I slot print_timing: id 2 | task 37555 | n_decoded = 6054, tg = 19.53 t/s, tg_3s = 17.69 t/s
39.03.655.219 I slot print_timing: id 2 | task 37555 | n_decoded = 6107, tg = 19.51 t/s, tg_3s = 17.51 t/s
39.06.679.888 I slot print_timing: id 2 | task 37555 | n_decoded = 6161, tg = 19.49 t/s, tg_3s = 17.85 t/s
39.09.707.768 I slot print_timing: id 2 | task 37555 | n_decoded = 6215, tg = 19.48 t/s, tg_3s = 17.83 t/s
39.12.745.710 I slot print_timing: id 2 | task 37555 | n_decoded = 6269, tg = 19.46 t/s, tg_3s = 17.78 t/s
39.15.790.319 I slot print_timing: id 2 | task 37555 | n_decoded = 6322, tg = 19.44 t/s, tg_3s = 17.41 t/s
39.18.797.086 I slot print_timing: id 2 | task 37555 | n_decoded = 6375, tg = 19.42 t/s, tg_3s = 17.63 t/s
39.21.854.029 I slot print_timing: id 2 | task 37555 | n_decoded = 6428, tg = 19.40 t/s, tg_3s = 17.34 t/s
39.24.907.712 I slot print_timing: id 2 | task 37555 | n_decoded = 6481, tg = 19.39 t/s, tg_3s = 17.36 t/s
39.27.958.846 I slot print_timing: id 2 | task 37555 | n_decoded = 6535, tg = 19.37 t/s, tg_3s = 17.70 t/s
39.30.982.830 I slot print_timing: id 2 | task 37555 | n_decoded = 6588, tg = 19.35 t/s, tg_3s = 17.53 t/s
39.33.996.906 I slot print_timing: id 2 | task 37555 | n_decoded = 6641, tg = 19.34 t/s, tg_3s = 17.58 t/s
39.37.010.212 I slot print_timing: id 2 | task 37555 | n_decoded = 6694, tg = 19.32 t/s, tg_3s = 17.59 t/s
39.40.048.951 I slot print_timing: id 2 | task 37555 | n_decoded = 6747, tg = 19.31 t/s, tg_3s = 17.44 t/s
39.43.082.401 I slot print_timing: id 2 | task 37555 | n_decoded = 6800, tg = 19.29 t/s, tg_3s = 17.47 t/s
39.46.103.279 I slot print_timing: id 2 | task 37555 | n_decoded = 6853, tg = 19.28 t/s, tg_3s = 17.54 t/s
39.49.150.786 I slot print_timing: id 2 | task 37555 | n_decoded = 6906, tg = 19.26 t/s, tg_3s = 17.39 t/s
39.52.193.767 I slot print_timing: id 2 | task 37555 | n_decoded = 6959, tg = 19.25 t/s, tg_3s = 17.42 t/s
39.55.204.104 I slot print_timing: id 2 | task 37555 | n_decoded = 7011, tg = 19.23 t/s, tg_3s = 17.27 t/s
39.58.229.899 I slot print_timing: id 2 | task 37555 | n_decoded = 7063, tg = 19.21 t/s, tg_3s = 17.19 t/s
40.01.243.974 I slot print_timing: id 2 | task 37555 | n_decoded = 7115, tg = 19.20 t/s, tg_3s = 17.25 t/s
40.04.254.340 I slot print_timing: id 2 | task 37555 | n_decoded = 7167, tg = 19.18 t/s, tg_3s = 17.27 t/s
40.07.291.312 I slot print_timing: id 2 | task 37555 | n_decoded = 7220, tg = 19.17 t/s, tg_3s = 17.45 t/s
40.10.316.401 I slot print_timing: id 2 | task 37555 | n_decoded = 7273, tg = 19.15 t/s, tg_3s = 17.52 t/s
40.13.317.965 I slot print_timing: id 2 | task 37555 | n_decoded = 7325, tg = 19.14 t/s, tg_3s = 17.32 t/s
40.16.352.173 I slot print_timing: id 2 | task 37555 | n_decoded = 7377, tg = 19.12 t/s, tg_3s = 17.14 t/s
40.19.388.797 I slot print_timing: id 2 | task 37555 | n_decoded = 7429, tg = 19.11 t/s, tg_3s = 17.12 t/s
40.22.420.959 I slot print_timing: id 2 | task 37555 | n_decoded = 7481, tg = 19.09 t/s, tg_3s = 17.15 t/s
40.25.450.706 I slot print_timing: id 2 | task 37555 | n_decoded = 7533, tg = 19.08 t/s, tg_3s = 17.16 t/s
40.28.492.745 I slot print_timing: id 2 | task 37555 | n_decoded = 7585, tg = 19.06 t/s, tg_3s = 17.09 t/s
40.31.506.688 I slot print_timing: id 2 | task 37555 | n_decoded = 7636, tg = 19.05 t/s, tg_3s = 16.92 t/s
40.34.560.161 I slot print_timing: id 2 | task 37555 | n_decoded = 7688, tg = 19.03 t/s, tg_3s = 17.03 t/s
40.37.578.550 I slot print_timing: id 2 | task 37555 | n_decoded = 7739, tg = 19.02 t/s, tg_3s = 16.90 t/s
40.40.631.789 I slot print_timing: id 2 | task 37555 | n_decoded = 7790, tg = 19.00 t/s, tg_3s = 16.70 t/s
40.43.645.163 I slot print_timing: id 2 | task 37555 | n_decoded = 7841, tg = 18.98 t/s, tg_3s = 16.92 t/s
40.46.645.621 I slot print_timing: id 2 | task 37555 | n_decoded = 7892, tg = 18.97 t/s, tg_3s = 17.00 t/s
40.49.694.557 I slot print_timing: id 2 | task 37555 | n_decoded = 7943, tg = 18.95 t/s, tg_3s = 16.73 t/s
40.52.698.454 I slot print_timing: id 2 | task 37555 | n_decoded = 7994, tg = 18.94 t/s, tg_3s = 16.98 t/s
40.55.706.852 I slot print_timing: id 2 | task 37555 | n_decoded = 8045, tg = 18.92 t/s, tg_3s = 16.95 t/s
40.58.747.376 I slot print_timing: id 2 | task 37555 | n_decoded = 8096, tg = 18.91 t/s, tg_3s = 16.77 t/s
41.01.773.648 I slot print_timing: id 2 | task 37555 | n_decoded = 8147, tg = 18.89 t/s, tg_3s = 16.85 t/s
41.04.445.164 I slot print_timing: id 2 | task 37555 | prompt eval time = 1928.78 ms / 56 tokens ( 34.44 ms per token, 29.03 tokens per second)
41.04.445.169 I slot print_timing: id 2 | task 37555 | eval time = 433849.55 ms / 8192 tokens ( 52.96 ms per token, 18.88 tokens per second)
41.04.445.169 I slot print_timing: id 2 | task 37555 | total time = 435778.33 ms / 8248 tokens
41.04.445.170 I slot print_timing: id 2 | task 37555 | graphs reused = 45527
41.04.445.203 I slot release: id 2 | task 37555 | stop processing: n_tokens = 8310, truncated = 0
41.04.445.253 I srv update_slots: all slots are idle
41.04.446.845 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
41.04.466.224 I srv operator(): Chat format: peg-native
41.04.472.498 I slot get_availabl: id 1 | task -1 | selected slot by LRU, t_last = -1
41.04.472.500 I srv get_availabl: updating prompt cache
41.04.472.502 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
41.04.472.507 I srv update: - cache state: 10 prompts, 7235.223 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
41.04.472.509 I srv update: - prompt 0x6354c6628780: 650 tokens, checkpoints: 0, 157.431 MiB
41.04.472.510 I srv update: - prompt 0x6354c28b1d40: 423 tokens, checkpoints: 0, 102.452 MiB
41.04.472.511 I srv update: - prompt 0x6354c9e4fcf0: 533 tokens, checkpoints: 0, 129.093 MiB
41.04.472.511 I srv update: - prompt 0x6354c2d32b70: 1311 tokens, checkpoints: 0, 317.524 MiB
41.04.472.512 I srv update: - prompt 0x6354c2d09130: 8261 tokens, checkpoints: 0, 2000.807 MiB
41.04.472.515 I srv update: - prompt 0x6354c2ebdcd0: 8290 tokens, checkpoints: 0, 2007.831 MiB
41.04.472.516 I srv update: - prompt 0x6354c2cffe70: 655 tokens, checkpoints: 0, 158.642 MiB
41.04.472.517 I srv update: - prompt 0x6354ca9ab8c0: 803 tokens, checkpoints: 0, 194.487 MiB
41.04.472.517 I srv update: - prompt 0x6354c2a6f0c0: 8320 tokens, checkpoints: 0, 2015.097 MiB
41.04.472.518 I srv update: - prompt 0x6354c2a2a820: 627 tokens, checkpoints: 0, 151.860 MiB
41.04.472.519 I srv get_availabl: prompt cache update took 0.02 ms
41.04.474.214 I slot launch_slot_: id 1 | task 45748 | processing task, is_child = 0
41.04.474.216 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
41.04.474.216 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
41.04.474.248 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
41.04.474.497 W srv prompt_save: - saving prompt with length 8310, total state size = 2012.675 MiB (draft: 0.000 MiB)
41.05.874.299 W srv update: - cache size limit reached, removing oldest entry (size = 157.431 MiB)
41.05.893.781 W srv update: - cache size limit reached, removing oldest entry (size = 102.452 MiB)
41.05.901.592 W srv update: - cache size limit reached, removing oldest entry (size = 129.093 MiB)
41.05.919.688 W srv update: - cache size limit reached, removing oldest entry (size = 317.524 MiB)
41.05.953.168 W srv update: - cache size limit reached, removing oldest entry (size = 2000.807 MiB)
41.06.131.805 I srv update: - cache state: 6 prompts, 6540.591 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
41.06.131.810 I srv update: - prompt 0x6354c2ebdcd0: 8290 tokens, checkpoints: 0, 2007.831 MiB
41.06.131.811 I srv update: - prompt 0x6354c2cffe70: 655 tokens, checkpoints: 0, 158.642 MiB
41.06.131.812 I srv update: - prompt 0x6354ca9ab8c0: 803 tokens, checkpoints: 0, 194.487 MiB
41.06.131.813 I srv update: - prompt 0x6354c2a6f0c0: 8320 tokens, checkpoints: 0, 2015.097 MiB
41.06.131.813 I srv update: - prompt 0x6354c2a2a820: 627 tokens, checkpoints: 0, 151.860 MiB
41.06.131.814 I srv update: - prompt 0x6354c9e731d0: 8310 tokens, checkpoints: 0, 2012.675 MiB
41.06.131.816 I slot prompt_clear: id 2 | task -1 | clearing prompt with 8310 tokens
41.06.133.006 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
41.06.133.008 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
41.14.536.930 I slot print_timing: id 1 | task 45748 | n_decoded = 100, tg = 21.90 t/s, tg_3s = 21.90 t/s
41.17.559.620 I slot print_timing: id 1 | task 45748 | n_decoded = 165, tg = 21.74 t/s, tg_3s = 21.50 t/s
41.20.597.039 I slot print_timing: id 1 | task 45748 | n_decoded = 231, tg = 21.74 t/s, tg_3s = 21.73 t/s
41.23.640.481 I slot print_timing: id 1 | task 45748 | n_decoded = 297, tg = 21.73 t/s, tg_3s = 21.69 t/s
41.26.678.238 I slot print_timing: id 1 | task 45748 | n_decoded = 363, tg = 21.73 t/s, tg_3s = 21.73 t/s
41.29.690.355 I slot print_timing: id 1 | task 45748 | n_decoded = 428, tg = 21.70 t/s, tg_3s = 21.58 t/s
41.32.696.240 I slot print_timing: id 1 | task 45748 | n_decoded = 493, tg = 21.69 t/s, tg_3s = 21.62 t/s
41.34.178.422 I slot print_timing: id 1 | task 45748 | prompt eval time = 3836.40 ms / 130 tokens ( 29.51 ms per token, 33.89 tokens per second)
41.34.178.427 I slot print_timing: id 1 | task 45748 | eval time = 24208.52 ms / 525 tokens ( 46.11 ms per token, 21.69 tokens per second)
41.34.178.428 I slot print_timing: id 1 | task 45748 | total time = 28044.92 ms / 655 tokens
41.34.178.429 I slot print_timing: id 1 | task 45748 | graphs reused = 46048
41.34.178.471 I slot release: id 1 | task 45748 | stop processing: n_tokens = 654, truncated = 0
41.34.178.484 I srv update_slots: all slots are idle
41.34.237.586 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
41.34.256.728 I srv operator(): Chat format: peg-native
41.34.262.794 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.685 (> 0.100 thold), f_keep = 0.096
41.34.262.796 I srv get_availabl: updating prompt cache
41.34.262.870 W srv prompt_save: - saving prompt with length 654, total state size = 158.400 MiB (draft: 0.000 MiB)
41.34.403.573 I srv load: - looking for better prompt, base f_keep = 0.096, sim = 0.685
41.34.403.581 I srv update: - cache state: 7 prompts, 6698.991 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
41.34.403.582 I srv update: - prompt 0x6354c2ebdcd0: 8290 tokens, checkpoints: 0, 2007.831 MiB
41.34.403.583 I srv update: - prompt 0x6354c2cffe70: 655 tokens, checkpoints: 0, 158.642 MiB
41.34.403.584 I srv update: - prompt 0x6354ca9ab8c0: 803 tokens, checkpoints: 0, 194.487 MiB
41.34.403.585 I srv update: - prompt 0x6354c2a6f0c0: 8320 tokens, checkpoints: 0, 2015.097 MiB
41.34.403.586 I srv update: - prompt 0x6354c2a2a820: 627 tokens, checkpoints: 0, 151.860 MiB
41.34.403.587 I srv update: - prompt 0x6354c9e731d0: 8310 tokens, checkpoints: 0, 2012.675 MiB
41.34.403.588 I srv update: - prompt 0x6354c2c51b40: 654 tokens, checkpoints: 0, 158.400 MiB
41.34.403.592 I srv get_availabl: prompt cache update took 140.79 ms
41.34.405.229 I slot launch_slot_: id 1 | task 46275 | processing task, is_child = 0
41.34.405.232 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
41.34.405.232 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
41.34.405.250 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
41.34.405.250 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
41.34.405.264 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
41.34.405.264 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
41.39.864.160 I slot print_timing: id 1 | task 46275 | n_decoded = 100, tg = 21.89 t/s, tg_3s = 21.89 t/s
41.42.911.288 I slot print_timing: id 1 | task 46275 | n_decoded = 166, tg = 21.80 t/s, tg_3s = 21.66 t/s
41.45.914.502 I slot print_timing: id 1 | task 46275 | n_decoded = 231, tg = 21.75 t/s, tg_3s = 21.64 t/s
41.48.953.239 I slot print_timing: id 1 | task 46275 | n_decoded = 297, tg = 21.75 t/s, tg_3s = 21.72 t/s
41.51.994.655 I slot print_timing: id 1 | task 46275 | n_decoded = 363, tg = 21.74 t/s, tg_3s = 21.70 t/s
41.54.995.034 I slot print_timing: id 1 | task 46275 | n_decoded = 427, tg = 21.68 t/s, tg_3s = 21.33 t/s
41.58.040.608 I slot print_timing: id 1 | task 46275 | n_decoded = 492, tg = 21.63 t/s, tg_3s = 21.34 t/s
41.58.369.102 I slot print_timing: id 1 | task 46275 | prompt eval time = 890.19 ms / 29 tokens ( 30.70 ms per token, 32.58 tokens per second)
41.58.369.107 I slot print_timing: id 1 | task 46275 | eval time = 23073.62 ms / 499 tokens ( 46.24 ms per token, 21.63 tokens per second)
41.58.369.108 I slot print_timing: id 1 | task 46275 | total time = 23963.80 ms / 528 tokens
41.58.369.109 I slot print_timing: id 1 | task 46275 | graphs reused = 46543
41.58.369.140 I slot release: id 1 | task 46275 | stop processing: n_tokens = 590, truncated = 0
41.58.369.156 I srv update_slots: all slots are idle
41.58.436.456 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
41.58.455.526 I srv operator(): Chat format: peg-native
41.58.461.572 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.624 (> 0.100 thold), f_keep = 0.107
41.58.461.575 I srv get_availabl: updating prompt cache
41.58.461.645 W srv prompt_save: - saving prompt with length 590, total state size = 142.899 MiB (draft: 0.000 MiB)
41.58.564.312 I srv load: - looking for better prompt, base f_keep = 0.107, sim = 0.624
41.58.564.320 I srv update: - cache state: 8 prompts, 6841.890 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
41.58.564.322 I srv update: - prompt 0x6354c2ebdcd0: 8290 tokens, checkpoints: 0, 2007.831 MiB
41.58.564.322 I srv update: - prompt 0x6354c2cffe70: 655 tokens, checkpoints: 0, 158.642 MiB
41.58.564.323 I srv update: - prompt 0x6354ca9ab8c0: 803 tokens, checkpoints: 0, 194.487 MiB
41.58.564.323 I srv update: - prompt 0x6354c2a6f0c0: 8320 tokens, checkpoints: 0, 2015.097 MiB
41.58.564.324 I srv update: - prompt 0x6354c2a2a820: 627 tokens, checkpoints: 0, 151.860 MiB
41.58.564.325 I srv update: - prompt 0x6354c9e731d0: 8310 tokens, checkpoints: 0, 2012.675 MiB
41.58.564.326 I srv update: - prompt 0x6354c2c51b40: 654 tokens, checkpoints: 0, 158.400 MiB
41.58.564.335 I srv update: - prompt 0x6354c2ecfe50: 590 tokens, checkpoints: 0, 142.899 MiB
41.58.564.337 I srv get_availabl: prompt cache update took 102.76 ms
41.58.566.080 I slot launch_slot_: id 1 | task 46775 | processing task, is_child = 0
41.58.566.083 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
41.58.566.083 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
41.58.566.098 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
41.58.566.098 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
41.58.566.111 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
41.58.566.111 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
42.04.621.075 I slot print_timing: id 1 | task 46775 | n_decoded = 100, tg = 21.78 t/s, tg_3s = 21.78 t/s
42.07.630.626 I slot print_timing: id 1 | task 46775 | n_decoded = 165, tg = 21.71 t/s, tg_3s = 21.60 t/s
42.10.660.860 I slot print_timing: id 1 | task 46775 | n_decoded = 230, tg = 21.63 t/s, tg_3s = 21.45 t/s
42.13.705.850 I slot print_timing: id 1 | task 46775 | n_decoded = 296, tg = 21.64 t/s, tg_3s = 21.67 t/s
42.16.741.954 I slot print_timing: id 1 | task 46775 | n_decoded = 360, tg = 21.54 t/s, tg_3s = 21.08 t/s
42.19.749.757 I slot print_timing: id 1 | task 46775 | n_decoded = 423, tg = 21.45 t/s, tg_3s = 20.95 t/s
42.22.764.592 I slot print_timing: id 1 | task 46775 | n_decoded = 486, tg = 21.38 t/s, tg_3s = 20.90 t/s
42.25.773.323 I slot print_timing: id 1 | task 46775 | n_decoded = 551, tg = 21.40 t/s, tg_3s = 21.60 t/s
42.28.783.691 I slot print_timing: id 1 | task 46775 | n_decoded = 616, tg = 21.42 t/s, tg_3s = 21.59 t/s
42.31.818.056 I slot print_timing: id 1 | task 46775 | n_decoded = 681, tg = 21.42 t/s, tg_3s = 21.42 t/s
42.34.828.022 I slot print_timing: id 1 | task 46775 | n_decoded = 745, tg = 21.41 t/s, tg_3s = 21.26 t/s
42.37.851.077 I slot print_timing: id 1 | task 46775 | n_decoded = 809, tg = 21.39 t/s, tg_3s = 21.17 t/s
42.40.870.456 I slot print_timing: id 1 | task 46775 | n_decoded = 873, tg = 21.38 t/s, tg_3s = 21.20 t/s
42.43.891.649 I slot print_timing: id 1 | task 46775 | n_decoded = 937, tg = 21.36 t/s, tg_3s = 21.18 t/s
42.46.901.529 I slot print_timing: id 1 | task 46775 | n_decoded = 1001, tg = 21.36 t/s, tg_3s = 21.26 t/s
42.49.912.373 I slot print_timing: id 1 | task 46775 | n_decoded = 1065, tg = 21.35 t/s, tg_3s = 21.26 t/s
42.51.368.240 I slot print_timing: id 1 | task 46775 | prompt eval time = 1463.78 ms / 38 tokens ( 38.52 ms per token, 25.96 tokens per second)
42.51.368.245 I slot print_timing: id 1 | task 46775 | eval time = 51338.32 ms / 1096 tokens ( 46.84 ms per token, 21.35 tokens per second)
42.51.368.246 I slot print_timing: id 1 | task 46775 | total time = 52802.10 ms / 1134 tokens
42.51.368.248 I slot print_timing: id 1 | task 46775 | graphs reused = 47633
42.51.368.286 I slot release: id 1 | task 46775 | stop processing: n_tokens = 1196, truncated = 0
42.51.368.308 I srv update_slots: all slots are idle
42.51.430.724 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
42.51.449.093 I srv operator(): Chat format: peg-native
42.51.454.660 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.492 (> 0.100 thold), f_keep = 0.053
42.51.454.662 I srv get_availabl: updating prompt cache
42.51.454.726 W srv prompt_save: - saving prompt with length 1196, total state size = 289.671 MiB (draft: 0.000 MiB)
42.51.639.446 I srv load: - looking for better prompt, base f_keep = 0.053, sim = 0.492
42.51.639.453 I srv update: - cache state: 9 prompts, 7131.561 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
42.51.639.454 I srv update: - prompt 0x6354c2ebdcd0: 8290 tokens, checkpoints: 0, 2007.831 MiB
42.51.639.455 I srv update: - prompt 0x6354c2cffe70: 655 tokens, checkpoints: 0, 158.642 MiB
42.51.639.455 I srv update: - prompt 0x6354ca9ab8c0: 803 tokens, checkpoints: 0, 194.487 MiB
42.51.639.456 I srv update: - prompt 0x6354c2a6f0c0: 8320 tokens, checkpoints: 0, 2015.097 MiB
42.51.639.456 I srv update: - prompt 0x6354c2a2a820: 627 tokens, checkpoints: 0, 151.860 MiB
42.51.639.457 I srv update: - prompt 0x6354c9e731d0: 8310 tokens, checkpoints: 0, 2012.675 MiB
42.51.639.457 I srv update: - prompt 0x6354c2c51b40: 654 tokens, checkpoints: 0, 158.400 MiB
42.51.639.458 I srv update: - prompt 0x6354c2ecfe50: 590 tokens, checkpoints: 0, 142.899 MiB
42.51.639.458 I srv update: - prompt 0x6354c9ddc610: 1196 tokens, checkpoints: 0, 289.671 MiB
42.51.639.460 I srv get_availabl: prompt cache update took 184.80 ms
42.51.641.080 I slot launch_slot_: id 1 | task 47872 | processing task, is_child = 0
42.51.641.081 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
42.51.641.082 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
42.51.641.096 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
42.51.641.097 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
42.51.641.109 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
42.51.641.109 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
42.58.059.100 I slot print_timing: id 1 | task 47872 | n_decoded = 100, tg = 21.85 t/s, tg_3s = 21.85 t/s
43.01.062.120 I slot print_timing: id 1 | task 47872 | n_decoded = 165, tg = 21.77 t/s, tg_3s = 21.64 t/s
43.04.098.181 I slot print_timing: id 1 | task 47872 | n_decoded = 231, tg = 21.76 t/s, tg_3s = 21.74 t/s
43.07.134.883 I slot print_timing: id 1 | task 47872 | n_decoded = 297, tg = 21.75 t/s, tg_3s = 21.73 t/s
43.10.177.498 I slot print_timing: id 1 | task 47872 | n_decoded = 362, tg = 21.68 t/s, tg_3s = 21.36 t/s
43.13.205.408 I slot print_timing: id 1 | task 47872 | n_decoded = 427, tg = 21.65 t/s, tg_3s = 21.47 t/s
43.16.212.416 I slot print_timing: id 1 | task 47872 | n_decoded = 492, tg = 21.65 t/s, tg_3s = 21.62 t/s
43.19.247.998 I slot print_timing: id 1 | task 47872 | n_decoded = 557, tg = 21.62 t/s, tg_3s = 21.41 t/s
43.22.285.183 I slot print_timing: id 1 | task 47872 | n_decoded = 622, tg = 21.60 t/s, tg_3s = 21.40 t/s
43.25.328.840 I slot print_timing: id 1 | task 47872 | n_decoded = 685, tg = 21.51 t/s, tg_3s = 20.70 t/s
43.28.343.394 I slot print_timing: id 1 | task 47872 | n_decoded = 749, tg = 21.49 t/s, tg_3s = 21.23 t/s
43.31.352.104 I slot print_timing: id 1 | task 47872 | n_decoded = 813, tg = 21.47 t/s, tg_3s = 21.27 t/s
43.34.366.877 I slot print_timing: id 1 | task 47872 | n_decoded = 877, tg = 21.45 t/s, tg_3s = 21.23 t/s
43.37.386.004 I slot print_timing: id 1 | task 47872 | n_decoded = 941, tg = 21.43 t/s, tg_3s = 21.20 t/s
43.40.402.821 I slot print_timing: id 1 | task 47872 | n_decoded = 1005, tg = 21.42 t/s, tg_3s = 21.21 t/s
43.43.407.997 I slot print_timing: id 1 | task 47872 | n_decoded = 1069, tg = 21.41 t/s, tg_3s = 21.30 t/s
43.46.415.932 I slot print_timing: id 1 | task 47872 | n_decoded = 1133, tg = 21.40 t/s, tg_3s = 21.28 t/s
43.49.436.251 I slot print_timing: id 1 | task 47872 | n_decoded = 1197, tg = 21.39 t/s, tg_3s = 21.19 t/s
43.52.445.284 I slot print_timing: id 1 | task 47872 | n_decoded = 1261, tg = 21.39 t/s, tg_3s = 21.27 t/s
43.55.458.834 I slot print_timing: id 1 | task 47872 | n_decoded = 1325, tg = 21.38 t/s, tg_3s = 21.24 t/s
43.58.468.221 I slot print_timing: id 1 | task 47872 | n_decoded = 1389, tg = 21.37 t/s, tg_3s = 21.27 t/s
44.01.507.313 I slot print_timing: id 1 | task 47872 | n_decoded = 1453, tg = 21.36 t/s, tg_3s = 21.06 t/s
44.04.519.423 I slot print_timing: id 1 | task 47872 | n_decoded = 1517, tg = 21.36 t/s, tg_3s = 21.25 t/s
44.07.533.488 I slot print_timing: id 1 | task 47872 | n_decoded = 1581, tg = 21.35 t/s, tg_3s = 21.23 t/s
44.10.558.581 I slot print_timing: id 1 | task 47872 | n_decoded = 1645, tg = 21.34 t/s, tg_3s = 21.16 t/s
44.13.578.079 I slot print_timing: id 1 | task 47872 | n_decoded = 1705, tg = 21.29 t/s, tg_3s = 19.87 t/s
44.16.591.642 I slot print_timing: id 1 | task 47872 | n_decoded = 1766, tg = 21.25 t/s, tg_3s = 20.24 t/s
44.19.601.235 I slot print_timing: id 1 | task 47872 | n_decoded = 1827, tg = 21.21 t/s, tg_3s = 20.27 t/s
44.22.623.435 I slot print_timing: id 1 | task 47872 | n_decoded = 1888, tg = 21.18 t/s, tg_3s = 20.18 t/s
44.25.671.795 I slot print_timing: id 1 | task 47872 | n_decoded = 1948, tg = 21.13 t/s, tg_3s = 19.68 t/s
44.28.716.721 I slot print_timing: id 1 | task 47872 | n_decoded = 2009, tg = 21.10 t/s, tg_3s = 20.03 t/s
44.31.763.367 I slot print_timing: id 1 | task 47872 | n_decoded = 2069, tg = 21.05 t/s, tg_3s = 19.69 t/s
44.34.792.428 I slot print_timing: id 1 | task 47872 | n_decoded = 2130, tg = 21.02 t/s, tg_3s = 20.14 t/s
44.37.795.361 I slot print_timing: id 1 | task 47872 | n_decoded = 2190, tg = 20.99 t/s, tg_3s = 19.98 t/s
44.40.808.626 I slot print_timing: id 1 | task 47872 | n_decoded = 2249, tg = 20.95 t/s, tg_3s = 19.58 t/s
44.43.820.789 I slot print_timing: id 1 | task 47872 | n_decoded = 2309, tg = 20.93 t/s, tg_3s = 19.92 t/s
44.46.829.560 I slot print_timing: id 1 | task 47872 | n_decoded = 2369, tg = 20.90 t/s, tg_3s = 19.94 t/s
44.49.876.192 I slot print_timing: id 1 | task 47872 | n_decoded = 2429, tg = 20.87 t/s, tg_3s = 19.69 t/s
44.52.876.337 I slot print_timing: id 1 | task 47872 | n_decoded = 2488, tg = 20.84 t/s, tg_3s = 19.67 t/s
44.55.883.257 I slot print_timing: id 1 | task 47872 | n_decoded = 2547, tg = 20.81 t/s, tg_3s = 19.62 t/s
44.58.922.706 I slot print_timing: id 1 | task 47872 | n_decoded = 2607, tg = 20.78 t/s, tg_3s = 19.74 t/s
45.01.957.921 I slot print_timing: id 1 | task 47872 | n_decoded = 2667, tg = 20.76 t/s, tg_3s = 19.77 t/s
45.04.990.394 I slot print_timing: id 1 | task 47872 | n_decoded = 2726, tg = 20.73 t/s, tg_3s = 19.46 t/s
45.08.009.501 I slot print_timing: id 1 | task 47872 | n_decoded = 2785, tg = 20.70 t/s, tg_3s = 19.54 t/s
45.11.043.944 I slot print_timing: id 1 | task 47872 | n_decoded = 2844, tg = 20.67 t/s, tg_3s = 19.44 t/s
45.14.061.554 I slot print_timing: id 1 | task 47872 | n_decoded = 2903, tg = 20.65 t/s, tg_3s = 19.55 t/s
45.17.091.718 I slot print_timing: id 1 | task 47872 | n_decoded = 2962, tg = 20.63 t/s, tg_3s = 19.47 t/s
45.20.127.688 I slot print_timing: id 1 | task 47872 | n_decoded = 3021, tg = 20.60 t/s, tg_3s = 19.43 t/s
45.23.170.193 I slot print_timing: id 1 | task 47872 | n_decoded = 3080, tg = 20.58 t/s, tg_3s = 19.39 t/s
45.26.206.823 I slot print_timing: id 1 | task 47872 | n_decoded = 3139, tg = 20.55 t/s, tg_3s = 19.43 t/s
45.29.233.785 I slot print_timing: id 1 | task 47872 | n_decoded = 3197, tg = 20.53 t/s, tg_3s = 19.16 t/s
45.32.250.763 I slot print_timing: id 1 | task 47872 | n_decoded = 3255, tg = 20.50 t/s, tg_3s = 19.22 t/s
45.35.280.514 I slot print_timing: id 1 | task 47872 | n_decoded = 3313, tg = 20.48 t/s, tg_3s = 19.14 t/s
45.38.323.453 I slot print_timing: id 1 | task 47872 | n_decoded = 3372, tg = 20.46 t/s, tg_3s = 19.39 t/s
45.41.368.915 I slot print_timing: id 1 | task 47872 | n_decoded = 3431, tg = 20.44 t/s, tg_3s = 19.37 t/s
45.44.395.008 I slot print_timing: id 1 | task 47872 | n_decoded = 3489, tg = 20.41 t/s, tg_3s = 19.17 t/s
45.47.419.619 I slot print_timing: id 1 | task 47872 | n_decoded = 3547, tg = 20.39 t/s, tg_3s = 19.18 t/s
45.50.464.232 I slot print_timing: id 1 | task 47872 | n_decoded = 3605, tg = 20.37 t/s, tg_3s = 19.05 t/s
45.53.494.153 I slot print_timing: id 1 | task 47872 | n_decoded = 3663, tg = 20.35 t/s, tg_3s = 19.14 t/s
45.56.540.667 I slot print_timing: id 1 | task 47872 | n_decoded = 3721, tg = 20.33 t/s, tg_3s = 19.04 t/s
45.59.588.964 I slot print_timing: id 1 | task 47872 | n_decoded = 3779, tg = 20.31 t/s, tg_3s = 19.03 t/s
46.02.635.531 I slot print_timing: id 1 | task 47872 | n_decoded = 3837, tg = 20.29 t/s, tg_3s = 19.04 t/s
46.05.681.882 I slot print_timing: id 1 | task 47872 | n_decoded = 3895, tg = 20.27 t/s, tg_3s = 19.04 t/s
46.08.686.048 I slot print_timing: id 1 | task 47872 | n_decoded = 3952, tg = 20.25 t/s, tg_3s = 18.97 t/s
46.11.717.175 I slot print_timing: id 1 | task 47872 | n_decoded = 4009, tg = 20.22 t/s, tg_3s = 18.80 t/s
46.14.728.934 I slot print_timing: id 1 | task 47872 | n_decoded = 4066, tg = 20.20 t/s, tg_3s = 18.93 t/s
46.17.745.922 I slot print_timing: id 1 | task 47872 | n_decoded = 4122, tg = 20.18 t/s, tg_3s = 18.56 t/s
46.20.783.836 I slot print_timing: id 1 | task 47872 | n_decoded = 4179, tg = 20.16 t/s, tg_3s = 18.76 t/s
46.23.787.423 I slot print_timing: id 1 | task 47872 | n_decoded = 4235, tg = 20.14 t/s, tg_3s = 18.64 t/s
46.26.832.489 I slot print_timing: id 1 | task 47872 | n_decoded = 4292, tg = 20.12 t/s, tg_3s = 18.72 t/s
46.29.864.550 I slot print_timing: id 1 | task 47872 | n_decoded = 4349, tg = 20.10 t/s, tg_3s = 18.80 t/s
46.32.898.528 I slot print_timing: id 1 | task 47872 | n_decoded = 4406, tg = 20.08 t/s, tg_3s = 18.79 t/s
46.35.944.763 I slot print_timing: id 1 | task 47872 | n_decoded = 4463, tg = 20.06 t/s, tg_3s = 18.71 t/s
46.38.959.486 I slot print_timing: id 1 | task 47872 | n_decoded = 4519, tg = 20.04 t/s, tg_3s = 18.58 t/s
46.41.981.121 I slot print_timing: id 1 | task 47872 | n_decoded = 4575, tg = 20.02 t/s, tg_3s = 18.53 t/s
46.45.009.396 I slot print_timing: id 1 | task 47872 | n_decoded = 4631, tg = 20.00 t/s, tg_3s = 18.49 t/s
46.48.053.712 I slot print_timing: id 1 | task 47872 | n_decoded = 4687, tg = 19.98 t/s, tg_3s = 18.39 t/s
46.51.082.227 I slot print_timing: id 1 | task 47872 | n_decoded = 4742, tg = 19.96 t/s, tg_3s = 18.16 t/s
46.54.110.438 I slot print_timing: id 1 | task 47872 | n_decoded = 4797, tg = 19.94 t/s, tg_3s = 18.16 t/s
46.57.143.693 I slot print_timing: id 1 | task 47872 | n_decoded = 4852, tg = 19.91 t/s, tg_3s = 18.13 t/s
47.00.175.531 I slot print_timing: id 1 | task 47872 | n_decoded = 4907, tg = 19.89 t/s, tg_3s = 18.14 t/s
47.03.221.079 I slot print_timing: id 1 | task 47872 | n_decoded = 4963, tg = 19.87 t/s, tg_3s = 18.39 t/s
47.06.265.673 I slot print_timing: id 1 | task 47872 | n_decoded = 5019, tg = 19.85 t/s, tg_3s = 18.39 t/s
47.09.292.055 I slot print_timing: id 1 | task 47872 | n_decoded = 5074, tg = 19.84 t/s, tg_3s = 18.17 t/s
47.12.298.536 I slot print_timing: id 1 | task 47872 | n_decoded = 5129, tg = 19.82 t/s, tg_3s = 18.29 t/s
47.15.348.753 I slot print_timing: id 1 | task 47872 | n_decoded = 5185, tg = 19.80 t/s, tg_3s = 18.36 t/s
47.18.385.415 I slot print_timing: id 1 | task 47872 | n_decoded = 5241, tg = 19.78 t/s, tg_3s = 18.44 t/s
47.21.395.336 I slot print_timing: id 1 | task 47872 | n_decoded = 5296, tg = 19.77 t/s, tg_3s = 18.27 t/s
47.24.396.214 I slot print_timing: id 1 | task 47872 | n_decoded = 5351, tg = 19.75 t/s, tg_3s = 18.33 t/s
47.27.412.879 I slot print_timing: id 1 | task 47872 | n_decoded = 5406, tg = 19.73 t/s, tg_3s = 18.23 t/s
47.30.421.874 I slot print_timing: id 1 | task 47872 | n_decoded = 5461, tg = 19.72 t/s, tg_3s = 18.28 t/s
47.33.423.967 I slot print_timing: id 1 | task 47872 | n_decoded = 5515, tg = 19.70 t/s, tg_3s = 17.99 t/s
47.36.470.252 I slot print_timing: id 1 | task 47872 | n_decoded = 5570, tg = 19.68 t/s, tg_3s = 18.05 t/s
47.39.484.141 I slot print_timing: id 1 | task 47872 | n_decoded = 5624, tg = 19.66 t/s, tg_3s = 17.92 t/s
47.42.530.438 I slot print_timing: id 1 | task 47872 | n_decoded = 5679, tg = 19.65 t/s, tg_3s = 18.05 t/s
47.45.538.974 I slot print_timing: id 1 | task 47872 | n_decoded = 5733, tg = 19.63 t/s, tg_3s = 17.95 t/s
47.48.550.625 I slot print_timing: id 1 | task 47872 | n_decoded = 5787, tg = 19.61 t/s, tg_3s = 17.93 t/s
47.51.588.783 I slot print_timing: id 1 | task 47872 | n_decoded = 5842, tg = 19.60 t/s, tg_3s = 18.10 t/s
47.54.635.570 I slot print_timing: id 1 | task 47872 | n_decoded = 5897, tg = 19.58 t/s, tg_3s = 18.05 t/s
47.57.681.087 I slot print_timing: id 1 | task 47872 | n_decoded = 5952, tg = 19.57 t/s, tg_3s = 18.06 t/s
48.00.712.527 I slot print_timing: id 1 | task 47872 | n_decoded = 6006, tg = 19.55 t/s, tg_3s = 17.81 t/s
48.03.733.107 I slot print_timing: id 1 | task 47872 | n_decoded = 6060, tg = 19.53 t/s, tg_3s = 17.88 t/s
48.06.757.302 I slot print_timing: id 1 | task 47872 | n_decoded = 6113, tg = 19.51 t/s, tg_3s = 17.53 t/s
48.09.802.248 I slot print_timing: id 1 | task 47872 | n_decoded = 6167, tg = 19.50 t/s, tg_3s = 17.73 t/s
48.12.820.354 I slot print_timing: id 1 | task 47872 | n_decoded = 6221, tg = 19.48 t/s, tg_3s = 17.89 t/s
48.15.879.658 I slot print_timing: id 1 | task 47872 | n_decoded = 6275, tg = 19.46 t/s, tg_3s = 17.65 t/s
48.18.885.761 I slot print_timing: id 1 | task 47872 | n_decoded = 6328, tg = 19.45 t/s, tg_3s = 17.63 t/s
48.21.939.549 I slot print_timing: id 1 | task 47872 | n_decoded = 6382, tg = 19.43 t/s, tg_3s = 17.68 t/s
48.24.995.512 I slot print_timing: id 1 | task 47872 | n_decoded = 6436, tg = 19.41 t/s, tg_3s = 17.67 t/s
48.28.025.816 I slot print_timing: id 1 | task 47872 | n_decoded = 6489, tg = 19.40 t/s, tg_3s = 17.49 t/s
48.31.080.300 I slot print_timing: id 1 | task 47872 | n_decoded = 6542, tg = 19.38 t/s, tg_3s = 17.35 t/s
48.34.101.313 I slot print_timing: id 1 | task 47872 | n_decoded = 6595, tg = 19.36 t/s, tg_3s = 17.54 t/s
48.37.128.186 I slot print_timing: id 1 | task 47872 | n_decoded = 6648, tg = 19.35 t/s, tg_3s = 17.51 t/s
48.40.169.653 I slot print_timing: id 1 | task 47872 | n_decoded = 6701, tg = 19.33 t/s, tg_3s = 17.43 t/s
48.43.182.511 I slot print_timing: id 1 | task 47872 | n_decoded = 6754, tg = 19.31 t/s, tg_3s = 17.59 t/s
48.46.219.893 I slot print_timing: id 1 | task 47872 | n_decoded = 6807, tg = 19.30 t/s, tg_3s = 17.45 t/s
48.49.254.194 I slot print_timing: id 1 | task 47872 | n_decoded = 6860, tg = 19.28 t/s, tg_3s = 17.47 t/s
48.52.285.580 I slot print_timing: id 1 | task 47872 | n_decoded = 6913, tg = 19.27 t/s, tg_3s = 17.48 t/s
48.55.314.216 I slot print_timing: id 1 | task 47872 | n_decoded = 6966, tg = 19.25 t/s, tg_3s = 17.50 t/s
48.58.359.501 I slot print_timing: id 1 | task 47872 | n_decoded = 7019, tg = 19.24 t/s, tg_3s = 17.40 t/s
49.01.403.426 I slot print_timing: id 1 | task 47872 | n_decoded = 7072, tg = 19.22 t/s, tg_3s = 17.41 t/s
49.04.427.853 I slot print_timing: id 1 | task 47872 | n_decoded = 7124, tg = 19.20 t/s, tg_3s = 17.19 t/s
49.07.456.158 I slot print_timing: id 1 | task 47872 | n_decoded = 7177, tg = 19.19 t/s, tg_3s = 17.50 t/s
49.10.477.723 I slot print_timing: id 1 | task 47872 | n_decoded = 7230, tg = 19.18 t/s, tg_3s = 17.54 t/s
49.13.501.839 I slot print_timing: id 1 | task 47872 | n_decoded = 7282, tg = 19.16 t/s, tg_3s = 17.20 t/s
49.16.545.662 I slot print_timing: id 1 | task 47872 | n_decoded = 7334, tg = 19.15 t/s, tg_3s = 17.08 t/s
49.19.600.051 I slot print_timing: id 1 | task 47872 | n_decoded = 7386, tg = 19.13 t/s, tg_3s = 17.02 t/s
49.22.658.183 I slot print_timing: id 1 | task 47872 | n_decoded = 7438, tg = 19.11 t/s, tg_3s = 17.00 t/s
49.25.673.071 I slot print_timing: id 1 | task 47872 | n_decoded = 7489, tg = 19.10 t/s, tg_3s = 16.92 t/s
49.28.680.769 I slot print_timing: id 1 | task 47872 | n_decoded = 7540, tg = 19.08 t/s, tg_3s = 16.96 t/s
49.31.687.682 I slot print_timing: id 1 | task 47872 | n_decoded = 7591, tg = 19.06 t/s, tg_3s = 16.96 t/s
49.34.713.089 I slot print_timing: id 1 | task 47872 | n_decoded = 7642, tg = 19.05 t/s, tg_3s = 16.86 t/s
49.37.733.526 I slot print_timing: id 1 | task 47872 | n_decoded = 7693, tg = 19.03 t/s, tg_3s = 16.88 t/s
49.40.764.893 I slot print_timing: id 1 | task 47872 | n_decoded = 7745, tg = 19.02 t/s, tg_3s = 17.15 t/s
49.43.798.884 I slot print_timing: id 1 | task 47872 | n_decoded = 7797, tg = 19.00 t/s, tg_3s = 17.14 t/s
49.46.803.954 I slot print_timing: id 1 | task 47872 | n_decoded = 7848, tg = 18.99 t/s, tg_3s = 16.97 t/s
49.49.851.648 I slot print_timing: id 1 | task 47872 | n_decoded = 7900, tg = 18.97 t/s, tg_3s = 17.06 t/s
49.52.903.345 I slot print_timing: id 1 | task 47872 | n_decoded = 7952, tg = 18.96 t/s, tg_3s = 17.04 t/s
49.55.908.127 I slot print_timing: id 1 | task 47872 | n_decoded = 8003, tg = 18.95 t/s, tg_3s = 16.97 t/s
49.58.947.055 I slot print_timing: id 1 | task 47872 | n_decoded = 8054, tg = 18.93 t/s, tg_3s = 16.78 t/s
50.01.964.404 I slot print_timing: id 1 | task 47872 | n_decoded = 8104, tg = 18.91 t/s, tg_3s = 16.57 t/s
50.04.978.437 I slot print_timing: id 1 | task 47872 | n_decoded = 8155, tg = 18.90 t/s, tg_3s = 16.92 t/s
50.07.169.244 I slot print_timing: id 1 | task 47872 | prompt eval time = 1841.60 ms / 65 tokens ( 28.33 ms per token, 35.30 tokens per second)
50.07.169.248 I slot print_timing: id 1 | task 47872 | eval time = 433686.51 ms / 8192 tokens ( 52.94 ms per token, 18.89 tokens per second)
50.07.169.249 I slot print_timing: id 1 | task 47872 | total time = 435528.10 ms / 8257 tokens
50.07.169.250 I slot print_timing: id 1 | task 47872 | graphs reused = 55791
50.07.169.282 I slot release: id 1 | task 47872 | stop processing: n_tokens = 8319, truncated = 0
50.07.169.330 I srv update_slots: all slots are idle
50.07.508.562 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
50.07.525.932 I srv operator(): Chat format: peg-native
50.07.531.463 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.618 (> 0.100 thold), f_keep = 0.008
50.07.531.464 I srv get_availabl: updating prompt cache
50.07.531.710 W srv prompt_save: - saving prompt with length 8319, total state size = 2014.854 MiB (draft: 0.000 MiB)
50.08.726.428 I srv load: - looking for better prompt, base f_keep = 0.008, sim = 0.618
50.08.726.434 W srv update: - cache size limit reached, removing oldest entry (size = 2007.831 MiB)
50.08.886.780 I srv update: - cache state: 9 prompts, 7138.585 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
50.08.886.785 I srv update: - prompt 0x6354c2cffe70: 655 tokens, checkpoints: 0, 158.642 MiB
50.08.886.786 I srv update: - prompt 0x6354ca9ab8c0: 803 tokens, checkpoints: 0, 194.487 MiB
50.08.886.786 I srv update: - prompt 0x6354c2a6f0c0: 8320 tokens, checkpoints: 0, 2015.097 MiB
50.08.886.787 I srv update: - prompt 0x6354c2a2a820: 627 tokens, checkpoints: 0, 151.860 MiB
50.08.886.788 I srv update: - prompt 0x6354c9e731d0: 8310 tokens, checkpoints: 0, 2012.675 MiB
50.08.886.789 I srv update: - prompt 0x6354c2c51b40: 654 tokens, checkpoints: 0, 158.400 MiB
50.08.886.789 I srv update: - prompt 0x6354c2ecfe50: 590 tokens, checkpoints: 0, 142.899 MiB
50.08.886.790 I srv update: - prompt 0x6354c9ddc610: 1196 tokens, checkpoints: 0, 289.671 MiB
50.08.886.791 I srv update: - prompt 0x6354c2ae9ff0: 8319 tokens, checkpoints: 0, 2014.854 MiB
50.08.886.793 I srv get_availabl: prompt cache update took 1355.33 ms
50.08.888.785 I slot launch_slot_: id 1 | task 56065 | processing task, is_child = 0
50.08.888.787 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
50.08.888.788 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
50.08.888.823 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
50.08.888.824 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
50.08.888.841 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
50.08.888.841 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
50.14.962.921 I slot print_timing: id 1 | task 56065 | n_decoded = 100, tg = 21.80 t/s, tg_3s = 21.80 t/s
50.17.967.477 I slot print_timing: id 1 | task 56065 | n_decoded = 165, tg = 21.73 t/s, tg_3s = 21.63 t/s
50.20.983.059 I slot print_timing: id 1 | task 56065 | n_decoded = 230, tg = 21.68 t/s, tg_3s = 21.55 t/s
50.24.019.432 I slot print_timing: id 1 | task 56065 | n_decoded = 295, tg = 21.62 t/s, tg_3s = 21.41 t/s
50.27.061.903 I slot print_timing: id 1 | task 56065 | n_decoded = 361, tg = 21.63 t/s, tg_3s = 21.69 t/s
50.30.078.168 I slot print_timing: id 1 | task 56065 | n_decoded = 426, tg = 21.62 t/s, tg_3s = 21.55 t/s
50.33.085.732 I slot print_timing: id 1 | task 56065 | n_decoded = 491, tg = 21.62 t/s, tg_3s = 21.61 t/s
50.36.130.392 I slot print_timing: id 1 | task 56065 | n_decoded = 556, tg = 21.59 t/s, tg_3s = 21.35 t/s
50.39.142.409 I slot print_timing: id 1 | task 56065 | n_decoded = 621, tg = 21.59 t/s, tg_3s = 21.58 t/s
50.40.996.686 I slot print_timing: id 1 | task 56065 | prompt eval time = 1486.49 ms / 39 tokens ( 38.12 ms per token, 26.24 tokens per second)
50.40.996.689 I slot print_timing: id 1 | task 56065 | eval time = 30621.31 ms / 661 tokens ( 46.33 ms per token, 21.59 tokens per second)
50.40.996.690 I slot print_timing: id 1 | task 56065 | total time = 32107.81 ms / 700 tokens
50.40.996.691 I slot print_timing: id 1 | task 56065 | graphs reused = 56448
50.40.996.724 I slot release: id 1 | task 56065 | stop processing: n_tokens = 762, truncated = 0
50.40.996.749 I srv update_slots: all slots are idle
50.41.042.142 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
50.41.061.577 I srv operator(): Chat format: peg-native
50.41.067.622 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.513 (> 0.100 thold), f_keep = 0.076
50.41.067.624 I srv get_availabl: updating prompt cache
50.41.067.691 W srv prompt_save: - saving prompt with length 762, total state size = 184.557 MiB (draft: 0.000 MiB)
50.41.193.772 I srv load: - looking for better prompt, base f_keep = 0.076, sim = 0.513
50.41.193.781 I srv update: - cache state: 10 prompts, 7323.142 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
50.41.193.781 I srv update: - prompt 0x6354c2cffe70: 655 tokens, checkpoints: 0, 158.642 MiB
50.41.193.789 I srv update: - prompt 0x6354ca9ab8c0: 803 tokens, checkpoints: 0, 194.487 MiB
50.41.193.790 I srv update: - prompt 0x6354c2a6f0c0: 8320 tokens, checkpoints: 0, 2015.097 MiB
50.41.193.790 I srv update: - prompt 0x6354c2a2a820: 627 tokens, checkpoints: 0, 151.860 MiB
50.41.193.791 I srv update: - prompt 0x6354c9e731d0: 8310 tokens, checkpoints: 0, 2012.675 MiB
50.41.193.791 I srv update: - prompt 0x6354c2c51b40: 654 tokens, checkpoints: 0, 158.400 MiB
50.41.193.792 I srv update: - prompt 0x6354c2ecfe50: 590 tokens, checkpoints: 0, 142.899 MiB
50.41.193.792 I srv update: - prompt 0x6354c9ddc610: 1196 tokens, checkpoints: 0, 289.671 MiB
50.41.193.793 I srv update: - prompt 0x6354c2ae9ff0: 8319 tokens, checkpoints: 0, 2014.854 MiB
50.41.193.793 I srv update: - prompt 0x6354c2a52e50: 762 tokens, checkpoints: 0, 184.557 MiB
50.41.193.794 I srv get_availabl: prompt cache update took 126.17 ms
50.41.195.449 I slot launch_slot_: id 1 | task 56727 | processing task, is_child = 0
50.41.195.451 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
50.41.195.452 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
50.41.195.467 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
50.41.195.468 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
50.41.195.482 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
50.41.195.483 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
50.47.684.969 I slot print_timing: id 1 | task 56727 | n_decoded = 100, tg = 21.89 t/s, tg_3s = 21.89 t/s
50.50.691.763 I slot print_timing: id 1 | task 56727 | n_decoded = 164, tg = 21.65 t/s, tg_3s = 21.29 t/s
50.53.456.160 I slot print_timing: id 1 | task 56727 | prompt eval time = 1921.34 ms / 55 tokens ( 34.93 ms per token, 28.63 tokens per second)
50.53.456.164 I slot print_timing: id 1 | task 56727 | eval time = 10339.30 ms / 224 tokens ( 46.16 ms per token, 21.66 tokens per second)
50.53.456.165 I slot print_timing: id 1 | task 56727 | total time = 12260.64 ms / 279 tokens
50.53.456.165 I slot print_timing: id 1 | task 56727 | graphs reused = 56669
50.53.456.195 I slot release: id 1 | task 56727 | stop processing: n_tokens = 336, truncated = 0
50.53.456.207 I srv update_slots: all slots are idle
50.53.457.184 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
50.53.475.894 I srv operator(): Chat format: peg-native
50.53.481.463 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.580 (> 0.100 thold), f_keep = 0.173
50.53.481.465 I srv get_availabl: updating prompt cache
50.53.481.508 W srv prompt_save: - saving prompt with length 336, total state size = 81.380 MiB (draft: 0.000 MiB)
50.53.539.395 I srv load: - looking for better prompt, base f_keep = 0.173, sim = 0.580
50.53.539.407 I srv update: - cache state: 11 prompts, 7404.522 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
50.53.539.408 I srv update: - prompt 0x6354c2cffe70: 655 tokens, checkpoints: 0, 158.642 MiB
50.53.539.408 I srv update: - prompt 0x6354ca9ab8c0: 803 tokens, checkpoints: 0, 194.487 MiB
50.53.539.409 I srv update: - prompt 0x6354c2a6f0c0: 8320 tokens, checkpoints: 0, 2015.097 MiB
50.53.539.411 I srv update: - prompt 0x6354c2a2a820: 627 tokens, checkpoints: 0, 151.860 MiB
50.53.539.412 I srv update: - prompt 0x6354c9e731d0: 8310 tokens, checkpoints: 0, 2012.675 MiB
50.53.539.412 I srv update: - prompt 0x6354c2c51b40: 654 tokens, checkpoints: 0, 158.400 MiB
50.53.539.413 I srv update: - prompt 0x6354c2ecfe50: 590 tokens, checkpoints: 0, 142.899 MiB
50.53.539.413 I srv update: - prompt 0x6354c9ddc610: 1196 tokens, checkpoints: 0, 289.671 MiB
50.53.539.414 I srv update: - prompt 0x6354c2ae9ff0: 8319 tokens, checkpoints: 0, 2014.854 MiB
50.53.539.414 I srv update: - prompt 0x6354c2a52e50: 762 tokens, checkpoints: 0, 184.557 MiB
50.53.539.415 I srv update: - prompt 0x6354c2cef280: 336 tokens, checkpoints: 0, 81.380 MiB
50.53.539.416 I srv get_availabl: prompt cache update took 57.95 ms
50.53.541.098 I slot launch_slot_: id 1 | task 56952 | processing task, is_child = 0
50.53.541.101 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
50.53.541.102 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
50.53.541.124 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
50.53.541.124 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
50.53.541.139 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
50.53.541.140 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
50.59.735.875 I slot print_timing: id 1 | task 56952 | n_decoded = 100, tg = 21.86 t/s, tg_3s = 21.86 t/s
51.02.741.105 I slot print_timing: id 1 | task 56952 | n_decoded = 165, tg = 21.77 t/s, tg_3s = 21.63 t/s
51.05.780.146 I slot print_timing: id 1 | task 56952 | n_decoded = 231, tg = 21.75 t/s, tg_3s = 21.72 t/s
51.08.820.390 I slot print_timing: id 1 | task 56952 | n_decoded = 297, tg = 21.74 t/s, tg_3s = 21.71 t/s
51.11.861.570 I slot print_timing: id 1 | task 56952 | n_decoded = 363, tg = 21.74 t/s, tg_3s = 21.70 t/s
51.14.866.433 I slot print_timing: id 1 | task 56952 | n_decoded = 428, tg = 21.72 t/s, tg_3s = 21.63 t/s
51.17.911.511 I slot print_timing: id 1 | task 56952 | n_decoded = 494, tg = 21.71 t/s, tg_3s = 21.67 t/s
51.20.914.695 I slot print_timing: id 1 | task 56952 | n_decoded = 559, tg = 21.71 t/s, tg_3s = 21.64 t/s
51.23.931.706 I slot print_timing: id 1 | task 56952 | n_decoded = 624, tg = 21.69 t/s, tg_3s = 21.54 t/s
51.26.960.369 I slot print_timing: id 1 | task 56952 | n_decoded = 689, tg = 21.67 t/s, tg_3s = 21.46 t/s
51.29.964.053 I slot print_timing: id 1 | task 56952 | n_decoded = 753, tg = 21.64 t/s, tg_3s = 21.31 t/s
51.32.970.115 I slot print_timing: id 1 | task 56952 | n_decoded = 817, tg = 21.61 t/s, tg_3s = 21.29 t/s
51.35.979.062 I slot print_timing: id 1 | task 56952 | n_decoded = 881, tg = 21.58 t/s, tg_3s = 21.27 t/s
51.39.002.465 I slot print_timing: id 1 | task 56952 | n_decoded = 945, tg = 21.56 t/s, tg_3s = 21.17 t/s
51.42.016.443 I slot print_timing: id 1 | task 56952 | n_decoded = 1009, tg = 21.53 t/s, tg_3s = 21.23 t/s
51.45.027.461 I slot print_timing: id 1 | task 56952 | n_decoded = 1073, tg = 21.52 t/s, tg_3s = 21.26 t/s
51.48.040.469 I slot print_timing: id 1 | task 56952 | n_decoded = 1136, tg = 21.48 t/s, tg_3s = 20.91 t/s
51.51.068.351 I slot print_timing: id 1 | task 56952 | n_decoded = 1200, tg = 21.46 t/s, tg_3s = 21.14 t/s
51.54.078.721 I slot print_timing: id 1 | task 56952 | n_decoded = 1264, tg = 21.45 t/s, tg_3s = 21.26 t/s
51.57.087.306 I slot print_timing: id 1 | task 56952 | n_decoded = 1328, tg = 21.44 t/s, tg_3s = 21.27 t/s
52.00.105.071 I slot print_timing: id 1 | task 56952 | n_decoded = 1392, tg = 21.43 t/s, tg_3s = 21.21 t/s
52.03.124.302 I slot print_timing: id 1 | task 56952 | n_decoded = 1456, tg = 21.42 t/s, tg_3s = 21.20 t/s
52.06.145.281 I slot print_timing: id 1 | task 56952 | n_decoded = 1520, tg = 21.41 t/s, tg_3s = 21.19 t/s
52.09.187.863 I slot print_timing: id 1 | task 56952 | n_decoded = 1584, tg = 21.40 t/s, tg_3s = 21.03 t/s
52.12.193.670 I slot print_timing: id 1 | task 56952 | n_decoded = 1647, tg = 21.38 t/s, tg_3s = 20.96 t/s
52.15.211.650 I slot print_timing: id 1 | task 56952 | n_decoded = 1710, tg = 21.36 t/s, tg_3s = 20.87 t/s
52.18.219.144 I slot print_timing: id 1 | task 56952 | n_decoded = 1771, tg = 21.32 t/s, tg_3s = 20.28 t/s
52.21.225.326 I slot print_timing: id 1 | task 56952 | n_decoded = 1832, tg = 21.29 t/s, tg_3s = 20.29 t/s
52.24.229.307 I slot print_timing: id 1 | task 56952 | n_decoded = 1893, tg = 21.25 t/s, tg_3s = 20.31 t/s
52.27.249.790 I slot print_timing: id 1 | task 56952 | n_decoded = 1954, tg = 21.22 t/s, tg_3s = 20.20 t/s
52.30.284.265 I slot print_timing: id 1 | task 56952 | n_decoded = 2015, tg = 21.18 t/s, tg_3s = 20.10 t/s
52.33.317.568 I slot print_timing: id 1 | task 56952 | n_decoded = 2076, tg = 21.15 t/s, tg_3s = 20.11 t/s
52.36.356.074 I slot print_timing: id 1 | task 56952 | n_decoded = 2137, tg = 21.12 t/s, tg_3s = 20.08 t/s
52.39.382.547 I slot print_timing: id 1 | task 56952 | n_decoded = 2197, tg = 21.08 t/s, tg_3s = 19.83 t/s
52.42.422.641 I slot print_timing: id 1 | task 56952 | n_decoded = 2257, tg = 21.04 t/s, tg_3s = 19.74 t/s
52.45.434.069 I slot print_timing: id 1 | task 56952 | n_decoded = 2317, tg = 21.01 t/s, tg_3s = 19.92 t/s
52.48.447.640 I slot print_timing: id 1 | task 56952 | n_decoded = 2377, tg = 20.98 t/s, tg_3s = 19.91 t/s
52.51.487.071 I slot print_timing: id 1 | task 56952 | n_decoded = 2437, tg = 20.95 t/s, tg_3s = 19.74 t/s
52.54.519.083 I slot print_timing: id 1 | task 56952 | n_decoded = 2497, tg = 20.92 t/s, tg_3s = 19.79 t/s
52.57.558.705 I slot print_timing: id 1 | task 56952 | n_decoded = 2557, tg = 20.89 t/s, tg_3s = 19.74 t/s
53.00.581.414 I slot print_timing: id 1 | task 56952 | n_decoded = 2616, tg = 20.86 t/s, tg_3s = 19.52 t/s
53.03.621.341 I slot print_timing: id 1 | task 56952 | n_decoded = 2676, tg = 20.83 t/s, tg_3s = 19.74 t/s
53.06.669.728 I slot print_timing: id 1 | task 56952 | n_decoded = 2735, tg = 20.80 t/s, tg_3s = 19.35 t/s
53.09.703.459 I slot print_timing: id 1 | task 56952 | n_decoded = 2794, tg = 20.77 t/s, tg_3s = 19.45 t/s
53.12.724.689 I slot print_timing: id 1 | task 56952 | n_decoded = 2853, tg = 20.74 t/s, tg_3s = 19.53 t/s
53.15.741.695 I slot print_timing: id 1 | task 56952 | n_decoded = 2912, tg = 20.71 t/s, tg_3s = 19.56 t/s
53.18.774.462 I slot print_timing: id 1 | task 56952 | n_decoded = 2970, tg = 20.68 t/s, tg_3s = 19.12 t/s
53.21.807.855 I slot print_timing: id 1 | task 56952 | n_decoded = 3028, tg = 20.65 t/s, tg_3s = 19.12 t/s
53.24.810.254 I slot print_timing: id 1 | task 56952 | n_decoded = 3086, tg = 20.62 t/s, tg_3s = 19.32 t/s
53.27.840.155 I slot print_timing: id 1 | task 56952 | n_decoded = 3144, tg = 20.59 t/s, tg_3s = 19.14 t/s
53.30.847.052 I slot print_timing: id 1 | task 56952 | n_decoded = 3202, tg = 20.57 t/s, tg_3s = 19.29 t/s
53.33.863.625 I slot print_timing: id 1 | task 56952 | n_decoded = 3260, tg = 20.54 t/s, tg_3s = 19.23 t/s
53.36.909.200 I slot print_timing: id 1 | task 56952 | n_decoded = 3319, tg = 20.52 t/s, tg_3s = 19.37 t/s
53.39.951.867 I slot print_timing: id 1 | task 56952 | n_decoded = 3378, tg = 20.50 t/s, tg_3s = 19.39 t/s
53.42.954.405 I slot print_timing: id 1 | task 56952 | n_decoded = 3436, tg = 20.48 t/s, tg_3s = 19.32 t/s
53.45.972.309 I slot print_timing: id 1 | task 56952 | n_decoded = 3494, tg = 20.46 t/s, tg_3s = 19.22 t/s
53.48.993.042 I slot print_timing: id 1 | task 56952 | n_decoded = 3552, tg = 20.43 t/s, tg_3s = 19.20 t/s
53.52.038.109 I slot print_timing: id 1 | task 56952 | n_decoded = 3610, tg = 20.41 t/s, tg_3s = 19.05 t/s
53.55.060.123 I slot print_timing: id 1 | task 56952 | n_decoded = 3668, tg = 20.39 t/s, tg_3s = 19.19 t/s
53.58.082.104 I slot print_timing: id 1 | task 56952 | n_decoded = 3726, tg = 20.37 t/s, tg_3s = 19.19 t/s
54.01.130.822 I slot print_timing: id 1 | task 56952 | n_decoded = 3784, tg = 20.35 t/s, tg_3s = 19.02 t/s
54.04.130.951 I slot print_timing: id 1 | task 56952 | n_decoded = 3841, tg = 20.33 t/s, tg_3s = 19.00 t/s
54.07.177.320 I slot print_timing: id 1 | task 56952 | n_decoded = 3899, tg = 20.31 t/s, tg_3s = 19.04 t/s
54.10.222.326 I slot print_timing: id 1 | task 56952 | n_decoded = 3957, tg = 20.29 t/s, tg_3s = 19.05 t/s
54.13.247.026 I slot print_timing: id 1 | task 56952 | n_decoded = 4014, tg = 20.26 t/s, tg_3s = 18.84 t/s
54.16.275.252 I slot print_timing: id 1 | task 56952 | n_decoded = 4070, tg = 20.24 t/s, tg_3s = 18.49 t/s
54.19.310.083 I slot print_timing: id 1 | task 56952 | n_decoded = 4127, tg = 20.22 t/s, tg_3s = 18.78 t/s
54.22.326.943 I slot print_timing: id 1 | task 56952 | n_decoded = 4184, tg = 20.20 t/s, tg_3s = 18.89 t/s
54.25.345.118 I slot print_timing: id 1 | task 56952 | n_decoded = 4241, tg = 20.18 t/s, tg_3s = 18.89 t/s
54.28.396.883 I slot print_timing: id 1 | task 56952 | n_decoded = 4298, tg = 20.16 t/s, tg_3s = 18.68 t/s
54.31.437.445 I slot print_timing: id 1 | task 56952 | n_decoded = 4355, tg = 20.14 t/s, tg_3s = 18.75 t/s
54.34.473.622 I slot print_timing: id 1 | task 56952 | n_decoded = 4412, tg = 20.12 t/s, tg_3s = 18.77 t/s
54.37.502.974 I slot print_timing: id 1 | task 56952 | n_decoded = 4469, tg = 20.10 t/s, tg_3s = 18.82 t/s
54.40.554.706 I slot print_timing: id 1 | task 56952 | n_decoded = 4526, tg = 20.08 t/s, tg_3s = 18.68 t/s
54.43.579.962 I slot print_timing: id 1 | task 56952 | n_decoded = 4582, tg = 20.06 t/s, tg_3s = 18.51 t/s
54.46.609.990 I slot print_timing: id 1 | task 56952 | n_decoded = 4638, tg = 20.04 t/s, tg_3s = 18.48 t/s
54.49.625.102 I slot print_timing: id 1 | task 56952 | n_decoded = 4693, tg = 20.02 t/s, tg_3s = 18.24 t/s
54.52.638.519 I slot print_timing: id 1 | task 56952 | n_decoded = 4748, tg = 19.99 t/s, tg_3s = 18.25 t/s
54.55.652.497 I slot print_timing: id 1 | task 56952 | n_decoded = 4803, tg = 19.97 t/s, tg_3s = 18.25 t/s
54.58.702.116 I slot print_timing: id 1 | task 56952 | n_decoded = 4859, tg = 19.95 t/s, tg_3s = 18.36 t/s
55.01.744.704 I slot print_timing: id 1 | task 56952 | n_decoded = 4915, tg = 19.93 t/s, tg_3s = 18.41 t/s
55.04.783.672 I slot print_timing: id 1 | task 56952 | n_decoded = 4971, tg = 19.91 t/s, tg_3s = 18.43 t/s
55.07.808.814 I slot print_timing: id 1 | task 56952 | n_decoded = 5026, tg = 19.89 t/s, tg_3s = 18.18 t/s
55.10.851.054 I slot print_timing: id 1 | task 56952 | n_decoded = 5082, tg = 19.88 t/s, tg_3s = 18.41 t/s
55.13.896.094 I slot print_timing: id 1 | task 56952 | n_decoded = 5138, tg = 19.86 t/s, tg_3s = 18.39 t/s
55.16.933.722 I slot print_timing: id 1 | task 56952 | n_decoded = 5194, tg = 19.84 t/s, tg_3s = 18.44 t/s
55.19.965.235 I slot print_timing: id 1 | task 56952 | n_decoded = 5250, tg = 19.83 t/s, tg_3s = 18.47 t/s
55.23.019.535 I slot print_timing: id 1 | task 56952 | n_decoded = 5306, tg = 19.81 t/s, tg_3s = 18.33 t/s
55.26.028.375 I slot print_timing: id 1 | task 56952 | n_decoded = 5360, tg = 19.79 t/s, tg_3s = 17.95 t/s
55.29.073.709 I slot print_timing: id 1 | task 56952 | n_decoded = 5415, tg = 19.77 t/s, tg_3s = 18.06 t/s
55.32.127.365 I slot print_timing: id 1 | task 56952 | n_decoded = 5471, tg = 19.75 t/s, tg_3s = 18.34 t/s
55.35.134.674 I slot print_timing: id 1 | task 56952 | n_decoded = 5526, tg = 19.74 t/s, tg_3s = 18.29 t/s
55.38.184.956 I slot print_timing: id 1 | task 56952 | n_decoded = 5581, tg = 19.72 t/s, tg_3s = 18.03 t/s
55.41.231.437 I slot print_timing: id 1 | task 56952 | n_decoded = 5636, tg = 19.70 t/s, tg_3s = 18.05 t/s
55.44.236.489 I slot print_timing: id 1 | task 56952 | n_decoded = 5690, tg = 19.68 t/s, tg_3s = 17.97 t/s
55.47.280.985 I slot print_timing: id 1 | task 56952 | n_decoded = 5745, tg = 19.67 t/s, tg_3s = 18.07 t/s
55.50.282.809 I slot print_timing: id 1 | task 56952 | n_decoded = 5799, tg = 19.65 t/s, tg_3s = 17.99 t/s
55.53.302.377 I slot print_timing: id 1 | task 56952 | n_decoded = 5853, tg = 19.63 t/s, tg_3s = 17.88 t/s
55.56.331.212 I slot print_timing: id 1 | task 56952 | n_decoded = 5907, tg = 19.61 t/s, tg_3s = 17.83 t/s
55.59.335.825 I slot print_timing: id 1 | task 56952 | n_decoded = 5960, tg = 19.59 t/s, tg_3s = 17.64 t/s
56.02.348.084 I slot print_timing: id 1 | task 56952 | n_decoded = 6014, tg = 19.58 t/s, tg_3s = 17.93 t/s
56.05.367.545 I slot print_timing: id 1 | task 56952 | n_decoded = 6068, tg = 19.56 t/s, tg_3s = 17.88 t/s
56.08.405.268 I slot print_timing: id 1 | task 56952 | n_decoded = 6122, tg = 19.54 t/s, tg_3s = 17.78 t/s
56.11.425.734 I slot print_timing: id 1 | task 56952 | n_decoded = 6176, tg = 19.53 t/s, tg_3s = 17.88 t/s
56.14.438.397 I slot print_timing: id 1 | task 56952 | n_decoded = 6229, tg = 19.51 t/s, tg_3s = 17.59 t/s
56.17.480.997 I slot print_timing: id 1 | task 56952 | n_decoded = 6282, tg = 19.49 t/s, tg_3s = 17.42 t/s
56.20.535.802 I slot print_timing: id 1 | task 56952 | n_decoded = 6336, tg = 19.47 t/s, tg_3s = 17.68 t/s
56.23.557.286 I slot print_timing: id 1 | task 56952 | n_decoded = 6389, tg = 19.46 t/s, tg_3s = 17.54 t/s
56.26.564.513 I slot print_timing: id 1 | task 56952 | n_decoded = 6442, tg = 19.44 t/s, tg_3s = 17.62 t/s
56.29.611.596 I slot print_timing: id 1 | task 56952 | n_decoded = 6496, tg = 19.42 t/s, tg_3s = 17.72 t/s
56.32.616.721 I slot print_timing: id 1 | task 56952 | n_decoded = 6549, tg = 19.41 t/s, tg_3s = 17.64 t/s
56.35.627.829 I slot print_timing: id 1 | task 56952 | n_decoded = 6601, tg = 19.39 t/s, tg_3s = 17.27 t/s
56.38.651.959 I slot print_timing: id 1 | task 56952 | n_decoded = 6653, tg = 19.37 t/s, tg_3s = 17.20 t/s
56.41.700.407 I slot print_timing: id 1 | task 56952 | n_decoded = 6706, tg = 19.35 t/s, tg_3s = 17.39 t/s
56.44.721.233 I slot print_timing: id 1 | task 56952 | n_decoded = 6759, tg = 19.34 t/s, tg_3s = 17.54 t/s
56.47.772.135 I slot print_timing: id 1 | task 56952 | n_decoded = 6812, tg = 19.32 t/s, tg_3s = 17.37 t/s
56.50.773.772 I slot print_timing: id 1 | task 56952 | n_decoded = 6863, tg = 19.30 t/s, tg_3s = 16.99 t/s
56.53.781.524 I slot print_timing: id 1 | task 56952 | n_decoded = 6915, tg = 19.28 t/s, tg_3s = 17.29 t/s
56.56.835.134 I slot print_timing: id 1 | task 56952 | n_decoded = 6968, tg = 19.27 t/s, tg_3s = 17.36 t/s
56.59.869.596 I slot print_timing: id 1 | task 56952 | n_decoded = 7021, tg = 19.25 t/s, tg_3s = 17.47 t/s
57.02.873.502 I slot print_timing: id 1 | task 56952 | n_decoded = 7073, tg = 19.24 t/s, tg_3s = 17.31 t/s
57.05.914.821 I slot print_timing: id 1 | task 56952 | n_decoded = 7125, tg = 19.22 t/s, tg_3s = 17.10 t/s
57.08.942.481 I slot print_timing: id 1 | task 56952 | n_decoded = 7178, tg = 19.20 t/s, tg_3s = 17.51 t/s
57.11.965.621 I slot print_timing: id 1 | task 56952 | n_decoded = 7231, tg = 19.19 t/s, tg_3s = 17.53 t/s
57.15.000.876 I slot print_timing: id 1 | task 56952 | n_decoded = 7284, tg = 19.18 t/s, tg_3s = 17.46 t/s
57.18.048.755 I slot print_timing: id 1 | task 56952 | n_decoded = 7337, tg = 19.16 t/s, tg_3s = 17.39 t/s
57.21.083.938 I slot print_timing: id 1 | task 56952 | n_decoded = 7389, tg = 19.15 t/s, tg_3s = 17.13 t/s
57.24.130.042 I slot print_timing: id 1 | task 56952 | n_decoded = 7440, tg = 19.13 t/s, tg_3s = 16.74 t/s
57.27.144.777 I slot print_timing: id 1 | task 56952 | n_decoded = 7491, tg = 19.11 t/s, tg_3s = 16.92 t/s
57.30.152.693 I slot print_timing: id 1 | task 56952 | n_decoded = 7542, tg = 19.09 t/s, tg_3s = 16.96 t/s
57.33.205.895 I slot print_timing: id 1 | task 56952 | n_decoded = 7594, tg = 19.08 t/s, tg_3s = 17.03 t/s
57.36.237.780 I slot print_timing: id 1 | task 56952 | n_decoded = 7646, tg = 19.06 t/s, tg_3s = 17.15 t/s
57.39.276.123 I slot print_timing: id 1 | task 56952 | n_decoded = 7698, tg = 19.05 t/s, tg_3s = 17.11 t/s
57.42.315.833 I slot print_timing: id 1 | task 56952 | n_decoded = 7750, tg = 19.03 t/s, tg_3s = 17.11 t/s
57.45.333.838 I slot print_timing: id 1 | task 56952 | n_decoded = 7801, tg = 19.02 t/s, tg_3s = 16.90 t/s
57.48.361.069 I slot print_timing: id 1 | task 56952 | n_decoded = 7852, tg = 19.00 t/s, tg_3s = 16.85 t/s
57.51.406.103 I slot print_timing: id 1 | task 56952 | n_decoded = 7904, tg = 18.99 t/s, tg_3s = 17.08 t/s
57.54.407.909 I slot print_timing: id 1 | task 56952 | n_decoded = 7955, tg = 18.97 t/s, tg_3s = 16.99 t/s
57.57.410.245 I slot print_timing: id 1 | task 56952 | n_decoded = 8006, tg = 18.96 t/s, tg_3s = 16.99 t/s
58.00.466.118 I slot print_timing: id 1 | task 56952 | n_decoded = 8058, tg = 18.95 t/s, tg_3s = 17.02 t/s
58.03.512.286 I slot print_timing: id 1 | task 56952 | n_decoded = 8109, tg = 18.93 t/s, tg_3s = 16.74 t/s
58.06.512.688 I slot print_timing: id 1 | task 56952 | n_decoded = 8159, tg = 18.91 t/s, tg_3s = 16.66 t/s
58.08.491.386 I slot print_timing: id 1 | task 56952 | prompt eval time = 1620.06 ms / 42 tokens ( 38.57 ms per token, 25.93 tokens per second)
58.08.491.393 I slot print_timing: id 1 | task 56952 | eval time = 433330.15 ms / 8192 tokens ( 52.90 ms per token, 18.90 tokens per second)
58.08.491.394 I slot print_timing: id 1 | task 56952 | total time = 434950.21 ms / 8234 tokens
58.08.491.395 I slot print_timing: id 1 | task 56952 | graphs reused = 64827
58.08.491.428 I slot release: id 1 | task 56952 | stop processing: n_tokens = 8291, truncated = 0
58.08.491.472 I srv update_slots: all slots are idle
58.08.807.912 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
58.08.827.683 I srv operator(): Chat format: peg-native
58.08.833.759 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.630 (> 0.100 thold), f_keep = 0.008
58.08.833.762 I srv get_availabl: updating prompt cache
58.08.834.067 W srv prompt_save: - saving prompt with length 8291, total state size = 2008.073 MiB (draft: 0.000 MiB)
58.10.024.264 I srv load: - looking for better prompt, base f_keep = 0.008, sim = 0.630
58.10.024.272 W srv update: - cache size limit reached, removing oldest entry (size = 158.642 MiB)
58.10.041.886 W srv update: - cache size limit reached, removing oldest entry (size = 194.487 MiB)
58.10.064.766 W srv update: - cache size limit reached, removing oldest entry (size = 2015.097 MiB)
58.10.222.815 I srv update: - cache state: 9 prompts, 7044.369 MiB (limits: 8192.000 MiB, 24576 tokens, 33823 est)
58.10.222.819 I srv update: - prompt 0x6354c2a2a820: 627 tokens, checkpoints: 0, 151.860 MiB
58.10.222.820 I srv update: - prompt 0x6354c9e731d0: 8310 tokens, checkpoints: 0, 2012.675 MiB
58.10.222.821 I srv update: - prompt 0x6354c2c51b40: 654 tokens, checkpoints: 0, 158.400 MiB
58.10.222.821 I srv update: - prompt 0x6354c2ecfe50: 590 tokens, checkpoints: 0, 142.899 MiB
58.10.222.822 I srv update: - prompt 0x6354c9ddc610: 1196 tokens, checkpoints: 0, 289.671 MiB
58.10.222.823 I srv update: - prompt 0x6354c2ae9ff0: 8319 tokens, checkpoints: 0, 2014.854 MiB
58.10.222.823 I srv update: - prompt 0x6354c2a52e50: 762 tokens, checkpoints: 0, 184.557 MiB
58.10.222.824 I srv update: - prompt 0x6354c2cef280: 336 tokens, checkpoints: 0, 81.380 MiB
58.10.222.825 I srv update: - prompt 0x6354c2d02990: 8291 tokens, checkpoints: 0, 2008.073 MiB
58.10.222.826 I srv get_availabl: prompt cache update took 1389.06 ms
58.10.225.162 I slot launch_slot_: id 1 | task 65145 | processing task, is_child = 0
58.10.225.164 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
58.10.225.165 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
58.10.225.197 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
58.10.225.198 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
58.10.225.219 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
58.10.225.219 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
58.16.230.162 I slot print_timing: id 1 | task 65145 | n_decoded = 100, tg = 21.83 t/s, tg_3s = 21.83 t/s
58.19.232.566 I slot print_timing: id 1 | task 65145 | n_decoded = 165, tg = 21.76 t/s, tg_3s = 21.65 t/s
58.22.248.405 I slot print_timing: id 1 | task 65145 | n_decoded = 229, tg = 21.61 t/s, tg_3s = 21.22 t/s
58.25.288.582 I slot print_timing: id 1 | task 65145 | n_decoded = 295, tg = 21.63 t/s, tg_3s = 21.71 t/s
58.28.333.550 I slot print_timing: id 1 | task 65145 | n_decoded = 360, tg = 21.58 t/s, tg_3s = 21.35 t/s
58.31.350.574 I slot print_timing: id 1 | task 65145 | n_decoded = 424, tg = 21.52 t/s, tg_3s = 21.21 t/s
58.34.390.522 I slot print_timing: id 1 | task 65145 | n_decoded = 489, tg = 21.50 t/s, tg_3s = 21.38 t/s
58.37.415.552 I slot print_timing: id 1 | task 65145 | n_decoded = 554, tg = 21.50 t/s, tg_3s = 21.49 t/s
58.40.460.921 I slot print_timing: id 1 | task 65145 | n_decoded = 620, tg = 21.52 t/s, tg_3s = 21.67 t/s
58.43.493.822 I slot print_timing: id 1 | task 65145 | n_decoded = 685, tg = 21.51 t/s, tg_3s = 21.43 t/s
58.46.536.266 I slot print_timing: id 1 | task 65145 | n_decoded = 750, tg = 21.50 t/s, tg_3s = 21.36 t/s
58.49.565.641 I slot print_timing: id 1 | task 65145 | n_decoded = 814, tg = 21.47 t/s, tg_3s = 21.13 t/s
58.52.596.292 I slot print_timing: id 1 | task 65145 | n_decoded = 877, tg = 21.42 t/s, tg_3s = 20.79 t/s
58.55.626.491 I slot print_timing: id 1 | task 65145 | n_decoded = 941, tg = 21.40 t/s, tg_3s = 21.12 t/s
58.58.670.930 I slot print_timing: id 1 | task 65145 | n_decoded = 1005, tg = 21.37 t/s, tg_3s = 21.02 t/s
59.01.675.435 I slot print_timing: id 1 | task 65145 | n_decoded = 1068, tg = 21.35 t/s, tg_3s = 20.97 t/s
59.04.682.483 I slot print_timing: id 1 | task 65145 | n_decoded = 1132, tg = 21.35 t/s, tg_3s = 21.28 t/s
59.07.716.866 I slot print_timing: id 1 | task 65145 | n_decoded = 1196, tg = 21.33 t/s, tg_3s = 21.09 t/s
59.10.724.796 I slot print_timing: id 1 | task 65145 | n_decoded = 1259, tg = 21.31 t/s, tg_3s = 20.94 t/s
59.13.741.705 I slot print_timing: id 1 | task 65145 | n_decoded = 1323, tg = 21.31 t/s, tg_3s = 21.21 t/s
59.16.753.857 I slot print_timing: id 1 | task 65145 | n_decoded = 1387, tg = 21.30 t/s, tg_3s = 21.25 t/s
59.19.765.113 I slot print_timing: id 1 | task 65145 | n_decoded = 1450, tg = 21.29 t/s, tg_3s = 20.92 t/s
59.20.190.192 I slot print_timing: id 1 | task 65145 | prompt eval time = 1424.35 ms / 37 tokens ( 38.50 ms per token, 25.98 tokens per second)
59.20.190.196 I slot print_timing: id 1 | task 65145 | eval time = 68540.58 ms / 1459 tokens ( 46.98 ms per token, 21.29 tokens per second)
59.20.190.196 I slot print_timing: id 1 | task 65145 | total time = 69964.94 ms / 1496 tokens
59.20.190.197 I slot print_timing: id 1 | task 65145 | graphs reused = 66278
59.20.190.227 I slot release: id 1 | task 65145 | stop processing: n_tokens = 1558, truncated = 0
59.20.190.246 I srv update_slots: all slots are idle
59.20.251.093 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink