0.00.064.429 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.064.433 I device_info: 0.00.064.453 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.064.457 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.064.464 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free) 0.00.064.523 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.064.526 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true 0.00.064.567 I srv init: running without SSL 0.00.064.627 I srv init: using 23 threads for HTTP server 0.00.064.955 I srv start: binding port with default address family 0.00.066.162 I srv llama_server: loading model 0.00.066.170 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/Ministral-3-14B-Reasoning-2512-Q5_K_M.gguf' 0.00.066.236 I common_init_result: fitting params to device memory ... 0.00.066.240 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) 0.06.792.079 W llama_context: setting new yarn_attn_factor = 1.0000 (mscale == 1.0, mscale_all_dim = 1.0) 0.06.792.089 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized 0.06.850.562 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) 0.06.992.461 I srv load_model: initializing slots, n_slots = 4 0.07.038.147 W common_speculative_init: no implementations specified for speculative decoding 0.07.038.149 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 0.07.038.153 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384 0.07.038.154 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384 0.07.038.154 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384 0.07.038.332 I srv load_model: prompt cache is enabled, size limit: 8192 MiB 0.07.038.333 I srv load_model: use `--cache-ram 0` to disable the prompt cache 0.07.038.334 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 0.07.038.334 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 0.07.038.354 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task 0.07.041.403 I init: chat template, example_format: '[SYSTEM_PROMPT]You are a helpful assistant[/SYSTEM_PROMPT][INST]Hello[/INST]Hi there[INST]How are you?[/INST]' 0.07.041.809 I srv init: init: chat template, thinking = 1 0.07.041.843 I srv llama_server: model loaded 0.07.041.847 I srv llama_server: server is listening on http://127.0.0.1:8090 0.07.041.851 I srv update_slots: all slots are idle 0.09.965.988 I srv operator(): Chat format: peg-native 0.09.966.195 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1 0.09.966.198 I srv get_availabl: updating prompt cache 0.09.966.204 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 0.09.966.212 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) 0.09.966.214 I srv get_availabl: prompt cache update took 0.01 ms 0.09.966.253 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0 0.09.966.254 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.09.966.255 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.09.966.268 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.09.966.269 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.09.966.278 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.09.966.279 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.13.421.692 I slot print_timing: id 3 | task 0 | n_decoded = 100, tg = 32.10 t/s, tg_3s = 32.10 t/s 0.16.442.788 I slot print_timing: id 3 | task 0 | n_decoded = 196, tg = 31.94 t/s, tg_3s = 31.78 t/s 0.19.469.830 I slot print_timing: id 3 | task 0 | n_decoded = 292, tg = 31.87 t/s, tg_3s = 31.71 t/s 0.22.471.819 I slot print_timing: id 3 | task 0 | n_decoded = 387, tg = 31.81 t/s, tg_3s = 31.65 t/s 0.25.477.977 I slot print_timing: id 3 | task 0 | n_decoded = 482, tg = 31.77 t/s, tg_3s = 31.60 t/s 0.28.493.228 I slot print_timing: id 3 | task 0 | n_decoded = 577, tg = 31.73 t/s, tg_3s = 31.51 t/s 0.31.504.161 I slot print_timing: id 3 | task 0 | n_decoded = 672, tg = 31.70 t/s, tg_3s = 31.55 t/s 0.34.535.208 I slot print_timing: id 3 | task 0 | n_decoded = 767, tg = 31.66 t/s, tg_3s = 31.34 t/s 0.37.543.097 I slot print_timing: id 3 | task 0 | n_decoded = 861, tg = 31.61 t/s, tg_3s = 31.25 t/s 0.40.549.214 I slot print_timing: id 3 | task 0 | n_decoded = 955, tg = 31.58 t/s, tg_3s = 31.27 t/s 0.43.559.656 I slot print_timing: id 3 | task 0 | n_decoded = 1049, tg = 31.55 t/s, tg_3s = 31.22 t/s 0.46.567.682 I slot print_timing: id 3 | task 0 | n_decoded = 1143, tg = 31.52 t/s, tg_3s = 31.25 t/s 0.49.593.093 I slot print_timing: id 3 | task 0 | n_decoded = 1237, tg = 31.49 t/s, tg_3s = 31.07 t/s 0.52.602.954 I slot print_timing: id 3 | task 0 | n_decoded = 1330, tg = 31.44 t/s, tg_3s = 30.90 t/s 0.55.615.917 I slot print_timing: id 3 | task 0 | n_decoded = 1423, tg = 31.41 t/s, tg_3s = 30.87 t/s 0.55.648.199 I slot print_timing: id 3 | task 0 | prompt eval time = 340.08 ms / 89 tokens ( 3.82 ms per token, 261.70 tokens per second) 0.55.648.202 I slot print_timing: id 3 | task 0 | eval time = 45341.81 ms / 1424 tokens ( 31.84 ms per token, 31.41 tokens per second) 0.55.648.202 I slot print_timing: id 3 | task 0 | total time = 45681.89 ms / 1513 tokens 0.55.648.207 I slot print_timing: id 3 | task 0 | graphs reused = 0 0.55.648.243 I slot release: id 3 | task 0 | stop processing: n_tokens = 1512, truncated = 0 0.55.648.253 I srv update_slots: all slots are idle 0.55.651.062 I srv operator(): Chat format: peg-native 0.55.651.175 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.046 0.55.651.177 I srv get_availabl: updating prompt cache 0.55.651.256 W srv prompt_save: - saving prompt with length 1512, total state size = 236.268 MiB (draft: 0.000 MiB) 0.55.937.629 I srv load: - looking for better prompt, base f_keep = 0.046, sim = 0.833 0.55.937.635 I srv update: - cache state: 1 prompts, 236.268 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 0.55.937.636 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 0.55.937.637 I srv get_availabl: prompt cache update took 286.46 ms 0.55.937.666 I slot launch_slot_: id 3 | task 1425 | processing task, is_child = 0 0.55.937.666 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.55.937.667 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.55.937.680 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.55.937.680 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.55.937.692 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.55.937.692 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.59.229.309 I slot print_timing: id 3 | task 1425 | n_decoded = 100, tg = 32.06 t/s, tg_3s = 32.06 t/s 1.02.242.898 I slot print_timing: id 3 | task 1425 | n_decoded = 196, tg = 31.96 t/s, tg_3s = 31.86 t/s 1.05.267.419 I slot print_timing: id 3 | task 1425 | n_decoded = 292, tg = 31.89 t/s, tg_3s = 31.74 t/s 1.08.291.430 I slot print_timing: id 3 | task 1425 | n_decoded = 388, tg = 31.85 t/s, tg_3s = 31.75 t/s 1.11.098.454 I slot print_timing: id 3 | task 1425 | prompt eval time = 172.86 ms / 14 tokens ( 12.35 ms per token, 80.99 tokens per second) 1.11.098.458 I slot print_timing: id 3 | task 1425 | eval time = 14987.88 ms / 477 tokens ( 31.42 ms per token, 31.83 tokens per second) 1.11.098.458 I slot print_timing: id 3 | task 1425 | total time = 15160.74 ms / 491 tokens 1.11.098.460 I slot print_timing: id 3 | task 1425 | graphs reused = 0 1.11.098.476 I slot release: id 3 | task 1425 | stop processing: n_tokens = 560, truncated = 0 1.11.098.485 I srv update_slots: all slots are idle 1.11.101.346 I srv operator(): Chat format: peg-native 1.11.101.456 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.125 1.11.101.458 I srv get_availabl: updating prompt cache 1.11.101.496 W srv prompt_save: - saving prompt with length 560, total state size = 87.507 MiB (draft: 0.000 MiB) 1.11.201.123 I srv load: - looking for better prompt, base f_keep = 0.125, sim = 0.833 1.11.201.132 I srv update: - cache state: 2 prompts, 323.776 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 1.11.201.133 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 1.11.201.134 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 1.11.201.135 I srv get_availabl: prompt cache update took 99.67 ms 1.11.201.160 I slot launch_slot_: id 3 | task 1903 | processing task, is_child = 0 1.11.201.161 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.11.201.161 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.11.201.174 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.11.201.174 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.11.201.184 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.11.201.184 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.14.502.737 I slot print_timing: id 3 | task 1903 | n_decoded = 100, tg = 31.96 t/s, tg_3s = 31.96 t/s 1.17.505.815 I slot print_timing: id 3 | task 1903 | n_decoded = 195, tg = 31.80 t/s, tg_3s = 31.63 t/s 1.20.513.573 I slot print_timing: id 3 | task 1903 | n_decoded = 290, tg = 31.73 t/s, tg_3s = 31.58 t/s 1.21.403.883 I slot print_timing: id 3 | task 1903 | prompt eval time = 172.88 ms / 14 tokens ( 12.35 ms per token, 80.98 tokens per second) 1.21.403.887 I slot print_timing: id 3 | task 1903 | eval time = 10029.80 ms / 318 tokens ( 31.54 ms per token, 31.71 tokens per second) 1.21.403.887 I slot print_timing: id 3 | task 1903 | total time = 10202.68 ms / 332 tokens 1.21.403.888 I slot print_timing: id 3 | task 1903 | graphs reused = 0 1.21.403.906 I slot release: id 3 | task 1903 | stop processing: n_tokens = 401, truncated = 0 1.21.403.914 I srv update_slots: all slots are idle 1.21.406.682 I srv operator(): Chat format: peg-native 1.21.406.785 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.843 (> 0.100 thold), f_keep = 0.175 1.21.406.786 I srv get_availabl: updating prompt cache 1.21.406.821 W srv prompt_save: - saving prompt with length 401, total state size = 62.662 MiB (draft: 0.000 MiB) 1.21.455.101 I srv load: - looking for better prompt, base f_keep = 0.175, sim = 0.843 1.21.455.110 I srv update: - cache state: 3 prompts, 386.437 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 1.21.455.111 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 1.21.455.112 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 1.21.455.112 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 1.21.455.113 I srv get_availabl: prompt cache update took 48.33 ms 1.21.455.160 I slot launch_slot_: id 3 | task 2222 | processing task, is_child = 0 1.21.455.161 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.21.455.162 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.21.455.177 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.21.455.177 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.21.455.186 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.21.455.187 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.24.757.458 I slot print_timing: id 3 | task 2222 | n_decoded = 100, tg = 31.95 t/s, tg_3s = 31.95 t/s 1.27.762.344 I slot print_timing: id 3 | task 2222 | n_decoded = 195, tg = 31.79 t/s, tg_3s = 31.62 t/s 1.30.773.648 I slot print_timing: id 3 | task 2222 | n_decoded = 290, tg = 31.71 t/s, tg_3s = 31.55 t/s 1.33.791.345 I slot print_timing: id 3 | task 2222 | n_decoded = 385, tg = 31.65 t/s, tg_3s = 31.48 t/s 1.36.806.975 I slot print_timing: id 3 | task 2222 | n_decoded = 480, tg = 31.62 t/s, tg_3s = 31.50 t/s 1.39.829.442 I slot print_timing: id 3 | task 2222 | n_decoded = 575, tg = 31.59 t/s, tg_3s = 31.43 t/s 1.42.861.392 I slot print_timing: id 3 | task 2222 | n_decoded = 670, tg = 31.55 t/s, tg_3s = 31.33 t/s 1.45.871.661 I slot print_timing: id 3 | task 2222 | n_decoded = 764, tg = 31.51 t/s, tg_3s = 31.23 t/s 1.48.879.710 I slot print_timing: id 3 | task 2222 | n_decoded = 858, tg = 31.48 t/s, tg_3s = 31.25 t/s 1.49.647.713 I slot print_timing: id 3 | task 2222 | prompt eval time = 172.27 ms / 13 tokens ( 13.25 ms per token, 75.46 tokens per second) 1.49.647.717 I slot print_timing: id 3 | task 2222 | eval time = 28020.23 ms / 882 tokens ( 31.77 ms per token, 31.48 tokens per second) 1.49.647.718 I slot print_timing: id 3 | task 2222 | total time = 28192.50 ms / 895 tokens 1.49.647.719 I slot print_timing: id 3 | task 2222 | graphs reused = 0 1.49.647.736 I slot release: id 3 | task 2222 | stop processing: n_tokens = 964, truncated = 0 1.49.647.746 I srv update_slots: all slots are idle 1.49.650.541 I srv operator(): Chat format: peg-native 1.49.650.646 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.729 (> 0.100 thold), f_keep = 0.073 1.49.650.648 I srv get_availabl: updating prompt cache 1.49.650.705 W srv prompt_save: - saving prompt with length 964, total state size = 150.637 MiB (draft: 0.000 MiB) 1.49.848.683 I srv load: - looking for better prompt, base f_keep = 0.073, sim = 0.729 1.49.848.689 I srv update: - cache state: 4 prompts, 537.074 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 1.49.848.690 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 1.49.848.691 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 1.49.848.691 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 1.49.848.692 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 1.49.848.694 I srv get_availabl: prompt cache update took 198.04 ms 1.49.848.722 I slot launch_slot_: id 3 | task 3105 | processing task, is_child = 0 1.49.848.722 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.49.848.723 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.49.848.736 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.49.848.736 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.49.848.745 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.49.848.745 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.53.199.872 I slot print_timing: id 3 | task 3105 | n_decoded = 100, tg = 32.07 t/s, tg_3s = 32.07 t/s 1.56.226.570 I slot print_timing: id 3 | task 3105 | n_decoded = 196, tg = 31.90 t/s, tg_3s = 31.72 t/s 1.59.250.551 I slot print_timing: id 3 | task 3105 | n_decoded = 292, tg = 31.85 t/s, tg_3s = 31.75 t/s 2.02.276.575 I slot print_timing: id 3 | task 3105 | n_decoded = 388, tg = 31.82 t/s, tg_3s = 31.72 t/s 2.05.281.146 I slot print_timing: id 3 | task 3105 | n_decoded = 483, tg = 31.78 t/s, tg_3s = 31.62 t/s 2.08.283.603 I slot print_timing: id 3 | task 3105 | n_decoded = 578, tg = 31.75 t/s, tg_3s = 31.64 t/s 2.11.290.475 I slot print_timing: id 3 | task 3105 | n_decoded = 673, tg = 31.73 t/s, tg_3s = 31.59 t/s 2.14.316.609 I slot print_timing: id 3 | task 3105 | n_decoded = 768, tg = 31.69 t/s, tg_3s = 31.39 t/s 2.17.341.491 I slot print_timing: id 3 | task 3105 | n_decoded = 863, tg = 31.66 t/s, tg_3s = 31.41 t/s 2.20.349.708 I slot print_timing: id 3 | task 3105 | n_decoded = 957, tg = 31.62 t/s, tg_3s = 31.25 t/s 2.23.357.717 I slot print_timing: id 3 | task 3105 | n_decoded = 1051, tg = 31.58 t/s, tg_3s = 31.25 t/s 2.26.365.674 I slot print_timing: id 3 | task 3105 | n_decoded = 1145, tg = 31.56 t/s, tg_3s = 31.25 t/s 2.29.395.958 I slot print_timing: id 3 | task 3105 | n_decoded = 1239, tg = 31.52 t/s, tg_3s = 31.02 t/s 2.32.412.987 I slot print_timing: id 3 | task 3105 | n_decoded = 1332, tg = 31.47 t/s, tg_3s = 30.83 t/s 2.35.434.257 I slot print_timing: id 3 | task 3105 | n_decoded = 1425, tg = 31.42 t/s, tg_3s = 30.78 t/s 2.38.460.938 I slot print_timing: id 3 | task 3105 | n_decoded = 1519, tg = 31.40 t/s, tg_3s = 31.06 t/s 2.41.469.980 I slot print_timing: id 3 | task 3105 | n_decoded = 1613, tg = 31.39 t/s, tg_3s = 31.24 t/s 2.44.256.171 I slot print_timing: id 3 | task 3105 | prompt eval time = 232.73 ms / 26 tokens ( 8.95 ms per token, 111.72 tokens per second) 2.44.256.173 I slot print_timing: id 3 | task 3105 | eval time = 54174.68 ms / 1700 tokens ( 31.87 ms per token, 31.38 tokens per second) 2.44.256.174 I slot print_timing: id 3 | task 3105 | total time = 54407.40 ms / 1726 tokens 2.44.256.175 I slot print_timing: id 3 | task 3105 | graphs reused = 0 2.44.256.195 I slot release: id 3 | task 3105 | stop processing: n_tokens = 1795, truncated = 0 2.44.256.210 I srv update_slots: all slots are idle 2.44.258.864 I srv operator(): Chat format: peg-native 2.44.258.969 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.824 (> 0.100 thold), f_keep = 0.039 2.44.258.971 I srv get_availabl: updating prompt cache 2.44.259.052 W srv prompt_save: - saving prompt with length 1795, total state size = 280.490 MiB (draft: 0.000 MiB) 2.44.531.411 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.824 2.44.531.418 I srv update: - cache state: 5 prompts, 817.565 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 2.44.531.419 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 2.44.531.420 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 2.44.531.420 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 2.44.531.421 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 2.44.531.421 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 2.44.531.423 I srv get_availabl: prompt cache update took 272.45 ms 2.44.531.453 I slot launch_slot_: id 3 | task 4806 | processing task, is_child = 0 2.44.531.453 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.44.531.454 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.44.531.468 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.44.531.468 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.44.531.477 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.44.531.477 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.47.827.260 I slot print_timing: id 3 | task 4806 | n_decoded = 100, tg = 32.02 t/s, tg_3s = 32.02 t/s 2.50.854.351 I slot print_timing: id 3 | task 4806 | n_decoded = 196, tg = 31.87 t/s, tg_3s = 31.71 t/s 2.53.860.940 I slot print_timing: id 3 | task 4806 | n_decoded = 291, tg = 31.78 t/s, tg_3s = 31.60 t/s 2.54.177.527 I slot print_timing: id 3 | task 4806 | prompt eval time = 172.77 ms / 15 tokens ( 11.52 ms per token, 86.82 tokens per second) 2.54.177.530 I slot print_timing: id 3 | task 4806 | eval time = 9473.26 ms / 301 tokens ( 31.47 ms per token, 31.77 tokens per second) 2.54.177.531 I slot print_timing: id 3 | task 4806 | total time = 9646.03 ms / 316 tokens 2.54.177.532 I slot print_timing: id 3 | task 4806 | graphs reused = 0 2.54.177.553 I slot release: id 3 | task 4806 | stop processing: n_tokens = 385, truncated = 0 2.54.177.561 I srv update_slots: all slots are idle 2.54.180.226 I srv operator(): Chat format: peg-native 2.54.180.332 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.877 (> 0.100 thold), f_keep = 0.184 2.54.180.332 I srv get_availabl: updating prompt cache 2.54.180.363 W srv prompt_save: - saving prompt with length 385, total state size = 60.162 MiB (draft: 0.000 MiB) 2.54.226.527 I srv load: - looking for better prompt, base f_keep = 0.184, sim = 0.877 2.54.226.535 I srv update: - cache state: 6 prompts, 877.726 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 2.54.226.536 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 2.54.226.537 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 2.54.226.537 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 2.54.226.538 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 2.54.226.538 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 2.54.226.539 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 2.54.226.541 I srv get_availabl: prompt cache update took 46.21 ms 2.54.226.586 I slot launch_slot_: id 3 | task 5108 | processing task, is_child = 0 2.54.226.586 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.54.226.588 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.54.226.600 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.54.226.600 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.54.226.609 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.54.226.609 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.57.526.940 I slot print_timing: id 3 | task 5108 | n_decoded = 100, tg = 31.97 t/s, tg_3s = 31.97 t/s 3.00.532.642 I slot print_timing: id 3 | task 5108 | n_decoded = 195, tg = 31.79 t/s, tg_3s = 31.61 t/s 3.03.544.372 I slot print_timing: id 3 | task 5108 | n_decoded = 290, tg = 31.71 t/s, tg_3s = 31.54 t/s 3.05.297.495 I slot print_timing: id 3 | task 5108 | prompt eval time = 172.47 ms / 10 tokens ( 17.25 ms per token, 57.98 tokens per second) 3.05.297.499 I slot print_timing: id 3 | task 5108 | eval time = 10898.40 ms / 345 tokens ( 31.59 ms per token, 31.66 tokens per second) 3.05.297.499 I slot print_timing: id 3 | task 5108 | total time = 11070.86 ms / 355 tokens 3.05.297.500 I slot print_timing: id 3 | task 5108 | graphs reused = 0 3.05.297.520 I slot release: id 3 | task 5108 | stop processing: n_tokens = 425, truncated = 0 3.05.297.528 I srv update_slots: all slots are idle 3.05.300.356 I srv operator(): Chat format: peg-native 3.05.300.478 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.855 (> 0.100 thold), f_keep = 0.167 3.05.300.480 I srv get_availabl: updating prompt cache 3.05.300.511 W srv prompt_save: - saving prompt with length 425, total state size = 66.412 MiB (draft: 0.000 MiB) 3.05.350.550 I srv load: - looking for better prompt, base f_keep = 0.167, sim = 0.855 3.05.350.562 I srv update: - cache state: 7 prompts, 944.138 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 3.05.350.563 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 3.05.350.564 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 3.05.350.564 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 3.05.350.565 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 3.05.350.565 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 3.05.350.566 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 3.05.350.566 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 3.05.350.568 I srv get_availabl: prompt cache update took 50.09 ms 3.05.350.609 I slot launch_slot_: id 3 | task 5454 | processing task, is_child = 0 3.05.350.610 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.05.350.611 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.05.350.625 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.05.350.625 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.05.350.634 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.05.350.634 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.08.646.453 I slot print_timing: id 3 | task 5454 | n_decoded = 100, tg = 32.02 t/s, tg_3s = 32.02 t/s 3.11.654.408 I slot print_timing: id 3 | task 5454 | n_decoded = 195, tg = 31.80 t/s, tg_3s = 31.58 t/s 3.14.685.053 I slot print_timing: id 3 | task 5454 | n_decoded = 290, tg = 31.65 t/s, tg_3s = 31.35 t/s 3.17.690.544 I slot print_timing: id 3 | task 5454 | n_decoded = 384, tg = 31.56 t/s, tg_3s = 31.28 t/s 3.20.698.543 I slot print_timing: id 3 | task 5454 | n_decoded = 478, tg = 31.50 t/s, tg_3s = 31.25 t/s 3.23.706.646 I slot print_timing: id 3 | task 5454 | n_decoded = 572, tg = 31.46 t/s, tg_3s = 31.25 t/s 3.26.714.482 I slot print_timing: id 3 | task 5454 | n_decoded = 666, tg = 31.43 t/s, tg_3s = 31.25 t/s 3.29.722.654 I slot print_timing: id 3 | task 5454 | n_decoded = 760, tg = 31.41 t/s, tg_3s = 31.25 t/s 3.32.736.721 I slot print_timing: id 3 | task 5454 | n_decoded = 855, tg = 31.42 t/s, tg_3s = 31.52 t/s 3.33.053.467 I slot print_timing: id 3 | task 5454 | prompt eval time = 172.31 ms / 12 tokens ( 14.36 ms per token, 69.64 tokens per second) 3.33.053.471 I slot print_timing: id 3 | task 5454 | eval time = 27530.51 ms / 865 tokens ( 31.83 ms per token, 31.42 tokens per second) 3.33.053.473 I slot print_timing: id 3 | task 5454 | total time = 27702.81 ms / 877 tokens 3.33.053.474 I slot print_timing: id 3 | task 5454 | graphs reused = 0 3.33.053.494 I slot release: id 3 | task 5454 | stop processing: n_tokens = 947, truncated = 0 3.33.053.505 I srv update_slots: all slots are idle 3.33.056.479 I srv operator(): Chat format: peg-native 3.33.056.591 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.631 (> 0.100 thold), f_keep = 0.074 3.33.056.593 I srv get_availabl: updating prompt cache 3.33.056.635 W srv prompt_save: - saving prompt with length 947, total state size = 147.981 MiB (draft: 0.000 MiB) 3.33.207.209 I srv load: - looking for better prompt, base f_keep = 0.074, sim = 0.631 3.33.207.219 I srv update: - cache state: 8 prompts, 1092.119 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 3.33.207.220 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 3.33.207.220 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 3.33.207.221 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 3.33.207.225 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 3.33.207.225 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 3.33.207.226 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 3.33.207.226 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 3.33.207.227 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 3.33.207.228 I srv get_availabl: prompt cache update took 150.63 ms 3.33.207.255 I slot launch_slot_: id 3 | task 6320 | processing task, is_child = 0 3.33.207.255 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.33.207.257 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.33.207.268 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.33.207.269 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.33.207.277 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.33.207.278 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.36.518.394 I slot print_timing: id 3 | task 6320 | n_decoded = 100, tg = 32.06 t/s, tg_3s = 32.06 t/s 3.39.543.029 I slot print_timing: id 3 | task 6320 | n_decoded = 196, tg = 31.90 t/s, tg_3s = 31.74 t/s 3.42.574.580 I slot print_timing: id 3 | task 6320 | n_decoded = 292, tg = 31.82 t/s, tg_3s = 31.67 t/s 3.43.525.056 I slot print_timing: id 3 | task 6320 | prompt eval time = 191.61 ms / 41 tokens ( 4.67 ms per token, 213.98 tokens per second) 3.43.525.062 I slot print_timing: id 3 | task 6320 | eval time = 10126.15 ms / 322 tokens ( 31.45 ms per token, 31.80 tokens per second) 3.43.525.064 I slot print_timing: id 3 | task 6320 | total time = 10317.76 ms / 363 tokens 3.43.525.064 I slot print_timing: id 3 | task 6320 | graphs reused = 0 3.43.525.086 I slot release: id 3 | task 6320 | stop processing: n_tokens = 432, truncated = 0 3.43.525.093 I srv update_slots: all slots are idle 3.43.528.119 I srv operator(): Chat format: peg-native 3.43.528.233 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.542 (> 0.100 thold), f_keep = 0.164 3.43.528.236 I srv get_availabl: updating prompt cache 3.43.528.277 W srv prompt_save: - saving prompt with length 432, total state size = 67.506 MiB (draft: 0.000 MiB) 3.43.578.721 I srv load: - looking for better prompt, base f_keep = 0.164, sim = 0.542 3.43.578.734 I srv update: - cache state: 9 prompts, 1159.625 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 3.43.578.736 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 3.43.578.740 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 3.43.578.740 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 3.43.578.741 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 3.43.578.742 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 3.43.578.743 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 3.43.578.744 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 3.43.578.745 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 3.43.578.746 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 3.43.578.747 I srv get_availabl: prompt cache update took 50.51 ms 3.43.578.789 I slot launch_slot_: id 3 | task 6643 | processing task, is_child = 0 3.43.578.790 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.43.578.791 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.43.578.807 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.43.578.807 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.43.578.829 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.43.578.829 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.46.967.577 I slot print_timing: id 3 | task 6643 | n_decoded = 100, tg = 32.04 t/s, tg_3s = 32.04 t/s 3.49.973.501 I slot print_timing: id 3 | task 6643 | n_decoded = 195, tg = 31.83 t/s, tg_3s = 31.60 t/s 3.52.985.687 I slot print_timing: id 3 | task 6643 | n_decoded = 290, tg = 31.73 t/s, tg_3s = 31.54 t/s 3.56.007.453 I slot print_timing: id 3 | task 6643 | n_decoded = 385, tg = 31.66 t/s, tg_3s = 31.44 t/s 3.56.873.436 I slot print_timing: id 3 | task 6643 | prompt eval time = 268.08 ms / 60 tokens ( 4.47 ms per token, 223.82 tokens per second) 3.56.873.440 I slot print_timing: id 3 | task 6643 | eval time = 13026.51 ms / 412 tokens ( 31.62 ms per token, 31.63 tokens per second) 3.56.873.440 I slot print_timing: id 3 | task 6643 | total time = 13294.58 ms / 472 tokens 3.56.873.441 I slot print_timing: id 3 | task 6643 | graphs reused = 0 3.56.873.464 I slot release: id 3 | task 6643 | stop processing: n_tokens = 542, truncated = 0 3.56.873.471 I srv update_slots: all slots are idle 3.56.876.351 I srv operator(): Chat format: peg-native 3.56.876.463 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.660 (> 0.100 thold), f_keep = 0.129 3.56.876.465 I srv get_availabl: updating prompt cache 3.56.876.500 W srv prompt_save: - saving prompt with length 542, total state size = 84.695 MiB (draft: 0.000 MiB) 3.56.956.602 I srv load: - looking for better prompt, base f_keep = 0.129, sim = 0.660 3.56.956.611 I srv update: - cache state: 10 prompts, 1244.319 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 3.56.956.612 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 3.56.956.616 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 3.56.956.617 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 3.56.956.618 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 3.56.956.618 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 3.56.956.619 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 3.56.956.619 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 3.56.956.627 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 3.56.956.628 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 3.56.956.629 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 3.56.956.630 I srv get_availabl: prompt cache update took 80.16 ms 3.56.956.657 I slot launch_slot_: id 3 | task 7056 | processing task, is_child = 0 3.56.956.658 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.56.956.659 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.56.956.670 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.56.956.670 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.56.956.690 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.56.956.691 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.00.272.089 I slot print_timing: id 3 | task 7056 | n_decoded = 100, tg = 31.85 t/s, tg_3s = 31.85 t/s 4.03.291.107 I slot print_timing: id 3 | task 7056 | n_decoded = 195, tg = 31.66 t/s, tg_3s = 31.47 t/s 4.06.292.482 I slot print_timing: id 3 | task 7056 | n_decoded = 289, tg = 31.55 t/s, tg_3s = 31.32 t/s 4.09.298.577 I slot print_timing: id 3 | task 7056 | n_decoded = 383, tg = 31.48 t/s, tg_3s = 31.27 t/s 4.12.306.695 I slot print_timing: id 3 | task 7056 | n_decoded = 477, tg = 31.44 t/s, tg_3s = 31.25 t/s 4.15.314.694 I slot print_timing: id 3 | task 7056 | n_decoded = 571, tg = 31.40 t/s, tg_3s = 31.25 t/s 4.17.446.697 I slot print_timing: id 3 | task 7056 | prompt eval time = 175.91 ms / 36 tokens ( 4.89 ms per token, 204.65 tokens per second) 4.17.446.703 I slot print_timing: id 3 | task 7056 | eval time = 20314.07 ms / 638 tokens ( 31.84 ms per token, 31.41 tokens per second) 4.17.446.703 I slot print_timing: id 3 | task 7056 | total time = 20489.97 ms / 674 tokens 4.17.446.704 I slot print_timing: id 3 | task 7056 | graphs reused = 0 4.17.446.724 I slot release: id 3 | task 7056 | stop processing: n_tokens = 743, truncated = 0 4.17.446.733 I srv update_slots: all slots are idle 4.17.449.531 I srv operator(): Chat format: peg-native 4.17.449.681 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.538 (> 0.100 thold), f_keep = 0.094 4.17.449.683 I srv get_availabl: updating prompt cache 4.17.449.731 W srv prompt_save: - saving prompt with length 743, total state size = 116.103 MiB (draft: 0.000 MiB) 4.17.538.952 I srv load: - looking for better prompt, base f_keep = 0.094, sim = 0.538 4.17.538.962 I srv update: - cache state: 11 prompts, 1360.422 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 4.17.538.963 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 4.17.538.964 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 4.17.538.964 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 4.17.538.965 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 4.17.538.966 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 4.17.538.970 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 4.17.538.970 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 4.17.538.971 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 4.17.538.971 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 4.17.538.972 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 4.17.538.972 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 4.17.538.973 I srv get_availabl: prompt cache update took 89.29 ms 4.17.539.001 I slot launch_slot_: id 3 | task 7695 | processing task, is_child = 0 4.17.539.002 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.17.539.003 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.17.539.014 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.17.539.015 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.17.539.023 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.17.539.023 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.20.918.434 I slot print_timing: id 3 | task 7695 | n_decoded = 100, tg = 32.14 t/s, tg_3s = 32.14 t/s 4.23.943.622 I slot print_timing: id 3 | task 7695 | n_decoded = 196, tg = 31.94 t/s, tg_3s = 31.73 t/s 4.26.971.081 I slot print_timing: id 3 | task 7695 | n_decoded = 292, tg = 31.86 t/s, tg_3s = 31.71 t/s 4.29.307.675 I slot print_timing: id 3 | task 7695 | prompt eval time = 267.53 ms / 60 tokens ( 4.46 ms per token, 224.27 tokens per second) 4.29.307.680 I slot print_timing: id 3 | task 7695 | eval time = 11501.09 ms / 366 tokens ( 31.42 ms per token, 31.82 tokens per second) 4.29.307.681 I slot print_timing: id 3 | task 7695 | total time = 11768.63 ms / 426 tokens 4.29.307.681 I slot print_timing: id 3 | task 7695 | graphs reused = 0 4.29.307.702 I slot release: id 3 | task 7695 | stop processing: n_tokens = 495, truncated = 0 4.29.307.712 I srv update_slots: all slots are idle 4.29.310.209 I srv operator(): Chat format: peg-native 4.29.310.318 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.569 (> 0.100 thold), f_keep = 0.141 4.29.310.320 I srv get_availabl: updating prompt cache 4.29.310.356 W srv prompt_save: - saving prompt with length 495, total state size = 77.350 MiB (draft: 0.000 MiB) 4.29.368.001 I srv load: - looking for better prompt, base f_keep = 0.141, sim = 0.569 4.29.368.013 I srv update: - cache state: 12 prompts, 1437.773 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 4.29.368.014 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 4.29.368.015 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 4.29.368.015 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 4.29.368.016 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 4.29.368.016 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 4.29.368.017 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 4.29.368.017 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 4.29.368.018 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 4.29.368.018 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 4.29.368.019 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 4.29.368.019 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 4.29.368.020 I srv update: - prompt 0x5aacf2e90fb0: 495 tokens, checkpoints: 0, 77.350 MiB 4.29.368.021 I srv get_availabl: prompt cache update took 57.70 ms 4.29.368.061 I slot launch_slot_: id 3 | task 8062 | processing task, is_child = 0 4.29.368.062 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.29.368.062 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.29.368.076 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.29.368.076 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.29.368.085 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.29.368.085 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.32.730.607 I slot print_timing: id 3 | task 8062 | n_decoded = 100, tg = 32.06 t/s, tg_3s = 32.06 t/s 4.35.761.737 I slot print_timing: id 3 | task 8062 | n_decoded = 196, tg = 31.87 t/s, tg_3s = 31.67 t/s 4.37.969.694 I slot print_timing: id 3 | task 8062 | prompt eval time = 242.92 ms / 53 tokens ( 4.58 ms per token, 218.18 tokens per second) 4.37.969.699 I slot print_timing: id 3 | task 8062 | eval time = 8358.67 ms / 266 tokens ( 31.42 ms per token, 31.82 tokens per second) 4.37.969.699 I slot print_timing: id 3 | task 8062 | total time = 8601.59 ms / 319 tokens 4.37.969.700 I slot print_timing: id 3 | task 8062 | graphs reused = 0 4.37.969.723 I slot release: id 3 | task 8062 | stop processing: n_tokens = 388, truncated = 0 4.37.969.731 I srv update_slots: all slots are idle 4.37.972.560 I srv operator(): Chat format: peg-native 4.37.972.667 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.628 (> 0.100 thold), f_keep = 0.183 4.37.972.670 I srv get_availabl: updating prompt cache 4.37.972.702 W srv prompt_save: - saving prompt with length 388, total state size = 60.630 MiB (draft: 0.000 MiB) 4.38.018.412 I srv load: - looking for better prompt, base f_keep = 0.183, sim = 0.628 4.38.018.427 I srv update: - cache state: 13 prompts, 1498.403 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 4.38.018.428 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 4.38.018.429 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 4.38.018.430 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 4.38.018.430 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 4.38.018.431 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 4.38.018.431 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 4.38.018.432 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 4.38.018.435 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 4.38.018.436 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 4.38.018.437 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 4.38.018.437 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 4.38.018.438 I srv update: - prompt 0x5aacf2e90fb0: 495 tokens, checkpoints: 0, 77.350 MiB 4.38.018.438 I srv update: - prompt 0x5aacf460a850: 388 tokens, checkpoints: 0, 60.630 MiB 4.38.018.439 I srv get_availabl: prompt cache update took 45.77 ms 4.38.018.477 I slot launch_slot_: id 3 | task 8329 | processing task, is_child = 0 4.38.018.478 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.38.018.478 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.38.018.494 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.38.018.494 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.38.018.503 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.38.018.504 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.41.335.664 I slot print_timing: id 3 | task 8329 | n_decoded = 100, tg = 32.00 t/s, tg_3s = 32.00 t/s 4.44.344.319 I slot print_timing: id 3 | task 8329 | n_decoded = 195, tg = 31.79 t/s, tg_3s = 31.58 t/s 4.47.355.494 I slot print_timing: id 3 | task 8329 | n_decoded = 290, tg = 31.71 t/s, tg_3s = 31.55 t/s 4.50.375.594 I slot print_timing: id 3 | task 8329 | n_decoded = 385, tg = 31.65 t/s, tg_3s = 31.46 t/s 4.53.403.487 I slot print_timing: id 3 | task 8329 | n_decoded = 480, tg = 31.59 t/s, tg_3s = 31.37 t/s 4.53.883.470 I slot print_timing: id 3 | task 8329 | prompt eval time = 191.82 ms / 42 tokens ( 4.57 ms per token, 218.95 tokens per second) 4.53.883.475 I slot print_timing: id 3 | task 8329 | eval time = 15673.12 ms / 495 tokens ( 31.66 ms per token, 31.58 tokens per second) 4.53.883.475 I slot print_timing: id 3 | task 8329 | total time = 15864.94 ms / 537 tokens 4.53.883.476 I slot print_timing: id 3 | task 8329 | graphs reused = 0 4.53.883.497 I slot release: id 3 | task 8329 | stop processing: n_tokens = 607, truncated = 0 4.53.883.504 I srv update_slots: all slots are idle 4.53.886.344 I srv operator(): Chat format: peg-native 4.53.886.452 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.588 (> 0.100 thold), f_keep = 0.115 4.53.886.454 I srv get_availabl: updating prompt cache 4.53.886.497 W srv prompt_save: - saving prompt with length 607, total state size = 94.852 MiB (draft: 0.000 MiB) 4.53.972.838 I srv load: - looking for better prompt, base f_keep = 0.115, sim = 0.588 4.53.972.848 I srv update: - cache state: 14 prompts, 1593.255 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 4.53.972.849 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 4.53.972.849 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 4.53.972.851 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 4.53.972.852 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 4.53.972.852 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 4.53.972.853 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 4.53.972.853 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 4.53.972.854 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 4.53.972.854 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 4.53.972.855 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 4.53.972.855 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 4.53.972.856 I srv update: - prompt 0x5aacf2e90fb0: 495 tokens, checkpoints: 0, 77.350 MiB 4.53.972.856 I srv update: - prompt 0x5aacf460a850: 388 tokens, checkpoints: 0, 60.630 MiB 4.53.972.857 I srv update: - prompt 0x5aacf4619bc0: 607 tokens, checkpoints: 0, 94.852 MiB 4.53.972.858 I srv get_availabl: prompt cache update took 86.40 ms 4.53.972.886 I slot launch_slot_: id 3 | task 8825 | processing task, is_child = 0 4.53.972.887 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.53.972.888 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.53.972.900 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.53.972.901 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.53.972.924 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.53.972.924 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.57.352.436 I slot print_timing: id 3 | task 8825 | n_decoded = 100, tg = 31.88 t/s, tg_3s = 31.88 t/s 5.00.382.518 I slot print_timing: id 3 | task 8825 | n_decoded = 195, tg = 31.62 t/s, tg_3s = 31.35 t/s 5.03.390.534 I slot print_timing: id 3 | task 8825 | n_decoded = 289, tg = 31.50 t/s, tg_3s = 31.25 t/s 5.06.398.604 I slot print_timing: id 3 | task 8825 | n_decoded = 383, tg = 31.44 t/s, tg_3s = 31.25 t/s 5.09.410.856 I slot print_timing: id 3 | task 8825 | prompt eval time = 242.82 ms / 49 tokens ( 4.96 ms per token, 201.80 tokens per second) 5.09.410.860 I slot print_timing: id 3 | task 8825 | eval time = 15195.09 ms / 477 tokens ( 31.86 ms per token, 31.39 tokens per second) 5.09.410.861 I slot print_timing: id 3 | task 8825 | total time = 15437.90 ms / 526 tokens 5.09.410.862 I slot print_timing: id 3 | task 8825 | graphs reused = 0 5.09.410.887 I slot release: id 3 | task 8825 | stop processing: n_tokens = 595, truncated = 0 5.09.410.896 I srv update_slots: all slots are idle 5.09.413.492 I srv operator(): Chat format: peg-native 5.09.413.606 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.118 5.09.413.608 I srv get_availabl: updating prompt cache 5.09.413.667 W srv prompt_save: - saving prompt with length 595, total state size = 92.976 MiB (draft: 0.000 MiB) 5.09.490.660 I srv load: - looking for better prompt, base f_keep = 0.118, sim = 0.667 5.09.490.669 I srv update: - cache state: 15 prompts, 1686.231 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 5.09.490.670 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 5.09.490.671 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 5.09.490.672 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 5.09.490.673 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 5.09.490.677 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 5.09.490.677 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 5.09.490.678 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 5.09.490.679 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 5.09.490.679 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 5.09.490.680 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 5.09.490.680 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 5.09.490.681 I srv update: - prompt 0x5aacf2e90fb0: 495 tokens, checkpoints: 0, 77.350 MiB 5.09.490.681 I srv update: - prompt 0x5aacf460a850: 388 tokens, checkpoints: 0, 60.630 MiB 5.09.490.682 I srv update: - prompt 0x5aacf4619bc0: 607 tokens, checkpoints: 0, 94.852 MiB 5.09.490.692 I srv update: - prompt 0x5aacf47a2aa0: 595 tokens, checkpoints: 0, 92.976 MiB 5.09.490.694 I srv get_availabl: prompt cache update took 77.08 ms 5.09.490.722 I slot launch_slot_: id 3 | task 9303 | processing task, is_child = 0 5.09.490.723 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 5.09.490.724 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 5.09.490.737 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 5.09.490.737 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 5.09.490.746 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 5.09.490.746 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 5.12.798.346 I slot print_timing: id 3 | task 9303 | n_decoded = 100, tg = 31.93 t/s, tg_3s = 31.93 t/s 5.15.826.471 I slot print_timing: id 3 | task 9303 | n_decoded = 196, tg = 31.82 t/s, tg_3s = 31.70 t/s 5.18.853.514 I slot print_timing: id 3 | task 9303 | n_decoded = 292, tg = 31.79 t/s, tg_3s = 31.71 t/s 5.21.854.514 I slot print_timing: id 3 | task 9303 | n_decoded = 387, tg = 31.75 t/s, tg_3s = 31.66 t/s 5.24.862.973 I slot print_timing: id 3 | task 9303 | n_decoded = 482, tg = 31.72 t/s, tg_3s = 31.58 t/s 5.27.877.664 I slot print_timing: id 3 | task 9303 | n_decoded = 577, tg = 31.68 t/s, tg_3s = 31.51 t/s 5.30.896.598 I slot print_timing: id 3 | task 9303 | n_decoded = 672, tg = 31.65 t/s, tg_3s = 31.47 t/s 5.33.904.623 I slot print_timing: id 3 | task 9303 | n_decoded = 766, tg = 31.60 t/s, tg_3s = 31.25 t/s 5.35.792.661 I slot print_timing: id 3 | task 9303 | prompt eval time = 176.10 ms / 35 tokens ( 5.03 ms per token, 198.75 tokens per second) 5.35.792.667 I slot print_timing: id 3 | task 9303 | eval time = 26125.80 ms / 825 tokens ( 31.67 ms per token, 31.58 tokens per second) 5.35.792.667 I slot print_timing: id 3 | task 9303 | total time = 26301.89 ms / 860 tokens 5.35.792.668 I slot print_timing: id 3 | task 9303 | graphs reused = 0 5.35.792.689 I slot release: id 3 | task 9303 | stop processing: n_tokens = 929, truncated = 0 5.35.792.698 I srv update_slots: all slots are idle 5.35.795.773 I srv operator(): Chat format: peg-native 5.35.795.879 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.075 5.35.795.881 I srv get_availabl: updating prompt cache 5.35.795.920 W srv prompt_save: - saving prompt with length 929, total state size = 145.168 MiB (draft: 0.000 MiB) 5.35.924.237 I srv load: - looking for better prompt, base f_keep = 0.075, sim = 0.667 5.35.924.248 I srv update: - cache state: 16 prompts, 1831.399 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 5.35.924.249 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 5.35.924.250 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 5.35.924.250 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 5.35.924.251 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 5.35.924.251 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 5.35.924.252 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 5.35.924.253 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 5.35.924.254 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 5.35.924.258 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 5.35.924.258 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 5.35.924.259 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 5.35.924.259 I srv update: - prompt 0x5aacf2e90fb0: 495 tokens, checkpoints: 0, 77.350 MiB 5.35.924.260 I srv update: - prompt 0x5aacf460a850: 388 tokens, checkpoints: 0, 60.630 MiB 5.35.924.260 I srv update: - prompt 0x5aacf4619bc0: 607 tokens, checkpoints: 0, 94.852 MiB 5.35.924.261 I srv update: - prompt 0x5aacf47a2aa0: 595 tokens, checkpoints: 0, 92.976 MiB 5.35.924.262 I srv update: - prompt 0x5aacf452c120: 929 tokens, checkpoints: 0, 145.168 MiB 5.35.924.264 I srv get_availabl: prompt cache update took 128.38 ms 5.35.924.291 I slot launch_slot_: id 3 | task 10129 | processing task, is_child = 0 5.35.924.292 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 5.35.924.293 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 5.35.924.305 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 5.35.924.305 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 5.35.924.315 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 5.35.924.315 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 5.39.238.592 I slot print_timing: id 3 | task 10129 | n_decoded = 100, tg = 31.87 t/s, tg_3s = 31.87 t/s 5.42.251.679 I slot print_timing: id 3 | task 10129 | n_decoded = 195, tg = 31.70 t/s, tg_3s = 31.53 t/s 5.45.271.701 I slot print_timing: id 3 | task 10129 | n_decoded = 290, tg = 31.62 t/s, tg_3s = 31.46 t/s 5.48.291.073 I slot print_timing: id 3 | task 10129 | n_decoded = 385, tg = 31.58 t/s, tg_3s = 31.46 t/s 5.51.293.683 I slot print_timing: id 3 | task 10129 | n_decoded = 479, tg = 31.53 t/s, tg_3s = 31.31 t/s 5.54.300.680 I slot print_timing: id 3 | task 10129 | n_decoded = 573, tg = 31.48 t/s, tg_3s = 31.26 t/s 5.57.300.814 I slot print_timing: id 3 | task 10129 | n_decoded = 667, tg = 31.46 t/s, tg_3s = 31.33 t/s 6.00.305.911 I slot print_timing: id 3 | task 10129 | n_decoded = 762, tg = 31.48 t/s, tg_3s = 31.61 t/s 6.03.123.499 I slot print_timing: id 3 | task 10129 | prompt eval time = 176.33 ms / 35 tokens ( 5.04 ms per token, 198.49 tokens per second) 6.03.123.502 I slot print_timing: id 3 | task 10129 | eval time = 27022.83 ms / 851 tokens ( 31.75 ms per token, 31.49 tokens per second) 6.03.123.503 I slot print_timing: id 3 | task 10129 | total time = 27199.16 ms / 886 tokens 6.03.123.504 I slot print_timing: id 3 | task 10129 | graphs reused = 0 6.03.123.529 I slot release: id 3 | task 10129 | stop processing: n_tokens = 955, truncated = 0 6.03.123.538 I srv update_slots: all slots are idle 6.03.126.063 I srv operator(): Chat format: peg-native 6.03.126.178 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.722 (> 0.100 thold), f_keep = 0.073 6.03.126.179 I srv get_availabl: updating prompt cache 6.03.126.238 W srv prompt_save: - saving prompt with length 955, total state size = 149.231 MiB (draft: 0.000 MiB) 6.03.239.106 I srv load: - looking for better prompt, base f_keep = 0.073, sim = 0.722 6.03.239.116 I srv update: - cache state: 17 prompts, 1980.630 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 6.03.239.119 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 6.03.239.120 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 6.03.239.120 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 6.03.239.121 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 6.03.239.121 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 6.03.239.124 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 6.03.239.125 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 6.03.239.125 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 6.03.239.126 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 6.03.239.130 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 6.03.239.132 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 6.03.239.133 I srv update: - prompt 0x5aacf2e90fb0: 495 tokens, checkpoints: 0, 77.350 MiB 6.03.239.133 I srv update: - prompt 0x5aacf460a850: 388 tokens, checkpoints: 0, 60.630 MiB 6.03.239.134 I srv update: - prompt 0x5aacf4619bc0: 607 tokens, checkpoints: 0, 94.852 MiB 6.03.239.134 I srv update: - prompt 0x5aacf47a2aa0: 595 tokens, checkpoints: 0, 92.976 MiB 6.03.239.134 I srv update: - prompt 0x5aacf452c120: 929 tokens, checkpoints: 0, 145.168 MiB 6.03.239.137 I srv update: - prompt 0x5aacf46010f0: 955 tokens, checkpoints: 0, 149.231 MiB 6.03.239.138 I srv get_availabl: prompt cache update took 112.96 ms 6.03.239.164 I slot launch_slot_: id 3 | task 10981 | processing task, is_child = 0 6.03.239.165 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 6.03.239.166 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 6.03.239.177 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 6.03.239.177 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 6.03.239.197 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 6.03.239.198 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 6.06.581.452 I slot print_timing: id 3 | task 10981 | n_decoded = 100, tg = 32.15 t/s, tg_3s = 32.15 t/s 6.09.603.942 I slot print_timing: id 3 | task 10981 | n_decoded = 196, tg = 31.96 t/s, tg_3s = 31.76 t/s 6.12.629.735 I slot print_timing: id 3 | task 10981 | n_decoded = 292, tg = 31.88 t/s, tg_3s = 31.73 t/s 6.15.660.533 I slot print_timing: id 3 | task 10981 | n_decoded = 388, tg = 31.83 t/s, tg_3s = 31.67 t/s 6.18.667.495 I slot print_timing: id 3 | task 10981 | n_decoded = 483, tg = 31.78 t/s, tg_3s = 31.59 t/s 6.21.677.567 I slot print_timing: id 3 | task 10981 | n_decoded = 578, tg = 31.75 t/s, tg_3s = 31.56 t/s 6.23.358.668 I slot print_timing: id 3 | task 10981 | prompt eval time = 231.58 ms / 27 tokens ( 8.58 ms per token, 116.59 tokens per second) 6.23.358.675 I slot print_timing: id 3 | task 10981 | eval time = 19887.86 ms / 631 tokens ( 31.52 ms per token, 31.73 tokens per second) 6.23.358.675 I slot print_timing: id 3 | task 10981 | total time = 20119.44 ms / 658 tokens 6.23.358.676 I slot print_timing: id 3 | task 10981 | graphs reused = 0 6.23.358.695 I slot release: id 3 | task 10981 | stop processing: n_tokens = 727, truncated = 0 6.23.358.702 I srv update_slots: all slots are idle 6.23.361.489 I srv operator(): Chat format: peg-native 6.23.361.609 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.814 (> 0.100 thold), f_keep = 0.096 6.23.361.611 I srv get_availabl: updating prompt cache 6.23.361.650 W srv prompt_save: - saving prompt with length 727, total state size = 113.603 MiB (draft: 0.000 MiB) 6.23.454.016 I srv load: - looking for better prompt, base f_keep = 0.096, sim = 0.814 6.23.454.028 I srv update: - cache state: 18 prompts, 2094.233 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 6.23.454.029 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 6.23.454.033 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 6.23.454.033 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 6.23.454.034 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 6.23.454.035 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 6.23.454.035 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 6.23.454.037 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 6.23.454.037 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 6.23.454.038 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 6.23.454.039 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 6.23.454.039 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 6.23.454.040 I srv update: - prompt 0x5aacf2e90fb0: 495 tokens, checkpoints: 0, 77.350 MiB 6.23.454.040 I srv update: - prompt 0x5aacf460a850: 388 tokens, checkpoints: 0, 60.630 MiB 6.23.454.041 I srv update: - prompt 0x5aacf4619bc0: 607 tokens, checkpoints: 0, 94.852 MiB 6.23.454.041 I srv update: - prompt 0x5aacf47a2aa0: 595 tokens, checkpoints: 0, 92.976 MiB 6.23.454.042 I srv update: - prompt 0x5aacf452c120: 929 tokens, checkpoints: 0, 145.168 MiB 6.23.454.042 I srv update: - prompt 0x5aacf46010f0: 955 tokens, checkpoints: 0, 149.231 MiB 6.23.454.043 I srv update: - prompt 0x5aacf44f27d0: 727 tokens, checkpoints: 0, 113.603 MiB 6.23.454.044 I srv get_availabl: prompt cache update took 92.43 ms 6.23.454.069 I slot launch_slot_: id 3 | task 11613 | processing task, is_child = 0 6.23.454.069 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 6.23.454.070 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 6.23.454.081 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 6.23.454.082 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 6.23.454.102 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 6.23.454.102 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 6.26.753.807 I slot print_timing: id 3 | task 11613 | n_decoded = 100, tg = 31.98 t/s, tg_3s = 31.98 t/s 6.29.784.285 I slot print_timing: id 3 | task 11613 | n_decoded = 196, tg = 31.83 t/s, tg_3s = 31.68 t/s 6.31.917.338 I slot print_timing: id 3 | task 11613 | prompt eval time = 172.87 ms / 16 tokens ( 10.80 ms per token, 92.55 tokens per second) 6.31.917.341 I slot print_timing: id 3 | task 11613 | eval time = 8290.33 ms / 263 tokens ( 31.52 ms per token, 31.72 tokens per second) 6.31.917.342 I slot print_timing: id 3 | task 11613 | total time = 8463.20 ms / 279 tokens 6.31.917.343 I slot print_timing: id 3 | task 11613 | graphs reused = 0 6.31.917.362 I slot release: id 3 | task 11613 | stop processing: n_tokens = 348, truncated = 0 6.31.917.370 I srv update_slots: all slots are idle 6.31.920.083 I srv operator(): Chat format: peg-native 6.31.920.222 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.648 (> 0.100 thold), f_keep = 0.201 6.31.920.224 I srv get_availabl: updating prompt cache 6.31.920.254 W srv prompt_save: - saving prompt with length 348, total state size = 54.380 MiB (draft: 0.000 MiB) 6.31.961.238 I srv load: - looking for better prompt, base f_keep = 0.201, sim = 0.648 6.31.961.252 I srv update: - cache state: 19 prompts, 2148.613 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 6.31.961.253 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 6.31.961.253 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 6.31.961.254 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 6.31.961.254 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 6.31.961.255 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 6.31.961.255 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 6.31.961.256 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 6.31.961.257 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 6.31.961.263 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 6.31.961.263 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 6.31.961.264 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 6.31.961.264 I srv update: - prompt 0x5aacf2e90fb0: 495 tokens, checkpoints: 0, 77.350 MiB 6.31.961.265 I srv update: - prompt 0x5aacf460a850: 388 tokens, checkpoints: 0, 60.630 MiB 6.31.961.265 I srv update: - prompt 0x5aacf4619bc0: 607 tokens, checkpoints: 0, 94.852 MiB 6.31.961.266 I srv update: - prompt 0x5aacf47a2aa0: 595 tokens, checkpoints: 0, 92.976 MiB 6.31.961.266 I srv update: - prompt 0x5aacf452c120: 929 tokens, checkpoints: 0, 145.168 MiB 6.31.961.269 I srv update: - prompt 0x5aacf46010f0: 955 tokens, checkpoints: 0, 149.231 MiB 6.31.961.270 I srv update: - prompt 0x5aacf44f27d0: 727 tokens, checkpoints: 0, 113.603 MiB 6.31.961.270 I srv update: - prompt 0x5aacf47ba950: 348 tokens, checkpoints: 0, 54.380 MiB 6.31.961.272 I srv get_availabl: prompt cache update took 41.05 ms 6.31.961.310 I slot launch_slot_: id 3 | task 11877 | processing task, is_child = 0 6.31.961.310 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 6.31.961.311 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 6.31.961.325 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 6.31.961.325 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 6.31.961.334 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 6.31.961.334 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 6.35.274.586 I slot print_timing: id 3 | task 11877 | n_decoded = 100, tg = 31.88 t/s, tg_3s = 31.88 t/s 6.38.289.738 I slot print_timing: id 3 | task 11877 | n_decoded = 195, tg = 31.70 t/s, tg_3s = 31.51 t/s 6.41.319.704 I slot print_timing: id 3 | task 11877 | n_decoded = 290, tg = 31.58 t/s, tg_3s = 31.35 t/s 6.44.326.739 I slot print_timing: id 3 | task 11877 | n_decoded = 384, tg = 31.50 t/s, tg_3s = 31.26 t/s 6.47.334.811 I slot print_timing: id 3 | task 11877 | n_decoded = 478, tg = 31.45 t/s, tg_3s = 31.25 t/s 6.47.942.816 I slot print_timing: id 3 | task 11877 | prompt eval time = 176.43 ms / 38 tokens ( 4.64 ms per token, 215.38 tokens per second) 6.47.942.819 I slot print_timing: id 3 | task 11877 | eval time = 15805.03 ms / 497 tokens ( 31.80 ms per token, 31.45 tokens per second) 6.47.942.820 I slot print_timing: id 3 | task 11877 | total time = 15981.46 ms / 535 tokens 6.47.942.820 I slot print_timing: id 3 | task 11877 | graphs reused = 0 6.47.942.840 I slot release: id 3 | task 11877 | stop processing: n_tokens = 604, truncated = 0 6.47.942.848 I srv update_slots: all slots are idle 6.47.945.603 I srv operator(): Chat format: peg-native 6.47.945.712 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.729 (> 0.100 thold), f_keep = 0.116 6.47.945.713 I srv get_availabl: updating prompt cache 6.47.945.753 W srv prompt_save: - saving prompt with length 604, total state size = 94.383 MiB (draft: 0.000 MiB) 6.48.017.030 I srv load: - looking for better prompt, base f_keep = 0.116, sim = 0.729 6.48.017.041 I srv update: - cache state: 20 prompts, 2242.996 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 6.48.017.043 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 6.48.017.043 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 6.48.017.044 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 6.48.017.044 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 6.48.017.045 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 6.48.017.047 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 6.48.017.047 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 6.48.017.048 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 6.48.017.049 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 6.48.017.049 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 6.48.017.050 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 6.48.017.050 I srv update: - prompt 0x5aacf2e90fb0: 495 tokens, checkpoints: 0, 77.350 MiB 6.48.017.051 I srv update: - prompt 0x5aacf460a850: 388 tokens, checkpoints: 0, 60.630 MiB 6.48.017.051 I srv update: - prompt 0x5aacf4619bc0: 607 tokens, checkpoints: 0, 94.852 MiB 6.48.017.051 I srv update: - prompt 0x5aacf47a2aa0: 595 tokens, checkpoints: 0, 92.976 MiB 6.48.017.052 I srv update: - prompt 0x5aacf452c120: 929 tokens, checkpoints: 0, 145.168 MiB 6.48.017.052 I srv update: - prompt 0x5aacf46010f0: 955 tokens, checkpoints: 0, 149.231 MiB 6.48.017.053 I srv update: - prompt 0x5aacf44f27d0: 727 tokens, checkpoints: 0, 113.603 MiB 6.48.017.053 I srv update: - prompt 0x5aacf47ba950: 348 tokens, checkpoints: 0, 54.380 MiB 6.48.017.054 I srv update: - prompt 0x5aacf460f890: 604 tokens, checkpoints: 0, 94.383 MiB 6.48.017.055 I srv get_availabl: prompt cache update took 71.34 ms 6.48.017.082 I slot launch_slot_: id 3 | task 12375 | processing task, is_child = 0 6.48.017.083 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 6.48.017.084 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 6.48.017.094 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 6.48.017.094 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 6.48.017.115 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 6.48.017.115 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 6.51.390.731 I slot print_timing: id 3 | task 12375 | n_decoded = 100, tg = 31.82 t/s, tg_3s = 31.82 t/s 6.54.416.667 I slot print_timing: id 3 | task 12375 | n_decoded = 196, tg = 31.77 t/s, tg_3s = 31.73 t/s 6.57.441.978 I slot print_timing: id 3 | task 12375 | n_decoded = 292, tg = 31.76 t/s, tg_3s = 31.73 t/s 7.00.472.565 I slot print_timing: id 3 | task 12375 | n_decoded = 388, tg = 31.74 t/s, tg_3s = 31.68 t/s 7.03.480.597 I slot print_timing: id 3 | task 12375 | n_decoded = 483, tg = 31.71 t/s, tg_3s = 31.58 t/s 7.06.490.626 I slot print_timing: id 3 | task 12375 | n_decoded = 578, tg = 31.68 t/s, tg_3s = 31.56 t/s 7.07.504.645 I slot print_timing: id 3 | task 12375 | prompt eval time = 230.67 ms / 26 tokens ( 8.87 ms per token, 112.71 tokens per second) 7.07.504.652 I slot print_timing: id 3 | task 12375 | eval time = 19256.82 ms / 610 tokens ( 31.57 ms per token, 31.68 tokens per second) 7.07.504.653 I slot print_timing: id 3 | task 12375 | total time = 19487.49 ms / 636 tokens 7.07.504.653 I slot print_timing: id 3 | task 12375 | graphs reused = 0 7.07.504.672 I slot release: id 3 | task 12375 | stop processing: n_tokens = 705, truncated = 0 7.07.504.681 I srv update_slots: all slots are idle 7.07.507.540 I srv operator(): Chat format: peg-native 7.07.507.655 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.673 (> 0.100 thold), f_keep = 0.099 7.07.507.657 I srv get_availabl: updating prompt cache 7.07.507.693 W srv prompt_save: - saving prompt with length 705, total state size = 110.165 MiB (draft: 0.000 MiB) 7.07.591.264 I srv load: - looking for better prompt, base f_keep = 0.099, sim = 0.673 7.07.591.274 I srv update: - cache state: 21 prompts, 2353.161 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 7.07.591.277 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 7.07.591.278 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 7.07.591.278 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 7.07.591.279 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 7.07.591.279 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 7.07.591.280 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 7.07.591.280 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 7.07.591.281 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 7.07.591.282 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 7.07.591.282 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 7.07.591.283 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 7.07.591.283 I srv update: - prompt 0x5aacf2e90fb0: 495 tokens, checkpoints: 0, 77.350 MiB 7.07.591.284 I srv update: - prompt 0x5aacf460a850: 388 tokens, checkpoints: 0, 60.630 MiB 7.07.591.284 I srv update: - prompt 0x5aacf4619bc0: 607 tokens, checkpoints: 0, 94.852 MiB 7.07.591.285 I srv update: - prompt 0x5aacf47a2aa0: 595 tokens, checkpoints: 0, 92.976 MiB 7.07.591.285 I srv update: - prompt 0x5aacf452c120: 929 tokens, checkpoints: 0, 145.168 MiB 7.07.591.286 I srv update: - prompt 0x5aacf46010f0: 955 tokens, checkpoints: 0, 149.231 MiB 7.07.591.286 I srv update: - prompt 0x5aacf44f27d0: 727 tokens, checkpoints: 0, 113.603 MiB 7.07.591.287 I srv update: - prompt 0x5aacf47ba950: 348 tokens, checkpoints: 0, 54.380 MiB 7.07.591.287 I srv update: - prompt 0x5aacf460f890: 604 tokens, checkpoints: 0, 94.383 MiB 7.07.591.288 I srv update: - prompt 0x5aacf126e120: 705 tokens, checkpoints: 0, 110.165 MiB 7.07.591.289 I srv get_availabl: prompt cache update took 83.63 ms 7.07.591.314 I slot launch_slot_: id 3 | task 12986 | processing task, is_child = 0 7.07.591.314 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 7.07.591.315 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 7.07.591.327 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 7.07.591.328 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 7.07.591.336 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 7.07.591.336 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 7.10.887.965 I slot print_timing: id 3 | task 12986 | n_decoded = 100, tg = 32.04 t/s, tg_3s = 32.04 t/s 7.13.919.044 I slot print_timing: id 3 | task 12986 | n_decoded = 196, tg = 31.86 t/s, tg_3s = 31.67 t/s 7.16.450.671 I slot print_timing: id 3 | task 12986 | prompt eval time = 175.90 ms / 34 tokens ( 5.17 ms per token, 193.29 tokens per second) 7.16.450.677 I slot print_timing: id 3 | task 12986 | eval time = 8683.41 ms / 276 tokens ( 31.46 ms per token, 31.78 tokens per second) 7.16.450.678 I slot print_timing: id 3 | task 12986 | total time = 8859.31 ms / 310 tokens 7.16.450.678 I slot print_timing: id 3 | task 12986 | graphs reused = 0 7.16.450.699 I slot release: id 3 | task 12986 | stop processing: n_tokens = 379, truncated = 0 7.16.450.708 I srv update_slots: all slots are idle 7.16.453.578 I srv operator(): Chat format: peg-native 7.16.453.705 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.636 (> 0.100 thold), f_keep = 0.185 7.16.453.708 I srv get_availabl: updating prompt cache 7.16.453.742 W srv prompt_save: - saving prompt with length 379, total state size = 59.224 MiB (draft: 0.000 MiB) 7.16.498.503 I srv load: - looking for better prompt, base f_keep = 0.185, sim = 0.636 7.16.498.518 I srv update: - cache state: 22 prompts, 2412.385 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 7.16.498.518 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 7.16.498.519 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 7.16.498.520 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 7.16.498.520 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 7.16.498.521 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 7.16.498.521 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 7.16.498.522 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 7.16.498.533 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 7.16.498.534 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 7.16.498.535 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 7.16.498.535 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 7.16.498.536 I srv update: - prompt 0x5aacf2e90fb0: 495 tokens, checkpoints: 0, 77.350 MiB 7.16.498.536 I srv update: - prompt 0x5aacf460a850: 388 tokens, checkpoints: 0, 60.630 MiB 7.16.498.537 I srv update: - prompt 0x5aacf4619bc0: 607 tokens, checkpoints: 0, 94.852 MiB 7.16.498.537 I srv update: - prompt 0x5aacf47a2aa0: 595 tokens, checkpoints: 0, 92.976 MiB 7.16.498.538 I srv update: - prompt 0x5aacf452c120: 929 tokens, checkpoints: 0, 145.168 MiB 7.16.498.538 I srv update: - prompt 0x5aacf46010f0: 955 tokens, checkpoints: 0, 149.231 MiB 7.16.498.539 I srv update: - prompt 0x5aacf44f27d0: 727 tokens, checkpoints: 0, 113.603 MiB 7.16.498.539 I srv update: - prompt 0x5aacf47ba950: 348 tokens, checkpoints: 0, 54.380 MiB 7.16.498.539 I srv update: - prompt 0x5aacf460f890: 604 tokens, checkpoints: 0, 94.383 MiB 7.16.498.540 I srv update: - prompt 0x5aacf126e120: 705 tokens, checkpoints: 0, 110.165 MiB 7.16.498.540 I srv update: - prompt 0x5aacf4620480: 379 tokens, checkpoints: 0, 59.224 MiB 7.16.498.542 I srv get_availabl: prompt cache update took 44.83 ms 7.16.498.580 I slot launch_slot_: id 3 | task 13263 | processing task, is_child = 0 7.16.498.581 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 7.16.498.582 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 7.16.498.596 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 7.16.498.596 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 7.16.498.605 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 7.16.498.605 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 7.19.803.372 I slot print_timing: id 3 | task 13263 | n_decoded = 100, tg = 31.97 t/s, tg_3s = 31.97 t/s 7.22.814.678 I slot print_timing: id 3 | task 13263 | n_decoded = 195, tg = 31.76 t/s, tg_3s = 31.55 t/s 7.25.835.091 I slot print_timing: id 3 | task 13263 | n_decoded = 290, tg = 31.66 t/s, tg_3s = 31.45 t/s 7.28.854.640 I slot print_timing: id 3 | task 13263 | n_decoded = 385, tg = 31.61 t/s, tg_3s = 31.46 t/s 7.31.879.958 I slot print_timing: id 3 | task 13263 | n_decoded = 480, tg = 31.57 t/s, tg_3s = 31.40 t/s 7.33.031.849 I slot print_timing: id 3 | task 13263 | prompt eval time = 176.50 ms / 40 tokens ( 4.41 ms per token, 226.63 tokens per second) 7.33.031.852 I slot print_timing: id 3 | task 13263 | eval time = 16356.73 ms / 516 tokens ( 31.70 ms per token, 31.55 tokens per second) 7.33.031.853 I slot print_timing: id 3 | task 13263 | total time = 16533.22 ms / 556 tokens 7.33.031.854 I slot print_timing: id 3 | task 13263 | graphs reused = 0 7.33.031.880 I slot release: id 3 | task 13263 | stop processing: n_tokens = 625, truncated = 0 7.33.031.888 I srv update_slots: all slots are idle 7.33.034.438 I srv operator(): Chat format: peg-native 7.33.034.548 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.737 (> 0.100 thold), f_keep = 0.112 7.33.034.550 I srv get_availabl: updating prompt cache 7.33.034.595 W srv prompt_save: - saving prompt with length 625, total state size = 97.664 MiB (draft: 0.000 MiB) 7.33.109.698 I srv load: - looking for better prompt, base f_keep = 0.112, sim = 0.737 7.33.109.710 I srv update: - cache state: 23 prompts, 2510.049 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est) 7.33.109.711 I srv update: - prompt 0x5aacf45160c0: 1512 tokens, checkpoints: 0, 236.268 MiB 7.33.109.712 I srv update: - prompt 0x5aacf461de00: 560 tokens, checkpoints: 0, 87.507 MiB 7.33.109.713 I srv update: - prompt 0x5aacf3e76170: 401 tokens, checkpoints: 0, 62.662 MiB 7.33.109.713 I srv update: - prompt 0x5aacf47ba8a0: 964 tokens, checkpoints: 0, 150.637 MiB 7.33.109.714 I srv update: - prompt 0x5aacf44fe780: 1795 tokens, checkpoints: 0, 280.490 MiB 7.33.109.715 I srv update: - prompt 0x5aacf3fc9390: 385 tokens, checkpoints: 0, 60.162 MiB 7.33.109.716 I srv update: - prompt 0x5aacf1374510: 425 tokens, checkpoints: 0, 66.412 MiB 7.33.109.716 I srv update: - prompt 0x5aacf4525f80: 947 tokens, checkpoints: 0, 147.981 MiB 7.33.109.717 I srv update: - prompt 0x5aacf44ed2e0: 432 tokens, checkpoints: 0, 67.506 MiB 7.33.109.717 I srv update: - prompt 0x5aacf45200e0: 542 tokens, checkpoints: 0, 84.695 MiB 7.33.109.718 I srv update: - prompt 0x5aacf47c5be0: 743 tokens, checkpoints: 0, 116.103 MiB 7.33.109.719 I srv update: - prompt 0x5aacf2e90fb0: 495 tokens, checkpoints: 0, 77.350 MiB 7.33.109.719 I srv update: - prompt 0x5aacf460a850: 388 tokens, checkpoints: 0, 60.630 MiB 7.33.109.720 I srv update: - prompt 0x5aacf4619bc0: 607 tokens, checkpoints: 0, 94.852 MiB 7.33.109.721 I srv update: - prompt 0x5aacf47a2aa0: 595 tokens, checkpoints: 0, 92.976 MiB 7.33.109.721 I srv update: - prompt 0x5aacf452c120: 929 tokens, checkpoints: 0, 145.168 MiB 7.33.109.722 I srv update: - prompt 0x5aacf46010f0: 955 tokens, checkpoints: 0, 149.231 MiB 7.33.109.723 I srv update: - prompt 0x5aacf44f27d0: 727 tokens, checkpoints: 0, 113.603 MiB 7.33.109.723 I srv update: - prompt 0x5aacf47ba950: 348 tokens, checkpoints: 0, 54.380 MiB 7.33.109.724 I srv update: - prompt 0x5aacf460f890: 604 tokens, checkpoints: 0, 94.383 MiB 7.33.109.724 I srv update: - prompt 0x5aacf126e120: 705 tokens, checkpoints: 0, 110.165 MiB 7.33.109.725 I srv update: - prompt 0x5aacf4620480: 379 tokens, checkpoints: 0, 59.224 MiB 7.33.109.726 I srv update: - prompt 0x5aacf460fca0: 625 tokens, checkpoints: 0, 97.664 MiB 7.33.109.727 I srv get_availabl: prompt cache update took 75.18 ms 7.33.109.755 I slot launch_slot_: id 3 | task 13780 | processing task, is_child = 0 7.33.109.756 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 7.33.109.757 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 7.33.109.774 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 7.33.109.774 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 7.33.109.789 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 7.33.109.790 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 7.36.478.926 I slot print_timing: id 3 | task 13780 | n_decoded = 100, tg = 31.87 t/s, tg_3s = 31.87 t/s 7.39.505.469 I slot print_timing: id 3 | task 13780 | n_decoded = 196, tg = 31.79 t/s, tg_3s = 31.72 t/s 7.42.534.609 I slot print_timing: id 3 | task 13780 | n_decoded = 292, tg = 31.76 t/s, tg_3s = 31.69 t/s 7.45.566.030 I slot print_timing: id 3 | task 13780 | n_decoded = 388, tg = 31.74 t/s, tg_3s = 31.67 t/s 7.48.573.613 I slot print_timing: id 3 | task 13780 | n_decoded = 483, tg = 31.71 t/s, tg_3s = 31.59 t/s 7.51.587.265 I slot print_timing: id 3 | task 13780 | n_decoded = 578, tg = 31.68 t/s, tg_3s = 31.52 t/s 7.54.317.724 I slot print_timing: id 3 | task 13780 | prompt eval time = 230.95 ms / 25 tokens ( 9.24 ms per token, 108.25 tokens per second) 7.54.317.730 I slot print_timing: id 3 | task 13780 | eval time = 20976.96 ms / 664 tokens ( 31.59 ms per token, 31.65 tokens per second) 7.54.317.730 I slot print_timing: id 3 | task 13780 | total time = 21207.91 ms / 689 tokens 7.54.317.731 I slot print_timing: id 3 | task 13780 | graphs reused = 0 7.54.317.750 I slot release: id 3 | task 13780 | stop processing: n_tokens = 758, truncated = 0 7.54.317.758 I srv update_slots: all slots are idle 7.54.318.548 I srv operator(): operator(): cleaning up before exit...