0.00.066.472 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.066.476 I device_info: 0.00.066.495 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.066.500 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.066.506 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free) 0.00.066.561 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.066.564 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true 0.00.066.605 I srv init: running without SSL 0.00.066.666 I srv init: using 23 threads for HTTP server 0.00.066.992 I srv start: binding port with default address family 0.00.068.197 I srv llama_server: loading model 0.00.068.206 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/GLM-4.6V-UD-IQ2_M.gguf' 0.00.068.263 I common_init_result: fitting params to device memory ... 0.00.068.265 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) 0.01.160.665 W load: special_eot_id is not in special_eog_ids - the tokenizer config may be incorrect 0.01.160.672 W load: special_eom_id is not in special_eog_ids - the tokenizer config may be incorrect 0.27.540.426 W llama_context: n_ctx_seq (16384) < n_ctx_train (131072) -- the full capacity of the model will not be utilized 0.27.706.120 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) 0.27.923.093 I srv load_model: initializing slots, n_slots = 4 0.27.995.106 W common_speculative_init: no implementations specified for speculative decoding 0.27.995.111 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 0.27.995.115 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384 0.27.995.115 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384 0.27.995.116 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384 0.27.995.157 I srv load_model: prompt cache is enabled, size limit: 8192 MiB 0.27.995.158 I srv load_model: use `--cache-ram 0` to disable the prompt cache 0.27.995.158 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 0.27.995.158 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 0.27.995.192 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task 0.28.010.203 I init: chat template, example_format: '[gMASK]<|system|> You are a helpful assistant<|user|> Hello<|assistant|> Hi there<|user|> How are you?<|assistant|> ' 0.28.019.395 I srv init: init: chat template, thinking = 1 0.28.019.432 I srv llama_server: model loaded 0.28.019.436 I srv llama_server: server is listening on http://127.0.0.1:8090 0.28.019.446 I srv update_slots: all slots are idle 0.29.990.711 I srv operator(): Chat format: peg-native 0.29.990.994 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1 0.29.990.997 I srv get_availabl: updating prompt cache 0.29.991.004 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 0.29.991.009 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) 0.29.991.009 I srv get_availabl: prompt cache update took 0.01 ms 0.29.991.065 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0 0.29.991.066 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.29.991.067 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.29.991.081 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.29.991.081 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.29.991.092 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.29.991.092 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.34.884.595 I slot print_timing: id 3 | task 0 | n_decoded = 100, tg = 31.67 t/s, tg_3s = 31.67 t/s 0.37.893.784 I slot print_timing: id 3 | task 0 | n_decoded = 194, tg = 31.46 t/s, tg_3s = 31.24 t/s 0.40.925.239 I slot print_timing: id 3 | task 0 | n_decoded = 288, tg = 31.31 t/s, tg_3s = 31.01 t/s 0.43.930.817 I slot print_timing: id 3 | task 0 | n_decoded = 381, tg = 31.22 t/s, tg_3s = 30.94 t/s 0.46.946.666 I slot print_timing: id 3 | task 0 | n_decoded = 474, tg = 31.14 t/s, tg_3s = 30.84 t/s 0.49.957.664 I slot print_timing: id 3 | task 0 | n_decoded = 567, tg = 31.10 t/s, tg_3s = 30.89 t/s 0.52.965.710 I slot print_timing: id 3 | task 0 | n_decoded = 660, tg = 31.08 t/s, tg_3s = 30.92 t/s 0.55.992.442 I slot print_timing: id 3 | task 0 | n_decoded = 751, tg = 30.95 t/s, tg_3s = 30.07 t/s 0.58.997.483 I slot print_timing: id 3 | task 0 | n_decoded = 841, tg = 30.84 t/s, tg_3s = 29.95 t/s 1.01.998.349 I slot print_timing: id 3 | task 0 | n_decoded = 931, tg = 30.75 t/s, tg_3s = 29.99 t/s 1.04.998.738 I slot print_timing: id 3 | task 0 | n_decoded = 1019, tg = 30.63 t/s, tg_3s = 29.33 t/s 1.08.017.800 I slot print_timing: id 3 | task 0 | n_decoded = 1109, tg = 30.56 t/s, tg_3s = 29.81 t/s 1.11.050.365 I slot print_timing: id 3 | task 0 | n_decoded = 1199, tg = 30.49 t/s, tg_3s = 29.68 t/s 1.14.064.667 I slot print_timing: id 3 | task 0 | n_decoded = 1288, tg = 30.42 t/s, tg_3s = 29.53 t/s 1.17.073.152 I slot print_timing: id 3 | task 0 | n_decoded = 1377, tg = 30.37 t/s, tg_3s = 29.58 t/s 1.20.103.292 I slot print_timing: id 3 | task 0 | n_decoded = 1466, tg = 30.30 t/s, tg_3s = 29.37 t/s 1.23.129.315 I slot print_timing: id 3 | task 0 | n_decoded = 1555, tg = 30.25 t/s, tg_3s = 29.41 t/s 1.26.156.363 I slot print_timing: id 3 | task 0 | n_decoded = 1644, tg = 30.20 t/s, tg_3s = 29.40 t/s 1.29.177.858 I slot print_timing: id 3 | task 0 | n_decoded = 1732, tg = 30.15 t/s, tg_3s = 29.12 t/s 1.32.191.665 I slot print_timing: id 3 | task 0 | n_decoded = 1819, tg = 30.08 t/s, tg_3s = 28.87 t/s 1.35.214.712 I slot print_timing: id 3 | task 0 | n_decoded = 1906, tg = 30.02 t/s, tg_3s = 28.78 t/s 1.38.224.165 I slot print_timing: id 3 | task 0 | n_decoded = 1992, tg = 29.96 t/s, tg_3s = 28.58 t/s 1.41.242.279 I slot print_timing: id 3 | task 0 | n_decoded = 2078, tg = 29.89 t/s, tg_3s = 28.49 t/s 1.44.252.237 I slot print_timing: id 3 | task 0 | n_decoded = 2164, tg = 29.84 t/s, tg_3s = 28.57 t/s 1.47.278.256 I slot print_timing: id 3 | task 0 | n_decoded = 2250, tg = 29.78 t/s, tg_3s = 28.42 t/s 1.50.298.936 I slot print_timing: id 3 | task 0 | n_decoded = 2336, tg = 29.73 t/s, tg_3s = 28.47 t/s 1.53.316.997 I slot print_timing: id 3 | task 0 | n_decoded = 2422, tg = 29.68 t/s, tg_3s = 28.50 t/s 1.56.337.393 I slot print_timing: id 3 | task 0 | n_decoded = 2508, tg = 29.64 t/s, tg_3s = 28.47 t/s 1.59.351.074 I slot print_timing: id 3 | task 0 | n_decoded = 2594, tg = 29.60 t/s, tg_3s = 28.54 t/s 2.02.368.775 I slot print_timing: id 3 | task 0 | n_decoded = 2680, tg = 29.57 t/s, tg_3s = 28.50 t/s 2.05.393.085 I slot print_timing: id 3 | task 0 | n_decoded = 2765, tg = 29.52 t/s, tg_3s = 28.11 t/s 2.08.419.376 I slot print_timing: id 3 | task 0 | n_decoded = 2849, tg = 29.46 t/s, tg_3s = 27.76 t/s 2.10.867.190 I slot print_timing: id 3 | task 0 | prompt eval time = 1735.67 ms / 93 tokens ( 18.66 ms per token, 53.58 tokens per second) 2.10.867.193 I slot print_timing: id 3 | task 0 | eval time = 99140.40 ms / 2917 tokens ( 33.99 ms per token, 29.42 tokens per second) 2.10.867.194 I slot print_timing: id 3 | task 0 | total time = 100876.07 ms / 3010 tokens 2.10.867.198 I slot print_timing: id 3 | task 0 | graphs reused = 2905 2.10.867.254 I slot release: id 3 | task 0 | stop processing: n_tokens = 3009, truncated = 0 2.10.867.268 I srv update_slots: all slots are idle 2.10.879.745 I srv operator(): Chat format: peg-native 2.10.879.984 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.828 (> 0.100 thold), f_keep = 0.024 2.10.879.986 I srv get_availabl: updating prompt cache 2.10.880.123 W srv prompt_save: - saving prompt with length 3009, total state size = 540.738 MiB (draft: 0.000 MiB) 2.11.511.735 I srv load: - looking for better prompt, base f_keep = 0.024, sim = 0.828 2.11.511.740 I srv update: - cache state: 1 prompts, 540.738 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 2.11.511.741 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 2.11.511.742 I srv get_availabl: prompt cache update took 631.75 ms 2.11.511.800 I slot launch_slot_: id 3 | task 2919 | processing task, is_child = 0 2.11.511.800 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.11.511.801 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.11.511.818 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.11.511.818 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.11.511.838 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.11.511.838 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.15.032.133 I slot print_timing: id 3 | task 2919 | n_decoded = 100, tg = 31.71 t/s, tg_3s = 31.71 t/s 2.18.042.803 I slot print_timing: id 3 | task 2919 | n_decoded = 194, tg = 31.47 t/s, tg_3s = 31.22 t/s 2.21.058.878 I slot print_timing: id 3 | task 2919 | n_decoded = 288, tg = 31.37 t/s, tg_3s = 31.17 t/s 2.24.081.897 I slot print_timing: id 3 | task 2919 | n_decoded = 382, tg = 31.30 t/s, tg_3s = 31.09 t/s 2.27.084.904 I slot print_timing: id 3 | task 2919 | n_decoded = 475, tg = 31.24 t/s, tg_3s = 30.97 t/s 2.30.090.202 I slot print_timing: id 3 | task 2919 | n_decoded = 568, tg = 31.19 t/s, tg_3s = 30.95 t/s 2.33.098.391 I slot print_timing: id 3 | task 2919 | n_decoded = 661, tg = 31.15 t/s, tg_3s = 30.92 t/s 2.36.119.868 I slot print_timing: id 3 | task 2919 | n_decoded = 752, tg = 31.02 t/s, tg_3s = 30.12 t/s 2.39.138.729 I slot print_timing: id 3 | task 2919 | n_decoded = 843, tg = 30.92 t/s, tg_3s = 30.14 t/s 2.42.145.833 I slot print_timing: id 3 | task 2919 | n_decoded = 934, tg = 30.86 t/s, tg_3s = 30.26 t/s 2.45.178.720 I slot print_timing: id 3 | task 2919 | n_decoded = 1023, tg = 30.72 t/s, tg_3s = 29.34 t/s 2.48.206.970 I slot print_timing: id 3 | task 2919 | n_decoded = 1113, tg = 30.64 t/s, tg_3s = 29.72 t/s 2.49.054.094 I slot print_timing: id 3 | task 2919 | prompt eval time = 366.77 ms / 15 tokens ( 24.45 ms per token, 40.90 tokens per second) 2.49.054.098 I slot print_timing: id 3 | task 2919 | eval time = 37175.45 ms / 1138 tokens ( 32.67 ms per token, 30.61 tokens per second) 2.49.054.099 I slot print_timing: id 3 | task 2919 | total time = 37542.22 ms / 1153 tokens 2.49.054.099 I slot print_timing: id 3 | task 2919 | graphs reused = 4037 2.49.054.137 I slot release: id 3 | task 2919 | stop processing: n_tokens = 1224, truncated = 0 2.49.054.149 I srv update_slots: all slots are idle 2.49.065.404 I srv operator(): Chat format: peg-native 2.49.065.640 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.818 (> 0.100 thold), f_keep = 0.059 2.49.065.641 I srv get_availabl: updating prompt cache 2.49.065.716 W srv prompt_save: - saving prompt with length 1224, total state size = 219.962 MiB (draft: 0.000 MiB) 2.49.239.854 I srv load: - looking for better prompt, base f_keep = 0.059, sim = 0.818 2.49.239.859 I srv update: - cache state: 2 prompts, 760.700 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 2.49.239.860 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 2.49.239.860 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 2.49.239.861 I srv get_availabl: prompt cache update took 174.22 ms 2.49.239.911 I slot launch_slot_: id 3 | task 4058 | processing task, is_child = 0 2.49.239.911 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.49.239.912 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.49.239.925 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.49.239.926 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.49.239.935 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.49.239.935 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.52.750.416 I slot print_timing: id 3 | task 4058 | n_decoded = 100, tg = 31.72 t/s, tg_3s = 31.72 t/s 2.55.753.685 I slot print_timing: id 3 | task 4058 | n_decoded = 194, tg = 31.52 t/s, tg_3s = 31.30 t/s 2.58.762.881 I slot print_timing: id 3 | task 4058 | n_decoded = 288, tg = 31.43 t/s, tg_3s = 31.24 t/s 3.01.771.888 I slot print_timing: id 3 | task 4058 | n_decoded = 382, tg = 31.38 t/s, tg_3s = 31.24 t/s 3.04.795.696 I slot print_timing: id 3 | task 4058 | n_decoded = 476, tg = 31.32 t/s, tg_3s = 31.09 t/s 3.07.811.735 I slot print_timing: id 3 | task 4058 | n_decoded = 570, tg = 31.30 t/s, tg_3s = 31.17 t/s 3.10.827.778 I slot print_timing: id 3 | task 4058 | n_decoded = 664, tg = 31.28 t/s, tg_3s = 31.17 t/s 3.13.846.464 I slot print_timing: id 3 | task 4058 | n_decoded = 755, tg = 31.14 t/s, tg_3s = 30.15 t/s 3.16.871.974 I slot print_timing: id 3 | task 4058 | n_decoded = 846, tg = 31.02 t/s, tg_3s = 30.08 t/s 3.19.896.919 I slot print_timing: id 3 | task 4058 | n_decoded = 937, tg = 30.93 t/s, tg_3s = 30.08 t/s 3.22.897.236 I slot print_timing: id 3 | task 4058 | n_decoded = 1025, tg = 30.78 t/s, tg_3s = 29.33 t/s 3.25.925.216 I slot print_timing: id 3 | task 4058 | n_decoded = 1114, tg = 30.67 t/s, tg_3s = 29.39 t/s 3.28.925.450 I slot print_timing: id 3 | task 4058 | n_decoded = 1202, tg = 30.56 t/s, tg_3s = 29.33 t/s 3.31.947.134 I slot print_timing: id 3 | task 4058 | n_decoded = 1291, tg = 30.48 t/s, tg_3s = 29.45 t/s 3.34.955.152 I slot print_timing: id 3 | task 4058 | n_decoded = 1380, tg = 30.43 t/s, tg_3s = 29.59 t/s 3.37.977.163 I slot print_timing: id 3 | task 4058 | n_decoded = 1469, tg = 30.36 t/s, tg_3s = 29.45 t/s 3.41.003.198 I slot print_timing: id 3 | task 4058 | n_decoded = 1558, tg = 30.31 t/s, tg_3s = 29.41 t/s 3.43.825.212 I slot print_timing: id 3 | task 4058 | prompt eval time = 358.26 ms / 16 tokens ( 22.39 ms per token, 44.66 tokens per second) 3.43.825.216 I slot print_timing: id 3 | task 4058 | eval time = 54226.99 ms / 1641 tokens ( 33.05 ms per token, 30.26 tokens per second) 3.43.825.217 I slot print_timing: id 3 | task 4058 | total time = 54585.25 ms / 1657 tokens 3.43.825.218 I slot print_timing: id 3 | task 4058 | graphs reused = 5670 3.43.825.253 I slot release: id 3 | task 4058 | stop processing: n_tokens = 1728, truncated = 0 3.43.825.266 I srv update_slots: all slots are idle 3.43.836.372 I srv operator(): Chat format: peg-native 3.43.836.605 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.837 (> 0.100 thold), f_keep = 0.042 3.43.836.606 I srv get_availabl: updating prompt cache 3.43.836.684 W srv prompt_save: - saving prompt with length 1728, total state size = 310.534 MiB (draft: 0.000 MiB) 3.44.134.922 I srv load: - looking for better prompt, base f_keep = 0.042, sim = 0.837 3.44.134.929 I srv update: - cache state: 3 prompts, 1071.234 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 3.44.134.930 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 3.44.134.931 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 3.44.134.931 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 3.44.134.933 I srv get_availabl: prompt cache update took 298.33 ms 3.44.134.986 I slot launch_slot_: id 3 | task 5700 | processing task, is_child = 0 3.44.134.987 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.44.134.987 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.44.135.002 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.44.135.002 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.44.135.012 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.44.135.020 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.47.641.685 I slot print_timing: id 3 | task 5700 | n_decoded = 100, tg = 31.76 t/s, tg_3s = 31.76 t/s 3.50.649.723 I slot print_timing: id 3 | task 5700 | n_decoded = 194, tg = 31.51 t/s, tg_3s = 31.25 t/s 3.53.662.737 I slot print_timing: id 3 | task 5700 | n_decoded = 288, tg = 31.41 t/s, tg_3s = 31.20 t/s 3.56.682.058 I slot print_timing: id 3 | task 5700 | n_decoded = 382, tg = 31.34 t/s, tg_3s = 31.13 t/s 3.59.690.699 I slot print_timing: id 3 | task 5700 | n_decoded = 475, tg = 31.26 t/s, tg_3s = 30.91 t/s 4.02.697.098 I slot print_timing: id 3 | task 5700 | n_decoded = 568, tg = 31.20 t/s, tg_3s = 30.93 t/s 4.05.698.717 I slot print_timing: id 3 | task 5700 | n_decoded = 661, tg = 31.17 t/s, tg_3s = 30.98 t/s 4.08.711.104 I slot print_timing: id 3 | task 5700 | n_decoded = 752, tg = 31.05 t/s, tg_3s = 30.21 t/s 4.11.743.010 I slot print_timing: id 3 | task 5700 | n_decoded = 843, tg = 30.94 t/s, tg_3s = 30.01 t/s 4.14.744.384 I slot print_timing: id 3 | task 5700 | n_decoded = 933, tg = 30.84 t/s, tg_3s = 29.99 t/s 4.17.767.333 I slot print_timing: id 3 | task 5700 | n_decoded = 1022, tg = 30.71 t/s, tg_3s = 29.44 t/s 4.20.789.350 I slot print_timing: id 3 | task 5700 | n_decoded = 1111, tg = 30.61 t/s, tg_3s = 29.45 t/s 4.23.790.524 I slot print_timing: id 3 | task 5700 | n_decoded = 1199, tg = 30.51 t/s, tg_3s = 29.32 t/s 4.26.807.352 I slot print_timing: id 3 | task 5700 | n_decoded = 1288, tg = 30.44 t/s, tg_3s = 29.50 t/s 4.29.822.377 I slot print_timing: id 3 | task 5700 | n_decoded = 1377, tg = 30.38 t/s, tg_3s = 29.52 t/s 4.32.852.350 I slot print_timing: id 3 | task 5700 | n_decoded = 1466, tg = 30.31 t/s, tg_3s = 29.37 t/s 4.35.878.494 I slot print_timing: id 3 | task 5700 | n_decoded = 1555, tg = 30.26 t/s, tg_3s = 29.41 t/s 4.38.904.373 I slot print_timing: id 3 | task 5700 | n_decoded = 1644, tg = 30.21 t/s, tg_3s = 29.41 t/s 4.41.917.993 I slot print_timing: id 3 | task 5700 | n_decoded = 1732, tg = 30.16 t/s, tg_3s = 29.20 t/s 4.44.931.048 I slot print_timing: id 3 | task 5700 | n_decoded = 1819, tg = 30.10 t/s, tg_3s = 28.87 t/s 4.47.942.432 I slot print_timing: id 3 | task 5700 | n_decoded = 1906, tg = 30.04 t/s, tg_3s = 28.89 t/s 4.50.955.530 I slot print_timing: id 3 | task 5700 | n_decoded = 1992, tg = 29.97 t/s, tg_3s = 28.54 t/s 4.52.950.697 I slot print_timing: id 3 | task 5700 | prompt eval time = 358.27 ms / 14 tokens ( 25.59 ms per token, 39.08 tokens per second) 4.52.950.700 I slot print_timing: id 3 | task 5700 | eval time = 68457.39 ms / 2049 tokens ( 33.41 ms per token, 29.93 tokens per second) 4.52.950.701 I slot print_timing: id 3 | task 5700 | total time = 68815.65 ms / 2063 tokens 4.52.950.701 I slot print_timing: id 3 | task 5700 | graphs reused = 7709 4.52.950.735 I slot release: id 3 | task 5700 | stop processing: n_tokens = 2134, truncated = 0 4.52.950.749 I srv update_slots: all slots are idle 4.52.962.203 I srv operator(): Chat format: peg-native 4.52.962.442 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.727 (> 0.100 thold), f_keep = 0.034 4.52.962.444 I srv get_availabl: updating prompt cache 4.52.962.530 W srv prompt_save: - saving prompt with length 2134, total state size = 383.495 MiB (draft: 0.000 MiB) 4.53.292.587 I srv load: - looking for better prompt, base f_keep = 0.034, sim = 0.727 4.53.292.593 I srv update: - cache state: 4 prompts, 1454.729 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 4.53.292.594 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 4.53.292.595 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 4.53.292.596 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 4.53.292.596 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 4.53.292.598 I srv get_availabl: prompt cache update took 330.15 ms 4.53.292.653 I slot launch_slot_: id 3 | task 7750 | processing task, is_child = 0 4.53.292.654 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.53.292.655 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.53.292.669 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.53.292.670 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.53.292.679 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.53.292.679 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.57.022.178 I slot print_timing: id 3 | task 7750 | n_decoded = 100, tg = 31.69 t/s, tg_3s = 31.69 t/s 5.00.042.239 I slot print_timing: id 3 | task 7750 | n_decoded = 194, tg = 31.41 t/s, tg_3s = 31.13 t/s 5.03.042.604 I slot print_timing: id 3 | task 7750 | n_decoded = 287, tg = 31.28 t/s, tg_3s = 31.00 t/s 5.06.064.284 I slot print_timing: id 3 | task 7750 | n_decoded = 381, tg = 31.23 t/s, tg_3s = 31.11 t/s 5.09.072.770 I slot print_timing: id 3 | task 7750 | n_decoded = 474, tg = 31.17 t/s, tg_3s = 30.91 t/s 5.12.102.628 I slot print_timing: id 3 | task 7750 | n_decoded = 568, tg = 31.15 t/s, tg_3s = 31.02 t/s 5.15.117.046 I slot print_timing: id 3 | task 7750 | n_decoded = 662, tg = 31.15 t/s, tg_3s = 31.18 t/s 5.18.122.889 I slot print_timing: id 3 | task 7750 | n_decoded = 753, tg = 31.04 t/s, tg_3s = 30.27 t/s 5.21.128.929 I slot print_timing: id 3 | task 7750 | n_decoded = 844, tg = 30.96 t/s, tg_3s = 30.27 t/s 5.24.147.143 I slot print_timing: id 3 | task 7750 | n_decoded = 935, tg = 30.88 t/s, tg_3s = 30.15 t/s 5.27.179.198 I slot print_timing: id 3 | task 7750 | n_decoded = 1025, tg = 30.77 t/s, tg_3s = 29.68 t/s 5.30.183.145 I slot print_timing: id 3 | task 7750 | n_decoded = 1114, tg = 30.67 t/s, tg_3s = 29.63 t/s 5.33.198.284 I slot print_timing: id 3 | task 7750 | n_decoded = 1203, tg = 30.59 t/s, tg_3s = 29.52 t/s 5.36.222.327 I slot print_timing: id 3 | task 7750 | n_decoded = 1292, tg = 30.50 t/s, tg_3s = 29.43 t/s 5.39.249.317 I slot print_timing: id 3 | task 7750 | n_decoded = 1381, tg = 30.43 t/s, tg_3s = 29.40 t/s 5.42.250.479 I slot print_timing: id 3 | task 7750 | n_decoded = 1469, tg = 30.36 t/s, tg_3s = 29.32 t/s 5.45.278.503 I slot print_timing: id 3 | task 7750 | n_decoded = 1558, tg = 30.30 t/s, tg_3s = 29.39 t/s 5.48.308.589 I slot print_timing: id 3 | task 7750 | n_decoded = 1647, tg = 30.25 t/s, tg_3s = 29.37 t/s 5.51.340.438 I slot print_timing: id 3 | task 7750 | n_decoded = 1735, tg = 30.19 t/s, tg_3s = 29.03 t/s 5.54.362.568 I slot print_timing: id 3 | task 7750 | n_decoded = 1822, tg = 30.12 t/s, tg_3s = 28.79 t/s 5.57.392.276 I slot print_timing: id 3 | task 7750 | n_decoded = 1909, tg = 30.05 t/s, tg_3s = 28.72 t/s 6.00.401.796 I slot print_timing: id 3 | task 7750 | n_decoded = 1995, tg = 29.98 t/s, tg_3s = 28.58 t/s 6.03.411.827 I slot print_timing: id 3 | task 7750 | n_decoded = 2081, tg = 29.92 t/s, tg_3s = 28.57 t/s 6.06.416.543 I slot print_timing: id 3 | task 7750 | n_decoded = 2167, tg = 29.87 t/s, tg_3s = 28.62 t/s 6.09.428.659 I slot print_timing: id 3 | task 7750 | n_decoded = 2253, tg = 29.82 t/s, tg_3s = 28.55 t/s 6.12.438.660 I slot print_timing: id 3 | task 7750 | n_decoded = 2339, tg = 29.77 t/s, tg_3s = 28.57 t/s 6.15.448.721 I slot print_timing: id 3 | task 7750 | n_decoded = 2425, tg = 29.72 t/s, tg_3s = 28.57 t/s 6.18.467.866 I slot print_timing: id 3 | task 7750 | n_decoded = 2511, tg = 29.68 t/s, tg_3s = 28.48 t/s 6.21.094.852 I slot print_timing: id 3 | task 7750 | prompt eval time = 573.66 ms / 27 tokens ( 21.25 ms per token, 47.07 tokens per second) 6.21.094.855 I slot print_timing: id 3 | task 7750 | eval time = 87228.49 ms / 2586 tokens ( 33.73 ms per token, 29.65 tokens per second) 6.21.094.856 I slot print_timing: id 3 | task 7750 | total time = 87802.15 ms / 2613 tokens 6.21.094.856 I slot print_timing: id 3 | task 7750 | graphs reused = 10283 6.21.094.889 I slot release: id 3 | task 7750 | stop processing: n_tokens = 2684, truncated = 0 6.21.094.906 I srv update_slots: all slots are idle 6.21.106.609 I srv operator(): Chat format: peg-native 6.21.106.841 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.809 (> 0.100 thold), f_keep = 0.027 6.21.106.843 I srv get_availabl: updating prompt cache 6.21.106.951 W srv prompt_save: - saving prompt with length 2684, total state size = 482.334 MiB (draft: 0.000 MiB) 6.21.488.637 I srv load: - looking for better prompt, base f_keep = 0.027, sim = 0.809 6.21.488.645 I srv update: - cache state: 5 prompts, 1937.063 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 6.21.488.646 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 6.21.488.647 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 6.21.488.647 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 6.21.488.648 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 6.21.488.649 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 6.21.488.653 I srv get_availabl: prompt cache update took 381.81 ms 6.21.488.704 I slot launch_slot_: id 3 | task 10337 | processing task, is_child = 0 6.21.488.705 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 6.21.488.706 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 6.21.488.722 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 6.21.488.722 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 6.21.488.732 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 6.21.488.732 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 6.25.123.213 I slot print_timing: id 3 | task 10337 | n_decoded = 100, tg = 31.78 t/s, tg_3s = 31.78 t/s 6.28.124.912 I slot print_timing: id 3 | task 10337 | n_decoded = 194, tg = 31.55 t/s, tg_3s = 31.32 t/s 6.31.152.797 I slot print_timing: id 3 | task 10337 | n_decoded = 288, tg = 31.38 t/s, tg_3s = 31.04 t/s 6.34.182.843 I slot print_timing: id 3 | task 10337 | n_decoded = 382, tg = 31.29 t/s, tg_3s = 31.02 t/s 6.37.192.174 I slot print_timing: id 3 | task 10337 | n_decoded = 475, tg = 31.22 t/s, tg_3s = 30.90 t/s 6.40.201.964 I slot print_timing: id 3 | task 10337 | n_decoded = 568, tg = 31.16 t/s, tg_3s = 30.90 t/s 6.43.209.194 I slot print_timing: id 3 | task 10337 | n_decoded = 661, tg = 31.13 t/s, tg_3s = 30.93 t/s 6.46.215.943 I slot print_timing: id 3 | task 10337 | n_decoded = 752, tg = 31.02 t/s, tg_3s = 30.27 t/s 6.49.227.106 I slot print_timing: id 3 | task 10337 | n_decoded = 843, tg = 30.93 t/s, tg_3s = 30.22 t/s 6.52.232.831 I slot print_timing: id 3 | task 10337 | n_decoded = 934, tg = 30.87 t/s, tg_3s = 30.28 t/s 6.55.259.770 I slot print_timing: id 3 | task 10337 | n_decoded = 1024, tg = 30.77 t/s, tg_3s = 29.73 t/s 6.58.284.119 I slot print_timing: id 3 | task 10337 | n_decoded = 1114, tg = 30.68 t/s, tg_3s = 29.76 t/s 7.01.289.162 I slot print_timing: id 3 | task 10337 | n_decoded = 1203, tg = 30.60 t/s, tg_3s = 29.62 t/s 7.04.302.807 I slot print_timing: id 3 | task 10337 | n_decoded = 1292, tg = 30.52 t/s, tg_3s = 29.53 t/s 7.07.316.288 I slot print_timing: id 3 | task 10337 | n_decoded = 1381, tg = 30.46 t/s, tg_3s = 29.53 t/s 7.10.342.341 I slot print_timing: id 3 | task 10337 | n_decoded = 1470, tg = 30.39 t/s, tg_3s = 29.41 t/s 7.13.368.372 I slot print_timing: id 3 | task 10337 | n_decoded = 1559, tg = 30.34 t/s, tg_3s = 29.41 t/s 7.16.394.425 I slot print_timing: id 3 | task 10337 | n_decoded = 1648, tg = 30.28 t/s, tg_3s = 29.41 t/s 7.19.417.380 I slot print_timing: id 3 | task 10337 | n_decoded = 1736, tg = 30.22 t/s, tg_3s = 29.11 t/s 7.22.433.378 I slot print_timing: id 3 | task 10337 | n_decoded = 1823, tg = 30.15 t/s, tg_3s = 28.85 t/s 7.24.933.181 I slot print_timing: id 3 | task 10337 | prompt eval time = 487.52 ms / 17 tokens ( 28.68 ms per token, 34.87 tokens per second) 7.24.933.185 I slot print_timing: id 3 | task 10337 | eval time = 62956.90 ms / 1895 tokens ( 33.22 ms per token, 30.10 tokens per second) 7.24.933.186 I slot print_timing: id 3 | task 10337 | total time = 63444.43 ms / 1912 tokens 7.24.933.186 I slot print_timing: id 3 | task 10337 | graphs reused = 12169 7.24.933.222 I slot release: id 3 | task 10337 | stop processing: n_tokens = 1983, truncated = 0 7.24.933.236 I srv update_slots: all slots are idle 7.24.944.376 I srv operator(): Chat format: peg-native 7.24.944.611 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.869 (> 0.100 thold), f_keep = 0.037 7.24.944.613 I srv get_availabl: updating prompt cache 7.24.944.699 W srv prompt_save: - saving prompt with length 1983, total state size = 356.359 MiB (draft: 0.000 MiB) 7.25.203.254 I srv load: - looking for better prompt, base f_keep = 0.037, sim = 0.869 7.25.203.263 I srv update: - cache state: 6 prompts, 2293.422 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 7.25.203.264 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 7.25.203.265 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 7.25.203.266 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 7.25.203.266 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 7.25.203.267 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 7.25.203.268 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 7.25.203.269 I srv get_availabl: prompt cache update took 258.65 ms 7.25.203.331 I slot launch_slot_: id 3 | task 12233 | processing task, is_child = 0 7.25.203.332 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 7.25.203.333 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 7.25.203.351 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 7.25.203.352 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 7.25.203.362 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 7.25.203.363 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 7.28.682.502 I slot print_timing: id 3 | task 12233 | n_decoded = 100, tg = 31.73 t/s, tg_3s = 31.73 t/s 7.31.688.046 I slot print_timing: id 3 | task 12233 | n_decoded = 194, tg = 31.51 t/s, tg_3s = 31.28 t/s 7.34.697.064 I slot print_timing: id 3 | task 12233 | n_decoded = 287, tg = 31.31 t/s, tg_3s = 30.91 t/s 7.37.704.054 I slot print_timing: id 3 | task 12233 | n_decoded = 380, tg = 31.22 t/s, tg_3s = 30.93 t/s 7.40.734.081 I slot print_timing: id 3 | task 12233 | n_decoded = 474, tg = 31.18 t/s, tg_3s = 31.02 t/s 7.43.736.141 I slot print_timing: id 3 | task 12233 | n_decoded = 567, tg = 31.14 t/s, tg_3s = 30.98 t/s 7.46.757.749 I slot print_timing: id 3 | task 12233 | n_decoded = 661, tg = 31.14 t/s, tg_3s = 31.11 t/s 7.49.779.712 I slot print_timing: id 3 | task 12233 | n_decoded = 753, tg = 31.05 t/s, tg_3s = 30.44 t/s 7.52.782.941 I slot print_timing: id 3 | task 12233 | n_decoded = 844, tg = 30.97 t/s, tg_3s = 30.30 t/s 7.55.787.992 I slot print_timing: id 3 | task 12233 | n_decoded = 935, tg = 30.90 t/s, tg_3s = 30.28 t/s 7.58.792.013 I slot print_timing: id 3 | task 12233 | n_decoded = 1024, tg = 30.79 t/s, tg_3s = 29.63 t/s 8.01.793.534 I slot print_timing: id 3 | task 12233 | n_decoded = 1113, tg = 30.69 t/s, tg_3s = 29.65 t/s 8.04.805.472 I slot print_timing: id 3 | task 12233 | n_decoded = 1202, tg = 30.60 t/s, tg_3s = 29.55 t/s 8.07.821.663 I slot print_timing: id 3 | task 12233 | n_decoded = 1291, tg = 30.53 t/s, tg_3s = 29.51 t/s 8.09.585.847 I slot print_timing: id 3 | task 12233 | prompt eval time = 327.32 ms / 11 tokens ( 29.76 ms per token, 33.61 tokens per second) 8.09.585.852 I slot print_timing: id 3 | task 12233 | eval time = 44055.14 ms / 1343 tokens ( 32.80 ms per token, 30.48 tokens per second) 8.09.585.853 I slot print_timing: id 3 | task 12233 | total time = 44382.46 ms / 1354 tokens 8.09.585.854 I slot print_timing: id 3 | task 12233 | graphs reused = 13505 8.09.585.899 I slot release: id 3 | task 12233 | stop processing: n_tokens = 1426, truncated = 0 8.09.585.916 I srv update_slots: all slots are idle 8.09.597.380 I srv operator(): Chat format: peg-native 8.09.597.612 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.849 (> 0.100 thold), f_keep = 0.051 8.09.597.614 I srv get_availabl: updating prompt cache 8.09.597.686 W srv prompt_save: - saving prompt with length 1426, total state size = 256.263 MiB (draft: 0.000 MiB) 8.09.780.498 I srv load: - looking for better prompt, base f_keep = 0.051, sim = 0.849 8.09.780.506 I srv update: - cache state: 7 prompts, 2549.684 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 8.09.780.507 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 8.09.780.508 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 8.09.780.509 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 8.09.780.513 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 8.09.780.514 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 8.09.780.515 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 8.09.780.516 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 8.09.780.517 I srv get_availabl: prompt cache update took 182.90 ms 8.09.780.570 I slot launch_slot_: id 3 | task 13577 | processing task, is_child = 0 8.09.780.571 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 8.09.780.571 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 8.09.780.586 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 8.09.780.587 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 8.09.780.597 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 8.09.780.597 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 8.13.273.909 I slot print_timing: id 3 | task 13577 | n_decoded = 100, tg = 31.75 t/s, tg_3s = 31.75 t/s 8.16.280.148 I slot print_timing: id 3 | task 13577 | n_decoded = 194, tg = 31.52 t/s, tg_3s = 31.27 t/s 8.19.286.588 I slot print_timing: id 3 | task 13577 | n_decoded = 287, tg = 31.33 t/s, tg_3s = 30.93 t/s 8.22.296.344 I slot print_timing: id 3 | task 13577 | n_decoded = 380, tg = 31.22 t/s, tg_3s = 30.90 t/s 8.25.306.660 I slot print_timing: id 3 | task 13577 | n_decoded = 473, tg = 31.16 t/s, tg_3s = 30.89 t/s 8.28.324.244 I slot print_timing: id 3 | task 13577 | n_decoded = 567, tg = 31.15 t/s, tg_3s = 31.15 t/s 8.31.343.084 I slot print_timing: id 3 | task 13577 | n_decoded = 661, tg = 31.15 t/s, tg_3s = 31.14 t/s 8.34.367.956 I slot print_timing: id 3 | task 13577 | n_decoded = 753, tg = 31.06 t/s, tg_3s = 30.41 t/s 8.37.372.928 I slot print_timing: id 3 | task 13577 | n_decoded = 844, tg = 30.97 t/s, tg_3s = 30.28 t/s 8.40.375.966 I slot print_timing: id 3 | task 13577 | n_decoded = 935, tg = 30.91 t/s, tg_3s = 30.30 t/s 8.43.408.149 I slot print_timing: id 3 | task 13577 | n_decoded = 1025, tg = 30.80 t/s, tg_3s = 29.68 t/s 8.46.437.864 I slot print_timing: id 3 | task 13577 | n_decoded = 1115, tg = 30.71 t/s, tg_3s = 29.71 t/s 8.49.446.640 I slot print_timing: id 3 | task 13577 | n_decoded = 1204, tg = 30.62 t/s, tg_3s = 29.58 t/s 8.52.457.206 I slot print_timing: id 3 | task 13577 | n_decoded = 1293, tg = 30.54 t/s, tg_3s = 29.56 t/s 8.55.474.919 I slot print_timing: id 3 | task 13577 | n_decoded = 1382, tg = 30.47 t/s, tg_3s = 29.49 t/s 8.58.475.212 I slot print_timing: id 3 | task 13577 | n_decoded = 1470, tg = 30.40 t/s, tg_3s = 29.33 t/s 9.01.502.817 I slot print_timing: id 3 | task 13577 | n_decoded = 1559, tg = 30.34 t/s, tg_3s = 29.40 t/s 9.04.531.194 I slot print_timing: id 3 | task 13577 | n_decoded = 1648, tg = 30.29 t/s, tg_3s = 29.39 t/s 9.07.555.848 I slot print_timing: id 3 | task 13577 | n_decoded = 1736, tg = 30.23 t/s, tg_3s = 29.09 t/s 9.10.582.517 I slot print_timing: id 3 | task 13577 | n_decoded = 1823, tg = 30.15 t/s, tg_3s = 28.74 t/s 9.13.615.449 I slot print_timing: id 3 | task 13577 | n_decoded = 1910, tg = 30.08 t/s, tg_3s = 28.69 t/s 9.16.628.808 I slot print_timing: id 3 | task 13577 | n_decoded = 1996, tg = 30.01 t/s, tg_3s = 28.54 t/s 9.19.638.761 I slot print_timing: id 3 | task 13577 | n_decoded = 2082, tg = 29.95 t/s, tg_3s = 28.57 t/s 9.21.143.862 I slot print_timing: id 3 | task 13577 | prompt eval time = 343.98 ms / 13 tokens ( 26.46 ms per token, 37.79 tokens per second) 9.21.143.865 I slot print_timing: id 3 | task 13577 | eval time = 71019.26 ms / 2125 tokens ( 33.42 ms per token, 29.92 tokens per second) 9.21.143.866 I slot print_timing: id 3 | task 13577 | total time = 71363.24 ms / 2138 tokens 9.21.143.866 I slot print_timing: id 3 | task 13577 | graphs reused = 15620 9.21.143.901 I slot release: id 3 | task 13577 | stop processing: n_tokens = 2210, truncated = 0 9.21.143.917 I srv update_slots: all slots are idle 9.21.155.258 I srv operator(): Chat format: peg-native 9.21.155.497 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.637 (> 0.100 thold), f_keep = 0.033 9.21.155.499 I srv get_availabl: updating prompt cache 9.21.155.593 W srv prompt_save: - saving prompt with length 2210, total state size = 397.153 MiB (draft: 0.000 MiB) 9.21.430.705 I srv load: - looking for better prompt, base f_keep = 0.033, sim = 0.637 9.21.430.716 I srv update: - cache state: 8 prompts, 2946.837 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 9.21.430.717 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 9.21.430.718 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 9.21.430.719 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 9.21.430.720 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 9.21.430.720 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 9.21.430.721 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 9.21.430.722 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 9.21.430.723 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 9.21.430.729 I srv get_availabl: prompt cache update took 275.23 ms 9.21.430.785 I slot launch_slot_: id 3 | task 15703 | processing task, is_child = 0 9.21.430.785 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 9.21.430.786 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 9.21.430.805 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 9.21.430.805 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 9.21.430.821 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 9.21.430.822 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 9.25.522.851 I slot print_timing: id 3 | task 15703 | n_decoded = 100, tg = 31.56 t/s, tg_3s = 31.56 t/s 9.28.553.125 I slot print_timing: id 3 | task 15703 | n_decoded = 195, tg = 31.46 t/s, tg_3s = 31.35 t/s 9.31.567.139 I slot print_timing: id 3 | task 15703 | n_decoded = 289, tg = 31.37 t/s, tg_3s = 31.19 t/s 9.34.590.185 I slot print_timing: id 3 | task 15703 | n_decoded = 383, tg = 31.30 t/s, tg_3s = 31.09 t/s 9.37.619.130 I slot print_timing: id 3 | task 15703 | n_decoded = 477, tg = 31.25 t/s, tg_3s = 31.03 t/s 9.40.642.149 I slot print_timing: id 3 | task 15703 | n_decoded = 571, tg = 31.22 t/s, tg_3s = 31.09 t/s 9.43.649.098 I slot print_timing: id 3 | task 15703 | n_decoded = 664, tg = 31.18 t/s, tg_3s = 30.93 t/s 9.46.652.132 I slot print_timing: id 3 | task 15703 | n_decoded = 755, tg = 31.07 t/s, tg_3s = 30.30 t/s 9.49.659.177 I slot print_timing: id 3 | task 15703 | n_decoded = 846, tg = 30.98 t/s, tg_3s = 30.26 t/s 9.52.660.788 I slot print_timing: id 3 | task 15703 | n_decoded = 936, tg = 30.88 t/s, tg_3s = 29.98 t/s 9.55.686.806 I slot print_timing: id 3 | task 15703 | n_decoded = 1025, tg = 30.75 t/s, tg_3s = 29.41 t/s 9.58.712.851 I slot print_timing: id 3 | task 15703 | n_decoded = 1114, tg = 30.64 t/s, tg_3s = 29.41 t/s 10.01.713.046 I slot print_timing: id 3 | task 15703 | n_decoded = 1202, tg = 30.54 t/s, tg_3s = 29.33 t/s 10.04.739.041 I slot print_timing: id 3 | task 15703 | n_decoded = 1291, tg = 30.46 t/s, tg_3s = 29.41 t/s 10.05.351.120 I slot print_timing: id 3 | task 15703 | prompt eval time = 923.36 ms / 41 tokens ( 22.52 ms per token, 44.40 tokens per second) 10.05.351.124 I slot print_timing: id 3 | task 15703 | eval time = 42996.92 ms / 1309 tokens ( 32.85 ms per token, 30.44 tokens per second) 10.05.351.125 I slot print_timing: id 3 | task 15703 | total time = 43920.28 ms / 1350 tokens 10.05.351.126 I slot print_timing: id 3 | task 15703 | graphs reused = 16922 10.05.351.168 I slot release: id 3 | task 15703 | stop processing: n_tokens = 1421, truncated = 0 10.05.351.182 I srv update_slots: all slots are idle 10.05.362.889 I srv operator(): Chat format: peg-native 10.05.363.126 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.557 (> 0.100 thold), f_keep = 0.051 10.05.363.129 I srv get_availabl: updating prompt cache 10.05.363.208 W srv prompt_save: - saving prompt with length 1421, total state size = 255.364 MiB (draft: 0.000 MiB) 10.05.537.798 I srv load: - looking for better prompt, base f_keep = 0.051, sim = 0.557 10.05.537.807 I srv update: - cache state: 9 prompts, 3202.201 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 10.05.537.808 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 10.05.537.809 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 10.05.537.817 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 10.05.537.818 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 10.05.537.818 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 10.05.537.819 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 10.05.537.819 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 10.05.537.820 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 10.05.537.821 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 10.05.537.822 I srv get_availabl: prompt cache update took 174.69 ms 10.05.537.869 I slot launch_slot_: id 3 | task 17013 | processing task, is_child = 0 10.05.537.870 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 10.05.537.871 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 10.05.537.884 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 10.05.537.884 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 10.05.537.893 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 10.05.537.893 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 10.09.899.944 I slot print_timing: id 3 | task 17013 | n_decoded = 100, tg = 31.49 t/s, tg_3s = 31.49 t/s 10.12.925.149 I slot print_timing: id 3 | task 17013 | n_decoded = 194, tg = 31.29 t/s, tg_3s = 31.07 t/s 10.15.933.186 I slot print_timing: id 3 | task 17013 | n_decoded = 288, tg = 31.27 t/s, tg_3s = 31.25 t/s 10.18.943.212 I slot print_timing: id 3 | task 17013 | n_decoded = 382, tg = 31.26 t/s, tg_3s = 31.23 t/s 10.21.961.271 I slot print_timing: id 3 | task 17013 | n_decoded = 476, tg = 31.24 t/s, tg_3s = 31.15 t/s 10.24.969.269 I slot print_timing: id 3 | task 17013 | n_decoded = 570, tg = 31.24 t/s, tg_3s = 31.25 t/s 10.27.980.877 I slot print_timing: id 3 | task 17013 | n_decoded = 663, tg = 31.19 t/s, tg_3s = 30.88 t/s 10.30.994.906 I slot print_timing: id 3 | task 17013 | n_decoded = 754, tg = 31.07 t/s, tg_3s = 30.19 t/s 10.34.010.965 I slot print_timing: id 3 | task 17013 | n_decoded = 845, tg = 30.97 t/s, tg_3s = 30.17 t/s 10.37.030.831 I slot print_timing: id 3 | task 17013 | n_decoded = 935, tg = 30.85 t/s, tg_3s = 29.80 t/s 10.40.042.759 I slot print_timing: id 3 | task 17013 | n_decoded = 1024, tg = 30.73 t/s, tg_3s = 29.55 t/s 10.43.057.493 I slot print_timing: id 3 | task 17013 | n_decoded = 1113, tg = 30.63 t/s, tg_3s = 29.52 t/s 10.46.090.975 I slot print_timing: id 3 | task 17013 | n_decoded = 1202, tg = 30.53 t/s, tg_3s = 29.34 t/s 10.48.915.859 I slot print_timing: id 3 | task 17013 | prompt eval time = 1186.53 ms / 58 tokens ( 20.46 ms per token, 48.88 tokens per second) 10.48.915.863 I slot print_timing: id 3 | task 17013 | eval time = 42191.42 ms / 1285 tokens ( 32.83 ms per token, 30.46 tokens per second) 10.48.915.864 I slot print_timing: id 3 | task 17013 | total time = 43377.94 ms / 1343 tokens 10.48.915.865 I slot print_timing: id 3 | task 17013 | graphs reused = 18200 10.48.915.915 I slot release: id 3 | task 17013 | stop processing: n_tokens = 1415, truncated = 0 10.48.915.930 I srv update_slots: all slots are idle 10.48.927.207 I srv operator(): Chat format: peg-native 10.48.927.451 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.673 (> 0.100 thold), f_keep = 0.051 10.48.927.453 I srv get_availabl: updating prompt cache 10.48.927.525 W srv prompt_save: - saving prompt with length 1415, total state size = 254.286 MiB (draft: 0.000 MiB) 10.49.102.001 I srv load: - looking for better prompt, base f_keep = 0.051, sim = 0.673 10.49.102.012 I srv update: - cache state: 10 prompts, 3456.487 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 10.49.102.013 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 10.49.102.014 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 10.49.102.015 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 10.49.102.016 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 10.49.102.016 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 10.49.102.017 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 10.49.102.022 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 10.49.102.023 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 10.49.102.024 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 10.49.102.024 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 10.49.102.026 I srv get_availabl: prompt cache update took 174.57 ms 10.49.102.082 I slot launch_slot_: id 3 | task 18299 | processing task, is_child = 0 10.49.102.083 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 10.49.102.083 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 10.49.102.096 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 10.49.102.096 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 10.49.102.107 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 10.49.102.107 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 10.53.035.354 I slot print_timing: id 3 | task 18299 | n_decoded = 100, tg = 31.67 t/s, tg_3s = 31.67 t/s 10.56.056.363 I slot print_timing: id 3 | task 18299 | n_decoded = 194, tg = 31.40 t/s, tg_3s = 31.12 t/s 10.59.078.301 I slot print_timing: id 3 | task 18299 | n_decoded = 288, tg = 31.30 t/s, tg_3s = 31.11 t/s 11.02.092.464 I slot print_timing: id 3 | task 18299 | n_decoded = 382, tg = 31.27 t/s, tg_3s = 31.19 t/s 11.05.119.429 I slot print_timing: id 3 | task 18299 | n_decoded = 476, tg = 31.23 t/s, tg_3s = 31.05 t/s 11.08.131.397 I slot print_timing: id 3 | task 18299 | n_decoded = 570, tg = 31.23 t/s, tg_3s = 31.21 t/s 11.11.152.554 I slot print_timing: id 3 | task 18299 | n_decoded = 664, tg = 31.21 t/s, tg_3s = 31.11 t/s 11.11.218.902 I slot print_timing: id 3 | task 18299 | prompt eval time = 775.55 ms / 35 tokens ( 22.16 ms per token, 45.13 tokens per second) 11.11.218.905 I slot print_timing: id 3 | task 18299 | eval time = 21341.22 ms / 666 tokens ( 32.04 ms per token, 31.21 tokens per second) 11.11.218.906 I slot print_timing: id 3 | task 18299 | total time = 22116.77 ms / 701 tokens 11.11.218.906 I slot print_timing: id 3 | task 18299 | graphs reused = 18861 11.11.218.943 I slot release: id 3 | task 18299 | stop processing: n_tokens = 772, truncated = 0 11.11.218.953 I srv update_slots: all slots are idle 11.11.230.269 I srv operator(): Chat format: peg-native 11.11.230.515 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.590 (> 0.100 thold), f_keep = 0.093 11.11.230.516 I srv get_availabl: updating prompt cache 11.11.230.567 W srv prompt_save: - saving prompt with length 772, total state size = 138.735 MiB (draft: 0.000 MiB) 11.11.329.361 I srv load: - looking for better prompt, base f_keep = 0.093, sim = 0.590 11.11.329.370 I srv update: - cache state: 11 prompts, 3595.222 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 11.11.329.371 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 11.11.329.372 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 11.11.329.373 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 11.11.329.373 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 11.11.329.374 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 11.11.329.374 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 11.11.329.375 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 11.11.329.375 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 11.11.329.376 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 11.11.329.377 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 11.11.329.377 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 11.11.329.378 I srv get_availabl: prompt cache update took 98.86 ms 11.11.329.426 I slot launch_slot_: id 3 | task 18966 | processing task, is_child = 0 11.11.329.426 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 11.11.329.427 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 11.11.329.438 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 11.11.329.439 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 11.11.329.447 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 11.11.329.448 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 11.15.540.736 I slot print_timing: id 3 | task 18966 | n_decoded = 100, tg = 31.85 t/s, tg_3s = 31.85 t/s 11.18.558.255 I slot print_timing: id 3 | task 18966 | n_decoded = 194, tg = 31.51 t/s, tg_3s = 31.15 t/s 11.21.570.305 I slot print_timing: id 3 | task 18966 | n_decoded = 288, tg = 31.41 t/s, tg_3s = 31.21 t/s 11.24.586.337 I slot print_timing: id 3 | task 18966 | n_decoded = 382, tg = 31.35 t/s, tg_3s = 31.17 t/s 11.27.610.425 I slot print_timing: id 3 | task 18966 | n_decoded = 476, tg = 31.30 t/s, tg_3s = 31.08 t/s 11.30.623.441 I slot print_timing: id 3 | task 18966 | n_decoded = 570, tg = 31.28 t/s, tg_3s = 31.20 t/s 11.33.641.862 I slot print_timing: id 3 | task 18966 | n_decoded = 663, tg = 31.21 t/s, tg_3s = 30.81 t/s 11.36.645.887 I slot print_timing: id 3 | task 18966 | n_decoded = 753, tg = 31.06 t/s, tg_3s = 29.96 t/s 11.39.675.506 I slot print_timing: id 3 | task 18966 | n_decoded = 844, tg = 30.95 t/s, tg_3s = 30.04 t/s 11.42.702.832 I slot print_timing: id 3 | task 18966 | n_decoded = 934, tg = 30.82 t/s, tg_3s = 29.73 t/s 11.42.974.873 I slot print_timing: id 3 | task 18966 | prompt eval time = 1071.98 ms / 50 tokens ( 21.44 ms per token, 46.64 tokens per second) 11.42.974.876 I slot print_timing: id 3 | task 18966 | eval time = 30573.43 ms / 942 tokens ( 32.46 ms per token, 30.81 tokens per second) 11.42.974.877 I slot print_timing: id 3 | task 18966 | total time = 31645.41 ms / 992 tokens 11.42.974.877 I slot print_timing: id 3 | task 18966 | graphs reused = 19797 11.42.974.911 I slot release: id 3 | task 18966 | stop processing: n_tokens = 1063, truncated = 0 11.42.974.921 I srv update_slots: all slots are idle 11.42.986.445 I srv operator(): Chat format: peg-native 11.42.986.677 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.581 (> 0.100 thold), f_keep = 0.068 11.42.986.678 I srv get_availabl: updating prompt cache 11.42.986.732 W srv prompt_save: - saving prompt with length 1063, total state size = 191.029 MiB (draft: 0.000 MiB) 11.43.120.111 I srv load: - looking for better prompt, base f_keep = 0.068, sim = 0.581 11.43.120.121 I srv update: - cache state: 12 prompts, 3786.251 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 11.43.120.122 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 11.43.120.122 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 11.43.120.123 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 11.43.120.123 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 11.43.120.124 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 11.43.120.124 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 11.43.120.125 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 11.43.120.126 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 11.43.120.126 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 11.43.120.127 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 11.43.120.128 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 11.43.120.128 I srv update: - prompt 0x6437c16c1840: 1063 tokens, checkpoints: 0, 191.029 MiB 11.43.120.129 I srv get_availabl: prompt cache update took 133.45 ms 11.43.120.176 I slot launch_slot_: id 3 | task 19909 | processing task, is_child = 0 11.43.120.177 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 11.43.120.177 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 11.43.120.188 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 11.43.120.188 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 11.43.120.197 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 11.43.120.197 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 11.47.370.894 I slot print_timing: id 3 | task 19909 | n_decoded = 100, tg = 31.50 t/s, tg_3s = 31.50 t/s 11.50.391.332 I slot print_timing: id 3 | task 19909 | n_decoded = 194, tg = 31.32 t/s, tg_3s = 31.12 t/s 11.53.400.218 I slot print_timing: id 3 | task 19909 | n_decoded = 288, tg = 31.29 t/s, tg_3s = 31.24 t/s 11.56.408.236 I slot print_timing: id 3 | task 19909 | n_decoded = 382, tg = 31.28 t/s, tg_3s = 31.25 t/s 11.59.426.267 I slot print_timing: id 3 | task 19909 | n_decoded = 476, tg = 31.25 t/s, tg_3s = 31.15 t/s 12.02.435.272 I slot print_timing: id 3 | task 19909 | n_decoded = 570, tg = 31.25 t/s, tg_3s = 31.24 t/s 12.05.441.826 I slot print_timing: id 3 | task 19909 | n_decoded = 663, tg = 31.21 t/s, tg_3s = 30.93 t/s 12.08.457.733 I slot print_timing: id 3 | task 19909 | n_decoded = 754, tg = 31.08 t/s, tg_3s = 30.17 t/s 12.10.744.390 I slot print_timing: id 3 | task 19909 | prompt eval time = 1076.35 ms / 52 tokens ( 20.70 ms per token, 48.31 tokens per second) 12.10.744.393 I slot print_timing: id 3 | task 19909 | eval time = 26547.82 ms / 823 tokens ( 32.26 ms per token, 31.00 tokens per second) 12.10.744.394 I slot print_timing: id 3 | task 19909 | total time = 27624.17 ms / 875 tokens 12.10.744.395 I slot print_timing: id 3 | task 19909 | graphs reused = 20615 12.10.744.428 I slot release: id 3 | task 19909 | stop processing: n_tokens = 946, truncated = 0 12.10.744.438 I srv update_slots: all slots are idle 12.10.755.499 I srv operator(): Chat format: peg-native 12.10.755.737 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.658 (> 0.100 thold), f_keep = 0.077 12.10.755.739 I srv get_availabl: updating prompt cache 12.10.755.789 W srv prompt_save: - saving prompt with length 946, total state size = 170.003 MiB (draft: 0.000 MiB) 12.10.875.255 I srv load: - looking for better prompt, base f_keep = 0.077, sim = 0.658 12.10.875.265 I srv update: - cache state: 13 prompts, 3956.254 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 12.10.875.266 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 12.10.875.266 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 12.10.875.267 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 12.10.875.268 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 12.10.875.268 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 12.10.875.269 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 12.10.875.269 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 12.10.875.270 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 12.10.875.270 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 12.10.875.271 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 12.10.875.271 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 12.10.875.272 I srv update: - prompt 0x6437c16c1840: 1063 tokens, checkpoints: 0, 191.029 MiB 12.10.875.272 I srv update: - prompt 0x6437c1492470: 946 tokens, checkpoints: 0, 170.003 MiB 12.10.875.273 I srv get_availabl: prompt cache update took 119.53 ms 12.10.875.329 I slot launch_slot_: id 3 | task 20733 | processing task, is_child = 0 12.10.875.331 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 12.10.875.331 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 12.10.875.342 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 12.10.875.342 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 12.10.875.351 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 12.10.875.351 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 12.14.813.072 I slot print_timing: id 3 | task 20733 | n_decoded = 100, tg = 31.68 t/s, tg_3s = 31.68 t/s 12.17.828.439 I slot print_timing: id 3 | task 20733 | n_decoded = 194, tg = 31.43 t/s, tg_3s = 31.17 t/s 12.20.848.288 I slot print_timing: id 3 | task 20733 | n_decoded = 288, tg = 31.33 t/s, tg_3s = 31.13 t/s 12.23.875.435 I slot print_timing: id 3 | task 20733 | n_decoded = 382, tg = 31.26 t/s, tg_3s = 31.05 t/s 12.26.884.733 I slot print_timing: id 3 | task 20733 | n_decoded = 475, tg = 31.19 t/s, tg_3s = 30.90 t/s 12.29.896.362 I slot print_timing: id 3 | task 20733 | n_decoded = 568, tg = 31.14 t/s, tg_3s = 30.88 t/s 12.30.446.236 I slot print_timing: id 3 | task 20733 | prompt eval time = 781.33 ms / 38 tokens ( 20.56 ms per token, 48.64 tokens per second) 12.30.446.239 I slot print_timing: id 3 | task 20733 | eval time = 18789.54 ms / 585 tokens ( 32.12 ms per token, 31.13 tokens per second) 12.30.446.240 I slot print_timing: id 3 | task 20733 | total time = 19570.86 ms / 623 tokens 12.30.446.241 I slot print_timing: id 3 | task 20733 | graphs reused = 21196 12.30.446.276 I slot release: id 3 | task 20733 | stop processing: n_tokens = 695, truncated = 0 12.30.446.285 I srv update_slots: all slots are idle 12.30.458.083 I srv operator(): Chat format: peg-native 12.30.458.318 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.615 (> 0.100 thold), f_keep = 0.104 12.30.458.329 I srv get_availabl: updating prompt cache 12.30.458.376 W srv prompt_save: - saving prompt with length 695, total state size = 124.897 MiB (draft: 0.000 MiB) 12.30.549.362 I srv load: - looking for better prompt, base f_keep = 0.104, sim = 0.615 12.30.549.373 I srv update: - cache state: 14 prompts, 4081.151 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 12.30.549.374 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 12.30.549.374 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 12.30.549.375 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 12.30.549.375 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 12.30.549.376 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 12.30.549.376 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 12.30.549.377 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 12.30.549.377 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 12.30.549.378 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 12.30.549.379 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 12.30.549.379 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 12.30.549.380 I srv update: - prompt 0x6437c16c1840: 1063 tokens, checkpoints: 0, 191.029 MiB 12.30.549.380 I srv update: - prompt 0x6437c1492470: 946 tokens, checkpoints: 0, 170.003 MiB 12.30.549.381 I srv update: - prompt 0x6437c1416d50: 695 tokens, checkpoints: 0, 124.897 MiB 12.30.549.382 I srv get_availabl: prompt cache update took 91.05 ms 12.30.549.429 I slot launch_slot_: id 3 | task 21319 | processing task, is_child = 0 12.30.549.430 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 12.30.549.431 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 12.30.549.441 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 12.30.549.441 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 12.30.549.450 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 12.30.549.450 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 12.34.669.765 I slot print_timing: id 3 | task 21319 | n_decoded = 100, tg = 31.53 t/s, tg_3s = 31.53 t/s 12.37.672.449 I slot print_timing: id 3 | task 21319 | n_decoded = 193, tg = 31.26 t/s, tg_3s = 30.97 t/s 12.40.687.377 I slot print_timing: id 3 | task 21319 | n_decoded = 287, tg = 31.23 t/s, tg_3s = 31.18 t/s 12.43.699.358 I slot print_timing: id 3 | task 21319 | n_decoded = 381, tg = 31.23 t/s, tg_3s = 31.21 t/s 12.46.722.328 I slot print_timing: id 3 | task 21319 | n_decoded = 475, tg = 31.20 t/s, tg_3s = 31.10 t/s 12.49.732.385 I slot print_timing: id 3 | task 21319 | n_decoded = 569, tg = 31.21 t/s, tg_3s = 31.23 t/s 12.52.762.465 I slot print_timing: id 3 | task 21319 | n_decoded = 663, tg = 31.18 t/s, tg_3s = 31.02 t/s 12.55.781.382 I slot print_timing: id 3 | task 21319 | n_decoded = 754, tg = 31.05 t/s, tg_3s = 30.14 t/s 12.58.803.469 I slot print_timing: id 3 | task 21319 | n_decoded = 845, tg = 30.95 t/s, tg_3s = 30.11 t/s 13.01.807.640 I slot print_timing: id 3 | task 21319 | n_decoded = 935, tg = 30.85 t/s, tg_3s = 29.96 t/s 13.04.839.729 I slot print_timing: id 3 | task 21319 | n_decoded = 1025, tg = 30.74 t/s, tg_3s = 29.68 t/s 13.07.841.791 I slot print_timing: id 3 | task 21319 | n_decoded = 1114, tg = 30.65 t/s, tg_3s = 29.65 t/s 13.10.867.792 I slot print_timing: id 3 | task 21319 | n_decoded = 1203, tg = 30.56 t/s, tg_3s = 29.41 t/s 13.13.893.740 I slot print_timing: id 3 | task 21319 | n_decoded = 1292, tg = 30.48 t/s, tg_3s = 29.41 t/s 13.14.029.851 I slot print_timing: id 3 | task 21319 | prompt eval time = 949.15 ms / 45 tokens ( 21.09 ms per token, 47.41 tokens per second) 13.14.029.854 I slot print_timing: id 3 | task 21319 | eval time = 42531.23 ms / 1296 tokens ( 32.82 ms per token, 30.47 tokens per second) 13.14.029.855 I slot print_timing: id 3 | task 21319 | total time = 43480.38 ms / 1341 tokens 13.14.029.855 I slot print_timing: id 3 | task 21319 | graphs reused = 22485 13.14.029.894 I slot release: id 3 | task 21319 | stop processing: n_tokens = 1412, truncated = 0 13.14.029.907 I srv update_slots: all slots are idle 13.14.041.529 I srv operator(): Chat format: peg-native 13.14.041.762 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.679 (> 0.100 thold), f_keep = 0.051 13.14.041.764 I srv get_availabl: updating prompt cache 13.14.041.829 W srv prompt_save: - saving prompt with length 1412, total state size = 253.747 MiB (draft: 0.000 MiB) 13.14.213.873 I srv load: - looking for better prompt, base f_keep = 0.051, sim = 0.679 13.14.213.884 I srv update: - cache state: 15 prompts, 4334.898 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 13.14.213.885 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 13.14.213.886 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 13.14.213.886 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 13.14.213.887 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 13.14.213.887 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 13.14.213.888 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 13.14.213.888 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 13.14.213.889 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 13.14.213.889 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 13.14.213.890 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 13.14.213.890 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 13.14.213.891 I srv update: - prompt 0x6437c16c1840: 1063 tokens, checkpoints: 0, 191.029 MiB 13.14.213.891 I srv update: - prompt 0x6437c1492470: 946 tokens, checkpoints: 0, 170.003 MiB 13.14.213.892 I srv update: - prompt 0x6437c1416d50: 695 tokens, checkpoints: 0, 124.897 MiB 13.14.213.892 I srv update: - prompt 0x6437c509b7a0: 1412 tokens, checkpoints: 0, 253.747 MiB 13.14.213.894 I srv get_availabl: prompt cache update took 172.13 ms 13.14.213.941 I slot launch_slot_: id 3 | task 22616 | processing task, is_child = 0 13.14.213.942 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 13.14.213.943 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 13.14.213.954 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 13.14.213.954 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 13.14.213.963 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 13.14.213.963 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 13.18.136.677 I slot print_timing: id 3 | task 22616 | n_decoded = 100, tg = 31.60 t/s, tg_3s = 31.60 t/s 13.21.158.436 I slot print_timing: id 3 | task 22616 | n_decoded = 194, tg = 31.36 t/s, tg_3s = 31.11 t/s 13.24.182.211 I slot print_timing: id 3 | task 22616 | n_decoded = 288, tg = 31.27 t/s, tg_3s = 31.09 t/s 13.27.196.214 I slot print_timing: id 3 | task 22616 | n_decoded = 382, tg = 31.25 t/s, tg_3s = 31.19 t/s 13.30.216.257 I slot print_timing: id 3 | task 22616 | n_decoded = 476, tg = 31.23 t/s, tg_3s = 31.13 t/s 13.33.226.451 I slot print_timing: id 3 | task 22616 | n_decoded = 570, tg = 31.23 t/s, tg_3s = 31.23 t/s 13.36.245.266 I slot print_timing: id 3 | task 22616 | n_decoded = 664, tg = 31.21 t/s, tg_3s = 31.14 t/s 13.39.259.526 I slot print_timing: id 3 | task 22616 | n_decoded = 755, tg = 31.09 t/s, tg_3s = 30.19 t/s 13.42.265.777 I slot print_timing: id 3 | task 22616 | n_decoded = 846, tg = 31.00 t/s, tg_3s = 30.27 t/s 13.45.290.039 I slot print_timing: id 3 | task 22616 | n_decoded = 937, tg = 30.91 t/s, tg_3s = 30.09 t/s 13.48.321.819 I slot print_timing: id 3 | task 22616 | n_decoded = 1027, tg = 30.80 t/s, tg_3s = 29.69 t/s 13.51.353.700 I slot print_timing: id 3 | task 22616 | n_decoded = 1117, tg = 30.70 t/s, tg_3s = 29.68 t/s 13.54.371.747 I slot print_timing: id 3 | task 22616 | n_decoded = 1206, tg = 30.61 t/s, tg_3s = 29.49 t/s 13.57.395.865 I slot print_timing: id 3 | task 22616 | n_decoded = 1295, tg = 30.53 t/s, tg_3s = 29.43 t/s 14.00.421.943 I slot print_timing: id 3 | task 22616 | n_decoded = 1384, tg = 30.45 t/s, tg_3s = 29.41 t/s 14.03.424.145 I slot print_timing: id 3 | task 22616 | n_decoded = 1472, tg = 30.38 t/s, tg_3s = 29.31 t/s 14.06.454.102 I slot print_timing: id 3 | task 22616 | n_decoded = 1561, tg = 30.32 t/s, tg_3s = 29.37 t/s 14.07.644.913 I slot print_timing: id 3 | task 22616 | prompt eval time = 758.41 ms / 34 tokens ( 22.31 ms per token, 44.83 tokens per second) 14.07.644.916 I slot print_timing: id 3 | task 22616 | eval time = 52672.51 ms / 1596 tokens ( 33.00 ms per token, 30.30 tokens per second) 14.07.644.917 I slot print_timing: id 3 | task 22616 | total time = 53430.93 ms / 1630 tokens 14.07.644.918 I slot print_timing: id 3 | task 22616 | graphs reused = 24073 14.07.644.956 I slot release: id 3 | task 22616 | stop processing: n_tokens = 1701, truncated = 0 14.07.644.968 I srv update_slots: all slots are idle 14.07.656.171 I srv operator(): Chat format: peg-native 14.07.656.412 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.042 14.07.656.413 I srv get_availabl: updating prompt cache 14.07.656.496 W srv prompt_save: - saving prompt with length 1701, total state size = 305.682 MiB (draft: 0.000 MiB) 14.07.867.337 I srv load: - looking for better prompt, base f_keep = 0.042, sim = 0.667 14.07.867.349 I srv update: - cache state: 16 prompts, 4640.580 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 14.07.867.350 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 14.07.867.351 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 14.07.867.351 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 14.07.867.352 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 14.07.867.352 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 14.07.867.353 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 14.07.867.353 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 14.07.867.354 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 14.07.867.354 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 14.07.867.355 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 14.07.867.355 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 14.07.867.356 I srv update: - prompt 0x6437c16c1840: 1063 tokens, checkpoints: 0, 191.029 MiB 14.07.867.356 I srv update: - prompt 0x6437c1492470: 946 tokens, checkpoints: 0, 170.003 MiB 14.07.867.357 I srv update: - prompt 0x6437c1416d50: 695 tokens, checkpoints: 0, 124.897 MiB 14.07.867.357 I srv update: - prompt 0x6437c509b7a0: 1412 tokens, checkpoints: 0, 253.747 MiB 14.07.867.358 I srv update: - prompt 0x6437c159f8a0: 1701 tokens, checkpoints: 0, 305.682 MiB 14.07.867.359 I srv get_availabl: prompt cache update took 210.94 ms 14.07.867.409 I slot launch_slot_: id 3 | task 24213 | processing task, is_child = 0 14.07.867.410 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 14.07.867.411 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 14.07.867.425 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 14.07.867.426 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 14.07.867.435 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 14.07.867.435 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 14.11.810.146 I slot print_timing: id 3 | task 24213 | n_decoded = 100, tg = 31.59 t/s, tg_3s = 31.59 t/s 14.14.836.045 I slot print_timing: id 3 | task 24213 | n_decoded = 194, tg = 31.33 t/s, tg_3s = 31.07 t/s 14.17.843.704 I slot print_timing: id 3 | task 24213 | n_decoded = 287, tg = 31.20 t/s, tg_3s = 30.92 t/s 14.20.854.121 I slot print_timing: id 3 | task 24213 | n_decoded = 380, tg = 31.12 t/s, tg_3s = 30.89 t/s 14.23.883.780 I slot print_timing: id 3 | task 24213 | n_decoded = 473, tg = 31.04 t/s, tg_3s = 30.70 t/s 14.26.909.425 I slot print_timing: id 3 | task 24213 | n_decoded = 567, tg = 31.04 t/s, tg_3s = 31.07 t/s 14.29.917.385 I slot print_timing: id 3 | task 24213 | n_decoded = 661, tg = 31.07 t/s, tg_3s = 31.25 t/s 14.32.929.960 I slot print_timing: id 3 | task 24213 | n_decoded = 752, tg = 30.97 t/s, tg_3s = 30.21 t/s 14.35.934.480 I slot print_timing: id 3 | task 24213 | n_decoded = 843, tg = 30.89 t/s, tg_3s = 30.29 t/s 14.38.958.974 I slot print_timing: id 3 | task 24213 | n_decoded = 934, tg = 30.81 t/s, tg_3s = 30.09 t/s 14.41.991.859 I slot print_timing: id 3 | task 24213 | n_decoded = 1024, tg = 30.71 t/s, tg_3s = 29.67 t/s 14.44.992.822 I slot print_timing: id 3 | task 24213 | n_decoded = 1113, tg = 30.62 t/s, tg_3s = 29.66 t/s 14.48.008.108 I slot print_timing: id 3 | task 24213 | n_decoded = 1202, tg = 30.54 t/s, tg_3s = 29.52 t/s 14.51.034.885 I slot print_timing: id 3 | task 24213 | n_decoded = 1291, tg = 30.46 t/s, tg_3s = 29.40 t/s 14.54.053.839 I slot print_timing: id 3 | task 24213 | n_decoded = 1380, tg = 30.39 t/s, tg_3s = 29.48 t/s 14.57.084.326 I slot print_timing: id 3 | task 24213 | n_decoded = 1469, tg = 30.33 t/s, tg_3s = 29.37 t/s 15.00.110.674 I slot print_timing: id 3 | task 24213 | n_decoded = 1558, tg = 30.27 t/s, tg_3s = 29.41 t/s 15.01.810.772 I slot print_timing: id 3 | task 24213 | prompt eval time = 777.21 ms / 36 tokens ( 21.59 ms per token, 46.32 tokens per second) 15.01.810.776 I slot print_timing: id 3 | task 24213 | eval time = 53166.11 ms / 1608 tokens ( 33.06 ms per token, 30.24 tokens per second) 15.01.810.776 I slot print_timing: id 3 | task 24213 | total time = 53943.32 ms / 1644 tokens 15.01.810.777 I slot print_timing: id 3 | task 24213 | graphs reused = 25673 15.01.810.809 I slot release: id 3 | task 24213 | stop processing: n_tokens = 1715, truncated = 0 15.01.810.823 I srv update_slots: all slots are idle 15.01.822.145 I srv operator(): Chat format: peg-native 15.01.822.385 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.735 (> 0.100 thold), f_keep = 0.042 15.01.822.387 I srv get_availabl: updating prompt cache 15.01.822.460 W srv prompt_save: - saving prompt with length 1715, total state size = 308.198 MiB (draft: 0.000 MiB) 15.02.031.788 I srv load: - looking for better prompt, base f_keep = 0.042, sim = 0.735 15.02.031.802 I srv update: - cache state: 17 prompts, 4948.778 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 15.02.031.803 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 15.02.031.803 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 15.02.031.804 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 15.02.031.804 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 15.02.031.805 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 15.02.031.805 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 15.02.031.806 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 15.02.031.806 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 15.02.031.807 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 15.02.031.807 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 15.02.031.808 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 15.02.031.808 I srv update: - prompt 0x6437c16c1840: 1063 tokens, checkpoints: 0, 191.029 MiB 15.02.031.809 I srv update: - prompt 0x6437c1492470: 946 tokens, checkpoints: 0, 170.003 MiB 15.02.031.810 I srv update: - prompt 0x6437c1416d50: 695 tokens, checkpoints: 0, 124.897 MiB 15.02.031.810 I srv update: - prompt 0x6437c509b7a0: 1412 tokens, checkpoints: 0, 253.747 MiB 15.02.031.811 I srv update: - prompt 0x6437c159f8a0: 1701 tokens, checkpoints: 0, 305.682 MiB 15.02.031.811 I srv update: - prompt 0x6437c147ecd0: 1715 tokens, checkpoints: 0, 308.198 MiB 15.02.031.812 I srv get_availabl: prompt cache update took 209.43 ms 15.02.031.859 I slot launch_slot_: id 3 | task 25822 | processing task, is_child = 0 15.02.031.859 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 15.02.031.860 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 15.02.031.874 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 15.02.031.874 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 15.02.031.883 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 15.02.031.883 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 15.05.757.800 I slot print_timing: id 3 | task 25822 | n_decoded = 100, tg = 31.72 t/s, tg_3s = 31.72 t/s 15.08.769.203 I slot print_timing: id 3 | task 25822 | n_decoded = 194, tg = 31.47 t/s, tg_3s = 31.21 t/s 15.11.776.707 I slot print_timing: id 3 | task 25822 | n_decoded = 287, tg = 31.29 t/s, tg_3s = 30.92 t/s 15.14.786.737 I slot print_timing: id 3 | task 25822 | n_decoded = 381, tg = 31.28 t/s, tg_3s = 31.23 t/s 15.16.172.781 I slot print_timing: id 3 | task 25822 | prompt eval time = 573.18 ms / 26 tokens ( 22.05 ms per token, 45.36 tokens per second) 15.16.172.785 I slot print_timing: id 3 | task 25822 | eval time = 13567.69 ms / 424 tokens ( 32.00 ms per token, 31.25 tokens per second) 15.16.172.785 I slot print_timing: id 3 | task 25822 | total time = 14140.88 ms / 450 tokens 15.16.172.786 I slot print_timing: id 3 | task 25822 | graphs reused = 26093 15.16.172.814 I slot release: id 3 | task 25822 | stop processing: n_tokens = 521, truncated = 0 15.16.172.822 I srv update_slots: all slots are idle 15.16.184.071 I srv operator(): Chat format: peg-native 15.16.184.306 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.800 (> 0.100 thold), f_keep = 0.138 15.16.184.308 I srv get_availabl: updating prompt cache 15.16.184.352 W srv prompt_save: - saving prompt with length 521, total state size = 93.628 MiB (draft: 0.000 MiB) 15.16.254.096 I srv load: - looking for better prompt, base f_keep = 0.138, sim = 0.800 15.16.254.106 I srv update: - cache state: 18 prompts, 5042.406 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 15.16.254.107 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 15.16.254.108 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 15.16.254.108 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 15.16.254.109 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 15.16.254.109 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 15.16.254.110 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 15.16.254.110 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 15.16.254.111 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 15.16.254.111 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 15.16.254.112 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 15.16.254.112 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 15.16.254.113 I srv update: - prompt 0x6437c16c1840: 1063 tokens, checkpoints: 0, 191.029 MiB 15.16.254.114 I srv update: - prompt 0x6437c1492470: 946 tokens, checkpoints: 0, 170.003 MiB 15.16.254.114 I srv update: - prompt 0x6437c1416d50: 695 tokens, checkpoints: 0, 124.897 MiB 15.16.254.114 I srv update: - prompt 0x6437c509b7a0: 1412 tokens, checkpoints: 0, 253.747 MiB 15.16.254.115 I srv update: - prompt 0x6437c159f8a0: 1701 tokens, checkpoints: 0, 305.682 MiB 15.16.254.115 I srv update: - prompt 0x6437c147ecd0: 1715 tokens, checkpoints: 0, 308.198 MiB 15.16.254.116 I srv update: - prompt 0x6437c1572990: 521 tokens, checkpoints: 0, 93.628 MiB 15.16.254.117 I srv get_availabl: prompt cache update took 69.81 ms 15.16.254.164 I slot launch_slot_: id 3 | task 26247 | processing task, is_child = 0 15.16.254.165 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 15.16.254.166 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 15.16.254.176 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 15.16.254.176 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 15.16.254.184 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 15.16.254.185 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 15.19.885.836 I slot print_timing: id 3 | task 26247 | n_decoded = 100, tg = 31.84 t/s, tg_3s = 31.84 t/s 15.22.909.175 I slot print_timing: id 3 | task 26247 | n_decoded = 195, tg = 31.64 t/s, tg_3s = 31.42 t/s 15.25.922.131 I slot print_timing: id 3 | task 26247 | n_decoded = 289, tg = 31.49 t/s, tg_3s = 31.20 t/s 15.28.949.140 I slot print_timing: id 3 | task 26247 | n_decoded = 383, tg = 31.38 t/s, tg_3s = 31.05 t/s 15.31.951.224 I slot print_timing: id 3 | task 26247 | n_decoded = 476, tg = 31.30 t/s, tg_3s = 30.98 t/s 15.34.977.133 I slot print_timing: id 3 | task 26247 | n_decoded = 570, tg = 31.26 t/s, tg_3s = 31.07 t/s 15.38.004.805 I slot print_timing: id 3 | task 26247 | n_decoded = 664, tg = 31.23 t/s, tg_3s = 31.05 t/s 15.41.009.704 I slot print_timing: id 3 | task 26247 | n_decoded = 755, tg = 31.12 t/s, tg_3s = 30.28 t/s 15.44.015.825 I slot print_timing: id 3 | task 26247 | n_decoded = 846, tg = 31.02 t/s, tg_3s = 30.27 t/s 15.47.036.649 I slot print_timing: id 3 | task 26247 | n_decoded = 937, tg = 30.93 t/s, tg_3s = 30.12 t/s 15.50.044.381 I slot print_timing: id 3 | task 26247 | n_decoded = 1026, tg = 30.81 t/s, tg_3s = 29.59 t/s 15.53.053.652 I slot print_timing: id 3 | task 26247 | n_decoded = 1115, tg = 30.71 t/s, tg_3s = 29.58 t/s 15.56.075.993 I slot print_timing: id 3 | task 26247 | n_decoded = 1204, tg = 30.61 t/s, tg_3s = 29.45 t/s 15.59.102.383 I slot print_timing: id 3 | task 26247 | n_decoded = 1293, tg = 30.53 t/s, tg_3s = 29.41 t/s 15.59.646.450 I slot print_timing: id 3 | task 26247 | prompt eval time = 491.19 ms / 18 tokens ( 27.29 ms per token, 36.65 tokens per second) 15.59.646.454 I slot print_timing: id 3 | task 26247 | eval time = 42901.05 ms / 1309 tokens ( 32.77 ms per token, 30.51 tokens per second) 15.59.646.454 I slot print_timing: id 3 | task 26247 | total time = 43392.24 ms / 1327 tokens 15.59.646.455 I slot print_timing: id 3 | task 26247 | graphs reused = 27395 15.59.646.484 I slot release: id 3 | task 26247 | stop processing: n_tokens = 1398, truncated = 0 15.59.646.495 I srv update_slots: all slots are idle 15.59.658.030 I srv operator(): Chat format: peg-native 15.59.658.268 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.661 (> 0.100 thold), f_keep = 0.052 15.59.658.270 I srv get_availabl: updating prompt cache 15.59.658.338 W srv prompt_save: - saving prompt with length 1398, total state size = 251.231 MiB (draft: 0.000 MiB) 15.59.831.574 I srv load: - looking for better prompt, base f_keep = 0.052, sim = 0.661 15.59.831.586 I srv update: - cache state: 19 prompts, 5293.637 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 15.59.831.587 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 15.59.831.587 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 15.59.831.588 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 15.59.831.590 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 15.59.831.591 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 15.59.831.591 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 15.59.831.592 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 15.59.831.592 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 15.59.831.593 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 15.59.831.593 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 15.59.831.594 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 15.59.831.594 I srv update: - prompt 0x6437c16c1840: 1063 tokens, checkpoints: 0, 191.029 MiB 15.59.831.595 I srv update: - prompt 0x6437c1492470: 946 tokens, checkpoints: 0, 170.003 MiB 15.59.831.595 I srv update: - prompt 0x6437c1416d50: 695 tokens, checkpoints: 0, 124.897 MiB 15.59.831.596 I srv update: - prompt 0x6437c509b7a0: 1412 tokens, checkpoints: 0, 253.747 MiB 15.59.831.596 I srv update: - prompt 0x6437c159f8a0: 1701 tokens, checkpoints: 0, 305.682 MiB 15.59.831.597 I srv update: - prompt 0x6437c147ecd0: 1715 tokens, checkpoints: 0, 308.198 MiB 15.59.831.597 I srv update: - prompt 0x6437c1572990: 521 tokens, checkpoints: 0, 93.628 MiB 15.59.831.598 I srv update: - prompt 0x6437c9929f50: 1398 tokens, checkpoints: 0, 251.231 MiB 15.59.831.599 I srv get_availabl: prompt cache update took 173.33 ms 15.59.831.648 I slot launch_slot_: id 3 | task 27557 | processing task, is_child = 0 15.59.831.648 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 15.59.831.649 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 15.59.831.660 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 15.59.831.660 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 15.59.831.669 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 15.59.831.670 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 16.03.782.882 I slot print_timing: id 3 | task 27557 | n_decoded = 100, tg = 31.54 t/s, tg_3s = 31.54 t/s 16.06.794.306 I slot print_timing: id 3 | task 27557 | n_decoded = 194, tg = 31.38 t/s, tg_3s = 31.21 t/s 16.09.802.715 I slot print_timing: id 3 | task 27557 | n_decoded = 288, tg = 31.34 t/s, tg_3s = 31.25 t/s 16.12.811.309 I slot print_timing: id 3 | task 27557 | n_decoded = 382, tg = 31.31 t/s, tg_3s = 31.24 t/s 16.15.812.911 I slot print_timing: id 3 | task 27557 | n_decoded = 475, tg = 31.25 t/s, tg_3s = 30.98 t/s 16.17.034.807 I slot print_timing: id 3 | task 27557 | prompt eval time = 780.79 ms / 37 tokens ( 21.10 ms per token, 47.39 tokens per second) 16.17.034.811 I slot print_timing: id 3 | task 27557 | eval time = 16422.32 ms / 513 tokens ( 32.01 ms per token, 31.24 tokens per second) 16.17.034.811 I slot print_timing: id 3 | task 27557 | total time = 17203.12 ms / 550 tokens 16.17.034.812 I slot print_timing: id 3 | task 27557 | graphs reused = 27904 16.17.034.848 I slot release: id 3 | task 27557 | stop processing: n_tokens = 621, truncated = 0 16.17.034.858 I srv update_slots: all slots are idle 16.17.046.708 I srv operator(): Chat format: peg-native 16.17.046.946 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.735 (> 0.100 thold), f_keep = 0.116 16.17.046.948 I srv get_availabl: updating prompt cache 16.17.046.988 W srv prompt_save: - saving prompt with length 621, total state size = 111.599 MiB (draft: 0.000 MiB) 16.17.127.718 I srv load: - looking for better prompt, base f_keep = 0.116, sim = 0.735 16.17.127.732 I srv update: - cache state: 20 prompts, 5405.236 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 16.17.127.733 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 16.17.127.734 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 16.17.127.735 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 16.17.127.735 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 16.17.127.736 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 16.17.127.736 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 16.17.127.737 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 16.17.127.737 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 16.17.127.738 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 16.17.127.738 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 16.17.127.739 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 16.17.127.739 I srv update: - prompt 0x6437c16c1840: 1063 tokens, checkpoints: 0, 191.029 MiB 16.17.127.740 I srv update: - prompt 0x6437c1492470: 946 tokens, checkpoints: 0, 170.003 MiB 16.17.127.740 I srv update: - prompt 0x6437c1416d50: 695 tokens, checkpoints: 0, 124.897 MiB 16.17.127.741 I srv update: - prompt 0x6437c509b7a0: 1412 tokens, checkpoints: 0, 253.747 MiB 16.17.127.741 I srv update: - prompt 0x6437c159f8a0: 1701 tokens, checkpoints: 0, 305.682 MiB 16.17.127.742 I srv update: - prompt 0x6437c147ecd0: 1715 tokens, checkpoints: 0, 308.198 MiB 16.17.127.742 I srv update: - prompt 0x6437c1572990: 521 tokens, checkpoints: 0, 93.628 MiB 16.17.127.743 I srv update: - prompt 0x6437c9929f50: 1398 tokens, checkpoints: 0, 251.231 MiB 16.17.127.743 I srv update: - prompt 0x6437c91a6de0: 621 tokens, checkpoints: 0, 111.599 MiB 16.17.127.745 I srv get_availabl: prompt cache update took 80.80 ms 16.17.127.792 I slot launch_slot_: id 3 | task 28071 | processing task, is_child = 0 16.17.127.793 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 16.17.127.794 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 16.17.127.804 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 16.17.127.804 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 16.17.127.813 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 16.17.127.813 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 16.20.855.424 I slot print_timing: id 3 | task 28071 | n_decoded = 100, tg = 31.79 t/s, tg_3s = 31.79 t/s 16.23.858.674 I slot print_timing: id 3 | task 28071 | n_decoded = 194, tg = 31.55 t/s, tg_3s = 31.30 t/s 16.26.885.353 I slot print_timing: id 3 | task 28071 | n_decoded = 288, tg = 31.39 t/s, tg_3s = 31.06 t/s 16.29.885.908 I slot print_timing: id 3 | task 28071 | n_decoded = 381, tg = 31.29 t/s, tg_3s = 30.99 t/s 16.32.892.749 I slot print_timing: id 3 | task 28071 | n_decoded = 474, tg = 31.22 t/s, tg_3s = 30.93 t/s 16.35.572.768 I slot print_timing: id 3 | task 28071 | prompt eval time = 582.15 ms / 26 tokens ( 22.39 ms per token, 44.66 tokens per second) 16.35.572.771 I slot print_timing: id 3 | task 28071 | eval time = 17862.79 ms / 557 tokens ( 32.07 ms per token, 31.18 tokens per second) 16.35.572.772 I slot print_timing: id 3 | task 28071 | total time = 18444.94 ms / 583 tokens 16.35.572.773 I slot print_timing: id 3 | task 28071 | graphs reused = 28457 16.35.572.802 I slot release: id 3 | task 28071 | stop processing: n_tokens = 654, truncated = 0 16.35.572.812 I srv update_slots: all slots are idle 16.35.584.133 I srv operator(): Chat format: peg-native 16.35.584.374 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.699 (> 0.100 thold), f_keep = 0.110 16.35.584.375 I srv get_availabl: updating prompt cache 16.35.584.418 W srv prompt_save: - saving prompt with length 654, total state size = 117.529 MiB (draft: 0.000 MiB) 16.35.669.746 I srv load: - looking for better prompt, base f_keep = 0.110, sim = 0.699 16.35.669.758 I srv update: - cache state: 21 prompts, 5522.765 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 16.35.669.759 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 16.35.669.761 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 16.35.669.762 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 16.35.669.762 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 16.35.669.763 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 16.35.669.763 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 16.35.669.764 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 16.35.669.764 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 16.35.669.765 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 16.35.669.765 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 16.35.669.766 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 16.35.669.766 I srv update: - prompt 0x6437c16c1840: 1063 tokens, checkpoints: 0, 191.029 MiB 16.35.669.767 I srv update: - prompt 0x6437c1492470: 946 tokens, checkpoints: 0, 170.003 MiB 16.35.669.767 I srv update: - prompt 0x6437c1416d50: 695 tokens, checkpoints: 0, 124.897 MiB 16.35.669.768 I srv update: - prompt 0x6437c509b7a0: 1412 tokens, checkpoints: 0, 253.747 MiB 16.35.669.768 I srv update: - prompt 0x6437c159f8a0: 1701 tokens, checkpoints: 0, 305.682 MiB 16.35.669.769 I srv update: - prompt 0x6437c147ecd0: 1715 tokens, checkpoints: 0, 308.198 MiB 16.35.669.769 I srv update: - prompt 0x6437c1572990: 521 tokens, checkpoints: 0, 93.628 MiB 16.35.669.770 I srv update: - prompt 0x6437c9929f50: 1398 tokens, checkpoints: 0, 251.231 MiB 16.35.669.770 I srv update: - prompt 0x6437c91a6de0: 621 tokens, checkpoints: 0, 111.599 MiB 16.35.669.771 I srv update: - prompt 0x6437b948e9f0: 654 tokens, checkpoints: 0, 117.529 MiB 16.35.669.772 I srv get_availabl: prompt cache update took 85.40 ms 16.35.669.817 I slot launch_slot_: id 3 | task 28629 | processing task, is_child = 0 16.35.669.818 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 16.35.669.819 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 16.35.669.829 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 16.35.669.829 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 16.35.669.849 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 16.35.669.849 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 16.39.441.803 I slot print_timing: id 3 | task 28629 | n_decoded = 100, tg = 31.61 t/s, tg_3s = 31.61 t/s 16.42.462.998 I slot print_timing: id 3 | task 28629 | n_decoded = 194, tg = 31.37 t/s, tg_3s = 31.11 t/s 16.45.467.173 I slot print_timing: id 3 | task 28629 | n_decoded = 287, tg = 31.23 t/s, tg_3s = 30.96 t/s 16.48.474.206 I slot print_timing: id 3 | task 28629 | n_decoded = 380, tg = 31.16 t/s, tg_3s = 30.93 t/s 16.51.482.700 I slot print_timing: id 3 | task 28629 | n_decoded = 473, tg = 31.11 t/s, tg_3s = 30.91 t/s 16.54.490.784 I slot print_timing: id 3 | task 28629 | n_decoded = 567, tg = 31.13 t/s, tg_3s = 31.25 t/s 16.57.498.808 I slot print_timing: id 3 | task 28629 | n_decoded = 661, tg = 31.15 t/s, tg_3s = 31.25 t/s 17.00.509.509 I slot print_timing: id 3 | task 28629 | n_decoded = 752, tg = 31.03 t/s, tg_3s = 30.23 t/s 17.03.522.520 I slot print_timing: id 3 | task 28629 | n_decoded = 843, tg = 30.94 t/s, tg_3s = 30.20 t/s 17.06.548.789 I slot print_timing: id 3 | task 28629 | n_decoded = 934, tg = 30.86 t/s, tg_3s = 30.07 t/s 17.09.582.374 I slot print_timing: id 3 | task 28629 | n_decoded = 1024, tg = 30.75 t/s, tg_3s = 29.67 t/s 17.12.586.296 I slot print_timing: id 3 | task 28629 | n_decoded = 1113, tg = 30.65 t/s, tg_3s = 29.63 t/s 17.15.596.909 I slot print_timing: id 3 | task 28629 | n_decoded = 1202, tg = 30.57 t/s, tg_3s = 29.56 t/s 17.18.614.345 I slot print_timing: id 3 | task 28629 | n_decoded = 1291, tg = 30.49 t/s, tg_3s = 29.50 t/s 17.21.631.880 I slot print_timing: id 3 | task 28629 | n_decoded = 1380, tg = 30.43 t/s, tg_3s = 29.49 t/s 17.24.633.898 I slot print_timing: id 3 | task 28629 | n_decoded = 1468, tg = 30.36 t/s, tg_3s = 29.31 t/s 17.25.790.066 I slot print_timing: id 3 | task 28629 | prompt eval time = 608.80 ms / 31 tokens ( 19.64 ms per token, 50.92 tokens per second) 17.25.790.070 I slot print_timing: id 3 | task 28629 | eval time = 49511.38 ms / 1502 tokens ( 32.96 ms per token, 30.34 tokens per second) 17.25.790.071 I slot print_timing: id 3 | task 28629 | total time = 50120.18 ms / 1533 tokens 17.25.790.071 I slot print_timing: id 3 | task 28629 | graphs reused = 29951 17.25.790.156 I slot release: id 3 | task 28629 | stop processing: n_tokens = 1604, truncated = 0 17.25.790.169 I srv update_slots: all slots are idle 17.25.801.281 I srv operator(): Chat format: peg-native 17.25.801.521 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.643 (> 0.100 thold), f_keep = 0.045 17.25.801.522 I srv get_availabl: updating prompt cache 17.25.801.584 W srv prompt_save: - saving prompt with length 1604, total state size = 288.250 MiB (draft: 0.000 MiB) 17.25.997.832 I srv load: - looking for better prompt, base f_keep = 0.045, sim = 0.643 17.25.997.845 I srv update: - cache state: 22 prompts, 5811.015 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 17.25.997.846 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 17.25.997.847 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 17.25.997.847 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 17.25.997.848 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 17.25.997.849 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 17.25.997.849 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 17.25.997.851 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 17.25.997.852 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 17.25.997.852 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 17.25.997.853 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 17.25.997.853 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 17.25.997.854 I srv update: - prompt 0x6437c16c1840: 1063 tokens, checkpoints: 0, 191.029 MiB 17.25.997.855 I srv update: - prompt 0x6437c1492470: 946 tokens, checkpoints: 0, 170.003 MiB 17.25.997.855 I srv update: - prompt 0x6437c1416d50: 695 tokens, checkpoints: 0, 124.897 MiB 17.25.997.856 I srv update: - prompt 0x6437c509b7a0: 1412 tokens, checkpoints: 0, 253.747 MiB 17.25.997.856 I srv update: - prompt 0x6437c159f8a0: 1701 tokens, checkpoints: 0, 305.682 MiB 17.25.997.857 I srv update: - prompt 0x6437c147ecd0: 1715 tokens, checkpoints: 0, 308.198 MiB 17.25.997.857 I srv update: - prompt 0x6437c1572990: 521 tokens, checkpoints: 0, 93.628 MiB 17.25.997.858 I srv update: - prompt 0x6437c9929f50: 1398 tokens, checkpoints: 0, 251.231 MiB 17.25.997.858 I srv update: - prompt 0x6437c91a6de0: 621 tokens, checkpoints: 0, 111.599 MiB 17.25.997.859 I srv update: - prompt 0x6437b948e9f0: 654 tokens, checkpoints: 0, 117.529 MiB 17.25.997.859 I srv update: - prompt 0x6437c99e0270: 1604 tokens, checkpoints: 0, 288.250 MiB 17.25.997.861 I srv get_availabl: prompt cache update took 196.34 ms 17.25.997.905 I slot launch_slot_: id 3 | task 30132 | processing task, is_child = 0 17.25.997.906 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 17.25.997.906 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 17.25.997.918 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 17.25.997.919 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 17.25.997.928 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 17.25.997.928 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 17.29.969.861 I slot print_timing: id 3 | task 30132 | n_decoded = 100, tg = 31.61 t/s, tg_3s = 31.61 t/s 17.32.996.099 I slot print_timing: id 3 | task 30132 | n_decoded = 194, tg = 31.34 t/s, tg_3s = 31.06 t/s 17.36.006.260 I slot print_timing: id 3 | task 30132 | n_decoded = 287, tg = 31.20 t/s, tg_3s = 30.90 t/s 17.39.016.774 I slot print_timing: id 3 | task 30132 | n_decoded = 380, tg = 31.12 t/s, tg_3s = 30.89 t/s 17.42.042.991 I slot print_timing: id 3 | task 30132 | n_decoded = 473, tg = 31.04 t/s, tg_3s = 30.73 t/s 17.43.473.050 I slot print_timing: id 3 | task 30132 | prompt eval time = 808.66 ms / 40 tokens ( 20.22 ms per token, 49.46 tokens per second) 17.43.473.053 I slot print_timing: id 3 | task 30132 | eval time = 16666.44 ms / 517 tokens ( 32.24 ms per token, 31.02 tokens per second) 17.43.473.054 I slot print_timing: id 3 | task 30132 | total time = 17475.10 ms / 557 tokens 17.43.473.054 I slot print_timing: id 3 | task 30132 | graphs reused = 30464 17.43.473.094 I slot release: id 3 | task 30132 | stop processing: n_tokens = 628, truncated = 0 17.43.473.104 I srv update_slots: all slots are idle 17.43.484.447 I srv operator(): Chat format: peg-native 17.43.484.684 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.115 17.43.484.684 I srv get_availabl: updating prompt cache 17.43.484.726 W srv prompt_save: - saving prompt with length 628, total state size = 112.857 MiB (draft: 0.000 MiB) 17.43.566.051 I srv load: - looking for better prompt, base f_keep = 0.115, sim = 0.742 17.43.566.065 I srv update: - cache state: 23 prompts, 5923.872 MiB (limits: 8192.000 MiB, 16384 tokens, 45585 est) 17.43.566.069 I srv update: - prompt 0x6437c94571b0: 3009 tokens, checkpoints: 0, 540.738 MiB 17.43.566.069 I srv update: - prompt 0x6437c162f170: 1224 tokens, checkpoints: 0, 219.962 MiB 17.43.566.070 I srv update: - prompt 0x6437c9940840: 1728 tokens, checkpoints: 0, 310.534 MiB 17.43.566.070 I srv update: - prompt 0x6437c131b970: 2134 tokens, checkpoints: 0, 383.495 MiB 17.43.566.071 I srv update: - prompt 0x6437be4f16e0: 2684 tokens, checkpoints: 0, 482.334 MiB 17.43.566.072 I srv update: - prompt 0x6437be7faee0: 1983 tokens, checkpoints: 0, 356.359 MiB 17.43.566.073 I srv update: - prompt 0x6437c9985af0: 1426 tokens, checkpoints: 0, 256.263 MiB 17.43.566.074 I srv update: - prompt 0x6437c1694f10: 2210 tokens, checkpoints: 0, 397.153 MiB 17.43.566.075 I srv update: - prompt 0x6437c14eb660: 1421 tokens, checkpoints: 0, 255.364 MiB 17.43.566.075 I srv update: - prompt 0x6437c13415b0: 1415 tokens, checkpoints: 0, 254.286 MiB 17.43.566.076 I srv update: - prompt 0x6437c1117500: 772 tokens, checkpoints: 0, 138.735 MiB 17.43.566.076 I srv update: - prompt 0x6437c16c1840: 1063 tokens, checkpoints: 0, 191.029 MiB 17.43.566.077 I srv update: - prompt 0x6437c1492470: 946 tokens, checkpoints: 0, 170.003 MiB 17.43.566.078 I srv update: - prompt 0x6437c1416d50: 695 tokens, checkpoints: 0, 124.897 MiB 17.43.566.078 I srv update: - prompt 0x6437c509b7a0: 1412 tokens, checkpoints: 0, 253.747 MiB 17.43.566.079 I srv update: - prompt 0x6437c159f8a0: 1701 tokens, checkpoints: 0, 305.682 MiB 17.43.566.079 I srv update: - prompt 0x6437c147ecd0: 1715 tokens, checkpoints: 0, 308.198 MiB 17.43.566.080 I srv update: - prompt 0x6437c1572990: 521 tokens, checkpoints: 0, 93.628 MiB 17.43.566.080 I srv update: - prompt 0x6437c9929f50: 1398 tokens, checkpoints: 0, 251.231 MiB 17.43.566.081 I srv update: - prompt 0x6437c91a6de0: 621 tokens, checkpoints: 0, 111.599 MiB 17.43.566.081 I srv update: - prompt 0x6437b948e9f0: 654 tokens, checkpoints: 0, 117.529 MiB 17.43.566.082 I srv update: - prompt 0x6437c99e0270: 1604 tokens, checkpoints: 0, 288.250 MiB 17.43.566.082 I srv update: - prompt 0x6437c15e9e50: 628 tokens, checkpoints: 0, 112.857 MiB 17.43.566.084 I srv get_availabl: prompt cache update took 81.40 ms 17.43.566.128 I slot launch_slot_: id 3 | task 30650 | processing task, is_child = 0 17.43.566.129 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 17.43.566.130 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 17.43.566.140 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 17.43.566.140 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 17.43.566.148 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 17.43.566.149 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 17.47.267.969 I slot print_timing: id 3 | task 30650 | n_decoded = 100, tg = 31.78 t/s, tg_3s = 31.78 t/s 17.50.299.745 I slot print_timing: id 3 | task 30650 | n_decoded = 195, tg = 31.56 t/s, tg_3s = 31.33 t/s 17.53.315.789 I slot print_timing: id 3 | task 30650 | n_decoded = 289, tg = 31.43 t/s, tg_3s = 31.17 t/s 17.56.325.840 I slot print_timing: id 3 | task 30650 | n_decoded = 383, tg = 31.38 t/s, tg_3s = 31.23 t/s 17.59.351.573 I slot print_timing: id 3 | task 30650 | n_decoded = 477, tg = 31.32 t/s, tg_3s = 31.07 t/s 18.01.561.903 I slot print_timing: id 3 | task 30650 | prompt eval time = 554.72 ms / 25 tokens ( 22.19 ms per token, 45.07 tokens per second) 18.01.561.906 I slot print_timing: id 3 | task 30650 | eval time = 17441.01 ms / 546 tokens ( 31.94 ms per token, 31.31 tokens per second) 18.01.561.907 I slot print_timing: id 3 | task 30650 | total time = 17995.73 ms / 571 tokens 18.01.561.908 I slot print_timing: id 3 | task 30650 | graphs reused = 31006 18.01.561.945 I slot release: id 3 | task 30650 | stop processing: n_tokens = 642, truncated = 0 18.01.561.954 I srv update_slots: all slots are idle 18.01.562.681 I srv operator(): operator(): cleaning up before exit...