0.00.065.276 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.065.279 I device_info: 0.00.065.299 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.065.303 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.065.310 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free) 0.00.065.364 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.065.368 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true 0.00.065.408 I srv init: running without SSL 0.00.065.473 I srv init: using 23 threads for HTTP server 0.00.065.798 I srv start: binding port with default address family 0.00.067.000 I srv llama_server: loading model 0.00.067.009 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/zai-org_GLM-4.7-Flash-Q5_K_M.gguf' 0.00.067.068 I common_init_result: fitting params to device memory ... 0.00.067.070 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) 0.01.172.880 W load: special_eot_id is not in special_eog_ids - the tokenizer config may be incorrect 0.01.172.887 W load: special_eom_id is not in special_eog_ids - the tokenizer config may be incorrect 0.14.477.848 W llama_context: n_ctx_seq (16384) < n_ctx_train (202752) -- the full capacity of the model will not be utilized 0.14.576.566 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) 0.14.760.255 I srv load_model: initializing slots, n_slots = 4 0.14.810.782 W common_speculative_init: no implementations specified for speculative decoding 0.14.810.787 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 0.14.810.792 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384 0.14.810.792 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384 0.14.810.793 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384 0.14.810.833 I srv load_model: prompt cache is enabled, size limit: 8192 MiB 0.14.810.834 I srv load_model: use `--cache-ram 0` to disable the prompt cache 0.14.810.835 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 0.14.810.835 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 0.14.810.853 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task 0.14.822.971 I init: chat template, example_format: '[gMASK]<|system|>You are a helpful assistant<|user|>Hello<|assistant|>Hi there<|user|>How are you?<|assistant|>' 0.14.832.242 I srv init: init: chat template, thinking = 1 0.14.832.276 I srv llama_server: model loaded 0.14.832.279 I srv llama_server: server is listening on http://127.0.0.1:8090 0.14.832.285 I srv update_slots: all slots are idle 0.14.975.167 I srv operator(): Chat format: peg-native 0.14.975.419 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1 0.14.975.421 I srv get_availabl: updating prompt cache 0.14.975.425 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 0.14.975.429 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) 0.14.975.430 I srv get_availabl: prompt cache update took 0.01 ms 0.14.975.449 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0 0.14.975.450 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.14.975.450 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.14.975.462 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.14.975.462 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.14.975.471 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.14.975.471 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.18.774.933 I slot print_timing: id 3 | task 0 | n_decoded = 137, tg = 45.54 t/s, tg_3s = 45.54 t/s 0.21.785.772 I slot print_timing: id 3 | task 0 | n_decoded = 273, tg = 45.36 t/s, tg_3s = 45.17 t/s 0.24.790.037 I slot print_timing: id 3 | task 0 | n_decoded = 409, tg = 45.33 t/s, tg_3s = 45.27 t/s 0.27.792.122 I slot print_timing: id 3 | task 0 | n_decoded = 543, tg = 45.15 t/s, tg_3s = 44.64 t/s 0.30.808.668 I slot print_timing: id 3 | task 0 | n_decoded = 678, tg = 45.07 t/s, tg_3s = 44.75 t/s 0.33.831.004 I slot print_timing: id 3 | task 0 | n_decoded = 812, tg = 44.95 t/s, tg_3s = 44.34 t/s 0.36.834.227 I slot print_timing: id 3 | task 0 | n_decoded = 945, tg = 44.86 t/s, tg_3s = 44.29 t/s 0.39.834.712 I slot print_timing: id 3 | task 0 | n_decoded = 1078, tg = 44.79 t/s, tg_3s = 44.33 t/s 0.42.849.674 I slot print_timing: id 3 | task 0 | n_decoded = 1211, tg = 44.71 t/s, tg_3s = 44.11 t/s 0.45.858.881 I slot print_timing: id 3 | task 0 | n_decoded = 1345, tg = 44.70 t/s, tg_3s = 44.53 t/s 0.48.878.637 I slot print_timing: id 3 | task 0 | n_decoded = 1479, tg = 44.67 t/s, tg_3s = 44.37 t/s 0.51.893.740 I slot print_timing: id 3 | task 0 | n_decoded = 1613, tg = 44.65 t/s, tg_3s = 44.44 t/s 0.54.904.620 I slot print_timing: id 3 | task 0 | n_decoded = 1746, tg = 44.61 t/s, tg_3s = 44.17 t/s 0.55.334.887 I slot print_timing: id 3 | task 0 | prompt eval time = 791.24 ms / 91 tokens ( 8.69 ms per token, 115.01 tokens per second) 0.55.334.891 I slot print_timing: id 3 | task 0 | eval time = 39568.15 ms / 1765 tokens ( 22.42 ms per token, 44.61 tokens per second) 0.55.334.892 I slot print_timing: id 3 | task 0 | total time = 40359.39 ms / 1856 tokens 0.55.334.896 I slot print_timing: id 3 | task 0 | graphs reused = 1757 0.55.334.954 I slot release: id 3 | task 0 | stop processing: n_tokens = 1855, truncated = 0 0.55.334.966 I srv update_slots: all slots are idle 0.55.346.647 I srv operator(): Chat format: peg-native 0.55.346.876 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.824 (> 0.100 thold), f_keep = 0.038 0.55.346.878 I srv get_availabl: updating prompt cache 0.55.346.952 W srv prompt_save: - saving prompt with length 1855, total state size = 95.806 MiB (draft: 0.000 MiB) 0.55.452.002 I srv load: - looking for better prompt, base f_keep = 0.038, sim = 0.824 0.55.452.007 I srv update: - cache state: 1 prompts, 95.806 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 0.55.452.008 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 0.55.452.009 I srv get_availabl: prompt cache update took 105.13 ms 0.55.452.057 I slot launch_slot_: id 3 | task 1767 | processing task, is_child = 0 0.55.452.057 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.55.452.058 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.55.452.069 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.55.452.069 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.55.452.078 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.55.452.079 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.58.613.096 I slot print_timing: id 3 | task 1767 | n_decoded = 137, tg = 45.56 t/s, tg_3s = 45.56 t/s 1.01.620.453 I slot print_timing: id 3 | task 1767 | n_decoded = 273, tg = 45.39 t/s, tg_3s = 45.22 t/s 1.04.636.499 I slot print_timing: id 3 | task 1767 | n_decoded = 410, tg = 45.40 t/s, tg_3s = 45.42 t/s 1.07.657.633 I slot print_timing: id 3 | task 1767 | n_decoded = 545, tg = 45.22 t/s, tg_3s = 44.69 t/s 1.10.671.523 I slot print_timing: id 3 | task 1767 | n_decoded = 680, tg = 45.14 t/s, tg_3s = 44.79 t/s 1.13.686.674 I slot print_timing: id 3 | task 1767 | n_decoded = 814, tg = 45.02 t/s, tg_3s = 44.44 t/s 1.16.701.281 I slot print_timing: id 3 | task 1767 | n_decoded = 948, tg = 44.94 t/s, tg_3s = 44.45 t/s 1.19.718.347 I slot print_timing: id 3 | task 1767 | n_decoded = 1082, tg = 44.87 t/s, tg_3s = 44.41 t/s 1.21.340.981 I slot print_timing: id 3 | task 1767 | prompt eval time = 153.66 ms / 15 tokens ( 10.24 ms per token, 97.62 tokens per second) 1.21.340.984 I slot print_timing: id 3 | task 1767 | eval time = 25735.22 ms / 1154 tokens ( 22.30 ms per token, 44.84 tokens per second) 1.21.340.985 I slot print_timing: id 3 | task 1767 | total time = 25888.88 ms / 1169 tokens 1.21.340.985 I slot print_timing: id 3 | task 1767 | graphs reused = 2905 1.21.341.031 I slot release: id 3 | task 1767 | stop processing: n_tokens = 1238, truncated = 0 1.21.341.043 I srv update_slots: all slots are idle 1.21.352.430 I srv operator(): Chat format: peg-native 1.21.352.656 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.814 (> 0.100 thold), f_keep = 0.057 1.21.352.658 I srv get_availabl: updating prompt cache 1.21.352.718 W srv prompt_save: - saving prompt with length 1238, total state size = 63.940 MiB (draft: 0.000 MiB) 1.21.441.960 I srv load: - looking for better prompt, base f_keep = 0.057, sim = 0.814 1.21.441.966 I srv update: - cache state: 2 prompts, 159.746 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 1.21.441.967 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 1.21.441.968 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 1.21.441.969 I srv get_availabl: prompt cache update took 89.31 ms 1.21.442.008 I slot launch_slot_: id 3 | task 2922 | processing task, is_child = 0 1.21.442.009 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.21.442.009 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.21.442.024 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.21.442.024 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.21.442.033 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.21.442.034 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.24.597.561 I slot print_timing: id 3 | task 2922 | n_decoded = 136, tg = 45.27 t/s, tg_3s = 45.27 t/s 1.27.609.883 I slot print_timing: id 3 | task 2922 | n_decoded = 271, tg = 45.04 t/s, tg_3s = 44.82 t/s 1.30.616.397 I slot print_timing: id 3 | task 2922 | n_decoded = 406, tg = 45.00 t/s, tg_3s = 44.90 t/s 1.33.623.969 I slot print_timing: id 3 | task 2922 | n_decoded = 540, tg = 44.89 t/s, tg_3s = 44.55 t/s 1.36.594.805 I slot print_timing: id 3 | task 2922 | prompt eval time = 151.31 ms / 16 tokens ( 9.46 ms per token, 105.75 tokens per second) 1.36.594.808 I slot print_timing: id 3 | task 2922 | eval time = 15001.44 ms / 673 tokens ( 22.29 ms per token, 44.86 tokens per second) 1.36.594.809 I slot print_timing: id 3 | task 2922 | total time = 15152.75 ms / 689 tokens 1.36.594.810 I slot print_timing: id 3 | task 2922 | graphs reused = 3574 1.36.594.848 I slot release: id 3 | task 2922 | stop processing: n_tokens = 758, truncated = 0 1.36.594.858 I srv update_slots: all slots are idle 1.36.606.066 I srv operator(): Chat format: peg-native 1.36.606.296 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.092 1.36.606.298 I srv get_availabl: updating prompt cache 1.36.606.342 W srv prompt_save: - saving prompt with length 758, total state size = 39.149 MiB (draft: 0.000 MiB) 1.36.636.126 I srv load: - looking for better prompt, base f_keep = 0.092, sim = 0.833 1.36.636.136 I srv update: - cache state: 3 prompts, 198.895 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 1.36.636.137 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 1.36.636.138 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 1.36.636.139 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 1.36.636.140 I srv get_availabl: prompt cache update took 29.84 ms 1.36.636.192 I slot launch_slot_: id 3 | task 3596 | processing task, is_child = 0 1.36.636.192 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.36.636.193 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.36.636.206 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.36.636.207 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.36.636.215 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.36.636.216 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.39.790.700 I slot print_timing: id 3 | task 3596 | n_decoded = 137, tg = 45.62 t/s, tg_3s = 45.62 t/s 1.42.792.812 I slot print_timing: id 3 | task 3596 | n_decoded = 273, tg = 45.46 t/s, tg_3s = 45.30 t/s 1.45.793.908 I slot print_timing: id 3 | task 3596 | n_decoded = 409, tg = 45.41 t/s, tg_3s = 45.32 t/s 1.48.808.322 I slot print_timing: id 3 | task 3596 | n_decoded = 544, tg = 45.26 t/s, tg_3s = 44.78 t/s 1.51.813.477 I slot print_timing: id 3 | task 3596 | n_decoded = 679, tg = 45.19 t/s, tg_3s = 44.92 t/s 1.54.817.026 I slot print_timing: id 3 | task 3596 | n_decoded = 813, tg = 45.09 t/s, tg_3s = 44.61 t/s 1.57.820.585 I slot print_timing: id 3 | task 3596 | n_decoded = 947, tg = 45.02 t/s, tg_3s = 44.61 t/s 2.00.829.366 I slot print_timing: id 3 | task 3596 | n_decoded = 1081, tg = 44.96 t/s, tg_3s = 44.54 t/s 2.03.832.952 I slot print_timing: id 3 | task 3596 | n_decoded = 1214, tg = 44.89 t/s, tg_3s = 44.28 t/s 2.06.833.830 I slot print_timing: id 3 | task 3596 | n_decoded = 1347, tg = 44.83 t/s, tg_3s = 44.32 t/s 2.07.082.131 I slot print_timing: id 3 | task 3596 | prompt eval time = 151.33 ms / 14 tokens ( 10.81 ms per token, 92.51 tokens per second) 2.07.082.134 I slot print_timing: id 3 | task 3596 | eval time = 30294.56 ms / 1358 tokens ( 22.31 ms per token, 44.83 tokens per second) 2.07.082.135 I slot print_timing: id 3 | task 3596 | total time = 30445.89 ms / 1372 tokens 2.07.082.135 I slot print_timing: id 3 | task 3596 | graphs reused = 4925 2.07.082.172 I slot release: id 3 | task 3596 | stop processing: n_tokens = 1441, truncated = 0 2.07.082.186 I srv update_slots: all slots are idle 2.07.093.406 I srv operator(): Chat format: peg-native 2.07.093.637 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.722 (> 0.100 thold), f_keep = 0.049 2.07.093.639 I srv get_availabl: updating prompt cache 2.07.093.712 W srv prompt_save: - saving prompt with length 1441, total state size = 74.424 MiB (draft: 0.000 MiB) 2.07.156.606 I srv load: - looking for better prompt, base f_keep = 0.049, sim = 0.722 2.07.156.612 I srv update: - cache state: 4 prompts, 273.319 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 2.07.156.613 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 2.07.156.614 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 2.07.156.614 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 2.07.156.615 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 2.07.156.616 I srv get_availabl: prompt cache update took 62.98 ms 2.07.156.647 I slot launch_slot_: id 3 | task 4955 | processing task, is_child = 0 2.07.156.648 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.07.156.649 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.07.156.660 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.07.156.660 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.07.156.669 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.07.156.670 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.10.402.446 I slot print_timing: id 3 | task 4955 | n_decoded = 137, tg = 45.56 t/s, tg_3s = 45.56 t/s 2.13.407.861 I slot print_timing: id 3 | task 4955 | n_decoded = 272, tg = 45.24 t/s, tg_3s = 44.92 t/s 2.16.428.585 I slot print_timing: id 3 | task 4955 | n_decoded = 408, tg = 45.17 t/s, tg_3s = 45.02 t/s 2.19.438.808 I slot print_timing: id 3 | task 4955 | n_decoded = 542, tg = 45.00 t/s, tg_3s = 44.51 t/s 2.22.453.430 I slot print_timing: id 3 | task 4955 | n_decoded = 676, tg = 44.89 t/s, tg_3s = 44.45 t/s 2.25.459.794 I slot print_timing: id 3 | task 4955 | n_decoded = 811, tg = 44.89 t/s, tg_3s = 44.90 t/s 2.28.471.594 I slot print_timing: id 3 | task 4955 | n_decoded = 946, tg = 44.88 t/s, tg_3s = 44.82 t/s 2.31.490.448 I slot print_timing: id 3 | task 4955 | n_decoded = 1081, tg = 44.86 t/s, tg_3s = 44.72 t/s 2.34.505.203 I slot print_timing: id 3 | task 4955 | n_decoded = 1215, tg = 44.82 t/s, tg_3s = 44.45 t/s 2.37.519.604 I slot print_timing: id 3 | task 4955 | n_decoded = 1349, tg = 44.78 t/s, tg_3s = 44.45 t/s 2.40.523.479 I slot print_timing: id 3 | task 4955 | n_decoded = 1482, tg = 44.74 t/s, tg_3s = 44.28 t/s 2.43.540.896 I slot print_timing: id 3 | task 4955 | n_decoded = 1616, tg = 44.71 t/s, tg_3s = 44.41 t/s 2.46.552.591 I slot print_timing: id 3 | task 4955 | n_decoded = 1749, tg = 44.67 t/s, tg_3s = 44.16 t/s 2.49.565.980 I slot print_timing: id 3 | task 4955 | n_decoded = 1882, tg = 44.63 t/s, tg_3s = 44.14 t/s 2.49.611.523 I slot print_timing: id 3 | task 4955 | prompt eval time = 238.70 ms / 27 tokens ( 8.84 ms per token, 113.11 tokens per second) 2.49.611.526 I slot print_timing: id 3 | task 4955 | eval time = 42216.13 ms / 1884 tokens ( 22.41 ms per token, 44.63 tokens per second) 2.49.611.527 I slot print_timing: id 3 | task 4955 | total time = 42454.84 ms / 1911 tokens 2.49.611.527 I slot print_timing: id 3 | task 4955 | graphs reused = 6800 2.49.611.566 I slot release: id 3 | task 4955 | stop processing: n_tokens = 1980, truncated = 0 2.49.611.581 I srv update_slots: all slots are idle 2.49.623.172 I srv operator(): Chat format: peg-native 2.49.623.404 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.805 (> 0.100 thold), f_keep = 0.035 2.49.623.405 I srv get_availabl: updating prompt cache 2.49.623.480 W srv prompt_save: - saving prompt with length 1980, total state size = 102.262 MiB (draft: 0.000 MiB) 2.49.753.953 I srv load: - looking for better prompt, base f_keep = 0.035, sim = 0.805 2.49.753.962 I srv update: - cache state: 5 prompts, 375.581 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 2.49.753.963 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 2.49.753.964 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 2.49.753.965 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 2.49.753.965 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 2.49.753.966 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 2.49.753.967 I srv get_availabl: prompt cache update took 130.56 ms 2.49.754.009 I slot launch_slot_: id 3 | task 6840 | processing task, is_child = 0 2.49.754.010 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.49.754.011 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.49.754.031 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.49.754.032 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.49.754.048 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.49.754.048 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.52.947.453 I slot print_timing: id 3 | task 6840 | n_decoded = 137, tg = 45.54 t/s, tg_3s = 45.54 t/s 2.55.956.574 I slot print_timing: id 3 | task 6840 | n_decoded = 273, tg = 45.37 t/s, tg_3s = 45.20 t/s 2.58.957.305 I slot print_timing: id 3 | task 6840 | n_decoded = 408, tg = 45.24 t/s, tg_3s = 44.99 t/s 3.01.965.188 I slot print_timing: id 3 | task 6840 | n_decoded = 542, tg = 45.07 t/s, tg_3s = 44.55 t/s 3.04.972.532 I slot print_timing: id 3 | task 6840 | n_decoded = 676, tg = 44.97 t/s, tg_3s = 44.56 t/s 3.07.973.765 I slot print_timing: id 3 | task 6840 | n_decoded = 809, tg = 44.86 t/s, tg_3s = 44.32 t/s 3.10.978.300 I slot print_timing: id 3 | task 6840 | n_decoded = 942, tg = 44.77 t/s, tg_3s = 44.27 t/s 3.13.997.366 I slot print_timing: id 3 | task 6840 | n_decoded = 1076, tg = 44.73 t/s, tg_3s = 44.38 t/s 3.15.954.807 I slot print_timing: id 3 | task 6840 | prompt eval time = 185.36 ms / 17 tokens ( 10.90 ms per token, 91.71 tokens per second) 3.15.954.810 I slot print_timing: id 3 | task 6840 | eval time = 26015.37 ms / 1163 tokens ( 22.37 ms per token, 44.70 tokens per second) 3.15.954.811 I slot print_timing: id 3 | task 6840 | total time = 26200.73 ms / 1180 tokens 3.15.954.812 I slot print_timing: id 3 | task 6840 | graphs reused = 7957 3.15.954.849 I slot release: id 3 | task 6840 | stop processing: n_tokens = 1249, truncated = 0 3.15.954.861 I srv update_slots: all slots are idle 3.15.966.855 I srv operator(): Chat format: peg-native 3.15.967.100 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.866 (> 0.100 thold), f_keep = 0.057 3.15.967.102 I srv get_availabl: updating prompt cache 3.15.967.157 W srv prompt_save: - saving prompt with length 1249, total state size = 64.508 MiB (draft: 0.000 MiB) 3.16.021.086 I srv load: - looking for better prompt, base f_keep = 0.057, sim = 0.866 3.16.021.094 I srv update: - cache state: 6 prompts, 440.089 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 3.16.021.095 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 3.16.021.096 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 3.16.021.097 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 3.16.021.097 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 3.16.021.098 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 3.16.021.098 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 3.16.021.099 I srv get_availabl: prompt cache update took 54.00 ms 3.16.021.141 I slot launch_slot_: id 3 | task 8004 | processing task, is_child = 0 3.16.021.143 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.16.021.143 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.16.021.154 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.16.021.154 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.16.021.163 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.16.021.163 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.19.180.056 I slot print_timing: id 3 | task 8004 | n_decoded = 137, tg = 45.48 t/s, tg_3s = 45.48 t/s 3.22.186.812 I slot print_timing: id 3 | task 8004 | n_decoded = 273, tg = 45.35 t/s, tg_3s = 45.23 t/s 3.25.201.385 I slot print_timing: id 3 | task 8004 | n_decoded = 410, tg = 45.38 t/s, tg_3s = 45.45 t/s 3.28.214.783 I slot print_timing: id 3 | task 8004 | n_decoded = 545, tg = 45.24 t/s, tg_3s = 44.80 t/s 3.31.222.566 I slot print_timing: id 3 | task 8004 | n_decoded = 680, tg = 45.17 t/s, tg_3s = 44.88 t/s 3.34.223.588 I slot print_timing: id 3 | task 8004 | n_decoded = 814, tg = 45.08 t/s, tg_3s = 44.65 t/s 3.37.226.627 I slot print_timing: id 3 | task 8004 | n_decoded = 948, tg = 45.02 t/s, tg_3s = 44.62 t/s 3.40.227.785 I slot print_timing: id 3 | task 8004 | n_decoded = 1082, tg = 44.97 t/s, tg_3s = 44.65 t/s 3.41.863.744 I slot print_timing: id 3 | task 8004 | prompt eval time = 146.29 ms / 11 tokens ( 13.30 ms per token, 75.19 tokens per second) 3.41.863.747 I slot print_timing: id 3 | task 8004 | eval time = 25696.27 ms / 1155 tokens ( 22.25 ms per token, 44.95 tokens per second) 3.41.863.748 I slot print_timing: id 3 | task 8004 | total time = 25842.56 ms / 1166 tokens 3.41.863.749 I slot print_timing: id 3 | task 8004 | graphs reused = 9106 3.41.863.788 I slot release: id 3 | task 8004 | stop processing: n_tokens = 1236, truncated = 0 3.41.863.801 I srv update_slots: all slots are idle 3.41.875.140 I srv operator(): Chat format: peg-native 3.41.875.365 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.845 (> 0.100 thold), f_keep = 0.057 3.41.875.367 I srv get_availabl: updating prompt cache 3.41.875.418 W srv prompt_save: - saving prompt with length 1236, total state size = 63.836 MiB (draft: 0.000 MiB) 3.41.945.128 I srv load: - looking for better prompt, base f_keep = 0.057, sim = 0.845 3.41.945.136 I srv update: - cache state: 7 prompts, 503.926 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 3.41.945.137 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 3.41.945.138 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 3.41.945.140 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 3.41.945.140 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 3.41.945.141 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 3.41.945.142 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 3.41.945.142 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 3.41.945.143 I srv get_availabl: prompt cache update took 69.78 ms 3.41.945.187 I slot launch_slot_: id 3 | task 9160 | processing task, is_child = 0 3.41.945.187 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.41.945.188 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.41.945.198 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.41.945.198 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.41.945.207 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.41.945.207 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.45.105.513 I slot print_timing: id 3 | task 9160 | n_decoded = 137, tg = 45.53 t/s, tg_3s = 45.53 t/s 3.48.125.555 I slot print_timing: id 3 | task 9160 | n_decoded = 273, tg = 45.28 t/s, tg_3s = 45.03 t/s 3.51.139.613 I slot print_timing: id 3 | task 9160 | n_decoded = 409, tg = 45.23 t/s, tg_3s = 45.12 t/s 3.54.149.800 I slot print_timing: id 3 | task 9160 | n_decoded = 543, tg = 45.05 t/s, tg_3s = 44.52 t/s 3.57.154.584 I slot print_timing: id 3 | task 9160 | n_decoded = 677, tg = 44.96 t/s, tg_3s = 44.60 t/s 4.00.156.767 I slot print_timing: id 3 | task 9160 | n_decoded = 810, tg = 44.85 t/s, tg_3s = 44.30 t/s 4.03.160.463 I slot print_timing: id 3 | task 9160 | n_decoded = 943, tg = 44.77 t/s, tg_3s = 44.28 t/s 4.06.164.703 I slot print_timing: id 3 | task 9160 | n_decoded = 1078, tg = 44.79 t/s, tg_3s = 44.94 t/s 4.09.166.624 I slot print_timing: id 3 | task 9160 | n_decoded = 1212, tg = 44.77 t/s, tg_3s = 44.64 t/s 4.11.379.350 I slot print_timing: id 3 | task 9160 | prompt eval time = 151.53 ms / 13 tokens ( 11.66 ms per token, 85.79 tokens per second) 4.11.379.354 I slot print_timing: id 3 | task 9160 | eval time = 29282.59 ms / 1311 tokens ( 22.34 ms per token, 44.77 tokens per second) 4.11.379.355 I slot print_timing: id 3 | task 9160 | total time = 29434.12 ms / 1324 tokens 4.11.379.355 I slot print_timing: id 3 | task 9160 | graphs reused = 10410 4.11.379.398 I slot release: id 3 | task 9160 | stop processing: n_tokens = 1394, truncated = 0 4.11.379.411 I srv update_slots: all slots are idle 4.11.391.171 I srv operator(): Chat format: peg-native 4.11.391.406 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.631 (> 0.100 thold), f_keep = 0.050 4.11.391.407 I srv get_availabl: updating prompt cache 4.11.391.474 W srv prompt_save: - saving prompt with length 1394, total state size = 71.997 MiB (draft: 0.000 MiB) 4.11.479.748 I srv load: - looking for better prompt, base f_keep = 0.050, sim = 0.631 4.11.479.756 I srv update: - cache state: 8 prompts, 575.922 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 4.11.479.757 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 4.11.479.758 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 4.11.479.759 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 4.11.479.759 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 4.11.479.760 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 4.11.479.761 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 4.11.479.762 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 4.11.479.763 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 4.11.479.765 I srv get_availabl: prompt cache update took 88.36 ms 4.11.479.805 I slot launch_slot_: id 3 | task 10472 | processing task, is_child = 0 4.11.479.805 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.11.479.806 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.11.479.818 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.11.479.818 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.11.479.828 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.11.479.828 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.14.720.277 I slot print_timing: id 3 | task 10472 | n_decoded = 137, tg = 45.62 t/s, tg_3s = 45.62 t/s 4.17.726.512 I slot print_timing: id 3 | task 10472 | n_decoded = 273, tg = 45.43 t/s, tg_3s = 45.24 t/s 4.20.731.297 I slot print_timing: id 3 | task 10472 | n_decoded = 409, tg = 45.37 t/s, tg_3s = 45.26 t/s 4.23.738.322 I slot print_timing: id 3 | task 10472 | n_decoded = 544, tg = 45.25 t/s, tg_3s = 44.89 t/s 4.26.759.781 I slot print_timing: id 3 | task 10472 | n_decoded = 679, tg = 45.14 t/s, tg_3s = 44.68 t/s 4.29.778.103 I slot print_timing: id 3 | task 10472 | n_decoded = 814, tg = 45.07 t/s, tg_3s = 44.73 t/s 4.32.796.815 I slot print_timing: id 3 | task 10472 | n_decoded = 948, tg = 44.97 t/s, tg_3s = 44.39 t/s 4.35.799.373 I slot print_timing: id 3 | task 10472 | n_decoded = 1082, tg = 44.93 t/s, tg_3s = 44.63 t/s 4.38.821.636 I slot print_timing: id 3 | task 10472 | n_decoded = 1216, tg = 44.86 t/s, tg_3s = 44.34 t/s 4.41.842.500 I slot print_timing: id 3 | task 10472 | n_decoded = 1350, tg = 44.81 t/s, tg_3s = 44.36 t/s 4.43.331.234 I slot print_timing: id 3 | task 10472 | prompt eval time = 237.51 ms / 41 tokens ( 5.79 ms per token, 172.63 tokens per second) 4.43.331.238 I slot print_timing: id 3 | task 10472 | eval time = 31613.88 ms / 1416 tokens ( 22.33 ms per token, 44.79 tokens per second) 4.43.331.238 I slot print_timing: id 3 | task 10472 | total time = 31851.38 ms / 1457 tokens 4.43.331.239 I slot print_timing: id 3 | task 10472 | graphs reused = 11819 4.43.331.282 I slot release: id 3 | task 10472 | stop processing: n_tokens = 1526, truncated = 0 4.43.331.300 I srv update_slots: all slots are idle 4.43.342.822 I srv operator(): Chat format: peg-native 4.43.343.061 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.550 (> 0.100 thold), f_keep = 0.047 4.43.343.062 I srv get_availabl: updating prompt cache 4.43.343.125 W srv prompt_save: - saving prompt with length 1526, total state size = 78.814 MiB (draft: 0.000 MiB) 4.43.424.267 I srv load: - looking for better prompt, base f_keep = 0.047, sim = 0.550 4.43.424.276 I srv update: - cache state: 9 prompts, 654.736 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 4.43.424.277 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 4.43.424.277 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 4.43.424.278 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 4.43.424.278 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 4.43.424.279 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 4.43.424.279 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 4.43.424.280 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 4.43.424.285 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 4.43.424.285 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 4.43.424.286 I srv get_availabl: prompt cache update took 81.22 ms 4.43.424.329 I slot launch_slot_: id 3 | task 11889 | processing task, is_child = 0 4.43.424.330 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.43.424.331 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.43.424.342 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.43.424.342 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.43.424.362 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.43.424.362 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.46.803.690 I slot print_timing: id 3 | task 11889 | n_decoded = 136, tg = 45.04 t/s, tg_3s = 45.04 t/s 4.49.814.396 I slot print_timing: id 3 | task 11889 | n_decoded = 271, tg = 44.94 t/s, tg_3s = 44.84 t/s 4.52.829.844 I slot print_timing: id 3 | task 11889 | n_decoded = 406, tg = 44.88 t/s, tg_3s = 44.77 t/s 4.55.850.051 I slot print_timing: id 3 | task 11889 | n_decoded = 541, tg = 44.84 t/s, tg_3s = 44.70 t/s 4.58.857.533 I slot print_timing: id 3 | task 11889 | n_decoded = 676, tg = 44.85 t/s, tg_3s = 44.89 t/s 5.01.866.208 I slot print_timing: id 3 | task 11889 | n_decoded = 811, tg = 44.85 t/s, tg_3s = 44.87 t/s 5.04.873.271 I slot print_timing: id 3 | task 11889 | n_decoded = 945, tg = 44.81 t/s, tg_3s = 44.56 t/s 5.07.889.287 I slot print_timing: id 3 | task 11889 | n_decoded = 1080, tg = 44.80 t/s, tg_3s = 44.76 t/s 5.10.903.442 I slot print_timing: id 3 | task 11889 | n_decoded = 1214, tg = 44.77 t/s, tg_3s = 44.46 t/s 5.13.918.840 I slot print_timing: id 3 | task 11889 | n_decoded = 1348, tg = 44.73 t/s, tg_3s = 44.44 t/s 5.16.929.906 I slot print_timing: id 3 | task 11889 | n_decoded = 1481, tg = 44.68 t/s, tg_3s = 44.17 t/s 5.19.931.612 I slot print_timing: id 3 | task 11889 | n_decoded = 1614, tg = 44.65 t/s, tg_3s = 44.31 t/s 5.22.933.674 I slot print_timing: id 3 | task 11889 | n_decoded = 1746, tg = 44.60 t/s, tg_3s = 43.97 t/s 5.23.160.348 I slot print_timing: id 3 | task 11889 | prompt eval time = 359.80 ms / 58 tokens ( 6.20 ms per token, 161.20 tokens per second) 5.23.160.352 I slot print_timing: id 3 | task 11889 | eval time = 39376.15 ms / 1756 tokens ( 22.42 ms per token, 44.60 tokens per second) 5.23.160.352 I slot print_timing: id 3 | task 11889 | total time = 39735.95 ms / 1814 tokens 5.23.160.353 I slot print_timing: id 3 | task 11889 | graphs reused = 13566 5.23.160.396 I slot release: id 3 | task 11889 | stop processing: n_tokens = 1884, truncated = 0 5.23.160.410 I srv update_slots: all slots are idle 5.23.172.078 I srv operator(): Chat format: peg-native 5.23.172.317 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.037 5.23.172.318 I srv get_availabl: updating prompt cache 5.23.172.393 W srv prompt_save: - saving prompt with length 1884, total state size = 97.304 MiB (draft: 0.000 MiB) 5.23.272.370 I srv load: - looking for better prompt, base f_keep = 0.037, sim = 0.667 5.23.272.381 I srv update: - cache state: 10 prompts, 752.040 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 5.23.272.384 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 5.23.272.385 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 5.23.272.386 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 5.23.272.388 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 5.23.272.389 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 5.23.272.390 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 5.23.272.390 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 5.23.272.391 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 5.23.272.392 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 5.23.272.392 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 5.23.272.393 I srv get_availabl: prompt cache update took 100.07 ms 5.23.272.440 I slot launch_slot_: id 3 | task 13646 | processing task, is_child = 0 5.23.272.441 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 5.23.272.442 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 5.23.272.463 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 5.23.272.463 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 5.23.272.486 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 5.23.272.486 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 5.26.489.545 I slot print_timing: id 3 | task 13646 | n_decoded = 137, tg = 45.60 t/s, tg_3s = 45.60 t/s 5.29.510.467 I slot print_timing: id 3 | task 13646 | n_decoded = 273, tg = 45.31 t/s, tg_3s = 45.02 t/s 5.32.519.042 I slot print_timing: id 3 | task 13646 | n_decoded = 409, tg = 45.27 t/s, tg_3s = 45.20 t/s 5.35.521.280 I slot print_timing: id 3 | task 13646 | n_decoded = 543, tg = 45.11 t/s, tg_3s = 44.63 t/s 5.38.522.822 I slot print_timing: id 3 | task 13646 | n_decoded = 677, tg = 45.02 t/s, tg_3s = 44.64 t/s 5.41.529.722 I slot print_timing: id 3 | task 13646 | n_decoded = 812, tg = 45.00 t/s, tg_3s = 44.90 t/s 5.44.534.997 I slot print_timing: id 3 | task 13646 | n_decoded = 946, tg = 44.94 t/s, tg_3s = 44.59 t/s 5.47.536.754 I slot print_timing: id 3 | task 13646 | n_decoded = 1081, tg = 44.95 t/s, tg_3s = 44.97 t/s 5.50.538.100 I slot print_timing: id 3 | task 13646 | n_decoded = 1215, tg = 44.91 t/s, tg_3s = 44.65 t/s 5.53.545.020 I slot print_timing: id 3 | task 13646 | n_decoded = 1349, tg = 44.88 t/s, tg_3s = 44.56 t/s 5.56.567.080 I slot print_timing: id 3 | task 13646 | n_decoded = 1483, tg = 44.83 t/s, tg_3s = 44.34 t/s 5.59.582.174 I slot print_timing: id 3 | task 13646 | n_decoded = 1617, tg = 44.80 t/s, tg_3s = 44.44 t/s 6.02.588.295 I slot print_timing: id 3 | task 13646 | n_decoded = 1750, tg = 44.75 t/s, tg_3s = 44.24 t/s 6.04.347.773 I slot print_timing: id 3 | task 13646 | prompt eval time = 212.78 ms / 35 tokens ( 6.08 ms per token, 164.49 tokens per second) 6.04.347.777 I slot print_timing: id 3 | task 13646 | eval time = 40862.48 ms / 1828 tokens ( 22.35 ms per token, 44.74 tokens per second) 6.04.347.777 I slot print_timing: id 3 | task 13646 | total time = 41075.26 ms / 1863 tokens 6.04.347.778 I slot print_timing: id 3 | task 13646 | graphs reused = 15385 6.04.347.817 I slot release: id 3 | task 13646 | stop processing: n_tokens = 1932, truncated = 0 6.04.347.832 I srv update_slots: all slots are idle 6.04.359.211 I srv operator(): Chat format: peg-native 6.04.359.443 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.583 (> 0.100 thold), f_keep = 0.036 6.04.359.444 I srv get_availabl: updating prompt cache 6.04.359.527 W srv prompt_save: - saving prompt with length 1932, total state size = 99.783 MiB (draft: 0.000 MiB) 6.04.461.867 I srv load: - looking for better prompt, base f_keep = 0.036, sim = 0.583 6.04.461.875 I srv update: - cache state: 11 prompts, 851.823 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 6.04.461.876 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 6.04.461.876 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 6.04.461.877 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 6.04.461.878 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 6.04.461.878 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 6.04.461.879 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 6.04.461.879 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 6.04.461.880 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 6.04.461.881 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 6.04.461.885 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 6.04.461.886 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 6.04.461.887 I srv get_availabl: prompt cache update took 102.44 ms 6.04.461.924 I slot launch_slot_: id 3 | task 15475 | processing task, is_child = 0 6.04.461.925 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 6.04.461.925 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 6.04.461.936 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 6.04.461.937 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 6.04.461.946 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 6.04.461.946 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 6.07.803.789 I slot print_timing: id 3 | task 15475 | n_decoded = 137, tg = 45.35 t/s, tg_3s = 45.35 t/s 6.10.815.593 I slot print_timing: id 3 | task 15475 | n_decoded = 273, tg = 45.25 t/s, tg_3s = 45.16 t/s 6.13.831.040 I slot print_timing: id 3 | task 15475 | n_decoded = 408, tg = 45.09 t/s, tg_3s = 44.77 t/s 6.16.835.006 I slot print_timing: id 3 | task 15475 | n_decoded = 542, tg = 44.97 t/s, tg_3s = 44.61 t/s 6.19.849.293 I slot print_timing: id 3 | task 15475 | n_decoded = 676, tg = 44.87 t/s, tg_3s = 44.45 t/s 6.22.869.540 I slot print_timing: id 3 | task 15475 | n_decoded = 810, tg = 44.78 t/s, tg_3s = 44.37 t/s 6.25.876.696 I slot print_timing: id 3 | task 15475 | n_decoded = 943, tg = 44.70 t/s, tg_3s = 44.23 t/s 6.27.677.400 I slot print_timing: id 3 | task 15475 | prompt eval time = 320.74 ms / 50 tokens ( 6.41 ms per token, 155.89 tokens per second) 6.27.677.403 I slot print_timing: id 3 | task 15475 | eval time = 22894.70 ms / 1023 tokens ( 22.38 ms per token, 44.68 tokens per second) 6.27.677.404 I slot print_timing: id 3 | task 15475 | total time = 23215.43 ms / 1073 tokens 6.27.677.405 I slot print_timing: id 3 | task 15475 | graphs reused = 16402 6.27.677.447 I slot release: id 3 | task 15475 | stop processing: n_tokens = 1142, truncated = 0 6.27.677.457 I srv update_slots: all slots are idle 6.27.689.146 I srv operator(): Chat format: peg-native 6.27.689.376 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.574 (> 0.100 thold), f_keep = 0.061 6.27.689.378 I srv get_availabl: updating prompt cache 6.27.689.435 W srv prompt_save: - saving prompt with length 1142, total state size = 58.982 MiB (draft: 0.000 MiB) 6.27.745.908 I srv load: - looking for better prompt, base f_keep = 0.061, sim = 0.574 6.27.745.920 I srv update: - cache state: 12 prompts, 910.805 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 6.27.745.921 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 6.27.745.922 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 6.27.745.926 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 6.27.745.927 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 6.27.745.927 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 6.27.745.928 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 6.27.745.928 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 6.27.745.929 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 6.27.745.932 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 6.27.745.933 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 6.27.745.933 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 6.27.745.934 I srv update: - prompt 0x63c2ea969620: 1142 tokens, checkpoints: 0, 58.982 MiB 6.27.745.935 I srv get_availabl: prompt cache update took 56.56 ms 6.27.745.974 I slot launch_slot_: id 3 | task 16499 | processing task, is_child = 0 6.27.745.975 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 6.27.745.975 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 6.27.745.985 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 6.27.745.986 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 6.27.745.995 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 6.27.745.995 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 6.31.081.196 I slot print_timing: id 3 | task 16499 | n_decoded = 136, tg = 45.19 t/s, tg_3s = 45.19 t/s 6.34.083.080 I slot print_timing: id 3 | task 16499 | n_decoded = 270, tg = 44.92 t/s, tg_3s = 44.64 t/s 6.37.085.339 I slot print_timing: id 3 | task 16499 | n_decoded = 404, tg = 44.82 t/s, tg_3s = 44.63 t/s 6.40.104.537 I slot print_timing: id 3 | task 16499 | n_decoded = 541, tg = 44.96 t/s, tg_3s = 45.38 t/s 6.43.118.727 I slot print_timing: id 3 | task 16499 | n_decoded = 677, tg = 44.99 t/s, tg_3s = 45.12 t/s 6.46.138.723 I slot print_timing: id 3 | task 16499 | n_decoded = 813, tg = 45.00 t/s, tg_3s = 45.03 t/s 6.49.155.336 I slot print_timing: id 3 | task 16499 | n_decoded = 948, tg = 44.96 t/s, tg_3s = 44.75 t/s 6.52.168.738 I slot print_timing: id 3 | task 16499 | n_decoded = 1083, tg = 44.94 t/s, tg_3s = 44.80 t/s 6.54.777.722 I slot print_timing: id 3 | task 16499 | prompt eval time = 325.79 ms / 52 tokens ( 6.27 ms per token, 159.61 tokens per second) 6.54.777.725 I slot print_timing: id 3 | task 16499 | eval time = 26705.92 ms / 1199 tokens ( 22.27 ms per token, 44.90 tokens per second) 6.54.777.726 I slot print_timing: id 3 | task 16499 | total time = 27031.71 ms / 1251 tokens 6.54.777.726 I slot print_timing: id 3 | task 16499 | graphs reused = 17594 6.54.777.772 I slot release: id 3 | task 16499 | stop processing: n_tokens = 1320, truncated = 0 6.54.777.785 I srv update_slots: all slots are idle 6.54.789.275 I srv operator(): Chat format: peg-native 6.54.789.502 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.651 (> 0.100 thold), f_keep = 0.054 6.54.789.504 I srv get_availabl: updating prompt cache 6.54.789.557 W srv prompt_save: - saving prompt with length 1320, total state size = 68.175 MiB (draft: 0.000 MiB) 6.54.853.468 I srv load: - looking for better prompt, base f_keep = 0.054, sim = 0.651 6.54.853.477 I srv update: - cache state: 13 prompts, 978.980 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 6.54.853.479 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 6.54.853.482 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 6.54.853.483 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 6.54.853.483 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 6.54.853.484 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 6.54.853.485 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 6.54.853.485 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 6.54.853.488 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 6.54.853.489 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 6.54.853.489 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 6.54.853.490 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 6.54.853.490 I srv update: - prompt 0x63c2ea969620: 1142 tokens, checkpoints: 0, 58.982 MiB 6.54.853.491 I srv update: - prompt 0x63c2eaf04ba0: 1320 tokens, checkpoints: 0, 68.175 MiB 6.54.853.494 I srv get_availabl: prompt cache update took 63.99 ms 6.54.853.536 I slot launch_slot_: id 3 | task 17699 | processing task, is_child = 0 6.54.853.537 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 6.54.853.538 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 6.54.853.548 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 6.54.853.548 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 6.54.853.557 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 6.54.853.557 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 6.58.068.405 I slot print_timing: id 3 | task 17699 | n_decoded = 137, tg = 45.63 t/s, tg_3s = 45.63 t/s 7.01.076.615 I slot print_timing: id 3 | task 17699 | n_decoded = 273, tg = 45.42 t/s, tg_3s = 45.21 t/s 7.04.088.770 I slot print_timing: id 3 | task 17699 | n_decoded = 409, tg = 45.33 t/s, tg_3s = 45.15 t/s 7.07.096.062 I slot print_timing: id 3 | task 17699 | n_decoded = 544, tg = 45.22 t/s, tg_3s = 44.89 t/s 7.10.097.166 I slot print_timing: id 3 | task 17699 | n_decoded = 678, tg = 45.11 t/s, tg_3s = 44.65 t/s 7.13.103.610 I slot print_timing: id 3 | task 17699 | n_decoded = 812, tg = 45.02 t/s, tg_3s = 44.57 t/s 7.16.105.204 I slot print_timing: id 3 | task 17699 | n_decoded = 945, tg = 44.92 t/s, tg_3s = 44.31 t/s 7.19.121.180 I slot print_timing: id 3 | task 17699 | n_decoded = 1079, tg = 44.85 t/s, tg_3s = 44.43 t/s 7.22.127.689 I slot print_timing: id 3 | task 17699 | n_decoded = 1212, tg = 44.79 t/s, tg_3s = 44.24 t/s 7.25.140.041 I slot print_timing: id 3 | task 17699 | n_decoded = 1346, tg = 44.76 t/s, tg_3s = 44.48 t/s 7.28.141.442 I slot print_timing: id 3 | task 17699 | n_decoded = 1479, tg = 44.72 t/s, tg_3s = 44.31 t/s 7.31.156.481 I slot print_timing: id 3 | task 17699 | n_decoded = 1613, tg = 44.69 t/s, tg_3s = 44.44 t/s 7.34.161.310 I slot print_timing: id 3 | task 17699 | n_decoded = 1746, tg = 44.66 t/s, tg_3s = 44.26 t/s 7.37.164.658 I slot print_timing: id 3 | task 17699 | n_decoded = 1879, tg = 44.63 t/s, tg_3s = 44.28 t/s 7.40.186.409 I slot print_timing: id 3 | task 17699 | n_decoded = 2012, tg = 44.59 t/s, tg_3s = 44.01 t/s 7.43.198.444 I slot print_timing: id 3 | task 17699 | n_decoded = 2144, tg = 44.54 t/s, tg_3s = 43.82 t/s 7.46.209.542 I slot print_timing: id 3 | task 17699 | n_decoded = 2275, tg = 44.48 t/s, tg_3s = 43.51 t/s 7.49.231.761 I slot print_timing: id 3 | task 17699 | n_decoded = 2407, tg = 44.44 t/s, tg_3s = 43.68 t/s 7.52.251.517 I slot print_timing: id 3 | task 17699 | n_decoded = 2538, tg = 44.38 t/s, tg_3s = 43.38 t/s 7.55.264.540 I slot print_timing: id 3 | task 17699 | n_decoded = 2669, tg = 44.34 t/s, tg_3s = 43.48 t/s 7.58.268.320 I slot print_timing: id 3 | task 17699 | n_decoded = 2798, tg = 44.27 t/s, tg_3s = 42.95 t/s 8.01.268.838 I slot print_timing: id 3 | task 17699 | n_decoded = 2927, tg = 44.21 t/s, tg_3s = 42.99 t/s 8.04.271.082 I slot print_timing: id 3 | task 17699 | n_decoded = 3050, tg = 44.07 t/s, tg_3s = 40.97 t/s 8.07.291.459 I slot print_timing: id 3 | task 17699 | n_decoded = 3172, tg = 43.92 t/s, tg_3s = 40.39 t/s 8.09.684.188 I slot print_timing: id 3 | task 17699 | prompt eval time = 212.27 ms / 38 tokens ( 5.59 ms per token, 179.02 tokens per second) 8.09.684.191 I slot print_timing: id 3 | task 17699 | eval time = 74618.34 ms / 3268 tokens ( 22.83 ms per token, 43.80 tokens per second) 8.09.684.191 I slot print_timing: id 3 | task 17699 | total time = 74830.61 ms / 3306 tokens 8.09.684.192 I slot print_timing: id 3 | task 17699 | graphs reused = 20847 8.09.684.229 I slot release: id 3 | task 17699 | stop processing: n_tokens = 3376, truncated = 0 8.09.684.247 I srv update_slots: all slots are idle 8.09.695.719 I srv operator(): Chat format: peg-native 8.09.695.948 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.609 (> 0.100 thold), f_keep = 0.021 8.09.695.950 I srv get_availabl: updating prompt cache 8.09.696.077 W srv prompt_save: - saving prompt with length 3376, total state size = 174.361 MiB (draft: 0.000 MiB) 8.09.855.089 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.609 8.09.855.100 I srv update: - cache state: 14 prompts, 1153.341 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 8.09.855.101 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 8.09.855.102 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 8.09.855.103 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 8.09.855.103 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 8.09.855.104 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 8.09.855.105 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 8.09.855.106 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 8.09.855.106 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 8.09.855.107 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 8.09.855.107 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 8.09.855.108 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 8.09.855.108 I srv update: - prompt 0x63c2ea969620: 1142 tokens, checkpoints: 0, 58.982 MiB 8.09.855.109 I srv update: - prompt 0x63c2eaf04ba0: 1320 tokens, checkpoints: 0, 68.175 MiB 8.09.855.109 I srv update: - prompt 0x63c2f1f09fb0: 3376 tokens, checkpoints: 0, 174.361 MiB 8.09.855.110 I srv get_availabl: prompt cache update took 159.16 ms 8.09.855.152 I slot launch_slot_: id 3 | task 20968 | processing task, is_child = 0 8.09.855.152 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 8.09.855.153 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 8.09.855.168 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 8.09.855.168 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 8.09.855.178 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 8.09.855.178 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 8.13.102.368 I slot print_timing: id 3 | task 20968 | n_decoded = 136, tg = 45.30 t/s, tg_3s = 45.30 t/s 8.16.117.874 I slot print_timing: id 3 | task 20968 | n_decoded = 271, tg = 45.03 t/s, tg_3s = 44.77 t/s 8.19.131.574 I slot print_timing: id 3 | task 20968 | n_decoded = 407, tg = 45.07 t/s, tg_3s = 45.13 t/s 8.22.132.583 I slot print_timing: id 3 | task 20968 | n_decoded = 543, tg = 45.13 t/s, tg_3s = 45.32 t/s 8.25.143.932 I slot print_timing: id 3 | task 20968 | n_decoded = 679, tg = 45.14 t/s, tg_3s = 45.16 t/s 8.28.146.616 I slot print_timing: id 3 | task 20968 | n_decoded = 814, tg = 45.11 t/s, tg_3s = 44.96 t/s 8.31.168.006 I slot print_timing: id 3 | task 20968 | n_decoded = 949, tg = 45.05 t/s, tg_3s = 44.68 t/s 8.34.183.068 I slot print_timing: id 3 | task 20968 | n_decoded = 1084, tg = 45.01 t/s, tg_3s = 44.78 t/s 8.37.196.676 I slot print_timing: id 3 | task 20968 | n_decoded = 1218, tg = 44.95 t/s, tg_3s = 44.46 t/s 8.40.211.718 I slot print_timing: id 3 | task 20968 | n_decoded = 1352, tg = 44.90 t/s, tg_3s = 44.44 t/s 8.43.220.848 I slot print_timing: id 3 | task 20968 | n_decoded = 1485, tg = 44.84 t/s, tg_3s = 44.20 t/s 8.44.061.677 I slot print_timing: id 3 | task 20968 | prompt eval time = 245.03 ms / 45 tokens ( 5.45 ms per token, 183.65 tokens per second) 8.44.061.680 I slot print_timing: id 3 | task 20968 | eval time = 33961.45 ms / 1522 tokens ( 22.31 ms per token, 44.82 tokens per second) 8.44.061.681 I slot print_timing: id 3 | task 20968 | total time = 34206.48 ms / 1567 tokens 8.44.061.682 I slot print_timing: id 3 | task 20968 | graphs reused = 22361 8.44.061.727 I slot release: id 3 | task 20968 | stop processing: n_tokens = 1636, truncated = 0 8.44.061.741 I srv update_slots: all slots are idle 8.44.073.144 I srv operator(): Chat format: peg-native 8.44.073.372 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.673 (> 0.100 thold), f_keep = 0.043 8.44.073.373 I srv get_availabl: updating prompt cache 8.44.073.438 W srv prompt_save: - saving prompt with length 1636, total state size = 84.495 MiB (draft: 0.000 MiB) 8.44.147.491 I srv load: - looking for better prompt, base f_keep = 0.043, sim = 0.673 8.44.147.502 I srv update: - cache state: 15 prompts, 1237.837 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 8.44.147.506 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 8.44.147.507 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 8.44.147.507 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 8.44.147.508 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 8.44.147.508 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 8.44.147.511 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 8.44.147.512 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 8.44.147.512 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 8.44.147.513 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 8.44.147.513 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 8.44.147.514 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 8.44.147.515 I srv update: - prompt 0x63c2ea969620: 1142 tokens, checkpoints: 0, 58.982 MiB 8.44.147.516 I srv update: - prompt 0x63c2eaf04ba0: 1320 tokens, checkpoints: 0, 68.175 MiB 8.44.147.516 I srv update: - prompt 0x63c2f1f09fb0: 3376 tokens, checkpoints: 0, 174.361 MiB 8.44.147.517 I srv update: - prompt 0x63c2ea962870: 1636 tokens, checkpoints: 0, 84.495 MiB 8.44.147.517 I srv get_availabl: prompt cache update took 74.14 ms 8.44.147.556 I slot launch_slot_: id 3 | task 22491 | processing task, is_child = 0 8.44.147.557 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 8.44.147.558 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 8.44.147.568 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 8.44.147.569 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 8.44.147.589 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 8.44.147.589 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 8.47.362.420 I slot print_timing: id 3 | task 22491 | n_decoded = 137, tg = 45.54 t/s, tg_3s = 45.54 t/s 8.50.377.002 I slot print_timing: id 3 | task 22491 | n_decoded = 272, tg = 45.16 t/s, tg_3s = 44.78 t/s 8.53.386.920 I slot print_timing: id 3 | task 22491 | n_decoded = 407, tg = 45.06 t/s, tg_3s = 44.85 t/s 8.56.401.774 I slot print_timing: id 3 | task 22491 | n_decoded = 541, tg = 44.90 t/s, tg_3s = 44.45 t/s 8.59.419.611 I slot print_timing: id 3 | task 22491 | n_decoded = 675, tg = 44.80 t/s, tg_3s = 44.40 t/s 9.02.434.596 I slot print_timing: id 3 | task 22491 | n_decoded = 809, tg = 44.74 t/s, tg_3s = 44.44 t/s 9.05.441.327 I slot print_timing: id 3 | task 22491 | n_decoded = 943, tg = 44.72 t/s, tg_3s = 44.57 t/s 9.08.442.352 I slot print_timing: id 3 | task 22491 | n_decoded = 1077, tg = 44.71 t/s, tg_3s = 44.65 t/s 9.11.464.459 I slot print_timing: id 3 | task 22491 | n_decoded = 1211, tg = 44.67 t/s, tg_3s = 44.34 t/s 9.14.479.576 I slot print_timing: id 3 | task 22491 | n_decoded = 1345, tg = 44.65 t/s, tg_3s = 44.44 t/s 9.17.482.300 I slot print_timing: id 3 | task 22491 | n_decoded = 1478, tg = 44.61 t/s, tg_3s = 44.29 t/s 9.20.501.874 I slot print_timing: id 3 | task 22491 | n_decoded = 1612, tg = 44.59 t/s, tg_3s = 44.38 t/s 9.23.522.661 I slot print_timing: id 3 | task 22491 | n_decoded = 1745, tg = 44.55 t/s, tg_3s = 44.03 t/s 9.26.539.938 I slot print_timing: id 3 | task 22491 | n_decoded = 1878, tg = 44.52 t/s, tg_3s = 44.08 t/s 9.29.539.985 I slot print_timing: id 3 | task 22491 | n_decoded = 2009, tg = 44.46 t/s, tg_3s = 43.67 t/s 9.32.559.761 I slot print_timing: id 3 | task 22491 | n_decoded = 2141, tg = 44.41 t/s, tg_3s = 43.71 t/s 9.35.578.171 I slot print_timing: id 3 | task 22491 | n_decoded = 2272, tg = 44.35 t/s, tg_3s = 43.40 t/s 9.38.597.335 I slot print_timing: id 3 | task 22491 | n_decoded = 2403, tg = 44.30 t/s, tg_3s = 43.39 t/s 9.41.323.499 I slot print_timing: id 3 | task 22491 | prompt eval time = 206.34 ms / 34 tokens ( 6.07 ms per token, 164.77 tokens per second) 9.41.323.503 I slot print_timing: id 3 | task 22491 | eval time = 56969.53 ms / 2521 tokens ( 22.60 ms per token, 44.25 tokens per second) 9.41.323.503 I slot print_timing: id 3 | task 22491 | total time = 57175.88 ms / 2555 tokens 9.41.323.504 I slot print_timing: id 3 | task 22491 | graphs reused = 24870 9.41.323.543 I slot release: id 3 | task 22491 | stop processing: n_tokens = 2624, truncated = 0 9.41.323.559 I srv update_slots: all slots are idle 9.41.334.934 I srv operator(): Chat format: peg-native 9.41.335.173 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.660 (> 0.100 thold), f_keep = 0.027 9.41.335.175 I srv get_availabl: updating prompt cache 9.41.335.276 W srv prompt_save: - saving prompt with length 2624, total state size = 135.523 MiB (draft: 0.000 MiB) 9.41.451.753 I srv load: - looking for better prompt, base f_keep = 0.027, sim = 0.660 9.41.451.764 I srv update: - cache state: 16 prompts, 1373.359 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 9.41.451.768 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 9.41.451.769 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 9.41.451.769 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 9.41.451.770 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 9.41.451.773 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 9.41.451.773 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 9.41.451.774 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 9.41.451.774 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 9.41.451.775 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 9.41.451.775 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 9.41.451.778 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 9.41.451.778 I srv update: - prompt 0x63c2ea969620: 1142 tokens, checkpoints: 0, 58.982 MiB 9.41.451.779 I srv update: - prompt 0x63c2eaf04ba0: 1320 tokens, checkpoints: 0, 68.175 MiB 9.41.451.779 I srv update: - prompt 0x63c2f1f09fb0: 3376 tokens, checkpoints: 0, 174.361 MiB 9.41.451.780 I srv update: - prompt 0x63c2ea962870: 1636 tokens, checkpoints: 0, 84.495 MiB 9.41.451.780 I srv update: - prompt 0x63c2f1efda40: 2624 tokens, checkpoints: 0, 135.523 MiB 9.41.451.782 I srv get_availabl: prompt cache update took 116.61 ms 9.41.451.824 I slot launch_slot_: id 3 | task 25013 | processing task, is_child = 0 9.41.451.824 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 9.41.451.825 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 9.41.451.837 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 9.41.451.837 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 9.41.451.857 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 9.41.451.857 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 9.44.678.427 I slot print_timing: id 3 | task 25013 | n_decoded = 137, tg = 45.43 t/s, tg_3s = 45.43 t/s 9.47.695.749 I slot print_timing: id 3 | task 25013 | n_decoded = 272, tg = 45.08 t/s, tg_3s = 44.74 t/s 9.50.697.122 I slot print_timing: id 3 | task 25013 | n_decoded = 407, tg = 45.05 t/s, tg_3s = 44.98 t/s 9.53.709.906 I slot print_timing: id 3 | task 25013 | n_decoded = 541, tg = 44.91 t/s, tg_3s = 44.48 t/s 9.56.722.593 I slot print_timing: id 3 | task 25013 | n_decoded = 675, tg = 44.82 t/s, tg_3s = 44.48 t/s 9.59.727.192 I slot print_timing: id 3 | task 25013 | n_decoded = 810, tg = 44.84 t/s, tg_3s = 44.93 t/s 10.02.747.893 I slot print_timing: id 3 | task 25013 | n_decoded = 945, tg = 44.82 t/s, tg_3s = 44.69 t/s 10.05.764.949 I slot print_timing: id 3 | task 25013 | n_decoded = 1080, tg = 44.81 t/s, tg_3s = 44.75 t/s 10.08.776.572 I slot print_timing: id 3 | task 25013 | n_decoded = 1214, tg = 44.77 t/s, tg_3s = 44.49 t/s 10.11.790.641 I slot print_timing: id 3 | task 25013 | n_decoded = 1348, tg = 44.74 t/s, tg_3s = 44.46 t/s 10.14.804.020 I slot print_timing: id 3 | task 25013 | n_decoded = 1481, tg = 44.69 t/s, tg_3s = 44.14 t/s 10.17.818.030 I slot print_timing: id 3 | task 25013 | n_decoded = 1614, tg = 44.64 t/s, tg_3s = 44.13 t/s 10.20.824.058 I slot print_timing: id 3 | task 25013 | n_decoded = 1746, tg = 44.58 t/s, tg_3s = 43.91 t/s 10.23.844.232 I slot print_timing: id 3 | task 25013 | n_decoded = 1879, tg = 44.55 t/s, tg_3s = 44.04 t/s 10.26.847.149 I slot print_timing: id 3 | task 25013 | n_decoded = 2010, tg = 44.48 t/s, tg_3s = 43.62 t/s 10.29.864.191 I slot print_timing: id 3 | task 25013 | n_decoded = 2141, tg = 44.42 t/s, tg_3s = 43.42 t/s 10.32.867.384 I slot print_timing: id 3 | task 25013 | n_decoded = 2271, tg = 44.35 t/s, tg_3s = 43.29 t/s 10.35.880.543 I slot print_timing: id 3 | task 25013 | n_decoded = 2402, tg = 44.30 t/s, tg_3s = 43.48 t/s 10.38.881.749 I slot print_timing: id 3 | task 25013 | n_decoded = 2532, tg = 44.25 t/s, tg_3s = 43.32 t/s 10.41.896.781 I slot print_timing: id 3 | task 25013 | n_decoded = 2663, tg = 44.21 t/s, tg_3s = 43.45 t/s 10.43.681.533 I slot print_timing: id 3 | task 25013 | prompt eval time = 210.68 ms / 36 tokens ( 5.85 ms per token, 170.88 tokens per second) 10.43.681.537 I slot print_timing: id 3 | task 25013 | eval time = 62018.97 ms / 2740 tokens ( 22.63 ms per token, 44.18 tokens per second) 10.43.681.538 I slot print_timing: id 3 | task 25013 | total time = 62229.64 ms / 2776 tokens 10.43.681.538 I slot print_timing: id 3 | task 25013 | graphs reused = 27597 10.43.681.578 I slot release: id 3 | task 25013 | stop processing: n_tokens = 2845, truncated = 0 10.43.681.595 I srv update_slots: all slots are idle 10.43.693.143 I srv operator(): Chat format: peg-native 10.43.693.380 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.729 (> 0.100 thold), f_keep = 0.025 10.43.693.382 I srv get_availabl: updating prompt cache 10.43.693.479 W srv prompt_save: - saving prompt with length 2845, total state size = 146.937 MiB (draft: 0.000 MiB) 10.43.821.188 I srv load: - looking for better prompt, base f_keep = 0.025, sim = 0.729 10.43.821.199 I srv update: - cache state: 17 prompts, 1520.296 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 10.43.821.200 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 10.43.821.200 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 10.43.821.201 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 10.43.821.202 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 10.43.821.206 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 10.43.821.207 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 10.43.821.207 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 10.43.821.208 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 10.43.821.208 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 10.43.821.209 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 10.43.821.209 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 10.43.821.212 I srv update: - prompt 0x63c2ea969620: 1142 tokens, checkpoints: 0, 58.982 MiB 10.43.821.213 I srv update: - prompt 0x63c2eaf04ba0: 1320 tokens, checkpoints: 0, 68.175 MiB 10.43.821.213 I srv update: - prompt 0x63c2f1f09fb0: 3376 tokens, checkpoints: 0, 174.361 MiB 10.43.821.214 I srv update: - prompt 0x63c2ea962870: 1636 tokens, checkpoints: 0, 84.495 MiB 10.43.821.214 I srv update: - prompt 0x63c2f1efda40: 2624 tokens, checkpoints: 0, 135.523 MiB 10.43.821.215 I srv update: - prompt 0x63c2f1e7c4d0: 2845 tokens, checkpoints: 0, 146.937 MiB 10.43.821.216 I srv get_availabl: prompt cache update took 127.83 ms 10.43.821.256 I slot launch_slot_: id 3 | task 27754 | processing task, is_child = 0 10.43.821.256 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 10.43.821.257 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 10.43.821.269 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 10.43.821.270 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 10.43.821.289 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 10.43.821.289 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 10.47.069.654 I slot print_timing: id 3 | task 27754 | n_decoded = 137, tg = 45.59 t/s, tg_3s = 45.59 t/s 10.50.076.799 I slot print_timing: id 3 | task 27754 | n_decoded = 273, tg = 45.41 t/s, tg_3s = 45.23 t/s 10.53.090.861 I slot print_timing: id 3 | task 27754 | n_decoded = 410, tg = 45.42 t/s, tg_3s = 45.45 t/s 10.56.096.112 I slot print_timing: id 3 | task 27754 | n_decoded = 545, tg = 45.30 t/s, tg_3s = 44.92 t/s 10.59.105.403 I slot print_timing: id 3 | task 27754 | n_decoded = 680, tg = 45.21 t/s, tg_3s = 44.86 t/s 11.02.117.969 I slot print_timing: id 3 | task 27754 | n_decoded = 815, tg = 45.14 t/s, tg_3s = 44.81 t/s 11.05.129.156 I slot print_timing: id 3 | task 27754 | n_decoded = 949, tg = 45.05 t/s, tg_3s = 44.50 t/s 11.08.129.194 I slot print_timing: id 3 | task 27754 | n_decoded = 1083, tg = 45.00 t/s, tg_3s = 44.67 t/s 11.11.135.956 I slot print_timing: id 3 | task 27754 | n_decoded = 1216, tg = 44.92 t/s, tg_3s = 44.23 t/s 11.14.157.284 I slot print_timing: id 3 | task 27754 | n_decoded = 1350, tg = 44.86 t/s, tg_3s = 44.35 t/s 11.17.172.466 I slot print_timing: id 3 | task 27754 | n_decoded = 1483, tg = 44.79 t/s, tg_3s = 44.11 t/s 11.17.851.371 I slot print_timing: id 3 | task 27754 | prompt eval time = 243.06 ms / 26 tokens ( 9.35 ms per token, 106.97 tokens per second) 11.17.851.374 I slot print_timing: id 3 | task 27754 | eval time = 33786.99 ms / 1513 tokens ( 22.33 ms per token, 44.78 tokens per second) 11.17.851.375 I slot print_timing: id 3 | task 27754 | total time = 34030.05 ms / 1539 tokens 11.17.851.376 I slot print_timing: id 3 | task 27754 | graphs reused = 29102 11.17.851.422 I slot release: id 3 | task 27754 | stop processing: n_tokens = 1608, truncated = 0 11.17.851.435 I srv update_slots: all slots are idle 11.17.862.796 I srv operator(): Chat format: peg-native 11.17.863.032 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.795 (> 0.100 thold), f_keep = 0.044 11.17.863.034 I srv get_availabl: updating prompt cache 11.17.863.095 W srv prompt_save: - saving prompt with length 1608, total state size = 83.049 MiB (draft: 0.000 MiB) 11.17.923.459 I srv load: - looking for better prompt, base f_keep = 0.044, sim = 0.795 11.17.923.472 I srv update: - cache state: 18 prompts, 1603.345 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 11.17.923.473 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 11.17.923.474 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 11.17.923.475 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 11.17.923.475 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 11.17.923.476 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 11.17.923.476 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 11.17.923.479 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 11.17.923.480 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 11.17.923.480 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 11.17.923.481 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 11.17.923.481 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 11.17.923.481 I srv update: - prompt 0x63c2ea969620: 1142 tokens, checkpoints: 0, 58.982 MiB 11.17.923.483 I srv update: - prompt 0x63c2eaf04ba0: 1320 tokens, checkpoints: 0, 68.175 MiB 11.17.923.484 I srv update: - prompt 0x63c2f1f09fb0: 3376 tokens, checkpoints: 0, 174.361 MiB 11.17.923.484 I srv update: - prompt 0x63c2ea962870: 1636 tokens, checkpoints: 0, 84.495 MiB 11.17.923.485 I srv update: - prompt 0x63c2f1efda40: 2624 tokens, checkpoints: 0, 135.523 MiB 11.17.923.485 I srv update: - prompt 0x63c2f1e7c4d0: 2845 tokens, checkpoints: 0, 146.937 MiB 11.17.923.486 I srv update: - prompt 0x63c2eaaf4520: 1608 tokens, checkpoints: 0, 83.049 MiB 11.17.923.487 I srv get_availabl: prompt cache update took 60.45 ms 11.17.923.530 I slot launch_slot_: id 3 | task 29268 | processing task, is_child = 0 11.17.923.531 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 11.17.923.532 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 11.17.923.542 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 11.17.923.543 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 11.17.923.552 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 11.17.923.552 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 11.21.123.021 I slot print_timing: id 3 | task 29268 | n_decoded = 137, tg = 45.62 t/s, tg_3s = 45.62 t/s 11.24.129.795 I slot print_timing: id 3 | task 29268 | n_decoded = 272, tg = 45.26 t/s, tg_3s = 44.90 t/s 11.27.134.464 I slot print_timing: id 3 | task 29268 | n_decoded = 407, tg = 45.15 t/s, tg_3s = 44.93 t/s 11.30.144.523 I slot print_timing: id 3 | task 29268 | n_decoded = 541, tg = 44.99 t/s, tg_3s = 44.52 t/s 11.33.153.790 I slot print_timing: id 3 | task 29268 | n_decoded = 675, tg = 44.90 t/s, tg_3s = 44.53 t/s 11.33.370.337 I slot print_timing: id 3 | task 29268 | prompt eval time = 196.05 ms / 18 tokens ( 10.89 ms per token, 91.81 tokens per second) 11.33.370.342 I slot print_timing: id 3 | task 29268 | eval time = 15250.71 ms / 684 tokens ( 22.30 ms per token, 44.85 tokens per second) 11.33.370.342 I slot print_timing: id 3 | task 29268 | total time = 15446.76 ms / 702 tokens 11.33.370.343 I slot print_timing: id 3 | task 29268 | graphs reused = 29781 11.33.370.383 I slot release: id 3 | task 29268 | stop processing: n_tokens = 771, truncated = 0 11.33.370.394 I srv update_slots: all slots are idle 11.33.381.781 I srv operator(): Chat format: peg-native 11.33.382.029 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.654 (> 0.100 thold), f_keep = 0.091 11.33.382.031 I srv get_availabl: updating prompt cache 11.33.382.083 W srv prompt_save: - saving prompt with length 771, total state size = 39.821 MiB (draft: 0.000 MiB) 11.33.412.028 I srv load: - looking for better prompt, base f_keep = 0.091, sim = 0.654 11.33.412.043 I srv update: - cache state: 19 prompts, 1643.166 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 11.33.412.043 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 11.33.412.044 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 11.33.412.045 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 11.33.412.045 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 11.33.412.046 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 11.33.412.047 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 11.33.412.047 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 11.33.412.048 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 11.33.412.052 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 11.33.412.053 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 11.33.412.053 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 11.33.412.054 I srv update: - prompt 0x63c2ea969620: 1142 tokens, checkpoints: 0, 58.982 MiB 11.33.412.054 I srv update: - prompt 0x63c2eaf04ba0: 1320 tokens, checkpoints: 0, 68.175 MiB 11.33.412.055 I srv update: - prompt 0x63c2f1f09fb0: 3376 tokens, checkpoints: 0, 174.361 MiB 11.33.412.055 I srv update: - prompt 0x63c2ea962870: 1636 tokens, checkpoints: 0, 84.495 MiB 11.33.412.058 I srv update: - prompt 0x63c2f1efda40: 2624 tokens, checkpoints: 0, 135.523 MiB 11.33.412.059 I srv update: - prompt 0x63c2f1e7c4d0: 2845 tokens, checkpoints: 0, 146.937 MiB 11.33.412.059 I srv update: - prompt 0x63c2eaaf4520: 1608 tokens, checkpoints: 0, 83.049 MiB 11.33.412.060 I srv update: - prompt 0x63c2f1e62ea0: 771 tokens, checkpoints: 0, 39.821 MiB 11.33.412.061 I srv get_availabl: prompt cache update took 30.03 ms 11.33.412.121 I slot launch_slot_: id 3 | task 29953 | processing task, is_child = 0 11.33.412.122 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 11.33.412.123 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 11.33.412.135 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 11.33.412.136 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 11.33.412.145 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 11.33.412.145 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 11.36.637.324 I slot print_timing: id 3 | task 29953 | n_decoded = 136, tg = 45.23 t/s, tg_3s = 45.23 t/s 11.39.642.338 I slot print_timing: id 3 | task 29953 | n_decoded = 272, tg = 45.24 t/s, tg_3s = 45.26 t/s 11.42.648.871 I slot print_timing: id 3 | task 29953 | n_decoded = 408, tg = 45.24 t/s, tg_3s = 45.23 t/s 11.45.651.502 I slot print_timing: id 3 | task 29953 | n_decoded = 543, tg = 45.17 t/s, tg_3s = 44.96 t/s 11.48.667.889 I slot print_timing: id 3 | task 29953 | n_decoded = 678, tg = 45.09 t/s, tg_3s = 44.76 t/s 11.51.681.237 I slot print_timing: id 3 | task 29953 | n_decoded = 813, tg = 45.04 t/s, tg_3s = 44.80 t/s 11.54.692.970 I slot print_timing: id 3 | task 29953 | n_decoded = 947, tg = 44.96 t/s, tg_3s = 44.49 t/s 11.57.693.245 I slot print_timing: id 3 | task 29953 | n_decoded = 1081, tg = 44.92 t/s, tg_3s = 44.66 t/s 11.58.790.061 I slot print_timing: id 3 | task 29953 | prompt eval time = 218.40 ms / 37 tokens ( 5.90 ms per token, 169.42 tokens per second) 11.58.790.064 I slot print_timing: id 3 | task 29953 | eval time = 25159.50 ms / 1130 tokens ( 22.27 ms per token, 44.91 tokens per second) 11.58.790.065 I slot print_timing: id 3 | task 29953 | total time = 25377.89 ms / 1167 tokens 11.58.790.065 I slot print_timing: id 3 | task 29953 | graphs reused = 30905 11.58.790.109 I slot release: id 3 | task 29953 | stop processing: n_tokens = 1236, truncated = 0 11.58.790.122 I srv update_slots: all slots are idle 11.58.801.209 I srv operator(): Chat format: peg-native 11.58.801.446 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.729 (> 0.100 thold), f_keep = 0.057 11.58.801.448 I srv get_availabl: updating prompt cache 11.58.801.502 W srv prompt_save: - saving prompt with length 1236, total state size = 63.836 MiB (draft: 0.000 MiB) 11.58.853.165 I srv load: - looking for better prompt, base f_keep = 0.057, sim = 0.729 11.58.853.176 I srv update: - cache state: 20 prompts, 1707.002 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 11.58.853.178 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 11.58.853.179 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 11.58.853.182 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 11.58.853.183 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 11.58.853.183 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 11.58.853.184 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 11.58.853.185 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 11.58.853.185 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 11.58.853.188 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 11.58.853.188 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 11.58.853.189 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 11.58.853.190 I srv update: - prompt 0x63c2ea969620: 1142 tokens, checkpoints: 0, 58.982 MiB 11.58.853.190 I srv update: - prompt 0x63c2eaf04ba0: 1320 tokens, checkpoints: 0, 68.175 MiB 11.58.853.193 I srv update: - prompt 0x63c2f1f09fb0: 3376 tokens, checkpoints: 0, 174.361 MiB 11.58.853.194 I srv update: - prompt 0x63c2ea962870: 1636 tokens, checkpoints: 0, 84.495 MiB 11.58.853.194 I srv update: - prompt 0x63c2f1efda40: 2624 tokens, checkpoints: 0, 135.523 MiB 11.58.853.195 I srv update: - prompt 0x63c2f1e7c4d0: 2845 tokens, checkpoints: 0, 146.937 MiB 11.58.853.195 I srv update: - prompt 0x63c2eaaf4520: 1608 tokens, checkpoints: 0, 83.049 MiB 11.58.853.197 I srv update: - prompt 0x63c2f1e62ea0: 771 tokens, checkpoints: 0, 39.821 MiB 11.58.853.198 I srv update: - prompt 0x63c2eaa9f9b0: 1236 tokens, checkpoints: 0, 63.836 MiB 11.58.853.199 I srv get_availabl: prompt cache update took 51.75 ms 11.58.853.238 I slot launch_slot_: id 3 | task 31084 | processing task, is_child = 0 11.58.853.239 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 11.58.853.240 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 11.58.853.250 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 11.58.853.250 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 11.58.853.259 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 11.58.853.259 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 12.02.110.072 I slot print_timing: id 3 | task 31084 | n_decoded = 137, tg = 45.42 t/s, tg_3s = 45.42 t/s 12.05.131.116 I slot print_timing: id 3 | task 31084 | n_decoded = 273, tg = 45.22 t/s, tg_3s = 45.02 t/s 12.08.145.404 I slot print_timing: id 3 | task 31084 | n_decoded = 409, tg = 45.18 t/s, tg_3s = 45.12 t/s 12.11.150.566 I slot print_timing: id 3 | task 31084 | n_decoded = 543, tg = 45.04 t/s, tg_3s = 44.59 t/s 12.14.157.381 I slot print_timing: id 3 | task 31084 | n_decoded = 677, tg = 44.94 t/s, tg_3s = 44.57 t/s 12.17.173.934 I slot print_timing: id 3 | task 31084 | n_decoded = 811, tg = 44.86 t/s, tg_3s = 44.42 t/s 12.20.179.147 I slot print_timing: id 3 | task 31084 | n_decoded = 944, tg = 44.77 t/s, tg_3s = 44.26 t/s 12.23.181.754 I slot print_timing: id 3 | task 31084 | n_decoded = 1077, tg = 44.71 t/s, tg_3s = 44.29 t/s 12.25.119.972 I slot print_timing: id 3 | task 31084 | prompt eval time = 240.22 ms / 26 tokens ( 9.24 ms per token, 108.24 tokens per second) 12.25.119.976 I slot print_timing: id 3 | task 31084 | eval time = 26026.48 ms / 1164 tokens ( 22.36 ms per token, 44.72 tokens per second) 12.25.119.976 I slot print_timing: id 3 | task 31084 | total time = 26266.69 ms / 1190 tokens 12.25.119.977 I slot print_timing: id 3 | task 31084 | graphs reused = 32063 12.25.120.029 I slot release: id 3 | task 31084 | stop processing: n_tokens = 1259, truncated = 0 12.25.120.041 I srv update_slots: all slots are idle 12.25.131.527 I srv operator(): Chat format: peg-native 12.25.131.765 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.693 (> 0.100 thold), f_keep = 0.056 12.25.131.766 I srv get_availabl: updating prompt cache 12.25.131.823 W srv prompt_save: - saving prompt with length 1259, total state size = 65.024 MiB (draft: 0.000 MiB) 12.25.185.900 I srv load: - looking for better prompt, base f_keep = 0.056, sim = 0.693 12.25.185.913 I srv update: - cache state: 21 prompts, 1772.027 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 12.25.185.915 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 12.25.185.915 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 12.25.185.916 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 12.25.185.916 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 12.25.185.917 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 12.25.185.917 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 12.25.185.918 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 12.25.185.921 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 12.25.185.921 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 12.25.185.922 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 12.25.185.922 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 12.25.185.923 I srv update: - prompt 0x63c2ea969620: 1142 tokens, checkpoints: 0, 58.982 MiB 12.25.185.923 I srv update: - prompt 0x63c2eaf04ba0: 1320 tokens, checkpoints: 0, 68.175 MiB 12.25.185.924 I srv update: - prompt 0x63c2f1f09fb0: 3376 tokens, checkpoints: 0, 174.361 MiB 12.25.185.925 I srv update: - prompt 0x63c2ea962870: 1636 tokens, checkpoints: 0, 84.495 MiB 12.25.185.926 I srv update: - prompt 0x63c2f1efda40: 2624 tokens, checkpoints: 0, 135.523 MiB 12.25.185.926 I srv update: - prompt 0x63c2f1e7c4d0: 2845 tokens, checkpoints: 0, 146.937 MiB 12.25.185.927 I srv update: - prompt 0x63c2eaaf4520: 1608 tokens, checkpoints: 0, 83.049 MiB 12.25.185.928 I srv update: - prompt 0x63c2f1e62ea0: 771 tokens, checkpoints: 0, 39.821 MiB 12.25.185.928 I srv update: - prompt 0x63c2eaa9f9b0: 1236 tokens, checkpoints: 0, 63.836 MiB 12.25.185.928 I srv update: - prompt 0x63c2f1dc6ec0: 1259 tokens, checkpoints: 0, 65.024 MiB 12.25.185.930 I srv get_availabl: prompt cache update took 54.16 ms 12.25.185.975 I slot launch_slot_: id 3 | task 32249 | processing task, is_child = 0 12.25.185.976 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 12.25.185.976 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 12.25.185.990 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 12.25.185.990 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 12.25.185.999 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 12.25.185.999 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 12.28.441.788 I slot print_timing: id 3 | task 32249 | n_decoded = 137, tg = 45.58 t/s, tg_3s = 45.58 t/s 12.31.451.825 I slot print_timing: id 3 | task 32249 | n_decoded = 273, tg = 45.38 t/s, tg_3s = 45.18 t/s 12.34.467.842 I slot print_timing: id 3 | task 32249 | n_decoded = 410, tg = 45.39 t/s, tg_3s = 45.42 t/s 12.37.480.026 I slot print_timing: id 3 | task 32249 | n_decoded = 545, tg = 45.25 t/s, tg_3s = 44.82 t/s 12.40.495.761 I slot print_timing: id 3 | task 32249 | n_decoded = 680, tg = 45.15 t/s, tg_3s = 44.77 t/s 12.43.515.314 I slot print_timing: id 3 | task 32249 | n_decoded = 815, tg = 45.08 t/s, tg_3s = 44.71 t/s 12.46.232.327 I slot print_timing: id 3 | task 32249 | prompt eval time = 249.88 ms / 31 tokens ( 8.06 ms per token, 124.06 tokens per second) 12.46.232.331 I slot print_timing: id 3 | task 32249 | eval time = 20796.42 ms / 936 tokens ( 22.22 ms per token, 45.01 tokens per second) 12.46.232.332 I slot print_timing: id 3 | task 32249 | total time = 21046.30 ms / 967 tokens 12.46.232.332 I slot print_timing: id 3 | task 32249 | graphs reused = 32993 12.46.232.371 I slot release: id 3 | task 32249 | stop processing: n_tokens = 1036, truncated = 0 12.46.232.383 I srv update_slots: all slots are idle 12.46.243.736 I srv operator(): Chat format: peg-native 12.46.243.973 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.636 (> 0.100 thold), f_keep = 0.068 12.46.243.976 I srv get_availabl: updating prompt cache 12.46.244.053 W srv prompt_save: - saving prompt with length 1036, total state size = 53.507 MiB (draft: 0.000 MiB) 12.46.292.886 I srv load: - looking for better prompt, base f_keep = 0.068, sim = 0.636 12.46.292.899 I srv update: - cache state: 22 prompts, 1825.534 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 12.46.292.900 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 12.46.292.901 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 12.46.292.902 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 12.46.292.902 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 12.46.292.903 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 12.46.292.903 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 12.46.292.904 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 12.46.292.904 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 12.46.292.905 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 12.46.292.905 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 12.46.292.906 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 12.46.292.906 I srv update: - prompt 0x63c2ea969620: 1142 tokens, checkpoints: 0, 58.982 MiB 12.46.292.907 I srv update: - prompt 0x63c2eaf04ba0: 1320 tokens, checkpoints: 0, 68.175 MiB 12.46.292.907 I srv update: - prompt 0x63c2f1f09fb0: 3376 tokens, checkpoints: 0, 174.361 MiB 12.46.292.908 I srv update: - prompt 0x63c2ea962870: 1636 tokens, checkpoints: 0, 84.495 MiB 12.46.292.908 I srv update: - prompt 0x63c2f1efda40: 2624 tokens, checkpoints: 0, 135.523 MiB 12.46.292.909 I srv update: - prompt 0x63c2f1e7c4d0: 2845 tokens, checkpoints: 0, 146.937 MiB 12.46.292.909 I srv update: - prompt 0x63c2eaaf4520: 1608 tokens, checkpoints: 0, 83.049 MiB 12.46.292.910 I srv update: - prompt 0x63c2f1e62ea0: 771 tokens, checkpoints: 0, 39.821 MiB 12.46.292.910 I srv update: - prompt 0x63c2eaa9f9b0: 1236 tokens, checkpoints: 0, 63.836 MiB 12.46.292.911 I srv update: - prompt 0x63c2f1dc6ec0: 1259 tokens, checkpoints: 0, 65.024 MiB 12.46.292.911 I srv update: - prompt 0x63c2f1f48c20: 1036 tokens, checkpoints: 0, 53.507 MiB 12.46.292.912 I srv get_availabl: prompt cache update took 48.94 ms 12.46.292.953 I slot launch_slot_: id 3 | task 33186 | processing task, is_child = 0 12.46.292.953 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 12.46.292.954 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 12.46.292.964 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 12.46.292.964 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 12.46.292.973 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 12.46.292.973 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 12.49.516.969 I slot print_timing: id 3 | task 33186 | n_decoded = 137, tg = 45.62 t/s, tg_3s = 45.62 t/s 12.52.518.120 I slot print_timing: id 3 | task 33186 | n_decoded = 272, tg = 45.30 t/s, tg_3s = 44.98 t/s 12.55.533.993 I slot print_timing: id 3 | task 33186 | n_decoded = 408, tg = 45.23 t/s, tg_3s = 45.09 t/s 12.58.554.101 I slot print_timing: id 3 | task 33186 | n_decoded = 543, tg = 45.10 t/s, tg_3s = 44.70 t/s 13.01.572.332 I slot print_timing: id 3 | task 33186 | n_decoded = 677, tg = 44.96 t/s, tg_3s = 44.40 t/s 13.04.594.484 I slot print_timing: id 3 | task 33186 | n_decoded = 812, tg = 44.91 t/s, tg_3s = 44.67 t/s 13.06.182.922 I slot print_timing: id 3 | task 33186 | prompt eval time = 220.69 ms / 40 tokens ( 5.52 ms per token, 181.25 tokens per second) 13.06.182.927 I slot print_timing: id 3 | task 33186 | eval time = 19669.24 ms / 883 tokens ( 22.28 ms per token, 44.89 tokens per second) 13.06.182.928 I slot print_timing: id 3 | task 33186 | total time = 19889.93 ms / 923 tokens 13.06.182.929 I slot print_timing: id 3 | task 33186 | graphs reused = 33871 13.06.182.982 I slot release: id 3 | task 33186 | stop processing: n_tokens = 992, truncated = 0 13.06.182.994 I srv update_slots: all slots are idle 13.06.193.968 I srv operator(): Chat format: peg-native 13.06.194.217 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.737 (> 0.100 thold), f_keep = 0.071 13.06.194.219 I srv get_availabl: updating prompt cache 13.06.194.290 W srv prompt_save: - saving prompt with length 992, total state size = 51.235 MiB (draft: 0.000 MiB) 13.06.236.830 I srv load: - looking for better prompt, base f_keep = 0.071, sim = 0.737 13.06.236.845 I srv update: - cache state: 23 prompts, 1876.768 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est) 13.06.236.850 I srv update: - prompt 0x63c2eae7a0e0: 1855 tokens, checkpoints: 0, 95.806 MiB 13.06.236.851 I srv update: - prompt 0x63c2f1995a20: 1238 tokens, checkpoints: 0, 63.940 MiB 13.06.236.852 I srv update: - prompt 0x63c2ea8de9c0: 758 tokens, checkpoints: 0, 39.149 MiB 13.06.236.853 I srv update: - prompt 0x63c2f1951910: 1441 tokens, checkpoints: 0, 74.424 MiB 13.06.236.857 I srv update: - prompt 0x63c2f1db0040: 1980 tokens, checkpoints: 0, 102.262 MiB 13.06.236.857 I srv update: - prompt 0x63c2f1dbd710: 1249 tokens, checkpoints: 0, 64.508 MiB 13.06.236.858 I srv update: - prompt 0x63c2f1df9610: 1236 tokens, checkpoints: 0, 63.836 MiB 13.06.236.859 I srv update: - prompt 0x63c2eae7e5d0: 1394 tokens, checkpoints: 0, 71.997 MiB 13.06.236.859 I srv update: - prompt 0x63c2ea8d81a0: 1526 tokens, checkpoints: 0, 78.814 MiB 13.06.236.861 I srv update: - prompt 0x63c2f1dbb070: 1884 tokens, checkpoints: 0, 97.304 MiB 13.06.236.862 I srv update: - prompt 0x63c2f1e6a330: 1932 tokens, checkpoints: 0, 99.783 MiB 13.06.236.862 I srv update: - prompt 0x63c2ea969620: 1142 tokens, checkpoints: 0, 58.982 MiB 13.06.236.863 I srv update: - prompt 0x63c2eaf04ba0: 1320 tokens, checkpoints: 0, 68.175 MiB 13.06.236.864 I srv update: - prompt 0x63c2f1f09fb0: 3376 tokens, checkpoints: 0, 174.361 MiB 13.06.236.864 I srv update: - prompt 0x63c2ea962870: 1636 tokens, checkpoints: 0, 84.495 MiB 13.06.236.865 I srv update: - prompt 0x63c2f1efda40: 2624 tokens, checkpoints: 0, 135.523 MiB 13.06.236.865 I srv update: - prompt 0x63c2f1e7c4d0: 2845 tokens, checkpoints: 0, 146.937 MiB 13.06.236.866 I srv update: - prompt 0x63c2eaaf4520: 1608 tokens, checkpoints: 0, 83.049 MiB 13.06.236.866 I srv update: - prompt 0x63c2f1e62ea0: 771 tokens, checkpoints: 0, 39.821 MiB 13.06.236.867 I srv update: - prompt 0x63c2eaa9f9b0: 1236 tokens, checkpoints: 0, 63.836 MiB 13.06.236.868 I srv update: - prompt 0x63c2f1dc6ec0: 1259 tokens, checkpoints: 0, 65.024 MiB 13.06.236.868 I srv update: - prompt 0x63c2f1f48c20: 1036 tokens, checkpoints: 0, 53.507 MiB 13.06.236.869 I srv update: - prompt 0x63c2eaabfd80: 992 tokens, checkpoints: 0, 51.235 MiB 13.06.236.870 I srv get_availabl: prompt cache update took 42.65 ms 13.06.236.919 I slot launch_slot_: id 3 | task 34070 | processing task, is_child = 0 13.06.236.920 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 13.06.236.921 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 13.06.236.932 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 13.06.236.933 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 13.06.236.956 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 13.06.236.957 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 13.09.490.194 I slot print_timing: id 3 | task 34070 | n_decoded = 137, tg = 45.33 t/s, tg_3s = 45.33 t/s 13.12.500.669 I slot print_timing: id 3 | task 34070 | n_decoded = 272, tg = 45.09 t/s, tg_3s = 44.84 t/s 13.15.501.680 I slot print_timing: id 3 | task 34070 | n_decoded = 407, tg = 45.05 t/s, tg_3s = 44.98 t/s 13.18.510.052 I slot print_timing: id 3 | task 34070 | n_decoded = 542, tg = 45.01 t/s, tg_3s = 44.87 t/s 13.21.530.237 I slot print_timing: id 3 | task 34070 | n_decoded = 678, tg = 45.01 t/s, tg_3s = 45.03 t/s 13.24.537.244 I slot print_timing: id 3 | task 34070 | n_decoded = 813, tg = 44.99 t/s, tg_3s = 44.90 t/s 13.27.559.286 I slot print_timing: id 3 | task 34070 | n_decoded = 948, tg = 44.95 t/s, tg_3s = 44.67 t/s 13.30.577.153 I slot print_timing: id 3 | task 34070 | n_decoded = 1083, tg = 44.92 t/s, tg_3s = 44.73 t/s 13.33.588.275 I slot print_timing: id 3 | task 34070 | n_decoded = 1217, tg = 44.87 t/s, tg_3s = 44.50 t/s 13.34.914.083 I slot print_timing: id 3 | task 34070 | prompt eval time = 231.20 ms / 25 tokens ( 9.25 ms per token, 108.13 tokens per second) 13.34.914.087 I slot print_timing: id 3 | task 34070 | eval time = 28445.89 ms / 1276 tokens ( 22.29 ms per token, 44.86 tokens per second) 13.34.914.088 I slot print_timing: id 3 | task 34070 | total time = 28677.09 ms / 1301 tokens 13.34.914.089 I slot print_timing: id 3 | task 34070 | graphs reused = 35140 13.34.914.133 I slot release: id 3 | task 34070 | stop processing: n_tokens = 1370, truncated = 0 13.34.914.148 I srv update_slots: all slots are idle 13.34.914.741 I srv operator(): operator(): cleaning up before exit...