0.00.066.133 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.066.136 I device_info: 0.00.066.157 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.066.161 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.066.167 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free) 0.00.066.225 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.066.230 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true 0.00.066.268 I srv init: running without SSL 0.00.066.333 I srv init: using 23 threads for HTTP server 0.00.066.667 I srv start: binding port with default address family 0.00.067.871 I srv llama_server: loading model 0.00.067.879 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/openai_gpt-oss-120b-Q4_K_M/openai_gpt-oss-120b-Q4_K_M-00001-of-00002.gguf' 0.00.067.927 I common_init_result: fitting params to device memory ... 0.00.067.929 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) 0.08.132.208 W load: setting token '<|message|>' (200008) attribute to USER_DEFINED (16), old attributes: 8 0.08.132.213 W load: setting token '<|start|>' (200006) attribute to USER_DEFINED (16), old attributes: 8 0.08.132.214 W load: setting token '<|constrain|>' (200003) attribute to USER_DEFINED (16), old attributes: 8 0.08.133.384 W load: setting token '<|channel|>' (200005) attribute to USER_DEFINED (16), old attributes: 8 0.08.138.239 W load: special_eog_ids contains both '<|return|>' and '<|call|>', or '<|calls|>' and '<|flush|>' tokens, removing '<|end|>' token from EOG list 0.45.595.825 W llama_context: n_ctx_seq (16384) < n_ctx_train (131072) -- the full capacity of the model will not be utilized 0.45.822.723 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) 0.46.010.445 I srv load_model: initializing slots, n_slots = 4 0.46.056.224 W common_speculative_init: no implementations specified for speculative decoding 0.46.056.230 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 0.46.056.234 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384 0.46.056.235 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384 0.46.056.235 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384 0.46.056.279 I srv load_model: prompt cache is enabled, size limit: 8192 MiB 0.46.056.280 I srv load_model: use `--cache-ram 0` to disable the prompt cache 0.46.056.280 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 0.46.056.280 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 0.46.056.302 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task 0.46.062.610 I init: chat template, example_format: '<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI. Knowledge cutoff: 2024-06 Current date: 2026-07-11 Reasoning: medium # Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>developer<|message|># Instructions You are a helpful assistant<|end|><|start|>user<|message|>Hello<|end|><|start|>assistant<|channel|>final<|message|>Hi there<|end|><|start|>user<|message|>How are you?<|end|><|start|>assistant' 0.46.063.245 I srv init: init: chat template, thinking = 1 0.46.063.423 I srv llama_server: model loaded 0.46.063.427 I srv llama_server: server is listening on http://127.0.0.1:8090 0.46.063.433 I srv update_slots: all slots are idle 0.50.013.637 I srv operator(): Chat format: peg-native 0.50.018.005 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1 0.50.018.007 I srv get_availabl: updating prompt cache 0.50.018.013 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 0.50.018.017 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) 0.50.018.018 I srv get_availabl: prompt cache update took 0.01 ms 0.50.018.950 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0 0.50.018.952 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.50.018.954 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.50.018.967 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.50.018.968 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.50.018.976 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.50.018.977 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.51.277.957 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 133, n_tokens = 134, size = 4.502 MiB) 0.54.550.157 I slot print_timing: id 3 | task 0 | n_decoded = 188, tg = 62.36 t/s, tg_3s = 62.36 t/s 0.57.553.333 I slot print_timing: id 3 | task 0 | n_decoded = 375, tg = 62.31 t/s, tg_3s = 62.27 t/s 1.00.565.379 I slot print_timing: id 3 | task 0 | n_decoded = 563, tg = 62.35 t/s, tg_3s = 62.42 t/s 1.03.581.291 I slot print_timing: id 3 | task 0 | n_decoded = 748, tg = 62.10 t/s, tg_3s = 61.34 t/s 1.06.588.505 I slot print_timing: id 3 | task 0 | n_decoded = 931, tg = 61.85 t/s, tg_3s = 60.85 t/s 1.09.596.140 I slot print_timing: id 3 | task 0 | n_decoded = 1113, tg = 61.62 t/s, tg_3s = 60.51 t/s 1.12.601.228 I slot print_timing: id 3 | task 0 | n_decoded = 1294, tg = 61.43 t/s, tg_3s = 60.23 t/s 1.13.691.156 I slot print_timing: id 3 | task 0 | prompt eval time = 1516.26 ms / 159 tokens ( 9.54 ms per token, 104.86 tokens per second) 1.13.691.160 I slot print_timing: id 3 | task 0 | eval time = 22155.89 ms / 1360 tokens ( 16.29 ms per token, 61.38 tokens per second) 1.13.691.160 I slot print_timing: id 3 | task 0 | total time = 23672.15 ms / 1519 tokens 1.13.691.164 I slot print_timing: id 3 | task 0 | graphs reused = 1354 1.13.691.216 I slot release: id 3 | task 0 | stop processing: n_tokens = 1518, truncated = 0 1.13.691.225 I srv update_slots: all slots are idle 1.13.700.997 I srv operator(): Chat format: peg-native 1.13.705.299 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.895 (> 0.100 thold), f_keep = 0.090 1.13.705.300 I srv get_availabl: updating prompt cache 1.13.705.369 W srv prompt_save: - saving prompt with length 1518, total state size = 57.887 MiB (draft: 0.000 MiB) 1.13.791.708 I srv load: - looking for better prompt, base f_keep = 0.090, sim = 0.895 1.13.791.714 I srv update: - cache state: 1 prompts, 62.389 MiB (limits: 8192.000 MiB, 16384 tokens, 199321 est) 1.13.791.715 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 1.13.791.717 I srv get_availabl: prompt cache update took 86.42 ms 1.13.792.565 I slot launch_slot_: id 3 | task 1363 | processing task, is_child = 0 1.13.792.568 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.13.792.569 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.13.792.587 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.13.792.587 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.13.792.596 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.13.792.596 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.13.792.612 I slot update_slots: id 3 | task 1363 | Checking checkpoint with [0, 133] against 9... 1.13.795.335 W slot update_slots: id 3 | task 1363 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 4.502 MiB) 1.17.009.226 I slot print_timing: id 3 | task 1363 | n_decoded = 187, tg = 62.17 t/s, tg_3s = 62.17 t/s 1.20.010.304 I slot print_timing: id 3 | task 1363 | n_decoded = 373, tg = 62.07 t/s, tg_3s = 61.98 t/s 1.23.015.492 I slot print_timing: id 3 | task 1363 | n_decoded = 557, tg = 61.79 t/s, tg_3s = 61.23 t/s 1.25.856.119 I slot print_timing: id 3 | task 1363 | prompt eval time = 208.63 ms / 20 tokens ( 10.43 ms per token, 95.87 tokens per second) 1.25.856.122 I slot print_timing: id 3 | task 1363 | eval time = 11854.88 ms / 729 tokens ( 16.26 ms per token, 61.49 tokens per second) 1.25.856.123 I slot print_timing: id 3 | task 1363 | total time = 12063.50 ms / 749 tokens 1.25.856.124 I slot print_timing: id 3 | task 1363 | graphs reused = 2075 1.25.856.156 I slot release: id 3 | task 1363 | stop processing: n_tokens = 881, truncated = 0 1.25.856.167 I srv update_slots: all slots are idle 1.25.864.616 I srv operator(): Chat format: peg-native 1.25.868.486 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.895 (> 0.100 thold), f_keep = 0.156 1.25.868.488 I srv get_availabl: updating prompt cache 1.25.868.546 W srv prompt_save: - saving prompt with length 881, total state size = 35.485 MiB (draft: 0.000 MiB) 1.25.896.454 I srv load: - looking for better prompt, base f_keep = 0.156, sim = 0.895 1.25.896.461 I srv update: - cache state: 2 prompts, 102.376 MiB (limits: 8192.000 MiB, 16384 tokens, 191965 est) 1.25.896.462 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 1.25.896.463 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 1.25.896.464 I srv get_availabl: prompt cache update took 27.98 ms 1.25.897.318 I slot launch_slot_: id 3 | task 2095 | processing task, is_child = 0 1.25.897.320 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.25.897.321 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.25.897.338 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.25.897.339 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.25.897.348 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.25.897.348 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.29.065.901 I slot print_timing: id 3 | task 2095 | n_decoded = 186, tg = 61.78 t/s, tg_3s = 61.78 t/s 1.32.077.025 I slot print_timing: id 3 | task 2095 | n_decoded = 370, tg = 61.44 t/s, tg_3s = 61.11 t/s 1.35.083.465 I slot print_timing: id 3 | task 2095 | n_decoded = 552, tg = 61.14 t/s, tg_3s = 60.54 t/s 1.35.669.496 I slot print_timing: id 3 | task 2095 | prompt eval time = 157.84 ms / 16 tokens ( 9.86 ms per token, 101.37 tokens per second) 1.35.669.499 I slot print_timing: id 3 | task 2095 | eval time = 9614.29 ms / 588 tokens ( 16.35 ms per token, 61.16 tokens per second) 1.35.669.499 I slot print_timing: id 3 | task 2095 | total time = 9772.13 ms / 604 tokens 1.35.669.500 I slot print_timing: id 3 | task 2095 | graphs reused = 2657 1.35.669.537 I slot release: id 3 | task 2095 | stop processing: n_tokens = 740, truncated = 0 1.35.669.547 I srv update_slots: all slots are idle 1.35.677.654 I srv operator(): Chat format: peg-native 1.35.681.529 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.901 (> 0.100 thold), f_keep = 0.185 1.35.681.530 I srv get_availabl: updating prompt cache 1.35.681.584 W srv prompt_save: - saving prompt with length 740, total state size = 30.526 MiB (draft: 0.000 MiB) 1.35.706.670 I srv load: - looking for better prompt, base f_keep = 0.185, sim = 0.901 1.35.706.678 I srv update: - cache state: 3 prompts, 137.404 MiB (limits: 8192.000 MiB, 16384 tokens, 187146 est) 1.35.706.679 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 1.35.706.680 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 1.35.706.683 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 1.35.706.685 I srv get_availabl: prompt cache update took 25.15 ms 1.35.707.642 I slot launch_slot_: id 3 | task 2685 | processing task, is_child = 0 1.35.707.644 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.35.707.645 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.35.707.663 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.35.707.664 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.35.707.673 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.35.707.673 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.38.872.290 I slot print_timing: id 3 | task 2685 | n_decoded = 187, tg = 62.31 t/s, tg_3s = 62.31 t/s 1.41.880.414 I slot print_timing: id 3 | task 2685 | n_decoded = 374, tg = 62.24 t/s, tg_3s = 62.17 t/s 1.44.881.550 I slot print_timing: id 3 | task 2685 | n_decoded = 561, tg = 62.26 t/s, tg_3s = 62.31 t/s 1.47.891.435 I slot print_timing: id 3 | task 2685 | n_decoded = 746, tg = 62.06 t/s, tg_3s = 61.46 t/s 1.50.895.111 I slot print_timing: id 3 | task 2685 | n_decoded = 930, tg = 61.90 t/s, tg_3s = 61.26 t/s 1.53.895.240 I slot print_timing: id 3 | task 2685 | n_decoded = 1113, tg = 61.75 t/s, tg_3s = 61.00 t/s 1.56.910.272 I slot print_timing: id 3 | task 2685 | n_decoded = 1295, tg = 61.55 t/s, tg_3s = 60.36 t/s 1.59.917.313 I slot print_timing: id 3 | task 2685 | n_decoded = 1476, tg = 61.38 t/s, tg_3s = 60.19 t/s 2.00.930.373 I slot print_timing: id 3 | task 2685 | prompt eval time = 163.34 ms / 15 tokens ( 10.89 ms per token, 91.83 tokens per second) 2.00.930.377 I slot print_timing: id 3 | task 2685 | eval time = 25059.34 ms / 1537 tokens ( 16.30 ms per token, 61.33 tokens per second) 2.00.930.377 I slot print_timing: id 3 | task 2685 | total time = 25222.68 ms / 1552 tokens 2.00.930.378 I slot print_timing: id 3 | task 2685 | graphs reused = 4183 2.00.930.410 I slot release: id 3 | task 2685 | stop processing: n_tokens = 1688, truncated = 0 2.00.930.423 I srv update_slots: all slots are idle 2.00.939.289 I srv operator(): Chat format: peg-native 2.00.943.157 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.830 (> 0.100 thold), f_keep = 0.081 2.00.943.158 I srv get_availabl: updating prompt cache 2.00.943.227 W srv prompt_save: - saving prompt with length 1688, total state size = 63.865 MiB (draft: 0.000 MiB) 2.01.017.064 I srv load: - looking for better prompt, base f_keep = 0.081, sim = 0.830 2.01.017.071 I srv update: - cache state: 4 prompts, 205.771 MiB (limits: 8192.000 MiB, 16384 tokens, 192168 est) 2.01.017.072 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 2.01.017.073 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 2.01.017.074 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 2.01.017.077 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 2.01.017.079 I srv get_availabl: prompt cache update took 73.92 ms 2.01.018.099 I slot launch_slot_: id 3 | task 4224 | processing task, is_child = 0 2.01.018.102 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.01.018.102 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.01.018.121 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.01.018.122 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.01.018.139 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.01.018.139 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.01.018.166 I slot update_slots: id 3 | task 4224 | Checking checkpoint with [0, 133] against 9... 2.01.020.977 W slot update_slots: id 3 | task 4224 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 4.502 MiB) 2.04.357.595 I slot print_timing: id 3 | task 4224 | n_decoded = 187, tg = 62.11 t/s, tg_3s = 62.11 t/s 2.07.361.457 I slot print_timing: id 3 | task 4224 | n_decoded = 373, tg = 62.01 t/s, tg_3s = 61.92 t/s 2.10.371.670 I slot print_timing: id 3 | task 4224 | n_decoded = 557, tg = 61.72 t/s, tg_3s = 61.13 t/s 2.13.384.204 I slot print_timing: id 3 | task 4224 | n_decoded = 739, tg = 61.39 t/s, tg_3s = 60.41 t/s 2.16.391.409 I slot print_timing: id 3 | task 4224 | n_decoded = 921, tg = 61.22 t/s, tg_3s = 60.52 t/s 2.19.399.224 I slot print_timing: id 3 | task 4224 | n_decoded = 1103, tg = 61.10 t/s, tg_3s = 60.51 t/s 2.22.404.522 I slot print_timing: id 3 | task 4224 | n_decoded = 1285, tg = 61.02 t/s, tg_3s = 60.56 t/s 2.25.419.528 I slot print_timing: id 3 | task 4224 | n_decoded = 1468, tg = 60.98 t/s, tg_3s = 60.70 t/s 2.28.431.548 I slot print_timing: id 3 | task 4224 | n_decoded = 1650, tg = 60.92 t/s, tg_3s = 60.42 t/s 2.31.440.574 I slot print_timing: id 3 | task 4224 | n_decoded = 1830, tg = 60.81 t/s, tg_3s = 59.82 t/s 2.34.452.999 I slot print_timing: id 3 | task 4224 | n_decoded = 2010, tg = 60.71 t/s, tg_3s = 59.75 t/s 2.37.459.624 I slot print_timing: id 3 | task 4224 | n_decoded = 2190, tg = 60.64 t/s, tg_3s = 59.87 t/s 2.37.610.036 I slot print_timing: id 3 | task 4224 | prompt eval time = 328.46 ms / 32 tokens ( 10.26 ms per token, 97.42 tokens per second) 2.37.610.040 I slot print_timing: id 3 | task 4224 | eval time = 36263.40 ms / 2199 tokens ( 16.49 ms per token, 60.64 tokens per second) 2.37.610.040 I slot print_timing: id 3 | task 4224 | total time = 36591.87 ms / 2231 tokens 2.37.610.041 I slot print_timing: id 3 | task 4224 | graphs reused = 6368 2.37.610.074 I slot release: id 3 | task 4224 | stop processing: n_tokens = 2363, truncated = 0 2.37.610.088 I srv update_slots: all slots are idle 2.37.619.417 I srv operator(): Chat format: peg-native 2.37.623.605 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.890 (> 0.100 thold), f_keep = 0.058 2.37.623.607 I srv get_availabl: updating prompt cache 2.37.623.711 W srv prompt_save: - saving prompt with length 2363, total state size = 87.604 MiB (draft: 0.000 MiB) 2.37.736.064 I srv load: - looking for better prompt, base f_keep = 0.058, sim = 0.890 2.37.736.072 I srv update: - cache state: 5 prompts, 297.877 MiB (limits: 8192.000 MiB, 16384 tokens, 197734 est) 2.37.736.072 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 2.37.736.074 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 2.37.736.075 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 2.37.736.075 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 2.37.736.076 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 2.37.736.077 I srv get_availabl: prompt cache update took 112.47 ms 2.37.736.951 I slot launch_slot_: id 3 | task 6426 | processing task, is_child = 0 2.37.736.952 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.37.736.953 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.37.736.971 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.37.736.971 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.37.736.981 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.37.736.981 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.37.736.997 I slot update_slots: id 3 | task 6426 | Checking checkpoint with [0, 133] against 9... 2.37.739.614 W slot update_slots: id 3 | task 6426 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 4.502 MiB) 2.40.954.213 I slot print_timing: id 3 | task 6426 | n_decoded = 188, tg = 62.45 t/s, tg_3s = 62.45 t/s 2.43.960.316 I slot print_timing: id 3 | task 6426 | n_decoded = 375, tg = 62.33 t/s, tg_3s = 62.21 t/s 2.46.975.769 I slot print_timing: id 3 | task 6426 | n_decoded = 561, tg = 62.11 t/s, tg_3s = 61.68 t/s 2.49.989.277 I slot print_timing: id 3 | task 6426 | n_decoded = 744, tg = 61.77 t/s, tg_3s = 60.73 t/s 2.52.997.934 I slot print_timing: id 3 | task 6426 | n_decoded = 926, tg = 61.51 t/s, tg_3s = 60.49 t/s 2.56.004.417 I slot print_timing: id 3 | task 6426 | n_decoded = 1108, tg = 61.35 t/s, tg_3s = 60.54 t/s 2.59.005.519 I slot print_timing: id 3 | task 6426 | n_decoded = 1288, tg = 61.15 t/s, tg_3s = 59.98 t/s 3.02.014.189 I slot print_timing: id 3 | task 6426 | n_decoded = 1468, tg = 60.99 t/s, tg_3s = 59.83 t/s 3.05.019.114 I slot print_timing: id 3 | task 6426 | n_decoded = 1647, tg = 60.83 t/s, tg_3s = 59.57 t/s 3.08.029.231 I slot print_timing: id 3 | task 6426 | n_decoded = 1824, tg = 60.63 t/s, tg_3s = 58.80 t/s 3.08.335.233 I slot print_timing: id 3 | task 6426 | prompt eval time = 206.73 ms / 21 tokens ( 9.84 ms per token, 101.58 tokens per second) 3.08.335.237 I slot print_timing: id 3 | task 6426 | eval time = 30391.50 ms / 1842 tokens ( 16.50 ms per token, 60.61 tokens per second) 3.08.335.237 I slot print_timing: id 3 | task 6426 | total time = 30598.23 ms / 1863 tokens 3.08.335.238 I slot print_timing: id 3 | task 6426 | graphs reused = 8198 3.08.335.278 I slot release: id 3 | task 6426 | stop processing: n_tokens = 1995, truncated = 0 3.08.335.294 I srv update_slots: all slots are idle 3.08.344.094 I srv operator(): Chat format: peg-native 3.08.348.146 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.920 (> 0.100 thold), f_keep = 0.069 3.08.348.148 I srv get_availabl: updating prompt cache 3.08.348.234 W srv prompt_save: - saving prompt with length 1995, total state size = 74.662 MiB (draft: 0.000 MiB) 3.08.448.528 I srv load: - looking for better prompt, base f_keep = 0.069, sim = 0.920 3.08.448.536 I srv update: - cache state: 6 prompts, 377.041 MiB (limits: 8192.000 MiB, 16384 tokens, 199563 est) 3.08.448.537 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 3.08.448.537 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 3.08.448.538 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 3.08.448.540 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 3.08.448.540 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 3.08.448.541 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 3.08.448.542 I srv get_availabl: prompt cache update took 100.39 ms 3.08.449.496 I slot launch_slot_: id 3 | task 8271 | processing task, is_child = 0 3.08.449.497 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.08.449.498 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.08.449.515 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.08.449.515 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.08.449.525 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.08.449.526 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.08.449.544 I slot update_slots: id 3 | task 8271 | Checking checkpoint with [0, 133] against 10... 3.08.452.281 W slot update_slots: id 3 | task 8271 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 4.502 MiB) 3.11.647.980 I slot print_timing: id 3 | task 8271 | n_decoded = 185, tg = 61.53 t/s, tg_3s = 61.53 t/s 3.14.660.305 I slot print_timing: id 3 | task 8271 | n_decoded = 368, tg = 61.14 t/s, tg_3s = 60.75 t/s 3.17.665.321 I slot print_timing: id 3 | task 8271 | n_decoded = 554, tg = 61.39 t/s, tg_3s = 61.90 t/s 3.20.675.449 I slot print_timing: id 3 | task 8271 | n_decoded = 739, tg = 61.41 t/s, tg_3s = 61.46 t/s 3.22.691.927 I slot print_timing: id 3 | task 8271 | prompt eval time = 191.76 ms / 17 tokens ( 11.28 ms per token, 88.65 tokens per second) 3.22.691.930 I slot print_timing: id 3 | task 8271 | eval time = 14050.62 ms / 863 tokens ( 16.28 ms per token, 61.42 tokens per second) 3.22.691.931 I slot print_timing: id 3 | task 8271 | total time = 14242.38 ms / 880 tokens 3.22.691.932 I slot print_timing: id 3 | task 8271 | graphs reused = 9053 3.22.691.967 I slot release: id 3 | task 8271 | stop processing: n_tokens = 1012, truncated = 0 3.22.691.978 I srv update_slots: all slots are idle 3.22.699.733 I srv operator(): Chat format: peg-native 3.22.703.178 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.908 (> 0.100 thold), f_keep = 0.136 3.22.703.180 I srv get_availabl: updating prompt cache 3.22.703.254 W srv prompt_save: - saving prompt with length 1012, total state size = 40.092 MiB (draft: 0.000 MiB) 3.22.737.826 I srv load: - looking for better prompt, base f_keep = 0.136, sim = 0.908 3.22.737.837 I srv update: - cache state: 7 prompts, 421.635 MiB (limits: 8192.000 MiB, 16384 tokens, 198119 est) 3.22.737.839 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 3.22.737.840 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 3.22.737.844 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 3.22.737.845 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 3.22.737.846 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 3.22.737.847 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 3.22.737.848 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 3.22.737.852 I srv get_availabl: prompt cache update took 34.67 ms 3.22.738.926 I slot launch_slot_: id 3 | task 9137 | processing task, is_child = 0 3.22.738.929 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.22.738.930 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.22.738.953 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.22.738.953 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.22.738.966 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.22.738.967 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.25.895.265 I slot print_timing: id 3 | task 9137 | n_decoded = 187, tg = 62.32 t/s, tg_3s = 62.32 t/s 3.28.904.430 I slot print_timing: id 3 | task 9137 | n_decoded = 374, tg = 62.23 t/s, tg_3s = 62.14 t/s 3.31.913.195 I slot print_timing: id 3 | task 9137 | n_decoded = 560, tg = 62.10 t/s, tg_3s = 61.82 t/s 3.34.926.958 I slot print_timing: id 3 | task 9137 | n_decoded = 744, tg = 61.83 t/s, tg_3s = 61.05 t/s 3.36.909.926 I slot print_timing: id 3 | task 9137 | prompt eval time = 155.81 ms / 14 tokens ( 11.13 ms per token, 89.85 tokens per second) 3.36.909.930 I slot print_timing: id 3 | task 9137 | eval time = 14015.12 ms / 865 tokens ( 16.20 ms per token, 61.72 tokens per second) 3.36.909.931 I slot print_timing: id 3 | task 9137 | total time = 14170.93 ms / 879 tokens 3.36.909.932 I slot print_timing: id 3 | task 9137 | graphs reused = 9910 3.36.909.966 I slot release: id 3 | task 9137 | stop processing: n_tokens = 1016, truncated = 0 3.36.909.978 I srv update_slots: all slots are idle 3.36.918.923 I srv operator(): Chat format: peg-native 3.36.922.863 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.761 (> 0.100 thold), f_keep = 0.135 3.36.922.865 I srv get_availabl: updating prompt cache 3.36.922.941 W srv prompt_save: - saving prompt with length 1016, total state size = 40.233 MiB (draft: 0.000 MiB) 3.36.954.775 I srv load: - looking for better prompt, base f_keep = 0.135, sim = 0.761 3.36.954.788 I srv update: - cache state: 8 prompts, 466.369 MiB (limits: 8192.000 MiB, 16384 tokens, 196961 est) 3.36.954.788 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 3.36.954.789 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 3.36.954.790 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 3.36.954.791 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 3.36.954.792 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 3.36.954.793 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 3.36.954.794 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 3.36.954.794 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 3.36.954.795 I srv get_availabl: prompt cache update took 31.93 ms 3.36.955.643 I slot launch_slot_: id 3 | task 10004 | processing task, is_child = 0 3.36.955.645 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.36.955.646 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.36.955.663 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.36.955.664 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.36.955.672 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.36.955.673 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.40.367.387 I slot print_timing: id 3 | task 10004 | n_decoded = 187, tg = 62.16 t/s, tg_3s = 62.16 t/s 3.43.377.923 I slot print_timing: id 3 | task 10004 | n_decoded = 373, tg = 61.97 t/s, tg_3s = 61.78 t/s 3.46.393.887 I slot print_timing: id 3 | task 10004 | n_decoded = 558, tg = 61.76 t/s, tg_3s = 61.34 t/s 3.46.557.307 I slot print_timing: id 3 | task 10004 | prompt eval time = 403.43 ms / 43 tokens ( 9.38 ms per token, 106.59 tokens per second) 3.46.557.309 I slot print_timing: id 3 | task 10004 | eval time = 9198.18 ms / 568 tokens ( 16.19 ms per token, 61.75 tokens per second) 3.46.557.310 I slot print_timing: id 3 | task 10004 | total time = 9601.62 ms / 611 tokens 3.46.557.311 I slot print_timing: id 3 | task 10004 | graphs reused = 10472 3.46.557.346 I slot release: id 3 | task 10004 | stop processing: n_tokens = 747, truncated = 0 3.46.557.354 I srv update_slots: all slots are idle 3.46.566.321 I srv operator(): Chat format: peg-native 3.46.569.838 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.708 (> 0.100 thold), f_keep = 0.185 3.46.569.840 I srv get_availabl: updating prompt cache 3.46.569.881 W srv prompt_save: - saving prompt with length 747, total state size = 30.773 MiB (draft: 0.000 MiB) 3.46.595.165 I srv load: - looking for better prompt, base f_keep = 0.185, sim = 0.708 3.46.595.179 I srv update: - cache state: 9 prompts, 501.644 MiB (limits: 8192.000 MiB, 16384 tokens, 195310 est) 3.46.595.179 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 3.46.595.180 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 3.46.595.180 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 3.46.595.181 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 3.46.595.182 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 3.46.595.186 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 3.46.595.186 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 3.46.595.187 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 3.46.595.187 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 3.46.595.188 I srv get_availabl: prompt cache update took 25.35 ms 3.46.596.054 I slot launch_slot_: id 3 | task 10574 | processing task, is_child = 0 3.46.596.056 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.46.596.056 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.46.596.072 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.46.596.072 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.46.596.081 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.46.596.081 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.50.139.860 I slot print_timing: id 3 | task 10574 | n_decoded = 186, tg = 61.99 t/s, tg_3s = 61.99 t/s 3.53.146.978 I slot print_timing: id 3 | task 10574 | n_decoded = 370, tg = 61.59 t/s, tg_3s = 61.19 t/s 3.53.425.015 I slot print_timing: id 3 | task 10574 | prompt eval time = 543.05 ms / 57 tokens ( 9.53 ms per token, 104.96 tokens per second) 3.53.425.018 I slot print_timing: id 3 | task 10574 | eval time = 6285.87 ms / 387 tokens ( 16.24 ms per token, 61.57 tokens per second) 3.53.425.018 I slot print_timing: id 3 | task 10574 | total time = 6828.91 ms / 444 tokens 3.53.425.019 I slot print_timing: id 3 | task 10574 | graphs reused = 10853 3.53.425.052 I slot release: id 3 | task 10574 | stop processing: n_tokens = 581, truncated = 0 3.53.425.061 I srv update_slots: all slots are idle 3.53.432.786 I srv operator(): Chat format: peg-native 3.53.436.245 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.787 (> 0.100 thold), f_keep = 0.236 3.53.436.247 I srv get_availabl: updating prompt cache 3.53.436.313 W srv prompt_save: - saving prompt with length 581, total state size = 24.935 MiB (draft: 0.000 MiB) 3.53.459.309 I srv load: - looking for better prompt, base f_keep = 0.236, sim = 0.787 3.53.459.324 I srv update: - cache state: 10 prompts, 531.080 MiB (limits: 8192.000 MiB, 16384 tokens, 193446 est) 3.53.459.328 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 3.53.459.328 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 3.53.459.329 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 3.53.459.330 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 3.53.459.330 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 3.53.459.331 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 3.53.459.332 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 3.53.459.333 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 3.53.459.333 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 3.53.459.334 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 3.53.459.335 I srv get_availabl: prompt cache update took 23.09 ms 3.53.460.188 I slot launch_slot_: id 3 | task 10963 | processing task, is_child = 0 3.53.460.190 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.53.460.191 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.53.460.206 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.53.460.206 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.53.460.215 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.53.460.215 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.56.844.279 I slot print_timing: id 3 | task 10963 | n_decoded = 187, tg = 62.20 t/s, tg_3s = 62.20 t/s 3.59.850.936 I slot print_timing: id 3 | task 10963 | n_decoded = 372, tg = 61.86 t/s, tg_3s = 61.53 t/s 4.02.851.734 I slot print_timing: id 3 | task 10963 | n_decoded = 558, tg = 61.90 t/s, tg_3s = 61.98 t/s 4.05.548.912 I slot print_timing: id 3 | task 10963 | prompt eval time = 377.47 ms / 37 tokens ( 10.20 ms per token, 98.02 tokens per second) 4.05.548.916 I slot print_timing: id 3 | task 10963 | eval time = 11711.21 ms / 724 tokens ( 16.18 ms per token, 61.82 tokens per second) 4.05.548.916 I slot print_timing: id 3 | task 10963 | total time = 12088.68 ms / 761 tokens 4.05.548.917 I slot print_timing: id 3 | task 10963 | graphs reused = 11569 4.05.548.953 I slot release: id 3 | task 10963 | stop processing: n_tokens = 897, truncated = 0 4.05.548.965 I srv update_slots: all slots are idle 4.05.557.842 I srv operator(): Chat format: peg-native 4.05.561.856 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.729 (> 0.100 thold), f_keep = 0.153 4.05.561.858 I srv get_availabl: updating prompt cache 4.05.561.910 W srv prompt_save: - saving prompt with length 897, total state size = 36.048 MiB (draft: 0.000 MiB) 4.05.590.495 I srv load: - looking for better prompt, base f_keep = 0.153, sim = 0.729 4.05.590.508 I srv update: - cache state: 11 prompts, 571.630 MiB (limits: 8192.000 MiB, 16384 tokens, 192579 est) 4.05.590.509 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 4.05.590.510 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 4.05.590.511 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 4.05.590.511 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 4.05.590.512 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 4.05.590.512 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 4.05.590.513 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 4.05.590.513 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 4.05.590.514 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 4.05.590.514 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 4.05.590.515 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 4.05.590.515 I srv get_availabl: prompt cache update took 28.66 ms 4.05.591.439 I slot launch_slot_: id 3 | task 11689 | processing task, is_child = 0 4.05.591.441 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.05.591.442 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.05.591.459 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.05.591.459 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.05.591.471 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.05.591.472 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.09.083.022 I slot print_timing: id 3 | task 11689 | n_decoded = 188, tg = 62.38 t/s, tg_3s = 62.38 t/s 4.12.092.999 I slot print_timing: id 3 | task 11689 | n_decoded = 375, tg = 62.25 t/s, tg_3s = 62.13 t/s 4.13.414.125 I slot print_timing: id 3 | task 11689 | prompt eval time = 477.81 ms / 51 tokens ( 9.37 ms per token, 106.74 tokens per second) 4.13.414.128 I slot print_timing: id 3 | task 11689 | eval time = 7344.81 ms / 457 tokens ( 16.07 ms per token, 62.22 tokens per second) 4.13.414.129 I slot print_timing: id 3 | task 11689 | total time = 7822.63 ms / 508 tokens 4.13.414.130 I slot print_timing: id 3 | task 11689 | graphs reused = 12020 4.13.414.166 I slot release: id 3 | task 11689 | stop processing: n_tokens = 644, truncated = 0 4.13.414.177 I srv update_slots: all slots are idle 4.13.423.096 I srv operator(): Chat format: peg-native 4.13.426.953 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.213 4.13.426.954 I srv get_availabl: updating prompt cache 4.13.427.008 W srv prompt_save: - saving prompt with length 644, total state size = 27.150 MiB (draft: 0.000 MiB) 4.13.450.051 I srv load: - looking for better prompt, base f_keep = 0.213, sim = 0.721 4.13.450.064 I srv update: - cache state: 12 prompts, 603.282 MiB (limits: 8192.000 MiB, 16384 tokens, 191220 est) 4.13.450.065 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 4.13.450.065 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 4.13.450.066 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 4.13.450.066 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 4.13.450.067 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 4.13.450.067 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 4.13.450.068 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 4.13.450.068 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 4.13.450.069 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 4.13.450.069 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 4.13.450.070 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 4.13.450.070 I srv update: - prompt 0x5e39f11a39f0: 644 tokens, checkpoints: 1, 31.652 MiB 4.13.450.071 I srv get_availabl: prompt cache update took 23.12 ms 4.13.450.955 I slot launch_slot_: id 3 | task 12148 | processing task, is_child = 0 4.13.450.957 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.13.450.957 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.13.450.974 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.13.450.974 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.13.450.983 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.13.450.983 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.16.993.755 I slot print_timing: id 3 | task 12148 | n_decoded = 188, tg = 62.39 t/s, tg_3s = 62.39 t/s 4.19.293.227 I slot print_timing: id 3 | task 12148 | prompt eval time = 529.51 ms / 53 tokens ( 9.99 ms per token, 100.09 tokens per second) 4.19.293.230 I slot print_timing: id 3 | task 12148 | eval time = 5312.72 ms / 331 tokens ( 16.05 ms per token, 62.30 tokens per second) 4.19.293.231 I slot print_timing: id 3 | task 12148 | total time = 5842.22 ms / 384 tokens 4.19.293.231 I slot print_timing: id 3 | task 12148 | graphs reused = 12345 4.19.293.270 I slot release: id 3 | task 12148 | stop processing: n_tokens = 520, truncated = 0 4.19.293.279 I srv update_slots: all slots are idle 4.19.301.713 I srv operator(): Chat format: peg-native 4.19.305.532 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.780 (> 0.100 thold), f_keep = 0.265 4.19.305.534 I srv get_availabl: updating prompt cache 4.19.305.573 W srv prompt_save: - saving prompt with length 520, total state size = 22.790 MiB (draft: 0.000 MiB) 4.19.325.337 I srv load: - looking for better prompt, base f_keep = 0.265, sim = 0.780 4.19.325.350 I srv update: - cache state: 13 prompts, 630.574 MiB (limits: 8192.000 MiB, 16384 tokens, 189699 est) 4.19.325.351 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 4.19.325.351 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 4.19.325.352 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 4.19.325.352 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 4.19.325.353 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 4.19.325.353 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 4.19.325.353 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 4.19.325.354 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 4.19.325.354 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 4.19.325.355 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 4.19.325.355 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 4.19.325.356 I srv update: - prompt 0x5e39f11a39f0: 644 tokens, checkpoints: 1, 31.652 MiB 4.19.325.356 I srv update: - prompt 0x5e39f1080b60: 520 tokens, checkpoints: 1, 27.291 MiB 4.19.325.357 I srv get_availabl: prompt cache update took 19.82 ms 4.19.326.205 I slot launch_slot_: id 3 | task 12481 | processing task, is_child = 0 4.19.326.206 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.19.326.207 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.19.326.221 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.19.326.221 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.19.326.230 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.19.326.231 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.22.723.815 I slot print_timing: id 3 | task 12481 | n_decoded = 188, tg = 62.47 t/s, tg_3s = 62.47 t/s 4.25.725.622 I slot print_timing: id 3 | task 12481 | n_decoded = 374, tg = 62.22 t/s, tg_3s = 61.96 t/s 4.27.831.673 I slot print_timing: id 3 | task 12481 | prompt eval time = 388.00 ms / 39 tokens ( 9.95 ms per token, 100.52 tokens per second) 4.27.831.676 I slot print_timing: id 3 | task 12481 | eval time = 8117.43 ms / 503 tokens ( 16.14 ms per token, 61.97 tokens per second) 4.27.831.676 I slot print_timing: id 3 | task 12481 | total time = 8505.42 ms / 542 tokens 4.27.831.677 I slot print_timing: id 3 | task 12481 | graphs reused = 12842 4.27.831.726 I slot release: id 3 | task 12481 | stop processing: n_tokens = 679, truncated = 0 4.27.831.735 I srv update_slots: all slots are idle 4.27.840.854 I srv operator(): Chat format: peg-native 4.27.844.921 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.745 (> 0.100 thold), f_keep = 0.202 4.27.844.923 I srv get_availabl: updating prompt cache 4.27.844.979 W srv prompt_save: - saving prompt with length 679, total state size = 28.381 MiB (draft: 0.000 MiB) 4.27.869.632 I srv load: - looking for better prompt, base f_keep = 0.202, sim = 0.745 4.27.869.649 I srv update: - cache state: 14 prompts, 663.457 MiB (limits: 8192.000 MiB, 16384 tokens, 188681 est) 4.27.869.650 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 4.27.869.652 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 4.27.869.652 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 4.27.869.653 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 4.27.869.654 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 4.27.869.654 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 4.27.869.655 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 4.27.869.655 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 4.27.869.656 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 4.27.869.656 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 4.27.869.657 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 4.27.869.658 I srv update: - prompt 0x5e39f11a39f0: 644 tokens, checkpoints: 1, 31.652 MiB 4.27.869.658 I srv update: - prompt 0x5e39f1080b60: 520 tokens, checkpoints: 1, 27.291 MiB 4.27.869.659 I srv update: - prompt 0x5e39f1267390: 679 tokens, checkpoints: 1, 32.883 MiB 4.27.869.660 I srv get_availabl: prompt cache update took 24.74 ms 4.27.870.740 I slot launch_slot_: id 3 | task 12986 | processing task, is_child = 0 4.27.870.743 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.27.870.744 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.27.870.762 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.27.870.762 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.27.870.774 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.27.870.774 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.31.354.128 I slot print_timing: id 3 | task 12986 | n_decoded = 187, tg = 62.14 t/s, tg_3s = 62.14 t/s 4.34.363.659 I slot print_timing: id 3 | task 12986 | n_decoded = 372, tg = 61.80 t/s, tg_3s = 61.47 t/s 4.37.371.172 I slot print_timing: id 3 | task 12986 | n_decoded = 556, tg = 61.60 t/s, tg_3s = 61.18 t/s 4.38.765.082 I slot print_timing: id 3 | task 12986 | prompt eval time = 473.84 ms / 47 tokens ( 10.08 ms per token, 99.19 tokens per second) 4.38.765.085 I slot print_timing: id 3 | task 12986 | eval time = 10420.44 ms / 640 tokens ( 16.28 ms per token, 61.42 tokens per second) 4.38.765.086 I slot print_timing: id 3 | task 12986 | total time = 10894.28 ms / 687 tokens 4.38.765.086 I slot print_timing: id 3 | task 12986 | graphs reused = 13474 4.38.765.126 I slot release: id 3 | task 12986 | stop processing: n_tokens = 823, truncated = 0 4.38.765.136 I srv update_slots: all slots are idle 4.38.773.755 I srv operator(): Chat format: peg-native 4.38.777.327 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.801 (> 0.100 thold), f_keep = 0.166 4.38.777.329 I srv get_availabl: updating prompt cache 4.38.777.382 W srv prompt_save: - saving prompt with length 823, total state size = 33.445 MiB (draft: 0.000 MiB) 4.38.804.381 I srv load: - looking for better prompt, base f_keep = 0.166, sim = 0.801 4.38.804.395 I srv update: - cache state: 15 prompts, 701.404 MiB (limits: 8192.000 MiB, 16384 tokens, 188085 est) 4.38.804.396 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 4.38.804.397 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 4.38.804.398 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 4.38.804.399 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 4.38.804.403 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 4.38.804.403 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 4.38.804.404 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 4.38.804.404 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 4.38.804.405 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 4.38.804.405 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 4.38.804.405 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 4.38.804.406 I srv update: - prompt 0x5e39f11a39f0: 644 tokens, checkpoints: 1, 31.652 MiB 4.38.804.406 I srv update: - prompt 0x5e39f1080b60: 520 tokens, checkpoints: 1, 27.291 MiB 4.38.804.407 I srv update: - prompt 0x5e39f1267390: 679 tokens, checkpoints: 1, 32.883 MiB 4.38.804.408 I srv update: - prompt 0x5e39f1142a10: 823 tokens, checkpoints: 1, 37.947 MiB 4.38.804.412 I srv get_availabl: prompt cache update took 27.08 ms 4.38.805.275 I slot launch_slot_: id 3 | task 13628 | processing task, is_child = 0 4.38.805.278 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.38.805.278 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.38.805.294 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.38.805.295 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.38.805.306 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.38.805.306 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.42.136.627 I slot print_timing: id 3 | task 13628 | n_decoded = 186, tg = 61.89 t/s, tg_3s = 61.89 t/s 4.45.144.766 I slot print_timing: id 3 | task 13628 | n_decoded = 370, tg = 61.53 t/s, tg_3s = 61.17 t/s 4.48.160.619 I slot print_timing: id 3 | task 13628 | n_decoded = 553, tg = 61.24 t/s, tg_3s = 60.68 t/s 4.51.160.709 I slot print_timing: id 3 | task 13628 | n_decoded = 734, tg = 61.02 t/s, tg_3s = 60.33 t/s 4.53.451.884 I slot print_timing: id 3 | task 13628 | prompt eval time = 325.77 ms / 34 tokens ( 9.58 ms per token, 104.37 tokens per second) 4.53.451.888 I slot print_timing: id 3 | task 13628 | eval time = 14320.79 ms / 872 tokens ( 16.42 ms per token, 60.89 tokens per second) 4.53.451.888 I slot print_timing: id 3 | task 13628 | total time = 14646.56 ms / 906 tokens 4.53.451.889 I slot print_timing: id 3 | task 13628 | graphs reused = 14337 4.53.451.923 I slot release: id 3 | task 13628 | stop processing: n_tokens = 1042, truncated = 0 4.53.451.935 I srv update_slots: all slots are idle 4.53.460.731 I srv operator(): Chat format: peg-native 4.53.464.799 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.787 (> 0.100 thold), f_keep = 0.131 4.53.464.801 I srv get_availabl: updating prompt cache 4.53.464.872 W srv prompt_save: - saving prompt with length 1042, total state size = 41.147 MiB (draft: 0.000 MiB) 4.53.510.231 I srv load: - looking for better prompt, base f_keep = 0.131, sim = 0.787 4.53.510.244 I srv update: - cache state: 16 prompts, 747.053 MiB (limits: 8192.000 MiB, 16384 tokens, 188018 est) 4.53.510.246 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 4.53.510.250 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 4.53.510.250 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 4.53.510.251 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 4.53.510.252 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 4.53.510.252 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 4.53.510.253 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 4.53.510.253 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 4.53.510.254 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 4.53.510.254 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 4.53.510.255 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 4.53.510.255 I srv update: - prompt 0x5e39f11a39f0: 644 tokens, checkpoints: 1, 31.652 MiB 4.53.510.256 I srv update: - prompt 0x5e39f1080b60: 520 tokens, checkpoints: 1, 27.291 MiB 4.53.510.259 I srv update: - prompt 0x5e39f1267390: 679 tokens, checkpoints: 1, 32.883 MiB 4.53.510.259 I srv update: - prompt 0x5e39f1142a10: 823 tokens, checkpoints: 1, 37.947 MiB 4.53.510.260 I srv update: - prompt 0x5e39f111bb90: 1042 tokens, checkpoints: 1, 45.649 MiB 4.53.510.261 I srv get_availabl: prompt cache update took 45.46 ms 4.53.511.413 I slot launch_slot_: id 3 | task 14502 | processing task, is_child = 0 4.53.511.416 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.53.511.417 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.53.511.434 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.53.511.434 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.53.511.447 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.53.511.448 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.53.511.468 I slot update_slots: id 3 | task 14502 | Checking checkpoint with [0, 133] against 9... 4.53.514.236 W slot update_slots: id 3 | task 14502 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 4.502 MiB) 4.56.942.008 I slot print_timing: id 3 | task 14502 | n_decoded = 188, tg = 62.33 t/s, tg_3s = 62.33 t/s 4.59.947.214 I slot print_timing: id 3 | task 14502 | n_decoded = 375, tg = 62.28 t/s, tg_3s = 62.23 t/s 5.02.950.678 I slot print_timing: id 3 | task 14502 | n_decoded = 562, tg = 62.27 t/s, tg_3s = 62.26 t/s 5.05.966.298 I slot print_timing: id 3 | task 14502 | n_decoded = 746, tg = 61.96 t/s, tg_3s = 61.02 t/s 5.08.971.838 I slot print_timing: id 3 | task 14502 | n_decoded = 929, tg = 61.74 t/s, tg_3s = 60.89 t/s 5.10.467.382 I slot print_timing: id 3 | task 14502 | prompt eval time = 414.56 ms / 41 tokens ( 10.11 ms per token, 98.90 tokens per second) 5.10.467.385 I slot print_timing: id 3 | task 14502 | eval time = 16541.35 ms / 1020 tokens ( 16.22 ms per token, 61.66 tokens per second) 5.10.467.386 I slot print_timing: id 3 | task 14502 | total time = 16955.91 ms / 1061 tokens 5.10.467.386 I slot print_timing: id 3 | task 14502 | graphs reused = 15348 5.10.467.418 I slot release: id 3 | task 14502 | stop processing: n_tokens = 1193, truncated = 0 5.10.467.431 I srv update_slots: all slots are idle 5.10.475.969 I srv operator(): Chat format: peg-native 5.10.479.836 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.835 (> 0.100 thold), f_keep = 0.115 5.10.479.838 I srv get_availabl: updating prompt cache 5.10.479.918 W srv prompt_save: - saving prompt with length 1193, total state size = 46.457 MiB (draft: 0.000 MiB) 5.10.522.044 I srv load: - looking for better prompt, base f_keep = 0.115, sim = 0.835 5.10.522.061 I srv update: - cache state: 17 prompts, 798.012 MiB (limits: 8192.000 MiB, 16384 tokens, 188259 est) 5.10.522.064 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 5.10.522.065 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 5.10.522.065 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 5.10.522.066 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 5.10.522.067 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 5.10.522.067 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 5.10.522.068 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 5.10.522.069 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 5.10.522.069 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 5.10.522.071 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 5.10.522.072 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 5.10.522.073 I srv update: - prompt 0x5e39f11a39f0: 644 tokens, checkpoints: 1, 31.652 MiB 5.10.522.073 I srv update: - prompt 0x5e39f1080b60: 520 tokens, checkpoints: 1, 27.291 MiB 5.10.522.074 I srv update: - prompt 0x5e39f1267390: 679 tokens, checkpoints: 1, 32.883 MiB 5.10.522.074 I srv update: - prompt 0x5e39f1142a10: 823 tokens, checkpoints: 1, 37.947 MiB 5.10.522.075 I srv update: - prompt 0x5e39f111bb90: 1042 tokens, checkpoints: 1, 45.649 MiB 5.10.522.076 I srv update: - prompt 0x5e39f1251a60: 1193 tokens, checkpoints: 1, 50.959 MiB 5.10.522.078 I srv get_availabl: prompt cache update took 42.24 ms 5.10.522.959 I slot launch_slot_: id 3 | task 15525 | processing task, is_child = 0 5.10.522.962 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 5.10.522.963 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 5.10.522.977 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 5.10.522.977 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 5.10.522.988 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 5.10.522.988 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 5.10.523.006 I slot update_slots: id 3 | task 15525 | Checking checkpoint with [0, 133] against 9... 5.10.525.556 W slot update_slots: id 3 | task 15525 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 4.502 MiB) 5.13.873.197 I slot print_timing: id 3 | task 15525 | n_decoded = 188, tg = 62.39 t/s, tg_3s = 62.39 t/s 5.16.887.172 I slot print_timing: id 3 | task 15525 | n_decoded = 375, tg = 62.22 t/s, tg_3s = 62.04 t/s 5.19.891.756 I slot print_timing: id 3 | task 15525 | n_decoded = 559, tg = 61.89 t/s, tg_3s = 61.24 t/s 5.22.895.293 I slot print_timing: id 3 | task 15525 | n_decoded = 741, tg = 61.57 t/s, tg_3s = 60.60 t/s 5.25.906.381 I slot print_timing: id 3 | task 15525 | n_decoded = 923, tg = 61.34 t/s, tg_3s = 60.44 t/s 5.27.353.959 I slot print_timing: id 3 | task 15525 | prompt eval time = 337.07 ms / 31 tokens ( 10.87 ms per token, 91.97 tokens per second) 5.27.353.963 I slot print_timing: id 3 | task 15525 | eval time = 16493.88 ms / 1010 tokens ( 16.33 ms per token, 61.23 tokens per second) 5.27.353.964 I slot print_timing: id 3 | task 15525 | total time = 16830.95 ms / 1041 tokens 5.27.353.964 I slot print_timing: id 3 | task 15525 | graphs reused = 16349 5.27.354.009 I slot release: id 3 | task 15525 | stop processing: n_tokens = 1173, truncated = 0 5.27.354.023 I srv update_slots: all slots are idle 5.27.363.187 I srv operator(): Chat format: peg-native 5.27.367.585 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.878 (> 0.100 thold), f_keep = 0.117 5.27.367.588 I srv get_availabl: updating prompt cache 5.27.367.681 W srv prompt_save: - saving prompt with length 1173, total state size = 45.754 MiB (draft: 0.000 MiB) 5.27.411.722 I srv load: - looking for better prompt, base f_keep = 0.117, sim = 0.878 5.27.411.736 I srv update: - cache state: 18 prompts, 848.268 MiB (limits: 8192.000 MiB, 16384 tokens, 188433 est) 5.27.411.737 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 5.27.411.737 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 5.27.411.738 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 5.27.411.738 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 5.27.411.739 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 5.27.411.739 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 5.27.411.739 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 5.27.411.740 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 5.27.411.740 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 5.27.411.741 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 5.27.411.741 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 5.27.411.742 I srv update: - prompt 0x5e39f11a39f0: 644 tokens, checkpoints: 1, 31.652 MiB 5.27.411.742 I srv update: - prompt 0x5e39f1080b60: 520 tokens, checkpoints: 1, 27.291 MiB 5.27.411.742 I srv update: - prompt 0x5e39f1267390: 679 tokens, checkpoints: 1, 32.883 MiB 5.27.411.743 I srv update: - prompt 0x5e39f1142a10: 823 tokens, checkpoints: 1, 37.947 MiB 5.27.411.743 I srv update: - prompt 0x5e39f111bb90: 1042 tokens, checkpoints: 1, 45.649 MiB 5.27.411.744 I srv update: - prompt 0x5e39f1251a60: 1193 tokens, checkpoints: 1, 50.959 MiB 5.27.411.744 I srv update: - prompt 0x5e39f13b7990: 1173 tokens, checkpoints: 1, 50.256 MiB 5.27.411.745 I srv get_availabl: prompt cache update took 44.16 ms 5.27.412.558 I slot launch_slot_: id 3 | task 16538 | processing task, is_child = 0 5.27.412.560 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 5.27.412.561 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 5.27.412.573 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 5.27.412.573 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 5.27.412.582 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 5.27.412.582 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 5.27.412.597 I slot update_slots: id 3 | task 16538 | Checking checkpoint with [0, 133] against 9... 5.27.415.154 W slot update_slots: id 3 | task 16538 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 4.502 MiB) 5.30.667.855 I slot print_timing: id 3 | task 16538 | n_decoded = 186, tg = 61.69 t/s, tg_3s = 61.69 t/s 5.33.670.057 I slot print_timing: id 3 | task 16538 | n_decoded = 369, tg = 61.32 t/s, tg_3s = 60.96 t/s 5.36.677.851 I slot print_timing: id 3 | task 16538 | n_decoded = 553, tg = 61.27 t/s, tg_3s = 61.17 t/s 5.39.681.424 I slot print_timing: id 3 | task 16538 | n_decoded = 733, tg = 60.94 t/s, tg_3s = 59.93 t/s 5.42.692.793 I slot print_timing: id 3 | task 16538 | n_decoded = 914, tg = 60.77 t/s, tg_3s = 60.11 t/s 5.45.706.760 I slot print_timing: id 3 | task 16538 | n_decoded = 1098, tg = 60.82 t/s, tg_3s = 61.05 t/s 5.46.098.792 I slot print_timing: id 3 | task 16538 | prompt eval time = 240.22 ms / 23 tokens ( 10.44 ms per token, 95.75 tokens per second) 5.46.098.795 I slot print_timing: id 3 | task 16538 | eval time = 18445.97 ms / 1122 tokens ( 16.44 ms per token, 60.83 tokens per second) 5.46.098.796 I slot print_timing: id 3 | task 16538 | total time = 18686.19 ms / 1145 tokens 5.46.098.796 I slot print_timing: id 3 | task 16538 | graphs reused = 17462 5.46.098.830 I slot release: id 3 | task 16538 | stop processing: n_tokens = 1277, truncated = 0 5.46.098.841 I srv update_slots: all slots are idle 5.46.107.542 I srv operator(): Chat format: peg-native 5.46.111.365 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.783 (> 0.100 thold), f_keep = 0.107 5.46.111.367 I srv get_availabl: updating prompt cache 5.46.111.425 W srv prompt_save: - saving prompt with length 1277, total state size = 49.411 MiB (draft: 0.000 MiB) 5.46.152.102 I srv load: - looking for better prompt, base f_keep = 0.107, sim = 0.783 5.46.152.116 I srv update: - cache state: 19 prompts, 902.181 MiB (limits: 8192.000 MiB, 16384 tokens, 188768 est) 5.46.152.117 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 5.46.152.117 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 5.46.152.118 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 5.46.152.118 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 5.46.152.119 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 5.46.152.119 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 5.46.152.120 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 5.46.152.120 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 5.46.152.121 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 5.46.152.121 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 5.46.152.122 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 5.46.152.122 I srv update: - prompt 0x5e39f11a39f0: 644 tokens, checkpoints: 1, 31.652 MiB 5.46.152.123 I srv update: - prompt 0x5e39f1080b60: 520 tokens, checkpoints: 1, 27.291 MiB 5.46.152.123 I srv update: - prompt 0x5e39f1267390: 679 tokens, checkpoints: 1, 32.883 MiB 5.46.152.123 I srv update: - prompt 0x5e39f1142a10: 823 tokens, checkpoints: 1, 37.947 MiB 5.46.152.124 I srv update: - prompt 0x5e39f111bb90: 1042 tokens, checkpoints: 1, 45.649 MiB 5.46.152.124 I srv update: - prompt 0x5e39f1251a60: 1193 tokens, checkpoints: 1, 50.959 MiB 5.46.152.125 I srv update: - prompt 0x5e39f13b7990: 1173 tokens, checkpoints: 1, 50.256 MiB 5.46.152.125 I srv update: - prompt 0x5e39f11765f0: 1277 tokens, checkpoints: 1, 53.913 MiB 5.46.152.126 I srv get_availabl: prompt cache update took 40.76 ms 5.46.153.004 I slot launch_slot_: id 3 | task 17663 | processing task, is_child = 0 5.46.153.006 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 5.46.153.006 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 5.46.153.018 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 5.46.153.018 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 5.46.153.027 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 5.46.153.028 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 5.46.153.043 I slot update_slots: id 3 | task 17663 | Checking checkpoint with [0, 133] against 9... 5.46.155.633 W slot update_slots: id 3 | task 17663 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 4.502 MiB) 5.49.596.585 I slot print_timing: id 3 | task 17663 | n_decoded = 188, tg = 62.36 t/s, tg_3s = 62.36 t/s 5.52.599.722 I slot print_timing: id 3 | task 17663 | n_decoded = 375, tg = 62.32 t/s, tg_3s = 62.27 t/s 5.54.279.739 I slot print_timing: id 3 | task 17663 | prompt eval time = 428.92 ms / 42 tokens ( 10.21 ms per token, 97.92 tokens per second) 5.54.279.742 I slot print_timing: id 3 | task 17663 | eval time = 7697.78 ms / 480 tokens ( 16.04 ms per token, 62.36 tokens per second) 5.54.279.743 I slot print_timing: id 3 | task 17663 | total time = 8126.69 ms / 522 tokens 5.54.279.743 I slot print_timing: id 3 | task 17663 | graphs reused = 17936 5.54.279.776 I slot release: id 3 | task 17663 | stop processing: n_tokens = 654, truncated = 0 5.54.279.784 I srv update_slots: all slots are idle 5.54.287.979 I srv operator(): Chat format: peg-native 5.54.291.441 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.835 (> 0.100 thold), f_keep = 0.209 5.54.291.443 I srv get_availabl: updating prompt cache 5.54.291.483 W srv prompt_save: - saving prompt with length 654, total state size = 27.502 MiB (draft: 0.000 MiB) 5.54.314.024 I srv load: - looking for better prompt, base f_keep = 0.209, sim = 0.835 5.54.314.041 I srv update: - cache state: 20 prompts, 934.185 MiB (limits: 8192.000 MiB, 16384 tokens, 188036 est) 5.54.314.042 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 5.54.314.042 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 5.54.314.043 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 5.54.314.043 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 5.54.314.044 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 5.54.314.044 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 5.54.314.044 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 5.54.314.045 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 5.54.314.045 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 5.54.314.046 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 5.54.314.046 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 5.54.314.047 I srv update: - prompt 0x5e39f11a39f0: 644 tokens, checkpoints: 1, 31.652 MiB 5.54.314.047 I srv update: - prompt 0x5e39f1080b60: 520 tokens, checkpoints: 1, 27.291 MiB 5.54.314.048 I srv update: - prompt 0x5e39f1267390: 679 tokens, checkpoints: 1, 32.883 MiB 5.54.314.048 I srv update: - prompt 0x5e39f1142a10: 823 tokens, checkpoints: 1, 37.947 MiB 5.54.314.048 I srv update: - prompt 0x5e39f111bb90: 1042 tokens, checkpoints: 1, 45.649 MiB 5.54.314.049 I srv update: - prompt 0x5e39f1251a60: 1193 tokens, checkpoints: 1, 50.959 MiB 5.54.314.049 I srv update: - prompt 0x5e39f13b7990: 1173 tokens, checkpoints: 1, 50.256 MiB 5.54.314.050 I srv update: - prompt 0x5e39f11765f0: 1277 tokens, checkpoints: 1, 53.913 MiB 5.54.314.050 I srv update: - prompt 0x5e39f127cbc0: 654 tokens, checkpoints: 1, 32.004 MiB 5.54.314.051 I srv get_availabl: prompt cache update took 22.61 ms 5.54.314.912 I slot launch_slot_: id 3 | task 18146 | processing task, is_child = 0 5.54.314.914 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 5.54.314.915 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 5.54.314.929 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 5.54.314.930 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 5.54.314.938 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 5.54.314.938 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 5.57.579.590 I slot print_timing: id 3 | task 18146 | n_decoded = 188, tg = 62.45 t/s, tg_3s = 62.45 t/s 6.00.591.798 I slot print_timing: id 3 | task 18146 | n_decoded = 375, tg = 62.26 t/s, tg_3s = 62.08 t/s 6.03.597.067 I slot print_timing: id 3 | task 18146 | n_decoded = 561, tg = 62.14 t/s, tg_3s = 61.89 t/s 6.06.597.936 I slot print_timing: id 3 | task 18146 | n_decoded = 744, tg = 61.85 t/s, tg_3s = 60.98 t/s 6.09.606.573 I slot print_timing: id 3 | task 18146 | n_decoded = 927, tg = 61.65 t/s, tg_3s = 60.82 t/s 6.12.614.002 I slot print_timing: id 3 | task 18146 | n_decoded = 1109, tg = 61.46 t/s, tg_3s = 60.52 t/s 6.15.626.011 I slot print_timing: id 3 | task 18146 | n_decoded = 1291, tg = 61.31 t/s, tg_3s = 60.42 t/s 6.18.636.857 I slot print_timing: id 3 | task 18146 | n_decoded = 1472, tg = 61.16 t/s, tg_3s = 60.12 t/s 6.21.650.857 I slot print_timing: id 3 | task 18146 | n_decoded = 1652, tg = 61.00 t/s, tg_3s = 59.72 t/s 6.22.432.871 I slot print_timing: id 3 | task 18146 | prompt eval time = 254.01 ms / 27 tokens ( 9.41 ms per token, 106.30 tokens per second) 6.22.432.874 I slot print_timing: id 3 | task 18146 | eval time = 27863.90 ms / 1698 tokens ( 16.41 ms per token, 60.94 tokens per second) 6.22.432.875 I slot print_timing: id 3 | task 18146 | total time = 28117.91 ms / 1725 tokens 6.22.432.875 I slot print_timing: id 3 | task 18146 | graphs reused = 19622 6.22.432.912 I slot release: id 3 | task 18146 | stop processing: n_tokens = 1861, truncated = 0 6.22.432.928 I srv update_slots: all slots are idle 6.22.441.564 I srv operator(): Chat format: peg-native 6.22.445.377 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.811 (> 0.100 thold), f_keep = 0.074 6.22.445.379 I srv get_availabl: updating prompt cache 6.22.445.480 W srv prompt_save: - saving prompt with length 1861, total state size = 69.949 MiB (draft: 0.000 MiB) 6.22.508.154 I srv load: - looking for better prompt, base f_keep = 0.074, sim = 0.811 6.22.508.169 I srv update: - cache state: 21 prompts, 1008.637 MiB (limits: 8192.000 MiB, 16384 tokens, 189271 est) 6.22.508.170 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 6.22.508.171 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 6.22.508.171 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 6.22.508.172 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 6.22.508.172 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 6.22.508.173 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 6.22.508.173 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 6.22.508.174 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 6.22.508.174 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 6.22.508.174 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 6.22.508.175 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 6.22.508.176 I srv update: - prompt 0x5e39f11a39f0: 644 tokens, checkpoints: 1, 31.652 MiB 6.22.508.176 I srv update: - prompt 0x5e39f1080b60: 520 tokens, checkpoints: 1, 27.291 MiB 6.22.508.177 I srv update: - prompt 0x5e39f1267390: 679 tokens, checkpoints: 1, 32.883 MiB 6.22.508.177 I srv update: - prompt 0x5e39f1142a10: 823 tokens, checkpoints: 1, 37.947 MiB 6.22.508.178 I srv update: - prompt 0x5e39f111bb90: 1042 tokens, checkpoints: 1, 45.649 MiB 6.22.508.178 I srv update: - prompt 0x5e39f1251a60: 1193 tokens, checkpoints: 1, 50.959 MiB 6.22.508.178 I srv update: - prompt 0x5e39f13b7990: 1173 tokens, checkpoints: 1, 50.256 MiB 6.22.508.179 I srv update: - prompt 0x5e39f11765f0: 1277 tokens, checkpoints: 1, 53.913 MiB 6.22.508.179 I srv update: - prompt 0x5e39f127cbc0: 654 tokens, checkpoints: 1, 32.004 MiB 6.22.508.180 I srv update: - prompt 0x5e39f116eb10: 1861 tokens, checkpoints: 1, 74.451 MiB 6.22.508.181 I srv get_availabl: prompt cache update took 62.80 ms 6.22.509.113 I slot launch_slot_: id 3 | task 19846 | processing task, is_child = 0 6.22.509.115 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 6.22.509.116 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 6.22.509.136 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 6.22.509.136 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 6.22.509.150 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 6.22.509.150 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 6.22.509.170 I slot update_slots: id 3 | task 19846 | Checking checkpoint with [0, 133] against 9... 6.22.511.851 W slot update_slots: id 3 | task 19846 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 4.502 MiB) 6.25.862.822 I slot print_timing: id 3 | task 19846 | n_decoded = 186, tg = 61.92 t/s, tg_3s = 61.92 t/s 6.28.872.941 I slot print_timing: id 3 | task 19846 | n_decoded = 370, tg = 61.52 t/s, tg_3s = 61.13 t/s 6.31.877.852 I slot print_timing: id 3 | task 19846 | n_decoded = 552, tg = 61.20 t/s, tg_3s = 60.57 t/s 6.34.889.655 I slot print_timing: id 3 | task 19846 | n_decoded = 734, tg = 61.01 t/s, tg_3s = 60.43 t/s 6.37.897.686 I slot print_timing: id 3 | task 19846 | n_decoded = 917, tg = 60.98 t/s, tg_3s = 60.84 t/s 6.40.913.631 I slot print_timing: id 3 | task 19846 | n_decoded = 1101, tg = 60.98 t/s, tg_3s = 61.01 t/s 6.43.927.933 I slot print_timing: id 3 | task 19846 | n_decoded = 1283, tg = 60.89 t/s, tg_3s = 60.38 t/s 6.44.425.111 I slot print_timing: id 3 | task 19846 | prompt eval time = 349.57 ms / 36 tokens ( 9.71 ms per token, 102.98 tokens per second) 6.44.425.115 I slot print_timing: id 3 | task 19846 | eval time = 21566.37 ms / 1313 tokens ( 16.43 ms per token, 60.88 tokens per second) 6.44.425.116 I slot print_timing: id 3 | task 19846 | total time = 21915.94 ms / 1349 tokens 6.44.425.116 I slot print_timing: id 3 | task 19846 | graphs reused = 20925 6.44.425.186 I slot release: id 3 | task 19846 | stop processing: n_tokens = 1481, truncated = 0 6.44.425.201 I srv update_slots: all slots are idle 6.44.434.083 I srv operator(): Chat format: peg-native 6.44.438.258 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.770 (> 0.100 thold), f_keep = 0.093 6.44.438.261 I srv get_availabl: updating prompt cache 6.44.438.351 W srv prompt_save: - saving prompt with length 1481, total state size = 56.586 MiB (draft: 0.000 MiB) 6.44.489.360 I srv load: - looking for better prompt, base f_keep = 0.093, sim = 0.770 6.44.489.375 I srv update: - cache state: 22 prompts, 1069.724 MiB (limits: 8192.000 MiB, 16384 tokens, 189804 est) 6.44.489.376 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 6.44.489.377 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 6.44.489.377 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 6.44.489.378 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 6.44.489.378 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 6.44.489.379 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 6.44.489.379 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 6.44.489.379 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 6.44.489.380 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 6.44.489.380 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 6.44.489.381 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 6.44.489.381 I srv update: - prompt 0x5e39f11a39f0: 644 tokens, checkpoints: 1, 31.652 MiB 6.44.489.382 I srv update: - prompt 0x5e39f1080b60: 520 tokens, checkpoints: 1, 27.291 MiB 6.44.489.382 I srv update: - prompt 0x5e39f1267390: 679 tokens, checkpoints: 1, 32.883 MiB 6.44.489.382 I srv update: - prompt 0x5e39f1142a10: 823 tokens, checkpoints: 1, 37.947 MiB 6.44.489.383 I srv update: - prompt 0x5e39f111bb90: 1042 tokens, checkpoints: 1, 45.649 MiB 6.44.489.383 I srv update: - prompt 0x5e39f1251a60: 1193 tokens, checkpoints: 1, 50.959 MiB 6.44.489.384 I srv update: - prompt 0x5e39f13b7990: 1173 tokens, checkpoints: 1, 50.256 MiB 6.44.489.384 I srv update: - prompt 0x5e39f11765f0: 1277 tokens, checkpoints: 1, 53.913 MiB 6.44.489.385 I srv update: - prompt 0x5e39f127cbc0: 654 tokens, checkpoints: 1, 32.004 MiB 6.44.489.385 I srv update: - prompt 0x5e39f116eb10: 1861 tokens, checkpoints: 1, 74.451 MiB 6.44.489.386 I srv update: - prompt 0x5e39f10a1bc0: 1481 tokens, checkpoints: 1, 61.088 MiB 6.44.489.386 I srv get_availabl: prompt cache update took 51.12 ms 6.44.490.228 I slot launch_slot_: id 3 | task 21162 | processing task, is_child = 0 6.44.490.231 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 6.44.490.232 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 6.44.490.244 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 6.44.490.244 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 6.44.490.253 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 6.44.490.253 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 6.44.490.266 I slot update_slots: id 3 | task 21162 | Checking checkpoint with [0, 133] against 9... 6.44.492.808 W slot update_slots: id 3 | task 21162 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 4.502 MiB) 6.47.938.961 I slot print_timing: id 3 | task 21162 | n_decoded = 188, tg = 62.44 t/s, tg_3s = 62.44 t/s 6.50.940.809 I slot print_timing: id 3 | task 21162 | n_decoded = 374, tg = 62.20 t/s, tg_3s = 61.96 t/s 6.53.952.559 I slot print_timing: id 3 | task 21162 | n_decoded = 559, tg = 61.94 t/s, tg_3s = 61.43 t/s 6.56.712.611 I slot print_timing: id 3 | task 21162 | prompt eval time = 437.85 ms / 45 tokens ( 9.73 ms per token, 102.78 tokens per second) 6.56.712.614 I slot print_timing: id 3 | task 21162 | eval time = 11784.49 ms / 726 tokens ( 16.23 ms per token, 61.61 tokens per second) 6.56.712.615 I slot print_timing: id 3 | task 21162 | total time = 12222.34 ms / 771 tokens 6.56.712.616 I slot print_timing: id 3 | task 21162 | graphs reused = 21643 6.56.712.648 I slot release: id 3 | task 21162 | stop processing: n_tokens = 903, truncated = 0 6.56.712.659 I srv update_slots: all slots are idle 6.56.721.345 I srv operator(): Chat format: peg-native 6.56.725.327 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.835 (> 0.100 thold), f_keep = 0.152 6.56.725.329 I srv get_availabl: updating prompt cache 6.56.725.391 W srv prompt_save: - saving prompt with length 903, total state size = 36.259 MiB (draft: 0.000 MiB) 6.56.755.633 I srv load: - looking for better prompt, base f_keep = 0.152, sim = 0.835 6.56.755.652 I srv update: - cache state: 23 prompts, 1110.485 MiB (limits: 8192.000 MiB, 16384 tokens, 189499 est) 6.56.755.656 I srv update: - prompt 0x5e39f714ab50: 1518 tokens, checkpoints: 1, 62.389 MiB 6.56.755.657 I srv update: - prompt 0x5e39f781ae10: 881 tokens, checkpoints: 1, 39.987 MiB 6.56.755.657 I srv update: - prompt 0x5e39f76b38e0: 740 tokens, checkpoints: 1, 35.028 MiB 6.56.755.658 I srv update: - prompt 0x5e39f781c9a0: 1688 tokens, checkpoints: 1, 68.367 MiB 6.56.755.670 I srv update: - prompt 0x5e39f76b78b0: 2363 tokens, checkpoints: 1, 92.105 MiB 6.56.755.671 I srv update: - prompt 0x5e39f136e070: 1995 tokens, checkpoints: 1, 79.164 MiB 6.56.755.671 I srv update: - prompt 0x5e39f10cb540: 1012 tokens, checkpoints: 1, 44.594 MiB 6.56.755.672 I srv update: - prompt 0x5e39f127a3a0: 1016 tokens, checkpoints: 1, 44.735 MiB 6.56.755.672 I srv update: - prompt 0x5e39f128d1c0: 747 tokens, checkpoints: 1, 35.274 MiB 6.56.755.673 I srv update: - prompt 0x5e39f117b700: 581 tokens, checkpoints: 1, 29.437 MiB 6.56.755.673 I srv update: - prompt 0x5e39f13b8c60: 897 tokens, checkpoints: 1, 40.550 MiB 6.56.755.674 I srv update: - prompt 0x5e39f11a39f0: 644 tokens, checkpoints: 1, 31.652 MiB 6.56.755.674 I srv update: - prompt 0x5e39f1080b60: 520 tokens, checkpoints: 1, 27.291 MiB 6.56.755.675 I srv update: - prompt 0x5e39f1267390: 679 tokens, checkpoints: 1, 32.883 MiB 6.56.755.675 I srv update: - prompt 0x5e39f1142a10: 823 tokens, checkpoints: 1, 37.947 MiB 6.56.755.676 I srv update: - prompt 0x5e39f111bb90: 1042 tokens, checkpoints: 1, 45.649 MiB 6.56.755.676 I srv update: - prompt 0x5e39f1251a60: 1193 tokens, checkpoints: 1, 50.959 MiB 6.56.755.677 I srv update: - prompt 0x5e39f13b7990: 1173 tokens, checkpoints: 1, 50.256 MiB 6.56.755.677 I srv update: - prompt 0x5e39f11765f0: 1277 tokens, checkpoints: 1, 53.913 MiB 6.56.755.678 I srv update: - prompt 0x5e39f127cbc0: 654 tokens, checkpoints: 1, 32.004 MiB 6.56.755.678 I srv update: - prompt 0x5e39f116eb10: 1861 tokens, checkpoints: 1, 74.451 MiB 6.56.755.679 I srv update: - prompt 0x5e39f10a1bc0: 1481 tokens, checkpoints: 1, 61.088 MiB 6.56.755.679 I srv update: - prompt 0x5e39f1176810: 903 tokens, checkpoints: 1, 40.761 MiB 6.56.755.680 I srv get_availabl: prompt cache update took 30.35 ms 6.56.756.618 I slot launch_slot_: id 3 | task 21891 | processing task, is_child = 0 6.56.756.620 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 6.56.756.621 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 6.56.756.637 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 6.56.756.637 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 6.56.756.647 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 6.56.756.647 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 7.00.010.411 I slot print_timing: id 3 | task 21891 | n_decoded = 187, tg = 62.20 t/s, tg_3s = 62.20 t/s 7.03.011.106 I slot print_timing: id 3 | task 21891 | n_decoded = 372, tg = 61.93 t/s, tg_3s = 61.65 t/s 7.06.015.493 I slot print_timing: id 3 | task 21891 | n_decoded = 556, tg = 61.70 t/s, tg_3s = 61.24 t/s 7.09.024.655 I slot print_timing: id 3 | task 21891 | n_decoded = 738, tg = 61.39 t/s, tg_3s = 60.48 t/s 7.12.038.069 I slot print_timing: id 3 | task 21891 | n_decoded = 920, tg = 61.19 t/s, tg_3s = 60.40 t/s 7.15.041.682 I slot print_timing: id 3 | task 21891 | n_decoded = 1101, tg = 61.04 t/s, tg_3s = 60.26 t/s 7.17.541.908 I slot print_timing: id 3 | task 21891 | prompt eval time = 247.33 ms / 27 tokens ( 9.16 ms per token, 109.17 tokens per second) 7.17.541.911 I slot print_timing: id 3 | task 21891 | eval time = 20537.91 ms / 1252 tokens ( 16.40 ms per token, 60.96 tokens per second) 7.17.541.912 I slot print_timing: id 3 | task 21891 | total time = 20785.24 ms / 1279 tokens 7.17.541.912 I slot print_timing: id 3 | task 21891 | graphs reused = 22885 7.17.541.945 I slot release: id 3 | task 21891 | stop processing: n_tokens = 1415, truncated = 0 7.17.541.957 I srv update_slots: all slots are idle 7.17.542.641 I srv operator(): operator(): cleaning up before exit...