0.00.067.106 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.067.109 I device_info: 0.00.067.127 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.067.132 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.067.138 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free) 0.00.067.191 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.067.195 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true 0.00.067.235 I srv init: running without SSL 0.00.067.296 I srv init: using 23 threads for HTTP server 0.00.067.618 I srv start: binding port with default address family 0.00.068.833 I srv llama_server: loading model 0.00.068.840 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/openai_gpt-oss-20b-Q5_K_M.gguf' 0.00.068.895 I common_init_result: fitting params to device memory ... 0.00.068.896 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) 0.01.552.532 W load: setting token '<|message|>' (200008) attribute to USER_DEFINED (16), old attributes: 8 0.01.552.538 W load: setting token '<|start|>' (200006) attribute to USER_DEFINED (16), old attributes: 8 0.01.552.539 W load: setting token '<|constrain|>' (200003) attribute to USER_DEFINED (16), old attributes: 8 0.01.553.701 W load: setting token '<|channel|>' (200005) attribute to USER_DEFINED (16), old attributes: 8 0.01.558.807 W load: special_eog_ids contains both '<|return|>' and '<|call|>', or '<|calls|>' and '<|flush|>' tokens, removing '<|end|>' token from EOG list 0.08.713.868 W llama_context: n_ctx_seq (16384) < n_ctx_train (131072) -- the full capacity of the model will not be utilized 0.08.776.024 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) 0.08.944.476 I srv load_model: initializing slots, n_slots = 4 0.08.977.444 W common_speculative_init: no implementations specified for speculative decoding 0.08.977.449 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 0.08.977.454 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384 0.08.977.455 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384 0.08.977.455 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384 0.08.977.507 I srv load_model: prompt cache is enabled, size limit: 8192 MiB 0.08.977.508 I srv load_model: use `--cache-ram 0` to disable the prompt cache 0.08.977.508 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 0.08.977.508 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 0.08.977.527 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task 0.08.981.123 I init: chat template, example_format: '<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI. Knowledge cutoff: 2024-06 Current date: 2026-07-12 Reasoning: medium # Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>developer<|message|># Instructions You are a helpful assistant<|end|><|start|>user<|message|>Hello<|end|><|start|>assistant<|channel|>final<|message|>Hi there<|end|><|start|>user<|message|>How are you?<|end|><|start|>assistant' 0.08.981.693 I srv init: init: chat template, thinking = 1 0.08.981.728 I srv llama_server: model loaded 0.08.981.731 I srv llama_server: server is listening on http://127.0.0.1:8090 0.08.981.736 I srv update_slots: all slots are idle 0.09.971.610 I srv operator(): Chat format: peg-native 0.09.975.555 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1 0.09.975.557 I srv get_availabl: updating prompt cache 0.09.975.561 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 0.09.975.564 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) 0.09.975.565 I srv get_availabl: prompt cache update took 0.01 ms 0.09.976.456 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0 0.09.976.458 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.09.976.460 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.09.976.472 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.09.976.473 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.09.976.482 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.09.976.482 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.10.664.858 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 133, n_tokens = 134, size = 3.002 MiB) 0.13.779.890 I slot print_timing: id 3 | task 0 | n_decoded = 271, tg = 90.23 t/s, tg_3s = 90.23 t/s 0.16.782.280 I slot print_timing: id 3 | task 0 | n_decoded = 533, tg = 88.75 t/s, tg_3s = 87.26 t/s 0.19.790.794 I slot print_timing: id 3 | task 0 | n_decoded = 794, tg = 88.08 t/s, tg_3s = 86.75 t/s 0.22.801.683 I slot print_timing: id 3 | task 0 | n_decoded = 1056, tg = 87.82 t/s, tg_3s = 87.02 t/s 0.25.812.494 I slot print_timing: id 3 | task 0 | n_decoded = 1317, tg = 87.59 t/s, tg_3s = 86.69 t/s 0.28.297.013 I slot print_timing: id 3 | task 0 | prompt eval time = 800.10 ms / 159 tokens ( 5.03 ms per token, 198.72 tokens per second) 0.28.297.017 I slot print_timing: id 3 | task 0 | eval time = 17520.40 ms / 1530 tokens ( 11.45 ms per token, 87.33 tokens per second) 0.28.297.018 I slot print_timing: id 3 | task 0 | total time = 18320.50 ms / 1689 tokens 0.28.297.021 I slot print_timing: id 3 | task 0 | graphs reused = 1523 0.28.297.072 I slot release: id 3 | task 0 | stop processing: n_tokens = 1688, truncated = 0 0.28.297.081 I srv update_slots: all slots are idle 0.28.305.739 I srv operator(): Chat format: peg-native 0.28.309.829 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.895 (> 0.100 thold), f_keep = 0.081 0.28.309.831 I srv get_availabl: updating prompt cache 0.28.309.916 W srv prompt_save: - saving prompt with length 1688, total state size = 42.584 MiB (draft: 0.000 MiB) 0.28.342.596 I srv load: - looking for better prompt, base f_keep = 0.081, sim = 0.895 0.28.342.603 I srv update: - cache state: 1 prompts, 45.586 MiB (limits: 8192.000 MiB, 16384 tokens, 303343 est) 0.28.342.604 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 0.28.342.604 I srv get_availabl: prompt cache update took 32.77 ms 0.28.343.531 I slot launch_slot_: id 3 | task 1533 | processing task, is_child = 0 0.28.343.534 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.28.343.534 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.28.343.547 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.28.343.547 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.28.343.557 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.28.343.558 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.28.343.574 I slot update_slots: id 3 | task 1533 | Checking checkpoint with [0, 133] against 9... 0.28.345.409 W slot update_slots: id 3 | task 1533 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 3.002 MiB) 0.31.451.329 I slot print_timing: id 3 | task 1533 | n_decoded = 268, tg = 89.08 t/s, tg_3s = 89.08 t/s 0.32.626.384 I slot print_timing: id 3 | task 1533 | prompt eval time = 99.15 ms / 20 tokens ( 4.96 ms per token, 201.72 tokens per second) 0.32.626.387 I slot print_timing: id 3 | task 1533 | eval time = 4183.66 ms / 371 tokens ( 11.28 ms per token, 88.68 tokens per second) 0.32.626.388 I slot print_timing: id 3 | task 1533 | total time = 4282.80 ms / 391 tokens 0.32.626.389 I slot print_timing: id 3 | task 1533 | graphs reused = 1888 0.32.626.420 I slot release: id 3 | task 1533 | stop processing: n_tokens = 523, truncated = 0 0.32.626.429 I srv update_slots: all slots are idle 0.32.634.989 I srv operator(): Chat format: peg-native 0.32.639.007 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.895 (> 0.100 thold), f_keep = 0.262 0.32.639.008 I srv get_availabl: updating prompt cache 0.32.639.052 W srv prompt_save: - saving prompt with length 523, total state size = 15.266 MiB (draft: 0.000 MiB) 0.32.648.012 I srv load: - looking for better prompt, base f_keep = 0.262, sim = 0.895 0.32.648.020 I srv update: - cache state: 2 prompts, 63.853 MiB (limits: 8192.000 MiB, 16384 tokens, 283658 est) 0.32.648.021 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 0.32.648.022 I srv update: - prompt 0x64ea8007b040: 523 tokens, checkpoints: 1, 18.268 MiB 0.32.648.023 I srv get_availabl: prompt cache update took 9.01 ms 0.32.648.956 I slot launch_slot_: id 3 | task 1907 | processing task, is_child = 0 0.32.648.958 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.32.648.959 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.32.648.973 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.32.648.973 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.32.648.983 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.32.648.983 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.35.733.368 I slot print_timing: id 3 | task 1907 | n_decoded = 266, tg = 88.38 t/s, tg_3s = 88.38 t/s 0.36.469.975 I slot print_timing: id 3 | task 1907 | prompt eval time = 74.65 ms / 16 tokens ( 4.67 ms per token, 214.33 tokens per second) 0.36.469.979 I slot print_timing: id 3 | task 1907 | eval time = 3746.32 ms / 331 tokens ( 11.32 ms per token, 88.35 tokens per second) 0.36.469.979 I slot print_timing: id 3 | task 1907 | total time = 3820.97 ms / 347 tokens 0.36.469.980 I slot print_timing: id 3 | task 1907 | graphs reused = 2215 0.36.470.011 I slot release: id 3 | task 1907 | stop processing: n_tokens = 483, truncated = 0 0.36.470.020 I srv update_slots: all slots are idle 0.36.478.362 I srv operator(): Chat format: peg-native 0.36.482.470 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.901 (> 0.100 thold), f_keep = 0.284 0.36.482.471 I srv get_availabl: updating prompt cache 0.36.482.505 W srv prompt_save: - saving prompt with length 483, total state size = 14.328 MiB (draft: 0.000 MiB) 0.36.495.925 I srv load: - looking for better prompt, base f_keep = 0.284, sim = 0.901 0.36.495.932 I srv update: - cache state: 3 prompts, 81.183 MiB (limits: 8192.000 MiB, 16384 tokens, 271846 est) 0.36.495.933 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 0.36.495.933 I srv update: - prompt 0x64ea8007b040: 523 tokens, checkpoints: 1, 18.268 MiB 0.36.495.934 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 0.36.495.935 I srv get_availabl: prompt cache update took 13.46 ms 0.36.496.876 I slot launch_slot_: id 3 | task 2240 | processing task, is_child = 0 0.36.496.878 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.36.496.879 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.36.496.895 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.36.496.896 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.36.496.905 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.36.496.905 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.39.580.878 I slot print_timing: id 3 | task 2240 | n_decoded = 266, tg = 88.63 t/s, tg_3s = 88.63 t/s 0.42.589.376 I slot print_timing: id 3 | task 2240 | n_decoded = 529, tg = 88.02 t/s, tg_3s = 87.42 t/s 0.45.600.029 I slot print_timing: id 3 | task 2240 | n_decoded = 787, tg = 87.25 t/s, tg_3s = 85.70 t/s 0.48.606.076 I slot print_timing: id 3 | task 2240 | n_decoded = 1043, tg = 86.73 t/s, tg_3s = 85.16 t/s 0.51.612.649 I slot print_timing: id 3 | task 2240 | n_decoded = 1299, tg = 86.41 t/s, tg_3s = 85.15 t/s 0.54.620.698 I slot print_timing: id 3 | task 2240 | n_decoded = 1553, tg = 86.08 t/s, tg_3s = 84.44 t/s 0.54.904.690 I slot print_timing: id 3 | task 2240 | prompt eval time = 82.76 ms / 15 tokens ( 5.52 ms per token, 181.24 tokens per second) 0.54.904.693 I slot print_timing: id 3 | task 2240 | eval time = 18325.01 ms / 1577 tokens ( 11.62 ms per token, 86.06 tokens per second) 0.54.904.694 I slot print_timing: id 3 | task 2240 | total time = 18407.77 ms / 1592 tokens 0.54.904.695 I slot print_timing: id 3 | task 2240 | graphs reused = 3781 0.54.904.728 I slot release: id 3 | task 2240 | stop processing: n_tokens = 1728, truncated = 0 0.54.904.742 I srv update_slots: all slots are idle 0.54.913.035 I srv operator(): Chat format: peg-native 0.54.916.868 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.830 (> 0.100 thold), f_keep = 0.079 0.54.916.870 I srv get_availabl: updating prompt cache 0.54.916.942 W srv prompt_save: - saving prompt with length 1728, total state size = 43.522 MiB (draft: 0.000 MiB) 0.54.968.788 I srv load: - looking for better prompt, base f_keep = 0.079, sim = 0.830 0.54.968.793 I srv load: - found better prompt with f_keep = 0.264, sim = 0.836 0.54.975.031 I srv update: - cache state: 3 prompts, 109.439 MiB (limits: 8192.000 MiB, 16384 tokens, 291857 est) 0.54.975.035 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 0.54.975.035 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 0.54.975.036 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 0.54.975.037 I srv get_availabl: prompt cache update took 58.17 ms 0.54.975.965 I slot launch_slot_: id 3 | task 3819 | processing task, is_child = 0 0.54.975.967 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.54.975.968 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.54.975.983 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.54.975.983 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.54.975.992 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.54.975.993 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.54.976.008 I slot update_slots: id 3 | task 3819 | Checking checkpoint with [0, 133] against 10... 0.54.977.976 W slot update_slots: id 3 | task 3819 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 3.002 MiB) 0.58.126.024 I slot print_timing: id 3 | task 3819 | n_decoded = 262, tg = 87.02 t/s, tg_3s = 87.02 t/s 1.01.126.794 I slot print_timing: id 3 | task 3819 | n_decoded = 520, tg = 86.50 t/s, tg_3s = 85.98 t/s 1.04.130.111 I slot print_timing: id 3 | task 3819 | n_decoded = 775, tg = 85.97 t/s, tg_3s = 84.91 t/s 1.07.133.964 I slot print_timing: id 3 | task 3819 | n_decoded = 1031, tg = 85.78 t/s, tg_3s = 85.22 t/s 1.10.138.551 I slot print_timing: id 3 | task 3819 | n_decoded = 1283, tg = 85.40 t/s, tg_3s = 83.87 t/s 1.13.144.033 I slot print_timing: id 3 | task 3819 | n_decoded = 1539, tg = 85.36 t/s, tg_3s = 85.18 t/s 1.16.151.948 I slot print_timing: id 3 | task 3819 | n_decoded = 1796, tg = 85.37 t/s, tg_3s = 85.44 t/s 1.16.938.196 I slot print_timing: id 3 | task 3819 | prompt eval time = 139.05 ms / 32 tokens ( 4.35 ms per token, 230.13 tokens per second) 1.16.938.198 I slot print_timing: id 3 | task 3819 | eval time = 21823.13 ms / 1863 tokens ( 11.71 ms per token, 85.37 tokens per second) 1.16.938.199 I slot print_timing: id 3 | task 3819 | total time = 21962.18 ms / 1895 tokens 1.16.938.200 I slot print_timing: id 3 | task 3819 | graphs reused = 5632 1.16.938.229 I slot release: id 3 | task 3819 | stop processing: n_tokens = 2027, truncated = 0 1.16.938.243 I srv update_slots: all slots are idle 1.16.946.722 I srv operator(): Chat format: peg-native 1.16.950.617 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.890 (> 0.100 thold), f_keep = 0.068 1.16.950.619 I srv get_availabl: updating prompt cache 1.16.950.700 W srv prompt_save: - saving prompt with length 2027, total state size = 50.533 MiB (draft: 0.000 MiB) 1.17.022.171 I srv load: - looking for better prompt, base f_keep = 0.068, sim = 0.890 1.17.022.179 I srv update: - cache state: 4 prompts, 162.974 MiB (limits: 8192.000 MiB, 16384 tokens, 297875 est) 1.17.022.180 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 1.17.022.181 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 1.17.022.181 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 1.17.022.182 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 1.17.022.183 I srv get_availabl: prompt cache update took 71.56 ms 1.17.023.036 I slot launch_slot_: id 3 | task 5685 | processing task, is_child = 0 1.17.023.038 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.17.023.038 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.17.023.052 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.17.023.052 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.17.023.061 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.17.023.061 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.17.023.077 I slot update_slots: id 3 | task 5685 | Checking checkpoint with [0, 133] against 9... 1.17.025.181 W slot update_slots: id 3 | task 5685 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 3.002 MiB) 1.20.123.040 I slot print_timing: id 3 | task 5685 | n_decoded = 269, tg = 89.57 t/s, tg_3s = 89.57 t/s 1.23.124.766 I slot print_timing: id 3 | task 5685 | n_decoded = 531, tg = 88.43 t/s, tg_3s = 87.28 t/s 1.26.132.187 I slot print_timing: id 3 | task 5685 | n_decoded = 791, tg = 87.77 t/s, tg_3s = 86.45 t/s 1.29.132.937 I slot print_timing: id 3 | task 5685 | n_decoded = 1049, tg = 87.32 t/s, tg_3s = 85.98 t/s 1.32.140.205 I slot print_timing: id 3 | task 5685 | n_decoded = 1306, tg = 86.95 t/s, tg_3s = 85.46 t/s 1.32.562.719 I slot print_timing: id 3 | task 5685 | prompt eval time = 96.70 ms / 21 tokens ( 4.60 ms per token, 217.16 tokens per second) 1.32.562.723 I slot print_timing: id 3 | task 5685 | eval time = 15442.94 ms / 1342 tokens ( 11.51 ms per token, 86.90 tokens per second) 1.32.562.724 I slot print_timing: id 3 | task 5685 | total time = 15539.64 ms / 1363 tokens 1.32.562.724 I slot print_timing: id 3 | task 5685 | graphs reused = 6964 1.32.562.757 I slot release: id 3 | task 5685 | stop processing: n_tokens = 1495, truncated = 0 1.32.562.768 I srv update_slots: all slots are idle 1.32.571.297 I srv operator(): Chat format: peg-native 1.32.575.190 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.920 (> 0.100 thold), f_keep = 0.092 1.32.575.193 I srv get_availabl: updating prompt cache 1.32.575.271 W srv prompt_save: - saving prompt with length 1495, total state size = 38.058 MiB (draft: 0.000 MiB) 1.32.630.039 I srv load: - looking for better prompt, base f_keep = 0.092, sim = 0.920 1.32.630.047 I srv update: - cache state: 5 prompts, 204.034 MiB (limits: 8192.000 MiB, 16384 tokens, 297954 est) 1.32.630.048 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 1.32.630.049 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 1.32.630.050 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 1.32.630.050 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 1.32.630.051 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 1.32.630.051 I srv get_availabl: prompt cache update took 54.86 ms 1.32.631.013 I slot launch_slot_: id 3 | task 7030 | processing task, is_child = 0 1.32.631.016 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.32.631.016 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.32.631.029 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.32.631.029 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.32.631.040 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.32.631.040 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.32.631.056 I slot update_slots: id 3 | task 7030 | Checking checkpoint with [0, 133] against 10... 1.32.633.307 W slot update_slots: id 3 | task 7030 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 3.002 MiB) 1.35.726.048 I slot print_timing: id 3 | task 7030 | n_decoded = 263, tg = 87.59 t/s, tg_3s = 87.59 t/s 1.37.391.128 I slot print_timing: id 3 | task 7030 | prompt eval time = 92.41 ms / 17 tokens ( 5.44 ms per token, 183.97 tokens per second) 1.37.391.131 I slot print_timing: id 3 | task 7030 | eval time = 4667.66 ms / 407 tokens ( 11.47 ms per token, 87.20 tokens per second) 1.37.391.132 I slot print_timing: id 3 | task 7030 | total time = 4760.07 ms / 424 tokens 1.37.391.133 I slot print_timing: id 3 | task 7030 | graphs reused = 7365 1.37.391.163 I slot release: id 3 | task 7030 | stop processing: n_tokens = 556, truncated = 0 1.37.391.172 I srv update_slots: all slots are idle 1.37.399.340 I srv operator(): Chat format: peg-native 1.37.403.207 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.908 (> 0.100 thold), f_keep = 0.248 1.37.403.209 I srv get_availabl: updating prompt cache 1.37.403.259 W srv prompt_save: - saving prompt with length 556, total state size = 16.040 MiB (draft: 0.000 MiB) 1.37.416.895 I srv load: - looking for better prompt, base f_keep = 0.248, sim = 0.908 1.37.416.904 I srv update: - cache state: 6 prompts, 223.075 MiB (limits: 8192.000 MiB, 16384 tokens, 292939 est) 1.37.416.906 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 1.37.416.906 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 1.37.416.907 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 1.37.416.907 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 1.37.416.908 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 1.37.416.908 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 1.37.416.909 I srv get_availabl: prompt cache update took 13.70 ms 1.37.417.857 I slot launch_slot_: id 3 | task 7440 | processing task, is_child = 0 1.37.417.859 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.37.417.860 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.37.417.875 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.37.417.875 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.37.417.884 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.37.417.885 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.40.504.498 I slot print_timing: id 3 | task 7440 | n_decoded = 264, tg = 87.74 t/s, tg_3s = 87.74 t/s 1.43.510.557 I slot print_timing: id 3 | task 7440 | n_decoded = 524, tg = 87.11 t/s, tg_3s = 86.49 t/s 1.46.515.683 I slot print_timing: id 3 | task 7440 | n_decoded = 782, tg = 86.69 t/s, tg_3s = 85.85 t/s 1.49.524.744 I slot print_timing: id 3 | task 7440 | n_decoded = 1038, tg = 86.29 t/s, tg_3s = 85.08 t/s 1.50.807.033 I slot print_timing: id 3 | task 7440 | prompt eval time = 77.55 ms / 14 tokens ( 5.54 ms per token, 180.53 tokens per second) 1.50.807.038 I slot print_timing: id 3 | task 7440 | eval time = 13311.58 ms / 1146 tokens ( 11.62 ms per token, 86.09 tokens per second) 1.50.807.038 I slot print_timing: id 3 | task 7440 | total time = 13389.13 ms / 1160 tokens 1.50.807.039 I slot print_timing: id 3 | task 7440 | graphs reused = 8501 1.50.807.072 I slot release: id 3 | task 7440 | stop processing: n_tokens = 1297, truncated = 0 1.50.807.085 I srv update_slots: all slots are idle 1.50.815.431 I srv operator(): Chat format: peg-native 1.50.819.275 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.761 (> 0.100 thold), f_keep = 0.106 1.50.819.278 I srv get_availabl: updating prompt cache 1.50.819.345 W srv prompt_save: - saving prompt with length 1297, total state size = 33.415 MiB (draft: 0.000 MiB) 1.50.866.313 I srv load: - looking for better prompt, base f_keep = 0.106, sim = 0.761 1.50.866.323 I srv update: - cache state: 7 prompts, 259.492 MiB (limits: 8192.000 MiB, 16384 tokens, 292774 est) 1.50.866.324 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 1.50.866.324 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 1.50.866.325 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 1.50.866.325 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 1.50.866.326 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 1.50.866.327 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 1.50.866.327 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 1.50.866.328 I srv get_availabl: prompt cache update took 47.05 ms 1.50.867.255 I slot launch_slot_: id 3 | task 8588 | processing task, is_child = 0 1.50.867.257 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.50.867.257 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.50.867.277 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.50.867.278 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.50.867.300 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.50.867.300 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.50.867.322 I slot update_slots: id 3 | task 8588 | Checking checkpoint with [0, 133] against 9... 1.50.869.491 W slot update_slots: id 3 | task 8588 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 3.002 MiB) 1.54.060.893 I slot print_timing: id 3 | task 8588 | n_decoded = 265, tg = 88.13 t/s, tg_3s = 88.13 t/s 1.54.886.212 I slot print_timing: id 3 | task 8588 | prompt eval time = 186.63 ms / 47 tokens ( 3.97 ms per token, 251.84 tokens per second) 1.54.886.215 I slot print_timing: id 3 | task 8588 | eval time = 3832.26 ms / 337 tokens ( 11.37 ms per token, 87.94 tokens per second) 1.54.886.216 I slot print_timing: id 3 | task 8588 | total time = 4018.89 ms / 384 tokens 1.54.886.217 I slot print_timing: id 3 | task 8588 | graphs reused = 8833 1.54.886.251 I slot release: id 3 | task 8588 | stop processing: n_tokens = 516, truncated = 0 1.54.886.259 I srv update_slots: all slots are idle 1.54.894.976 I srv operator(): Chat format: peg-native 1.54.898.834 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.708 (> 0.100 thold), f_keep = 0.267 1.54.898.836 I srv get_availabl: updating prompt cache 1.54.898.876 W srv prompt_save: - saving prompt with length 516, total state size = 15.102 MiB (draft: 0.000 MiB) 1.54.911.823 I srv load: - looking for better prompt, base f_keep = 0.267, sim = 0.708 1.54.911.834 I srv update: - cache state: 8 prompts, 277.596 MiB (limits: 8192.000 MiB, 16384 tokens, 288908 est) 1.54.911.835 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 1.54.911.835 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 1.54.911.836 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 1.54.911.836 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 1.54.911.837 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 1.54.911.837 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 1.54.911.838 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 1.54.911.838 I srv update: - prompt 0x64ea78fe4e40: 516 tokens, checkpoints: 1, 18.103 MiB 1.54.911.840 I srv get_availabl: prompt cache update took 13.00 ms 1.54.912.791 I slot launch_slot_: id 3 | task 8928 | processing task, is_child = 0 1.54.912.793 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.54.912.794 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.54.912.809 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.54.912.809 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.54.912.818 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.54.912.819 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.58.114.751 I slot print_timing: id 3 | task 8928 | n_decoded = 264, tg = 87.74 t/s, tg_3s = 87.74 t/s 2.01.120.300 I slot print_timing: id 3 | task 8928 | n_decoded = 529, tg = 87.95 t/s, tg_3s = 88.17 t/s 2.04.129.741 I slot print_timing: id 3 | task 8928 | n_decoded = 791, tg = 87.66 t/s, tg_3s = 87.06 t/s 2.07.137.410 I slot print_timing: id 3 | task 8928 | n_decoded = 1052, tg = 87.44 t/s, tg_3s = 86.78 t/s 2.10.000.362 I slot print_timing: id 3 | task 8928 | prompt eval time = 192.99 ms / 57 tokens ( 3.39 ms per token, 295.35 tokens per second) 2.10.000.366 I slot print_timing: id 3 | task 8928 | eval time = 14894.53 ms / 1297 tokens ( 11.48 ms per token, 87.08 tokens per second) 2.10.000.366 I slot print_timing: id 3 | task 8928 | total time = 15087.52 ms / 1354 tokens 2.10.000.367 I slot print_timing: id 3 | task 8928 | graphs reused = 10120 2.10.000.403 I slot release: id 3 | task 8928 | stop processing: n_tokens = 1491, truncated = 0 2.10.000.416 I srv update_slots: all slots are idle 2.10.009.011 I srv operator(): Chat format: peg-native 2.10.012.875 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.787 (> 0.100 thold), f_keep = 0.092 2.10.012.877 I srv get_availabl: updating prompt cache 2.10.012.944 W srv prompt_save: - saving prompt with length 1491, total state size = 37.964 MiB (draft: 0.000 MiB) 2.10.066.730 I srv load: - looking for better prompt, base f_keep = 0.092, sim = 0.787 2.10.066.740 I srv update: - cache state: 9 prompts, 318.562 MiB (limits: 8192.000 MiB, 16384 tokens, 290097 est) 2.10.066.741 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 2.10.066.741 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 2.10.066.742 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 2.10.066.742 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 2.10.066.743 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 2.10.066.744 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 2.10.066.744 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 2.10.066.745 I srv update: - prompt 0x64ea78fe4e40: 516 tokens, checkpoints: 1, 18.103 MiB 2.10.066.745 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 2.10.066.746 I srv get_availabl: prompt cache update took 53.87 ms 2.10.067.695 I slot launch_slot_: id 3 | task 10227 | processing task, is_child = 0 2.10.067.697 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.10.067.698 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.10.067.710 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.10.067.710 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.10.067.720 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.10.067.721 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.10.067.738 I slot update_slots: id 3 | task 10227 | Checking checkpoint with [0, 133] against 9... 2.10.069.678 W slot update_slots: id 3 | task 10227 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 3.002 MiB) 2.13.234.273 I slot print_timing: id 3 | task 10227 | n_decoded = 261, tg = 86.86 t/s, tg_3s = 86.86 t/s 2.16.240.399 I slot print_timing: id 3 | task 10227 | n_decoded = 521, tg = 86.67 t/s, tg_3s = 86.49 t/s 2.16.992.924 I slot print_timing: id 3 | task 10227 | prompt eval time = 161.63 ms / 41 tokens ( 3.94 ms per token, 253.67 tokens per second) 2.16.992.929 I slot print_timing: id 3 | task 10227 | eval time = 6763.55 ms / 586 tokens ( 11.54 ms per token, 86.64 tokens per second) 2.16.992.929 I slot print_timing: id 3 | task 10227 | total time = 6925.18 ms / 627 tokens 2.16.992.930 I slot print_timing: id 3 | task 10227 | graphs reused = 10700 2.16.992.967 I slot release: id 3 | task 10227 | stop processing: n_tokens = 759, truncated = 0 2.16.992.979 I srv update_slots: all slots are idle 2.17.001.411 I srv operator(): Chat format: peg-native 2.17.005.597 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.729 (> 0.100 thold), f_keep = 0.181 2.17.005.599 I srv get_availabl: updating prompt cache 2.17.005.645 W srv prompt_save: - saving prompt with length 759, total state size = 20.800 MiB (draft: 0.000 MiB) 2.17.023.468 I srv load: - looking for better prompt, base f_keep = 0.181, sim = 0.729 2.17.023.479 I srv update: - cache state: 10 prompts, 342.363 MiB (limits: 8192.000 MiB, 16384 tokens, 288090 est) 2.17.023.480 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 2.17.023.482 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 2.17.023.483 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 2.17.023.483 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 2.17.023.484 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 2.17.023.484 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 2.17.023.485 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 2.17.023.485 I srv update: - prompt 0x64ea78fe4e40: 516 tokens, checkpoints: 1, 18.103 MiB 2.17.023.486 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 2.17.023.486 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 2.17.023.487 I srv get_availabl: prompt cache update took 17.89 ms 2.17.024.403 I slot launch_slot_: id 3 | task 10816 | processing task, is_child = 0 2.17.024.405 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.17.024.406 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.17.024.422 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.17.024.422 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.17.024.431 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.17.024.432 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.20.199.770 I slot print_timing: id 3 | task 10816 | n_decoded = 262, tg = 87.32 t/s, tg_3s = 87.32 t/s 2.22.461.835 I slot print_timing: id 3 | task 10816 | prompt eval time = 174.90 ms / 51 tokens ( 3.43 ms per token, 291.59 tokens per second) 2.22.461.838 I slot print_timing: id 3 | task 10816 | eval time = 5262.48 ms / 458 tokens ( 11.49 ms per token, 87.03 tokens per second) 2.22.461.839 I slot print_timing: id 3 | task 10816 | total time = 5437.38 ms / 509 tokens 2.22.461.839 I slot print_timing: id 3 | task 10816 | graphs reused = 11152 2.22.461.875 I slot release: id 3 | task 10816 | stop processing: n_tokens = 645, truncated = 0 2.22.461.884 I srv update_slots: all slots are idle 2.22.470.298 I srv operator(): Chat format: peg-native 2.22.474.150 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.212 2.22.474.152 I srv get_availabl: updating prompt cache 2.22.474.202 W srv prompt_save: - saving prompt with length 645, total state size = 18.127 MiB (draft: 0.000 MiB) 2.22.490.053 I srv load: - looking for better prompt, base f_keep = 0.212, sim = 0.721 2.22.490.062 I srv load: - found better prompt with f_keep = 0.267, sim = 0.726 2.22.495.946 I srv update: - cache state: 10 prompts, 345.388 MiB (limits: 8192.000 MiB, 16384 tokens, 288627 est) 2.22.495.950 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 2.22.495.950 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 2.22.495.951 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 2.22.495.951 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 2.22.495.952 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 2.22.495.953 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 2.22.495.953 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 2.22.495.954 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 2.22.495.954 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 2.22.495.955 I srv update: - prompt 0x64ea78e58610: 645 tokens, checkpoints: 1, 21.128 MiB 2.22.495.956 I srv get_availabl: prompt cache update took 21.80 ms 2.22.496.832 I slot launch_slot_: id 3 | task 11276 | processing task, is_child = 0 2.22.496.834 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.22.496.836 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.22.496.849 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.22.496.849 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.22.496.858 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.22.496.858 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.22.496.873 I slot update_slots: id 3 | task 11276 | Checking checkpoint with [0, 133] against 10... 2.22.498.760 W slot update_slots: id 3 | task 11276 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 3.002 MiB) 2.25.712.325 I slot print_timing: id 3 | task 11276 | n_decoded = 260, tg = 86.49 t/s, tg_3s = 86.49 t/s 2.28.066.941 I slot print_timing: id 3 | task 11276 | prompt eval time = 209.43 ms / 57 tokens ( 3.67 ms per token, 272.17 tokens per second) 2.28.066.945 I slot print_timing: id 3 | task 11276 | eval time = 5360.64 ms / 464 tokens ( 11.55 ms per token, 86.56 tokens per second) 2.28.066.945 I slot print_timing: id 3 | task 11276 | total time = 5570.06 ms / 521 tokens 2.28.066.946 I slot print_timing: id 3 | task 11276 | graphs reused = 11610 2.28.066.984 I slot release: id 3 | task 11276 | stop processing: n_tokens = 653, truncated = 0 2.28.066.994 I srv update_slots: all slots are idle 2.28.075.360 I srv operator(): Chat format: peg-native 2.28.079.186 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.780 (> 0.100 thold), f_keep = 0.211 2.28.079.188 I srv get_availabl: updating prompt cache 2.28.079.234 W srv prompt_save: - saving prompt with length 653, total state size = 18.314 MiB (draft: 0.000 MiB) 2.28.094.474 I srv load: - looking for better prompt, base f_keep = 0.211, sim = 0.780 2.28.094.486 I srv update: - cache state: 11 prompts, 366.704 MiB (limits: 8192.000 MiB, 16384 tokens, 286437 est) 2.28.094.487 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 2.28.094.488 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 2.28.094.488 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 2.28.094.489 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 2.28.094.490 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 2.28.094.490 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 2.28.094.491 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 2.28.094.491 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 2.28.094.492 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 2.28.094.492 I srv update: - prompt 0x64ea78e58610: 645 tokens, checkpoints: 1, 21.128 MiB 2.28.094.493 I srv update: - prompt 0x64ea78c08210: 653 tokens, checkpoints: 1, 21.316 MiB 2.28.094.494 I srv get_availabl: prompt cache update took 15.31 ms 2.28.095.423 I slot launch_slot_: id 3 | task 11743 | processing task, is_child = 0 2.28.095.426 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.28.095.426 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.28.095.443 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.28.095.443 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.28.095.453 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.28.095.453 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.31.243.442 I slot print_timing: id 3 | task 11743 | n_decoded = 261, tg = 86.96 t/s, tg_3s = 86.96 t/s 2.33.541.345 I slot print_timing: id 3 | task 11743 | prompt eval time = 146.71 ms / 39 tokens ( 3.76 ms per token, 265.82 tokens per second) 2.33.541.350 I slot print_timing: id 3 | task 11743 | eval time = 5299.15 ms / 459 tokens ( 11.54 ms per token, 86.62 tokens per second) 2.33.541.350 I slot print_timing: id 3 | task 11743 | total time = 5445.87 ms / 498 tokens 2.33.541.351 I slot print_timing: id 3 | task 11743 | graphs reused = 12063 2.33.541.386 I slot release: id 3 | task 11743 | stop processing: n_tokens = 635, truncated = 0 2.33.541.397 I srv update_slots: all slots are idle 2.33.549.946 I srv operator(): Chat format: peg-native 2.33.553.835 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.745 (> 0.100 thold), f_keep = 0.216 2.33.553.837 I srv get_availabl: updating prompt cache 2.33.553.884 W srv prompt_save: - saving prompt with length 635, total state size = 17.892 MiB (draft: 0.000 MiB) 2.33.569.271 I srv load: - looking for better prompt, base f_keep = 0.216, sim = 0.745 2.33.569.284 I srv update: - cache state: 12 prompts, 387.598 MiB (limits: 8192.000 MiB, 16384 tokens, 284417 est) 2.33.569.285 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 2.33.569.285 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 2.33.569.286 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 2.33.569.286 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 2.33.569.287 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 2.33.569.287 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 2.33.569.288 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 2.33.569.288 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 2.33.569.289 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 2.33.569.290 I srv update: - prompt 0x64ea78e58610: 645 tokens, checkpoints: 1, 21.128 MiB 2.33.569.290 I srv update: - prompt 0x64ea78c08210: 653 tokens, checkpoints: 1, 21.316 MiB 2.33.569.291 I srv update: - prompt 0x64ea8011af90: 635 tokens, checkpoints: 1, 20.894 MiB 2.33.569.291 I srv get_availabl: prompt cache update took 15.45 ms 2.33.570.222 I slot launch_slot_: id 3 | task 12204 | processing task, is_child = 0 2.33.570.224 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.33.570.225 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.33.570.240 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.33.570.241 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.33.570.251 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.33.570.252 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.36.746.931 I slot print_timing: id 3 | task 12204 | n_decoded = 261, tg = 86.82 t/s, tg_3s = 86.82 t/s 2.39.747.977 I slot print_timing: id 3 | task 12204 | n_decoded = 518, tg = 86.23 t/s, tg_3s = 85.64 t/s 2.42.752.488 I slot print_timing: id 3 | task 12204 | n_decoded = 773, tg = 85.78 t/s, tg_3s = 84.87 t/s 2.45.754.468 I slot print_timing: id 3 | task 12204 | n_decoded = 1023, tg = 85.15 t/s, tg_3s = 83.28 t/s 2.48.760.622 I slot print_timing: id 3 | task 12204 | n_decoded = 1273, tg = 84.75 t/s, tg_3s = 83.16 t/s 2.51.767.632 I slot print_timing: id 3 | task 12204 | n_decoded = 1525, tg = 84.60 t/s, tg_3s = 83.80 t/s 2.54.320.015 I slot print_timing: id 3 | task 12204 | prompt eval time = 170.45 ms / 47 tokens ( 3.63 ms per token, 275.74 tokens per second) 2.54.320.019 I slot print_timing: id 3 | task 12204 | eval time = 20579.29 ms / 1743 tokens ( 11.81 ms per token, 84.70 tokens per second) 2.54.320.020 I slot print_timing: id 3 | task 12204 | total time = 20749.74 ms / 1790 tokens 2.54.320.020 I slot print_timing: id 3 | task 12204 | graphs reused = 13794 2.54.320.059 I slot release: id 3 | task 12204 | stop processing: n_tokens = 1926, truncated = 0 2.54.320.073 I srv update_slots: all slots are idle 2.54.328.537 I srv operator(): Chat format: peg-native 2.54.332.403 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.801 (> 0.100 thold), f_keep = 0.071 2.54.332.405 I srv get_availabl: updating prompt cache 2.54.332.499 W srv prompt_save: - saving prompt with length 1926, total state size = 48.165 MiB (draft: 0.000 MiB) 2.54.371.181 I srv load: - looking for better prompt, base f_keep = 0.071, sim = 0.801 2.54.371.191 I srv update: - cache state: 13 prompts, 438.765 MiB (limits: 8192.000 MiB, 16384 tokens, 287209 est) 2.54.371.192 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 2.54.371.193 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 2.54.371.193 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 2.54.371.194 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 2.54.371.194 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 2.54.371.195 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 2.54.371.195 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 2.54.371.196 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 2.54.371.196 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 2.54.371.197 I srv update: - prompt 0x64ea78e58610: 645 tokens, checkpoints: 1, 21.128 MiB 2.54.371.197 I srv update: - prompt 0x64ea78c08210: 653 tokens, checkpoints: 1, 21.316 MiB 2.54.371.198 I srv update: - prompt 0x64ea8011af90: 635 tokens, checkpoints: 1, 20.894 MiB 2.54.371.198 I srv update: - prompt 0x64ea800cc880: 1926 tokens, checkpoints: 1, 51.166 MiB 2.54.371.200 I srv get_availabl: prompt cache update took 38.79 ms 2.54.372.138 I slot launch_slot_: id 3 | task 13949 | processing task, is_child = 0 2.54.372.140 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 2.54.372.141 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 2.54.372.153 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 2.54.372.154 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 2.54.372.164 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 2.54.372.164 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 2.54.372.180 I slot update_slots: id 3 | task 13949 | Checking checkpoint with [0, 133] against 9... 2.54.374.128 W slot update_slots: id 3 | task 13949 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 3.002 MiB) 2.57.536.795 I slot print_timing: id 3 | task 13949 | n_decoded = 266, tg = 88.54 t/s, tg_3s = 88.54 t/s 3.00.541.544 I slot print_timing: id 3 | task 13949 | n_decoded = 529, tg = 88.03 t/s, tg_3s = 87.53 t/s 3.03.544.038 I slot print_timing: id 3 | task 13949 | n_decoded = 789, tg = 87.55 t/s, tg_3s = 86.59 t/s 3.03.647.601 I slot print_timing: id 3 | task 13949 | prompt eval time = 160.22 ms / 38 tokens ( 4.22 ms per token, 237.17 tokens per second) 3.03.647.604 I slot print_timing: id 3 | task 13949 | eval time = 9115.20 ms / 798 tokens ( 11.42 ms per token, 87.55 tokens per second) 3.03.647.604 I slot print_timing: id 3 | task 13949 | total time = 9275.42 ms / 836 tokens 3.03.647.605 I slot print_timing: id 3 | task 13949 | graphs reused = 14584 3.03.647.639 I slot release: id 3 | task 13949 | stop processing: n_tokens = 968, truncated = 0 3.03.647.652 I srv update_slots: all slots are idle 3.03.656.028 I srv operator(): Chat format: peg-native 3.03.659.890 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.787 (> 0.100 thold), f_keep = 0.142 3.03.659.892 I srv get_availabl: updating prompt cache 3.03.659.951 W srv prompt_save: - saving prompt with length 968, total state size = 25.701 MiB (draft: 0.000 MiB) 3.03.681.154 I srv load: - looking for better prompt, base f_keep = 0.142, sim = 0.787 3.03.681.171 I srv update: - cache state: 14 prompts, 467.467 MiB (limits: 8192.000 MiB, 16384 tokens, 286538 est) 3.03.681.172 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 3.03.681.172 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 3.03.681.173 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 3.03.681.174 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 3.03.681.176 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 3.03.681.177 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 3.03.681.178 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 3.03.681.179 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 3.03.681.180 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 3.03.681.180 I srv update: - prompt 0x64ea78e58610: 645 tokens, checkpoints: 1, 21.128 MiB 3.03.681.181 I srv update: - prompt 0x64ea78c08210: 653 tokens, checkpoints: 1, 21.316 MiB 3.03.681.182 I srv update: - prompt 0x64ea8011af90: 635 tokens, checkpoints: 1, 20.894 MiB 3.03.681.182 I srv update: - prompt 0x64ea800cc880: 1926 tokens, checkpoints: 1, 51.166 MiB 3.03.681.183 I srv update: - prompt 0x64ea78f02ed0: 968 tokens, checkpoints: 1, 28.702 MiB 3.03.681.184 I srv get_availabl: prompt cache update took 21.29 ms 3.03.682.161 I slot launch_slot_: id 3 | task 14750 | processing task, is_child = 0 3.03.682.163 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.03.682.164 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.03.682.183 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.03.682.184 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.03.682.196 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.03.682.196 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.06.825.830 I slot print_timing: id 3 | task 14750 | n_decoded = 265, tg = 88.16 t/s, tg_3s = 88.16 t/s 3.09.832.909 I slot print_timing: id 3 | task 14750 | n_decoded = 527, tg = 87.65 t/s, tg_3s = 87.13 t/s 3.12.843.645 I slot print_timing: id 3 | task 14750 | n_decoded = 787, tg = 87.22 t/s, tg_3s = 86.36 t/s 3.15.854.119 I slot print_timing: id 3 | task 14750 | n_decoded = 1044, tg = 86.75 t/s, tg_3s = 85.37 t/s 3.16.299.535 I slot print_timing: id 3 | task 14750 | prompt eval time = 137.85 ms / 37 tokens ( 3.73 ms per token, 268.40 tokens per second) 3.16.299.538 I slot print_timing: id 3 | task 14750 | eval time = 12479.46 ms / 1082 tokens ( 11.53 ms per token, 86.70 tokens per second) 3.16.299.539 I slot print_timing: id 3 | task 14750 | total time = 12617.31 ms / 1119 tokens 3.16.299.539 I slot print_timing: id 3 | task 14750 | graphs reused = 15657 3.16.299.575 I slot release: id 3 | task 14750 | stop processing: n_tokens = 1255, truncated = 0 3.16.299.587 I srv update_slots: all slots are idle 3.16.307.981 I srv operator(): Chat format: peg-native 3.16.311.834 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.835 (> 0.100 thold), f_keep = 0.109 3.16.311.836 I srv get_availabl: updating prompt cache 3.16.311.896 W srv prompt_save: - saving prompt with length 1255, total state size = 32.430 MiB (draft: 0.000 MiB) 3.16.346.897 I srv load: - looking for better prompt, base f_keep = 0.109, sim = 0.835 3.16.346.911 I srv update: - cache state: 15 prompts, 502.899 MiB (limits: 8192.000 MiB, 16384 tokens, 286793 est) 3.16.346.912 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 3.16.346.913 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 3.16.346.913 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 3.16.346.914 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 3.16.346.914 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 3.16.346.915 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 3.16.346.916 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 3.16.346.916 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 3.16.346.917 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 3.16.346.917 I srv update: - prompt 0x64ea78e58610: 645 tokens, checkpoints: 1, 21.128 MiB 3.16.346.918 I srv update: - prompt 0x64ea78c08210: 653 tokens, checkpoints: 1, 21.316 MiB 3.16.346.918 I srv update: - prompt 0x64ea8011af90: 635 tokens, checkpoints: 1, 20.894 MiB 3.16.346.918 I srv update: - prompt 0x64ea800cc880: 1926 tokens, checkpoints: 1, 51.166 MiB 3.16.346.919 I srv update: - prompt 0x64ea78f02ed0: 968 tokens, checkpoints: 1, 28.702 MiB 3.16.346.919 I srv update: - prompt 0x64ea78fbfd70: 1255 tokens, checkpoints: 1, 35.432 MiB 3.16.346.921 I srv get_availabl: prompt cache update took 35.08 ms 3.16.347.846 I slot launch_slot_: id 3 | task 15834 | processing task, is_child = 0 3.16.347.848 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.16.347.849 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.16.347.861 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.16.347.861 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.16.347.871 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.16.347.871 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.16.347.886 I slot update_slots: id 3 | task 15834 | Checking checkpoint with [0, 133] against 9... 3.16.349.644 W slot update_slots: id 3 | task 15834 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 3.002 MiB) 3.19.487.466 I slot print_timing: id 3 | task 15834 | n_decoded = 262, tg = 87.24 t/s, tg_3s = 87.24 t/s 3.22.496.011 I slot print_timing: id 3 | task 15834 | n_decoded = 523, tg = 87.00 t/s, tg_3s = 86.75 t/s 3.25.500.524 I slot print_timing: id 3 | task 15834 | n_decoded = 781, tg = 86.62 t/s, tg_3s = 85.87 t/s 3.27.964.976 I slot print_timing: id 3 | task 15834 | prompt eval time = 136.46 ms / 31 tokens ( 4.40 ms per token, 227.17 tokens per second) 3.27.964.979 I slot print_timing: id 3 | task 15834 | eval time = 11480.62 ms / 991 tokens ( 11.58 ms per token, 86.32 tokens per second) 3.27.964.979 I slot print_timing: id 3 | task 15834 | total time = 11617.08 ms / 1022 tokens 3.27.964.980 I slot print_timing: id 3 | task 15834 | graphs reused = 16639 3.27.965.014 I slot release: id 3 | task 15834 | stop processing: n_tokens = 1154, truncated = 0 3.27.965.025 I srv update_slots: all slots are idle 3.27.974.019 I srv operator(): Chat format: peg-native 3.27.978.029 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.878 (> 0.100 thold), f_keep = 0.119 3.27.978.030 I srv get_availabl: updating prompt cache 3.27.978.089 W srv prompt_save: - saving prompt with length 1154, total state size = 30.062 MiB (draft: 0.000 MiB) 3.28.005.992 I srv load: - looking for better prompt, base f_keep = 0.119, sim = 0.878 3.28.006.005 I srv update: - cache state: 16 prompts, 535.963 MiB (limits: 8192.000 MiB, 16384 tokens, 286739 est) 3.28.006.006 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 3.28.006.007 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 3.28.006.007 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 3.28.006.008 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 3.28.006.008 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 3.28.006.009 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 3.28.006.009 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 3.28.006.009 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 3.28.006.010 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 3.28.006.010 I srv update: - prompt 0x64ea78e58610: 645 tokens, checkpoints: 1, 21.128 MiB 3.28.006.011 I srv update: - prompt 0x64ea78c08210: 653 tokens, checkpoints: 1, 21.316 MiB 3.28.006.011 I srv update: - prompt 0x64ea8011af90: 635 tokens, checkpoints: 1, 20.894 MiB 3.28.006.012 I srv update: - prompt 0x64ea800cc880: 1926 tokens, checkpoints: 1, 51.166 MiB 3.28.006.012 I srv update: - prompt 0x64ea78f02ed0: 968 tokens, checkpoints: 1, 28.702 MiB 3.28.006.013 I srv update: - prompt 0x64ea78fbfd70: 1255 tokens, checkpoints: 1, 35.432 MiB 3.28.006.013 I srv update: - prompt 0x64ea790877e0: 1154 tokens, checkpoints: 1, 33.064 MiB 3.28.006.014 I srv get_availabl: prompt cache update took 27.98 ms 3.28.006.875 I slot launch_slot_: id 3 | task 16828 | processing task, is_child = 0 3.28.006.876 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 3.28.006.877 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 3.28.006.888 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 3.28.006.888 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 3.28.006.897 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 3.28.006.897 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 3.28.006.912 I slot update_slots: id 3 | task 16828 | Checking checkpoint with [0, 133] against 9... 3.28.008.647 W slot update_slots: id 3 | task 16828 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 3.002 MiB) 3.31.118.032 I slot print_timing: id 3 | task 16828 | n_decoded = 263, tg = 87.56 t/s, tg_3s = 87.56 t/s 3.34.122.718 I slot print_timing: id 3 | task 16828 | n_decoded = 525, tg = 87.38 t/s, tg_3s = 87.20 t/s 3.37.126.618 I slot print_timing: id 3 | task 16828 | n_decoded = 782, tg = 86.77 t/s, tg_3s = 85.56 t/s 3.40.129.728 I slot print_timing: id 3 | task 16828 | n_decoded = 1042, tg = 86.72 t/s, tg_3s = 86.58 t/s 3.43.138.863 I slot print_timing: id 3 | task 16828 | n_decoded = 1303, tg = 86.72 t/s, tg_3s = 86.74 t/s 3.46.139.460 I slot print_timing: id 3 | task 16828 | n_decoded = 1561, tg = 86.60 t/s, tg_3s = 85.98 t/s 3.49.142.340 I slot print_timing: id 3 | task 16828 | n_decoded = 1815, tg = 86.31 t/s, tg_3s = 84.59 t/s 3.52.152.307 I slot print_timing: id 3 | task 16828 | n_decoded = 2067, tg = 85.99 t/s, tg_3s = 83.72 t/s 3.55.154.362 I slot print_timing: id 3 | task 16828 | n_decoded = 2317, tg = 85.69 t/s, tg_3s = 83.28 t/s 3.58.157.399 I slot print_timing: id 3 | task 16828 | n_decoded = 2567, tg = 85.44 t/s, tg_3s = 83.25 t/s 4.01.160.462 I slot print_timing: id 3 | task 16828 | n_decoded = 2817, tg = 85.24 t/s, tg_3s = 83.25 t/s 4.04.164.447 I slot print_timing: id 3 | task 16828 | n_decoded = 3067, tg = 85.08 t/s, tg_3s = 83.22 t/s 4.07.174.475 I slot print_timing: id 3 | task 16828 | n_decoded = 3317, tg = 84.92 t/s, tg_3s = 83.06 t/s 4.10.183.633 I slot print_timing: id 3 | task 16828 | n_decoded = 3566, tg = 84.76 t/s, tg_3s = 82.75 t/s 4.13.193.729 I slot print_timing: id 3 | task 16828 | n_decoded = 3815, tg = 84.63 t/s, tg_3s = 82.72 t/s 4.16.200.795 I slot print_timing: id 3 | task 16828 | n_decoded = 4065, tg = 84.54 t/s, tg_3s = 83.14 t/s 4.16.572.799 I slot print_timing: id 3 | task 16828 | prompt eval time = 107.40 ms / 23 tokens ( 4.67 ms per token, 214.15 tokens per second) 4.16.572.803 I slot print_timing: id 3 | task 16828 | eval time = 48458.48 ms / 4096 tokens ( 11.83 ms per token, 84.53 tokens per second) 4.16.572.803 I slot print_timing: id 3 | task 16828 | total time = 48565.88 ms / 4119 tokens 4.16.572.804 I slot print_timing: id 3 | task 16828 | graphs reused = 20714 4.16.572.839 I slot release: id 3 | task 16828 | stop processing: n_tokens = 4251, truncated = 0 4.16.572.863 I srv update_slots: all slots are idle 4.16.581.284 I srv operator(): Chat format: peg-native 4.16.585.297 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.783 (> 0.100 thold), f_keep = 0.032 4.16.585.298 I srv get_availabl: updating prompt cache 4.16.585.465 W srv prompt_save: - saving prompt with length 4251, total state size = 102.684 MiB (draft: 0.000 MiB) 4.16.678.747 I srv load: - looking for better prompt, base f_keep = 0.032, sim = 0.783 4.16.678.762 I srv update: - cache state: 17 prompts, 641.649 MiB (limits: 8192.000 MiB, 16384 tokens, 293784 est) 4.16.678.763 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 4.16.678.764 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 4.16.678.764 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 4.16.678.765 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 4.16.678.765 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 4.16.678.766 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 4.16.678.766 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 4.16.678.767 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 4.16.678.767 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 4.16.678.768 I srv update: - prompt 0x64ea78e58610: 645 tokens, checkpoints: 1, 21.128 MiB 4.16.678.768 I srv update: - prompt 0x64ea78c08210: 653 tokens, checkpoints: 1, 21.316 MiB 4.16.678.769 I srv update: - prompt 0x64ea8011af90: 635 tokens, checkpoints: 1, 20.894 MiB 4.16.678.769 I srv update: - prompt 0x64ea800cc880: 1926 tokens, checkpoints: 1, 51.166 MiB 4.16.678.770 I srv update: - prompt 0x64ea78f02ed0: 968 tokens, checkpoints: 1, 28.702 MiB 4.16.678.770 I srv update: - prompt 0x64ea78fbfd70: 1255 tokens, checkpoints: 1, 35.432 MiB 4.16.678.771 I srv update: - prompt 0x64ea790877e0: 1154 tokens, checkpoints: 1, 33.064 MiB 4.16.678.771 I srv update: - prompt 0x64ea800eb4a0: 4251 tokens, checkpoints: 1, 105.685 MiB 4.16.678.772 I srv get_availabl: prompt cache update took 93.47 ms 4.16.679.730 I slot launch_slot_: id 3 | task 20927 | processing task, is_child = 0 4.16.679.732 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.16.679.732 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.16.679.749 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.16.679.749 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.16.679.760 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.16.679.760 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.16.679.779 I slot update_slots: id 3 | task 20927 | Checking checkpoint with [0, 133] against 9... 4.16.681.817 W slot update_slots: id 3 | task 20927 | restored context checkpoint (pos_min = 0, pos_max = 133, n_tokens = 134, n_past = 133, size = 3.002 MiB) 4.19.848.813 I slot print_timing: id 3 | task 20927 | n_decoded = 259, tg = 86.10 t/s, tg_3s = 86.10 t/s 4.20.894.866 I slot print_timing: id 3 | task 20927 | prompt eval time = 160.92 ms / 42 tokens ( 3.83 ms per token, 261.00 tokens per second) 4.20.894.869 I slot print_timing: id 3 | task 20927 | eval time = 4054.16 ms / 348 tokens ( 11.65 ms per token, 85.84 tokens per second) 4.20.894.869 I slot print_timing: id 3 | task 20927 | total time = 4215.08 ms / 390 tokens 4.20.894.870 I slot print_timing: id 3 | task 20927 | graphs reused = 21056 4.20.894.904 I slot release: id 3 | task 20927 | stop processing: n_tokens = 522, truncated = 0 4.20.894.913 I srv update_slots: all slots are idle 4.20.903.739 I srv operator(): Chat format: peg-native 4.20.907.638 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.835 (> 0.100 thold), f_keep = 0.262 4.20.907.639 I srv get_availabl: updating prompt cache 4.20.907.686 W srv prompt_save: - saving prompt with length 522, total state size = 15.242 MiB (draft: 0.000 MiB) 4.20.920.985 I srv load: - looking for better prompt, base f_keep = 0.262, sim = 0.835 4.20.921.001 I srv update: - cache state: 18 prompts, 659.893 MiB (limits: 8192.000 MiB, 16384 tokens, 292141 est) 4.20.921.001 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 4.20.921.002 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 4.20.921.002 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 4.20.921.003 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 4.20.921.003 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 4.20.921.004 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 4.20.921.004 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 4.20.921.005 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 4.20.921.005 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 4.20.921.006 I srv update: - prompt 0x64ea78e58610: 645 tokens, checkpoints: 1, 21.128 MiB 4.20.921.006 I srv update: - prompt 0x64ea78c08210: 653 tokens, checkpoints: 1, 21.316 MiB 4.20.921.007 I srv update: - prompt 0x64ea8011af90: 635 tokens, checkpoints: 1, 20.894 MiB 4.20.921.007 I srv update: - prompt 0x64ea800cc880: 1926 tokens, checkpoints: 1, 51.166 MiB 4.20.921.007 I srv update: - prompt 0x64ea78f02ed0: 968 tokens, checkpoints: 1, 28.702 MiB 4.20.921.008 I srv update: - prompt 0x64ea78fbfd70: 1255 tokens, checkpoints: 1, 35.432 MiB 4.20.921.008 I srv update: - prompt 0x64ea790877e0: 1154 tokens, checkpoints: 1, 33.064 MiB 4.20.921.009 I srv update: - prompt 0x64ea800eb4a0: 4251 tokens, checkpoints: 1, 105.685 MiB 4.20.921.009 I srv update: - prompt 0x64ea80084860: 522 tokens, checkpoints: 1, 18.244 MiB 4.20.921.010 I srv get_availabl: prompt cache update took 13.37 ms 4.20.921.842 I slot launch_slot_: id 3 | task 21278 | processing task, is_child = 0 4.20.921.843 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.20.921.844 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.20.921.857 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.20.921.857 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.20.921.866 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.20.921.866 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.24.040.216 I slot print_timing: id 3 | task 21278 | n_decoded = 259, tg = 86.01 t/s, tg_3s = 86.01 t/s 4.25.145.240 I slot print_timing: id 3 | task 21278 | prompt eval time = 107.01 ms / 27 tokens ( 3.96 ms per token, 252.31 tokens per second) 4.25.145.243 I slot print_timing: id 3 | task 21278 | eval time = 4116.34 ms / 353 tokens ( 11.66 ms per token, 85.76 tokens per second) 4.25.145.243 I slot print_timing: id 3 | task 21278 | total time = 4223.35 ms / 380 tokens 4.25.145.244 I slot print_timing: id 3 | task 21278 | graphs reused = 21404 4.25.145.277 I slot release: id 3 | task 21278 | stop processing: n_tokens = 516, truncated = 0 4.25.145.285 I srv update_slots: all slots are idle 4.25.153.029 I srv operator(): Chat format: peg-native 4.25.156.489 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.811 (> 0.100 thold), f_keep = 0.266 4.25.156.491 I srv get_availabl: updating prompt cache 4.25.156.525 W srv prompt_save: - saving prompt with length 516, total state size = 15.102 MiB (draft: 0.000 MiB) 4.25.169.649 I srv load: - looking for better prompt, base f_keep = 0.266, sim = 0.811 4.25.169.667 I srv update: - cache state: 19 prompts, 677.996 MiB (limits: 8192.000 MiB, 16384 tokens, 290576 est) 4.25.169.668 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 4.25.169.669 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 4.25.169.669 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 4.25.169.670 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 4.25.169.670 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 4.25.169.671 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 4.25.169.671 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 4.25.169.672 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 4.25.169.676 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 4.25.169.677 I srv update: - prompt 0x64ea78e58610: 645 tokens, checkpoints: 1, 21.128 MiB 4.25.169.677 I srv update: - prompt 0x64ea78c08210: 653 tokens, checkpoints: 1, 21.316 MiB 4.25.169.678 I srv update: - prompt 0x64ea8011af90: 635 tokens, checkpoints: 1, 20.894 MiB 4.25.169.678 I srv update: - prompt 0x64ea800cc880: 1926 tokens, checkpoints: 1, 51.166 MiB 4.25.169.679 I srv update: - prompt 0x64ea78f02ed0: 968 tokens, checkpoints: 1, 28.702 MiB 4.25.169.679 I srv update: - prompt 0x64ea78fbfd70: 1255 tokens, checkpoints: 1, 35.432 MiB 4.25.169.679 I srv update: - prompt 0x64ea790877e0: 1154 tokens, checkpoints: 1, 33.064 MiB 4.25.169.680 I srv update: - prompt 0x64ea800eb4a0: 4251 tokens, checkpoints: 1, 105.685 MiB 4.25.169.680 I srv update: - prompt 0x64ea80084860: 522 tokens, checkpoints: 1, 18.244 MiB 4.25.169.681 I srv update: - prompt 0x64ea78eb3a90: 516 tokens, checkpoints: 1, 18.103 MiB 4.25.169.682 I srv get_availabl: prompt cache update took 13.19 ms 4.25.170.523 I slot launch_slot_: id 3 | task 21633 | processing task, is_child = 0 4.25.170.525 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.25.170.526 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.25.170.540 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.25.170.541 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.25.170.549 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.25.170.549 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.28.292.231 I slot print_timing: id 3 | task 21633 | n_decoded = 258, tg = 85.98 t/s, tg_3s = 85.98 t/s 4.31.295.887 I slot print_timing: id 3 | task 21633 | n_decoded = 513, tg = 85.44 t/s, tg_3s = 84.90 t/s 4.32.384.506 I slot print_timing: id 3 | task 21633 | prompt eval time = 121.13 ms / 32 tokens ( 3.79 ms per token, 264.17 tokens per second) 4.32.384.509 I slot print_timing: id 3 | task 21633 | eval time = 7092.80 ms / 608 tokens ( 11.67 ms per token, 85.72 tokens per second) 4.32.384.510 I slot print_timing: id 3 | task 21633 | total time = 7213.94 ms / 640 tokens 4.32.384.511 I slot print_timing: id 3 | task 21633 | graphs reused = 22004 4.32.384.550 I slot release: id 3 | task 21633 | stop processing: n_tokens = 776, truncated = 0 4.32.384.563 I srv update_slots: all slots are idle 4.32.393.221 I srv operator(): Chat format: peg-native 4.32.397.083 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.770 (> 0.100 thold), f_keep = 0.177 4.32.397.085 I srv get_availabl: updating prompt cache 4.32.397.163 W srv prompt_save: - saving prompt with length 776, total state size = 21.198 MiB (draft: 0.000 MiB) 4.32.415.577 I srv load: - looking for better prompt, base f_keep = 0.177, sim = 0.770 4.32.415.594 I srv update: - cache state: 20 prompts, 702.196 MiB (limits: 8192.000 MiB, 16384 tokens, 289614 est) 4.32.415.595 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 4.32.415.595 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 4.32.415.596 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 4.32.415.596 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 4.32.415.597 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 4.32.415.597 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 4.32.415.598 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 4.32.415.598 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 4.32.415.599 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 4.32.415.599 I srv update: - prompt 0x64ea78e58610: 645 tokens, checkpoints: 1, 21.128 MiB 4.32.415.600 I srv update: - prompt 0x64ea78c08210: 653 tokens, checkpoints: 1, 21.316 MiB 4.32.415.600 I srv update: - prompt 0x64ea8011af90: 635 tokens, checkpoints: 1, 20.894 MiB 4.32.415.601 I srv update: - prompt 0x64ea800cc880: 1926 tokens, checkpoints: 1, 51.166 MiB 4.32.415.601 I srv update: - prompt 0x64ea78f02ed0: 968 tokens, checkpoints: 1, 28.702 MiB 4.32.415.602 I srv update: - prompt 0x64ea78fbfd70: 1255 tokens, checkpoints: 1, 35.432 MiB 4.32.415.602 I srv update: - prompt 0x64ea790877e0: 1154 tokens, checkpoints: 1, 33.064 MiB 4.32.415.603 I srv update: - prompt 0x64ea800eb4a0: 4251 tokens, checkpoints: 1, 105.685 MiB 4.32.415.603 I srv update: - prompt 0x64ea80084860: 522 tokens, checkpoints: 1, 18.244 MiB 4.32.415.604 I srv update: - prompt 0x64ea78eb3a90: 516 tokens, checkpoints: 1, 18.103 MiB 4.32.415.604 I srv update: - prompt 0x64ea7b97b9c0: 776 tokens, checkpoints: 1, 24.200 MiB 4.32.415.605 I srv get_availabl: prompt cache update took 18.52 ms 4.32.416.614 I slot launch_slot_: id 3 | task 22243 | processing task, is_child = 0 4.32.416.618 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.32.416.618 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.32.416.636 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.32.416.636 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.32.416.650 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.32.416.650 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.35.570.577 I slot print_timing: id 3 | task 22243 | n_decoded = 266, tg = 88.45 t/s, tg_3s = 88.45 t/s 4.36.161.329 I slot print_timing: id 3 | task 22243 | prompt eval time = 146.67 ms / 41 tokens ( 3.58 ms per token, 279.53 tokens per second) 4.36.161.332 I slot print_timing: id 3 | task 22243 | eval time = 3597.98 ms / 318 tokens ( 11.31 ms per token, 88.38 tokens per second) 4.36.161.333 I slot print_timing: id 3 | task 22243 | total time = 3744.65 ms / 359 tokens 4.36.161.333 I slot print_timing: id 3 | task 22243 | graphs reused = 22318 4.36.161.369 I slot release: id 3 | task 22243 | stop processing: n_tokens = 495, truncated = 0 4.36.161.378 I srv update_slots: all slots are idle 4.36.169.611 I srv operator(): Chat format: peg-native 4.36.173.406 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.835 (> 0.100 thold), f_keep = 0.277 4.36.173.408 I srv get_availabl: updating prompt cache 4.36.173.458 W srv prompt_save: - saving prompt with length 495, total state size = 14.609 MiB (draft: 0.000 MiB) 4.36.187.174 I srv load: - looking for better prompt, base f_keep = 0.277, sim = 0.835 4.36.187.192 I srv update: - cache state: 21 prompts, 719.807 MiB (limits: 8192.000 MiB, 16384 tokens, 288162 est) 4.36.187.193 I srv update: - prompt 0x64ea78fb5d80: 1688 tokens, checkpoints: 1, 45.586 MiB 4.36.187.193 I srv update: - prompt 0x64ea78e666d0: 483 tokens, checkpoints: 1, 17.330 MiB 4.36.187.194 I srv update: - prompt 0x64ea800b3fc0: 1728 tokens, checkpoints: 1, 46.524 MiB 4.36.187.194 I srv update: - prompt 0x64ea78d2e0c0: 2027 tokens, checkpoints: 1, 53.535 MiB 4.36.187.195 I srv update: - prompt 0x64ea7c3e1ff0: 1495 tokens, checkpoints: 1, 41.060 MiB 4.36.187.196 I srv update: - prompt 0x64ea78e7de00: 556 tokens, checkpoints: 1, 19.041 MiB 4.36.187.196 I srv update: - prompt 0x64ea7907df00: 1297 tokens, checkpoints: 1, 36.417 MiB 4.36.187.197 I srv update: - prompt 0x64ea78f01d70: 1491 tokens, checkpoints: 1, 40.966 MiB 4.36.187.197 I srv update: - prompt 0x64ea78f37f80: 759 tokens, checkpoints: 1, 23.802 MiB 4.36.187.198 I srv update: - prompt 0x64ea78e58610: 645 tokens, checkpoints: 1, 21.128 MiB 4.36.187.198 I srv update: - prompt 0x64ea78c08210: 653 tokens, checkpoints: 1, 21.316 MiB 4.36.187.198 I srv update: - prompt 0x64ea8011af90: 635 tokens, checkpoints: 1, 20.894 MiB 4.36.187.199 I srv update: - prompt 0x64ea800cc880: 1926 tokens, checkpoints: 1, 51.166 MiB 4.36.187.199 I srv update: - prompt 0x64ea78f02ed0: 968 tokens, checkpoints: 1, 28.702 MiB 4.36.187.200 I srv update: - prompt 0x64ea78fbfd70: 1255 tokens, checkpoints: 1, 35.432 MiB 4.36.187.200 I srv update: - prompt 0x64ea790877e0: 1154 tokens, checkpoints: 1, 33.064 MiB 4.36.187.201 I srv update: - prompt 0x64ea800eb4a0: 4251 tokens, checkpoints: 1, 105.685 MiB 4.36.187.201 I srv update: - prompt 0x64ea80084860: 522 tokens, checkpoints: 1, 18.244 MiB 4.36.187.202 I srv update: - prompt 0x64ea78eb3a90: 516 tokens, checkpoints: 1, 18.103 MiB 4.36.187.202 I srv update: - prompt 0x64ea7b97b9c0: 776 tokens, checkpoints: 1, 24.200 MiB 4.36.187.203 I srv update: - prompt 0x64ea800d6bb0: 495 tokens, checkpoints: 1, 17.611 MiB 4.36.187.204 I srv get_availabl: prompt cache update took 13.79 ms 4.36.188.164 I slot launch_slot_: id 3 | task 22563 | processing task, is_child = 0 4.36.188.166 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.36.188.167 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.36.188.182 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.36.188.182 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.36.188.192 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.36.188.192 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.39.300.267 I slot print_timing: id 3 | task 22563 | n_decoded = 265, tg = 88.06 t/s, tg_3s = 88.06 t/s 4.42.307.817 I slot print_timing: id 3 | task 22563 | n_decoded = 527, tg = 87.59 t/s, tg_3s = 87.11 t/s 4.45.215.955 I slot print_timing: id 3 | task 22563 | prompt eval time = 102.63 ms / 27 tokens ( 3.80 ms per token, 263.09 tokens per second) 4.45.215.959 I slot print_timing: id 3 | task 22563 | eval time = 8925.11 ms / 779 tokens ( 11.46 ms per token, 87.28 tokens per second) 4.45.215.959 I slot print_timing: id 3 | task 22563 | total time = 9027.74 ms / 806 tokens 4.45.215.960 I slot print_timing: id 3 | task 22563 | graphs reused = 23089 4.45.215.994 I slot release: id 3 | task 22563 | stop processing: n_tokens = 942, truncated = 0 4.45.216.005 I srv update_slots: all slots are idle 4.45.216.593 I srv operator(): operator(): cleaning up before exit...