0.00.065.558 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.065.562 I device_info: 0.00.065.582 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.065.587 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.065.593 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free) 0.00.065.649 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.065.653 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true 0.00.065.693 I srv init: running without SSL 0.00.065.754 I srv init: using 23 threads for HTTP server 0.00.066.078 I srv start: binding port with default address family 0.00.067.281 I srv llama_server: loading model 0.00.067.289 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/mistralai_Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf' 0.00.067.340 I common_init_result: fitting params to device memory ... 0.00.067.342 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) 0.05.202.222 W llama_context: setting new yarn_attn_factor = 1.0000 (mscale == 1.0, mscale_all_dim = 1.0) 0.05.202.232 W llama_context: n_ctx_seq (32768) < n_ctx_train (393216) -- the full capacity of the model will not be utilized 0.05.292.687 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) 0.05.466.573 I srv load_model: initializing slots, n_slots = 4 0.05.536.969 W common_speculative_init: no implementations specified for speculative decoding 0.05.536.972 I slot load_model: id 0 | task -1 | new slot, n_ctx = 32768 0.05.536.975 I slot load_model: id 1 | task -1 | new slot, n_ctx = 32768 0.05.536.976 I slot load_model: id 2 | task -1 | new slot, n_ctx = 32768 0.05.536.976 I slot load_model: id 3 | task -1 | new slot, n_ctx = 32768 0.05.537.084 I srv load_model: prompt cache is enabled, size limit: 8192 MiB 0.05.537.085 I srv load_model: use `--cache-ram 0` to disable the prompt cache 0.05.537.085 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 0.05.537.086 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 0.05.537.102 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task 0.05.540.066 I init: chat template, example_format: '[SYSTEM_PROMPT]You are a helpful assistant[/SYSTEM_PROMPT][INST]Hello[/INST]Hi there[INST]How are you?[/INST]' 0.05.540.465 I srv init: init: chat template, thinking = 1 0.05.540.498 I srv llama_server: model loaded 0.05.540.501 I srv llama_server: server is listening on http://127.0.0.1:8090 0.05.540.506 I srv update_slots: all slots are idle 0.09.966.518 I srv operator(): Chat format: peg-native 0.09.966.729 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1 0.09.966.733 I srv get_availabl: updating prompt cache 0.09.966.740 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 0.09.966.745 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 32768 tokens, 8589934592 est) 0.09.966.745 I srv get_availabl: prompt cache update took 0.01 ms 0.09.966.784 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0 0.09.966.785 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.09.966.786 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.09.966.809 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.09.966.810 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.09.966.828 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.09.966.829 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.15.585.904 I slot print_timing: id 3 | task 0 | n_decoded = 100, tg = 24.33 t/s, tg_3s = 24.33 t/s 0.18.243.494 I slot print_timing: id 3 | task 0 | prompt eval time = 1509.63 ms / 397 tokens ( 3.80 ms per token, 262.98 tokens per second) 0.18.243.498 I slot print_timing: id 3 | task 0 | eval time = 6766.99 ms / 164 tokens ( 41.26 ms per token, 24.24 tokens per second) 0.18.243.498 I slot print_timing: id 3 | task 0 | total time = 8276.63 ms / 561 tokens 0.18.243.503 I slot print_timing: id 3 | task 0 | graphs reused = 0 0.18.243.558 I slot release: id 3 | task 0 | stop processing: n_tokens = 560, truncated = 0 0.18.243.568 I srv update_slots: all slots are idle 0.18.248.491 I srv operator(): Chat format: peg-native 0.18.248.614 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.940 (> 0.100 thold), f_keep = 1.000 0.18.248.638 I slot launch_slot_: id 3 | task 165 | processing task, is_child = 0 0.18.248.640 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.18.248.640 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.18.248.664 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.18.248.664 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.18.248.685 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.18.248.689 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.22.657.890 I slot print_timing: id 3 | task 165 | n_decoded = 100, tg = 24.25 t/s, tg_3s = 24.25 t/s 0.25.324.556 I slot print_timing: id 3 | task 165 | prompt eval time = 285.12 ms / 36 tokens ( 7.92 ms per token, 126.26 tokens per second) 0.25.324.561 I slot print_timing: id 3 | task 165 | eval time = 6790.72 ms / 164 tokens ( 41.41 ms per token, 24.15 tokens per second) 0.25.324.561 I slot print_timing: id 3 | task 165 | total time = 7075.84 ms / 200 tokens 0.25.324.562 I slot print_timing: id 3 | task 165 | graphs reused = 0 0.25.324.609 I slot release: id 3 | task 165 | stop processing: n_tokens = 759, truncated = 0 0.25.324.617 I srv update_slots: all slots are idle 0.25.331.065 I srv operator(): Chat format: peg-native 0.25.331.172 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.974 (> 0.100 thold), f_keep = 1.000 0.25.331.190 I slot launch_slot_: id 3 | task 330 | processing task, is_child = 0 0.25.331.191 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.25.331.192 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.25.331.213 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.25.331.213 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.25.331.230 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.25.331.230 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.26.428.050 I slot print_timing: id 3 | task 330 | prompt eval time = 215.77 ms / 20 tokens ( 10.79 ms per token, 92.69 tokens per second) 0.26.428.054 I slot print_timing: id 3 | task 330 | eval time = 881.03 ms / 22 tokens ( 40.05 ms per token, 24.97 tokens per second) 0.26.428.055 I slot print_timing: id 3 | task 330 | total time = 1096.79 ms / 42 tokens 0.26.428.055 I slot print_timing: id 3 | task 330 | graphs reused = 0 0.26.428.107 I slot release: id 3 | task 330 | stop processing: n_tokens = 800, truncated = 0 0.26.428.113 I srv update_slots: all slots are idle 0.26.451.493 I srv operator(): Chat format: peg-native 0.26.451.605 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.915 (> 0.100 thold), f_keep = 1.000 0.26.451.628 I slot launch_slot_: id 3 | task 353 | processing task, is_child = 0 0.26.451.630 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.26.451.630 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.26.451.656 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.26.451.656 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.26.451.677 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.26.451.677 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.29.077.333 I slot print_timing: id 3 | task 353 | prompt eval time = 441.64 ms / 74 tokens ( 5.97 ms per token, 167.56 tokens per second) 0.29.077.337 I slot print_timing: id 3 | task 353 | eval time = 2183.99 ms / 53 tokens ( 41.21 ms per token, 24.27 tokens per second) 0.29.077.338 I slot print_timing: id 3 | task 353 | total time = 2625.62 ms / 127 tokens 0.29.077.338 I slot print_timing: id 3 | task 353 | graphs reused = 0 0.29.077.404 I slot release: id 3 | task 353 | stop processing: n_tokens = 926, truncated = 0 0.29.077.413 I srv update_slots: all slots are idle 0.29.097.845 I srv operator(): Chat format: peg-native 0.29.097.954 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.796 (> 0.100 thold), f_keep = 0.321 0.29.097.956 I srv get_availabl: updating prompt cache 0.29.098.026 W srv prompt_save: - saving prompt with length 926, total state size = 144.699 MiB (draft: 0.000 MiB) 0.29.228.913 I srv load: - looking for better prompt, base f_keep = 0.321, sim = 0.796 0.29.228.919 I srv update: - cache state: 1 prompts, 144.699 MiB (limits: 8192.000 MiB, 32768 tokens, 52424 est) 0.29.228.920 I srv update: - prompt 0x5e7d93f9e760: 926 tokens, checkpoints: 0, 144.699 MiB 0.29.228.921 I srv get_availabl: prompt cache update took 130.96 ms 0.29.228.947 I slot launch_slot_: id 3 | task 407 | processing task, is_child = 0 0.29.228.948 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.29.228.949 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.29.228.968 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.29.228.968 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.29.228.985 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.29.228.985 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.30.417.634 I slot print_timing: id 3 | task 407 | prompt eval time = 439.12 ms / 76 tokens ( 5.78 ms per token, 173.07 tokens per second) 0.30.417.638 I slot print_timing: id 3 | task 407 | eval time = 749.50 ms / 19 tokens ( 39.45 ms per token, 25.35 tokens per second) 0.30.417.638 I slot print_timing: id 3 | task 407 | total time = 1188.61 ms / 95 tokens 0.30.417.639 I slot print_timing: id 3 | task 407 | graphs reused = 0 0.30.417.679 I slot release: id 3 | task 407 | stop processing: n_tokens = 391, truncated = 0 0.30.417.687 I srv update_slots: all slots are idle 0.30.422.394 I srv operator(): Chat format: peg-native 0.30.422.573 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.813 (> 0.100 thold), f_keep = 1.000 0.30.422.594 I slot launch_slot_: id 3 | task 427 | processing task, is_child = 0 0.30.422.594 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.30.422.596 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.30.422.619 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.30.422.619 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.30.422.638 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.30.422.639 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.34.969.815 I slot print_timing: id 3 | task 427 | n_decoded = 100, tg = 24.17 t/s, tg_3s = 24.17 t/s 0.35.932.840 I slot print_timing: id 3 | task 427 | prompt eval time = 410.34 ms / 90 tokens ( 4.56 ms per token, 219.33 tokens per second) 0.35.932.844 I slot print_timing: id 3 | task 427 | eval time = 5099.84 ms / 123 tokens ( 41.46 ms per token, 24.12 tokens per second) 0.35.932.844 I slot print_timing: id 3 | task 427 | total time = 5510.17 ms / 213 tokens 0.35.932.845 I slot print_timing: id 3 | task 427 | graphs reused = 0 0.35.932.887 I slot release: id 3 | task 427 | stop processing: n_tokens = 603, truncated = 0 0.35.932.895 I srv update_slots: all slots are idle 0.35.939.295 I srv operator(): Chat format: peg-native 0.35.939.412 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.971 (> 0.100 thold), f_keep = 1.000 0.35.939.435 I slot launch_slot_: id 3 | task 551 | processing task, is_child = 0 0.35.939.435 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.35.939.436 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.35.939.458 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.35.939.458 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.35.939.477 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.35.939.478 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.36.937.654 I slot print_timing: id 3 | task 551 | prompt eval time = 213.50 ms / 18 tokens ( 11.86 ms per token, 84.31 tokens per second) 0.36.937.657 I slot print_timing: id 3 | task 551 | eval time = 784.65 ms / 20 tokens ( 39.23 ms per token, 25.49 tokens per second) 0.36.937.658 I slot print_timing: id 3 | task 551 | total time = 998.15 ms / 38 tokens 0.36.937.658 I slot print_timing: id 3 | task 551 | graphs reused = 0 0.36.937.705 I slot release: id 3 | task 551 | stop processing: n_tokens = 640, truncated = 0 0.36.937.711 I srv update_slots: all slots are idle 0.36.960.896 I srv operator(): Chat format: peg-native 0.36.961.009 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.970 (> 0.100 thold), f_keep = 1.000 0.36.961.032 I slot launch_slot_: id 3 | task 572 | processing task, is_child = 0 0.36.961.033 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.36.961.034 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.36.961.053 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.36.961.053 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.36.961.070 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.36.961.071 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.38.952.676 I slot print_timing: id 3 | task 572 | prompt eval time = 214.30 ms / 20 tokens ( 10.71 ms per token, 93.33 tokens per second) 0.38.952.679 I slot print_timing: id 3 | task 572 | eval time = 1777.28 ms / 44 tokens ( 40.39 ms per token, 24.76 tokens per second) 0.38.952.680 I slot print_timing: id 3 | task 572 | total time = 1991.58 ms / 64 tokens 0.38.952.680 I slot print_timing: id 3 | task 572 | graphs reused = 0 0.38.952.730 I slot release: id 3 | task 572 | stop processing: n_tokens = 703, truncated = 0 0.38.952.737 I srv update_slots: all slots are idle 0.38.972.606 I srv operator(): Chat format: peg-native 0.38.972.743 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.780 (> 0.100 thold), f_keep = 0.422 0.38.972.745 I srv get_availabl: updating prompt cache 0.38.972.797 W srv prompt_save: - saving prompt with length 703, total state size = 109.853 MiB (draft: 0.000 MiB) 0.39.062.144 I srv load: - looking for better prompt, base f_keep = 0.422, sim = 0.780 0.39.062.150 I srv update: - cache state: 2 prompts, 254.552 MiB (limits: 8192.000 MiB, 32768 tokens, 52424 est) 0.39.062.151 I srv update: - prompt 0x5e7d93f9e760: 926 tokens, checkpoints: 0, 144.699 MiB 0.39.062.152 I srv update: - prompt 0x5e7d93fb7fd0: 703 tokens, checkpoints: 0, 109.853 MiB 0.39.062.153 I srv get_availabl: prompt cache update took 89.41 ms 0.39.062.179 I slot launch_slot_: id 3 | task 617 | processing task, is_child = 0 0.39.062.180 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.39.062.181 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.39.062.199 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.39.062.200 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.39.062.216 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.39.062.217 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.41.953.801 I slot print_timing: id 3 | task 617 | prompt eval time = 408.69 ms / 84 tokens ( 4.87 ms per token, 205.53 tokens per second) 0.41.953.805 I slot print_timing: id 3 | task 617 | eval time = 2482.86 ms / 61 tokens ( 40.70 ms per token, 24.57 tokens per second) 0.41.953.805 I slot print_timing: id 3 | task 617 | total time = 2891.55 ms / 145 tokens 0.41.953.806 I slot print_timing: id 3 | task 617 | graphs reused = 0 0.41.953.847 I slot release: id 3 | task 617 | stop processing: n_tokens = 441, truncated = 0 0.41.953.853 I srv update_slots: all slots are idle 0.41.958.587 I srv operator(): Chat format: peg-native 0.41.958.693 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.965 (> 0.100 thold), f_keep = 1.000 0.41.958.708 I slot launch_slot_: id 3 | task 679 | processing task, is_child = 0 0.41.958.708 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.41.958.709 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.41.958.729 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.41.958.729 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.41.958.746 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.41.958.747 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.42.828.693 I slot print_timing: id 3 | task 679 | prompt eval time = 125.22 ms / 16 tokens ( 7.83 ms per token, 127.77 tokens per second) 0.42.828.697 I slot print_timing: id 3 | task 679 | eval time = 744.70 ms / 19 tokens ( 39.19 ms per token, 25.51 tokens per second) 0.42.828.697 I slot print_timing: id 3 | task 679 | total time = 869.92 ms / 35 tokens 0.42.828.698 I slot print_timing: id 3 | task 679 | graphs reused = 0 0.42.828.741 I slot release: id 3 | task 679 | stop processing: n_tokens = 475, truncated = 0 0.42.828.747 I srv update_slots: all slots are idle 0.42.849.442 I srv operator(): Chat format: peg-native 0.42.849.556 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.965 (> 0.100 thold), f_keep = 1.000 0.42.849.583 I slot launch_slot_: id 3 | task 699 | processing task, is_child = 0 0.42.849.585 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.42.849.586 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.42.849.605 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.42.849.605 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.42.849.622 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.42.849.622 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.43.766.267 I slot print_timing: id 3 | task 699 | prompt eval time = 213.43 ms / 17 tokens ( 12.55 ms per token, 79.65 tokens per second) 0.43.766.270 I slot print_timing: id 3 | task 699 | eval time = 703.19 ms / 18 tokens ( 39.07 ms per token, 25.60 tokens per second) 0.43.766.271 I slot print_timing: id 3 | task 699 | total time = 916.62 ms / 35 tokens 0.43.766.271 I slot print_timing: id 3 | task 699 | graphs reused = 0 0.43.766.315 I slot release: id 3 | task 699 | stop processing: n_tokens = 509, truncated = 0 0.43.766.321 I srv update_slots: all slots are idle 0.43.773.429 I srv operator(): Chat format: peg-native 0.43.773.535 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.898 (> 0.100 thold), f_keep = 1.000 0.43.773.554 I slot launch_slot_: id 3 | task 718 | processing task, is_child = 0 0.43.773.554 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.43.773.555 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.43.773.577 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.43.773.577 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.43.773.596 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.43.773.596 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.46.498.956 I slot print_timing: id 3 | task 718 | prompt eval time = 313.04 ms / 58 tokens ( 5.40 ms per token, 185.28 tokens per second) 0.46.498.960 I slot print_timing: id 3 | task 718 | eval time = 2412.29 ms / 59 tokens ( 40.89 ms per token, 24.46 tokens per second) 0.46.498.960 I slot print_timing: id 3 | task 718 | total time = 2725.33 ms / 117 tokens 0.46.498.961 I slot print_timing: id 3 | task 718 | graphs reused = 0 0.46.499.006 I slot release: id 3 | task 718 | stop processing: n_tokens = 625, truncated = 0 0.46.499.014 I srv update_slots: all slots are idle 0.46.507.723 I srv operator(): Chat format: peg-native 0.46.507.852 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.974 (> 0.100 thold), f_keep = 1.000 0.46.507.869 I slot launch_slot_: id 3 | task 778 | processing task, is_child = 0 0.46.507.870 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.46.507.871 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.46.507.891 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.46.507.892 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.46.507.908 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.46.507.909 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.47.467.985 I slot print_timing: id 3 | task 778 | prompt eval time = 213.32 ms / 17 tokens ( 12.55 ms per token, 79.69 tokens per second) 0.47.467.988 I slot print_timing: id 3 | task 778 | eval time = 746.73 ms / 19 tokens ( 39.30 ms per token, 25.44 tokens per second) 0.47.467.989 I slot print_timing: id 3 | task 778 | total time = 960.05 ms / 36 tokens 0.47.467.990 I slot print_timing: id 3 | task 778 | graphs reused = 0 0.47.468.034 I slot release: id 3 | task 778 | stop processing: n_tokens = 660, truncated = 0 0.47.468.041 I srv update_slots: all slots are idle 0.47.493.037 I srv operator(): Chat format: peg-native 0.47.493.150 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.969 (> 0.100 thold), f_keep = 1.000 0.47.493.174 I slot launch_slot_: id 3 | task 798 | processing task, is_child = 0 0.47.493.181 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.47.493.181 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.47.493.204 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.47.493.204 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.47.493.223 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.47.493.224 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.49.948.013 I slot print_timing: id 3 | task 798 | prompt eval time = 214.19 ms / 21 tokens ( 10.20 ms per token, 98.04 tokens per second) 0.49.948.016 I slot print_timing: id 3 | task 798 | eval time = 2240.57 ms / 55 tokens ( 40.74 ms per token, 24.55 tokens per second) 0.49.948.017 I slot print_timing: id 3 | task 798 | total time = 2454.76 ms / 76 tokens 0.49.948.018 I slot print_timing: id 3 | task 798 | graphs reused = 0 0.49.948.064 I slot release: id 3 | task 798 | stop processing: n_tokens = 735, truncated = 0 0.49.948.071 I srv update_slots: all slots are idle 0.49.966.603 I srv operator(): Chat format: peg-native 0.49.966.710 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.730 (> 0.100 thold), f_keep = 0.404 0.49.966.712 I srv get_availabl: updating prompt cache 0.49.966.762 W srv prompt_save: - saving prompt with length 735, total state size = 114.853 MiB (draft: 0.000 MiB) 0.50.116.332 I srv load: - looking for better prompt, base f_keep = 0.404, sim = 0.730 0.50.116.339 I srv update: - cache state: 3 prompts, 369.405 MiB (limits: 8192.000 MiB, 32768 tokens, 52424 est) 0.50.116.340 I srv update: - prompt 0x5e7d93f9e760: 926 tokens, checkpoints: 0, 144.699 MiB 0.50.116.341 I srv update: - prompt 0x5e7d93fb7fd0: 703 tokens, checkpoints: 0, 109.853 MiB 0.50.116.341 I srv update: - prompt 0x5e7d94748bf0: 735 tokens, checkpoints: 0, 114.853 MiB 0.50.116.343 I srv get_availabl: prompt cache update took 149.63 ms 0.50.116.375 I slot launch_slot_: id 3 | task 854 | processing task, is_child = 0 0.50.116.376 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.50.116.377 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.50.116.396 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.50.116.396 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.50.116.413 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.50.116.413 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.54.692.614 I slot print_timing: id 3 | task 854 | n_decoded = 100, tg = 24.33 t/s, tg_3s = 24.33 t/s 0.55.232.006 I slot print_timing: id 3 | task 854 | prompt eval time = 466.78 ms / 110 tokens ( 4.24 ms per token, 235.66 tokens per second) 0.55.232.010 I slot print_timing: id 3 | task 854 | eval time = 4648.79 ms / 113 tokens ( 41.14 ms per token, 24.31 tokens per second) 0.55.232.010 I slot print_timing: id 3 | task 854 | total time = 5115.57 ms / 223 tokens 0.55.232.011 I slot print_timing: id 3 | task 854 | graphs reused = 0 0.55.232.049 I slot release: id 3 | task 854 | stop processing: n_tokens = 519, truncated = 0 0.55.232.056 I srv update_slots: all slots are idle 0.55.236.946 I srv operator(): Chat format: peg-native 0.55.237.051 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.968 (> 0.100 thold), f_keep = 1.000 0.55.237.071 I slot launch_slot_: id 3 | task 968 | processing task, is_child = 0 0.55.237.072 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.55.237.072 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.55.237.093 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.55.237.094 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.55.237.113 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.55.237.113 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.56.198.711 I slot print_timing: id 3 | task 968 | prompt eval time = 213.60 ms / 17 tokens ( 12.56 ms per token, 79.59 tokens per second) 0.56.198.715 I slot print_timing: id 3 | task 968 | eval time = 747.97 ms / 19 tokens ( 39.37 ms per token, 25.40 tokens per second) 0.56.198.715 I slot print_timing: id 3 | task 968 | total time = 961.57 ms / 36 tokens 0.56.198.716 I slot print_timing: id 3 | task 968 | graphs reused = 0 0.56.198.758 I slot release: id 3 | task 968 | stop processing: n_tokens = 554, truncated = 0 0.56.198.765 I srv update_slots: all slots are idle 0.56.219.783 I srv operator(): Chat format: peg-native 0.56.219.888 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.954 (> 0.100 thold), f_keep = 1.000 0.56.219.913 I slot launch_slot_: id 3 | task 988 | processing task, is_child = 0 0.56.219.916 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.56.219.916 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.56.219.937 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.56.219.937 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.56.219.954 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.56.219.954 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.58.009.984 I slot print_timing: id 3 | task 988 | prompt eval time = 251.95 ms / 27 tokens ( 9.33 ms per token, 107.16 tokens per second) 0.58.009.988 I slot print_timing: id 3 | task 988 | eval time = 1538.05 ms / 38 tokens ( 40.48 ms per token, 24.71 tokens per second) 0.58.009.988 I slot print_timing: id 3 | task 988 | total time = 1790.00 ms / 65 tokens 0.58.009.989 I slot print_timing: id 3 | task 988 | graphs reused = 0 0.58.010.036 I slot release: id 3 | task 988 | stop processing: n_tokens = 618, truncated = 0 0.58.010.042 I srv update_slots: all slots are idle 0.58.028.339 I srv operator(): Chat format: peg-native 0.58.028.448 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.727 (> 0.100 thold), f_keep = 0.484 0.58.028.450 I srv get_availabl: updating prompt cache 0.58.028.489 W srv prompt_save: - saving prompt with length 618, total state size = 96.571 MiB (draft: 0.000 MiB) 0.58.134.999 I srv load: - looking for better prompt, base f_keep = 0.484, sim = 0.727 0.58.135.008 I srv update: - cache state: 4 prompts, 465.975 MiB (limits: 8192.000 MiB, 32768 tokens, 52424 est) 0.58.135.010 I srv update: - prompt 0x5e7d93f9e760: 926 tokens, checkpoints: 0, 144.699 MiB 0.58.135.010 I srv update: - prompt 0x5e7d93fb7fd0: 703 tokens, checkpoints: 0, 109.853 MiB 0.58.135.011 I srv update: - prompt 0x5e7d94748bf0: 735 tokens, checkpoints: 0, 114.853 MiB 0.58.135.012 I srv update: - prompt 0x5e7d947409d0: 618 tokens, checkpoints: 0, 96.571 MiB 0.58.135.013 I srv get_availabl: prompt cache update took 106.56 ms 0.58.135.039 I slot launch_slot_: id 3 | task 1027 | processing task, is_child = 0 0.58.135.040 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.58.135.040 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.58.135.061 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.58.135.061 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.58.135.078 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.58.135.078 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.02.714.035 I slot print_timing: id 3 | task 1027 | n_decoded = 100, tg = 24.32 t/s, tg_3s = 24.32 t/s 1.05.712.025 I slot print_timing: id 3 | task 1027 | prompt eval time = 467.13 ms / 112 tokens ( 4.17 ms per token, 239.76 tokens per second) 1.05.712.028 I slot print_timing: id 3 | task 1027 | eval time = 7109.79 ms / 172 tokens ( 41.34 ms per token, 24.19 tokens per second) 1.05.712.029 I slot print_timing: id 3 | task 1027 | total time = 7576.92 ms / 284 tokens 1.05.712.029 I slot print_timing: id 3 | task 1027 | graphs reused = 0 1.05.712.069 I slot release: id 3 | task 1027 | stop processing: n_tokens = 582, truncated = 0 1.05.712.077 I srv update_slots: all slots are idle 1.05.717.392 I srv operator(): Chat format: peg-native 1.05.717.503 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.972 (> 0.100 thold), f_keep = 1.000 1.05.717.521 I slot launch_slot_: id 3 | task 1200 | processing task, is_child = 0 1.05.717.521 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.05.717.522 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.05.717.541 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.05.717.542 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.05.717.558 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.05.717.559 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.06.680.649 I slot print_timing: id 3 | task 1200 | prompt eval time = 213.43 ms / 17 tokens ( 12.55 ms per token, 79.65 tokens per second) 1.06.680.653 I slot print_timing: id 3 | task 1200 | eval time = 749.64 ms / 19 tokens ( 39.45 ms per token, 25.35 tokens per second) 1.06.680.653 I slot print_timing: id 3 | task 1200 | total time = 963.06 ms / 36 tokens 1.06.680.654 I slot print_timing: id 3 | task 1200 | graphs reused = 0 1.06.680.700 I slot release: id 3 | task 1200 | stop processing: n_tokens = 617, truncated = 0 1.06.680.708 I srv update_slots: all slots are idle 1.06.701.341 I srv operator(): Chat format: peg-native 1.06.701.468 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.969 (> 0.100 thold), f_keep = 1.000 1.06.701.497 I slot launch_slot_: id 3 | task 1220 | processing task, is_child = 0 1.06.701.499 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 1.06.701.500 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 1.06.701.524 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 1.06.701.524 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 1.06.701.543 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 1.06.701.543 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.08.125.945 I slot print_timing: id 3 | task 1220 | prompt eval time = 214.77 ms / 20 tokens ( 10.74 ms per token, 93.12 tokens per second) 1.08.125.949 I slot print_timing: id 3 | task 1220 | eval time = 1209.61 ms / 30 tokens ( 40.32 ms per token, 24.80 tokens per second) 1.08.125.950 I slot print_timing: id 3 | task 1220 | total time = 1424.37 ms / 50 tokens 1.08.125.951 I slot print_timing: id 3 | task 1220 | graphs reused = 0 1.08.126.001 I slot release: id 3 | task 1220 | stop processing: n_tokens = 666, truncated = 0 1.08.126.008 I srv update_slots: all slots are idle 1.08.143.111 I srv operator(): operator(): cleaning up before exit...