0.00.376.955 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.376.958 I device_info: 0.00.376.979 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.376.994 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.377.003 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free) 0.00.377.065 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.377.069 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true 0.00.377.275 I srv init: running without SSL 0.00.377.470 I srv init: using 23 threads for HTTP server 0.00.377.790 I srv start: binding port with default address family 0.00.379.000 I srv llama_server: loading model 0.00.379.009 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/Devstral-2-123B-Instruct-2512-UD-IQ2_M.gguf' 0.00.379.060 I common_init_result: fitting params to device memory ... 0.00.379.062 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) 0.28.383.676 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized 0.28.569.698 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) 0.29.028.378 I srv load_model: initializing slots, n_slots = 4 0.29.395.491 W common_speculative_init: no implementations specified for speculative decoding 0.29.395.496 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 0.29.395.501 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384 0.29.395.501 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384 0.29.395.501 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384 0.29.395.546 I srv load_model: prompt cache is enabled, size limit: 8192 MiB 0.29.395.574 I srv load_model: use `--cache-ram 0` to disable the prompt cache 0.29.395.574 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 0.29.395.575 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 0.29.395.657 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task 0.29.403.558 I init: chat template, example_format: '[SYSTEM_PROMPT]You are a helpful assistant[/SYSTEM_PROMPT][INST]Hello[/INST]Hi there[INST]How are you?[/INST]' 0.29.404.246 I srv init: init: chat template, thinking = 1 0.29.404.291 I srv llama_server: model loaded 0.29.404.296 I srv llama_server: server is listening on http://127.0.0.1:8090 0.29.404.306 I srv update_slots: all slots are idle 0.29.926.700 I srv operator(): Chat format: peg-native 0.29.927.016 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1 0.29.927.020 I srv get_availabl: updating prompt cache 0.29.927.027 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 0.29.927.031 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) 0.29.927.032 I srv get_availabl: prompt cache update took 0.01 ms 0.29.927.099 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0 0.29.927.100 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.29.927.101 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.29.927.115 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.29.927.115 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.29.927.124 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.29.927.125 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 1.02.225.328 I slot print_timing: id 3 | task 0 | n_decoded = 100, tg = 3.42 t/s, tg_3s = 3.42 t/s 1.05.480.022 I slot print_timing: id 3 | task 0 | n_decoded = 111, tg = 3.42 t/s, tg_3s = 3.38 t/s 1.08.736.314 I slot print_timing: id 3 | task 0 | n_decoded = 122, tg = 3.41 t/s, tg_3s = 3.38 t/s 1.11.981.932 I slot print_timing: id 3 | task 0 | n_decoded = 133, tg = 3.41 t/s, tg_3s = 3.39 t/s 1.15.233.777 I slot print_timing: id 3 | task 0 | n_decoded = 144, tg = 3.41 t/s, tg_3s = 3.38 t/s 1.18.486.165 I slot print_timing: id 3 | task 0 | n_decoded = 155, tg = 3.41 t/s, tg_3s = 3.38 t/s 1.21.736.974 I slot print_timing: id 3 | task 0 | n_decoded = 166, tg = 3.41 t/s, tg_3s = 3.38 t/s 1.25.012.623 I slot print_timing: id 3 | task 0 | n_decoded = 177, tg = 3.40 t/s, tg_3s = 3.36 t/s 1.28.279.704 I slot print_timing: id 3 | task 0 | n_decoded = 188, tg = 3.40 t/s, tg_3s = 3.37 t/s 1.31.543.936 I slot print_timing: id 3 | task 0 | n_decoded = 199, tg = 3.40 t/s, tg_3s = 3.37 t/s 1.34.806.042 I slot print_timing: id 3 | task 0 | n_decoded = 210, tg = 3.40 t/s, tg_3s = 3.37 t/s 1.38.067.018 I slot print_timing: id 3 | task 0 | n_decoded = 221, tg = 3.40 t/s, tg_3s = 3.37 t/s 1.41.332.119 I slot print_timing: id 3 | task 0 | n_decoded = 232, tg = 3.39 t/s, tg_3s = 3.37 t/s 1.44.598.673 I slot print_timing: id 3 | task 0 | n_decoded = 243, tg = 3.39 t/s, tg_3s = 3.37 t/s 1.47.865.920 I slot print_timing: id 3 | task 0 | n_decoded = 254, tg = 3.39 t/s, tg_3s = 3.37 t/s 1.51.135.016 I slot print_timing: id 3 | task 0 | n_decoded = 265, tg = 3.39 t/s, tg_3s = 3.36 t/s 1.54.402.066 I slot print_timing: id 3 | task 0 | n_decoded = 276, tg = 3.39 t/s, tg_3s = 3.37 t/s 1.57.667.809 I slot print_timing: id 3 | task 0 | n_decoded = 287, tg = 3.39 t/s, tg_3s = 3.37 t/s 2.00.932.259 I slot print_timing: id 3 | task 0 | n_decoded = 298, tg = 3.39 t/s, tg_3s = 3.37 t/s 2.04.196.687 I slot print_timing: id 3 | task 0 | n_decoded = 309, tg = 3.39 t/s, tg_3s = 3.37 t/s 2.07.461.228 I slot print_timing: id 3 | task 0 | n_decoded = 320, tg = 3.39 t/s, tg_3s = 3.37 t/s 2.10.731.477 I slot print_timing: id 3 | task 0 | n_decoded = 331, tg = 3.39 t/s, tg_3s = 3.36 t/s 2.14.006.664 I slot print_timing: id 3 | task 0 | n_decoded = 342, tg = 3.39 t/s, tg_3s = 3.36 t/s 2.17.277.329 I slot print_timing: id 3 | task 0 | n_decoded = 353, tg = 3.38 t/s, tg_3s = 3.36 t/s 2.20.550.338 I slot print_timing: id 3 | task 0 | n_decoded = 364, tg = 3.38 t/s, tg_3s = 3.36 t/s 2.23.819.291 I slot print_timing: id 3 | task 0 | n_decoded = 375, tg = 3.38 t/s, tg_3s = 3.36 t/s 2.27.093.375 I slot print_timing: id 3 | task 0 | n_decoded = 386, tg = 3.38 t/s, tg_3s = 3.36 t/s 2.30.359.706 I slot print_timing: id 3 | task 0 | n_decoded = 397, tg = 3.38 t/s, tg_3s = 3.37 t/s 2.33.627.695 I slot print_timing: id 3 | task 0 | n_decoded = 408, tg = 3.38 t/s, tg_3s = 3.37 t/s 2.36.897.166 I slot print_timing: id 3 | task 0 | n_decoded = 419, tg = 3.38 t/s, tg_3s = 3.36 t/s 2.40.176.115 I slot print_timing: id 3 | task 0 | n_decoded = 430, tg = 3.38 t/s, tg_3s = 3.35 t/s 2.43.448.691 I slot print_timing: id 3 | task 0 | n_decoded = 441, tg = 3.38 t/s, tg_3s = 3.36 t/s 2.46.722.940 I slot print_timing: id 3 | task 0 | n_decoded = 452, tg = 3.38 t/s, tg_3s = 3.36 t/s 2.49.997.050 I slot print_timing: id 3 | task 0 | n_decoded = 463, tg = 3.38 t/s, tg_3s = 3.36 t/s 2.53.265.260 I slot print_timing: id 3 | task 0 | n_decoded = 474, tg = 3.38 t/s, tg_3s = 3.37 t/s 2.56.538.088 I slot print_timing: id 3 | task 0 | n_decoded = 485, tg = 3.38 t/s, tg_3s = 3.36 t/s 2.59.809.015 I slot print_timing: id 3 | task 0 | n_decoded = 496, tg = 3.38 t/s, tg_3s = 3.36 t/s 3.03.079.489 I slot print_timing: id 3 | task 0 | n_decoded = 507, tg = 3.38 t/s, tg_3s = 3.36 t/s 3.06.354.560 I slot print_timing: id 3 | task 0 | n_decoded = 518, tg = 3.38 t/s, tg_3s = 3.36 t/s 3.09.631.693 I slot print_timing: id 3 | task 0 | n_decoded = 529, tg = 3.38 t/s, tg_3s = 3.36 t/s 3.12.902.296 I slot print_timing: id 3 | task 0 | n_decoded = 540, tg = 3.38 t/s, tg_3s = 3.36 t/s 3.16.178.811 I slot print_timing: id 3 | task 0 | n_decoded = 551, tg = 3.38 t/s, tg_3s = 3.36 t/s 3.19.453.799 I slot print_timing: id 3 | task 0 | n_decoded = 562, tg = 3.38 t/s, tg_3s = 3.36 t/s 3.22.727.097 I slot print_timing: id 3 | task 0 | n_decoded = 573, tg = 3.38 t/s, tg_3s = 3.36 t/s 3.25.999.730 I slot print_timing: id 3 | task 0 | n_decoded = 584, tg = 3.38 t/s, tg_3s = 3.36 t/s 3.29.271.130 I slot print_timing: id 3 | task 0 | n_decoded = 595, tg = 3.38 t/s, tg_3s = 3.36 t/s 3.32.539.846 I slot print_timing: id 3 | task 0 | n_decoded = 606, tg = 3.38 t/s, tg_3s = 3.37 t/s 3.35.818.291 I slot print_timing: id 3 | task 0 | n_decoded = 617, tg = 3.37 t/s, tg_3s = 3.36 t/s 3.39.095.278 I slot print_timing: id 3 | task 0 | n_decoded = 628, tg = 3.37 t/s, tg_3s = 3.36 t/s 3.42.367.913 I slot print_timing: id 3 | task 0 | n_decoded = 639, tg = 3.37 t/s, tg_3s = 3.36 t/s 3.45.639.912 I slot print_timing: id 3 | task 0 | n_decoded = 650, tg = 3.37 t/s, tg_3s = 3.36 t/s 3.48.913.692 I slot print_timing: id 3 | task 0 | n_decoded = 661, tg = 3.37 t/s, tg_3s = 3.36 t/s 3.52.189.235 I slot print_timing: id 3 | task 0 | n_decoded = 672, tg = 3.37 t/s, tg_3s = 3.36 t/s 3.55.475.838 I slot print_timing: id 3 | task 0 | n_decoded = 683, tg = 3.37 t/s, tg_3s = 3.35 t/s 3.58.772.385 I slot print_timing: id 3 | task 0 | n_decoded = 694, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.02.070.610 I slot print_timing: id 3 | task 0 | n_decoded = 705, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.05.071.009 I slot print_timing: id 3 | task 0 | n_decoded = 715, tg = 3.37 t/s, tg_3s = 3.33 t/s 4.08.359.868 I slot print_timing: id 3 | task 0 | n_decoded = 726, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.11.654.954 I slot print_timing: id 3 | task 0 | n_decoded = 737, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.14.948.925 I slot print_timing: id 3 | task 0 | n_decoded = 748, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.18.242.253 I slot print_timing: id 3 | task 0 | n_decoded = 759, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.21.535.291 I slot print_timing: id 3 | task 0 | n_decoded = 770, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.24.830.287 I slot print_timing: id 3 | task 0 | n_decoded = 781, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.28.128.151 I slot print_timing: id 3 | task 0 | n_decoded = 792, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.31.421.437 I slot print_timing: id 3 | task 0 | n_decoded = 803, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.34.718.381 I slot print_timing: id 3 | task 0 | n_decoded = 814, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.38.016.591 I slot print_timing: id 3 | task 0 | n_decoded = 825, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.41.314.127 I slot print_timing: id 3 | task 0 | n_decoded = 836, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.44.611.282 I slot print_timing: id 3 | task 0 | n_decoded = 847, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.47.908.439 I slot print_timing: id 3 | task 0 | n_decoded = 858, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.51.200.979 I slot print_timing: id 3 | task 0 | n_decoded = 869, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.54.493.761 I slot print_timing: id 3 | task 0 | n_decoded = 880, tg = 3.37 t/s, tg_3s = 3.34 t/s 4.57.789.664 I slot print_timing: id 3 | task 0 | n_decoded = 891, tg = 3.36 t/s, tg_3s = 3.34 t/s 5.01.082.657 I slot print_timing: id 3 | task 0 | n_decoded = 902, tg = 3.36 t/s, tg_3s = 3.34 t/s 5.04.376.006 I slot print_timing: id 3 | task 0 | n_decoded = 913, tg = 3.36 t/s, tg_3s = 3.34 t/s 5.07.670.721 I slot print_timing: id 3 | task 0 | n_decoded = 924, tg = 3.36 t/s, tg_3s = 3.34 t/s 5.10.958.846 I slot print_timing: id 3 | task 0 | n_decoded = 935, tg = 3.36 t/s, tg_3s = 3.35 t/s 5.13.969.088 I slot print_timing: id 3 | task 0 | n_decoded = 945, tg = 3.36 t/s, tg_3s = 3.32 t/s 5.16.972.220 I slot print_timing: id 3 | task 0 | n_decoded = 955, tg = 3.36 t/s, tg_3s = 3.33 t/s 5.17.874.628 I slot print_timing: id 3 | task 0 | prompt eval time = 3065.19 ms / 89 tokens ( 34.44 ms per token, 29.04 tokens per second) 5.17.874.631 I slot print_timing: id 3 | task 0 | eval time = 284882.28 ms / 958 tokens ( 297.37 ms per token, 3.36 tokens per second) 5.17.874.632 I slot print_timing: id 3 | task 0 | total time = 287947.47 ms / 1047 tokens 5.17.874.636 I slot print_timing: id 3 | task 0 | graphs reused = 953 5.17.874.690 I slot release: id 3 | task 0 | stop processing: n_tokens = 1046, truncated = 0 5.17.874.729 I srv update_slots: all slots are idle 5.17.878.422 I srv operator(): Chat format: peg-native 5.17.878.543 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.067 5.17.878.546 I srv get_availabl: updating prompt cache 5.17.878.616 W srv prompt_save: - saving prompt with length 1046, total state size = 359.577 MiB (draft: 0.000 MiB) 5.18.129.974 I srv load: - looking for better prompt, base f_keep = 0.067, sim = 0.833 5.18.129.979 I srv update: - cache state: 1 prompts, 359.577 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 5.18.130.031 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 5.18.130.033 I srv get_availabl: prompt cache update took 251.49 ms 5.18.130.071 I slot launch_slot_: id 3 | task 959 | processing task, is_child = 0 5.18.130.072 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 5.18.130.073 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 5.18.130.085 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 5.18.130.085 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 5.18.130.094 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 5.18.130.101 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 5.48.431.415 I slot print_timing: id 3 | task 959 | n_decoded = 100, tg = 3.41 t/s, tg_3s = 3.41 t/s 5.51.696.800 I slot print_timing: id 3 | task 959 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 5.54.957.353 I slot print_timing: id 3 | task 959 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 5.58.218.218 I slot print_timing: id 3 | task 959 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 6.01.483.085 I slot print_timing: id 3 | task 959 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s 6.04.749.423 I slot print_timing: id 3 | task 959 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.37 t/s 6.08.019.113 I slot print_timing: id 3 | task 959 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s 6.11.297.901 I slot print_timing: id 3 | task 959 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.35 t/s 6.14.567.896 I slot print_timing: id 3 | task 959 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 6.17.840.877 I slot print_timing: id 3 | task 959 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.36 t/s 6.21.111.793 I slot print_timing: id 3 | task 959 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 6.24.382.418 I slot print_timing: id 3 | task 959 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 6.27.654.841 I slot print_timing: id 3 | task 959 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 6.30.929.148 I slot print_timing: id 3 | task 959 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 6.34.200.196 I slot print_timing: id 3 | task 959 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 6.37.471.225 I slot print_timing: id 3 | task 959 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 6.40.742.335 I slot print_timing: id 3 | task 959 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 6.44.016.683 I slot print_timing: id 3 | task 959 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 6.47.290.138 I slot print_timing: id 3 | task 959 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 6.49.075.486 I slot print_timing: id 3 | task 959 | prompt eval time = 935.94 ms / 14 tokens ( 66.85 ms per token, 14.96 tokens per second) 6.49.075.490 I slot print_timing: id 3 | task 959 | eval time = 90009.42 ms / 304 tokens ( 296.08 ms per token, 3.38 tokens per second) 6.49.075.491 I slot print_timing: id 3 | task 959 | total time = 90945.36 ms / 318 tokens 6.49.075.492 I slot print_timing: id 3 | task 959 | graphs reused = 1254 6.49.075.526 I slot release: id 3 | task 959 | stop processing: n_tokens = 387, truncated = 0 6.49.075.533 I srv update_slots: all slots are idle 6.49.078.747 I srv operator(): Chat format: peg-native 6.49.078.855 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.181 6.49.078.858 I srv get_availabl: updating prompt cache 6.49.078.906 W srv prompt_save: - saving prompt with length 387, total state size = 133.038 MiB (draft: 0.000 MiB) 6.49.178.125 I srv load: - looking for better prompt, base f_keep = 0.181, sim = 0.833 6.49.178.133 I srv update: - cache state: 2 prompts, 492.614 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 6.49.178.134 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 6.49.178.135 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 6.49.178.136 I srv get_availabl: prompt cache update took 99.28 ms 6.49.178.177 I slot launch_slot_: id 3 | task 1264 | processing task, is_child = 0 6.49.178.178 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 6.49.178.179 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 6.49.178.191 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 6.49.178.192 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 6.49.178.200 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 6.49.178.200 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 7.19.494.996 I slot print_timing: id 3 | task 1264 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s 7.22.759.352 I slot print_timing: id 3 | task 1264 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 7.26.026.187 I slot print_timing: id 3 | task 1264 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 7.29.294.478 I slot print_timing: id 3 | task 1264 | n_decoded = 133, tg = 3.39 t/s, tg_3s = 3.37 t/s 7.32.556.277 I slot print_timing: id 3 | task 1264 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s 7.35.818.578 I slot print_timing: id 3 | task 1264 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.37 t/s 7.38.193.819 I slot print_timing: id 3 | task 1264 | prompt eval time = 936.55 ms / 14 tokens ( 66.90 ms per token, 14.95 tokens per second) 7.38.193.823 I slot print_timing: id 3 | task 1264 | eval time = 48079.05 ms / 163 tokens ( 294.96 ms per token, 3.39 tokens per second) 7.38.193.824 I slot print_timing: id 3 | task 1264 | total time = 49015.60 ms / 177 tokens 7.38.193.824 I slot print_timing: id 3 | task 1264 | graphs reused = 1415 7.38.193.854 I slot release: id 3 | task 1264 | stop processing: n_tokens = 246, truncated = 0 7.38.193.862 I srv update_slots: all slots are idle 7.38.196.740 I srv operator(): Chat format: peg-native 7.38.196.847 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.843 (> 0.100 thold), f_keep = 0.285 7.38.196.849 I srv get_availabl: updating prompt cache 7.38.196.885 W srv prompt_save: - saving prompt with length 246, total state size = 84.567 MiB (draft: 0.000 MiB) 7.38.262.012 I srv load: - looking for better prompt, base f_keep = 0.285, sim = 0.843 7.38.262.019 I srv update: - cache state: 3 prompts, 577.182 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 7.38.262.058 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 7.38.262.060 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 7.38.262.061 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 7.38.262.062 I srv get_availabl: prompt cache update took 65.21 ms 7.38.262.111 I slot launch_slot_: id 3 | task 1428 | processing task, is_child = 0 7.38.262.112 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 7.38.262.113 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 7.38.262.126 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 7.38.262.126 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 7.38.262.135 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 7.38.262.135 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 8.08.601.411 I slot print_timing: id 3 | task 1428 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s 8.11.867.320 I slot print_timing: id 3 | task 1428 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 8.15.135.247 I slot print_timing: id 3 | task 1428 | n_decoded = 122, tg = 3.39 t/s, tg_3s = 3.37 t/s 8.18.404.286 I slot print_timing: id 3 | task 1428 | n_decoded = 133, tg = 3.39 t/s, tg_3s = 3.36 t/s 8.21.669.967 I slot print_timing: id 3 | task 1428 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s 8.24.935.918 I slot print_timing: id 3 | task 1428 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.37 t/s 8.28.200.703 I slot print_timing: id 3 | task 1428 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.37 t/s 8.31.481.033 I slot print_timing: id 3 | task 1428 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.35 t/s 8.34.755.802 I slot print_timing: id 3 | task 1428 | n_decoded = 188, tg = 3.38 t/s, tg_3s = 3.36 t/s 8.38.030.206 I slot print_timing: id 3 | task 1428 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.36 t/s 8.41.308.190 I slot print_timing: id 3 | task 1428 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 8.44.586.129 I slot print_timing: id 3 | task 1428 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 8.47.863.734 I slot print_timing: id 3 | task 1428 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 8.51.141.292 I slot print_timing: id 3 | task 1428 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 8.54.420.519 I slot print_timing: id 3 | task 1428 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.35 t/s 8.57.700.670 I slot print_timing: id 3 | task 1428 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.35 t/s 9.00.974.945 I slot print_timing: id 3 | task 1428 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 9.04.251.427 I slot print_timing: id 3 | task 1428 | n_decoded = 287, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.07.533.553 I slot print_timing: id 3 | task 1428 | n_decoded = 298, tg = 3.37 t/s, tg_3s = 3.35 t/s 9.10.812.436 I slot print_timing: id 3 | task 1428 | n_decoded = 309, tg = 3.37 t/s, tg_3s = 3.35 t/s 9.14.089.328 I slot print_timing: id 3 | task 1428 | n_decoded = 320, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.17.361.864 I slot print_timing: id 3 | task 1428 | n_decoded = 331, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.20.637.205 I slot print_timing: id 3 | task 1428 | n_decoded = 342, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.23.912.738 I slot print_timing: id 3 | task 1428 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.27.191.219 I slot print_timing: id 3 | task 1428 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.30.467.731 I slot print_timing: id 3 | task 1428 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.33.744.626 I slot print_timing: id 3 | task 1428 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.37.020.105 I slot print_timing: id 3 | task 1428 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.40.294.125 I slot print_timing: id 3 | task 1428 | n_decoded = 408, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.43.570.606 I slot print_timing: id 3 | task 1428 | n_decoded = 419, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.46.847.921 I slot print_timing: id 3 | task 1428 | n_decoded = 430, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.50.133.052 I slot print_timing: id 3 | task 1428 | n_decoded = 441, tg = 3.37 t/s, tg_3s = 3.35 t/s 9.53.407.604 I slot print_timing: id 3 | task 1428 | n_decoded = 452, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.56.685.109 I slot print_timing: id 3 | task 1428 | n_decoded = 463, tg = 3.37 t/s, tg_3s = 3.36 t/s 9.59.965.074 I slot print_timing: id 3 | task 1428 | n_decoded = 474, tg = 3.37 t/s, tg_3s = 3.35 t/s 10.03.242.773 I slot print_timing: id 3 | task 1428 | n_decoded = 485, tg = 3.37 t/s, tg_3s = 3.36 t/s 10.06.519.631 I slot print_timing: id 3 | task 1428 | n_decoded = 496, tg = 3.37 t/s, tg_3s = 3.36 t/s 10.09.800.142 I slot print_timing: id 3 | task 1428 | n_decoded = 507, tg = 3.37 t/s, tg_3s = 3.35 t/s 10.13.077.234 I slot print_timing: id 3 | task 1428 | n_decoded = 518, tg = 3.37 t/s, tg_3s = 3.36 t/s 10.16.355.250 I slot print_timing: id 3 | task 1428 | n_decoded = 529, tg = 3.37 t/s, tg_3s = 3.36 t/s 10.19.626.168 I slot print_timing: id 3 | task 1428 | n_decoded = 540, tg = 3.37 t/s, tg_3s = 3.36 t/s 10.22.900.162 I slot print_timing: id 3 | task 1428 | n_decoded = 551, tg = 3.37 t/s, tg_3s = 3.36 t/s 10.26.176.230 I slot print_timing: id 3 | task 1428 | n_decoded = 562, tg = 3.37 t/s, tg_3s = 3.36 t/s 10.29.451.091 I slot print_timing: id 3 | task 1428 | n_decoded = 573, tg = 3.37 t/s, tg_3s = 3.36 t/s 10.32.723.573 I slot print_timing: id 3 | task 1428 | n_decoded = 584, tg = 3.37 t/s, tg_3s = 3.36 t/s 10.36.002.719 I slot print_timing: id 3 | task 1428 | n_decoded = 595, tg = 3.37 t/s, tg_3s = 3.35 t/s 10.39.280.411 I slot print_timing: id 3 | task 1428 | prompt eval time = 935.26 ms / 13 tokens ( 71.94 ms per token, 13.90 tokens per second) 10.39.280.415 I slot print_timing: id 3 | task 1428 | eval time = 180082.99 ms / 606 tokens ( 297.17 ms per token, 3.37 tokens per second) 10.39.280.415 I slot print_timing: id 3 | task 1428 | total time = 181018.25 ms / 619 tokens 10.39.280.443 I slot print_timing: id 3 | task 1428 | graphs reused = 2017 10.39.280.481 I slot release: id 3 | task 1428 | stop processing: n_tokens = 688, truncated = 0 10.39.280.491 I srv update_slots: all slots are idle 10.39.284.091 I srv operator(): Chat format: peg-native 10.39.284.234 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.729 (> 0.100 thold), f_keep = 0.102 10.39.284.237 I srv get_availabl: updating prompt cache 10.39.284.290 W srv prompt_save: - saving prompt with length 688, total state size = 236.510 MiB (draft: 0.000 MiB) 10.39.456.922 I srv load: - looking for better prompt, base f_keep = 0.102, sim = 0.729 10.39.456.929 I srv update: - cache state: 4 prompts, 813.691 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 10.39.456.930 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 10.39.456.930 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 10.39.456.931 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 10.39.456.932 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 10.39.456.933 I srv get_availabl: prompt cache update took 172.69 ms 10.39.456.962 I slot launch_slot_: id 3 | task 2035 | processing task, is_child = 0 10.39.456.997 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 10.39.456.998 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 10.39.457.010 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 10.39.457.010 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 10.39.457.030 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 10.39.457.031 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 11.10.137.777 I slot print_timing: id 3 | task 2035 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s 11.13.401.556 I slot print_timing: id 3 | task 2035 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 11.16.669.499 I slot print_timing: id 3 | task 2035 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 11.19.937.057 I slot print_timing: id 3 | task 2035 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 11.23.200.015 I slot print_timing: id 3 | task 2035 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s 11.26.461.278 I slot print_timing: id 3 | task 2035 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.37 t/s 11.29.735.991 I slot print_timing: id 3 | task 2035 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s 11.33.009.487 I slot print_timing: id 3 | task 2035 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 11.36.283.133 I slot print_timing: id 3 | task 2035 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 11.39.555.346 I slot print_timing: id 3 | task 2035 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.36 t/s 11.42.829.098 I slot print_timing: id 3 | task 2035 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 11.46.105.378 I slot print_timing: id 3 | task 2035 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 11.49.381.128 I slot print_timing: id 3 | task 2035 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 11.52.655.254 I slot print_timing: id 3 | task 2035 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 11.55.928.526 I slot print_timing: id 3 | task 2035 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 11.59.201.261 I slot print_timing: id 3 | task 2035 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 12.02.471.774 I slot print_timing: id 3 | task 2035 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 12.05.741.574 I slot print_timing: id 3 | task 2035 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 12.09.018.158 I slot print_timing: id 3 | task 2035 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 12.12.289.990 I slot print_timing: id 3 | task 2035 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.36 t/s 12.15.565.428 I slot print_timing: id 3 | task 2035 | n_decoded = 320, tg = 3.38 t/s, tg_3s = 3.36 t/s 12.18.844.930 I slot print_timing: id 3 | task 2035 | n_decoded = 331, tg = 3.37 t/s, tg_3s = 3.35 t/s 12.22.124.681 I slot print_timing: id 3 | task 2035 | n_decoded = 342, tg = 3.37 t/s, tg_3s = 3.35 t/s 12.25.401.394 I slot print_timing: id 3 | task 2035 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.36 t/s 12.28.677.279 I slot print_timing: id 3 | task 2035 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.36 t/s 12.31.954.106 I slot print_timing: id 3 | task 2035 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.36 t/s 12.35.226.191 I slot print_timing: id 3 | task 2035 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.36 t/s 12.38.501.259 I slot print_timing: id 3 | task 2035 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.36 t/s 12.41.773.863 I slot print_timing: id 3 | task 2035 | n_decoded = 408, tg = 3.37 t/s, tg_3s = 3.36 t/s 12.45.058.014 I slot print_timing: id 3 | task 2035 | n_decoded = 419, tg = 3.37 t/s, tg_3s = 3.35 t/s 12.48.336.589 I slot print_timing: id 3 | task 2035 | n_decoded = 430, tg = 3.37 t/s, tg_3s = 3.36 t/s 12.51.610.834 I slot print_timing: id 3 | task 2035 | n_decoded = 441, tg = 3.37 t/s, tg_3s = 3.36 t/s 12.54.887.909 I slot print_timing: id 3 | task 2035 | n_decoded = 452, tg = 3.37 t/s, tg_3s = 3.36 t/s 12.58.167.340 I slot print_timing: id 3 | task 2035 | n_decoded = 463, tg = 3.37 t/s, tg_3s = 3.35 t/s 13.01.441.656 I slot print_timing: id 3 | task 2035 | n_decoded = 474, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.04.716.333 I slot print_timing: id 3 | task 2035 | n_decoded = 485, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.07.994.864 I slot print_timing: id 3 | task 2035 | n_decoded = 496, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.11.271.746 I slot print_timing: id 3 | task 2035 | n_decoded = 507, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.14.549.762 I slot print_timing: id 3 | task 2035 | n_decoded = 518, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.17.826.880 I slot print_timing: id 3 | task 2035 | n_decoded = 529, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.21.100.244 I slot print_timing: id 3 | task 2035 | n_decoded = 540, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.24.375.338 I slot print_timing: id 3 | task 2035 | n_decoded = 551, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.27.654.846 I slot print_timing: id 3 | task 2035 | n_decoded = 562, tg = 3.37 t/s, tg_3s = 3.35 t/s 13.30.936.254 I slot print_timing: id 3 | task 2035 | n_decoded = 573, tg = 3.37 t/s, tg_3s = 3.35 t/s 13.34.211.333 I slot print_timing: id 3 | task 2035 | n_decoded = 584, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.37.486.777 I slot print_timing: id 3 | task 2035 | n_decoded = 595, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.40.762.283 I slot print_timing: id 3 | task 2035 | n_decoded = 606, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.44.034.086 I slot print_timing: id 3 | task 2035 | n_decoded = 617, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.47.308.516 I slot print_timing: id 3 | task 2035 | n_decoded = 628, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.50.582.500 I slot print_timing: id 3 | task 2035 | n_decoded = 639, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.53.858.976 I slot print_timing: id 3 | task 2035 | n_decoded = 650, tg = 3.37 t/s, tg_3s = 3.36 t/s 13.57.129.396 I slot print_timing: id 3 | task 2035 | n_decoded = 661, tg = 3.37 t/s, tg_3s = 3.36 t/s 14.00.408.309 I slot print_timing: id 3 | task 2035 | n_decoded = 672, tg = 3.37 t/s, tg_3s = 3.35 t/s 14.03.413.124 I slot print_timing: id 3 | task 2035 | n_decoded = 682, tg = 3.37 t/s, tg_3s = 3.33 t/s 14.06.414.462 I slot print_timing: id 3 | task 2035 | n_decoded = 692, tg = 3.36 t/s, tg_3s = 3.33 t/s 14.09.711.769 I slot print_timing: id 3 | task 2035 | n_decoded = 703, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.13.003.595 I slot print_timing: id 3 | task 2035 | n_decoded = 714, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.16.299.838 I slot print_timing: id 3 | task 2035 | n_decoded = 725, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.19.595.218 I slot print_timing: id 3 | task 2035 | n_decoded = 736, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.22.890.005 I slot print_timing: id 3 | task 2035 | n_decoded = 747, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.26.183.796 I slot print_timing: id 3 | task 2035 | n_decoded = 758, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.29.479.610 I slot print_timing: id 3 | task 2035 | n_decoded = 769, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.32.776.205 I slot print_timing: id 3 | task 2035 | n_decoded = 780, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.36.072.796 I slot print_timing: id 3 | task 2035 | n_decoded = 791, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.39.370.958 I slot print_timing: id 3 | task 2035 | n_decoded = 802, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.42.670.362 I slot print_timing: id 3 | task 2035 | n_decoded = 813, tg = 3.36 t/s, tg_3s = 3.33 t/s 14.45.967.001 I slot print_timing: id 3 | task 2035 | n_decoded = 824, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.49.266.421 I slot print_timing: id 3 | task 2035 | n_decoded = 835, tg = 3.36 t/s, tg_3s = 3.33 t/s 14.52.562.452 I slot print_timing: id 3 | task 2035 | n_decoded = 846, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.55.855.955 I slot print_timing: id 3 | task 2035 | n_decoded = 857, tg = 3.36 t/s, tg_3s = 3.34 t/s 14.59.150.090 I slot print_timing: id 3 | task 2035 | n_decoded = 868, tg = 3.36 t/s, tg_3s = 3.34 t/s 15.02.444.967 I slot print_timing: id 3 | task 2035 | n_decoded = 879, tg = 3.36 t/s, tg_3s = 3.34 t/s 15.05.445.786 I slot print_timing: id 3 | task 2035 | n_decoded = 889, tg = 3.36 t/s, tg_3s = 3.33 t/s 15.08.742.672 I slot print_timing: id 3 | task 2035 | n_decoded = 900, tg = 3.36 t/s, tg_3s = 3.34 t/s 15.12.035.860 I slot print_timing: id 3 | task 2035 | n_decoded = 911, tg = 3.36 t/s, tg_3s = 3.34 t/s 15.15.325.763 I slot print_timing: id 3 | task 2035 | n_decoded = 922, tg = 3.36 t/s, tg_3s = 3.34 t/s 15.18.332.334 I slot print_timing: id 3 | task 2035 | n_decoded = 932, tg = 3.36 t/s, tg_3s = 3.33 t/s 15.21.337.095 I slot print_timing: id 3 | task 2035 | n_decoded = 942, tg = 3.36 t/s, tg_3s = 3.33 t/s 15.24.339.434 I slot print_timing: id 3 | task 2035 | n_decoded = 952, tg = 3.36 t/s, tg_3s = 3.33 t/s 15.27.345.378 I slot print_timing: id 3 | task 2035 | n_decoded = 962, tg = 3.36 t/s, tg_3s = 3.33 t/s 15.30.348.831 I slot print_timing: id 3 | task 2035 | n_decoded = 972, tg = 3.36 t/s, tg_3s = 3.33 t/s 15.33.351.103 I slot print_timing: id 3 | task 2035 | n_decoded = 982, tg = 3.36 t/s, tg_3s = 3.33 t/s 15.36.352.848 I slot print_timing: id 3 | task 2035 | n_decoded = 992, tg = 3.36 t/s, tg_3s = 3.33 t/s 15.39.356.381 I slot print_timing: id 3 | task 2035 | n_decoded = 1002, tg = 3.36 t/s, tg_3s = 3.33 t/s 15.42.359.213 I slot print_timing: id 3 | task 2035 | n_decoded = 1012, tg = 3.36 t/s, tg_3s = 3.33 t/s 15.45.365.284 I slot print_timing: id 3 | task 2035 | n_decoded = 1022, tg = 3.36 t/s, tg_3s = 3.33 t/s 15.48.370.073 I slot print_timing: id 3 | task 2035 | n_decoded = 1032, tg = 3.35 t/s, tg_3s = 3.33 t/s 15.51.373.214 I slot print_timing: id 3 | task 2035 | n_decoded = 1042, tg = 3.35 t/s, tg_3s = 3.33 t/s 15.54.374.315 I slot print_timing: id 3 | task 2035 | n_decoded = 1052, tg = 3.35 t/s, tg_3s = 3.33 t/s 15.57.378.878 I slot print_timing: id 3 | task 2035 | n_decoded = 1062, tg = 3.35 t/s, tg_3s = 3.33 t/s 16.00.382.676 I slot print_timing: id 3 | task 2035 | n_decoded = 1072, tg = 3.35 t/s, tg_3s = 3.33 t/s 16.03.389.119 I slot print_timing: id 3 | task 2035 | n_decoded = 1082, tg = 3.35 t/s, tg_3s = 3.33 t/s 16.06.391.585 I slot print_timing: id 3 | task 2035 | n_decoded = 1092, tg = 3.35 t/s, tg_3s = 3.33 t/s 16.09.393.719 I slot print_timing: id 3 | task 2035 | n_decoded = 1102, tg = 3.35 t/s, tg_3s = 3.33 t/s 16.12.396.464 I slot print_timing: id 3 | task 2035 | n_decoded = 1112, tg = 3.35 t/s, tg_3s = 3.33 t/s 16.15.400.176 I slot print_timing: id 3 | task 2035 | n_decoded = 1122, tg = 3.35 t/s, tg_3s = 3.33 t/s 16.18.406.434 I slot print_timing: id 3 | task 2035 | n_decoded = 1132, tg = 3.35 t/s, tg_3s = 3.33 t/s 16.21.415.734 I slot print_timing: id 3 | task 2035 | n_decoded = 1142, tg = 3.35 t/s, tg_3s = 3.32 t/s 16.22.017.740 I slot print_timing: id 3 | task 2035 | prompt eval time = 1307.67 ms / 26 tokens ( 50.29 ms per token, 19.88 tokens per second) 16.22.017.744 I slot print_timing: id 3 | task 2035 | eval time = 341253.01 ms / 1144 tokens ( 298.30 ms per token, 3.35 tokens per second) 16.22.017.775 I slot print_timing: id 3 | task 2035 | total time = 342560.68 ms / 1170 tokens 16.22.017.777 I slot print_timing: id 3 | task 2035 | graphs reused = 3155 16.22.017.814 I slot release: id 3 | task 2035 | stop processing: n_tokens = 1239, truncated = 0 16.22.017.827 I srv update_slots: all slots are idle 16.22.021.025 I srv operator(): Chat format: peg-native 16.22.021.131 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.824 (> 0.100 thold), f_keep = 0.056 16.22.021.134 I srv get_availabl: updating prompt cache 16.22.021.216 W srv prompt_save: - saving prompt with length 1239, total state size = 425.922 MiB (draft: 0.000 MiB) 16.22.313.551 I srv load: - looking for better prompt, base f_keep = 0.056, sim = 0.824 16.22.313.558 I srv update: - cache state: 5 prompts, 1239.614 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 16.22.313.559 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 16.22.313.560 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 16.22.313.560 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 16.22.313.561 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 16.22.313.562 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 16.22.313.592 I srv get_availabl: prompt cache update took 292.43 ms 16.22.313.620 I slot launch_slot_: id 3 | task 3180 | processing task, is_child = 0 16.22.313.621 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 16.22.313.622 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 16.22.313.635 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 16.22.313.636 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 16.22.313.645 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 16.22.313.645 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 16.52.627.379 I slot print_timing: id 3 | task 3180 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s 16.55.896.452 I slot print_timing: id 3 | task 3180 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.36 t/s 16.59.157.158 I slot print_timing: id 3 | task 3180 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 17.02.423.614 I slot print_timing: id 3 | task 3180 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 17.05.688.673 I slot print_timing: id 3 | task 3180 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s 17.08.951.593 I slot print_timing: id 3 | task 3180 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.37 t/s 17.12.212.181 I slot print_timing: id 3 | task 3180 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.37 t/s 17.15.484.292 I slot print_timing: id 3 | task 3180 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 17.18.761.067 I slot print_timing: id 3 | task 3180 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 17.22.034.552 I slot print_timing: id 3 | task 3180 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.36 t/s 17.25.309.219 I slot print_timing: id 3 | task 3180 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 17.28.583.108 I slot print_timing: id 3 | task 3180 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 17.31.856.217 I slot print_timing: id 3 | task 3180 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 17.35.128.208 I slot print_timing: id 3 | task 3180 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 17.38.401.421 I slot print_timing: id 3 | task 3180 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 17.41.672.574 I slot print_timing: id 3 | task 3180 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 17.44.940.835 I slot print_timing: id 3 | task 3180 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.37 t/s 17.48.214.949 I slot print_timing: id 3 | task 3180 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 17.51.489.366 I slot print_timing: id 3 | task 3180 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 17.54.754.824 I slot print_timing: id 3 | task 3180 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.37 t/s 17.58.025.166 I slot print_timing: id 3 | task 3180 | n_decoded = 320, tg = 3.38 t/s, tg_3s = 3.36 t/s 18.01.297.385 I slot print_timing: id 3 | task 3180 | n_decoded = 331, tg = 3.38 t/s, tg_3s = 3.36 t/s 18.04.569.639 I slot print_timing: id 3 | task 3180 | n_decoded = 342, tg = 3.38 t/s, tg_3s = 3.36 t/s 18.07.843.340 I slot print_timing: id 3 | task 3180 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.36 t/s 18.11.117.437 I slot print_timing: id 3 | task 3180 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.36 t/s 18.14.386.647 I slot print_timing: id 3 | task 3180 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.36 t/s 18.17.659.724 I slot print_timing: id 3 | task 3180 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.36 t/s 18.20.930.828 I slot print_timing: id 3 | task 3180 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.36 t/s 18.24.205.924 I slot print_timing: id 3 | task 3180 | n_decoded = 408, tg = 3.37 t/s, tg_3s = 3.36 t/s 18.27.477.974 I slot print_timing: id 3 | task 3180 | n_decoded = 419, tg = 3.37 t/s, tg_3s = 3.36 t/s 18.30.761.118 I slot print_timing: id 3 | task 3180 | n_decoded = 430, tg = 3.37 t/s, tg_3s = 3.35 t/s 18.34.038.213 I slot print_timing: id 3 | task 3180 | n_decoded = 441, tg = 3.37 t/s, tg_3s = 3.36 t/s 18.37.315.882 I slot print_timing: id 3 | task 3180 | n_decoded = 452, tg = 3.37 t/s, tg_3s = 3.36 t/s 18.40.586.681 I slot print_timing: id 3 | task 3180 | n_decoded = 463, tg = 3.37 t/s, tg_3s = 3.36 t/s 18.43.856.637 I slot print_timing: id 3 | task 3180 | n_decoded = 474, tg = 3.37 t/s, tg_3s = 3.36 t/s 18.47.133.180 I slot print_timing: id 3 | task 3180 | n_decoded = 485, tg = 3.37 t/s, tg_3s = 3.36 t/s 18.50.412.313 I slot print_timing: id 3 | task 3180 | n_decoded = 496, tg = 3.37 t/s, tg_3s = 3.35 t/s 18.53.695.238 I slot print_timing: id 3 | task 3180 | n_decoded = 507, tg = 3.37 t/s, tg_3s = 3.35 t/s 18.56.977.634 I slot print_timing: id 3 | task 3180 | n_decoded = 518, tg = 3.37 t/s, tg_3s = 3.35 t/s 19.00.257.617 I slot print_timing: id 3 | task 3180 | n_decoded = 529, tg = 3.37 t/s, tg_3s = 3.35 t/s 19.03.532.242 I slot print_timing: id 3 | task 3180 | n_decoded = 540, tg = 3.37 t/s, tg_3s = 3.36 t/s 19.06.809.897 I slot print_timing: id 3 | task 3180 | n_decoded = 551, tg = 3.37 t/s, tg_3s = 3.36 t/s 19.10.085.137 I slot print_timing: id 3 | task 3180 | n_decoded = 562, tg = 3.37 t/s, tg_3s = 3.36 t/s 19.13.359.503 I slot print_timing: id 3 | task 3180 | n_decoded = 573, tg = 3.37 t/s, tg_3s = 3.36 t/s 19.16.636.136 I slot print_timing: id 3 | task 3180 | n_decoded = 584, tg = 3.37 t/s, tg_3s = 3.36 t/s 19.19.612.175 I slot print_timing: id 3 | task 3180 | prompt eval time = 936.72 ms / 15 tokens ( 62.45 ms per token, 16.01 tokens per second) 19.19.612.179 I slot print_timing: id 3 | task 3180 | eval time = 176361.79 ms / 594 tokens ( 296.91 ms per token, 3.37 tokens per second) 19.19.612.179 I slot print_timing: id 3 | task 3180 | total time = 177298.51 ms / 609 tokens 19.19.612.180 I slot print_timing: id 3 | task 3180 | graphs reused = 3745 19.19.612.217 I slot release: id 3 | task 3180 | stop processing: n_tokens = 678, truncated = 0 19.19.612.227 I srv update_slots: all slots are idle 19.19.615.128 I srv operator(): Chat format: peg-native 19.19.615.262 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.877 (> 0.100 thold), f_keep = 0.105 19.19.615.264 I srv get_availabl: updating prompt cache 19.19.615.312 W srv prompt_save: - saving prompt with length 678, total state size = 233.072 MiB (draft: 0.000 MiB) 19.19.781.917 I srv load: - looking for better prompt, base f_keep = 0.105, sim = 0.877 19.19.781.925 I srv update: - cache state: 6 prompts, 1472.686 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 19.19.781.926 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 19.19.781.927 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 19.19.781.928 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 19.19.781.928 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 19.19.781.929 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 19.19.781.929 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 19.19.781.930 I srv get_availabl: prompt cache update took 166.66 ms 19.19.781.960 I slot launch_slot_: id 3 | task 3775 | processing task, is_child = 0 19.19.781.961 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 19.19.781.961 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 19.19.782.009 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 19.19.782.009 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 19.19.782.037 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 19.19.782.037 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 19.50.097.287 I slot print_timing: id 3 | task 3775 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s 19.53.361.689 I slot print_timing: id 3 | task 3775 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 19.56.626.167 I slot print_timing: id 3 | task 3775 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 19.59.887.424 I slot print_timing: id 3 | task 3775 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 20.03.149.420 I slot print_timing: id 3 | task 3775 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s 20.06.412.563 I slot print_timing: id 3 | task 3775 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.37 t/s 20.09.672.705 I slot print_timing: id 3 | task 3775 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.37 t/s 20.12.935.142 I slot print_timing: id 3 | task 3775 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.37 t/s 20.16.212.267 I slot print_timing: id 3 | task 3775 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 20.19.479.021 I slot print_timing: id 3 | task 3775 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.37 t/s 20.22.755.381 I slot print_timing: id 3 | task 3775 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 20.26.032.212 I slot print_timing: id 3 | task 3775 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 20.29.304.248 I slot print_timing: id 3 | task 3775 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 20.32.578.773 I slot print_timing: id 3 | task 3775 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 20.35.851.783 I slot print_timing: id 3 | task 3775 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 20.39.123.357 I slot print_timing: id 3 | task 3775 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 20.42.398.095 I slot print_timing: id 3 | task 3775 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 20.45.674.738 I slot print_timing: id 3 | task 3775 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 20.48.056.379 I slot print_timing: id 3 | task 3775 | prompt eval time = 933.27 ms / 10 tokens ( 93.33 ms per token, 10.71 tokens per second) 20.48.056.382 I slot print_timing: id 3 | task 3775 | eval time = 87341.05 ms / 295 tokens ( 296.07 ms per token, 3.38 tokens per second) 20.48.056.383 I slot print_timing: id 3 | task 3775 | total time = 88274.32 ms / 305 tokens 20.48.056.384 I slot print_timing: id 3 | task 3775 | graphs reused = 4037 20.48.056.418 I slot release: id 3 | task 3775 | stop processing: n_tokens = 375, truncated = 0 20.48.056.426 I srv update_slots: all slots are idle 20.48.059.341 I srv operator(): Chat format: peg-native 20.48.059.443 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.855 (> 0.100 thold), f_keep = 0.189 20.48.059.445 I srv get_availabl: updating prompt cache 20.48.059.477 W srv prompt_save: - saving prompt with length 375, total state size = 128.913 MiB (draft: 0.000 MiB) 20.48.155.880 I srv load: - looking for better prompt, base f_keep = 0.189, sim = 0.855 20.48.155.891 I srv update: - cache state: 7 prompts, 1601.599 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 20.48.155.891 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 20.48.155.892 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 20.48.155.893 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 20.48.155.894 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 20.48.155.894 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 20.48.155.895 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 20.48.155.895 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 20.48.155.897 I srv get_availabl: prompt cache update took 96.45 ms 20.48.155.946 I slot launch_slot_: id 3 | task 4071 | processing task, is_child = 0 20.48.155.946 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 20.48.155.947 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 20.48.155.959 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 20.48.155.960 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 20.48.155.968 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 20.48.155.968 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 21.18.478.230 I slot print_timing: id 3 | task 4071 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s 21.21.743.247 I slot print_timing: id 3 | task 4071 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 21.25.006.678 I slot print_timing: id 3 | task 4071 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 21.28.269.867 I slot print_timing: id 3 | task 4071 | n_decoded = 133, tg = 3.39 t/s, tg_3s = 3.37 t/s 21.31.535.221 I slot print_timing: id 3 | task 4071 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s 21.34.799.096 I slot print_timing: id 3 | task 4071 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.37 t/s 21.38.057.409 I slot print_timing: id 3 | task 4071 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.38 t/s 21.41.330.371 I slot print_timing: id 3 | task 4071 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 21.44.603.120 I slot print_timing: id 3 | task 4071 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 21.47.874.851 I slot print_timing: id 3 | task 4071 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.36 t/s 21.51.152.839 I slot print_timing: id 3 | task 4071 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 21.54.426.621 I slot print_timing: id 3 | task 4071 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 21.57.702.580 I slot print_timing: id 3 | task 4071 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 22.00.974.618 I slot print_timing: id 3 | task 4071 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 22.04.241.836 I slot print_timing: id 3 | task 4071 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.37 t/s 22.07.513.908 I slot print_timing: id 3 | task 4071 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 22.10.783.853 I slot print_timing: id 3 | task 4071 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 22.14.052.409 I slot print_timing: id 3 | task 4071 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.37 t/s 22.17.320.512 I slot print_timing: id 3 | task 4071 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.37 t/s 22.20.589.260 I slot print_timing: id 3 | task 4071 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.37 t/s 22.23.854.859 I slot print_timing: id 3 | task 4071 | n_decoded = 320, tg = 3.38 t/s, tg_3s = 3.37 t/s 22.27.121.943 I slot print_timing: id 3 | task 4071 | n_decoded = 331, tg = 3.38 t/s, tg_3s = 3.37 t/s 22.30.396.050 I slot print_timing: id 3 | task 4071 | n_decoded = 342, tg = 3.38 t/s, tg_3s = 3.36 t/s 22.33.666.119 I slot print_timing: id 3 | task 4071 | n_decoded = 353, tg = 3.38 t/s, tg_3s = 3.36 t/s 22.36.934.668 I slot print_timing: id 3 | task 4071 | n_decoded = 364, tg = 3.38 t/s, tg_3s = 3.37 t/s 22.40.202.825 I slot print_timing: id 3 | task 4071 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.37 t/s 22.43.473.422 I slot print_timing: id 3 | task 4071 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.36 t/s 22.46.742.035 I slot print_timing: id 3 | task 4071 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.37 t/s 22.50.011.331 I slot print_timing: id 3 | task 4071 | n_decoded = 408, tg = 3.37 t/s, tg_3s = 3.36 t/s 22.53.284.235 I slot print_timing: id 3 | task 4071 | n_decoded = 419, tg = 3.37 t/s, tg_3s = 3.36 t/s 22.56.554.696 I slot print_timing: id 3 | task 4071 | n_decoded = 430, tg = 3.37 t/s, tg_3s = 3.36 t/s 22.59.837.227 I slot print_timing: id 3 | task 4071 | n_decoded = 441, tg = 3.37 t/s, tg_3s = 3.35 t/s 23.03.111.987 I slot print_timing: id 3 | task 4071 | n_decoded = 452, tg = 3.37 t/s, tg_3s = 3.36 t/s 23.06.384.596 I slot print_timing: id 3 | task 4071 | n_decoded = 463, tg = 3.37 t/s, tg_3s = 3.36 t/s 23.06.980.220 I slot print_timing: id 3 | task 4071 | prompt eval time = 935.12 ms / 12 tokens ( 77.93 ms per token, 12.83 tokens per second) 23.06.980.223 I slot print_timing: id 3 | task 4071 | eval time = 137889.11 ms / 465 tokens ( 296.54 ms per token, 3.37 tokens per second) 23.06.980.224 I slot print_timing: id 3 | task 4071 | total time = 138824.22 ms / 477 tokens 23.06.980.225 I slot print_timing: id 3 | task 4071 | graphs reused = 4498 23.06.980.266 I slot release: id 3 | task 4071 | stop processing: n_tokens = 547, truncated = 0 23.06.980.276 I srv update_slots: all slots are idle 23.06.983.285 I srv operator(): Chat format: peg-native 23.06.983.402 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.631 (> 0.100 thold), f_keep = 0.128 23.06.983.405 I srv get_availabl: updating prompt cache 23.06.983.457 W srv prompt_save: - saving prompt with length 547, total state size = 188.040 MiB (draft: 0.000 MiB) 23.07.119.016 I srv load: - looking for better prompt, base f_keep = 0.128, sim = 0.631 23.07.119.025 I srv update: - cache state: 8 prompts, 1789.638 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 23.07.119.026 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 23.07.119.027 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 23.07.119.029 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 23.07.119.032 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 23.07.119.033 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 23.07.119.034 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 23.07.119.035 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 23.07.119.035 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 23.07.119.039 I srv get_availabl: prompt cache update took 135.63 ms 23.07.119.071 I slot launch_slot_: id 3 | task 4537 | processing task, is_child = 0 23.07.119.072 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 23.07.119.073 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 23.07.119.085 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 23.07.119.085 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 23.07.119.095 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 23.07.119.096 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 23.38.086.571 I slot print_timing: id 3 | task 4537 | n_decoded = 100, tg = 3.41 t/s, tg_3s = 3.41 t/s 23.41.345.884 I slot print_timing: id 3 | task 4537 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 23.44.608.815 I slot print_timing: id 3 | task 4537 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 23.47.869.152 I slot print_timing: id 3 | task 4537 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 23.51.131.093 I slot print_timing: id 3 | task 4537 | n_decoded = 144, tg = 3.40 t/s, tg_3s = 3.37 t/s 23.54.412.979 I slot print_timing: id 3 | task 4537 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.35 t/s 23.57.690.688 I slot print_timing: id 3 | task 4537 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s 24.00.966.417 I slot print_timing: id 3 | task 4537 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 24.04.240.234 I slot print_timing: id 3 | task 4537 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 24.07.513.686 I slot print_timing: id 3 | task 4537 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.10.787.027 I slot print_timing: id 3 | task 4537 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.14.064.972 I slot print_timing: id 3 | task 4537 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.17.336.272 I slot print_timing: id 3 | task 4537 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.20.607.798 I slot print_timing: id 3 | task 4537 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.23.878.361 I slot print_timing: id 3 | task 4537 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.27.148.754 I slot print_timing: id 3 | task 4537 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.30.418.655 I slot print_timing: id 3 | task 4537 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.33.689.133 I slot print_timing: id 3 | task 4537 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.36.959.160 I slot print_timing: id 3 | task 4537 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.40.233.273 I slot print_timing: id 3 | task 4537 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.43.504.104 I slot print_timing: id 3 | task 4537 | n_decoded = 320, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.46.775.536 I slot print_timing: id 3 | task 4537 | n_decoded = 331, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.50.051.595 I slot print_timing: id 3 | task 4537 | n_decoded = 342, tg = 3.38 t/s, tg_3s = 3.36 t/s 24.53.328.916 I slot print_timing: id 3 | task 4537 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.36 t/s 24.56.604.516 I slot print_timing: id 3 | task 4537 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.36 t/s 24.58.689.367 I slot print_timing: id 3 | task 4537 | prompt eval time = 1599.21 ms / 41 tokens ( 39.01 ms per token, 25.64 tokens per second) 24.58.689.372 I slot print_timing: id 3 | task 4537 | eval time = 109971.03 ms / 371 tokens ( 296.42 ms per token, 3.37 tokens per second) 24.58.689.372 I slot print_timing: id 3 | task 4537 | total time = 111570.25 ms / 412 tokens 24.58.689.373 I slot print_timing: id 3 | task 4537 | graphs reused = 4866 24.58.689.415 I slot release: id 3 | task 4537 | stop processing: n_tokens = 481, truncated = 0 24.58.689.422 I srv update_slots: all slots are idle 24.58.692.570 I srv operator(): Chat format: peg-native 24.58.692.676 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.542 (> 0.100 thold), f_keep = 0.148 24.58.692.678 I srv get_availabl: updating prompt cache 24.58.692.719 W srv prompt_save: - saving prompt with length 481, total state size = 165.351 MiB (draft: 0.000 MiB) 24.58.815.897 I srv load: - looking for better prompt, base f_keep = 0.148, sim = 0.542 24.58.815.908 I srv update: - cache state: 9 prompts, 1954.990 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 24.58.815.909 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 24.58.815.910 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 24.58.815.910 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 24.58.815.911 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 24.58.815.911 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 24.58.815.913 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 24.58.815.913 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 24.58.815.914 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 24.58.815.915 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 24.58.815.918 I srv get_availabl: prompt cache update took 123.24 ms 24.58.815.963 I slot launch_slot_: id 3 | task 4909 | processing task, is_child = 0 24.58.815.964 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 24.58.815.965 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 24.58.815.978 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 24.58.815.978 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 24.58.815.993 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 24.58.815.993 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 25.30.115.892 I slot print_timing: id 3 | task 4909 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s 25.33.375.562 I slot print_timing: id 3 | task 4909 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 25.36.638.468 I slot print_timing: id 3 | task 4909 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 25.39.916.901 I slot print_timing: id 3 | task 4909 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.36 t/s 25.43.189.841 I slot print_timing: id 3 | task 4909 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.36 t/s 25.46.459.086 I slot print_timing: id 3 | task 4909 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.36 t/s 25.49.729.231 I slot print_timing: id 3 | task 4909 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s 25.52.999.828 I slot print_timing: id 3 | task 4909 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 25.56.276.404 I slot print_timing: id 3 | task 4909 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 25.59.554.637 I slot print_timing: id 3 | task 4909 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.36 t/s 26.02.834.291 I slot print_timing: id 3 | task 4909 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.35 t/s 26.06.108.236 I slot print_timing: id 3 | task 4909 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 26.09.378.995 I slot print_timing: id 3 | task 4909 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 26.12.648.710 I slot print_timing: id 3 | task 4909 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 26.15.917.757 I slot print_timing: id 3 | task 4909 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 26.19.185.759 I slot print_timing: id 3 | task 4909 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.37 t/s 26.22.458.199 I slot print_timing: id 3 | task 4909 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 26.25.731.885 I slot print_timing: id 3 | task 4909 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 26.29.005.620 I slot print_timing: id 3 | task 4909 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 26.32.280.391 I slot print_timing: id 3 | task 4909 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.36 t/s 26.35.553.501 I slot print_timing: id 3 | task 4909 | n_decoded = 320, tg = 3.38 t/s, tg_3s = 3.36 t/s 26.38.824.706 I slot print_timing: id 3 | task 4909 | n_decoded = 331, tg = 3.37 t/s, tg_3s = 3.36 t/s 26.42.098.104 I slot print_timing: id 3 | task 4909 | n_decoded = 342, tg = 3.37 t/s, tg_3s = 3.36 t/s 26.45.374.908 I slot print_timing: id 3 | task 4909 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.36 t/s 26.48.650.115 I slot print_timing: id 3 | task 4909 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.36 t/s 26.51.922.146 I slot print_timing: id 3 | task 4909 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.36 t/s 26.55.205.813 I slot print_timing: id 3 | task 4909 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.35 t/s 26.57.585.648 I slot print_timing: id 3 | task 4909 | prompt eval time = 1928.29 ms / 60 tokens ( 32.14 ms per token, 31.12 tokens per second) 26.57.585.651 I slot print_timing: id 3 | task 4909 | eval time = 116841.34 ms / 394 tokens ( 296.55 ms per token, 3.37 tokens per second) 26.57.585.652 I slot print_timing: id 3 | task 4909 | total time = 118769.63 ms / 454 tokens 26.57.585.653 I slot print_timing: id 3 | task 4909 | graphs reused = 5256 26.57.585.694 I slot release: id 3 | task 4909 | stop processing: n_tokens = 524, truncated = 0 26.57.585.702 I srv update_slots: all slots are idle 26.57.588.812 I srv operator(): Chat format: peg-native 26.57.588.922 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.660 (> 0.100 thold), f_keep = 0.134 26.57.588.924 I srv get_availabl: updating prompt cache 26.57.588.996 W srv prompt_save: - saving prompt with length 524, total state size = 180.133 MiB (draft: 0.000 MiB) 26.57.721.807 I srv load: - looking for better prompt, base f_keep = 0.134, sim = 0.660 26.57.721.816 I srv update: - cache state: 10 prompts, 2135.123 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 26.57.721.817 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 26.57.721.818 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 26.57.721.818 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 26.57.721.819 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 26.57.721.820 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 26.57.721.820 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 26.57.721.821 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 26.57.721.821 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 26.57.721.822 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 26.57.721.822 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 26.57.721.824 I srv get_availabl: prompt cache update took 132.90 ms 26.57.721.851 I slot launch_slot_: id 3 | task 5304 | processing task, is_child = 0 26.57.721.852 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 26.57.721.853 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 26.57.721.865 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 26.57.721.865 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 26.57.721.873 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 26.57.721.874 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 27.28.535.465 I slot print_timing: id 3 | task 5304 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s 27.31.804.485 I slot print_timing: id 3 | task 5304 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.36 t/s 27.35.071.419 I slot print_timing: id 3 | task 5304 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 27.38.336.557 I slot print_timing: id 3 | task 5304 | n_decoded = 133, tg = 3.39 t/s, tg_3s = 3.37 t/s 27.41.604.776 I slot print_timing: id 3 | task 5304 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s 27.44.885.588 I slot print_timing: id 3 | task 5304 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.35 t/s 27.48.157.051 I slot print_timing: id 3 | task 5304 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s 27.51.431.688 I slot print_timing: id 3 | task 5304 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 27.54.700.282 I slot print_timing: id 3 | task 5304 | n_decoded = 188, tg = 3.38 t/s, tg_3s = 3.37 t/s 27.57.975.943 I slot print_timing: id 3 | task 5304 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.36 t/s 28.01.255.144 I slot print_timing: id 3 | task 5304 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.35 t/s 28.04.534.366 I slot print_timing: id 3 | task 5304 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.35 t/s 28.07.813.104 I slot print_timing: id 3 | task 5304 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.35 t/s 28.11.088.114 I slot print_timing: id 3 | task 5304 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 28.14.363.433 I slot print_timing: id 3 | task 5304 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 28.17.636.913 I slot print_timing: id 3 | task 5304 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 28.20.911.631 I slot print_timing: id 3 | task 5304 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 28.24.184.106 I slot print_timing: id 3 | task 5304 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 28.27.457.716 I slot print_timing: id 3 | task 5304 | n_decoded = 298, tg = 3.37 t/s, tg_3s = 3.36 t/s 28.30.730.137 I slot print_timing: id 3 | task 5304 | n_decoded = 309, tg = 3.37 t/s, tg_3s = 3.36 t/s 28.34.002.659 I slot print_timing: id 3 | task 5304 | n_decoded = 320, tg = 3.37 t/s, tg_3s = 3.36 t/s 28.37.274.284 I slot print_timing: id 3 | task 5304 | n_decoded = 331, tg = 3.37 t/s, tg_3s = 3.36 t/s 28.40.547.461 I slot print_timing: id 3 | task 5304 | n_decoded = 342, tg = 3.37 t/s, tg_3s = 3.36 t/s 28.43.819.809 I slot print_timing: id 3 | task 5304 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.36 t/s 28.46.796.482 I slot print_timing: id 3 | task 5304 | prompt eval time = 1431.02 ms / 36 tokens ( 39.75 ms per token, 25.16 tokens per second) 28.46.796.486 I slot print_timing: id 3 | task 5304 | eval time = 107643.57 ms / 363 tokens ( 296.54 ms per token, 3.37 tokens per second) 28.46.796.487 I slot print_timing: id 3 | task 5304 | total time = 109074.59 ms / 399 tokens 28.46.796.488 I slot print_timing: id 3 | task 5304 | graphs reused = 5616 28.46.796.523 I slot release: id 3 | task 5304 | stop processing: n_tokens = 468, truncated = 0 28.46.796.532 I srv update_slots: all slots are idle 28.46.799.541 I srv operator(): Chat format: peg-native 28.46.799.648 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.538 (> 0.100 thold), f_keep = 0.150 28.46.799.649 I srv get_availabl: updating prompt cache 28.46.799.699 W srv prompt_save: - saving prompt with length 468, total state size = 160.882 MiB (draft: 0.000 MiB) 28.46.919.374 I srv load: - looking for better prompt, base f_keep = 0.150, sim = 0.538 28.46.919.389 I srv update: - cache state: 11 prompts, 2296.005 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 28.46.919.390 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 28.46.919.391 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 28.46.919.391 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 28.46.919.392 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 28.46.919.393 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 28.46.919.393 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 28.46.919.394 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 28.46.919.395 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 28.46.919.395 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 28.46.919.396 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 28.46.919.398 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 28.46.919.400 I srv get_availabl: prompt cache update took 119.75 ms 28.46.919.452 I slot launch_slot_: id 3 | task 5668 | processing task, is_child = 0 28.46.919.453 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 28.46.919.454 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 28.46.919.471 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 28.46.919.471 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 28.46.919.481 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 28.46.919.481 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 29.18.198.245 I slot print_timing: id 3 | task 5668 | n_decoded = 100, tg = 3.41 t/s, tg_3s = 3.41 t/s 29.21.461.246 I slot print_timing: id 3 | task 5668 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 29.24.726.173 I slot print_timing: id 3 | task 5668 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 29.28.008.392 I slot print_timing: id 3 | task 5668 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.35 t/s 29.31.285.828 I slot print_timing: id 3 | task 5668 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.36 t/s 29.34.563.688 I slot print_timing: id 3 | task 5668 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.36 t/s 29.37.837.337 I slot print_timing: id 3 | task 5668 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s 29.41.110.974 I slot print_timing: id 3 | task 5668 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 29.44.383.476 I slot print_timing: id 3 | task 5668 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 29.47.657.335 I slot print_timing: id 3 | task 5668 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.36 t/s 29.50.929.185 I slot print_timing: id 3 | task 5668 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 29.54.200.088 I slot print_timing: id 3 | task 5668 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 29.57.470.536 I slot print_timing: id 3 | task 5668 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 30.00.743.614 I slot print_timing: id 3 | task 5668 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 30.04.016.836 I slot print_timing: id 3 | task 5668 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 30.07.293.204 I slot print_timing: id 3 | task 5668 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 30.10.567.223 I slot print_timing: id 3 | task 5668 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 30.13.841.959 I slot print_timing: id 3 | task 5668 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 30.17.113.808 I slot print_timing: id 3 | task 5668 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 30.20.386.525 I slot print_timing: id 3 | task 5668 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.36 t/s 30.23.663.044 I slot print_timing: id 3 | task 5668 | n_decoded = 320, tg = 3.37 t/s, tg_3s = 3.36 t/s 30.26.937.385 I slot print_timing: id 3 | task 5668 | n_decoded = 331, tg = 3.37 t/s, tg_3s = 3.36 t/s 30.30.209.162 I slot print_timing: id 3 | task 5668 | n_decoded = 342, tg = 3.37 t/s, tg_3s = 3.36 t/s 30.33.482.334 I slot print_timing: id 3 | task 5668 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.36 t/s 30.36.756.097 I slot print_timing: id 3 | task 5668 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.36 t/s 30.40.024.184 I slot print_timing: id 3 | task 5668 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.37 t/s 30.43.301.490 I slot print_timing: id 3 | task 5668 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.36 t/s 30.46.576.250 I slot print_timing: id 3 | task 5668 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.36 t/s 30.49.847.261 I slot print_timing: id 3 | task 5668 | n_decoded = 408, tg = 3.37 t/s, tg_3s = 3.36 t/s 30.53.117.411 I slot print_timing: id 3 | task 5668 | n_decoded = 419, tg = 3.37 t/s, tg_3s = 3.36 t/s 30.54.605.545 I slot print_timing: id 3 | task 5668 | prompt eval time = 1926.34 ms / 60 tokens ( 32.11 ms per token, 31.15 tokens per second) 30.54.605.548 I slot print_timing: id 3 | task 5668 | eval time = 125759.70 ms / 424 tokens ( 296.60 ms per token, 3.37 tokens per second) 30.54.605.549 I slot print_timing: id 3 | task 5668 | total time = 127686.04 ms / 484 tokens 30.54.605.550 I slot print_timing: id 3 | task 5668 | graphs reused = 6036 30.54.605.589 I slot release: id 3 | task 5668 | stop processing: n_tokens = 553, truncated = 0 30.54.605.599 I srv update_slots: all slots are idle 30.54.608.512 I srv operator(): Chat format: peg-native 30.54.608.622 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.569 (> 0.100 thold), f_keep = 0.127 30.54.608.624 I srv get_availabl: updating prompt cache 30.54.608.690 W srv prompt_save: - saving prompt with length 553, total state size = 190.102 MiB (draft: 0.000 MiB) 30.54.745.422 I srv load: - looking for better prompt, base f_keep = 0.127, sim = 0.569 30.54.745.432 I srv update: - cache state: 12 prompts, 2486.107 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 30.54.745.433 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 30.54.745.436 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 30.54.745.437 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 30.54.745.437 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 30.54.745.438 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 30.54.745.438 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 30.54.745.440 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 30.54.745.441 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 30.54.745.441 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 30.54.745.442 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 30.54.745.442 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 30.54.745.443 I srv update: - prompt 0x60aca9ff8290: 553 tokens, checkpoints: 0, 190.102 MiB 30.54.745.444 I srv get_availabl: prompt cache update took 136.82 ms 30.54.745.474 I slot launch_slot_: id 3 | task 6093 | processing task, is_child = 0 30.54.745.475 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 30.54.745.476 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 30.54.745.487 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 30.54.745.487 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 30.54.745.496 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 30.54.745.496 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 31.25.898.720 I slot print_timing: id 3 | task 6093 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s 31.28.865.913 I slot print_timing: id 3 | task 6093 | prompt eval time = 1771.15 ms / 53 tokens ( 33.42 ms per token, 29.92 tokens per second) 31.28.865.916 I slot print_timing: id 3 | task 6093 | eval time = 32349.24 ms / 110 tokens ( 294.08 ms per token, 3.40 tokens per second) 31.28.865.917 I slot print_timing: id 3 | task 6093 | total time = 34120.39 ms / 163 tokens 31.28.865.918 I slot print_timing: id 3 | task 6093 | graphs reused = 6144 31.28.866.007 I slot release: id 3 | task 6093 | stop processing: n_tokens = 232, truncated = 0 31.28.866.016 I srv update_slots: all slots are idle 31.28.868.882 I srv operator(): Chat format: peg-native 31.28.869.004 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.628 (> 0.100 thold), f_keep = 0.306 31.28.869.005 I srv get_availabl: updating prompt cache 31.28.869.042 W srv prompt_save: - saving prompt with length 232, total state size = 79.755 MiB (draft: 0.000 MiB) 31.28.930.886 I srv load: - looking for better prompt, base f_keep = 0.306, sim = 0.628 31.28.930.900 I srv update: - cache state: 13 prompts, 2565.862 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 31.28.930.901 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 31.28.930.903 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 31.28.930.904 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 31.28.930.904 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 31.28.930.905 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 31.28.930.905 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 31.28.930.906 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 31.28.930.907 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 31.28.930.907 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 31.28.930.908 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 31.28.930.908 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 31.28.930.909 I srv update: - prompt 0x60aca9ff8290: 553 tokens, checkpoints: 0, 190.102 MiB 31.28.930.909 I srv update: - prompt 0x60aca9e15130: 232 tokens, checkpoints: 0, 79.755 MiB 31.28.930.910 I srv get_availabl: prompt cache update took 61.90 ms 31.28.930.963 I slot launch_slot_: id 3 | task 6204 | processing task, is_child = 0 31.28.930.964 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 31.28.930.965 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 31.28.930.978 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 31.28.930.979 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 31.28.930.996 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 31.28.930.997 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 31.59.867.148 I slot print_timing: id 3 | task 6204 | n_decoded = 100, tg = 3.41 t/s, tg_3s = 3.41 t/s 32.03.130.735 I slot print_timing: id 3 | task 6204 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 32.06.391.128 I slot print_timing: id 3 | task 6204 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 32.09.655.635 I slot print_timing: id 3 | task 6204 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 32.12.918.736 I slot print_timing: id 3 | task 6204 | n_decoded = 144, tg = 3.40 t/s, tg_3s = 3.37 t/s 32.16.198.936 I slot print_timing: id 3 | task 6204 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.35 t/s 32.19.469.612 I slot print_timing: id 3 | task 6204 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s 32.22.738.510 I slot print_timing: id 3 | task 6204 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.37 t/s 32.26.006.584 I slot print_timing: id 3 | task 6204 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.37 t/s 32.29.279.417 I slot print_timing: id 3 | task 6204 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.36 t/s 32.32.554.218 I slot print_timing: id 3 | task 6204 | n_decoded = 210, tg = 3.39 t/s, tg_3s = 3.36 t/s 32.35.828.418 I slot print_timing: id 3 | task 6204 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 32.39.104.078 I slot print_timing: id 3 | task 6204 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 32.42.378.228 I slot print_timing: id 3 | task 6204 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 32.45.646.078 I slot print_timing: id 3 | task 6204 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.37 t/s 32.48.915.199 I slot print_timing: id 3 | task 6204 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 32.52.187.710 I slot print_timing: id 3 | task 6204 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 32.55.455.128 I slot print_timing: id 3 | task 6204 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.37 t/s 32.58.725.484 I slot print_timing: id 3 | task 6204 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 33.01.998.606 I slot print_timing: id 3 | task 6204 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.36 t/s 33.04.972.756 I slot print_timing: id 3 | task 6204 | prompt eval time = 1600.07 ms / 42 tokens ( 38.10 ms per token, 26.25 tokens per second) 33.04.972.761 I slot print_timing: id 3 | task 6204 | eval time = 94441.66 ms / 319 tokens ( 296.06 ms per token, 3.38 tokens per second) 33.04.972.761 I slot print_timing: id 3 | task 6204 | total time = 96041.73 ms / 361 tokens 33.04.972.762 I slot print_timing: id 3 | task 6204 | graphs reused = 6460 33.04.972.799 I slot release: id 3 | task 6204 | stop processing: n_tokens = 431, truncated = 0 33.04.972.812 I srv update_slots: all slots are idle 33.04.975.873 I srv operator(): Chat format: peg-native 33.04.975.990 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.588 (> 0.100 thold), f_keep = 0.162 33.04.975.992 I srv get_availabl: updating prompt cache 33.04.976.040 W srv prompt_save: - saving prompt with length 431, total state size = 148.163 MiB (draft: 0.000 MiB) 33.05.085.859 I srv load: - looking for better prompt, base f_keep = 0.162, sim = 0.588 33.05.085.874 I srv update: - cache state: 14 prompts, 2714.025 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 33.05.085.878 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 33.05.085.878 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 33.05.085.879 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 33.05.085.880 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 33.05.085.880 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 33.05.085.883 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 33.05.085.884 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 33.05.085.884 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 33.05.085.885 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 33.05.085.886 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 33.05.085.889 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 33.05.085.889 I srv update: - prompt 0x60aca9ff8290: 553 tokens, checkpoints: 0, 190.102 MiB 33.05.085.890 I srv update: - prompt 0x60aca9e15130: 232 tokens, checkpoints: 0, 79.755 MiB 33.05.085.890 I srv update: - prompt 0x60aca9e75a30: 431 tokens, checkpoints: 0, 148.163 MiB 33.05.085.892 I srv get_availabl: prompt cache update took 109.90 ms 33.05.085.949 I slot launch_slot_: id 3 | task 6524 | processing task, is_child = 0 33.05.085.950 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 33.05.085.951 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 33.05.085.966 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 33.05.085.966 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 33.05.085.977 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 33.05.085.977 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 33.36.198.305 I slot print_timing: id 3 | task 6524 | n_decoded = 100, tg = 3.41 t/s, tg_3s = 3.41 t/s 33.39.458.881 I slot print_timing: id 3 | task 6524 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 33.42.721.224 I slot print_timing: id 3 | task 6524 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 33.45.984.107 I slot print_timing: id 3 | task 6524 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 33.49.262.420 I slot print_timing: id 3 | task 6524 | n_decoded = 144, tg = 3.40 t/s, tg_3s = 3.36 t/s 33.52.531.994 I slot print_timing: id 3 | task 6524 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.36 t/s 33.55.803.811 I slot print_timing: id 3 | task 6524 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s 33.59.076.700 I slot print_timing: id 3 | task 6524 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 34.02.347.637 I slot print_timing: id 3 | task 6524 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 34.05.621.421 I slot print_timing: id 3 | task 6524 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.36 t/s 34.08.894.229 I slot print_timing: id 3 | task 6524 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 34.12.165.036 I slot print_timing: id 3 | task 6524 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 34.15.437.149 I slot print_timing: id 3 | task 6524 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 34.18.708.623 I slot print_timing: id 3 | task 6524 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 34.21.979.719 I slot print_timing: id 3 | task 6524 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 34.25.245.267 I slot print_timing: id 3 | task 6524 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.37 t/s 34.28.513.895 I slot print_timing: id 3 | task 6524 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.37 t/s 34.29.108.676 I slot print_timing: id 3 | task 6524 | prompt eval time = 1769.12 ms / 49 tokens ( 36.10 ms per token, 27.70 tokens per second) 34.29.108.679 I slot print_timing: id 3 | task 6524 | eval time = 82253.55 ms / 278 tokens ( 295.88 ms per token, 3.38 tokens per second) 34.29.108.680 I slot print_timing: id 3 | task 6524 | total time = 84022.67 ms / 327 tokens 34.29.108.681 I slot print_timing: id 3 | task 6524 | graphs reused = 6735 34.29.108.719 I slot release: id 3 | task 6524 | stop processing: n_tokens = 396, truncated = 0 34.29.108.726 I srv update_slots: all slots are idle 34.29.111.804 I srv operator(): Chat format: peg-native 34.29.111.917 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.177 34.29.111.920 I srv get_availabl: updating prompt cache 34.29.111.958 W srv prompt_save: - saving prompt with length 396, total state size = 136.132 MiB (draft: 0.000 MiB) 34.29.213.443 I srv load: - looking for better prompt, base f_keep = 0.177, sim = 0.667 34.29.213.456 I srv update: - cache state: 15 prompts, 2850.157 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 34.29.213.457 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 34.29.213.458 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 34.29.213.459 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 34.29.213.460 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 34.29.213.461 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 34.29.213.461 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 34.29.213.462 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 34.29.213.462 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 34.29.213.463 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 34.29.213.464 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 34.29.213.464 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 34.29.213.465 I srv update: - prompt 0x60aca9ff8290: 553 tokens, checkpoints: 0, 190.102 MiB 34.29.213.465 I srv update: - prompt 0x60aca9e15130: 232 tokens, checkpoints: 0, 79.755 MiB 34.29.213.466 I srv update: - prompt 0x60aca9e75a30: 431 tokens, checkpoints: 0, 148.163 MiB 34.29.213.466 I srv update: - prompt 0x60aca9f65d00: 396 tokens, checkpoints: 0, 136.132 MiB 34.29.213.468 I srv get_availabl: prompt cache update took 101.55 ms 34.29.213.517 I slot launch_slot_: id 3 | task 6803 | processing task, is_child = 0 34.29.213.518 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 34.29.213.519 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 34.29.213.531 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 34.29.213.531 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 34.29.213.540 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 34.29.213.540 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 35.00.014.817 I slot print_timing: id 3 | task 6803 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s 35.03.286.950 I slot print_timing: id 3 | task 6803 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.36 t/s 35.06.552.349 I slot print_timing: id 3 | task 6803 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 35.09.814.665 I slot print_timing: id 3 | task 6803 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 35.13.075.367 I slot print_timing: id 3 | task 6803 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s 35.16.354.652 I slot print_timing: id 3 | task 6803 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.35 t/s 35.19.624.669 I slot print_timing: id 3 | task 6803 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s 35.22.899.204 I slot print_timing: id 3 | task 6803 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 35.26.172.540 I slot print_timing: id 3 | task 6803 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 35.29.451.691 I slot print_timing: id 3 | task 6803 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.35 t/s 35.32.729.050 I slot print_timing: id 3 | task 6803 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 35.36.000.368 I slot print_timing: id 3 | task 6803 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 35.39.273.944 I slot print_timing: id 3 | task 6803 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 35.42.543.682 I slot print_timing: id 3 | task 6803 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 35.45.819.590 I slot print_timing: id 3 | task 6803 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 35.49.094.778 I slot print_timing: id 3 | task 6803 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 35.52.366.799 I slot print_timing: id 3 | task 6803 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 35.55.636.252 I slot print_timing: id 3 | task 6803 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 35.58.906.477 I slot print_timing: id 3 | task 6803 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 36.02.177.979 I slot print_timing: id 3 | task 6803 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.36 t/s 36.05.450.323 I slot print_timing: id 3 | task 6803 | n_decoded = 320, tg = 3.38 t/s, tg_3s = 3.36 t/s 36.08.726.916 I slot print_timing: id 3 | task 6803 | n_decoded = 331, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.12.003.048 I slot print_timing: id 3 | task 6803 | n_decoded = 342, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.15.275.990 I slot print_timing: id 3 | task 6803 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.18.553.933 I slot print_timing: id 3 | task 6803 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.21.831.290 I slot print_timing: id 3 | task 6803 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.25.102.153 I slot print_timing: id 3 | task 6803 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.28.380.119 I slot print_timing: id 3 | task 6803 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.31.655.170 I slot print_timing: id 3 | task 6803 | n_decoded = 408, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.34.939.308 I slot print_timing: id 3 | task 6803 | n_decoded = 419, tg = 3.37 t/s, tg_3s = 3.35 t/s 36.38.207.952 I slot print_timing: id 3 | task 6803 | n_decoded = 430, tg = 3.37 t/s, tg_3s = 3.37 t/s 36.41.485.015 I slot print_timing: id 3 | task 6803 | n_decoded = 441, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.44.758.851 I slot print_timing: id 3 | task 6803 | n_decoded = 452, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.48.031.471 I slot print_timing: id 3 | task 6803 | n_decoded = 463, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.51.301.417 I slot print_timing: id 3 | task 6803 | n_decoded = 474, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.54.571.846 I slot print_timing: id 3 | task 6803 | n_decoded = 485, tg = 3.37 t/s, tg_3s = 3.36 t/s 36.57.841.052 I slot print_timing: id 3 | task 6803 | n_decoded = 496, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.01.110.679 I slot print_timing: id 3 | task 6803 | n_decoded = 507, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.04.382.105 I slot print_timing: id 3 | task 6803 | n_decoded = 518, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.07.651.198 I slot print_timing: id 3 | task 6803 | n_decoded = 529, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.10.924.587 I slot print_timing: id 3 | task 6803 | n_decoded = 540, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.14.195.902 I slot print_timing: id 3 | task 6803 | n_decoded = 551, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.17.463.054 I slot print_timing: id 3 | task 6803 | n_decoded = 562, tg = 3.37 t/s, tg_3s = 3.37 t/s 37.20.734.532 I slot print_timing: id 3 | task 6803 | n_decoded = 573, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.24.007.533 I slot print_timing: id 3 | task 6803 | n_decoded = 584, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.27.284.169 I slot print_timing: id 3 | task 6803 | n_decoded = 595, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.30.556.866 I slot print_timing: id 3 | task 6803 | n_decoded = 606, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.33.831.028 I slot print_timing: id 3 | task 6803 | n_decoded = 617, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.37.110.221 I slot print_timing: id 3 | task 6803 | n_decoded = 628, tg = 3.37 t/s, tg_3s = 3.35 t/s 37.40.389.350 I slot print_timing: id 3 | task 6803 | n_decoded = 639, tg = 3.37 t/s, tg_3s = 3.35 t/s 37.43.667.233 I slot print_timing: id 3 | task 6803 | n_decoded = 650, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.46.942.830 I slot print_timing: id 3 | task 6803 | n_decoded = 661, tg = 3.37 t/s, tg_3s = 3.36 t/s 37.50.240.705 I slot print_timing: id 3 | task 6803 | n_decoded = 672, tg = 3.37 t/s, tg_3s = 3.34 t/s 37.53.533.654 I slot print_timing: id 3 | task 6803 | n_decoded = 683, tg = 3.37 t/s, tg_3s = 3.34 t/s 37.56.826.707 I slot print_timing: id 3 | task 6803 | n_decoded = 694, tg = 3.37 t/s, tg_3s = 3.34 t/s 38.00.120.676 I slot print_timing: id 3 | task 6803 | n_decoded = 705, tg = 3.37 t/s, tg_3s = 3.34 t/s 38.03.115.423 I slot print_timing: id 3 | task 6803 | prompt eval time = 1429.61 ms / 35 tokens ( 40.85 ms per token, 24.48 tokens per second) 38.03.115.426 I slot print_timing: id 3 | task 6803 | eval time = 212472.25 ms / 715 tokens ( 297.16 ms per token, 3.37 tokens per second) 38.03.115.427 I slot print_timing: id 3 | task 6803 | total time = 213901.86 ms / 750 tokens 38.03.115.428 I slot print_timing: id 3 | task 6803 | graphs reused = 7445 38.03.115.467 I slot release: id 3 | task 6803 | stop processing: n_tokens = 819, truncated = 0 38.03.115.477 I srv update_slots: all slots are idle 38.03.118.426 I srv operator(): Chat format: peg-native 38.03.118.532 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.085 38.03.118.534 I srv get_availabl: updating prompt cache 38.03.118.582 W srv prompt_save: - saving prompt with length 819, total state size = 281.543 MiB (draft: 0.000 MiB) 38.03.315.347 I srv load: - looking for better prompt, base f_keep = 0.085, sim = 0.667 38.03.315.358 I srv update: - cache state: 16 prompts, 3131.699 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 38.03.315.359 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 38.03.315.360 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 38.03.315.361 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 38.03.315.362 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 38.03.315.362 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 38.03.315.364 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 38.03.315.365 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 38.03.315.365 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 38.03.315.366 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 38.03.315.366 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 38.03.315.367 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 38.03.315.367 I srv update: - prompt 0x60aca9ff8290: 553 tokens, checkpoints: 0, 190.102 MiB 38.03.315.368 I srv update: - prompt 0x60aca9e15130: 232 tokens, checkpoints: 0, 79.755 MiB 38.03.315.368 I srv update: - prompt 0x60aca9e75a30: 431 tokens, checkpoints: 0, 148.163 MiB 38.03.315.369 I srv update: - prompt 0x60aca9f65d00: 396 tokens, checkpoints: 0, 136.132 MiB 38.03.315.369 I srv update: - prompt 0x60aca9abc850: 819 tokens, checkpoints: 0, 281.543 MiB 38.03.315.371 I srv get_availabl: prompt cache update took 196.84 ms 38.03.315.401 I slot launch_slot_: id 3 | task 7519 | processing task, is_child = 0 38.03.315.402 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 38.03.315.403 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 38.03.315.419 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 38.03.315.419 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 38.03.315.443 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 38.03.315.443 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 38.34.125.228 I slot print_timing: id 3 | task 7519 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s 38.37.389.668 I slot print_timing: id 3 | task 7519 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 38.40.650.974 I slot print_timing: id 3 | task 7519 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 38.43.910.867 I slot print_timing: id 3 | task 7519 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 38.47.174.051 I slot print_timing: id 3 | task 7519 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s 38.50.452.941 I slot print_timing: id 3 | task 7519 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.35 t/s 38.53.729.556 I slot print_timing: id 3 | task 7519 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s 38.57.005.353 I slot print_timing: id 3 | task 7519 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 39.00.279.283 I slot print_timing: id 3 | task 7519 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 39.03.554.799 I slot print_timing: id 3 | task 7519 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.36 t/s 39.06.832.214 I slot print_timing: id 3 | task 7519 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 39.10.103.609 I slot print_timing: id 3 | task 7519 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 39.13.380.770 I slot print_timing: id 3 | task 7519 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 39.16.652.391 I slot print_timing: id 3 | task 7519 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 39.19.928.603 I slot print_timing: id 3 | task 7519 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 39.23.205.319 I slot print_timing: id 3 | task 7519 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 39.26.477.629 I slot print_timing: id 3 | task 7519 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 39.29.748.122 I slot print_timing: id 3 | task 7519 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 39.33.020.927 I slot print_timing: id 3 | task 7519 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 39.36.293.560 I slot print_timing: id 3 | task 7519 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.36 t/s 39.39.559.978 I slot print_timing: id 3 | task 7519 | n_decoded = 320, tg = 3.38 t/s, tg_3s = 3.37 t/s 39.40.152.104 I slot print_timing: id 3 | task 7519 | prompt eval time = 1430.24 ms / 35 tokens ( 40.86 ms per token, 24.47 tokens per second) 39.40.152.108 I slot print_timing: id 3 | task 7519 | eval time = 95406.39 ms / 322 tokens ( 296.29 ms per token, 3.38 tokens per second) 39.40.152.108 I slot print_timing: id 3 | task 7519 | total time = 96836.62 ms / 357 tokens 39.40.152.109 I slot print_timing: id 3 | task 7519 | graphs reused = 7764 39.40.152.146 I slot release: id 3 | task 7519 | stop processing: n_tokens = 426, truncated = 0 39.40.152.154 I srv update_slots: all slots are idle 39.40.155.181 I srv operator(): Chat format: peg-native 39.40.155.337 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.722 (> 0.100 thold), f_keep = 0.164 39.40.155.339 I srv get_availabl: updating prompt cache 39.40.155.384 W srv prompt_save: - saving prompt with length 426, total state size = 146.444 MiB (draft: 0.000 MiB) 39.40.264.473 I srv load: - looking for better prompt, base f_keep = 0.164, sim = 0.722 39.40.264.488 I srv update: - cache state: 17 prompts, 3278.144 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 39.40.264.489 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 39.40.264.489 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 39.40.264.490 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 39.40.264.491 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 39.40.264.491 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 39.40.264.492 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 39.40.264.492 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 39.40.264.493 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 39.40.264.493 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 39.40.264.494 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 39.40.264.494 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 39.40.264.495 I srv update: - prompt 0x60aca9ff8290: 553 tokens, checkpoints: 0, 190.102 MiB 39.40.264.495 I srv update: - prompt 0x60aca9e15130: 232 tokens, checkpoints: 0, 79.755 MiB 39.40.264.496 I srv update: - prompt 0x60aca9e75a30: 431 tokens, checkpoints: 0, 148.163 MiB 39.40.264.496 I srv update: - prompt 0x60aca9f65d00: 396 tokens, checkpoints: 0, 136.132 MiB 39.40.264.497 I srv update: - prompt 0x60aca9abc850: 819 tokens, checkpoints: 0, 281.543 MiB 39.40.264.497 I srv update: - prompt 0x60aca9db18f0: 426 tokens, checkpoints: 0, 146.444 MiB 39.40.264.505 I srv get_availabl: prompt cache update took 109.17 ms 39.40.264.547 I slot launch_slot_: id 3 | task 7842 | processing task, is_child = 0 39.40.264.548 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 39.40.264.549 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 39.40.264.562 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 39.40.264.562 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 39.40.264.571 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 39.40.264.571 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 40.10.921.054 I slot print_timing: id 3 | task 7842 | n_decoded = 100, tg = 3.41 t/s, tg_3s = 3.41 t/s 40.14.185.275 I slot print_timing: id 3 | task 7842 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 40.17.453.336 I slot print_timing: id 3 | task 7842 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 40.20.715.586 I slot print_timing: id 3 | task 7842 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 40.23.975.801 I slot print_timing: id 3 | task 7842 | n_decoded = 144, tg = 3.40 t/s, tg_3s = 3.37 t/s 40.27.238.815 I slot print_timing: id 3 | task 7842 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.37 t/s 40.30.521.310 I slot print_timing: id 3 | task 7842 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.35 t/s 40.33.799.545 I slot print_timing: id 3 | task 7842 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 40.37.074.537 I slot print_timing: id 3 | task 7842 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 40.40.348.724 I slot print_timing: id 3 | task 7842 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.36 t/s 40.43.619.206 I slot print_timing: id 3 | task 7842 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 40.46.887.465 I slot print_timing: id 3 | task 7842 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.37 t/s 40.50.158.571 I slot print_timing: id 3 | task 7842 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 40.53.427.186 I slot print_timing: id 3 | task 7842 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.37 t/s 40.56.700.623 I slot print_timing: id 3 | task 7842 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 40.59.970.690 I slot print_timing: id 3 | task 7842 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 41.03.241.552 I slot print_timing: id 3 | task 7842 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 41.06.510.473 I slot print_timing: id 3 | task 7842 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.37 t/s 41.09.783.775 I slot print_timing: id 3 | task 7842 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 41.13.058.642 I slot print_timing: id 3 | task 7842 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.36 t/s 41.14.545.898 I slot print_timing: id 3 | task 7842 | prompt eval time = 1307.79 ms / 27 tokens ( 48.44 ms per token, 20.65 tokens per second) 41.14.545.901 I slot print_timing: id 3 | task 7842 | eval time = 92973.51 ms / 314 tokens ( 296.09 ms per token, 3.38 tokens per second) 41.14.545.902 I slot print_timing: id 3 | task 7842 | total time = 94281.30 ms / 341 tokens 41.14.545.903 I slot print_timing: id 3 | task 7842 | graphs reused = 8075 41.14.545.947 I slot release: id 3 | task 7842 | stop processing: n_tokens = 410, truncated = 0 41.14.545.959 I srv update_slots: all slots are idle 41.14.548.868 I srv operator(): Chat format: peg-native 41.14.548.971 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.814 (> 0.100 thold), f_keep = 0.171 41.14.548.973 I srv get_availabl: updating prompt cache 41.14.549.020 W srv prompt_save: - saving prompt with length 410, total state size = 140.944 MiB (draft: 0.000 MiB) 41.14.653.929 I srv load: - looking for better prompt, base f_keep = 0.171, sim = 0.814 41.14.653.946 I srv update: - cache state: 18 prompts, 3419.088 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 41.14.653.947 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 41.14.653.948 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 41.14.653.948 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 41.14.653.949 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 41.14.653.949 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 41.14.653.950 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 41.14.653.950 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 41.14.653.951 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 41.14.653.951 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 41.14.653.952 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 41.14.653.952 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 41.14.653.953 I srv update: - prompt 0x60aca9ff8290: 553 tokens, checkpoints: 0, 190.102 MiB 41.14.653.953 I srv update: - prompt 0x60aca9e15130: 232 tokens, checkpoints: 0, 79.755 MiB 41.14.653.954 I srv update: - prompt 0x60aca9e75a30: 431 tokens, checkpoints: 0, 148.163 MiB 41.14.653.954 I srv update: - prompt 0x60aca9f65d00: 396 tokens, checkpoints: 0, 136.132 MiB 41.14.653.955 I srv update: - prompt 0x60aca9abc850: 819 tokens, checkpoints: 0, 281.543 MiB 41.14.653.955 I srv update: - prompt 0x60aca9db18f0: 426 tokens, checkpoints: 0, 146.444 MiB 41.14.653.956 I srv update: - prompt 0x60aca98dcad0: 410 tokens, checkpoints: 0, 140.944 MiB 41.14.653.957 I srv get_availabl: prompt cache update took 104.98 ms 41.14.654.007 I slot launch_slot_: id 3 | task 8157 | processing task, is_child = 0 41.14.654.008 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 41.14.654.009 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 41.14.654.021 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 41.14.654.022 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 41.14.654.030 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 41.14.654.030 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 41.44.916.319 I slot print_timing: id 3 | task 8157 | n_decoded = 100, tg = 3.41 t/s, tg_3s = 3.41 t/s 41.48.173.712 I slot print_timing: id 3 | task 8157 | n_decoded = 111, tg = 3.41 t/s, tg_3s = 3.38 t/s 41.51.432.216 I slot print_timing: id 3 | task 8157 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.38 t/s 41.54.687.282 I slot print_timing: id 3 | task 8157 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.38 t/s 41.57.944.730 I slot print_timing: id 3 | task 8157 | n_decoded = 144, tg = 3.40 t/s, tg_3s = 3.38 t/s 42.01.202.927 I slot print_timing: id 3 | task 8157 | n_decoded = 155, tg = 3.40 t/s, tg_3s = 3.38 t/s 42.04.459.562 I slot print_timing: id 3 | task 8157 | n_decoded = 166, tg = 3.40 t/s, tg_3s = 3.38 t/s 42.07.735.111 I slot print_timing: id 3 | task 8157 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 42.11.005.798 I slot print_timing: id 3 | task 8157 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 42.14.278.556 I slot print_timing: id 3 | task 8157 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.36 t/s 42.17.547.840 I slot print_timing: id 3 | task 8157 | n_decoded = 210, tg = 3.39 t/s, tg_3s = 3.36 t/s 42.20.822.500 I slot print_timing: id 3 | task 8157 | n_decoded = 221, tg = 3.39 t/s, tg_3s = 3.36 t/s 42.22.012.824 I slot print_timing: id 3 | task 8157 | prompt eval time = 938.55 ms / 16 tokens ( 58.66 ms per token, 17.05 tokens per second) 42.22.012.828 I slot print_timing: id 3 | task 8157 | eval time = 66420.22 ms / 225 tokens ( 295.20 ms per token, 3.39 tokens per second) 42.22.012.828 I slot print_timing: id 3 | task 8157 | total time = 67358.77 ms / 241 tokens 42.22.012.829 I slot print_timing: id 3 | task 8157 | graphs reused = 8297 42.22.012.865 I slot release: id 3 | task 8157 | stop processing: n_tokens = 310, truncated = 0 42.22.012.872 I srv update_slots: all slots are idle 42.22.015.781 I srv operator(): Chat format: peg-native 42.22.015.882 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.648 (> 0.100 thold), f_keep = 0.226 42.22.015.883 I srv get_availabl: updating prompt cache 42.22.015.928 W srv prompt_save: - saving prompt with length 310, total state size = 106.568 MiB (draft: 0.000 MiB) 42.22.097.711 I srv load: - looking for better prompt, base f_keep = 0.226, sim = 0.648 42.22.097.725 I srv update: - cache state: 19 prompts, 3525.656 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 42.22.097.726 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 42.22.097.727 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 42.22.097.727 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 42.22.097.728 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 42.22.097.728 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 42.22.097.729 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 42.22.097.729 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 42.22.097.730 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 42.22.097.730 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 42.22.097.731 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 42.22.097.731 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 42.22.097.732 I srv update: - prompt 0x60aca9ff8290: 553 tokens, checkpoints: 0, 190.102 MiB 42.22.097.732 I srv update: - prompt 0x60aca9e15130: 232 tokens, checkpoints: 0, 79.755 MiB 42.22.097.733 I srv update: - prompt 0x60aca9e75a30: 431 tokens, checkpoints: 0, 148.163 MiB 42.22.097.733 I srv update: - prompt 0x60aca9f65d00: 396 tokens, checkpoints: 0, 136.132 MiB 42.22.097.734 I srv update: - prompt 0x60aca9abc850: 819 tokens, checkpoints: 0, 281.543 MiB 42.22.097.735 I srv update: - prompt 0x60aca9db18f0: 426 tokens, checkpoints: 0, 146.444 MiB 42.22.097.735 I srv update: - prompt 0x60aca98dcad0: 410 tokens, checkpoints: 0, 140.944 MiB 42.22.097.736 I srv update: - prompt 0x60aca9f18250: 310 tokens, checkpoints: 0, 106.568 MiB 42.22.097.738 I srv get_availabl: prompt cache update took 81.85 ms 42.22.097.785 I slot launch_slot_: id 3 | task 8383 | processing task, is_child = 0 42.22.097.785 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 42.22.097.786 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 42.22.097.799 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 42.22.097.799 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 42.22.097.808 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 42.22.097.808 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 42.52.897.052 I slot print_timing: id 3 | task 8383 | n_decoded = 100, tg = 3.41 t/s, tg_3s = 3.41 t/s 42.56.160.760 I slot print_timing: id 3 | task 8383 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 42.59.421.849 I slot print_timing: id 3 | task 8383 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 43.02.686.056 I slot print_timing: id 3 | task 8383 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 43.05.949.114 I slot print_timing: id 3 | task 8383 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s 43.09.229.148 I slot print_timing: id 3 | task 8383 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.35 t/s 43.12.495.338 I slot print_timing: id 3 | task 8383 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.37 t/s 43.15.764.105 I slot print_timing: id 3 | task 8383 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.37 t/s 43.19.032.408 I slot print_timing: id 3 | task 8383 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.37 t/s 43.22.296.110 I slot print_timing: id 3 | task 8383 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.37 t/s 43.25.568.359 I slot print_timing: id 3 | task 8383 | n_decoded = 210, tg = 3.39 t/s, tg_3s = 3.36 t/s 43.28.841.263 I slot print_timing: id 3 | task 8383 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 43.32.112.188 I slot print_timing: id 3 | task 8383 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 43.35.382.252 I slot print_timing: id 3 | task 8383 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 43.38.654.551 I slot print_timing: id 3 | task 8383 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 43.41.926.653 I slot print_timing: id 3 | task 8383 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 43.45.204.542 I slot print_timing: id 3 | task 8383 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 43.48.478.076 I slot print_timing: id 3 | task 8383 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 43.51.751.385 I slot print_timing: id 3 | task 8383 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 43.55.025.313 I slot print_timing: id 3 | task 8383 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.36 t/s 43.58.300.362 I slot print_timing: id 3 | task 8383 | n_decoded = 320, tg = 3.38 t/s, tg_3s = 3.36 t/s 44.01.570.683 I slot print_timing: id 3 | task 8383 | n_decoded = 331, tg = 3.38 t/s, tg_3s = 3.36 t/s 44.02.463.635 I slot print_timing: id 3 | task 8383 | prompt eval time = 1432.26 ms / 38 tokens ( 37.69 ms per token, 26.53 tokens per second) 44.02.463.639 I slot print_timing: id 3 | task 8383 | eval time = 98933.55 ms / 334 tokens ( 296.21 ms per token, 3.38 tokens per second) 44.02.463.639 I slot print_timing: id 3 | task 8383 | total time = 100365.80 ms / 372 tokens 44.02.463.640 I slot print_timing: id 3 | task 8383 | graphs reused = 8628 44.02.463.677 I slot release: id 3 | task 8383 | stop processing: n_tokens = 441, truncated = 0 44.02.463.687 I srv update_slots: all slots are idle 44.02.466.560 I srv operator(): Chat format: peg-native 44.02.466.668 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.729 (> 0.100 thold), f_keep = 0.159 44.02.466.670 I srv get_availabl: updating prompt cache 44.02.466.707 W srv prompt_save: - saving prompt with length 441, total state size = 151.601 MiB (draft: 0.000 MiB) 44.02.579.220 I srv load: - looking for better prompt, base f_keep = 0.159, sim = 0.729 44.02.579.237 I srv update: - cache state: 20 prompts, 3677.257 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 44.02.579.238 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 44.02.579.239 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 44.02.579.239 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 44.02.579.240 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 44.02.579.241 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 44.02.579.241 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 44.02.579.242 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 44.02.579.242 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 44.02.579.243 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 44.02.579.243 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 44.02.579.244 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 44.02.579.244 I srv update: - prompt 0x60aca9ff8290: 553 tokens, checkpoints: 0, 190.102 MiB 44.02.579.245 I srv update: - prompt 0x60aca9e15130: 232 tokens, checkpoints: 0, 79.755 MiB 44.02.579.245 I srv update: - prompt 0x60aca9e75a30: 431 tokens, checkpoints: 0, 148.163 MiB 44.02.579.246 I srv update: - prompt 0x60aca9f65d00: 396 tokens, checkpoints: 0, 136.132 MiB 44.02.579.246 I srv update: - prompt 0x60aca9abc850: 819 tokens, checkpoints: 0, 281.543 MiB 44.02.579.247 I srv update: - prompt 0x60aca9db18f0: 426 tokens, checkpoints: 0, 146.444 MiB 44.02.579.247 I srv update: - prompt 0x60aca98dcad0: 410 tokens, checkpoints: 0, 140.944 MiB 44.02.579.248 I srv update: - prompt 0x60aca9f18250: 310 tokens, checkpoints: 0, 106.568 MiB 44.02.579.248 I srv update: - prompt 0x60aca9e9b310: 441 tokens, checkpoints: 0, 151.601 MiB 44.02.579.249 I srv get_availabl: prompt cache update took 112.58 ms 44.02.579.293 I slot launch_slot_: id 3 | task 8718 | processing task, is_child = 0 44.02.579.294 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 44.02.579.295 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 44.02.579.308 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 44.02.579.308 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 44.02.579.317 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 44.02.579.317 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 44.33.255.503 I slot print_timing: id 3 | task 8718 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s 44.36.515.060 I slot print_timing: id 3 | task 8718 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 44.39.773.116 I slot print_timing: id 3 | task 8718 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.38 t/s 44.43.036.268 I slot print_timing: id 3 | task 8718 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 44.46.298.730 I slot print_timing: id 3 | task 8718 | n_decoded = 144, tg = 3.40 t/s, tg_3s = 3.37 t/s 44.49.560.714 I slot print_timing: id 3 | task 8718 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.37 t/s 44.52.838.636 I slot print_timing: id 3 | task 8718 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s 44.56.115.334 I slot print_timing: id 3 | task 8718 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 44.59.385.199 I slot print_timing: id 3 | task 8718 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 45.02.655.400 I slot print_timing: id 3 | task 8718 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.36 t/s 45.05.923.458 I slot print_timing: id 3 | task 8718 | n_decoded = 210, tg = 3.39 t/s, tg_3s = 3.37 t/s 45.09.194.683 I slot print_timing: id 3 | task 8718 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 45.12.468.224 I slot print_timing: id 3 | task 8718 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 45.15.738.939 I slot print_timing: id 3 | task 8718 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 45.19.009.570 I slot print_timing: id 3 | task 8718 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 45.22.280.941 I slot print_timing: id 3 | task 8718 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 45.25.554.942 I slot print_timing: id 3 | task 8718 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 45.28.833.586 I slot print_timing: id 3 | task 8718 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 45.32.111.409 I slot print_timing: id 3 | task 8718 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 45.35.382.410 I slot print_timing: id 3 | task 8718 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.36 t/s 45.38.649.085 I slot print_timing: id 3 | task 8718 | n_decoded = 320, tg = 3.38 t/s, tg_3s = 3.37 t/s 45.41.925.161 I slot print_timing: id 3 | task 8718 | n_decoded = 331, tg = 3.38 t/s, tg_3s = 3.36 t/s 45.45.201.488 I slot print_timing: id 3 | task 8718 | n_decoded = 342, tg = 3.38 t/s, tg_3s = 3.36 t/s 45.48.476.566 I slot print_timing: id 3 | task 8718 | n_decoded = 353, tg = 3.38 t/s, tg_3s = 3.36 t/s 45.51.751.168 I slot print_timing: id 3 | task 8718 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.36 t/s 45.55.023.684 I slot print_timing: id 3 | task 8718 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.36 t/s 45.58.296.935 I slot print_timing: id 3 | task 8718 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.36 t/s 46.01.569.878 I slot print_timing: id 3 | task 8718 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.36 t/s 46.04.839.978 I slot print_timing: id 3 | task 8718 | n_decoded = 408, tg = 3.37 t/s, tg_3s = 3.36 t/s 46.08.121.916 I slot print_timing: id 3 | task 8718 | n_decoded = 419, tg = 3.37 t/s, tg_3s = 3.35 t/s 46.09.014.970 I slot print_timing: id 3 | task 8718 | prompt eval time = 1307.11 ms / 26 tokens ( 50.27 ms per token, 19.89 tokens per second) 46.09.014.974 I slot print_timing: id 3 | task 8718 | eval time = 125128.52 ms / 422 tokens ( 296.51 ms per token, 3.37 tokens per second) 46.09.014.975 I slot print_timing: id 3 | task 8718 | total time = 126435.63 ms / 448 tokens 46.09.014.976 I slot print_timing: id 3 | task 8718 | graphs reused = 9046 46.09.015.021 I slot release: id 3 | task 8718 | stop processing: n_tokens = 517, truncated = 0 46.09.015.031 I srv update_slots: all slots are idle 46.09.017.934 I srv operator(): Chat format: peg-native 46.09.018.048 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.673 (> 0.100 thold), f_keep = 0.135 46.09.018.049 I srv get_availabl: updating prompt cache 46.09.018.088 W srv prompt_save: - saving prompt with length 517, total state size = 177.727 MiB (draft: 0.000 MiB) 46.09.147.126 I srv load: - looking for better prompt, base f_keep = 0.135, sim = 0.673 46.09.147.139 I srv update: - cache state: 21 prompts, 3854.984 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 46.09.147.140 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 46.09.147.141 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 46.09.147.142 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 46.09.147.142 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 46.09.147.143 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 46.09.147.143 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 46.09.147.144 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 46.09.147.144 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 46.09.147.145 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 46.09.147.145 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 46.09.147.146 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 46.09.147.146 I srv update: - prompt 0x60aca9ff8290: 553 tokens, checkpoints: 0, 190.102 MiB 46.09.147.147 I srv update: - prompt 0x60aca9e15130: 232 tokens, checkpoints: 0, 79.755 MiB 46.09.147.147 I srv update: - prompt 0x60aca9e75a30: 431 tokens, checkpoints: 0, 148.163 MiB 46.09.147.148 I srv update: - prompt 0x60aca9f65d00: 396 tokens, checkpoints: 0, 136.132 MiB 46.09.147.148 I srv update: - prompt 0x60aca9abc850: 819 tokens, checkpoints: 0, 281.543 MiB 46.09.147.149 I srv update: - prompt 0x60aca9db18f0: 426 tokens, checkpoints: 0, 146.444 MiB 46.09.147.149 I srv update: - prompt 0x60aca98dcad0: 410 tokens, checkpoints: 0, 140.944 MiB 46.09.147.150 I srv update: - prompt 0x60aca9f18250: 310 tokens, checkpoints: 0, 106.568 MiB 46.09.147.151 I srv update: - prompt 0x60aca9e9b310: 441 tokens, checkpoints: 0, 151.601 MiB 46.09.147.151 I srv update: - prompt 0x60aca9c57c00: 517 tokens, checkpoints: 0, 177.727 MiB 46.09.147.152 I srv get_availabl: prompt cache update took 129.10 ms 46.09.147.179 I slot launch_slot_: id 3 | task 9141 | processing task, is_child = 0 46.09.147.179 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 46.09.147.180 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 46.09.147.192 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 46.09.147.192 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 46.09.147.200 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 46.09.147.201 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 46.39.944.352 I slot print_timing: id 3 | task 9141 | n_decoded = 100, tg = 3.41 t/s, tg_3s = 3.41 t/s 46.43.204.630 I slot print_timing: id 3 | task 9141 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 46.46.467.555 I slot print_timing: id 3 | task 9141 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s 46.49.727.434 I slot print_timing: id 3 | task 9141 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s 46.52.991.908 I slot print_timing: id 3 | task 9141 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s 46.56.262.420 I slot print_timing: id 3 | task 9141 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.36 t/s 46.59.532.178 I slot print_timing: id 3 | task 9141 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s 47.02.807.381 I slot print_timing: id 3 | task 9141 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.36 t/s 47.06.081.077 I slot print_timing: id 3 | task 9141 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 47.09.354.283 I slot print_timing: id 3 | task 9141 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.36 t/s 47.12.631.283 I slot print_timing: id 3 | task 9141 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s 47.15.904.101 I slot print_timing: id 3 | task 9141 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s 47.19.177.434 I slot print_timing: id 3 | task 9141 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 47.22.450.760 I slot print_timing: id 3 | task 9141 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 47.25.723.155 I slot print_timing: id 3 | task 9141 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 47.28.997.633 I slot print_timing: id 3 | task 9141 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 47.32.271.140 I slot print_timing: id 3 | task 9141 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 47.35.541.533 I slot print_timing: id 3 | task 9141 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 47.38.811.338 I slot print_timing: id 3 | task 9141 | prompt eval time = 1429.34 ms / 34 tokens ( 42.04 ms per token, 23.79 tokens per second) 47.38.811.342 I slot print_timing: id 3 | task 9141 | eval time = 88234.77 ms / 298 tokens ( 296.09 ms per token, 3.38 tokens per second) 47.38.811.342 I slot print_timing: id 3 | task 9141 | total time = 89664.11 ms / 332 tokens 47.38.811.344 I slot print_timing: id 3 | task 9141 | graphs reused = 9341 47.38.811.386 I slot release: id 3 | task 9141 | stop processing: n_tokens = 401, truncated = 0 47.38.811.394 I srv update_slots: all slots are idle 47.38.814.389 I srv operator(): Chat format: peg-native 47.38.814.498 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.636 (> 0.100 thold), f_keep = 0.175 47.38.814.500 I srv get_availabl: updating prompt cache 47.38.814.545 W srv prompt_save: - saving prompt with length 401, total state size = 137.850 MiB (draft: 0.000 MiB) 47.38.916.830 I srv load: - looking for better prompt, base f_keep = 0.175, sim = 0.636 47.38.916.847 I srv update: - cache state: 22 prompts, 3992.834 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 47.38.916.848 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 47.38.916.853 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 47.38.916.853 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 47.38.916.854 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 47.38.916.854 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 47.38.916.855 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 47.38.916.856 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 47.38.916.858 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 47.38.916.859 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 47.38.916.859 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 47.38.916.860 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 47.38.916.860 I srv update: - prompt 0x60aca9ff8290: 553 tokens, checkpoints: 0, 190.102 MiB 47.38.916.862 I srv update: - prompt 0x60aca9e15130: 232 tokens, checkpoints: 0, 79.755 MiB 47.38.916.862 I srv update: - prompt 0x60aca9e75a30: 431 tokens, checkpoints: 0, 148.163 MiB 47.38.916.862 I srv update: - prompt 0x60aca9f65d00: 396 tokens, checkpoints: 0, 136.132 MiB 47.38.916.863 I srv update: - prompt 0x60aca9abc850: 819 tokens, checkpoints: 0, 281.543 MiB 47.38.916.864 I srv update: - prompt 0x60aca9db18f0: 426 tokens, checkpoints: 0, 146.444 MiB 47.38.916.864 I srv update: - prompt 0x60aca98dcad0: 410 tokens, checkpoints: 0, 140.944 MiB 47.38.916.865 I srv update: - prompt 0x60aca9f18250: 310 tokens, checkpoints: 0, 106.568 MiB 47.38.916.865 I srv update: - prompt 0x60aca9e9b310: 441 tokens, checkpoints: 0, 151.601 MiB 47.38.916.866 I srv update: - prompt 0x60aca9c57c00: 517 tokens, checkpoints: 0, 177.727 MiB 47.38.916.866 I srv update: - prompt 0x60aca9d96d00: 401 tokens, checkpoints: 0, 137.850 MiB 47.38.916.867 I srv get_availabl: prompt cache update took 102.37 ms 47.38.916.911 I slot launch_slot_: id 3 | task 9440 | processing task, is_child = 0 47.38.916.911 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 47.38.916.912 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 47.38.916.924 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 47.38.916.924 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 47.38.916.933 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 47.38.916.933 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 48.09.698.710 I slot print_timing: id 3 | task 9440 | n_decoded = 100, tg = 3.41 t/s, tg_3s = 3.41 t/s 48.12.959.900 I slot print_timing: id 3 | task 9440 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s 48.16.215.956 I slot print_timing: id 3 | task 9440 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.38 t/s 48.19.471.805 I slot print_timing: id 3 | task 9440 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.38 t/s 48.22.727.120 I slot print_timing: id 3 | task 9440 | n_decoded = 144, tg = 3.40 t/s, tg_3s = 3.38 t/s 48.26.003.197 I slot print_timing: id 3 | task 9440 | n_decoded = 155, tg = 3.40 t/s, tg_3s = 3.36 t/s 48.29.270.930 I slot print_timing: id 3 | task 9440 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.37 t/s 48.32.538.709 I slot print_timing: id 3 | task 9440 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.37 t/s 48.35.801.791 I slot print_timing: id 3 | task 9440 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.37 t/s 48.39.065.810 I slot print_timing: id 3 | task 9440 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.37 t/s 48.40.255.784 I slot print_timing: id 3 | task 9440 | prompt eval time = 1433.46 ms / 40 tokens ( 35.84 ms per token, 27.90 tokens per second) 48.40.255.788 I slot print_timing: id 3 | task 9440 | eval time = 59905.37 ms / 203 tokens ( 295.10 ms per token, 3.39 tokens per second) 48.40.255.789 I slot print_timing: id 3 | task 9440 | total time = 61338.83 ms / 243 tokens 48.40.255.789 I slot print_timing: id 3 | task 9440 | graphs reused = 9541 48.40.255.826 I slot release: id 3 | task 9440 | stop processing: n_tokens = 312, truncated = 0 48.40.255.833 I srv update_slots: all slots are idle 48.40.258.685 I srv operator(): Chat format: peg-native 48.40.258.791 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.737 (> 0.100 thold), f_keep = 0.224 48.40.258.793 I srv get_availabl: updating prompt cache 48.40.258.826 W srv prompt_save: - saving prompt with length 312, total state size = 107.256 MiB (draft: 0.000 MiB) 48.40.339.808 I srv load: - looking for better prompt, base f_keep = 0.224, sim = 0.737 48.40.339.827 I srv update: - cache state: 23 prompts, 4100.090 MiB (limits: 8192.000 MiB, 16384 tokens, 23830 est) 48.40.339.828 I srv update: - prompt 0x60aca9e31f00: 1046 tokens, checkpoints: 0, 359.577 MiB 48.40.339.828 I srv update: - prompt 0x60aca9e54f90: 387 tokens, checkpoints: 0, 133.038 MiB 48.40.339.829 I srv update: - prompt 0x60aca9e4a0b0: 246 tokens, checkpoints: 0, 84.567 MiB 48.40.339.829 I srv update: - prompt 0x60aca9ddaf30: 688 tokens, checkpoints: 0, 236.510 MiB 48.40.339.830 I srv update: - prompt 0x60aca9df3e60: 1239 tokens, checkpoints: 0, 425.922 MiB 48.40.339.831 I srv update: - prompt 0x60aca9e8b3b0: 678 tokens, checkpoints: 0, 233.072 MiB 48.40.339.831 I srv update: - prompt 0x60acaae6a010: 375 tokens, checkpoints: 0, 128.913 MiB 48.40.339.832 I srv update: - prompt 0x60aca9b9b040: 547 tokens, checkpoints: 0, 188.040 MiB 48.40.339.832 I srv update: - prompt 0x60aca9c6ae20: 481 tokens, checkpoints: 0, 165.351 MiB 48.40.339.833 I srv update: - prompt 0x60aca9d9fab0: 524 tokens, checkpoints: 0, 180.133 MiB 48.40.339.833 I srv update: - prompt 0x60aca9b8c7c0: 468 tokens, checkpoints: 0, 160.882 MiB 48.40.339.834 I srv update: - prompt 0x60aca9ff8290: 553 tokens, checkpoints: 0, 190.102 MiB 48.40.339.834 I srv update: - prompt 0x60aca9e15130: 232 tokens, checkpoints: 0, 79.755 MiB 48.40.339.835 I srv update: - prompt 0x60aca9e75a30: 431 tokens, checkpoints: 0, 148.163 MiB 48.40.339.835 I srv update: - prompt 0x60aca9f65d00: 396 tokens, checkpoints: 0, 136.132 MiB 48.40.339.836 I srv update: - prompt 0x60aca9abc850: 819 tokens, checkpoints: 0, 281.543 MiB 48.40.339.836 I srv update: - prompt 0x60aca9db18f0: 426 tokens, checkpoints: 0, 146.444 MiB 48.40.339.837 I srv update: - prompt 0x60aca98dcad0: 410 tokens, checkpoints: 0, 140.944 MiB 48.40.339.837 I srv update: - prompt 0x60aca9f18250: 310 tokens, checkpoints: 0, 106.568 MiB 48.40.339.838 I srv update: - prompt 0x60aca9e9b310: 441 tokens, checkpoints: 0, 151.601 MiB 48.40.339.838 I srv update: - prompt 0x60aca9c57c00: 517 tokens, checkpoints: 0, 177.727 MiB 48.40.339.839 I srv update: - prompt 0x60aca9d96d00: 401 tokens, checkpoints: 0, 137.850 MiB 48.40.339.839 I srv update: - prompt 0x60aca98c2e20: 312 tokens, checkpoints: 0, 107.256 MiB 48.40.339.841 I srv get_availabl: prompt cache update took 81.05 ms 48.40.339.884 I slot launch_slot_: id 3 | task 9644 | processing task, is_child = 0 48.40.339.885 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 48.40.339.886 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 48.40.339.898 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 48.40.339.898 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 48.40.339.906 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 48.40.339.907 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 49.11.005.372 I slot print_timing: id 3 | task 9644 | n_decoded = 100, tg = 3.41 t/s, tg_3s = 3.41 t/s 49.14.258.845 I slot print_timing: id 3 | task 9644 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.38 t/s 49.17.516.725 I slot print_timing: id 3 | task 9644 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.38 t/s 49.20.774.376 I slot print_timing: id 3 | task 9644 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.38 t/s 49.24.031.774 I slot print_timing: id 3 | task 9644 | n_decoded = 144, tg = 3.40 t/s, tg_3s = 3.38 t/s 49.27.288.579 I slot print_timing: id 3 | task 9644 | n_decoded = 155, tg = 3.40 t/s, tg_3s = 3.38 t/s 49.30.556.872 I slot print_timing: id 3 | task 9644 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.37 t/s 49.33.823.130 I slot print_timing: id 3 | task 9644 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.37 t/s 49.37.093.032 I slot print_timing: id 3 | task 9644 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.36 t/s 49.40.363.371 I slot print_timing: id 3 | task 9644 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.36 t/s 49.43.637.108 I slot print_timing: id 3 | task 9644 | n_decoded = 210, tg = 3.39 t/s, tg_3s = 3.36 t/s 49.46.912.881 I slot print_timing: id 3 | task 9644 | n_decoded = 221, tg = 3.39 t/s, tg_3s = 3.36 t/s 49.50.189.187 I slot print_timing: id 3 | task 9644 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.36 t/s 49.53.464.959 I slot print_timing: id 3 | task 9644 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.36 t/s 49.56.736.045 I slot print_timing: id 3 | task 9644 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s 50.00.008.290 I slot print_timing: id 3 | task 9644 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s 50.03.281.615 I slot print_timing: id 3 | task 9644 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.36 t/s 50.06.557.774 I slot print_timing: id 3 | task 9644 | n_decoded = 287, tg = 3.38 t/s, tg_3s = 3.36 t/s 50.09.828.798 I slot print_timing: id 3 | task 9644 | n_decoded = 298, tg = 3.38 t/s, tg_3s = 3.36 t/s 50.13.100.159 I slot print_timing: id 3 | task 9644 | n_decoded = 309, tg = 3.38 t/s, tg_3s = 3.36 t/s 50.16.374.884 I slot print_timing: id 3 | task 9644 | n_decoded = 320, tg = 3.38 t/s, tg_3s = 3.36 t/s 50.19.650.439 I slot print_timing: id 3 | task 9644 | n_decoded = 331, tg = 3.38 t/s, tg_3s = 3.36 t/s 50.22.925.585 I slot print_timing: id 3 | task 9644 | n_decoded = 342, tg = 3.38 t/s, tg_3s = 3.36 t/s 50.26.200.945 I slot print_timing: id 3 | task 9644 | n_decoded = 353, tg = 3.38 t/s, tg_3s = 3.36 t/s 50.29.181.219 I slot print_timing: id 3 | task 9644 | prompt eval time = 1306.53 ms / 25 tokens ( 52.26 ms per token, 19.13 tokens per second) 50.29.181.223 I slot print_timing: id 3 | task 9644 | eval time = 107534.76 ms / 363 tokens ( 296.24 ms per token, 3.38 tokens per second) 50.29.181.224 I slot print_timing: id 3 | task 9644 | total time = 108841.29 ms / 388 tokens 50.29.181.225 I slot print_timing: id 3 | task 9644 | graphs reused = 9901 50.29.181.291 I slot release: id 3 | task 9644 | stop processing: n_tokens = 457, truncated = 0 50.29.181.302 I srv update_slots: all slots are idle 50.29.181.889 I srv operator(): operator(): cleaning up before exit...