0.00.380.391 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.380.394 I device_info: 0.00.380.415 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.380.420 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free) 0.00.380.427 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free) 0.00.380.487 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.380.491 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true 0.00.380.684 I srv init: running without SSL 0.00.380.872 I srv init: using 23 threads for HTTP server 0.00.381.203 I srv start: binding port with default address family 0.00.382.406 I srv llama_server: loading model 0.00.382.417 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/MiniMaxAI_MiniMax-M2-IQ2_S/MiniMaxAI_MiniMax-M2-IQ2_S-00001-of-00002.gguf' 0.00.382.464 I common_init_result: fitting params to device memory ... 0.00.382.469 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) 0.06.792.311 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect 0.06.883.252 W llama_model_loader: tensor overrides to CPU are used with mmap enabled - consider using --no-mmap for better performance 0.44.855.818 W llama_context: n_ctx_seq (16384) < n_ctx_train (196608) -- the full capacity of the model will not be utilized 0.45.091.936 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) 0.45.320.732 I srv load_model: initializing slots, n_slots = 4 0.45.428.723 W common_speculative_init: no implementations specified for speculative decoding 0.45.428.729 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384 0.45.428.733 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384 0.45.428.734 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384 0.45.428.734 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384 0.45.428.807 I srv load_model: prompt cache is enabled, size limit: 8192 MiB 0.45.428.808 I srv load_model: use `--cache-ram 0` to disable the prompt cache 0.45.428.808 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 0.45.428.809 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256 0.45.428.827 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task 0.45.432.525 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 0.45.446.455 I init: chat template, example_format: ']~!b[]~b]system You are a helpful assistant[e~[ ]~b]user Hello[e~[ ]~b]ai Hi there[e~[ ]~b]user How are you?[e~[ ]~b]ai ' 0.45.446.460 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 0.45.459.354 I srv init: init: chat template, thinking = 1 0.45.459.392 I srv llama_server: model loaded 0.45.459.397 I srv llama_server: server is listening on http://127.0.0.1:8090 0.45.459.402 I srv update_slots: all slots are idle 0.49.918.124 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 0.49.937.210 I srv operator(): Chat format: peg-native 0.49.943.242 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1 0.49.943.246 I srv get_availabl: updating prompt cache 0.49.943.253 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 0.49.943.257 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est) 0.49.943.258 I srv get_availabl: prompt cache update took 0.01 ms 0.49.945.037 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0 0.49.945.040 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 0.49.945.041 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 0.49.945.055 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 0.49.945.055 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 0.49.945.064 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 0.49.945.064 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 0.57.088.481 I slot print_timing: id 3 | task 0 | n_decoded = 100, tg = 22.57 t/s, tg_3s = 22.57 t/s 1.00.124.635 I slot print_timing: id 3 | task 0 | n_decoded = 168, tg = 22.50 t/s, tg_3s = 22.40 t/s 1.03.149.547 I slot print_timing: id 3 | task 0 | n_decoded = 236, tg = 22.49 t/s, tg_3s = 22.48 t/s 1.06.175.438 I slot print_timing: id 3 | task 0 | n_decoded = 304, tg = 22.49 t/s, tg_3s = 22.47 t/s 1.09.207.887 I slot print_timing: id 3 | task 0 | n_decoded = 372, tg = 22.48 t/s, tg_3s = 22.42 t/s 1.12.221.740 I slot print_timing: id 3 | task 0 | n_decoded = 439, tg = 22.44 t/s, tg_3s = 22.23 t/s 1.15.222.757 I slot print_timing: id 3 | task 0 | n_decoded = 506, tg = 22.42 t/s, tg_3s = 22.33 t/s 1.18.267.490 I slot print_timing: id 3 | task 0 | n_decoded = 574, tg = 22.41 t/s, tg_3s = 22.33 t/s 1.21.267.835 I slot print_timing: id 3 | task 0 | n_decoded = 641, tg = 22.40 t/s, tg_3s = 22.33 t/s 1.24.299.249 I slot print_timing: id 3 | task 0 | n_decoded = 707, tg = 22.34 t/s, tg_3s = 21.77 t/s 1.27.300.381 I slot print_timing: id 3 | task 0 | n_decoded = 773, tg = 22.31 t/s, tg_3s = 21.99 t/s 1.30.330.453 I slot print_timing: id 3 | task 0 | n_decoded = 839, tg = 22.27 t/s, tg_3s = 21.78 t/s 1.33.371.938 I slot print_timing: id 3 | task 0 | n_decoded = 906, tg = 22.25 t/s, tg_3s = 22.03 t/s 1.36.409.125 I slot print_timing: id 3 | task 0 | n_decoded = 972, tg = 22.22 t/s, tg_3s = 21.73 t/s 1.39.427.402 I slot print_timing: id 3 | task 0 | n_decoded = 1038, tg = 22.19 t/s, tg_3s = 21.87 t/s 1.42.436.510 I slot print_timing: id 3 | task 0 | n_decoded = 1104, tg = 22.18 t/s, tg_3s = 21.93 t/s 1.45.445.159 I slot print_timing: id 3 | task 0 | n_decoded = 1170, tg = 22.16 t/s, tg_3s = 21.94 t/s 1.48.470.589 I slot print_timing: id 3 | task 0 | n_decoded = 1236, tg = 22.15 t/s, tg_3s = 21.82 t/s 1.51.486.625 I slot print_timing: id 3 | task 0 | n_decoded = 1302, tg = 22.13 t/s, tg_3s = 21.88 t/s 1.54.518.282 I slot print_timing: id 3 | task 0 | n_decoded = 1367, tg = 22.10 t/s, tg_3s = 21.44 t/s 1.57.533.779 I slot print_timing: id 3 | task 0 | n_decoded = 1433, tg = 22.09 t/s, tg_3s = 21.89 t/s 2.00.554.236 I slot print_timing: id 3 | task 0 | n_decoded = 1499, tg = 22.08 t/s, tg_3s = 21.85 t/s 2.03.564.687 I slot print_timing: id 3 | task 0 | n_decoded = 1564, tg = 22.06 t/s, tg_3s = 21.59 t/s 2.06.578.724 I slot print_timing: id 3 | task 0 | n_decoded = 1630, tg = 22.05 t/s, tg_3s = 21.90 t/s 2.09.616.823 I slot print_timing: id 3 | task 0 | n_decoded = 1696, tg = 22.04 t/s, tg_3s = 21.72 t/s 2.12.622.142 I slot print_timing: id 3 | task 0 | n_decoded = 1758, tg = 21.98 t/s, tg_3s = 20.63 t/s 2.15.652.128 I slot print_timing: id 3 | task 0 | n_decoded = 1821, tg = 21.94 t/s, tg_3s = 20.79 t/s 2.18.673.299 I slot print_timing: id 3 | task 0 | n_decoded = 1884, tg = 21.90 t/s, tg_3s = 20.85 t/s 2.21.707.144 I slot print_timing: id 3 | task 0 | n_decoded = 1947, tg = 21.86 t/s, tg_3s = 20.77 t/s 2.24.741.940 I slot print_timing: id 3 | task 0 | n_decoded = 2009, tg = 21.82 t/s, tg_3s = 20.43 t/s 2.27.758.161 I slot print_timing: id 3 | task 0 | n_decoded = 2071, tg = 21.78 t/s, tg_3s = 20.56 t/s 2.30.763.971 I slot print_timing: id 3 | task 0 | n_decoded = 2133, tg = 21.74 t/s, tg_3s = 20.63 t/s 2.33.791.516 I slot print_timing: id 3 | task 0 | n_decoded = 2195, tg = 21.70 t/s, tg_3s = 20.48 t/s 2.36.834.721 I slot print_timing: id 3 | task 0 | n_decoded = 2257, tg = 21.66 t/s, tg_3s = 20.37 t/s 2.39.848.541 I slot print_timing: id 3 | task 0 | n_decoded = 2319, tg = 21.63 t/s, tg_3s = 20.57 t/s 2.42.860.951 I slot print_timing: id 3 | task 0 | n_decoded = 2381, tg = 21.61 t/s, tg_3s = 20.58 t/s 2.45.875.431 I slot print_timing: id 3 | task 0 | n_decoded = 2443, tg = 21.58 t/s, tg_3s = 20.57 t/s 2.48.875.986 I slot print_timing: id 3 | task 0 | n_decoded = 2504, tg = 21.55 t/s, tg_3s = 20.33 t/s 2.51.892.299 I slot print_timing: id 3 | task 0 | n_decoded = 2565, tg = 21.51 t/s, tg_3s = 20.22 t/s 2.54.926.259 I slot print_timing: id 3 | task 0 | n_decoded = 2626, tg = 21.48 t/s, tg_3s = 20.11 t/s 2.57.932.119 I slot print_timing: id 3 | task 0 | n_decoded = 2686, tg = 21.44 t/s, tg_3s = 19.96 t/s 3.00.944.332 I slot print_timing: id 3 | task 0 | n_decoded = 2746, tg = 21.41 t/s, tg_3s = 19.92 t/s 3.03.951.182 I slot print_timing: id 3 | task 0 | n_decoded = 2805, tg = 21.36 t/s, tg_3s = 19.62 t/s 3.06.966.207 I slot print_timing: id 3 | task 0 | n_decoded = 2865, tg = 21.33 t/s, tg_3s = 19.90 t/s 3.09.968.899 I slot print_timing: id 3 | task 0 | n_decoded = 2925, tg = 21.30 t/s, tg_3s = 19.98 t/s 3.12.970.795 I slot print_timing: id 3 | task 0 | n_decoded = 2985, tg = 21.27 t/s, tg_3s = 19.99 t/s 3.15.976.461 I slot print_timing: id 3 | task 0 | n_decoded = 3045, tg = 21.25 t/s, tg_3s = 19.96 t/s 3.18.982.621 I slot print_timing: id 3 | task 0 | n_decoded = 3105, tg = 21.22 t/s, tg_3s = 19.96 t/s 3.21.987.509 I slot print_timing: id 3 | task 0 | n_decoded = 3165, tg = 21.19 t/s, tg_3s = 19.97 t/s 3.24.997.205 I slot print_timing: id 3 | task 0 | n_decoded = 3225, tg = 21.17 t/s, tg_3s = 19.94 t/s 3.28.004.661 I slot print_timing: id 3 | task 0 | n_decoded = 3284, tg = 21.14 t/s, tg_3s = 19.62 t/s 3.31.018.744 I slot print_timing: id 3 | task 0 | n_decoded = 3344, tg = 21.12 t/s, tg_3s = 19.91 t/s 3.34.023.479 I slot print_timing: id 3 | task 0 | n_decoded = 3404, tg = 21.09 t/s, tg_3s = 19.97 t/s 3.37.073.288 I slot print_timing: id 3 | task 0 | n_decoded = 3465, tg = 21.07 t/s, tg_3s = 20.00 t/s 3.40.109.697 I slot print_timing: id 3 | task 0 | n_decoded = 3525, tg = 21.05 t/s, tg_3s = 19.76 t/s 3.43.149.633 I slot print_timing: id 3 | task 0 | n_decoded = 3585, tg = 21.03 t/s, tg_3s = 19.74 t/s 3.46.189.826 I slot print_timing: id 3 | task 0 | n_decoded = 3645, tg = 21.00 t/s, tg_3s = 19.74 t/s 3.49.230.106 I slot print_timing: id 3 | task 0 | n_decoded = 3705, tg = 20.98 t/s, tg_3s = 19.74 t/s 3.52.237.888 I slot print_timing: id 3 | task 0 | n_decoded = 3764, tg = 20.96 t/s, tg_3s = 19.62 t/s 3.55.247.373 I slot print_timing: id 3 | task 0 | n_decoded = 3823, tg = 20.94 t/s, tg_3s = 19.60 t/s 3.58.254.036 I slot print_timing: id 3 | task 0 | n_decoded = 3882, tg = 20.92 t/s, tg_3s = 19.62 t/s 4.01.262.135 I slot print_timing: id 3 | task 0 | n_decoded = 3941, tg = 20.90 t/s, tg_3s = 19.61 t/s 4.04.282.042 I slot print_timing: id 3 | task 0 | n_decoded = 4000, tg = 20.87 t/s, tg_3s = 19.54 t/s 4.07.309.508 I slot print_timing: id 3 | task 0 | n_decoded = 4059, tg = 20.85 t/s, tg_3s = 19.49 t/s 4.09.207.736 I slot print_timing: id 3 | task 0 | prompt eval time = 2712.07 ms / 99 tokens ( 27.39 ms per token, 36.50 tokens per second) 4.09.207.739 I slot print_timing: id 3 | task 0 | eval time = 196550.57 ms / 4096 tokens ( 47.99 ms per token, 20.84 tokens per second) 4.09.207.740 I slot print_timing: id 3 | task 0 | total time = 199262.64 ms / 4195 tokens 4.09.207.744 I slot print_timing: id 3 | task 0 | graphs reused = 4079 4.09.207.785 I slot release: id 3 | task 0 | stop processing: n_tokens = 4194, truncated = 0 4.09.207.808 I srv update_slots: all slots are idle 4.09.209.295 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 4.09.226.655 I srv operator(): Chat format: peg-native 4.09.232.072 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.796 (> 0.100 thold), f_keep = 0.018 4.09.232.074 I srv get_availabl: updating prompt cache 4.09.232.211 W srv prompt_save: - saving prompt with length 4194, total state size = 1015.784 MiB (draft: 0.000 MiB) 4.09.910.106 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.796 4.09.910.112 I srv update: - cache state: 1 prompts, 1015.784 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 4.09.910.113 I srv update: - prompt 0x561d97ec6750: 4194 tokens, checkpoints: 0, 1015.784 MiB 4.09.910.115 I srv get_availabl: prompt cache update took 678.04 ms 4.09.912.094 I slot launch_slot_: id 3 | task 4098 | processing task, is_child = 0 4.09.912.096 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.09.912.097 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.09.912.117 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.09.912.117 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.09.912.129 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.09.912.130 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.14.935.170 I slot print_timing: id 3 | task 4098 | n_decoded = 100, tg = 22.62 t/s, tg_3s = 22.62 t/s 4.17.972.727 I slot print_timing: id 3 | task 4098 | n_decoded = 168, tg = 22.52 t/s, tg_3s = 22.39 t/s 4.20.999.639 I slot print_timing: id 3 | task 4098 | n_decoded = 236, tg = 22.51 t/s, tg_3s = 22.47 t/s 4.24.030.393 I slot print_timing: id 3 | task 4098 | n_decoded = 304, tg = 22.49 t/s, tg_3s = 22.44 t/s 4.27.059.147 I slot print_timing: id 3 | task 4098 | n_decoded = 372, tg = 22.48 t/s, tg_3s = 22.45 t/s 4.30.099.396 I slot print_timing: id 3 | task 4098 | n_decoded = 440, tg = 22.47 t/s, tg_3s = 22.37 t/s 4.33.143.481 I slot print_timing: id 3 | task 4098 | n_decoded = 508, tg = 22.45 t/s, tg_3s = 22.34 t/s 4.36.150.344 I slot print_timing: id 3 | task 4098 | n_decoded = 575, tg = 22.43 t/s, tg_3s = 22.28 t/s 4.39.190.621 I slot print_timing: id 3 | task 4098 | n_decoded = 642, tg = 22.39 t/s, tg_3s = 22.04 t/s 4.39.235.420 I slot print_timing: id 3 | task 4098 | prompt eval time = 601.28 ms / 19 tokens ( 31.65 ms per token, 31.60 tokens per second) 4.39.235.423 I slot print_timing: id 3 | task 4098 | eval time = 28721.98 ms / 643 tokens ( 44.67 ms per token, 22.39 tokens per second) 4.39.235.424 I slot print_timing: id 3 | task 4098 | total time = 29323.26 ms / 662 tokens 4.39.235.425 I slot print_timing: id 3 | task 4098 | graphs reused = 4718 4.39.235.452 I slot release: id 3 | task 4098 | stop processing: n_tokens = 735, truncated = 0 4.39.235.466 I srv update_slots: all slots are idle 4.39.236.471 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 4.39.255.598 I srv operator(): Chat format: peg-native 4.39.261.910 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.787 (> 0.100 thold), f_keep = 0.101 4.39.261.912 I srv get_availabl: updating prompt cache 4.39.261.963 W srv prompt_save: - saving prompt with length 735, total state size = 178.018 MiB (draft: 0.000 MiB) 4.39.378.386 I srv load: - looking for better prompt, base f_keep = 0.101, sim = 0.787 4.39.378.392 I srv update: - cache state: 2 prompts, 1193.801 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 4.39.378.393 I srv update: - prompt 0x561d97ec6750: 4194 tokens, checkpoints: 0, 1015.784 MiB 4.39.378.394 I srv update: - prompt 0x561d90faec70: 735 tokens, checkpoints: 0, 178.018 MiB 4.39.378.395 I srv get_availabl: prompt cache update took 116.48 ms 4.39.380.019 I slot launch_slot_: id 3 | task 4742 | processing task, is_child = 0 4.39.380.021 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 4.39.380.022 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 4.39.380.032 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 4.39.380.032 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 4.39.380.040 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 4.39.380.040 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 4.44.395.109 I slot print_timing: id 3 | task 4742 | n_decoded = 100, tg = 22.64 t/s, tg_3s = 22.64 t/s 4.47.434.853 I slot print_timing: id 3 | task 4742 | n_decoded = 168, tg = 22.53 t/s, tg_3s = 22.37 t/s 4.50.466.213 I slot print_timing: id 3 | task 4742 | n_decoded = 236, tg = 22.50 t/s, tg_3s = 22.43 t/s 4.53.501.812 I slot print_timing: id 3 | task 4742 | n_decoded = 304, tg = 22.48 t/s, tg_3s = 22.40 t/s 4.56.538.735 I slot print_timing: id 3 | task 4742 | n_decoded = 372, tg = 22.46 t/s, tg_3s = 22.39 t/s 4.59.570.959 I slot print_timing: id 3 | task 4742 | n_decoded = 439, tg = 22.41 t/s, tg_3s = 22.10 t/s 5.02.599.951 I slot print_timing: id 3 | task 4742 | n_decoded = 506, tg = 22.37 t/s, tg_3s = 22.12 t/s 5.05.622.727 I slot print_timing: id 3 | task 4742 | n_decoded = 573, tg = 22.34 t/s, tg_3s = 22.17 t/s 5.06.656.955 I slot print_timing: id 3 | task 4742 | prompt eval time = 597.82 ms / 20 tokens ( 29.89 ms per token, 33.46 tokens per second) 5.06.656.958 I slot print_timing: id 3 | task 4742 | eval time = 26679.07 ms / 596 tokens ( 44.76 ms per token, 22.34 tokens per second) 5.06.656.959 I slot print_timing: id 3 | task 4742 | total time = 27276.89 ms / 616 tokens 5.06.656.960 I slot print_timing: id 3 | task 4742 | graphs reused = 5310 5.06.656.987 I slot release: id 3 | task 4742 | stop processing: n_tokens = 689, truncated = 0 5.06.657.001 I srv update_slots: all slots are idle 5.06.657.954 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 5.06.678.120 I srv operator(): Chat format: peg-native 5.06.684.211 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.796 (> 0.100 thold), f_keep = 0.107 5.06.684.214 I srv get_availabl: updating prompt cache 5.06.684.269 W srv prompt_save: - saving prompt with length 689, total state size = 166.877 MiB (draft: 0.000 MiB) 5.06.794.966 I srv load: - looking for better prompt, base f_keep = 0.107, sim = 0.796 5.06.794.972 I srv update: - cache state: 3 prompts, 1360.678 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 5.06.794.973 I srv update: - prompt 0x561d97ec6750: 4194 tokens, checkpoints: 0, 1015.784 MiB 5.06.794.974 I srv update: - prompt 0x561d90faec70: 735 tokens, checkpoints: 0, 178.018 MiB 5.06.794.975 I srv update: - prompt 0x561d97e885f0: 689 tokens, checkpoints: 0, 166.877 MiB 5.06.794.977 I srv get_availabl: prompt cache update took 110.76 ms 5.06.796.566 I slot launch_slot_: id 3 | task 5339 | processing task, is_child = 0 5.06.796.577 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 5.06.796.578 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 5.06.796.589 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 5.06.796.589 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 5.06.796.597 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 5.06.796.598 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 5.11.848.768 I slot print_timing: id 3 | task 5339 | n_decoded = 100, tg = 22.57 t/s, tg_3s = 22.57 t/s 5.14.853.728 I slot print_timing: id 3 | task 5339 | n_decoded = 167, tg = 22.46 t/s, tg_3s = 22.30 t/s 5.17.866.034 I slot print_timing: id 3 | task 5339 | n_decoded = 234, tg = 22.40 t/s, tg_3s = 22.24 t/s 5.20.904.078 I slot print_timing: id 3 | task 5339 | n_decoded = 302, tg = 22.39 t/s, tg_3s = 22.38 t/s 5.23.932.025 I slot print_timing: id 3 | task 5339 | n_decoded = 370, tg = 22.41 t/s, tg_3s = 22.46 t/s 5.26.945.838 I slot print_timing: id 3 | task 5339 | n_decoded = 437, tg = 22.38 t/s, tg_3s = 22.23 t/s 5.29.963.288 I slot print_timing: id 3 | task 5339 | n_decoded = 504, tg = 22.35 t/s, tg_3s = 22.20 t/s 5.32.970.622 I slot print_timing: id 3 | task 5339 | n_decoded = 571, tg = 22.35 t/s, tg_3s = 22.28 t/s 5.35.993.182 I slot print_timing: id 3 | task 5339 | n_decoded = 638, tg = 22.33 t/s, tg_3s = 22.17 t/s 5.39.037.477 I slot print_timing: id 3 | task 5339 | n_decoded = 705, tg = 22.30 t/s, tg_3s = 22.01 t/s 5.42.039.134 I slot print_timing: id 3 | task 5339 | n_decoded = 771, tg = 22.27 t/s, tg_3s = 21.99 t/s 5.45.084.547 I slot print_timing: id 3 | task 5339 | n_decoded = 838, tg = 22.25 t/s, tg_3s = 22.00 t/s 5.48.085.780 I slot print_timing: id 3 | task 5339 | n_decoded = 904, tg = 22.23 t/s, tg_3s = 21.99 t/s 5.51.123.969 I slot print_timing: id 3 | task 5339 | n_decoded = 969, tg = 22.17 t/s, tg_3s = 21.39 t/s 5.54.144.020 I slot print_timing: id 3 | task 5339 | n_decoded = 1034, tg = 22.13 t/s, tg_3s = 21.52 t/s 5.57.175.420 I slot print_timing: id 3 | task 5339 | n_decoded = 1100, tg = 22.11 t/s, tg_3s = 21.77 t/s 6.00.216.480 I slot print_timing: id 3 | task 5339 | n_decoded = 1167, tg = 22.10 t/s, tg_3s = 22.03 t/s 6.03.248.520 I slot print_timing: id 3 | task 5339 | n_decoded = 1232, tg = 22.07 t/s, tg_3s = 21.44 t/s 6.06.259.684 I slot print_timing: id 3 | task 5339 | n_decoded = 1297, tg = 22.04 t/s, tg_3s = 21.59 t/s 6.09.264.693 I slot print_timing: id 3 | task 5339 | n_decoded = 1363, tg = 22.04 t/s, tg_3s = 21.96 t/s 6.12.272.329 I slot print_timing: id 3 | task 5339 | n_decoded = 1429, tg = 22.03 t/s, tg_3s = 21.94 t/s 6.15.302.187 I slot print_timing: id 3 | task 5339 | n_decoded = 1495, tg = 22.02 t/s, tg_3s = 21.78 t/s 6.18.318.004 I slot print_timing: id 3 | task 5339 | n_decoded = 1561, tg = 22.02 t/s, tg_3s = 21.88 t/s 6.21.337.272 I slot print_timing: id 3 | task 5339 | n_decoded = 1627, tg = 22.01 t/s, tg_3s = 21.86 t/s 6.24.375.360 I slot print_timing: id 3 | task 5339 | n_decoded = 1693, tg = 22.00 t/s, tg_3s = 21.72 t/s 6.27.386.825 I slot print_timing: id 3 | task 5339 | n_decoded = 1755, tg = 21.95 t/s, tg_3s = 20.59 t/s 6.30.402.345 I slot print_timing: id 3 | task 5339 | n_decoded = 1818, tg = 21.91 t/s, tg_3s = 20.89 t/s 6.33.420.842 I slot print_timing: id 3 | task 5339 | n_decoded = 1881, tg = 21.87 t/s, tg_3s = 20.87 t/s 6.36.423.627 I slot print_timing: id 3 | task 5339 | n_decoded = 1943, tg = 21.83 t/s, tg_3s = 20.65 t/s 6.39.430.979 I slot print_timing: id 3 | task 5339 | n_decoded = 2005, tg = 21.79 t/s, tg_3s = 20.62 t/s 6.42.450.231 I slot print_timing: id 3 | task 5339 | n_decoded = 2067, tg = 21.75 t/s, tg_3s = 20.53 t/s 6.45.470.430 I slot print_timing: id 3 | task 5339 | n_decoded = 2129, tg = 21.71 t/s, tg_3s = 20.53 t/s 6.48.476.480 I slot print_timing: id 3 | task 5339 | n_decoded = 2191, tg = 21.68 t/s, tg_3s = 20.63 t/s 6.51.496.692 I slot print_timing: id 3 | task 5339 | n_decoded = 2253, tg = 21.65 t/s, tg_3s = 20.53 t/s 6.54.520.629 I slot print_timing: id 3 | task 5339 | n_decoded = 2315, tg = 21.61 t/s, tg_3s = 20.50 t/s 6.57.535.671 I slot print_timing: id 3 | task 5339 | n_decoded = 2377, tg = 21.59 t/s, tg_3s = 20.56 t/s 7.00.561.583 I slot print_timing: id 3 | task 5339 | n_decoded = 2439, tg = 21.56 t/s, tg_3s = 20.49 t/s 7.03.604.602 I slot print_timing: id 3 | task 5339 | n_decoded = 2501, tg = 21.53 t/s, tg_3s = 20.37 t/s 7.06.604.882 I slot print_timing: id 3 | task 5339 | n_decoded = 2562, tg = 21.50 t/s, tg_3s = 20.33 t/s 7.09.607.914 I slot print_timing: id 3 | task 5339 | n_decoded = 2623, tg = 21.47 t/s, tg_3s = 20.31 t/s 7.12.617.785 I slot print_timing: id 3 | task 5339 | n_decoded = 2683, tg = 21.43 t/s, tg_3s = 19.93 t/s 7.15.640.239 I slot print_timing: id 3 | task 5339 | n_decoded = 2744, tg = 21.40 t/s, tg_3s = 20.18 t/s 7.18.668.979 I slot print_timing: id 3 | task 5339 | n_decoded = 2805, tg = 21.37 t/s, tg_3s = 20.14 t/s 7.21.703.982 I slot print_timing: id 3 | task 5339 | n_decoded = 2866, tg = 21.34 t/s, tg_3s = 20.10 t/s 7.24.704.545 I slot print_timing: id 3 | task 5339 | n_decoded = 2926, tg = 21.31 t/s, tg_3s = 20.00 t/s 7.27.733.065 I slot print_timing: id 3 | task 5339 | n_decoded = 2986, tg = 21.28 t/s, tg_3s = 19.81 t/s 7.30.733.916 I slot print_timing: id 3 | task 5339 | n_decoded = 3046, tg = 21.25 t/s, tg_3s = 19.99 t/s 7.33.734.901 I slot print_timing: id 3 | task 5339 | n_decoded = 3106, tg = 21.23 t/s, tg_3s = 19.99 t/s 7.36.743.468 I slot print_timing: id 3 | task 5339 | n_decoded = 3166, tg = 21.20 t/s, tg_3s = 19.94 t/s 7.39.751.629 I slot print_timing: id 3 | task 5339 | n_decoded = 3226, tg = 21.18 t/s, tg_3s = 19.95 t/s 7.42.766.902 I slot print_timing: id 3 | task 5339 | n_decoded = 3286, tg = 21.15 t/s, tg_3s = 19.90 t/s 7.45.771.954 I slot print_timing: id 3 | task 5339 | n_decoded = 3346, tg = 21.13 t/s, tg_3s = 19.97 t/s 7.48.773.024 I slot print_timing: id 3 | task 5339 | n_decoded = 3406, tg = 21.11 t/s, tg_3s = 19.99 t/s 7.51.775.205 I slot print_timing: id 3 | task 5339 | n_decoded = 3466, tg = 21.09 t/s, tg_3s = 19.99 t/s 7.54.781.779 I slot print_timing: id 3 | task 5339 | n_decoded = 3525, tg = 21.06 t/s, tg_3s = 19.62 t/s 7.57.781.945 I slot print_timing: id 3 | task 5339 | n_decoded = 3584, tg = 21.04 t/s, tg_3s = 19.67 t/s 8.00.809.786 I slot print_timing: id 3 | task 5339 | n_decoded = 3643, tg = 21.01 t/s, tg_3s = 19.49 t/s 8.03.852.432 I slot print_timing: id 3 | task 5339 | n_decoded = 3703, tg = 20.99 t/s, tg_3s = 19.72 t/s 8.06.857.235 I slot print_timing: id 3 | task 5339 | n_decoded = 3762, tg = 20.97 t/s, tg_3s = 19.64 t/s 8.09.865.855 I slot print_timing: id 3 | task 5339 | n_decoded = 3821, tg = 20.94 t/s, tg_3s = 19.61 t/s 8.12.877.440 I slot print_timing: id 3 | task 5339 | n_decoded = 3880, tg = 20.92 t/s, tg_3s = 19.59 t/s 8.15.885.577 I slot print_timing: id 3 | task 5339 | n_decoded = 3939, tg = 20.90 t/s, tg_3s = 19.61 t/s 8.18.894.402 I slot print_timing: id 3 | task 5339 | n_decoded = 3998, tg = 20.88 t/s, tg_3s = 19.61 t/s 8.21.929.629 I slot print_timing: id 3 | task 5339 | n_decoded = 4057, tg = 20.86 t/s, tg_3s = 19.44 t/s 8.23.930.991 I slot print_timing: id 3 | task 5339 | prompt eval time = 621.31 ms / 19 tokens ( 32.70 ms per token, 30.58 tokens per second) 8.23.930.995 I slot print_timing: id 3 | task 5339 | eval time = 196513.05 ms / 4096 tokens ( 47.98 ms per token, 20.84 tokens per second) 8.23.930.996 I slot print_timing: id 3 | task 5339 | total time = 197134.36 ms / 4115 tokens 8.23.930.997 I slot print_timing: id 3 | task 5339 | graphs reused = 9388 8.23.931.025 I slot release: id 3 | task 5339 | stop processing: n_tokens = 4188, truncated = 0 8.23.931.053 I srv update_slots: all slots are idle 8.23.932.338 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 8.23.951.709 I srv operator(): Chat format: peg-native 8.23.957.815 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.698 (> 0.100 thold), f_keep = 0.018 8.23.957.817 I srv get_availabl: updating prompt cache 8.23.957.989 W srv prompt_save: - saving prompt with length 4188, total state size = 1014.331 MiB (draft: 0.000 MiB) 8.24.612.714 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.698 8.24.612.723 I srv update: - cache state: 4 prompts, 2375.009 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 8.24.612.724 I srv update: - prompt 0x561d97ec6750: 4194 tokens, checkpoints: 0, 1015.784 MiB 8.24.612.725 I srv update: - prompt 0x561d90faec70: 735 tokens, checkpoints: 0, 178.018 MiB 8.24.612.725 I srv update: - prompt 0x561d97e885f0: 689 tokens, checkpoints: 0, 166.877 MiB 8.24.612.726 I srv update: - prompt 0x561d97e68780: 4188 tokens, checkpoints: 0, 1014.331 MiB 8.24.612.727 I srv get_availabl: prompt cache update took 654.91 ms 8.24.614.526 I slot launch_slot_: id 3 | task 9436 | processing task, is_child = 0 8.24.614.528 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 8.24.614.529 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 8.24.614.548 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 8.24.614.548 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 8.24.614.558 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 8.24.614.559 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 8.30.119.390 I slot print_timing: id 3 | task 9436 | n_decoded = 100, tg = 22.20 t/s, tg_3s = 22.20 t/s 8.33.152.096 I slot print_timing: id 3 | task 9436 | n_decoded = 168, tg = 22.29 t/s, tg_3s = 22.42 t/s 8.36.179.130 I slot print_timing: id 3 | task 9436 | n_decoded = 236, tg = 22.34 t/s, tg_3s = 22.46 t/s 8.39.219.431 I slot print_timing: id 3 | task 9436 | n_decoded = 304, tg = 22.35 t/s, tg_3s = 22.37 t/s 8.42.252.267 I slot print_timing: id 3 | task 9436 | n_decoded = 372, tg = 22.36 t/s, tg_3s = 22.42 t/s 8.45.258.487 I slot print_timing: id 3 | task 9436 | n_decoded = 439, tg = 22.35 t/s, tg_3s = 22.29 t/s 8.48.287.708 I slot print_timing: id 3 | task 9436 | n_decoded = 506, tg = 22.32 t/s, tg_3s = 22.12 t/s 8.51.324.703 I slot print_timing: id 3 | task 9436 | n_decoded = 573, tg = 22.29 t/s, tg_3s = 22.06 t/s 8.54.336.019 I slot print_timing: id 3 | task 9436 | n_decoded = 640, tg = 22.28 t/s, tg_3s = 22.25 t/s 8.57.339.791 I slot print_timing: id 3 | task 9436 | n_decoded = 706, tg = 22.25 t/s, tg_3s = 21.97 t/s 9.00.345.061 I slot print_timing: id 3 | task 9436 | n_decoded = 772, tg = 22.23 t/s, tg_3s = 21.96 t/s 9.03.351.870 I slot print_timing: id 3 | task 9436 | n_decoded = 838, tg = 22.21 t/s, tg_3s = 21.95 t/s 9.06.360.220 I slot print_timing: id 3 | task 9436 | n_decoded = 904, tg = 22.19 t/s, tg_3s = 21.94 t/s 9.09.403.284 I slot print_timing: id 3 | task 9436 | n_decoded = 970, tg = 22.15 t/s, tg_3s = 21.69 t/s 9.12.427.471 I slot print_timing: id 3 | task 9436 | n_decoded = 1036, tg = 22.13 t/s, tg_3s = 21.82 t/s 9.15.442.862 I slot print_timing: id 3 | task 9436 | n_decoded = 1102, tg = 22.12 t/s, tg_3s = 21.89 t/s 9.18.474.103 I slot print_timing: id 3 | task 9436 | n_decoded = 1168, tg = 22.10 t/s, tg_3s = 21.77 t/s 9.21.505.887 I slot print_timing: id 3 | task 9436 | n_decoded = 1234, tg = 22.08 t/s, tg_3s = 21.77 t/s 9.24.519.040 I slot print_timing: id 3 | task 9436 | n_decoded = 1300, tg = 22.07 t/s, tg_3s = 21.90 t/s 9.27.547.408 I slot print_timing: id 3 | task 9436 | n_decoded = 1366, tg = 22.06 t/s, tg_3s = 21.79 t/s 9.30.565.776 I slot print_timing: id 3 | task 9436 | n_decoded = 1432, tg = 22.05 t/s, tg_3s = 21.87 t/s 9.33.609.755 I slot print_timing: id 3 | task 9436 | n_decoded = 1498, tg = 22.03 t/s, tg_3s = 21.68 t/s 9.36.626.586 I slot print_timing: id 3 | task 9436 | n_decoded = 1563, tg = 22.01 t/s, tg_3s = 21.55 t/s 9.39.666.774 I slot print_timing: id 3 | task 9436 | n_decoded = 1629, tg = 22.00 t/s, tg_3s = 21.71 t/s 9.42.676.732 I slot print_timing: id 3 | task 9436 | n_decoded = 1694, tg = 21.98 t/s, tg_3s = 21.59 t/s 9.45.701.808 I slot print_timing: id 3 | task 9436 | n_decoded = 1757, tg = 21.94 t/s, tg_3s = 20.83 t/s 9.48.741.450 I slot print_timing: id 3 | task 9436 | n_decoded = 1820, tg = 21.89 t/s, tg_3s = 20.73 t/s 9.51.742.627 I slot print_timing: id 3 | task 9436 | n_decoded = 1882, tg = 21.85 t/s, tg_3s = 20.66 t/s 9.54.791.455 I slot print_timing: id 3 | task 9436 | n_decoded = 1945, tg = 21.81 t/s, tg_3s = 20.66 t/s 9.57.816.103 I slot print_timing: id 3 | task 9436 | n_decoded = 2007, tg = 21.77 t/s, tg_3s = 20.50 t/s 10.00.833.058 I slot print_timing: id 3 | task 9436 | n_decoded = 2069, tg = 21.73 t/s, tg_3s = 20.55 t/s 10.03.880.881 I slot print_timing: id 3 | task 9436 | n_decoded = 2132, tg = 21.70 t/s, tg_3s = 20.67 t/s 10.06.926.511 I slot print_timing: id 3 | task 9436 | n_decoded = 2195, tg = 21.67 t/s, tg_3s = 20.69 t/s 10.09.962.937 I slot print_timing: id 3 | task 9436 | n_decoded = 2257, tg = 21.63 t/s, tg_3s = 20.42 t/s 10.12.982.460 I slot print_timing: id 3 | task 9436 | n_decoded = 2318, tg = 21.59 t/s, tg_3s = 20.20 t/s 10.15.998.331 I slot print_timing: id 3 | task 9436 | n_decoded = 2380, tg = 21.56 t/s, tg_3s = 20.56 t/s 10.19.009.155 I slot print_timing: id 3 | task 9436 | n_decoded = 2442, tg = 21.54 t/s, tg_3s = 20.59 t/s 10.22.055.834 I slot print_timing: id 3 | task 9436 | n_decoded = 2504, tg = 21.50 t/s, tg_3s = 20.35 t/s 10.25.099.812 I slot print_timing: id 3 | task 9436 | n_decoded = 2566, tg = 21.48 t/s, tg_3s = 20.37 t/s 10.28.147.787 I slot print_timing: id 3 | task 9436 | n_decoded = 2628, tg = 21.45 t/s, tg_3s = 20.34 t/s 10.31.185.882 I slot print_timing: id 3 | task 9436 | n_decoded = 2690, tg = 21.42 t/s, tg_3s = 20.41 t/s 10.34.209.544 I slot print_timing: id 3 | task 9436 | n_decoded = 2751, tg = 21.39 t/s, tg_3s = 20.17 t/s 10.37.223.982 I slot print_timing: id 3 | task 9436 | n_decoded = 2811, tg = 21.36 t/s, tg_3s = 19.90 t/s 10.40.250.048 I slot print_timing: id 3 | task 9436 | n_decoded = 2872, tg = 21.33 t/s, tg_3s = 20.16 t/s 10.43.278.634 I slot print_timing: id 3 | task 9436 | n_decoded = 2933, tg = 21.31 t/s, tg_3s = 20.14 t/s 10.46.323.852 I slot print_timing: id 3 | task 9436 | n_decoded = 2993, tg = 21.27 t/s, tg_3s = 19.70 t/s 10.49.324.587 I slot print_timing: id 3 | task 9436 | n_decoded = 3053, tg = 21.24 t/s, tg_3s = 20.00 t/s 10.52.327.155 I slot print_timing: id 3 | task 9436 | n_decoded = 3113, tg = 21.22 t/s, tg_3s = 19.98 t/s 10.55.340.944 I slot print_timing: id 3 | task 9436 | n_decoded = 3173, tg = 21.19 t/s, tg_3s = 19.91 t/s 10.58.355.083 I slot print_timing: id 3 | task 9436 | n_decoded = 3233, tg = 21.17 t/s, tg_3s = 19.91 t/s 11.01.404.918 I slot print_timing: id 3 | task 9436 | n_decoded = 3294, tg = 21.14 t/s, tg_3s = 20.00 t/s 11.04.409.090 I slot print_timing: id 3 | task 9436 | n_decoded = 3354, tg = 21.12 t/s, tg_3s = 19.97 t/s 11.07.418.253 I slot print_timing: id 3 | task 9436 | n_decoded = 3414, tg = 21.10 t/s, tg_3s = 19.94 t/s 11.10.442.846 I slot print_timing: id 3 | task 9436 | n_decoded = 3473, tg = 21.07 t/s, tg_3s = 19.51 t/s 11.13.490.518 I slot print_timing: id 3 | task 9436 | n_decoded = 3533, tg = 21.05 t/s, tg_3s = 19.69 t/s 11.16.533.655 I slot print_timing: id 3 | task 9436 | n_decoded = 3593, tg = 21.02 t/s, tg_3s = 19.72 t/s 11.19.578.994 I slot print_timing: id 3 | task 9436 | n_decoded = 3653, tg = 21.00 t/s, tg_3s = 19.70 t/s 11.22.624.718 I slot print_timing: id 3 | task 9436 | n_decoded = 3713, tg = 20.98 t/s, tg_3s = 19.70 t/s 11.25.656.018 I slot print_timing: id 3 | task 9436 | n_decoded = 3772, tg = 20.95 t/s, tg_3s = 19.46 t/s 11.28.666.104 I slot print_timing: id 3 | task 9436 | n_decoded = 3831, tg = 20.93 t/s, tg_3s = 19.60 t/s 11.31.682.490 I slot print_timing: id 3 | task 9436 | n_decoded = 3890, tg = 20.91 t/s, tg_3s = 19.56 t/s 11.34.709.547 I slot print_timing: id 3 | task 9436 | n_decoded = 3949, tg = 20.88 t/s, tg_3s = 19.49 t/s 11.37.744.204 I slot print_timing: id 3 | task 9436 | n_decoded = 4008, tg = 20.86 t/s, tg_3s = 19.44 t/s 11.40.785.459 I slot print_timing: id 3 | task 9436 | n_decoded = 4067, tg = 20.84 t/s, tg_3s = 19.40 t/s 11.42.271.844 I slot print_timing: id 3 | task 9436 | prompt eval time = 1000.09 ms / 32 tokens ( 31.25 ms per token, 32.00 tokens per second) 11.42.271.848 I slot print_timing: id 3 | task 9436 | eval time = 196657.16 ms / 4096 tokens ( 48.01 ms per token, 20.83 tokens per second) 11.42.271.849 I slot print_timing: id 3 | task 9436 | total time = 197657.25 ms / 4128 tokens 11.42.271.850 I slot print_timing: id 3 | task 9436 | graphs reused = 13466 11.42.271.876 I slot release: id 3 | task 9436 | stop processing: n_tokens = 4201, truncated = 0 11.42.271.900 I srv update_slots: all slots are idle 11.42.273.227 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 11.42.292.277 I srv operator(): Chat format: peg-native 11.42.298.689 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.779 (> 0.100 thold), f_keep = 0.018 11.42.298.692 I srv get_availabl: updating prompt cache 11.42.298.848 W srv prompt_save: - saving prompt with length 4201, total state size = 1017.479 MiB (draft: 0.000 MiB) 11.42.925.329 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.779 11.42.925.336 I srv update: - cache state: 5 prompts, 3392.488 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 11.42.925.337 I srv update: - prompt 0x561d97ec6750: 4194 tokens, checkpoints: 0, 1015.784 MiB 11.42.925.338 I srv update: - prompt 0x561d90faec70: 735 tokens, checkpoints: 0, 178.018 MiB 11.42.925.338 I srv update: - prompt 0x561d97e885f0: 689 tokens, checkpoints: 0, 166.877 MiB 11.42.925.339 I srv update: - prompt 0x561d97e68780: 4188 tokens, checkpoints: 0, 1014.331 MiB 11.42.925.339 I srv update: - prompt 0x561d982cf8c0: 4201 tokens, checkpoints: 0, 1017.479 MiB 11.42.925.340 I srv get_availabl: prompt cache update took 626.65 ms 11.42.926.981 I slot launch_slot_: id 3 | task 13533 | processing task, is_child = 0 11.42.926.984 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 11.42.926.984 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 11.42.927.004 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 11.42.927.004 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 11.42.927.014 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 11.42.927.014 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 11.47.993.140 I slot print_timing: id 3 | task 13533 | n_decoded = 100, tg = 22.53 t/s, tg_3s = 22.53 t/s 11.51.015.872 I slot print_timing: id 3 | task 13533 | n_decoded = 167, tg = 22.38 t/s, tg_3s = 22.17 t/s 11.54.039.724 I slot print_timing: id 3 | task 13533 | n_decoded = 234, tg = 22.32 t/s, tg_3s = 22.16 t/s 11.57.047.498 I slot print_timing: id 3 | task 13533 | n_decoded = 300, tg = 22.23 t/s, tg_3s = 21.94 t/s 12.00.081.030 I slot print_timing: id 3 | task 13533 | n_decoded = 368, tg = 22.27 t/s, tg_3s = 22.42 t/s 12.03.090.164 I slot print_timing: id 3 | task 13533 | n_decoded = 435, tg = 22.27 t/s, tg_3s = 22.27 t/s 12.06.111.133 I slot print_timing: id 3 | task 13533 | n_decoded = 502, tg = 22.26 t/s, tg_3s = 22.18 t/s 12.09.135.509 I slot print_timing: id 3 | task 13533 | n_decoded = 569, tg = 22.24 t/s, tg_3s = 22.15 t/s 12.12.163.511 I slot print_timing: id 3 | task 13533 | n_decoded = 636, tg = 22.23 t/s, tg_3s = 22.13 t/s 12.15.171.314 I slot print_timing: id 3 | task 13533 | n_decoded = 702, tg = 22.20 t/s, tg_3s = 21.94 t/s 12.18.209.162 I slot print_timing: id 3 | task 13533 | n_decoded = 768, tg = 22.16 t/s, tg_3s = 21.73 t/s 12.21.251.286 I slot print_timing: id 3 | task 13533 | n_decoded = 835, tg = 22.15 t/s, tg_3s = 22.02 t/s 12.24.275.782 I slot print_timing: id 3 | task 13533 | n_decoded = 901, tg = 22.13 t/s, tg_3s = 21.82 t/s 12.24.693.282 I slot print_timing: id 3 | task 13533 | prompt eval time = 627.47 ms / 21 tokens ( 29.88 ms per token, 33.47 tokens per second) 12.24.693.287 I slot print_timing: id 3 | task 13533 | eval time = 41138.77 ms / 910 tokens ( 45.21 ms per token, 22.12 tokens per second) 12.24.693.287 I slot print_timing: id 3 | task 13533 | total time = 41766.24 ms / 931 tokens 12.24.693.288 I slot print_timing: id 3 | task 13533 | graphs reused = 14371 12.24.693.317 I slot release: id 3 | task 13533 | stop processing: n_tokens = 1004, truncated = 0 12.24.693.334 I srv update_slots: all slots are idle 12.24.694.310 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 12.24.713.043 I srv operator(): Chat format: peg-native 12.24.719.327 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.824 (> 0.100 thold), f_keep = 0.075 12.24.719.329 I srv get_availabl: updating prompt cache 12.24.719.393 W srv prompt_save: - saving prompt with length 1004, total state size = 243.169 MiB (draft: 0.000 MiB) 12.24.884.119 I srv load: - looking for better prompt, base f_keep = 0.075, sim = 0.824 12.24.884.126 I srv update: - cache state: 6 prompts, 3635.657 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 12.24.884.127 I srv update: - prompt 0x561d97ec6750: 4194 tokens, checkpoints: 0, 1015.784 MiB 12.24.884.128 I srv update: - prompt 0x561d90faec70: 735 tokens, checkpoints: 0, 178.018 MiB 12.24.884.128 I srv update: - prompt 0x561d97e885f0: 689 tokens, checkpoints: 0, 166.877 MiB 12.24.884.133 I srv update: - prompt 0x561d97e68780: 4188 tokens, checkpoints: 0, 1014.331 MiB 12.24.884.134 I srv update: - prompt 0x561d982cf8c0: 4201 tokens, checkpoints: 0, 1017.479 MiB 12.24.884.135 I srv update: - prompt 0x561d90bf8cd0: 1004 tokens, checkpoints: 0, 243.169 MiB 12.24.884.136 I srv get_availabl: prompt cache update took 164.81 ms 12.24.885.882 I slot launch_slot_: id 3 | task 14444 | processing task, is_child = 0 12.24.885.885 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 12.24.885.885 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 12.24.885.896 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 12.24.885.896 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 12.24.885.905 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 12.24.885.905 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 12.29.756.884 I slot print_timing: id 3 | task 14444 | n_decoded = 100, tg = 22.44 t/s, tg_3s = 22.44 t/s 12.32.772.579 I slot print_timing: id 3 | task 14444 | n_decoded = 167, tg = 22.35 t/s, tg_3s = 22.22 t/s 12.35.780.303 I slot print_timing: id 3 | task 14444 | n_decoded = 234, tg = 22.33 t/s, tg_3s = 22.28 t/s 12.38.823.590 I slot print_timing: id 3 | task 14444 | n_decoded = 302, tg = 22.33 t/s, tg_3s = 22.34 t/s 12.41.827.004 I slot print_timing: id 3 | task 14444 | n_decoded = 369, tg = 22.33 t/s, tg_3s = 22.31 t/s 12.44.829.014 I slot print_timing: id 3 | task 14444 | n_decoded = 436, tg = 22.33 t/s, tg_3s = 22.32 t/s 12.47.848.391 I slot print_timing: id 3 | task 14444 | n_decoded = 502, tg = 22.26 t/s, tg_3s = 21.86 t/s 12.50.852.712 I slot print_timing: id 3 | task 14444 | n_decoded = 569, tg = 22.27 t/s, tg_3s = 22.30 t/s 12.53.896.596 I slot print_timing: id 3 | task 14444 | n_decoded = 637, tg = 22.28 t/s, tg_3s = 22.34 t/s 12.56.930.515 I slot print_timing: id 3 | task 14444 | n_decoded = 704, tg = 22.26 t/s, tg_3s = 22.08 t/s 12.59.969.746 I slot print_timing: id 3 | task 14444 | n_decoded = 771, tg = 22.24 t/s, tg_3s = 22.05 t/s 13.03.008.221 I slot print_timing: id 3 | task 14444 | n_decoded = 838, tg = 22.22 t/s, tg_3s = 22.05 t/s 13.06.010.545 I slot print_timing: id 3 | task 14444 | n_decoded = 904, tg = 22.21 t/s, tg_3s = 21.98 t/s 13.09.026.637 I slot print_timing: id 3 | task 14444 | n_decoded = 970, tg = 22.18 t/s, tg_3s = 21.88 t/s 13.12.060.288 I slot print_timing: id 3 | task 14444 | n_decoded = 1036, tg = 22.16 t/s, tg_3s = 21.76 t/s 13.15.092.904 I slot print_timing: id 3 | task 14444 | n_decoded = 1102, tg = 22.13 t/s, tg_3s = 21.76 t/s 13.18.113.140 I slot print_timing: id 3 | task 14444 | n_decoded = 1167, tg = 22.10 t/s, tg_3s = 21.52 t/s 13.21.122.414 I slot print_timing: id 3 | task 14444 | n_decoded = 1232, tg = 22.07 t/s, tg_3s = 21.60 t/s 13.24.149.519 I slot print_timing: id 3 | task 14444 | n_decoded = 1298, tg = 22.06 t/s, tg_3s = 21.80 t/s 13.27.175.421 I slot print_timing: id 3 | task 14444 | n_decoded = 1364, tg = 22.04 t/s, tg_3s = 21.81 t/s 13.30.191.641 I slot print_timing: id 3 | task 14444 | n_decoded = 1430, tg = 22.04 t/s, tg_3s = 21.88 t/s 13.33.211.858 I slot print_timing: id 3 | task 14444 | n_decoded = 1496, tg = 22.03 t/s, tg_3s = 21.85 t/s 13.36.222.884 I slot print_timing: id 3 | task 14444 | n_decoded = 1562, tg = 22.02 t/s, tg_3s = 21.92 t/s 13.39.236.240 I slot print_timing: id 3 | task 14444 | n_decoded = 1628, tg = 22.02 t/s, tg_3s = 21.90 t/s 13.42.267.099 I slot print_timing: id 3 | task 14444 | n_decoded = 1694, tg = 22.01 t/s, tg_3s = 21.78 t/s 13.45.307.636 I slot print_timing: id 3 | task 14444 | n_decoded = 1757, tg = 21.96 t/s, tg_3s = 20.72 t/s 13.48.322.818 I slot print_timing: id 3 | task 14444 | n_decoded = 1820, tg = 21.92 t/s, tg_3s = 20.89 t/s 13.51.340.401 I slot print_timing: id 3 | task 14444 | n_decoded = 1883, tg = 21.89 t/s, tg_3s = 20.88 t/s 13.54.357.678 I slot print_timing: id 3 | task 14444 | n_decoded = 1946, tg = 21.85 t/s, tg_3s = 20.88 t/s 13.57.402.275 I slot print_timing: id 3 | task 14444 | n_decoded = 2009, tg = 21.81 t/s, tg_3s = 20.69 t/s 14.00.436.176 I slot print_timing: id 3 | task 14444 | n_decoded = 2072, tg = 21.78 t/s, tg_3s = 20.77 t/s 14.03.476.294 I slot print_timing: id 3 | task 14444 | n_decoded = 2135, tg = 21.75 t/s, tg_3s = 20.72 t/s 14.06.518.241 I slot print_timing: id 3 | task 14444 | n_decoded = 2198, tg = 21.72 t/s, tg_3s = 20.71 t/s 14.09.566.165 I slot print_timing: id 3 | task 14444 | n_decoded = 2260, tg = 21.68 t/s, tg_3s = 20.34 t/s 14.12.590.465 I slot print_timing: id 3 | task 14444 | n_decoded = 2322, tg = 21.64 t/s, tg_3s = 20.50 t/s 14.15.595.578 I slot print_timing: id 3 | task 14444 | n_decoded = 2383, tg = 21.61 t/s, tg_3s = 20.30 t/s 14.18.614.552 I slot print_timing: id 3 | task 14444 | n_decoded = 2444, tg = 21.57 t/s, tg_3s = 20.21 t/s 14.21.657.825 I slot print_timing: id 3 | task 14444 | n_decoded = 2506, tg = 21.54 t/s, tg_3s = 20.37 t/s 14.24.687.178 I slot print_timing: id 3 | task 14444 | n_decoded = 2567, tg = 21.50 t/s, tg_3s = 20.14 t/s 14.27.690.490 I slot print_timing: id 3 | task 14444 | n_decoded = 2628, tg = 21.47 t/s, tg_3s = 20.31 t/s 14.30.735.345 I slot print_timing: id 3 | task 14444 | n_decoded = 2690, tg = 21.45 t/s, tg_3s = 20.36 t/s 14.33.750.708 I slot print_timing: id 3 | task 14444 | n_decoded = 2751, tg = 21.42 t/s, tg_3s = 20.23 t/s 14.36.780.462 I slot print_timing: id 3 | task 14444 | n_decoded = 2812, tg = 21.39 t/s, tg_3s = 20.13 t/s 14.39.804.448 I slot print_timing: id 3 | task 14444 | n_decoded = 2873, tg = 21.36 t/s, tg_3s = 20.17 t/s 14.42.830.252 I slot print_timing: id 3 | task 14444 | n_decoded = 2934, tg = 21.33 t/s, tg_3s = 20.16 t/s 14.45.878.019 I slot print_timing: id 3 | task 14444 | n_decoded = 2995, tg = 21.31 t/s, tg_3s = 20.01 t/s 14.48.884.797 I slot print_timing: id 3 | task 14444 | n_decoded = 3055, tg = 21.28 t/s, tg_3s = 19.95 t/s 14.51.885.603 I slot print_timing: id 3 | task 14444 | n_decoded = 3115, tg = 21.25 t/s, tg_3s = 19.99 t/s 14.54.890.750 I slot print_timing: id 3 | task 14444 | n_decoded = 3175, tg = 21.22 t/s, tg_3s = 19.97 t/s 14.57.895.750 I slot print_timing: id 3 | task 14444 | n_decoded = 3235, tg = 21.20 t/s, tg_3s = 19.97 t/s 15.00.908.068 I slot print_timing: id 3 | task 14444 | n_decoded = 3295, tg = 21.18 t/s, tg_3s = 19.92 t/s 15.03.918.863 I slot print_timing: id 3 | task 14444 | n_decoded = 3355, tg = 21.15 t/s, tg_3s = 19.93 t/s 15.06.932.063 I slot print_timing: id 3 | task 14444 | n_decoded = 3415, tg = 21.13 t/s, tg_3s = 19.91 t/s 15.09.941.526 I slot print_timing: id 3 | task 14444 | n_decoded = 3475, tg = 21.11 t/s, tg_3s = 19.94 t/s 15.12.962.725 I slot print_timing: id 3 | task 14444 | n_decoded = 3534, tg = 21.08 t/s, tg_3s = 19.53 t/s 15.16.010.847 I slot print_timing: id 3 | task 14444 | n_decoded = 3594, tg = 21.05 t/s, tg_3s = 19.68 t/s 15.19.013.166 I slot print_timing: id 3 | task 14444 | n_decoded = 3653, tg = 21.03 t/s, tg_3s = 19.65 t/s 15.22.016.401 I slot print_timing: id 3 | task 14444 | n_decoded = 3712, tg = 21.01 t/s, tg_3s = 19.65 t/s 15.25.020.860 I slot print_timing: id 3 | task 14444 | n_decoded = 3771, tg = 20.98 t/s, tg_3s = 19.64 t/s 15.28.045.154 I slot print_timing: id 3 | task 14444 | n_decoded = 3830, tg = 20.96 t/s, tg_3s = 19.51 t/s 15.31.053.952 I slot print_timing: id 3 | task 14444 | n_decoded = 3889, tg = 20.94 t/s, tg_3s = 19.61 t/s 15.34.078.338 I slot print_timing: id 3 | task 14444 | n_decoded = 3948, tg = 20.91 t/s, tg_3s = 19.51 t/s 15.37.084.597 I slot print_timing: id 3 | task 14444 | n_decoded = 4007, tg = 20.89 t/s, tg_3s = 19.63 t/s 15.40.118.414 I slot print_timing: id 3 | task 14444 | n_decoded = 4066, tg = 20.87 t/s, tg_3s = 19.45 t/s 15.41.657.189 I slot print_timing: id 3 | task 14444 | prompt eval time = 415.39 ms / 16 tokens ( 25.96 ms per token, 38.52 tokens per second) 15.41.657.194 I slot print_timing: id 3 | task 14444 | eval time = 196355.87 ms / 4096 tokens ( 47.94 ms per token, 20.86 tokens per second) 15.41.657.194 I slot print_timing: id 3 | task 14444 | total time = 196771.26 ms / 4112 tokens 15.41.657.195 I slot print_timing: id 3 | task 14444 | graphs reused = 18449 15.41.657.222 I slot release: id 3 | task 14444 | stop processing: n_tokens = 4186, truncated = 0 15.41.657.254 I srv update_slots: all slots are idle 15.41.658.592 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 15.41.677.996 I srv operator(): Chat format: peg-native 15.41.684.018 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.806 (> 0.100 thold), f_keep = 0.018 15.41.684.020 I srv get_availabl: updating prompt cache 15.41.684.180 W srv prompt_save: - saving prompt with length 4186, total state size = 1013.846 MiB (draft: 0.000 MiB) 15.42.305.763 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.806 15.42.305.770 I srv update: - cache state: 7 prompts, 4649.503 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 15.42.305.771 I srv update: - prompt 0x561d97ec6750: 4194 tokens, checkpoints: 0, 1015.784 MiB 15.42.305.772 I srv update: - prompt 0x561d90faec70: 735 tokens, checkpoints: 0, 178.018 MiB 15.42.305.772 I srv update: - prompt 0x561d97e885f0: 689 tokens, checkpoints: 0, 166.877 MiB 15.42.305.773 I srv update: - prompt 0x561d97e68780: 4188 tokens, checkpoints: 0, 1014.331 MiB 15.42.305.773 I srv update: - prompt 0x561d982cf8c0: 4201 tokens, checkpoints: 0, 1017.479 MiB 15.42.305.774 I srv update: - prompt 0x561d90bf8cd0: 1004 tokens, checkpoints: 0, 243.169 MiB 15.42.305.774 I srv update: - prompt 0x561d97edb570: 4186 tokens, checkpoints: 0, 1013.846 MiB 15.42.305.775 I srv get_availabl: prompt cache update took 621.75 ms 15.42.307.414 I slot launch_slot_: id 3 | task 18541 | processing task, is_child = 0 15.42.307.417 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 15.42.307.418 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 15.42.307.437 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 15.42.307.437 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 15.42.307.447 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 15.42.307.447 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 15.47.408.031 I slot print_timing: id 3 | task 18541 | n_decoded = 100, tg = 22.12 t/s, tg_3s = 22.12 t/s 15.50.436.397 I slot print_timing: id 3 | task 18541 | n_decoded = 167, tg = 22.12 t/s, tg_3s = 22.12 t/s 15.53.476.153 I slot print_timing: id 3 | task 18541 | n_decoded = 235, tg = 22.19 t/s, tg_3s = 22.37 t/s 15.56.481.036 I slot print_timing: id 3 | task 18541 | n_decoded = 302, tg = 22.22 t/s, tg_3s = 22.30 t/s 15.59.485.516 I slot print_timing: id 3 | task 18541 | n_decoded = 369, tg = 22.23 t/s, tg_3s = 22.30 t/s 16.02.528.840 I slot print_timing: id 3 | task 18541 | n_decoded = 437, tg = 22.25 t/s, tg_3s = 22.34 t/s 16.05.571.304 I slot print_timing: id 3 | task 18541 | n_decoded = 505, tg = 22.26 t/s, tg_3s = 22.35 t/s 16.08.571.681 I slot print_timing: id 3 | task 18541 | n_decoded = 572, tg = 22.27 t/s, tg_3s = 22.33 t/s 16.11.579.150 I slot print_timing: id 3 | task 18541 | n_decoded = 639, tg = 22.27 t/s, tg_3s = 22.28 t/s 16.14.606.529 I slot print_timing: id 3 | task 18541 | n_decoded = 706, tg = 22.26 t/s, tg_3s = 22.13 t/s 16.17.649.090 I slot print_timing: id 3 | task 18541 | n_decoded = 773, tg = 22.24 t/s, tg_3s = 22.02 t/s 16.20.663.721 I slot print_timing: id 3 | task 18541 | n_decoded = 838, tg = 22.18 t/s, tg_3s = 21.56 t/s 16.23.678.974 I slot print_timing: id 3 | task 18541 | n_decoded = 903, tg = 22.14 t/s, tg_3s = 21.56 t/s 16.26.723.407 I slot print_timing: id 3 | task 18541 | n_decoded = 969, tg = 22.11 t/s, tg_3s = 21.68 t/s 16.29.766.333 I slot print_timing: id 3 | task 18541 | n_decoded = 1035, tg = 22.08 t/s, tg_3s = 21.69 t/s 16.32.782.369 I slot print_timing: id 3 | task 18541 | n_decoded = 1100, tg = 22.05 t/s, tg_3s = 21.55 t/s 16.35.812.751 I slot print_timing: id 3 | task 18541 | n_decoded = 1166, tg = 22.03 t/s, tg_3s = 21.78 t/s 16.38.826.203 I slot print_timing: id 3 | task 18541 | n_decoded = 1231, tg = 22.01 t/s, tg_3s = 21.57 t/s 16.41.850.070 I slot print_timing: id 3 | task 18541 | n_decoded = 1297, tg = 22.00 t/s, tg_3s = 21.83 t/s 16.44.896.351 I slot print_timing: id 3 | task 18541 | n_decoded = 1363, tg = 21.98 t/s, tg_3s = 21.67 t/s 16.47.899.449 I slot print_timing: id 3 | task 18541 | n_decoded = 1428, tg = 21.97 t/s, tg_3s = 21.64 t/s 16.50.924.758 I slot print_timing: id 3 | task 18541 | n_decoded = 1494, tg = 21.96 t/s, tg_3s = 21.82 t/s 16.53.933.031 I slot print_timing: id 3 | task 18541 | n_decoded = 1560, tg = 21.96 t/s, tg_3s = 21.94 t/s 16.56.945.978 I slot print_timing: id 3 | task 18541 | n_decoded = 1626, tg = 21.96 t/s, tg_3s = 21.91 t/s 16.59.975.164 I slot print_timing: id 3 | task 18541 | n_decoded = 1692, tg = 21.95 t/s, tg_3s = 21.79 t/s 17.02.998.294 I slot print_timing: id 3 | task 18541 | n_decoded = 1754, tg = 21.89 t/s, tg_3s = 20.51 t/s 17.06.020.634 I slot print_timing: id 3 | task 18541 | n_decoded = 1817, tg = 21.86 t/s, tg_3s = 20.84 t/s 17.09.034.688 I slot print_timing: id 3 | task 18541 | n_decoded = 1880, tg = 21.82 t/s, tg_3s = 20.90 t/s 17.12.050.460 I slot print_timing: id 3 | task 18541 | n_decoded = 1943, tg = 21.79 t/s, tg_3s = 20.89 t/s 17.15.090.367 I slot print_timing: id 3 | task 18541 | n_decoded = 2006, tg = 21.76 t/s, tg_3s = 20.72 t/s 17.18.131.515 I slot print_timing: id 3 | task 18541 | n_decoded = 2069, tg = 21.72 t/s, tg_3s = 20.72 t/s 17.21.167.416 I slot print_timing: id 3 | task 18541 | n_decoded = 2132, tg = 21.69 t/s, tg_3s = 20.75 t/s 17.24.204.745 I slot print_timing: id 3 | task 18541 | n_decoded = 2195, tg = 21.66 t/s, tg_3s = 20.74 t/s 17.27.240.841 I slot print_timing: id 3 | task 18541 | n_decoded = 2257, tg = 21.63 t/s, tg_3s = 20.42 t/s 17.30.259.127 I slot print_timing: id 3 | task 18541 | n_decoded = 2319, tg = 21.60 t/s, tg_3s = 20.54 t/s 17.33.283.797 I slot print_timing: id 3 | task 18541 | n_decoded = 2381, tg = 21.57 t/s, tg_3s = 20.50 t/s 17.36.285.998 I slot print_timing: id 3 | task 18541 | n_decoded = 2442, tg = 21.53 t/s, tg_3s = 20.32 t/s 17.39.325.289 I slot print_timing: id 3 | task 18541 | n_decoded = 2504, tg = 21.51 t/s, tg_3s = 20.40 t/s 17.42.369.206 I slot print_timing: id 3 | task 18541 | n_decoded = 2566, tg = 21.48 t/s, tg_3s = 20.37 t/s 17.45.408.369 I slot print_timing: id 3 | task 18541 | n_decoded = 2628, tg = 21.45 t/s, tg_3s = 20.40 t/s 17.48.449.401 I slot print_timing: id 3 | task 18541 | n_decoded = 2690, tg = 21.42 t/s, tg_3s = 20.39 t/s 17.51.462.169 I slot print_timing: id 3 | task 18541 | n_decoded = 2751, tg = 21.40 t/s, tg_3s = 20.25 t/s 17.54.490.228 I slot print_timing: id 3 | task 18541 | n_decoded = 2812, tg = 21.37 t/s, tg_3s = 20.14 t/s 17.57.518.144 I slot print_timing: id 3 | task 18541 | n_decoded = 2873, tg = 21.34 t/s, tg_3s = 20.15 t/s 18.00.548.188 I slot print_timing: id 3 | task 18541 | n_decoded = 2934, tg = 21.31 t/s, tg_3s = 20.13 t/s 18.03.588.584 I slot print_timing: id 3 | task 18541 | n_decoded = 2995, tg = 21.29 t/s, tg_3s = 20.06 t/s 18.06.611.392 I slot print_timing: id 3 | task 18541 | n_decoded = 3055, tg = 21.26 t/s, tg_3s = 19.85 t/s 18.09.620.627 I slot print_timing: id 3 | task 18541 | n_decoded = 3115, tg = 21.23 t/s, tg_3s = 19.94 t/s 18.12.630.693 I slot print_timing: id 3 | task 18541 | n_decoded = 3175, tg = 21.20 t/s, tg_3s = 19.93 t/s 18.15.637.383 I slot print_timing: id 3 | task 18541 | n_decoded = 3235, tg = 21.18 t/s, tg_3s = 19.96 t/s 18.18.652.802 I slot print_timing: id 3 | task 18541 | n_decoded = 3295, tg = 21.15 t/s, tg_3s = 19.90 t/s 18.21.658.482 I slot print_timing: id 3 | task 18541 | n_decoded = 3355, tg = 21.13 t/s, tg_3s = 19.96 t/s 18.24.664.664 I slot print_timing: id 3 | task 18541 | n_decoded = 3415, tg = 21.11 t/s, tg_3s = 19.96 t/s 18.27.679.000 I slot print_timing: id 3 | task 18541 | n_decoded = 3475, tg = 21.09 t/s, tg_3s = 19.90 t/s 18.30.728.222 I slot print_timing: id 3 | task 18541 | n_decoded = 3535, tg = 21.06 t/s, tg_3s = 19.68 t/s 18.33.762.693 I slot print_timing: id 3 | task 18541 | n_decoded = 3595, tg = 21.04 t/s, tg_3s = 19.77 t/s 18.36.803.401 I slot print_timing: id 3 | task 18541 | n_decoded = 3655, tg = 21.02 t/s, tg_3s = 19.73 t/s 18.39.841.212 I slot print_timing: id 3 | task 18541 | n_decoded = 3715, tg = 20.99 t/s, tg_3s = 19.75 t/s 18.42.855.092 I slot print_timing: id 3 | task 18541 | n_decoded = 3774, tg = 20.97 t/s, tg_3s = 19.58 t/s 18.45.861.010 I slot print_timing: id 3 | task 18541 | n_decoded = 3833, tg = 20.95 t/s, tg_3s = 19.63 t/s 18.48.877.100 I slot print_timing: id 3 | task 18541 | n_decoded = 3892, tg = 20.93 t/s, tg_3s = 19.56 t/s 18.51.882.092 I slot print_timing: id 3 | task 18541 | n_decoded = 3951, tg = 20.91 t/s, tg_3s = 19.63 t/s 18.54.915.969 I slot print_timing: id 3 | task 18541 | n_decoded = 4010, tg = 20.88 t/s, tg_3s = 19.45 t/s 18.57.958.802 I slot print_timing: id 3 | task 18541 | n_decoded = 4068, tg = 20.85 t/s, tg_3s = 19.06 t/s 18.59.409.024 I slot print_timing: id 3 | task 18541 | prompt eval time = 580.43 ms / 18 tokens ( 32.25 ms per token, 31.01 tokens per second) 18.59.409.028 I slot print_timing: id 3 | task 18541 | eval time = 196521.12 ms / 4096 tokens ( 47.98 ms per token, 20.84 tokens per second) 18.59.409.029 I slot print_timing: id 3 | task 18541 | total time = 197101.55 ms / 4114 tokens 18.59.409.030 I slot print_timing: id 3 | task 18541 | graphs reused = 22527 18.59.409.057 I slot release: id 3 | task 18541 | stop processing: n_tokens = 4188, truncated = 0 18.59.409.082 I srv update_slots: all slots are idle 18.59.410.505 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 18.59.430.380 I srv operator(): Chat format: peg-native 18.59.436.528 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.617 (> 0.100 thold), f_keep = 0.018 18.59.436.531 I srv get_availabl: updating prompt cache 18.59.436.709 W srv prompt_save: - saving prompt with length 4188, total state size = 1014.331 MiB (draft: 0.000 MiB) 19.00.096.843 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.617 19.00.096.851 I srv update: - cache state: 8 prompts, 5663.834 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 19.00.096.852 I srv update: - prompt 0x561d97ec6750: 4194 tokens, checkpoints: 0, 1015.784 MiB 19.00.096.853 I srv update: - prompt 0x561d90faec70: 735 tokens, checkpoints: 0, 178.018 MiB 19.00.096.853 I srv update: - prompt 0x561d97e885f0: 689 tokens, checkpoints: 0, 166.877 MiB 19.00.096.854 I srv update: - prompt 0x561d97e68780: 4188 tokens, checkpoints: 0, 1014.331 MiB 19.00.096.855 I srv update: - prompt 0x561d982cf8c0: 4201 tokens, checkpoints: 0, 1017.479 MiB 19.00.096.855 I srv update: - prompt 0x561d90bf8cd0: 1004 tokens, checkpoints: 0, 243.169 MiB 19.00.096.856 I srv update: - prompt 0x561d97edb570: 4186 tokens, checkpoints: 0, 1013.846 MiB 19.00.096.856 I srv update: - prompt 0x561d983dc210: 4188 tokens, checkpoints: 0, 1014.331 MiB 19.00.096.857 I srv get_availabl: prompt cache update took 660.33 ms 19.00.098.641 I slot launch_slot_: id 3 | task 22638 | processing task, is_child = 0 19.00.098.643 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 19.00.098.644 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 19.00.098.663 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 19.00.098.663 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 19.00.098.674 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 19.00.098.674 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 19.05.862.299 I slot print_timing: id 3 | task 22638 | n_decoded = 100, tg = 22.17 t/s, tg_3s = 22.17 t/s 19.08.868.455 I slot print_timing: id 3 | task 22638 | n_decoded = 167, tg = 22.22 t/s, tg_3s = 22.29 t/s 19.11.906.504 I slot print_timing: id 3 | task 22638 | n_decoded = 234, tg = 22.17 t/s, tg_3s = 22.05 t/s 19.14.938.081 I slot print_timing: id 3 | task 22638 | n_decoded = 302, tg = 22.23 t/s, tg_3s = 22.43 t/s 19.17.980.705 I slot print_timing: id 3 | task 22638 | n_decoded = 370, tg = 22.25 t/s, tg_3s = 22.35 t/s 19.20.989.204 I slot print_timing: id 3 | task 22638 | n_decoded = 436, tg = 22.20 t/s, tg_3s = 21.94 t/s 19.24.015.496 I slot print_timing: id 3 | task 22638 | n_decoded = 503, tg = 22.19 t/s, tg_3s = 22.14 t/s 19.27.019.318 I slot print_timing: id 3 | task 22638 | n_decoded = 570, tg = 22.21 t/s, tg_3s = 22.30 t/s 19.30.019.336 I slot print_timing: id 3 | task 22638 | n_decoded = 637, tg = 22.22 t/s, tg_3s = 22.33 t/s 19.33.060.520 I slot print_timing: id 3 | task 22638 | n_decoded = 704, tg = 22.20 t/s, tg_3s = 22.03 t/s 19.36.073.182 I slot print_timing: id 3 | task 22638 | n_decoded = 770, tg = 22.18 t/s, tg_3s = 21.91 t/s 19.39.092.649 I slot print_timing: id 3 | task 22638 | n_decoded = 836, tg = 22.15 t/s, tg_3s = 21.86 t/s 19.42.137.931 I slot print_timing: id 3 | task 22638 | n_decoded = 903, tg = 22.14 t/s, tg_3s = 22.00 t/s 19.45.149.772 I slot print_timing: id 3 | task 22638 | n_decoded = 969, tg = 22.12 t/s, tg_3s = 21.91 t/s 19.48.152.143 I slot print_timing: id 3 | task 22638 | n_decoded = 1035, tg = 22.12 t/s, tg_3s = 21.98 t/s 19.51.190.216 I slot print_timing: id 3 | task 22638 | n_decoded = 1101, tg = 22.09 t/s, tg_3s = 21.72 t/s 19.54.203.740 I slot print_timing: id 3 | task 22638 | n_decoded = 1166, tg = 22.06 t/s, tg_3s = 21.57 t/s 19.57.216.342 I slot print_timing: id 3 | task 22638 | n_decoded = 1232, tg = 22.05 t/s, tg_3s = 21.91 t/s 20.00.234.039 I slot print_timing: id 3 | task 22638 | n_decoded = 1298, tg = 22.04 t/s, tg_3s = 21.87 t/s 20.03.262.867 I slot print_timing: id 3 | task 22638 | n_decoded = 1364, tg = 22.03 t/s, tg_3s = 21.79 t/s 20.06.282.241 I slot print_timing: id 3 | task 22638 | n_decoded = 1429, tg = 22.01 t/s, tg_3s = 21.53 t/s 20.09.318.371 I slot print_timing: id 3 | task 22638 | n_decoded = 1495, tg = 22.00 t/s, tg_3s = 21.74 t/s 20.12.333.592 I slot print_timing: id 3 | task 22638 | n_decoded = 1561, tg = 21.99 t/s, tg_3s = 21.89 t/s 20.15.341.772 I slot print_timing: id 3 | task 22638 | n_decoded = 1627, tg = 21.99 t/s, tg_3s = 21.94 t/s 20.18.361.593 I slot print_timing: id 3 | task 22638 | n_decoded = 1692, tg = 21.97 t/s, tg_3s = 21.52 t/s 20.21.372.288 I slot print_timing: id 3 | task 22638 | n_decoded = 1755, tg = 21.93 t/s, tg_3s = 20.93 t/s 20.24.381.431 I slot print_timing: id 3 | task 22638 | n_decoded = 1818, tg = 21.90 t/s, tg_3s = 20.94 t/s 20.27.417.556 I slot print_timing: id 3 | task 22638 | n_decoded = 1881, tg = 21.86 t/s, tg_3s = 20.75 t/s 20.30.449.710 I slot print_timing: id 3 | task 22638 | n_decoded = 1944, tg = 21.82 t/s, tg_3s = 20.78 t/s 20.33.486.112 I slot print_timing: id 3 | task 22638 | n_decoded = 2007, tg = 21.78 t/s, tg_3s = 20.75 t/s 20.36.521.984 I slot print_timing: id 3 | task 22638 | n_decoded = 2070, tg = 21.75 t/s, tg_3s = 20.75 t/s 20.39.546.166 I slot print_timing: id 3 | task 22638 | n_decoded = 2132, tg = 21.71 t/s, tg_3s = 20.50 t/s 20.42.557.883 I slot print_timing: id 3 | task 22638 | n_decoded = 2194, tg = 21.68 t/s, tg_3s = 20.59 t/s 20.45.595.046 I slot print_timing: id 3 | task 22638 | n_decoded = 2256, tg = 21.64 t/s, tg_3s = 20.41 t/s 20.48.625.972 I slot print_timing: id 3 | task 22638 | n_decoded = 2318, tg = 21.61 t/s, tg_3s = 20.46 t/s 20.51.643.973 I slot print_timing: id 3 | task 22638 | n_decoded = 2380, tg = 21.58 t/s, tg_3s = 20.54 t/s 20.54.661.904 I slot print_timing: id 3 | task 22638 | n_decoded = 2442, tg = 21.55 t/s, tg_3s = 20.54 t/s 20.57.675.585 I slot print_timing: id 3 | task 22638 | n_decoded = 2503, tg = 21.52 t/s, tg_3s = 20.24 t/s 21.00.719.361 I slot print_timing: id 3 | task 22638 | n_decoded = 2565, tg = 21.49 t/s, tg_3s = 20.37 t/s 21.03.745.327 I slot print_timing: id 3 | task 22638 | n_decoded = 2626, tg = 21.46 t/s, tg_3s = 20.16 t/s 21.06.791.499 I slot print_timing: id 3 | task 22638 | n_decoded = 2688, tg = 21.43 t/s, tg_3s = 20.35 t/s 21.09.823.626 I slot print_timing: id 3 | task 22638 | n_decoded = 2749, tg = 21.40 t/s, tg_3s = 20.12 t/s 21.12.854.340 I slot print_timing: id 3 | task 22638 | n_decoded = 2810, tg = 21.37 t/s, tg_3s = 20.13 t/s 21.15.878.240 I slot print_timing: id 3 | task 22638 | n_decoded = 2871, tg = 21.34 t/s, tg_3s = 20.17 t/s 21.18.894.032 I slot print_timing: id 3 | task 22638 | n_decoded = 2931, tg = 21.31 t/s, tg_3s = 19.90 t/s 21.21.911.506 I slot print_timing: id 3 | task 22638 | n_decoded = 2991, tg = 21.28 t/s, tg_3s = 19.88 t/s 21.24.930.795 I slot print_timing: id 3 | task 22638 | n_decoded = 3051, tg = 21.25 t/s, tg_3s = 19.87 t/s 21.27.950.744 I slot print_timing: id 3 | task 22638 | n_decoded = 3111, tg = 21.22 t/s, tg_3s = 19.87 t/s 21.30.953.905 I slot print_timing: id 3 | task 22638 | n_decoded = 3171, tg = 21.20 t/s, tg_3s = 19.98 t/s 21.33.972.791 I slot print_timing: id 3 | task 22638 | n_decoded = 3231, tg = 21.17 t/s, tg_3s = 19.87 t/s 21.36.979.013 I slot print_timing: id 3 | task 22638 | n_decoded = 3291, tg = 21.15 t/s, tg_3s = 19.96 t/s 21.39.984.673 I slot print_timing: id 3 | task 22638 | n_decoded = 3351, tg = 21.12 t/s, tg_3s = 19.96 t/s 21.43.021.290 I slot print_timing: id 3 | task 22638 | n_decoded = 3411, tg = 21.10 t/s, tg_3s = 19.76 t/s 21.46.050.118 I slot print_timing: id 3 | task 22638 | n_decoded = 3471, tg = 21.07 t/s, tg_3s = 19.81 t/s 21.49.068.599 I slot print_timing: id 3 | task 22638 | n_decoded = 3530, tg = 21.05 t/s, tg_3s = 19.55 t/s 21.52.070.323 I slot print_timing: id 3 | task 22638 | n_decoded = 3589, tg = 21.02 t/s, tg_3s = 19.66 t/s 21.55.116.702 I slot print_timing: id 3 | task 22638 | n_decoded = 3649, tg = 21.00 t/s, tg_3s = 19.70 t/s 21.58.155.495 I slot print_timing: id 3 | task 22638 | n_decoded = 3708, tg = 20.97 t/s, tg_3s = 19.42 t/s 22.01.165.520 I slot print_timing: id 3 | task 22638 | n_decoded = 3767, tg = 20.95 t/s, tg_3s = 19.60 t/s 22.04.215.451 I slot print_timing: id 3 | task 22638 | n_decoded = 3827, tg = 20.93 t/s, tg_3s = 19.67 t/s 22.07.228.259 I slot print_timing: id 3 | task 22638 | n_decoded = 3886, tg = 20.91 t/s, tg_3s = 19.58 t/s 22.10.238.855 I slot print_timing: id 3 | task 22638 | n_decoded = 3945, tg = 20.89 t/s, tg_3s = 19.60 t/s 22.13.247.559 I slot print_timing: id 3 | task 22638 | n_decoded = 4003, tg = 20.86 t/s, tg_3s = 19.28 t/s 22.16.252.824 I slot print_timing: id 3 | task 22638 | n_decoded = 4061, tg = 20.84 t/s, tg_3s = 19.30 t/s 22.18.058.791 I slot print_timing: id 3 | task 22638 | prompt eval time = 1253.45 ms / 46 tokens ( 27.25 ms per token, 36.70 tokens per second) 22.18.058.796 I slot print_timing: id 3 | task 22638 | eval time = 196706.64 ms / 4096 tokens ( 48.02 ms per token, 20.82 tokens per second) 22.18.058.797 I slot print_timing: id 3 | task 22638 | total time = 197960.09 ms / 4142 tokens 22.18.058.798 I slot print_timing: id 3 | task 22638 | graphs reused = 26605 22.18.058.835 I slot release: id 3 | task 22638 | stop processing: n_tokens = 4215, truncated = 0 22.18.058.866 I srv update_slots: all slots are idle 22.18.060.063 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 22.18.078.882 I srv operator(): Chat format: peg-native 22.18.084.327 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.560 (> 0.100 thold), f_keep = 0.018 22.18.084.329 I srv get_availabl: updating prompt cache 22.18.084.455 W srv prompt_save: - saving prompt with length 4215, total state size = 1020.870 MiB (draft: 0.000 MiB) 22.18.702.695 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.560 22.18.702.703 I srv update: - cache state: 9 prompts, 6684.704 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 22.18.702.704 I srv update: - prompt 0x561d97ec6750: 4194 tokens, checkpoints: 0, 1015.784 MiB 22.18.702.705 I srv update: - prompt 0x561d90faec70: 735 tokens, checkpoints: 0, 178.018 MiB 22.18.702.705 I srv update: - prompt 0x561d97e885f0: 689 tokens, checkpoints: 0, 166.877 MiB 22.18.702.706 I srv update: - prompt 0x561d97e68780: 4188 tokens, checkpoints: 0, 1014.331 MiB 22.18.702.706 I srv update: - prompt 0x561d982cf8c0: 4201 tokens, checkpoints: 0, 1017.479 MiB 22.18.702.707 I srv update: - prompt 0x561d90bf8cd0: 1004 tokens, checkpoints: 0, 243.169 MiB 22.18.702.707 I srv update: - prompt 0x561d97edb570: 4186 tokens, checkpoints: 0, 1013.846 MiB 22.18.702.708 I srv update: - prompt 0x561d983dc210: 4188 tokens, checkpoints: 0, 1014.331 MiB 22.18.702.708 I srv update: - prompt 0x561d90d122a0: 4215 tokens, checkpoints: 0, 1020.870 MiB 22.18.702.709 I srv get_availabl: prompt cache update took 618.38 ms 22.18.704.343 I slot launch_slot_: id 3 | task 26735 | processing task, is_child = 0 22.18.704.345 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 22.18.704.345 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 22.18.704.364 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 22.18.704.365 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 22.18.704.374 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 22.18.704.374 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 22.24.781.673 I slot print_timing: id 3 | task 26735 | n_decoded = 100, tg = 22.48 t/s, tg_3s = 22.48 t/s 22.27.793.008 I slot print_timing: id 3 | task 26735 | n_decoded = 167, tg = 22.39 t/s, tg_3s = 22.25 t/s 22.30.833.341 I slot print_timing: id 3 | task 26735 | n_decoded = 235, tg = 22.38 t/s, tg_3s = 22.37 t/s 22.33.854.845 I slot print_timing: id 3 | task 26735 | n_decoded = 302, tg = 22.34 t/s, tg_3s = 22.17 t/s 22.36.887.897 I slot print_timing: id 3 | task 26735 | n_decoded = 370, tg = 22.35 t/s, tg_3s = 22.42 t/s 22.39.932.193 I slot print_timing: id 3 | task 26735 | n_decoded = 438, tg = 22.35 t/s, tg_3s = 22.34 t/s 22.42.975.819 I slot print_timing: id 3 | task 26735 | n_decoded = 506, tg = 22.35 t/s, tg_3s = 22.34 t/s 22.45.997.848 I slot print_timing: id 3 | task 26735 | n_decoded = 573, tg = 22.33 t/s, tg_3s = 22.17 t/s 22.49.005.556 I slot print_timing: id 3 | task 26735 | n_decoded = 640, tg = 22.32 t/s, tg_3s = 22.28 t/s 22.52.008.026 I slot print_timing: id 3 | task 26735 | n_decoded = 705, tg = 22.26 t/s, tg_3s = 21.65 t/s 22.55.051.710 I slot print_timing: id 3 | task 26735 | n_decoded = 771, tg = 22.21 t/s, tg_3s = 21.68 t/s 22.58.054.132 I slot print_timing: id 3 | task 26735 | n_decoded = 837, tg = 22.19 t/s, tg_3s = 21.98 t/s 23.01.062.840 I slot print_timing: id 3 | task 26735 | n_decoded = 903, tg = 22.17 t/s, tg_3s = 21.94 t/s 23.04.067.045 I slot print_timing: id 3 | task 26735 | n_decoded = 969, tg = 22.16 t/s, tg_3s = 21.97 t/s 23.07.101.212 I slot print_timing: id 3 | task 26735 | n_decoded = 1035, tg = 22.13 t/s, tg_3s = 21.75 t/s 23.10.142.614 I slot print_timing: id 3 | task 26735 | n_decoded = 1101, tg = 22.10 t/s, tg_3s = 21.70 t/s 23.13.164.425 I slot print_timing: id 3 | task 26735 | n_decoded = 1167, tg = 22.09 t/s, tg_3s = 21.84 t/s 23.16.171.961 I slot print_timing: id 3 | task 26735 | n_decoded = 1233, tg = 22.08 t/s, tg_3s = 21.94 t/s 23.19.186.296 I slot print_timing: id 3 | task 26735 | n_decoded = 1298, tg = 22.06 t/s, tg_3s = 21.56 t/s 23.22.222.782 I slot print_timing: id 3 | task 26735 | n_decoded = 1363, tg = 22.02 t/s, tg_3s = 21.41 t/s 23.25.262.626 I slot print_timing: id 3 | task 26735 | n_decoded = 1429, tg = 22.01 t/s, tg_3s = 21.71 t/s 23.28.283.657 I slot print_timing: id 3 | task 26735 | n_decoded = 1495, tg = 22.00 t/s, tg_3s = 21.85 t/s 23.31.317.723 I slot print_timing: id 3 | task 26735 | n_decoded = 1561, tg = 21.99 t/s, tg_3s = 21.75 t/s 23.34.356.414 I slot print_timing: id 3 | task 26735 | n_decoded = 1627, tg = 21.98 t/s, tg_3s = 21.72 t/s 23.37.402.966 I slot print_timing: id 3 | task 26735 | n_decoded = 1692, tg = 21.95 t/s, tg_3s = 21.34 t/s 23.40.409.634 I slot print_timing: id 3 | task 26735 | n_decoded = 1755, tg = 21.92 t/s, tg_3s = 20.95 t/s 23.43.422.531 I slot print_timing: id 3 | task 26735 | n_decoded = 1818, tg = 21.88 t/s, tg_3s = 20.91 t/s 23.46.423.507 I slot print_timing: id 3 | task 26735 | n_decoded = 1880, tg = 21.84 t/s, tg_3s = 20.66 t/s 23.49.436.235 I slot print_timing: id 3 | task 26735 | n_decoded = 1942, tg = 21.80 t/s, tg_3s = 20.58 t/s 23.52.437.258 I slot print_timing: id 3 | task 26735 | n_decoded = 2004, tg = 21.76 t/s, tg_3s = 20.66 t/s 23.55.480.398 I slot print_timing: id 3 | task 26735 | n_decoded = 2067, tg = 21.72 t/s, tg_3s = 20.70 t/s 23.58.498.076 I slot print_timing: id 3 | task 26735 | n_decoded = 2129, tg = 21.69 t/s, tg_3s = 20.55 t/s 24.01.528.011 I slot print_timing: id 3 | task 26735 | n_decoded = 2191, tg = 21.65 t/s, tg_3s = 20.46 t/s 24.04.550.583 I slot print_timing: id 3 | task 26735 | n_decoded = 2252, tg = 21.61 t/s, tg_3s = 20.18 t/s 24.07.556.996 I slot print_timing: id 3 | task 26735 | n_decoded = 2313, tg = 21.57 t/s, tg_3s = 20.29 t/s 24.10.580.811 I slot print_timing: id 3 | task 26735 | n_decoded = 2375, tg = 21.54 t/s, tg_3s = 20.50 t/s 24.13.613.712 I slot print_timing: id 3 | task 26735 | n_decoded = 2437, tg = 21.51 t/s, tg_3s = 20.44 t/s 24.16.628.730 I slot print_timing: id 3 | task 26735 | n_decoded = 2498, tg = 21.48 t/s, tg_3s = 20.23 t/s 24.19.673.341 I slot print_timing: id 3 | task 26735 | n_decoded = 2560, tg = 21.45 t/s, tg_3s = 20.36 t/s 24.22.679.130 I slot print_timing: id 3 | task 26735 | n_decoded = 2620, tg = 21.41 t/s, tg_3s = 19.96 t/s 24.25.685.644 I slot print_timing: id 3 | task 26735 | n_decoded = 2681, tg = 21.39 t/s, tg_3s = 20.29 t/s 24.28.721.884 I slot print_timing: id 3 | task 26735 | n_decoded = 2742, tg = 21.36 t/s, tg_3s = 20.09 t/s 24.31.763.925 I slot print_timing: id 3 | task 26735 | n_decoded = 2803, tg = 21.33 t/s, tg_3s = 20.05 t/s 24.34.769.147 I slot print_timing: id 3 | task 26735 | n_decoded = 2863, tg = 21.30 t/s, tg_3s = 19.97 t/s 24.37.802.875 I slot print_timing: id 3 | task 26735 | n_decoded = 2924, tg = 21.27 t/s, tg_3s = 20.11 t/s 24.40.820.649 I slot print_timing: id 3 | task 26735 | n_decoded = 2984, tg = 21.24 t/s, tg_3s = 19.88 t/s 24.43.867.778 I slot print_timing: id 3 | task 26735 | n_decoded = 3044, tg = 21.21 t/s, tg_3s = 19.69 t/s 24.46.869.879 I slot print_timing: id 3 | task 26735 | n_decoded = 3104, tg = 21.18 t/s, tg_3s = 19.99 t/s 24.49.870.556 I slot print_timing: id 3 | task 26735 | n_decoded = 3164, tg = 21.16 t/s, tg_3s = 20.00 t/s 24.52.882.917 I slot print_timing: id 3 | task 26735 | n_decoded = 3224, tg = 21.13 t/s, tg_3s = 19.92 t/s 24.55.887.950 I slot print_timing: id 3 | task 26735 | n_decoded = 3284, tg = 21.11 t/s, tg_3s = 19.97 t/s 24.58.894.864 I slot print_timing: id 3 | task 26735 | n_decoded = 3344, tg = 21.09 t/s, tg_3s = 19.95 t/s 25.01.916.792 I slot print_timing: id 3 | task 26735 | n_decoded = 3404, tg = 21.07 t/s, tg_3s = 19.85 t/s 25.04.952.257 I slot print_timing: id 3 | task 26735 | n_decoded = 3464, tg = 21.04 t/s, tg_3s = 19.77 t/s 25.07.998.978 I slot print_timing: id 3 | task 26735 | n_decoded = 3524, tg = 21.02 t/s, tg_3s = 19.69 t/s 25.11.029.751 I slot print_timing: id 3 | task 26735 | n_decoded = 3583, tg = 20.99 t/s, tg_3s = 19.47 t/s 25.14.071.220 I slot print_timing: id 3 | task 26735 | n_decoded = 3643, tg = 20.97 t/s, tg_3s = 19.73 t/s 25.17.120.014 I slot print_timing: id 3 | task 26735 | n_decoded = 3703, tg = 20.95 t/s, tg_3s = 19.68 t/s 25.20.169.172 I slot print_timing: id 3 | task 26735 | n_decoded = 3762, tg = 20.92 t/s, tg_3s = 19.35 t/s 25.23.170.403 I slot print_timing: id 3 | task 26735 | n_decoded = 3821, tg = 20.90 t/s, tg_3s = 19.66 t/s 25.26.179.356 I slot print_timing: id 3 | task 26735 | n_decoded = 3880, tg = 20.88 t/s, tg_3s = 19.61 t/s 25.29.189.402 I slot print_timing: id 3 | task 26735 | n_decoded = 3939, tg = 20.86 t/s, tg_3s = 19.60 t/s 25.32.224.210 I slot print_timing: id 3 | task 26735 | n_decoded = 3998, tg = 20.83 t/s, tg_3s = 19.44 t/s 25.35.252.762 I slot print_timing: id 3 | task 26735 | n_decoded = 4057, tg = 20.81 t/s, tg_3s = 19.48 t/s 25.37.283.798 I slot print_timing: id 3 | task 26735 | prompt eval time = 1629.70 ms / 59 tokens ( 27.62 ms per token, 36.20 tokens per second) 25.37.283.803 I slot print_timing: id 3 | task 26735 | eval time = 196949.69 ms / 4096 tokens ( 48.08 ms per token, 20.80 tokens per second) 25.37.283.804 I slot print_timing: id 3 | task 26735 | total time = 198579.39 ms / 4155 tokens 25.37.283.804 I slot print_timing: id 3 | task 26735 | graphs reused = 30683 25.37.283.843 I slot release: id 3 | task 26735 | stop processing: n_tokens = 4229, truncated = 0 25.37.283.875 I srv update_slots: all slots are idle 25.37.285.088 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 25.37.304.319 I srv operator(): Chat format: peg-native 25.37.310.429 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.649 (> 0.100 thold), f_keep = 0.017 25.37.310.431 I srv get_availabl: updating prompt cache 25.37.310.604 W srv prompt_save: - saving prompt with length 4229, total state size = 1024.261 MiB (draft: 0.000 MiB) 25.37.978.938 I srv load: - looking for better prompt, base f_keep = 0.017, sim = 0.649 25.37.978.947 I srv update: - cache state: 10 prompts, 7708.965 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 25.37.978.948 I srv update: - prompt 0x561d97ec6750: 4194 tokens, checkpoints: 0, 1015.784 MiB 25.37.978.949 I srv update: - prompt 0x561d90faec70: 735 tokens, checkpoints: 0, 178.018 MiB 25.37.978.949 I srv update: - prompt 0x561d97e885f0: 689 tokens, checkpoints: 0, 166.877 MiB 25.37.978.950 I srv update: - prompt 0x561d97e68780: 4188 tokens, checkpoints: 0, 1014.331 MiB 25.37.978.950 I srv update: - prompt 0x561d982cf8c0: 4201 tokens, checkpoints: 0, 1017.479 MiB 25.37.978.951 I srv update: - prompt 0x561d90bf8cd0: 1004 tokens, checkpoints: 0, 243.169 MiB 25.37.978.951 I srv update: - prompt 0x561d97edb570: 4186 tokens, checkpoints: 0, 1013.846 MiB 25.37.978.952 I srv update: - prompt 0x561d983dc210: 4188 tokens, checkpoints: 0, 1014.331 MiB 25.37.978.952 I srv update: - prompt 0x561d90d122a0: 4215 tokens, checkpoints: 0, 1020.870 MiB 25.37.978.953 I srv update: - prompt 0x561d90ada720: 4229 tokens, checkpoints: 0, 1024.261 MiB 25.37.978.954 I srv get_availabl: prompt cache update took 668.52 ms 25.37.980.724 I slot launch_slot_: id 3 | task 30832 | processing task, is_child = 0 25.37.980.726 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 25.37.980.727 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 25.37.980.746 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 25.37.980.746 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 25.37.980.756 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 25.37.980.757 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 25.43.510.878 I slot print_timing: id 3 | task 30832 | n_decoded = 100, tg = 22.46 t/s, tg_3s = 22.46 t/s 25.46.517.542 I slot print_timing: id 3 | task 30832 | n_decoded = 167, tg = 22.39 t/s, tg_3s = 22.28 t/s 25.49.555.330 I slot print_timing: id 3 | task 30832 | n_decoded = 234, tg = 22.29 t/s, tg_3s = 22.06 t/s 25.52.590.398 I slot print_timing: id 3 | task 30832 | n_decoded = 301, tg = 22.24 t/s, tg_3s = 22.08 t/s 25.55.634.135 I slot print_timing: id 3 | task 30832 | n_decoded = 369, tg = 22.26 t/s, tg_3s = 22.34 t/s 25.58.669.979 I slot print_timing: id 3 | task 30832 | n_decoded = 436, tg = 22.23 t/s, tg_3s = 22.07 t/s 26.01.697.857 I slot print_timing: id 3 | task 30832 | n_decoded = 503, tg = 22.22 t/s, tg_3s = 22.13 t/s 26.04.705.761 I slot print_timing: id 3 | task 30832 | n_decoded = 570, tg = 22.22 t/s, tg_3s = 22.27 t/s 26.07.745.717 I slot print_timing: id 3 | task 30832 | n_decoded = 638, tg = 22.24 t/s, tg_3s = 22.37 t/s 26.10.782.144 I slot print_timing: id 3 | task 30832 | n_decoded = 705, tg = 22.22 t/s, tg_3s = 22.07 t/s 26.13.822.670 I slot print_timing: id 3 | task 30832 | n_decoded = 772, tg = 22.21 t/s, tg_3s = 22.04 t/s 26.16.824.231 I slot print_timing: id 3 | task 30832 | n_decoded = 838, tg = 22.19 t/s, tg_3s = 21.99 t/s 26.19.842.897 I slot print_timing: id 3 | task 30832 | n_decoded = 904, tg = 22.17 t/s, tg_3s = 21.86 t/s 26.22.868.578 I slot print_timing: id 3 | task 30832 | n_decoded = 970, tg = 22.14 t/s, tg_3s = 21.81 t/s 26.25.882.506 I slot print_timing: id 3 | task 30832 | n_decoded = 1036, tg = 22.13 t/s, tg_3s = 21.90 t/s 26.28.886.627 I slot print_timing: id 3 | task 30832 | n_decoded = 1102, tg = 22.12 t/s, tg_3s = 21.97 t/s 26.31.893.863 I slot print_timing: id 3 | task 30832 | n_decoded = 1168, tg = 22.11 t/s, tg_3s = 21.95 t/s 26.34.906.403 I slot print_timing: id 3 | task 30832 | n_decoded = 1234, tg = 22.10 t/s, tg_3s = 21.91 t/s 26.37.916.085 I slot print_timing: id 3 | task 30832 | n_decoded = 1300, tg = 22.09 t/s, tg_3s = 21.93 t/s 26.40.924.401 I slot print_timing: id 3 | task 30832 | n_decoded = 1366, tg = 22.08 t/s, tg_3s = 21.94 t/s 26.43.944.378 I slot print_timing: id 3 | task 30832 | n_decoded = 1432, tg = 22.07 t/s, tg_3s = 21.85 t/s 26.46.960.453 I slot print_timing: id 3 | task 30832 | n_decoded = 1498, tg = 22.06 t/s, tg_3s = 21.88 t/s 26.49.979.799 I slot print_timing: id 3 | task 30832 | n_decoded = 1564, tg = 22.05 t/s, tg_3s = 21.86 t/s 26.52.988.958 I slot print_timing: id 3 | task 30832 | n_decoded = 1630, tg = 22.05 t/s, tg_3s = 21.93 t/s 26.56.009.225 I slot print_timing: id 3 | task 30832 | n_decoded = 1695, tg = 22.03 t/s, tg_3s = 21.52 t/s 26.59.024.975 I slot print_timing: id 3 | task 30832 | n_decoded = 1758, tg = 21.98 t/s, tg_3s = 20.89 t/s 27.02.040.596 I slot print_timing: id 3 | task 30832 | n_decoded = 1820, tg = 21.93 t/s, tg_3s = 20.56 t/s 27.05.086.963 I slot print_timing: id 3 | task 30832 | n_decoded = 1883, tg = 21.89 t/s, tg_3s = 20.68 t/s 27.08.116.435 I slot print_timing: id 3 | task 30832 | n_decoded = 1946, tg = 21.85 t/s, tg_3s = 20.80 t/s 27.11.124.786 I slot print_timing: id 3 | task 30832 | n_decoded = 2008, tg = 21.81 t/s, tg_3s = 20.61 t/s 27.14.155.841 I slot print_timing: id 3 | task 30832 | n_decoded = 2071, tg = 21.78 t/s, tg_3s = 20.78 t/s 27.17.189.731 I slot print_timing: id 3 | task 30832 | n_decoded = 2134, tg = 21.75 t/s, tg_3s = 20.77 t/s 27.20.190.017 I slot print_timing: id 3 | task 30832 | n_decoded = 2196, tg = 21.71 t/s, tg_3s = 20.66 t/s 27.23.216.902 I slot print_timing: id 3 | task 30832 | n_decoded = 2258, tg = 21.68 t/s, tg_3s = 20.48 t/s 27.26.243.429 I slot print_timing: id 3 | task 30832 | n_decoded = 2320, tg = 21.64 t/s, tg_3s = 20.49 t/s 27.29.261.194 I slot print_timing: id 3 | task 30832 | n_decoded = 2382, tg = 21.61 t/s, tg_3s = 20.55 t/s 27.32.288.182 I slot print_timing: id 3 | task 30832 | n_decoded = 2444, tg = 21.58 t/s, tg_3s = 20.48 t/s 27.35.293.169 I slot print_timing: id 3 | task 30832 | n_decoded = 2505, tg = 21.55 t/s, tg_3s = 20.30 t/s 27.38.332.495 I slot print_timing: id 3 | task 30832 | n_decoded = 2567, tg = 21.52 t/s, tg_3s = 20.40 t/s 27.41.374.014 I slot print_timing: id 3 | task 30832 | n_decoded = 2629, tg = 21.49 t/s, tg_3s = 20.38 t/s 27.44.393.168 I slot print_timing: id 3 | task 30832 | n_decoded = 2690, tg = 21.46 t/s, tg_3s = 20.20 t/s 27.47.416.407 I slot print_timing: id 3 | task 30832 | n_decoded = 2750, tg = 21.42 t/s, tg_3s = 19.85 t/s 27.50.443.696 I slot print_timing: id 3 | task 30832 | n_decoded = 2811, tg = 21.40 t/s, tg_3s = 20.15 t/s 27.53.472.576 I slot print_timing: id 3 | task 30832 | n_decoded = 2872, tg = 21.37 t/s, tg_3s = 20.14 t/s 27.56.477.525 I slot print_timing: id 3 | task 30832 | n_decoded = 2932, tg = 21.34 t/s, tg_3s = 19.97 t/s 27.59.490.807 I slot print_timing: id 3 | task 30832 | n_decoded = 2992, tg = 21.31 t/s, tg_3s = 19.91 t/s 28.02.492.866 I slot print_timing: id 3 | task 30832 | n_decoded = 3051, tg = 21.27 t/s, tg_3s = 19.65 t/s 28.05.512.061 I slot print_timing: id 3 | task 30832 | n_decoded = 3111, tg = 21.24 t/s, tg_3s = 19.87 t/s 28.08.528.900 I slot print_timing: id 3 | task 30832 | n_decoded = 3171, tg = 21.21 t/s, tg_3s = 19.89 t/s 28.11.547.687 I slot print_timing: id 3 | task 30832 | n_decoded = 3231, tg = 21.19 t/s, tg_3s = 19.88 t/s 28.14.554.791 I slot print_timing: id 3 | task 30832 | n_decoded = 3291, tg = 21.16 t/s, tg_3s = 19.95 t/s 28.17.588.765 I slot print_timing: id 3 | task 30832 | n_decoded = 3351, tg = 21.14 t/s, tg_3s = 19.78 t/s 28.20.595.992 I slot print_timing: id 3 | task 30832 | n_decoded = 3411, tg = 21.12 t/s, tg_3s = 19.95 t/s 28.23.607.950 I slot print_timing: id 3 | task 30832 | n_decoded = 3471, tg = 21.09 t/s, tg_3s = 19.92 t/s 28.26.626.344 I slot print_timing: id 3 | task 30832 | n_decoded = 3530, tg = 21.07 t/s, tg_3s = 19.55 t/s 28.29.668.254 I slot print_timing: id 3 | task 30832 | n_decoded = 3589, tg = 21.04 t/s, tg_3s = 19.40 t/s 28.32.678.185 I slot print_timing: id 3 | task 30832 | n_decoded = 3648, tg = 21.01 t/s, tg_3s = 19.60 t/s 28.35.687.957 I slot print_timing: id 3 | task 30832 | n_decoded = 3707, tg = 20.99 t/s, tg_3s = 19.60 t/s 28.38.699.756 I slot print_timing: id 3 | task 30832 | n_decoded = 3766, tg = 20.96 t/s, tg_3s = 19.59 t/s 28.41.729.184 I slot print_timing: id 3 | task 30832 | n_decoded = 3825, tg = 20.94 t/s, tg_3s = 19.48 t/s 28.44.759.580 I slot print_timing: id 3 | task 30832 | n_decoded = 3884, tg = 20.92 t/s, tg_3s = 19.47 t/s 28.47.808.535 I slot print_timing: id 3 | task 30832 | n_decoded = 3943, tg = 20.89 t/s, tg_3s = 19.35 t/s 28.50.829.966 I slot print_timing: id 3 | task 30832 | n_decoded = 4002, tg = 20.87 t/s, tg_3s = 19.53 t/s 28.53.848.026 I slot print_timing: id 3 | task 30832 | n_decoded = 4061, tg = 20.85 t/s, tg_3s = 19.55 t/s 28.55.639.436 I slot print_timing: id 3 | task 30832 | prompt eval time = 1077.67 ms / 40 tokens ( 26.94 ms per token, 37.12 tokens per second) 28.55.639.440 I slot print_timing: id 3 | task 30832 | eval time = 196580.97 ms / 4096 tokens ( 47.99 ms per token, 20.84 tokens per second) 28.55.639.440 I slot print_timing: id 3 | task 30832 | total time = 197658.65 ms / 4136 tokens 28.55.639.441 I slot print_timing: id 3 | task 30832 | graphs reused = 34761 28.55.639.471 I slot release: id 3 | task 30832 | stop processing: n_tokens = 4209, truncated = 0 28.55.639.498 I srv update_slots: all slots are idle 28.55.640.784 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 28.55.659.956 I srv operator(): Chat format: peg-native 28.55.666.443 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.578 (> 0.100 thold), f_keep = 0.018 28.55.666.445 I srv get_availabl: updating prompt cache 28.55.666.621 W srv prompt_save: - saving prompt with length 4209, total state size = 1019.417 MiB (draft: 0.000 MiB) 28.56.292.470 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.578 28.56.292.480 W srv update: - cache size limit reached, removing oldest entry (size = 1015.784 MiB) 28.56.376.270 I srv update: - cache state: 10 prompts, 7712.598 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 28.56.376.274 I srv update: - prompt 0x561d90faec70: 735 tokens, checkpoints: 0, 178.018 MiB 28.56.376.274 I srv update: - prompt 0x561d97e885f0: 689 tokens, checkpoints: 0, 166.877 MiB 28.56.376.275 I srv update: - prompt 0x561d97e68780: 4188 tokens, checkpoints: 0, 1014.331 MiB 28.56.376.275 I srv update: - prompt 0x561d982cf8c0: 4201 tokens, checkpoints: 0, 1017.479 MiB 28.56.376.276 I srv update: - prompt 0x561d90bf8cd0: 1004 tokens, checkpoints: 0, 243.169 MiB 28.56.376.276 I srv update: - prompt 0x561d97edb570: 4186 tokens, checkpoints: 0, 1013.846 MiB 28.56.376.277 I srv update: - prompt 0x561d983dc210: 4188 tokens, checkpoints: 0, 1014.331 MiB 28.56.376.277 I srv update: - prompt 0x561d90d122a0: 4215 tokens, checkpoints: 0, 1020.870 MiB 28.56.376.278 I srv update: - prompt 0x561d90ada720: 4229 tokens, checkpoints: 0, 1024.261 MiB 28.56.376.278 I srv update: - prompt 0x561d90ef7dc0: 4209 tokens, checkpoints: 0, 1019.417 MiB 28.56.376.281 I srv get_availabl: prompt cache update took 709.84 ms 28.56.378.535 I slot launch_slot_: id 3 | task 34929 | processing task, is_child = 0 28.56.378.539 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 28.56.378.539 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 28.56.378.561 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 28.56.378.562 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 28.56.378.581 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 28.56.378.581 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 29.02.257.901 I slot print_timing: id 3 | task 34929 | n_decoded = 100, tg = 22.27 t/s, tg_3s = 22.27 t/s 29.05.298.586 I slot print_timing: id 3 | task 34929 | n_decoded = 168, tg = 22.31 t/s, tg_3s = 22.36 t/s 29.08.335.123 I slot print_timing: id 3 | task 34929 | n_decoded = 236, tg = 22.33 t/s, tg_3s = 22.39 t/s 29.11.355.457 I slot print_timing: id 3 | task 34929 | n_decoded = 303, tg = 22.30 t/s, tg_3s = 22.18 t/s 29.14.400.426 I slot print_timing: id 3 | task 34929 | n_decoded = 371, tg = 22.30 t/s, tg_3s = 22.33 t/s 29.17.425.134 I slot print_timing: id 3 | task 34929 | n_decoded = 438, tg = 22.28 t/s, tg_3s = 22.15 t/s 29.20.467.632 I slot print_timing: id 3 | task 34929 | n_decoded = 506, tg = 22.29 t/s, tg_3s = 22.35 t/s 29.23.504.064 I slot print_timing: id 3 | task 34929 | n_decoded = 574, tg = 22.30 t/s, tg_3s = 22.39 t/s 29.26.550.260 I slot print_timing: id 3 | task 34929 | n_decoded = 642, tg = 22.30 t/s, tg_3s = 22.32 t/s 29.29.552.439 I slot print_timing: id 3 | task 34929 | n_decoded = 708, tg = 22.27 t/s, tg_3s = 21.98 t/s 29.32.575.002 I slot print_timing: id 3 | task 34929 | n_decoded = 774, tg = 22.24 t/s, tg_3s = 21.84 t/s 29.35.579.909 I slot print_timing: id 3 | task 34929 | n_decoded = 840, tg = 22.21 t/s, tg_3s = 21.96 t/s 29.38.599.758 I slot print_timing: id 3 | task 34929 | n_decoded = 906, tg = 22.19 t/s, tg_3s = 21.86 t/s 29.41.627.937 I slot print_timing: id 3 | task 34929 | n_decoded = 972, tg = 22.16 t/s, tg_3s = 21.80 t/s 29.44.657.053 I slot print_timing: id 3 | task 34929 | n_decoded = 1038, tg = 22.14 t/s, tg_3s = 21.79 t/s 29.47.667.680 I slot print_timing: id 3 | task 34929 | n_decoded = 1104, tg = 22.12 t/s, tg_3s = 21.92 t/s 29.50.694.324 I slot print_timing: id 3 | task 34929 | n_decoded = 1170, tg = 22.11 t/s, tg_3s = 21.81 t/s 29.53.708.497 I slot print_timing: id 3 | task 34929 | n_decoded = 1236, tg = 22.09 t/s, tg_3s = 21.90 t/s 29.56.745.727 I slot print_timing: id 3 | task 34929 | n_decoded = 1302, tg = 22.08 t/s, tg_3s = 21.73 t/s 29.59.759.953 I slot print_timing: id 3 | task 34929 | n_decoded = 1367, tg = 22.05 t/s, tg_3s = 21.56 t/s 30.02.794.680 I slot print_timing: id 3 | task 34929 | n_decoded = 1433, tg = 22.04 t/s, tg_3s = 21.75 t/s 30.05.834.544 I slot print_timing: id 3 | task 34929 | n_decoded = 1498, tg = 22.01 t/s, tg_3s = 21.38 t/s 30.08.878.995 I slot print_timing: id 3 | task 34929 | n_decoded = 1563, tg = 21.98 t/s, tg_3s = 21.35 t/s 30.11.908.289 I slot print_timing: id 3 | task 34929 | n_decoded = 1629, tg = 21.97 t/s, tg_3s = 21.79 t/s 30.14.952.694 I slot print_timing: id 3 | task 34929 | n_decoded = 1694, tg = 21.95 t/s, tg_3s = 21.35 t/s 30.17.990.391 I slot print_timing: id 3 | task 34929 | n_decoded = 1757, tg = 21.90 t/s, tg_3s = 20.74 t/s 30.21.026.117 I slot print_timing: id 3 | task 34929 | n_decoded = 1820, tg = 21.86 t/s, tg_3s = 20.75 t/s 30.24.046.735 I slot print_timing: id 3 | task 34929 | n_decoded = 1883, tg = 21.82 t/s, tg_3s = 20.86 t/s 30.27.092.032 I slot print_timing: id 3 | task 34929 | n_decoded = 1946, tg = 21.79 t/s, tg_3s = 20.69 t/s 30.30.125.826 I slot print_timing: id 3 | task 34929 | n_decoded = 2009, tg = 21.75 t/s, tg_3s = 20.77 t/s 30.33.171.371 I slot print_timing: id 3 | task 34929 | n_decoded = 2072, tg = 21.72 t/s, tg_3s = 20.69 t/s 30.36.213.727 I slot print_timing: id 3 | task 34929 | n_decoded = 2135, tg = 21.69 t/s, tg_3s = 20.71 t/s 30.39.228.340 I slot print_timing: id 3 | task 34929 | n_decoded = 2197, tg = 21.65 t/s, tg_3s = 20.57 t/s 30.42.242.792 I slot print_timing: id 3 | task 34929 | n_decoded = 2259, tg = 21.62 t/s, tg_3s = 20.57 t/s 30.45.291.522 I slot print_timing: id 3 | task 34929 | n_decoded = 2321, tg = 21.59 t/s, tg_3s = 20.34 t/s 30.48.312.637 I slot print_timing: id 3 | task 34929 | n_decoded = 2382, tg = 21.55 t/s, tg_3s = 20.19 t/s 30.51.353.544 I slot print_timing: id 3 | task 34929 | n_decoded = 2444, tg = 21.52 t/s, tg_3s = 20.39 t/s 30.53.813.260 I slot print_timing: id 3 | task 34929 | prompt eval time = 1388.08 ms / 54 tokens ( 25.71 ms per token, 38.90 tokens per second) 30.53.813.265 I slot print_timing: id 3 | task 34929 | eval time = 116046.57 ms / 2494 tokens ( 46.53 ms per token, 21.49 tokens per second) 30.53.813.266 I slot print_timing: id 3 | task 34929 | total time = 117434.64 ms / 2548 tokens 30.53.813.267 I slot print_timing: id 3 | task 34929 | graphs reused = 37243 30.53.813.299 I slot release: id 3 | task 34929 | stop processing: n_tokens = 2621, truncated = 0 30.53.813.320 I srv update_slots: all slots are idle 30.53.814.440 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 30.53.834.365 I srv operator(): Chat format: peg-native 30.53.840.484 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.569 (> 0.100 thold), f_keep = 0.028 30.53.840.486 I srv get_availabl: updating prompt cache 30.53.840.623 W srv prompt_save: - saving prompt with length 2621, total state size = 634.805 MiB (draft: 0.000 MiB) 30.54.243.985 I srv load: - looking for better prompt, base f_keep = 0.028, sim = 0.569 30.54.243.993 W srv update: - cache size limit reached, removing oldest entry (size = 178.018 MiB) 30.54.263.162 I srv update: - cache state: 10 prompts, 8169.385 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 30.54.263.164 I srv update: - prompt 0x561d97e885f0: 689 tokens, checkpoints: 0, 166.877 MiB 30.54.263.165 I srv update: - prompt 0x561d97e68780: 4188 tokens, checkpoints: 0, 1014.331 MiB 30.54.263.166 I srv update: - prompt 0x561d982cf8c0: 4201 tokens, checkpoints: 0, 1017.479 MiB 30.54.263.166 I srv update: - prompt 0x561d90bf8cd0: 1004 tokens, checkpoints: 0, 243.169 MiB 30.54.263.167 I srv update: - prompt 0x561d97edb570: 4186 tokens, checkpoints: 0, 1013.846 MiB 30.54.263.167 I srv update: - prompt 0x561d983dc210: 4188 tokens, checkpoints: 0, 1014.331 MiB 30.54.263.168 I srv update: - prompt 0x561d90d122a0: 4215 tokens, checkpoints: 0, 1020.870 MiB 30.54.263.168 I srv update: - prompt 0x561d90ada720: 4229 tokens, checkpoints: 0, 1024.261 MiB 30.54.263.169 I srv update: - prompt 0x561d90ef7dc0: 4209 tokens, checkpoints: 0, 1019.417 MiB 30.54.263.169 I srv update: - prompt 0x561d983db950: 2621 tokens, checkpoints: 0, 634.805 MiB 30.54.263.171 I srv get_availabl: prompt cache update took 422.68 ms 30.54.265.039 I slot launch_slot_: id 3 | task 37424 | processing task, is_child = 0 30.54.265.042 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 30.54.265.043 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 30.54.265.061 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 30.54.265.061 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 30.54.265.070 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 30.54.265.070 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 31.00.131.003 I slot print_timing: id 3 | task 37424 | n_decoded = 100, tg = 22.58 t/s, tg_3s = 22.58 t/s 31.03.149.328 I slot print_timing: id 3 | task 37424 | n_decoded = 167, tg = 22.43 t/s, tg_3s = 22.20 t/s 31.06.175.356 I slot print_timing: id 3 | task 37424 | n_decoded = 235, tg = 22.44 t/s, tg_3s = 22.47 t/s 31.09.202.303 I slot print_timing: id 3 | task 37424 | n_decoded = 303, tg = 22.45 t/s, tg_3s = 22.46 t/s 31.12.203.623 I slot print_timing: id 3 | task 37424 | n_decoded = 370, tg = 22.42 t/s, tg_3s = 22.32 t/s 31.15.231.006 I slot print_timing: id 3 | task 37424 | n_decoded = 437, tg = 22.38 t/s, tg_3s = 22.13 t/s 31.18.238.930 I slot print_timing: id 3 | task 37424 | n_decoded = 504, tg = 22.36 t/s, tg_3s = 22.27 t/s 31.21.282.138 I slot print_timing: id 3 | task 37424 | n_decoded = 572, tg = 22.36 t/s, tg_3s = 22.34 t/s 31.24.285.528 I slot print_timing: id 3 | task 37424 | n_decoded = 639, tg = 22.36 t/s, tg_3s = 22.31 t/s 31.27.297.242 I slot print_timing: id 3 | task 37424 | n_decoded = 705, tg = 22.31 t/s, tg_3s = 21.91 t/s 31.30.299.595 I slot print_timing: id 3 | task 37424 | n_decoded = 771, tg = 22.29 t/s, tg_3s = 21.98 t/s 31.33.300.482 I slot print_timing: id 3 | task 37424 | n_decoded = 837, tg = 22.26 t/s, tg_3s = 21.99 t/s 31.36.312.303 I slot print_timing: id 3 | task 37424 | n_decoded = 903, tg = 22.24 t/s, tg_3s = 21.91 t/s 31.39.318.864 I slot print_timing: id 3 | task 37424 | n_decoded = 969, tg = 22.22 t/s, tg_3s = 21.95 t/s 31.42.324.037 I slot print_timing: id 3 | task 37424 | n_decoded = 1034, tg = 22.18 t/s, tg_3s = 21.63 t/s 31.45.336.797 I slot print_timing: id 3 | task 37424 | n_decoded = 1100, tg = 22.16 t/s, tg_3s = 21.91 t/s 31.48.376.799 I slot print_timing: id 3 | task 37424 | n_decoded = 1166, tg = 22.14 t/s, tg_3s = 21.71 t/s 31.51.413.727 I slot print_timing: id 3 | task 37424 | n_decoded = 1232, tg = 22.11 t/s, tg_3s = 21.73 t/s 31.54.423.563 I slot print_timing: id 3 | task 37424 | n_decoded = 1298, tg = 22.10 t/s, tg_3s = 21.93 t/s 31.57.455.686 I slot print_timing: id 3 | task 37424 | n_decoded = 1364, tg = 22.09 t/s, tg_3s = 21.77 t/s 32.00.487.277 I slot print_timing: id 3 | task 37424 | n_decoded = 1430, tg = 22.07 t/s, tg_3s = 21.77 t/s 32.03.502.288 I slot print_timing: id 3 | task 37424 | n_decoded = 1495, tg = 22.05 t/s, tg_3s = 21.56 t/s 32.06.524.629 I slot print_timing: id 3 | task 37424 | n_decoded = 1561, tg = 22.04 t/s, tg_3s = 21.84 t/s 32.09.555.464 I slot print_timing: id 3 | task 37424 | n_decoded = 1627, tg = 22.03 t/s, tg_3s = 21.78 t/s 32.12.566.722 I slot print_timing: id 3 | task 37424 | n_decoded = 1691, tg = 22.00 t/s, tg_3s = 21.25 t/s 32.15.582.109 I slot print_timing: id 3 | task 37424 | n_decoded = 1754, tg = 21.96 t/s, tg_3s = 20.89 t/s 32.18.608.981 I slot print_timing: id 3 | task 37424 | n_decoded = 1817, tg = 21.92 t/s, tg_3s = 20.81 t/s 32.21.620.037 I slot print_timing: id 3 | task 37424 | n_decoded = 1879, tg = 21.87 t/s, tg_3s = 20.59 t/s 32.24.625.339 I slot print_timing: id 3 | task 37424 | n_decoded = 1941, tg = 21.83 t/s, tg_3s = 20.63 t/s 32.27.669.123 I slot print_timing: id 3 | task 37424 | n_decoded = 2004, tg = 21.79 t/s, tg_3s = 20.70 t/s 32.30.679.647 I slot print_timing: id 3 | task 37424 | n_decoded = 2066, tg = 21.75 t/s, tg_3s = 20.59 t/s 32.33.698.715 I slot print_timing: id 3 | task 37424 | n_decoded = 2127, tg = 21.71 t/s, tg_3s = 20.20 t/s 32.36.725.931 I slot print_timing: id 3 | task 37424 | n_decoded = 2189, tg = 21.67 t/s, tg_3s = 20.48 t/s 32.39.740.804 I slot print_timing: id 3 | task 37424 | n_decoded = 2250, tg = 21.63 t/s, tg_3s = 20.23 t/s 32.42.752.290 I slot print_timing: id 3 | task 37424 | n_decoded = 2312, tg = 21.60 t/s, tg_3s = 20.59 t/s 32.45.763.469 I slot print_timing: id 3 | task 37424 | n_decoded = 2374, tg = 21.57 t/s, tg_3s = 20.59 t/s 32.48.794.221 I slot print_timing: id 3 | task 37424 | n_decoded = 2436, tg = 21.54 t/s, tg_3s = 20.46 t/s 32.51.840.250 I slot print_timing: id 3 | task 37424 | n_decoded = 2498, tg = 21.51 t/s, tg_3s = 20.35 t/s 32.54.883.978 I slot print_timing: id 3 | task 37424 | n_decoded = 2560, tg = 21.48 t/s, tg_3s = 20.37 t/s 32.57.888.741 I slot print_timing: id 3 | task 37424 | n_decoded = 2621, tg = 21.45 t/s, tg_3s = 20.30 t/s 33.00.891.835 I slot print_timing: id 3 | task 37424 | n_decoded = 2682, tg = 21.42 t/s, tg_3s = 20.31 t/s 33.03.914.814 I slot print_timing: id 3 | task 37424 | n_decoded = 2742, tg = 21.39 t/s, tg_3s = 19.85 t/s 33.06.958.413 I slot print_timing: id 3 | task 37424 | n_decoded = 2803, tg = 21.36 t/s, tg_3s = 20.04 t/s 33.09.989.431 I slot print_timing: id 3 | task 37424 | n_decoded = 2864, tg = 21.33 t/s, tg_3s = 20.13 t/s 33.13.021.411 I slot print_timing: id 3 | task 37424 | n_decoded = 2925, tg = 21.30 t/s, tg_3s = 20.12 t/s 33.16.033.925 I slot print_timing: id 3 | task 37424 | n_decoded = 2985, tg = 21.27 t/s, tg_3s = 19.92 t/s 33.19.076.042 I slot print_timing: id 3 | task 37424 | n_decoded = 3045, tg = 21.24 t/s, tg_3s = 19.72 t/s 33.22.120.950 I slot print_timing: id 3 | task 37424 | n_decoded = 3105, tg = 21.21 t/s, tg_3s = 19.71 t/s 33.25.137.331 I slot print_timing: id 3 | task 37424 | n_decoded = 3165, tg = 21.18 t/s, tg_3s = 19.89 t/s 33.28.178.116 I slot print_timing: id 3 | task 37424 | n_decoded = 3225, tg = 21.15 t/s, tg_3s = 19.73 t/s 33.31.180.895 I slot print_timing: id 3 | task 37424 | n_decoded = 3285, tg = 21.13 t/s, tg_3s = 19.98 t/s 33.34.184.874 I slot print_timing: id 3 | task 37424 | n_decoded = 3345, tg = 21.11 t/s, tg_3s = 19.97 t/s 33.37.217.897 I slot print_timing: id 3 | task 37424 | n_decoded = 3405, tg = 21.08 t/s, tg_3s = 19.78 t/s 33.40.267.717 I slot print_timing: id 3 | task 37424 | n_decoded = 3465, tg = 21.06 t/s, tg_3s = 19.67 t/s 33.43.307.349 I slot print_timing: id 3 | task 37424 | n_decoded = 3525, tg = 21.03 t/s, tg_3s = 19.74 t/s 33.46.309.646 I slot print_timing: id 3 | task 37424 | n_decoded = 3584, tg = 21.01 t/s, tg_3s = 19.65 t/s 33.49.347.291 I slot print_timing: id 3 | task 37424 | n_decoded = 3644, tg = 20.99 t/s, tg_3s = 19.75 t/s 33.52.373.313 I slot print_timing: id 3 | task 37424 | n_decoded = 3703, tg = 20.96 t/s, tg_3s = 19.50 t/s 33.55.383.965 I slot print_timing: id 3 | task 37424 | n_decoded = 3762, tg = 20.94 t/s, tg_3s = 19.60 t/s 33.58.392.855 I slot print_timing: id 3 | task 37424 | n_decoded = 3821, tg = 20.92 t/s, tg_3s = 19.61 t/s 34.01.399.009 I slot print_timing: id 3 | task 37424 | n_decoded = 3880, tg = 20.89 t/s, tg_3s = 19.63 t/s 34.04.422.799 I slot print_timing: id 3 | task 37424 | n_decoded = 3939, tg = 20.87 t/s, tg_3s = 19.51 t/s 34.07.453.897 I slot print_timing: id 3 | task 37424 | n_decoded = 3998, tg = 20.85 t/s, tg_3s = 19.46 t/s 34.10.490.679 I slot print_timing: id 3 | task 37424 | n_decoded = 4057, tg = 20.83 t/s, tg_3s = 19.43 t/s 34.12.500.408 I slot print_timing: id 3 | task 37424 | prompt eval time = 1438.09 ms / 56 tokens ( 25.68 ms per token, 38.94 tokens per second) 34.12.500.412 I slot print_timing: id 3 | task 37424 | eval time = 196797.22 ms / 4096 tokens ( 48.05 ms per token, 20.81 tokens per second) 34.12.500.413 I slot print_timing: id 3 | task 37424 | total time = 198235.31 ms / 4152 tokens 34.12.500.413 I slot print_timing: id 3 | task 37424 | graphs reused = 41321 34.12.500.441 I slot release: id 3 | task 37424 | stop processing: n_tokens = 4225, truncated = 0 34.12.500.466 I srv update_slots: all slots are idle 34.12.501.598 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 34.12.518.831 I srv operator(): Chat format: peg-native 34.12.524.385 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.636 (> 0.100 thold), f_keep = 0.018 34.12.524.388 I srv get_availabl: updating prompt cache 34.12.524.546 W srv prompt_save: - saving prompt with length 4225, total state size = 1023.292 MiB (draft: 0.000 MiB) 34.13.203.601 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.636 34.13.203.610 W srv update: - cache size limit reached, removing oldest entry (size = 166.877 MiB) 34.13.222.745 W srv update: - cache size limit reached, removing oldest entry (size = 1014.331 MiB) 34.13.315.645 I srv update: - cache state: 9 prompts, 8011.470 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 34.13.315.650 I srv update: - prompt 0x561d982cf8c0: 4201 tokens, checkpoints: 0, 1017.479 MiB 34.13.315.651 I srv update: - prompt 0x561d90bf8cd0: 1004 tokens, checkpoints: 0, 243.169 MiB 34.13.315.652 I srv update: - prompt 0x561d97edb570: 4186 tokens, checkpoints: 0, 1013.846 MiB 34.13.315.653 I srv update: - prompt 0x561d983dc210: 4188 tokens, checkpoints: 0, 1014.331 MiB 34.13.315.653 I srv update: - prompt 0x561d90d122a0: 4215 tokens, checkpoints: 0, 1020.870 MiB 34.13.315.654 I srv update: - prompt 0x561d90ada720: 4229 tokens, checkpoints: 0, 1024.261 MiB 34.13.315.655 I srv update: - prompt 0x561d90ef7dc0: 4209 tokens, checkpoints: 0, 1019.417 MiB 34.13.315.656 I srv update: - prompt 0x561d983db950: 2621 tokens, checkpoints: 0, 634.805 MiB 34.13.315.656 I srv update: - prompt 0x561d90a1b120: 4225 tokens, checkpoints: 0, 1023.292 MiB 34.13.315.658 I srv get_availabl: prompt cache update took 791.27 ms 34.13.317.459 I slot launch_slot_: id 3 | task 41521 | processing task, is_child = 0 34.13.317.462 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 34.13.317.463 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 34.13.317.487 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 34.13.317.487 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 34.13.317.498 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 34.13.317.498 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 34.18.908.577 I slot print_timing: id 3 | task 41521 | n_decoded = 100, tg = 22.62 t/s, tg_3s = 22.62 t/s 34.21.940.156 I slot print_timing: id 3 | task 41521 | n_decoded = 168, tg = 22.54 t/s, tg_3s = 22.43 t/s 34.24.967.243 I slot print_timing: id 3 | task 41521 | n_decoded = 236, tg = 22.52 t/s, tg_3s = 22.46 t/s 34.27.969.799 I slot print_timing: id 3 | task 41521 | n_decoded = 303, tg = 22.47 t/s, tg_3s = 22.31 t/s 34.31.008.872 I slot print_timing: id 3 | task 41521 | n_decoded = 370, tg = 22.40 t/s, tg_3s = 22.05 t/s 34.34.026.859 I slot print_timing: id 3 | task 41521 | n_decoded = 437, tg = 22.37 t/s, tg_3s = 22.20 t/s 34.37.035.828 I slot print_timing: id 3 | task 41521 | n_decoded = 504, tg = 22.35 t/s, tg_3s = 22.27 t/s 34.40.050.046 I slot print_timing: id 3 | task 41521 | n_decoded = 571, tg = 22.34 t/s, tg_3s = 22.23 t/s 34.43.052.441 I slot print_timing: id 3 | task 41521 | n_decoded = 638, tg = 22.34 t/s, tg_3s = 22.32 t/s 34.46.053.740 I slot print_timing: id 3 | task 41521 | n_decoded = 704, tg = 22.30 t/s, tg_3s = 21.99 t/s 34.49.058.478 I slot print_timing: id 3 | task 41521 | n_decoded = 770, tg = 22.27 t/s, tg_3s = 21.97 t/s 34.52.060.261 I slot print_timing: id 3 | task 41521 | n_decoded = 836, tg = 22.25 t/s, tg_3s = 21.99 t/s 34.55.060.776 I slot print_timing: id 3 | task 41521 | n_decoded = 902, tg = 22.23 t/s, tg_3s = 22.00 t/s 34.58.080.975 I slot print_timing: id 3 | task 41521 | n_decoded = 968, tg = 22.21 t/s, tg_3s = 21.85 t/s 35.01.085.069 I slot print_timing: id 3 | task 41521 | n_decoded = 1034, tg = 22.19 t/s, tg_3s = 21.97 t/s 35.04.105.636 I slot print_timing: id 3 | task 41521 | n_decoded = 1099, tg = 22.15 t/s, tg_3s = 21.52 t/s 35.07.118.100 I slot print_timing: id 3 | task 41521 | n_decoded = 1164, tg = 22.12 t/s, tg_3s = 21.58 t/s 35.10.158.431 I slot print_timing: id 3 | task 41521 | n_decoded = 1230, tg = 22.09 t/s, tg_3s = 21.71 t/s 35.13.163.631 I slot print_timing: id 3 | task 41521 | n_decoded = 1296, tg = 22.09 t/s, tg_3s = 21.96 t/s 35.16.195.700 I slot print_timing: id 3 | task 41521 | n_decoded = 1362, tg = 22.07 t/s, tg_3s = 21.77 t/s 35.19.229.433 I slot print_timing: id 3 | task 41521 | n_decoded = 1428, tg = 22.06 t/s, tg_3s = 21.76 t/s 35.22.241.441 I slot print_timing: id 3 | task 41521 | n_decoded = 1494, tg = 22.05 t/s, tg_3s = 21.91 t/s 35.25.250.522 I slot print_timing: id 3 | task 41521 | n_decoded = 1560, tg = 22.05 t/s, tg_3s = 21.93 t/s 35.28.294.733 I slot print_timing: id 3 | task 41521 | n_decoded = 1626, tg = 22.03 t/s, tg_3s = 21.68 t/s 35.31.304.232 I slot print_timing: id 3 | task 41521 | n_decoded = 1691, tg = 22.01 t/s, tg_3s = 21.60 t/s 35.34.314.365 I slot print_timing: id 3 | task 41521 | n_decoded = 1754, tg = 21.97 t/s, tg_3s = 20.93 t/s 35.37.325.204 I slot print_timing: id 3 | task 41521 | n_decoded = 1817, tg = 21.93 t/s, tg_3s = 20.92 t/s 35.40.341.738 I slot print_timing: id 3 | task 41521 | n_decoded = 1880, tg = 21.90 t/s, tg_3s = 20.88 t/s 35.43.382.592 I slot print_timing: id 3 | task 41521 | n_decoded = 1943, tg = 21.86 t/s, tg_3s = 20.72 t/s 35.46.384.786 I slot print_timing: id 3 | task 41521 | n_decoded = 2005, tg = 21.82 t/s, tg_3s = 20.65 t/s 35.49.389.986 I slot print_timing: id 3 | task 41521 | n_decoded = 2067, tg = 21.78 t/s, tg_3s = 20.63 t/s 35.52.425.661 I slot print_timing: id 3 | task 41521 | n_decoded = 2130, tg = 21.75 t/s, tg_3s = 20.75 t/s 35.55.434.158 I slot print_timing: id 3 | task 41521 | n_decoded = 2192, tg = 21.71 t/s, tg_3s = 20.61 t/s 35.58.467.690 I slot print_timing: id 3 | task 41521 | n_decoded = 2254, tg = 21.68 t/s, tg_3s = 20.44 t/s 36.01.476.214 I slot print_timing: id 3 | task 41521 | n_decoded = 2316, tg = 21.65 t/s, tg_3s = 20.61 t/s 36.04.483.498 I slot print_timing: id 3 | task 41521 | n_decoded = 2378, tg = 21.62 t/s, tg_3s = 20.62 t/s 36.07.498.804 I slot print_timing: id 3 | task 41521 | n_decoded = 2440, tg = 21.59 t/s, tg_3s = 20.56 t/s 36.10.546.499 I slot print_timing: id 3 | task 41521 | n_decoded = 2502, tg = 21.56 t/s, tg_3s = 20.34 t/s 36.13.548.370 I slot print_timing: id 3 | task 41521 | n_decoded = 2563, tg = 21.53 t/s, tg_3s = 20.32 t/s 36.16.593.965 I slot print_timing: id 3 | task 41521 | n_decoded = 2625, tg = 21.50 t/s, tg_3s = 20.36 t/s 36.19.626.231 I slot print_timing: id 3 | task 41521 | n_decoded = 2686, tg = 21.46 t/s, tg_3s = 20.12 t/s 36.22.667.493 I slot print_timing: id 3 | task 41521 | n_decoded = 2747, tg = 21.43 t/s, tg_3s = 20.06 t/s 36.25.695.561 I slot print_timing: id 3 | task 41521 | n_decoded = 2808, tg = 21.40 t/s, tg_3s = 20.14 t/s 36.28.734.769 I slot print_timing: id 3 | task 41521 | n_decoded = 2869, tg = 21.37 t/s, tg_3s = 20.07 t/s 36.31.776.554 I slot print_timing: id 3 | task 41521 | n_decoded = 2930, tg = 21.34 t/s, tg_3s = 20.05 t/s 36.34.824.939 I slot print_timing: id 3 | task 41521 | n_decoded = 2990, tg = 21.31 t/s, tg_3s = 19.68 t/s 36.37.837.374 I slot print_timing: id 3 | task 41521 | n_decoded = 3050, tg = 21.28 t/s, tg_3s = 19.92 t/s 36.40.844.778 I slot print_timing: id 3 | task 41521 | n_decoded = 3110, tg = 21.25 t/s, tg_3s = 19.95 t/s 36.43.851.434 I slot print_timing: id 3 | task 41521 | n_decoded = 3170, tg = 21.22 t/s, tg_3s = 19.96 t/s 36.46.868.485 I slot print_timing: id 3 | task 41521 | n_decoded = 3230, tg = 21.20 t/s, tg_3s = 19.89 t/s 36.49.879.643 I slot print_timing: id 3 | task 41521 | n_decoded = 3290, tg = 21.17 t/s, tg_3s = 19.93 t/s 36.52.880.717 I slot print_timing: id 3 | task 41521 | n_decoded = 3350, tg = 21.15 t/s, tg_3s = 19.99 t/s 36.55.884.990 I slot print_timing: id 3 | task 41521 | n_decoded = 3410, tg = 21.13 t/s, tg_3s = 19.97 t/s 36.58.918.046 I slot print_timing: id 3 | task 41521 | n_decoded = 3470, tg = 21.10 t/s, tg_3s = 19.78 t/s 37.01.921.068 I slot print_timing: id 3 | task 41521 | n_decoded = 3529, tg = 21.08 t/s, tg_3s = 19.65 t/s 37.04.945.205 I slot print_timing: id 3 | task 41521 | n_decoded = 3588, tg = 21.05 t/s, tg_3s = 19.51 t/s 37.07.987.644 I slot print_timing: id 3 | task 41521 | n_decoded = 3648, tg = 21.03 t/s, tg_3s = 19.72 t/s 37.11.027.784 I slot print_timing: id 3 | task 41521 | n_decoded = 3708, tg = 21.00 t/s, tg_3s = 19.74 t/s 37.14.040.091 I slot print_timing: id 3 | task 41521 | n_decoded = 3767, tg = 20.98 t/s, tg_3s = 19.59 t/s 37.17.051.177 I slot print_timing: id 3 | task 41521 | n_decoded = 3826, tg = 20.96 t/s, tg_3s = 19.59 t/s 37.20.057.173 I slot print_timing: id 3 | task 41521 | n_decoded = 3885, tg = 20.94 t/s, tg_3s = 19.63 t/s 37.23.065.401 I slot print_timing: id 3 | task 41521 | n_decoded = 3944, tg = 20.91 t/s, tg_3s = 19.61 t/s 37.26.097.012 I slot print_timing: id 3 | task 41521 | n_decoded = 4003, tg = 20.89 t/s, tg_3s = 19.46 t/s 37.29.130.880 I slot print_timing: id 3 | task 41521 | n_decoded = 4062, tg = 20.87 t/s, tg_3s = 19.45 t/s 37.30.874.584 I slot print_timing: id 3 | task 41521 | prompt eval time = 1170.33 ms / 43 tokens ( 27.22 ms per token, 36.74 tokens per second) 37.30.874.588 I slot print_timing: id 3 | task 41521 | eval time = 196386.72 ms / 4096 tokens ( 47.95 ms per token, 20.86 tokens per second) 37.30.874.589 I slot print_timing: id 3 | task 41521 | total time = 197557.05 ms / 4139 tokens 37.30.874.590 I slot print_timing: id 3 | task 41521 | graphs reused = 45399 37.30.874.619 I slot release: id 3 | task 41521 | stop processing: n_tokens = 4213, truncated = 0 37.30.874.645 I srv update_slots: all slots are idle 37.30.875.706 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 37.30.893.034 I srv operator(): Chat format: peg-native 37.30.898.598 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.592 (> 0.100 thold), f_keep = 0.018 37.30.898.600 I srv get_availabl: updating prompt cache 37.30.898.764 W srv prompt_save: - saving prompt with length 4213, total state size = 1020.386 MiB (draft: 0.000 MiB) 37.31.547.921 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.592 37.31.547.930 W srv update: - cache size limit reached, removing oldest entry (size = 1017.479 MiB) 37.31.627.058 I srv update: - cache state: 9 prompts, 8014.376 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 37.31.627.062 I srv update: - prompt 0x561d90bf8cd0: 1004 tokens, checkpoints: 0, 243.169 MiB 37.31.627.063 I srv update: - prompt 0x561d97edb570: 4186 tokens, checkpoints: 0, 1013.846 MiB 37.31.627.063 I srv update: - prompt 0x561d983dc210: 4188 tokens, checkpoints: 0, 1014.331 MiB 37.31.627.064 I srv update: - prompt 0x561d90d122a0: 4215 tokens, checkpoints: 0, 1020.870 MiB 37.31.627.064 I srv update: - prompt 0x561d90ada720: 4229 tokens, checkpoints: 0, 1024.261 MiB 37.31.627.065 I srv update: - prompt 0x561d90ef7dc0: 4209 tokens, checkpoints: 0, 1019.417 MiB 37.31.627.065 I srv update: - prompt 0x561d983db950: 2621 tokens, checkpoints: 0, 634.805 MiB 37.31.627.066 I srv update: - prompt 0x561d90a1b120: 4225 tokens, checkpoints: 0, 1023.292 MiB 37.31.627.066 I srv update: - prompt 0x561d90b66b80: 4213 tokens, checkpoints: 0, 1020.386 MiB 37.31.627.068 I srv get_availabl: prompt cache update took 728.47 ms 37.31.628.684 I slot launch_slot_: id 3 | task 45618 | processing task, is_child = 0 37.31.628.686 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 37.31.628.687 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 37.31.628.707 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 37.31.628.708 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 37.31.628.717 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 37.31.628.717 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 37.37.496.713 I slot print_timing: id 3 | task 45618 | n_decoded = 100, tg = 22.57 t/s, tg_3s = 22.57 t/s 37.40.538.654 I slot print_timing: id 3 | task 45618 | n_decoded = 168, tg = 22.48 t/s, tg_3s = 22.35 t/s 37.43.572.544 I slot print_timing: id 3 | task 45618 | n_decoded = 236, tg = 22.46 t/s, tg_3s = 22.41 t/s 37.46.600.948 I slot print_timing: id 3 | task 45618 | n_decoded = 304, tg = 22.46 t/s, tg_3s = 22.45 t/s 37.49.633.043 I slot print_timing: id 3 | task 45618 | n_decoded = 372, tg = 22.45 t/s, tg_3s = 22.43 t/s 37.52.657.191 I slot print_timing: id 3 | task 45618 | n_decoded = 439, tg = 22.41 t/s, tg_3s = 22.15 t/s 37.55.696.823 I slot print_timing: id 3 | task 45618 | n_decoded = 507, tg = 22.40 t/s, tg_3s = 22.37 t/s 37.58.699.221 I slot print_timing: id 3 | task 45618 | n_decoded = 574, tg = 22.39 t/s, tg_3s = 22.32 t/s 38.01.704.822 I slot print_timing: id 3 | task 45618 | n_decoded = 641, tg = 22.38 t/s, tg_3s = 22.29 t/s 38.04.712.601 I slot print_timing: id 3 | task 45618 | n_decoded = 707, tg = 22.34 t/s, tg_3s = 21.94 t/s 38.07.755.287 I slot print_timing: id 3 | task 45618 | n_decoded = 774, tg = 22.31 t/s, tg_3s = 22.02 t/s 38.10.798.387 I slot print_timing: id 3 | task 45618 | n_decoded = 841, tg = 22.29 t/s, tg_3s = 22.02 t/s 38.13.801.753 I slot print_timing: id 3 | task 45618 | n_decoded = 906, tg = 22.24 t/s, tg_3s = 21.64 t/s 38.16.804.097 I slot print_timing: id 3 | task 45618 | n_decoded = 972, tg = 22.22 t/s, tg_3s = 21.98 t/s 38.19.809.606 I slot print_timing: id 3 | task 45618 | n_decoded = 1038, tg = 22.21 t/s, tg_3s = 21.96 t/s 38.21.402.616 I slot print_timing: id 3 | task 45618 | prompt eval time = 1436.51 ms / 51 tokens ( 28.17 ms per token, 35.50 tokens per second) 38.21.402.622 I slot print_timing: id 3 | task 45618 | eval time = 48337.36 ms / 1073 tokens ( 45.05 ms per token, 22.20 tokens per second) 38.21.402.622 I slot print_timing: id 3 | task 45618 | total time = 49773.87 ms / 1124 tokens 38.21.402.623 I slot print_timing: id 3 | task 45618 | graphs reused = 46466 38.21.402.654 I slot release: id 3 | task 45618 | stop processing: n_tokens = 1197, truncated = 0 38.21.402.671 I srv update_slots: all slots are idle 38.21.403.567 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 38.21.420.871 I srv operator(): Chat format: peg-native 38.21.426.446 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.661 (> 0.100 thold), f_keep = 0.062 38.21.426.448 I srv get_availabl: updating prompt cache 38.21.426.528 W srv prompt_save: - saving prompt with length 1197, total state size = 289.914 MiB (draft: 0.000 MiB) 38.21.628.393 I srv load: - looking for better prompt, base f_keep = 0.062, sim = 0.661 38.21.628.403 W srv update: - cache size limit reached, removing oldest entry (size = 243.169 MiB) 38.21.655.989 I srv update: - cache state: 9 prompts, 8061.120 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 38.21.655.992 I srv update: - prompt 0x561d97edb570: 4186 tokens, checkpoints: 0, 1013.846 MiB 38.21.655.993 I srv update: - prompt 0x561d983dc210: 4188 tokens, checkpoints: 0, 1014.331 MiB 38.21.655.994 I srv update: - prompt 0x561d90d122a0: 4215 tokens, checkpoints: 0, 1020.870 MiB 38.21.655.994 I srv update: - prompt 0x561d90ada720: 4229 tokens, checkpoints: 0, 1024.261 MiB 38.21.655.995 I srv update: - prompt 0x561d90ef7dc0: 4209 tokens, checkpoints: 0, 1019.417 MiB 38.21.655.995 I srv update: - prompt 0x561d983db950: 2621 tokens, checkpoints: 0, 634.805 MiB 38.21.655.996 I srv update: - prompt 0x561d90a1b120: 4225 tokens, checkpoints: 0, 1023.292 MiB 38.21.655.996 I srv update: - prompt 0x561d90b66b80: 4213 tokens, checkpoints: 0, 1020.386 MiB 38.21.655.997 I srv update: - prompt 0x561d90a2a790: 1197 tokens, checkpoints: 0, 289.914 MiB 38.21.655.999 I srv get_availabl: prompt cache update took 229.55 ms 38.21.657.944 I slot launch_slot_: id 3 | task 46692 | processing task, is_child = 0 38.21.657.947 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 38.21.657.948 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 38.21.657.962 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 38.21.657.962 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 38.21.657.976 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 38.21.657.977 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 38.27.124.855 I slot print_timing: id 3 | task 46692 | n_decoded = 100, tg = 22.52 t/s, tg_3s = 22.52 t/s 38.30.167.853 I slot print_timing: id 3 | task 46692 | n_decoded = 168, tg = 22.45 t/s, tg_3s = 22.35 t/s 38.33.205.741 I slot print_timing: id 3 | task 46692 | n_decoded = 236, tg = 22.43 t/s, tg_3s = 22.38 t/s 38.36.249.104 I slot print_timing: id 3 | task 46692 | n_decoded = 304, tg = 22.41 t/s, tg_3s = 22.34 t/s 38.39.279.295 I slot print_timing: id 3 | task 46692 | n_decoded = 372, tg = 22.42 t/s, tg_3s = 22.44 t/s 38.42.282.981 I slot print_timing: id 3 | task 46692 | n_decoded = 439, tg = 22.40 t/s, tg_3s = 22.31 t/s 38.45.297.925 I slot print_timing: id 3 | task 46692 | n_decoded = 506, tg = 22.38 t/s, tg_3s = 22.22 t/s 38.48.301.074 I slot print_timing: id 3 | task 46692 | n_decoded = 573, tg = 22.37 t/s, tg_3s = 22.31 t/s 38.51.301.958 I slot print_timing: id 3 | task 46692 | n_decoded = 640, tg = 22.36 t/s, tg_3s = 22.33 t/s 38.54.348.055 I slot print_timing: id 3 | task 46692 | n_decoded = 707, tg = 22.33 t/s, tg_3s = 22.00 t/s 38.57.386.148 I slot print_timing: id 3 | task 46692 | n_decoded = 773, tg = 22.28 t/s, tg_3s = 21.72 t/s 39.00.393.429 I slot print_timing: id 3 | task 46692 | n_decoded = 839, tg = 22.25 t/s, tg_3s = 21.95 t/s 39.03.438.705 I slot print_timing: id 3 | task 46692 | n_decoded = 906, tg = 22.23 t/s, tg_3s = 22.00 t/s 39.06.454.038 I slot print_timing: id 3 | task 46692 | n_decoded = 972, tg = 22.21 t/s, tg_3s = 21.89 t/s 39.09.457.005 I slot print_timing: id 3 | task 46692 | n_decoded = 1038, tg = 22.19 t/s, tg_3s = 21.98 t/s 39.12.463.133 I slot print_timing: id 3 | task 46692 | n_decoded = 1104, tg = 22.18 t/s, tg_3s = 21.96 t/s 39.15.469.711 I slot print_timing: id 3 | task 46692 | n_decoded = 1170, tg = 22.17 t/s, tg_3s = 21.95 t/s 39.18.498.527 I slot print_timing: id 3 | task 46692 | n_decoded = 1236, tg = 22.15 t/s, tg_3s = 21.79 t/s 39.21.503.460 I slot print_timing: id 3 | task 46692 | n_decoded = 1302, tg = 22.14 t/s, tg_3s = 21.96 t/s 39.24.509.131 I slot print_timing: id 3 | task 46692 | n_decoded = 1368, tg = 22.13 t/s, tg_3s = 21.96 t/s 39.27.552.863 I slot print_timing: id 3 | task 46692 | n_decoded = 1434, tg = 22.11 t/s, tg_3s = 21.68 t/s 39.30.582.977 I slot print_timing: id 3 | task 46692 | n_decoded = 1500, tg = 22.09 t/s, tg_3s = 21.78 t/s 39.33.605.418 I slot print_timing: id 3 | task 46692 | n_decoded = 1566, tg = 22.08 t/s, tg_3s = 21.84 t/s 39.36.619.197 I slot print_timing: id 3 | task 46692 | n_decoded = 1632, tg = 22.07 t/s, tg_3s = 21.90 t/s 39.39.655.953 I slot print_timing: id 3 | task 46692 | n_decoded = 1697, tg = 22.05 t/s, tg_3s = 21.40 t/s 39.42.680.578 I slot print_timing: id 3 | task 46692 | n_decoded = 1760, tg = 22.00 t/s, tg_3s = 20.83 t/s 39.45.690.629 I slot print_timing: id 3 | task 46692 | n_decoded = 1821, tg = 21.94 t/s, tg_3s = 20.27 t/s 39.48.718.098 I slot print_timing: id 3 | task 46692 | n_decoded = 1884, tg = 21.90 t/s, tg_3s = 20.81 t/s 39.51.722.869 I slot print_timing: id 3 | task 46692 | n_decoded = 1946, tg = 21.86 t/s, tg_3s = 20.63 t/s 39.54.760.379 I slot print_timing: id 3 | task 46692 | n_decoded = 2008, tg = 21.81 t/s, tg_3s = 20.41 t/s 39.57.764.127 I slot print_timing: id 3 | task 46692 | n_decoded = 2069, tg = 21.76 t/s, tg_3s = 20.31 t/s 40.00.791.938 I slot print_timing: id 3 | task 46692 | n_decoded = 2131, tg = 21.72 t/s, tg_3s = 20.48 t/s 40.03.833.213 I slot print_timing: id 3 | task 46692 | n_decoded = 2194, tg = 21.69 t/s, tg_3s = 20.72 t/s 40.06.861.553 I slot print_timing: id 3 | task 46692 | n_decoded = 2255, tg = 21.65 t/s, tg_3s = 20.14 t/s 40.09.900.838 I slot print_timing: id 3 | task 46692 | n_decoded = 2316, tg = 21.60 t/s, tg_3s = 20.07 t/s 40.12.917.162 I slot print_timing: id 3 | task 46692 | n_decoded = 2378, tg = 21.57 t/s, tg_3s = 20.55 t/s 40.15.928.683 I slot print_timing: id 3 | task 46692 | n_decoded = 2440, tg = 21.55 t/s, tg_3s = 20.59 t/s 40.18.959.552 I slot print_timing: id 3 | task 46692 | n_decoded = 2501, tg = 21.51 t/s, tg_3s = 20.13 t/s 40.21.967.931 I slot print_timing: id 3 | task 46692 | n_decoded = 2562, tg = 21.48 t/s, tg_3s = 20.28 t/s 40.24.989.151 I slot print_timing: id 3 | task 46692 | n_decoded = 2623, tg = 21.45 t/s, tg_3s = 20.19 t/s 40.28.004.024 I slot print_timing: id 3 | task 46692 | n_decoded = 2684, tg = 21.42 t/s, tg_3s = 20.23 t/s 40.31.036.122 I slot print_timing: id 3 | task 46692 | n_decoded = 2745, tg = 21.39 t/s, tg_3s = 20.12 t/s 40.34.067.640 I slot print_timing: id 3 | task 46692 | n_decoded = 2806, tg = 21.36 t/s, tg_3s = 20.12 t/s 40.37.109.314 I slot print_timing: id 3 | task 46692 | n_decoded = 2866, tg = 21.32 t/s, tg_3s = 19.73 t/s 40.40.146.057 I slot print_timing: id 3 | task 46692 | n_decoded = 2926, tg = 21.29 t/s, tg_3s = 19.76 t/s 40.43.150.103 I slot print_timing: id 3 | task 46692 | n_decoded = 2986, tg = 21.26 t/s, tg_3s = 19.97 t/s 40.46.156.880 I slot print_timing: id 3 | task 46692 | n_decoded = 3046, tg = 21.23 t/s, tg_3s = 19.95 t/s 40.49.162.526 I slot print_timing: id 3 | task 46692 | n_decoded = 3106, tg = 21.20 t/s, tg_3s = 19.96 t/s 40.52.178.152 I slot print_timing: id 3 | task 46692 | n_decoded = 3166, tg = 21.18 t/s, tg_3s = 19.90 t/s 40.55.198.236 I slot print_timing: id 3 | task 46692 | n_decoded = 3226, tg = 21.15 t/s, tg_3s = 19.87 t/s 40.58.209.374 I slot print_timing: id 3 | task 46692 | n_decoded = 3286, tg = 21.13 t/s, tg_3s = 19.93 t/s 41.01.212.485 I slot print_timing: id 3 | task 46692 | n_decoded = 3346, tg = 21.11 t/s, tg_3s = 19.98 t/s 41.04.221.772 I slot print_timing: id 3 | task 46692 | n_decoded = 3406, tg = 21.09 t/s, tg_3s = 19.94 t/s 41.07.244.559 I slot print_timing: id 3 | task 46692 | n_decoded = 3466, tg = 21.06 t/s, tg_3s = 19.85 t/s 41.10.277.885 I slot print_timing: id 3 | task 46692 | n_decoded = 3525, tg = 21.03 t/s, tg_3s = 19.45 t/s 41.13.290.595 I slot print_timing: id 3 | task 46692 | n_decoded = 3584, tg = 21.01 t/s, tg_3s = 19.58 t/s 41.16.331.049 I slot print_timing: id 3 | task 46692 | n_decoded = 3644, tg = 20.99 t/s, tg_3s = 19.73 t/s 41.19.376.302 I slot print_timing: id 3 | task 46692 | n_decoded = 3704, tg = 20.96 t/s, tg_3s = 19.70 t/s 41.22.383.901 I slot print_timing: id 3 | task 46692 | n_decoded = 3763, tg = 20.94 t/s, tg_3s = 19.62 t/s 41.25.387.847 I slot print_timing: id 3 | task 46692 | n_decoded = 3822, tg = 20.92 t/s, tg_3s = 19.64 t/s 41.28.399.075 I slot print_timing: id 3 | task 46692 | n_decoded = 3881, tg = 20.90 t/s, tg_3s = 19.59 t/s 41.31.410.095 I slot print_timing: id 3 | task 46692 | n_decoded = 3940, tg = 20.88 t/s, tg_3s = 19.59 t/s 41.34.455.680 I slot print_timing: id 3 | task 46692 | n_decoded = 3998, tg = 20.85 t/s, tg_3s = 19.04 t/s 41.37.502.384 I slot print_timing: id 3 | task 46692 | n_decoded = 4057, tg = 20.82 t/s, tg_3s = 19.37 t/s 41.39.528.989 I slot print_timing: id 3 | task 46692 | prompt eval time = 1027.17 ms / 38 tokens ( 27.03 ms per token, 36.99 tokens per second) 41.39.528.994 I slot print_timing: id 3 | task 46692 | eval time = 196843.80 ms / 4096 tokens ( 48.06 ms per token, 20.81 tokens per second) 41.39.528.995 I slot print_timing: id 3 | task 46692 | total time = 197870.98 ms / 4134 tokens 41.39.528.996 I slot print_timing: id 3 | task 46692 | graphs reused = 50544 41.39.529.031 I slot release: id 3 | task 46692 | stop processing: n_tokens = 4207, truncated = 0 41.39.529.063 I srv update_slots: all slots are idle 41.39.530.055 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 41.39.549.234 I srv operator(): Chat format: peg-native 41.39.555.350 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.643 (> 0.100 thold), f_keep = 0.018 41.39.555.352 I srv get_availabl: updating prompt cache 41.39.555.528 W srv prompt_save: - saving prompt with length 4207, total state size = 1018.932 MiB (draft: 0.000 MiB) 41.40.214.948 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.643 41.40.214.958 W srv update: - cache size limit reached, removing oldest entry (size = 1013.846 MiB) 41.40.308.632 I srv update: - cache state: 9 prompts, 8066.207 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 41.40.308.637 I srv update: - prompt 0x561d983dc210: 4188 tokens, checkpoints: 0, 1014.331 MiB 41.40.308.638 I srv update: - prompt 0x561d90d122a0: 4215 tokens, checkpoints: 0, 1020.870 MiB 41.40.308.639 I srv update: - prompt 0x561d90ada720: 4229 tokens, checkpoints: 0, 1024.261 MiB 41.40.308.640 I srv update: - prompt 0x561d90ef7dc0: 4209 tokens, checkpoints: 0, 1019.417 MiB 41.40.308.640 I srv update: - prompt 0x561d983db950: 2621 tokens, checkpoints: 0, 634.805 MiB 41.40.308.641 I srv update: - prompt 0x561d90a1b120: 4225 tokens, checkpoints: 0, 1023.292 MiB 41.40.308.642 I srv update: - prompt 0x561d90b66b80: 4213 tokens, checkpoints: 0, 1020.386 MiB 41.40.308.643 I srv update: - prompt 0x561d90a2a790: 1197 tokens, checkpoints: 0, 289.914 MiB 41.40.308.643 I srv update: - prompt 0x561d90e0e700: 4207 tokens, checkpoints: 0, 1018.932 MiB 41.40.308.645 I srv get_availabl: prompt cache update took 753.29 ms 41.40.311.206 I slot launch_slot_: id 3 | task 50789 | processing task, is_child = 0 41.40.311.209 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 41.40.311.210 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 41.40.311.233 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 41.40.311.233 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 41.40.311.247 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 41.40.311.247 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 41.45.931.681 I slot print_timing: id 3 | task 50789 | n_decoded = 100, tg = 22.59 t/s, tg_3s = 22.59 t/s 41.48.974.254 I slot print_timing: id 3 | task 50789 | n_decoded = 168, tg = 22.49 t/s, tg_3s = 22.35 t/s 41.52.009.488 I slot print_timing: id 3 | task 50789 | n_decoded = 236, tg = 22.46 t/s, tg_3s = 22.40 t/s 41.55.034.072 I slot print_timing: id 3 | task 50789 | n_decoded = 304, tg = 22.47 t/s, tg_3s = 22.48 t/s 41.58.069.833 I slot print_timing: id 3 | task 50789 | n_decoded = 372, tg = 22.46 t/s, tg_3s = 22.40 t/s 42.01.076.305 I slot print_timing: id 3 | task 50789 | n_decoded = 439, tg = 22.43 t/s, tg_3s = 22.29 t/s 42.04.084.085 I slot print_timing: id 3 | task 50789 | n_decoded = 506, tg = 22.41 t/s, tg_3s = 22.28 t/s 42.07.126.898 I slot print_timing: id 3 | task 50789 | n_decoded = 574, tg = 22.40 t/s, tg_3s = 22.35 t/s 42.10.168.859 I slot print_timing: id 3 | task 50789 | n_decoded = 641, tg = 22.36 t/s, tg_3s = 22.03 t/s 42.13.209.312 I slot print_timing: id 3 | task 50789 | n_decoded = 707, tg = 22.30 t/s, tg_3s = 21.71 t/s 42.16.220.059 I slot print_timing: id 3 | task 50789 | n_decoded = 773, tg = 22.27 t/s, tg_3s = 21.92 t/s 42.19.222.081 I slot print_timing: id 3 | task 50789 | n_decoded = 839, tg = 22.24 t/s, tg_3s = 21.99 t/s 42.22.266.476 I slot print_timing: id 3 | task 50789 | n_decoded = 906, tg = 22.23 t/s, tg_3s = 22.01 t/s 42.25.287.554 I slot print_timing: id 3 | task 50789 | n_decoded = 972, tg = 22.20 t/s, tg_3s = 21.85 t/s 42.28.287.791 I slot print_timing: id 3 | task 50789 | n_decoded = 1037, tg = 22.17 t/s, tg_3s = 21.66 t/s 42.31.304.976 I slot print_timing: id 3 | task 50789 | n_decoded = 1103, tg = 22.15 t/s, tg_3s = 21.87 t/s 42.34.348.415 I slot print_timing: id 3 | task 50789 | n_decoded = 1169, tg = 22.12 t/s, tg_3s = 21.69 t/s 42.37.359.171 I slot print_timing: id 3 | task 50789 | n_decoded = 1235, tg = 22.11 t/s, tg_3s = 21.92 t/s 42.40.362.534 I slot print_timing: id 3 | task 50789 | n_decoded = 1301, tg = 22.10 t/s, tg_3s = 21.98 t/s 42.43.367.100 I slot print_timing: id 3 | task 50789 | n_decoded = 1367, tg = 22.10 t/s, tg_3s = 21.97 t/s 42.46.384.638 I slot print_timing: id 3 | task 50789 | n_decoded = 1433, tg = 22.09 t/s, tg_3s = 21.87 t/s 42.49.425.308 I slot print_timing: id 3 | task 50789 | n_decoded = 1499, tg = 22.07 t/s, tg_3s = 21.71 t/s 42.52.436.354 I slot print_timing: id 3 | task 50789 | n_decoded = 1565, tg = 22.06 t/s, tg_3s = 21.92 t/s 42.55.444.872 I slot print_timing: id 3 | task 50789 | n_decoded = 1631, tg = 22.06 t/s, tg_3s = 21.94 t/s 42.58.466.981 I slot print_timing: id 3 | task 50789 | n_decoded = 1696, tg = 22.04 t/s, tg_3s = 21.51 t/s 43.01.490.325 I slot print_timing: id 3 | task 50789 | n_decoded = 1759, tg = 21.99 t/s, tg_3s = 20.84 t/s 43.04.508.881 I slot print_timing: id 3 | task 50789 | n_decoded = 1822, tg = 21.95 t/s, tg_3s = 20.87 t/s 43.07.538.470 I slot print_timing: id 3 | task 50789 | n_decoded = 1885, tg = 21.91 t/s, tg_3s = 20.79 t/s 43.10.570.758 I slot print_timing: id 3 | task 50789 | n_decoded = 1948, tg = 21.87 t/s, tg_3s = 20.78 t/s 43.13.618.590 I slot print_timing: id 3 | task 50789 | n_decoded = 2011, tg = 21.83 t/s, tg_3s = 20.67 t/s 43.16.660.196 I slot print_timing: id 3 | task 50789 | n_decoded = 2074, tg = 21.80 t/s, tg_3s = 20.71 t/s 43.19.702.016 I slot print_timing: id 3 | task 50789 | n_decoded = 2137, tg = 21.76 t/s, tg_3s = 20.71 t/s 43.22.708.668 I slot print_timing: id 3 | task 50789 | n_decoded = 2199, tg = 21.73 t/s, tg_3s = 20.62 t/s 43.25.728.338 I slot print_timing: id 3 | task 50789 | n_decoded = 2261, tg = 21.69 t/s, tg_3s = 20.53 t/s 43.28.750.514 I slot print_timing: id 3 | task 50789 | n_decoded = 2323, tg = 21.66 t/s, tg_3s = 20.52 t/s 43.31.781.293 I slot print_timing: id 3 | task 50789 | n_decoded = 2385, tg = 21.63 t/s, tg_3s = 20.46 t/s 43.34.791.738 I slot print_timing: id 3 | task 50789 | n_decoded = 2447, tg = 21.60 t/s, tg_3s = 20.59 t/s 43.37.816.173 I slot print_timing: id 3 | task 50789 | n_decoded = 2508, tg = 21.56 t/s, tg_3s = 20.17 t/s 43.40.837.275 I slot print_timing: id 3 | task 50789 | n_decoded = 2569, tg = 21.53 t/s, tg_3s = 20.19 t/s 43.43.883.541 I slot print_timing: id 3 | task 50789 | n_decoded = 2631, tg = 21.50 t/s, tg_3s = 20.35 t/s 43.46.929.164 I slot print_timing: id 3 | task 50789 | n_decoded = 2693, tg = 21.47 t/s, tg_3s = 20.36 t/s 43.49.962.745 I slot print_timing: id 3 | task 50789 | n_decoded = 2754, tg = 21.44 t/s, tg_3s = 20.11 t/s 43.52.991.797 I slot print_timing: id 3 | task 50789 | n_decoded = 2815, tg = 21.41 t/s, tg_3s = 20.14 t/s 43.55.992.321 I slot print_timing: id 3 | task 50789 | n_decoded = 2875, tg = 21.38 t/s, tg_3s = 20.00 t/s 43.58.993.852 I slot print_timing: id 3 | task 50789 | n_decoded = 2935, tg = 21.35 t/s, tg_3s = 19.99 t/s 44.02.002.076 I slot print_timing: id 3 | task 50789 | n_decoded = 2995, tg = 21.32 t/s, tg_3s = 19.95 t/s 44.05.003.447 I slot print_timing: id 3 | task 50789 | n_decoded = 3055, tg = 21.29 t/s, tg_3s = 19.99 t/s 44.08.005.493 I slot print_timing: id 3 | task 50789 | n_decoded = 3115, tg = 21.26 t/s, tg_3s = 19.99 t/s 44.11.006.015 I slot print_timing: id 3 | task 50789 | n_decoded = 3175, tg = 21.24 t/s, tg_3s = 20.00 t/s 44.14.020.528 I slot print_timing: id 3 | task 50789 | n_decoded = 3235, tg = 21.21 t/s, tg_3s = 19.90 t/s 44.17.045.065 I slot print_timing: id 3 | task 50789 | n_decoded = 3295, tg = 21.18 t/s, tg_3s = 19.84 t/s 44.20.045.595 I slot print_timing: id 3 | task 50789 | n_decoded = 3355, tg = 21.16 t/s, tg_3s = 20.00 t/s 44.23.052.832 I slot print_timing: id 3 | task 50789 | n_decoded = 3414, tg = 21.13 t/s, tg_3s = 19.62 t/s 44.26.084.906 I slot print_timing: id 3 | task 50789 | n_decoded = 3474, tg = 21.11 t/s, tg_3s = 19.79 t/s 44.29.125.723 I slot print_timing: id 3 | task 50789 | n_decoded = 3534, tg = 21.08 t/s, tg_3s = 19.73 t/s 44.32.167.083 I slot print_timing: id 3 | task 50789 | n_decoded = 3594, tg = 21.06 t/s, tg_3s = 19.73 t/s 44.35.212.542 I slot print_timing: id 3 | task 50789 | n_decoded = 3654, tg = 21.04 t/s, tg_3s = 19.70 t/s 44.38.261.475 I slot print_timing: id 3 | task 50789 | n_decoded = 3714, tg = 21.01 t/s, tg_3s = 19.68 t/s 44.41.294.286 I slot print_timing: id 3 | task 50789 | n_decoded = 3773, tg = 20.99 t/s, tg_3s = 19.45 t/s 44.44.322.834 I slot print_timing: id 3 | task 50789 | n_decoded = 3832, tg = 20.96 t/s, tg_3s = 19.48 t/s 44.47.360.159 I slot print_timing: id 3 | task 50789 | n_decoded = 3891, tg = 20.94 t/s, tg_3s = 19.42 t/s 44.50.369.040 I slot print_timing: id 3 | task 50789 | n_decoded = 3950, tg = 20.91 t/s, tg_3s = 19.61 t/s 44.53.397.631 I slot print_timing: id 3 | task 50789 | n_decoded = 4009, tg = 20.89 t/s, tg_3s = 19.48 t/s 44.56.423.423 I slot print_timing: id 3 | task 50789 | n_decoded = 4068, tg = 20.87 t/s, tg_3s = 19.50 t/s 44.57.861.729 I slot print_timing: id 3 | task 50789 | prompt eval time = 1192.71 ms / 41 tokens ( 29.09 ms per token, 34.38 tokens per second) 44.57.861.733 I slot print_timing: id 3 | task 50789 | eval time = 196357.73 ms / 4096 tokens ( 47.94 ms per token, 20.86 tokens per second) 44.57.861.733 I slot print_timing: id 3 | task 50789 | total time = 197550.45 ms / 4137 tokens 44.57.861.734 I slot print_timing: id 3 | task 50789 | graphs reused = 54622 44.57.861.766 I slot release: id 3 | task 50789 | stop processing: n_tokens = 4210, truncated = 0 44.57.861.796 I srv update_slots: all slots are idle 44.57.862.870 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 44.57.880.774 I srv operator(): Chat format: peg-native 44.57.886.327 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.705 (> 0.100 thold), f_keep = 0.018 44.57.886.328 I srv get_availabl: updating prompt cache 44.57.886.458 W srv prompt_save: - saving prompt with length 4210, total state size = 1019.659 MiB (draft: 0.000 MiB) 44.58.510.053 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.705 44.58.510.061 W srv update: - cache size limit reached, removing oldest entry (size = 1014.331 MiB) 44.58.593.618 I srv update: - cache state: 9 prompts, 8071.535 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 44.58.593.622 I srv update: - prompt 0x561d90d122a0: 4215 tokens, checkpoints: 0, 1020.870 MiB 44.58.593.623 I srv update: - prompt 0x561d90ada720: 4229 tokens, checkpoints: 0, 1024.261 MiB 44.58.593.624 I srv update: - prompt 0x561d90ef7dc0: 4209 tokens, checkpoints: 0, 1019.417 MiB 44.58.593.625 I srv update: - prompt 0x561d983db950: 2621 tokens, checkpoints: 0, 634.805 MiB 44.58.593.625 I srv update: - prompt 0x561d90a1b120: 4225 tokens, checkpoints: 0, 1023.292 MiB 44.58.593.626 I srv update: - prompt 0x561d90b66b80: 4213 tokens, checkpoints: 0, 1020.386 MiB 44.58.593.627 I srv update: - prompt 0x561d90a2a790: 1197 tokens, checkpoints: 0, 289.914 MiB 44.58.593.627 I srv update: - prompt 0x561d90e0e700: 4207 tokens, checkpoints: 0, 1018.932 MiB 44.58.593.628 I srv update: - prompt 0x561d90aa1a40: 4210 tokens, checkpoints: 0, 1019.659 MiB 44.58.593.630 I srv get_availabl: prompt cache update took 707.30 ms 44.58.595.910 I slot launch_slot_: id 3 | task 54886 | processing task, is_child = 0 44.58.595.912 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 44.58.595.914 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 44.58.595.936 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 44.58.595.936 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 44.58.595.948 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 44.58.595.948 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 45.03.883.530 I slot print_timing: id 3 | task 54886 | n_decoded = 100, tg = 22.60 t/s, tg_3s = 22.60 t/s 45.06.900.295 I slot print_timing: id 3 | task 54886 | n_decoded = 167, tg = 22.44 t/s, tg_3s = 22.21 t/s 45.09.905.652 I slot print_timing: id 3 | task 54886 | n_decoded = 234, tg = 22.40 t/s, tg_3s = 22.29 t/s 45.12.907.891 I slot print_timing: id 3 | task 54886 | n_decoded = 301, tg = 22.38 t/s, tg_3s = 22.32 t/s 45.15.911.295 I slot print_timing: id 3 | task 54886 | n_decoded = 368, tg = 22.37 t/s, tg_3s = 22.31 t/s 45.18.919.893 I slot print_timing: id 3 | task 54886 | n_decoded = 435, tg = 22.35 t/s, tg_3s = 22.27 t/s 45.21.963.755 I slot print_timing: id 3 | task 54886 | n_decoded = 502, tg = 22.31 t/s, tg_3s = 22.01 t/s 45.24.967.066 I slot print_timing: id 3 | task 54886 | n_decoded = 569, tg = 22.31 t/s, tg_3s = 22.31 t/s 45.27.977.002 I slot print_timing: id 3 | task 54886 | n_decoded = 636, tg = 22.30 t/s, tg_3s = 22.26 t/s 45.30.982.147 I slot print_timing: id 3 | task 54886 | n_decoded = 702, tg = 22.27 t/s, tg_3s = 21.96 t/s 45.33.992.699 I slot print_timing: id 3 | task 54886 | n_decoded = 768, tg = 22.24 t/s, tg_3s = 21.92 t/s 45.37.035.211 I slot print_timing: id 3 | task 54886 | n_decoded = 835, tg = 22.22 t/s, tg_3s = 22.02 t/s 45.40.055.807 I slot print_timing: id 3 | task 54886 | n_decoded = 901, tg = 22.19 t/s, tg_3s = 21.85 t/s 45.43.069.510 I slot print_timing: id 3 | task 54886 | n_decoded = 967, tg = 22.17 t/s, tg_3s = 21.90 t/s 45.46.074.097 I slot print_timing: id 3 | task 54886 | n_decoded = 1033, tg = 22.16 t/s, tg_3s = 21.97 t/s 45.49.082.898 I slot print_timing: id 3 | task 54886 | n_decoded = 1099, tg = 22.15 t/s, tg_3s = 21.94 t/s 45.52.087.555 I slot print_timing: id 3 | task 54886 | n_decoded = 1165, tg = 22.14 t/s, tg_3s = 21.97 t/s 45.55.131.388 I slot print_timing: id 3 | task 54886 | n_decoded = 1231, tg = 22.11 t/s, tg_3s = 21.68 t/s 45.58.149.523 I slot print_timing: id 3 | task 54886 | n_decoded = 1297, tg = 22.10 t/s, tg_3s = 21.87 t/s 46.01.163.234 I slot print_timing: id 3 | task 54886 | n_decoded = 1363, tg = 22.09 t/s, tg_3s = 21.90 t/s 46.04.192.312 I slot print_timing: id 3 | task 54886 | n_decoded = 1429, tg = 22.08 t/s, tg_3s = 21.79 t/s 46.07.216.328 I slot print_timing: id 3 | task 54886 | n_decoded = 1495, tg = 22.06 t/s, tg_3s = 21.83 t/s 46.10.227.245 I slot print_timing: id 3 | task 54886 | n_decoded = 1561, tg = 22.06 t/s, tg_3s = 21.92 t/s 46.13.243.196 I slot print_timing: id 3 | task 54886 | n_decoded = 1627, tg = 22.05 t/s, tg_3s = 21.88 t/s 46.16.286.700 I slot print_timing: id 3 | task 54886 | n_decoded = 1693, tg = 22.04 t/s, tg_3s = 21.69 t/s 46.19.300.874 I slot print_timing: id 3 | task 54886 | n_decoded = 1756, tg = 21.99 t/s, tg_3s = 20.90 t/s 46.22.324.308 I slot print_timing: id 3 | task 54886 | n_decoded = 1819, tg = 21.95 t/s, tg_3s = 20.84 t/s 46.25.345.437 I slot print_timing: id 3 | task 54886 | n_decoded = 1882, tg = 21.91 t/s, tg_3s = 20.85 t/s 46.28.375.580 I slot print_timing: id 3 | task 54886 | n_decoded = 1945, tg = 21.87 t/s, tg_3s = 20.79 t/s 46.31.416.839 I slot print_timing: id 3 | task 54886 | n_decoded = 2008, tg = 21.84 t/s, tg_3s = 20.72 t/s 46.34.453.871 I slot print_timing: id 3 | task 54886 | n_decoded = 2071, tg = 21.80 t/s, tg_3s = 20.74 t/s 46.37.496.841 I slot print_timing: id 3 | task 54886 | n_decoded = 2134, tg = 21.77 t/s, tg_3s = 20.70 t/s 46.40.500.741 I slot print_timing: id 3 | task 54886 | n_decoded = 2196, tg = 21.73 t/s, tg_3s = 20.64 t/s 46.43.534.416 I slot print_timing: id 3 | task 54886 | n_decoded = 2258, tg = 21.70 t/s, tg_3s = 20.44 t/s 46.46.556.304 I slot print_timing: id 3 | task 54886 | n_decoded = 2320, tg = 21.66 t/s, tg_3s = 20.52 t/s 46.49.582.918 I slot print_timing: id 3 | task 54886 | n_decoded = 2382, tg = 21.63 t/s, tg_3s = 20.48 t/s 46.52.601.318 I slot print_timing: id 3 | task 54886 | n_decoded = 2444, tg = 21.60 t/s, tg_3s = 20.54 t/s 46.55.645.501 I slot print_timing: id 3 | task 54886 | n_decoded = 2506, tg = 21.57 t/s, tg_3s = 20.37 t/s 46.58.649.652 I slot print_timing: id 3 | task 54886 | n_decoded = 2567, tg = 21.54 t/s, tg_3s = 20.31 t/s 47.01.692.883 I slot print_timing: id 3 | task 54886 | n_decoded = 2628, tg = 21.50 t/s, tg_3s = 20.04 t/s 47.04.702.545 I slot print_timing: id 3 | task 54886 | n_decoded = 2689, tg = 21.47 t/s, tg_3s = 20.27 t/s 47.07.738.105 I slot print_timing: id 3 | task 54886 | n_decoded = 2750, tg = 21.44 t/s, tg_3s = 20.10 t/s 47.10.767.826 I slot print_timing: id 3 | task 54886 | n_decoded = 2811, tg = 21.41 t/s, tg_3s = 20.13 t/s 47.13.800.833 I slot print_timing: id 3 | task 54886 | n_decoded = 2872, tg = 21.38 t/s, tg_3s = 20.11 t/s 47.16.831.773 I slot print_timing: id 3 | task 54886 | n_decoded = 2933, tg = 21.35 t/s, tg_3s = 20.13 t/s 47.19.851.389 I slot print_timing: id 3 | task 54886 | n_decoded = 2993, tg = 21.32 t/s, tg_3s = 19.87 t/s 47.22.857.186 I slot print_timing: id 3 | task 54886 | n_decoded = 3053, tg = 21.29 t/s, tg_3s = 19.96 t/s 47.25.872.641 I slot print_timing: id 3 | task 54886 | n_decoded = 3113, tg = 21.26 t/s, tg_3s = 19.90 t/s 47.28.895.328 I slot print_timing: id 3 | task 54886 | n_decoded = 3173, tg = 21.23 t/s, tg_3s = 19.85 t/s 47.31.910.185 I slot print_timing: id 3 | task 54886 | n_decoded = 3233, tg = 21.21 t/s, tg_3s = 19.90 t/s 47.34.917.617 I slot print_timing: id 3 | task 54886 | n_decoded = 3293, tg = 21.18 t/s, tg_3s = 19.95 t/s 47.37.924.355 I slot print_timing: id 3 | task 54886 | n_decoded = 3353, tg = 21.16 t/s, tg_3s = 19.96 t/s 47.40.924.627 I slot print_timing: id 3 | task 54886 | n_decoded = 3413, tg = 21.14 t/s, tg_3s = 20.00 t/s 47.43.937.515 I slot print_timing: id 3 | task 54886 | n_decoded = 3472, tg = 21.11 t/s, tg_3s = 19.58 t/s 47.46.980.762 I slot print_timing: id 3 | task 54886 | n_decoded = 3532, tg = 21.08 t/s, tg_3s = 19.72 t/s 47.50.020.417 I slot print_timing: id 3 | task 54886 | n_decoded = 3592, tg = 21.06 t/s, tg_3s = 19.74 t/s 47.53.060.996 I slot print_timing: id 3 | task 54886 | n_decoded = 3652, tg = 21.04 t/s, tg_3s = 19.73 t/s 47.56.102.326 I slot print_timing: id 3 | task 54886 | n_decoded = 3712, tg = 21.01 t/s, tg_3s = 19.73 t/s 47.59.126.847 I slot print_timing: id 3 | task 54886 | n_decoded = 3771, tg = 20.99 t/s, tg_3s = 19.51 t/s 48.02.130.636 I slot print_timing: id 3 | task 54886 | n_decoded = 3830, tg = 20.97 t/s, tg_3s = 19.64 t/s 48.05.135.831 I slot print_timing: id 3 | task 54886 | n_decoded = 3889, tg = 20.94 t/s, tg_3s = 19.63 t/s 48.08.145.117 I slot print_timing: id 3 | task 54886 | n_decoded = 3948, tg = 20.92 t/s, tg_3s = 19.61 t/s 48.11.168.094 I slot print_timing: id 3 | task 54886 | n_decoded = 4007, tg = 20.90 t/s, tg_3s = 19.52 t/s 48.14.195.432 I slot print_timing: id 3 | task 54886 | n_decoded = 4066, tg = 20.88 t/s, tg_3s = 19.49 t/s 48.15.748.033 I slot print_timing: id 3 | task 54886 | prompt eval time = 862.52 ms / 31 tokens ( 27.82 ms per token, 35.94 tokens per second) 48.15.748.040 I slot print_timing: id 3 | task 54886 | eval time = 196289.53 ms / 4096 tokens ( 47.92 ms per token, 20.87 tokens per second) 48.15.748.041 I slot print_timing: id 3 | task 54886 | total time = 197152.05 ms / 4127 tokens 48.15.748.042 I slot print_timing: id 3 | task 54886 | graphs reused = 58700 48.15.748.073 I slot release: id 3 | task 54886 | stop processing: n_tokens = 4200, truncated = 0 48.15.748.103 I srv update_slots: all slots are idle 48.15.749.244 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 48.15.768.165 I srv operator(): Chat format: peg-native 48.15.774.301 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.771 (> 0.100 thold), f_keep = 0.018 48.15.774.303 I srv get_availabl: updating prompt cache 48.15.774.472 W srv prompt_save: - saving prompt with length 4200, total state size = 1017.237 MiB (draft: 0.000 MiB) 48.16.428.491 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.771 48.16.428.499 W srv update: - cache size limit reached, removing oldest entry (size = 1020.870 MiB) 48.16.522.826 I srv update: - cache state: 9 prompts, 8067.902 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 48.16.522.831 I srv update: - prompt 0x561d90ada720: 4229 tokens, checkpoints: 0, 1024.261 MiB 48.16.522.832 I srv update: - prompt 0x561d90ef7dc0: 4209 tokens, checkpoints: 0, 1019.417 MiB 48.16.522.832 I srv update: - prompt 0x561d983db950: 2621 tokens, checkpoints: 0, 634.805 MiB 48.16.522.833 I srv update: - prompt 0x561d90a1b120: 4225 tokens, checkpoints: 0, 1023.292 MiB 48.16.522.833 I srv update: - prompt 0x561d90b66b80: 4213 tokens, checkpoints: 0, 1020.386 MiB 48.16.522.834 I srv update: - prompt 0x561d90a2a790: 1197 tokens, checkpoints: 0, 289.914 MiB 48.16.522.834 I srv update: - prompt 0x561d90e0e700: 4207 tokens, checkpoints: 0, 1018.932 MiB 48.16.522.835 I srv update: - prompt 0x561d90aa1a40: 4210 tokens, checkpoints: 0, 1019.659 MiB 48.16.522.835 I srv update: - prompt 0x561d90ad2ae0: 4200 tokens, checkpoints: 0, 1017.237 MiB 48.16.522.837 I srv get_availabl: prompt cache update took 748.53 ms 48.16.524.603 I slot launch_slot_: id 3 | task 58983 | processing task, is_child = 0 48.16.524.605 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 48.16.524.606 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 48.16.524.629 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 48.16.524.629 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 48.16.524.644 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 48.16.524.646 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 48.21.584.945 I slot print_timing: id 3 | task 58983 | n_decoded = 100, tg = 22.55 t/s, tg_3s = 22.55 t/s 48.24.594.949 I slot print_timing: id 3 | task 58983 | n_decoded = 167, tg = 22.43 t/s, tg_3s = 22.26 t/s 48.27.600.395 I slot print_timing: id 3 | task 58983 | n_decoded = 234, tg = 22.39 t/s, tg_3s = 22.29 t/s 48.30.639.656 I slot print_timing: id 3 | task 58983 | n_decoded = 301, tg = 22.31 t/s, tg_3s = 22.04 t/s 48.33.666.765 I slot print_timing: id 3 | task 58983 | n_decoded = 369, tg = 22.34 t/s, tg_3s = 22.46 t/s 48.36.709.799 I slot print_timing: id 3 | task 58983 | n_decoded = 437, tg = 22.34 t/s, tg_3s = 22.35 t/s 48.39.752.070 I slot print_timing: id 3 | task 58983 | n_decoded = 505, tg = 22.34 t/s, tg_3s = 22.35 t/s 48.42.793.811 I slot print_timing: id 3 | task 58983 | n_decoded = 572, tg = 22.31 t/s, tg_3s = 22.03 t/s 48.45.836.790 I slot print_timing: id 3 | task 58983 | n_decoded = 640, tg = 22.31 t/s, tg_3s = 22.35 t/s 48.48.870.024 I slot print_timing: id 3 | task 58983 | n_decoded = 707, tg = 22.29 t/s, tg_3s = 22.09 t/s 48.50.142.307 I slot print_timing: id 3 | task 58983 | prompt eval time = 626.12 ms / 22 tokens ( 28.46 ms per token, 35.14 tokens per second) 48.50.142.311 I slot print_timing: id 3 | task 58983 | eval time = 32991.50 ms / 735 tokens ( 44.89 ms per token, 22.28 tokens per second) 48.50.142.311 I slot print_timing: id 3 | task 58983 | total time = 33617.62 ms / 757 tokens 48.50.142.312 I slot print_timing: id 3 | task 58983 | graphs reused = 59430 48.50.142.345 I slot release: id 3 | task 58983 | stop processing: n_tokens = 830, truncated = 0 48.50.142.362 I srv update_slots: all slots are idle 48.50.143.236 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 48.50.162.742 I srv operator(): Chat format: peg-native 48.50.168.855 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.655 (> 0.100 thold), f_keep = 0.089 48.50.168.857 I srv get_availabl: updating prompt cache 48.50.168.913 W srv prompt_save: - saving prompt with length 830, total state size = 201.027 MiB (draft: 0.000 MiB) 48.50.322.295 I srv load: - looking for better prompt, base f_keep = 0.089, sim = 0.655 48.50.322.303 W srv update: - cache size limit reached, removing oldest entry (size = 1024.261 MiB) 48.50.416.320 I srv update: - cache state: 9 prompts, 7244.668 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 48.50.416.324 I srv update: - prompt 0x561d90ef7dc0: 4209 tokens, checkpoints: 0, 1019.417 MiB 48.50.416.325 I srv update: - prompt 0x561d983db950: 2621 tokens, checkpoints: 0, 634.805 MiB 48.50.416.326 I srv update: - prompt 0x561d90a1b120: 4225 tokens, checkpoints: 0, 1023.292 MiB 48.50.416.327 I srv update: - prompt 0x561d90b66b80: 4213 tokens, checkpoints: 0, 1020.386 MiB 48.50.416.327 I srv update: - prompt 0x561d90a2a790: 1197 tokens, checkpoints: 0, 289.914 MiB 48.50.416.328 I srv update: - prompt 0x561d90e0e700: 4207 tokens, checkpoints: 0, 1018.932 MiB 48.50.416.329 I srv update: - prompt 0x561d90aa1a40: 4210 tokens, checkpoints: 0, 1019.659 MiB 48.50.416.329 I srv update: - prompt 0x561d90ad2ae0: 4200 tokens, checkpoints: 0, 1017.237 MiB 48.50.416.330 I srv update: - prompt 0x561d90afb3d0: 830 tokens, checkpoints: 0, 201.027 MiB 48.50.416.332 I srv get_availabl: prompt cache update took 247.47 ms 48.50.418.190 I slot launch_slot_: id 3 | task 59719 | processing task, is_child = 0 48.50.418.192 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 48.50.418.193 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 48.50.418.207 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 48.50.418.207 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 48.50.418.215 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 48.50.418.216 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 48.55.978.074 I slot print_timing: id 3 | task 59719 | n_decoded = 100, tg = 22.30 t/s, tg_3s = 22.30 t/s 48.58.979.996 I slot print_timing: id 3 | task 59719 | n_decoded = 167, tg = 22.31 t/s, tg_3s = 22.32 t/s 49.02.014.053 I slot print_timing: id 3 | task 59719 | n_decoded = 235, tg = 22.34 t/s, tg_3s = 22.41 t/s 49.05.051.242 I slot print_timing: id 3 | task 59719 | n_decoded = 303, tg = 22.35 t/s, tg_3s = 22.39 t/s 49.08.090.055 I slot print_timing: id 3 | task 59719 | n_decoded = 371, tg = 22.35 t/s, tg_3s = 22.38 t/s 49.11.106.187 I slot print_timing: id 3 | task 59719 | n_decoded = 438, tg = 22.33 t/s, tg_3s = 22.21 t/s 49.14.111.780 I slot print_timing: id 3 | task 59719 | n_decoded = 505, tg = 22.33 t/s, tg_3s = 22.29 t/s 49.17.118.231 I slot print_timing: id 3 | task 59719 | n_decoded = 572, tg = 22.32 t/s, tg_3s = 22.29 t/s 49.20.157.591 I slot print_timing: id 3 | task 59719 | n_decoded = 640, tg = 22.33 t/s, tg_3s = 22.37 t/s 49.23.193.434 I slot print_timing: id 3 | task 59719 | n_decoded = 707, tg = 22.30 t/s, tg_3s = 22.07 t/s 49.26.232.158 I slot print_timing: id 3 | task 59719 | n_decoded = 774, tg = 22.28 t/s, tg_3s = 22.05 t/s 49.29.264.549 I slot print_timing: id 3 | task 59719 | n_decoded = 840, tg = 22.24 t/s, tg_3s = 21.77 t/s 49.32.280.182 I slot print_timing: id 3 | task 59719 | n_decoded = 906, tg = 22.21 t/s, tg_3s = 21.89 t/s 49.35.287.123 I slot print_timing: id 3 | task 59719 | n_decoded = 972, tg = 22.20 t/s, tg_3s = 21.95 t/s 49.38.330.194 I slot print_timing: id 3 | task 59719 | n_decoded = 1039, tg = 22.18 t/s, tg_3s = 22.02 t/s 49.41.332.152 I slot print_timing: id 3 | task 59719 | n_decoded = 1105, tg = 22.17 t/s, tg_3s = 21.99 t/s 49.44.343.463 I slot print_timing: id 3 | task 59719 | n_decoded = 1171, tg = 22.16 t/s, tg_3s = 21.92 t/s 49.47.350.967 I slot print_timing: id 3 | task 59719 | n_decoded = 1237, tg = 22.15 t/s, tg_3s = 21.95 t/s 49.50.354.118 I slot print_timing: id 3 | task 59719 | n_decoded = 1302, tg = 22.12 t/s, tg_3s = 21.64 t/s 49.53.393.694 I slot print_timing: id 3 | task 59719 | n_decoded = 1367, tg = 22.08 t/s, tg_3s = 21.38 t/s 49.56.436.534 I slot print_timing: id 3 | task 59719 | n_decoded = 1433, tg = 22.07 t/s, tg_3s = 21.69 t/s 49.59.444.787 I slot print_timing: id 3 | task 59719 | n_decoded = 1498, tg = 22.05 t/s, tg_3s = 21.61 t/s 50.02.467.614 I slot print_timing: id 3 | task 59719 | n_decoded = 1564, tg = 22.04 t/s, tg_3s = 21.83 t/s 50.05.500.585 I slot print_timing: id 3 | task 59719 | n_decoded = 1630, tg = 22.03 t/s, tg_3s = 21.76 t/s 50.08.507.375 I slot print_timing: id 3 | task 59719 | n_decoded = 1694, tg = 22.00 t/s, tg_3s = 21.29 t/s 50.11.547.454 I slot print_timing: id 3 | task 59719 | n_decoded = 1757, tg = 21.95 t/s, tg_3s = 20.72 t/s 50.14.560.697 I slot print_timing: id 3 | task 59719 | n_decoded = 1820, tg = 21.91 t/s, tg_3s = 20.91 t/s 50.17.572.519 I slot print_timing: id 3 | task 59719 | n_decoded = 1883, tg = 21.88 t/s, tg_3s = 20.92 t/s 50.20.596.028 I slot print_timing: id 3 | task 59719 | n_decoded = 1946, tg = 21.84 t/s, tg_3s = 20.84 t/s 50.23.624.064 I slot print_timing: id 3 | task 59719 | n_decoded = 2009, tg = 21.81 t/s, tg_3s = 20.81 t/s 50.26.650.970 I slot print_timing: id 3 | task 59719 | n_decoded = 2072, tg = 21.77 t/s, tg_3s = 20.81 t/s 50.29.687.009 I slot print_timing: id 3 | task 59719 | n_decoded = 2135, tg = 21.74 t/s, tg_3s = 20.75 t/s 50.32.714.543 I slot print_timing: id 3 | task 59719 | n_decoded = 2197, tg = 21.71 t/s, tg_3s = 20.48 t/s 50.35.721.631 I slot print_timing: id 3 | task 59719 | n_decoded = 2258, tg = 21.66 t/s, tg_3s = 20.29 t/s 50.38.737.321 I slot print_timing: id 3 | task 59719 | n_decoded = 2320, tg = 21.63 t/s, tg_3s = 20.56 t/s 50.41.755.560 I slot print_timing: id 3 | task 59719 | n_decoded = 2382, tg = 21.60 t/s, tg_3s = 20.54 t/s 50.44.778.054 I slot print_timing: id 3 | task 59719 | n_decoded = 2444, tg = 21.57 t/s, tg_3s = 20.51 t/s 50.47.792.849 I slot print_timing: id 3 | task 59719 | n_decoded = 2505, tg = 21.54 t/s, tg_3s = 20.23 t/s 50.50.817.105 I slot print_timing: id 3 | task 59719 | n_decoded = 2566, tg = 21.50 t/s, tg_3s = 20.17 t/s 50.53.829.459 I slot print_timing: id 3 | task 59719 | n_decoded = 2627, tg = 21.47 t/s, tg_3s = 20.25 t/s 50.56.832.117 I slot print_timing: id 3 | task 59719 | n_decoded = 2688, tg = 21.45 t/s, tg_3s = 20.32 t/s 50.59.864.464 I slot print_timing: id 3 | task 59719 | n_decoded = 2749, tg = 21.41 t/s, tg_3s = 20.12 t/s 51.02.889.364 I slot print_timing: id 3 | task 59719 | n_decoded = 2810, tg = 21.39 t/s, tg_3s = 20.17 t/s 51.05.917.257 I slot print_timing: id 3 | task 59719 | n_decoded = 2871, tg = 21.36 t/s, tg_3s = 20.15 t/s 51.08.941.404 I slot print_timing: id 3 | task 59719 | n_decoded = 2932, tg = 21.33 t/s, tg_3s = 20.17 t/s 51.11.941.441 I slot print_timing: id 3 | task 59719 | n_decoded = 2992, tg = 21.30 t/s, tg_3s = 20.00 t/s 51.14.957.415 I slot print_timing: id 3 | task 59719 | n_decoded = 3052, tg = 21.27 t/s, tg_3s = 19.89 t/s 51.18.007.429 I slot print_timing: id 3 | task 59719 | n_decoded = 3113, tg = 21.25 t/s, tg_3s = 20.00 t/s 51.21.032.423 I slot print_timing: id 3 | task 59719 | n_decoded = 3173, tg = 21.22 t/s, tg_3s = 19.83 t/s 51.24.055.546 I slot print_timing: id 3 | task 59719 | n_decoded = 3233, tg = 21.19 t/s, tg_3s = 19.85 t/s 51.27.059.470 I slot print_timing: id 3 | task 59719 | n_decoded = 3293, tg = 21.17 t/s, tg_3s = 19.97 t/s 51.30.104.624 I slot print_timing: id 3 | task 59719 | n_decoded = 3353, tg = 21.14 t/s, tg_3s = 19.70 t/s 51.33.121.362 I slot print_timing: id 3 | task 59719 | n_decoded = 3413, tg = 21.12 t/s, tg_3s = 19.89 t/s 51.36.133.119 I slot print_timing: id 3 | task 59719 | n_decoded = 3473, tg = 21.09 t/s, tg_3s = 19.92 t/s 51.39.150.964 I slot print_timing: id 3 | task 59719 | n_decoded = 3532, tg = 21.07 t/s, tg_3s = 19.55 t/s 51.42.177.764 I slot print_timing: id 3 | task 59719 | n_decoded = 3591, tg = 21.04 t/s, tg_3s = 19.49 t/s 51.45.223.029 I slot print_timing: id 3 | task 59719 | n_decoded = 3651, tg = 21.02 t/s, tg_3s = 19.70 t/s 51.48.227.143 I slot print_timing: id 3 | task 59719 | n_decoded = 3710, tg = 20.99 t/s, tg_3s = 19.64 t/s 51.51.233.295 I slot print_timing: id 3 | task 59719 | n_decoded = 3768, tg = 20.96 t/s, tg_3s = 19.29 t/s 51.54.240.422 I slot print_timing: id 3 | task 59719 | n_decoded = 3827, tg = 20.94 t/s, tg_3s = 19.62 t/s 51.57.270.487 I slot print_timing: id 3 | task 59719 | n_decoded = 3886, tg = 20.92 t/s, tg_3s = 19.47 t/s 52.00.286.363 I slot print_timing: id 3 | task 59719 | n_decoded = 3945, tg = 20.90 t/s, tg_3s = 19.56 t/s 52.03.305.156 I slot print_timing: id 3 | task 59719 | n_decoded = 4004, tg = 20.87 t/s, tg_3s = 19.54 t/s 52.06.329.014 I slot print_timing: id 3 | task 59719 | n_decoded = 4063, tg = 20.85 t/s, tg_3s = 19.51 t/s 52.08.018.588 I slot print_timing: id 3 | task 59719 | prompt eval time = 1075.72 ms / 39 tokens ( 27.58 ms per token, 36.25 tokens per second) 52.08.018.592 I slot print_timing: id 3 | task 59719 | eval time = 196524.62 ms / 4096 tokens ( 47.98 ms per token, 20.84 tokens per second) 52.08.018.593 I slot print_timing: id 3 | task 59719 | total time = 197600.34 ms / 4135 tokens 52.08.018.594 I slot print_timing: id 3 | task 59719 | graphs reused = 63508 52.08.018.621 I slot release: id 3 | task 59719 | stop processing: n_tokens = 4208, truncated = 0 52.08.018.646 I srv update_slots: all slots are idle 52.08.019.929 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 52.08.039.953 I srv operator(): Chat format: peg-native 52.08.046.297 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.705 (> 0.100 thold), f_keep = 0.018 52.08.046.299 I srv get_availabl: updating prompt cache 52.08.046.482 W srv prompt_save: - saving prompt with length 4208, total state size = 1019.175 MiB (draft: 0.000 MiB) 52.08.673.332 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.705 52.08.673.340 W srv update: - cache size limit reached, removing oldest entry (size = 1019.417 MiB) 52.08.768.420 I srv update: - cache state: 9 prompts, 7244.426 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 52.08.768.425 I srv update: - prompt 0x561d983db950: 2621 tokens, checkpoints: 0, 634.805 MiB 52.08.768.425 I srv update: - prompt 0x561d90a1b120: 4225 tokens, checkpoints: 0, 1023.292 MiB 52.08.768.426 I srv update: - prompt 0x561d90b66b80: 4213 tokens, checkpoints: 0, 1020.386 MiB 52.08.768.427 I srv update: - prompt 0x561d90a2a790: 1197 tokens, checkpoints: 0, 289.914 MiB 52.08.768.427 I srv update: - prompt 0x561d90e0e700: 4207 tokens, checkpoints: 0, 1018.932 MiB 52.08.768.428 I srv update: - prompt 0x561d90aa1a40: 4210 tokens, checkpoints: 0, 1019.659 MiB 52.08.768.428 I srv update: - prompt 0x561d90ad2ae0: 4200 tokens, checkpoints: 0, 1017.237 MiB 52.08.768.429 I srv update: - prompt 0x561d90afb3d0: 830 tokens, checkpoints: 0, 201.027 MiB 52.08.768.429 I srv update: - prompt 0x561d90d37f60: 4208 tokens, checkpoints: 0, 1019.175 MiB 52.08.768.432 I srv get_availabl: prompt cache update took 722.13 ms 52.08.770.324 I slot launch_slot_: id 3 | task 63816 | processing task, is_child = 0 52.08.770.326 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 52.08.770.327 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 52.08.770.349 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 52.08.770.350 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 52.08.770.361 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 52.08.770.361 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 52.14.180.426 I slot print_timing: id 3 | task 63816 | n_decoded = 100, tg = 22.36 t/s, tg_3s = 22.36 t/s 52.17.183.592 I slot print_timing: id 3 | task 63816 | n_decoded = 167, tg = 22.34 t/s, tg_3s = 22.31 t/s 52.20.187.177 I slot print_timing: id 3 | task 63816 | n_decoded = 234, tg = 22.33 t/s, tg_3s = 22.31 t/s 52.23.224.195 I slot print_timing: id 3 | task 63816 | n_decoded = 302, tg = 22.34 t/s, tg_3s = 22.39 t/s 52.26.259.438 I slot print_timing: id 3 | task 63816 | n_decoded = 368, tg = 22.23 t/s, tg_3s = 21.74 t/s 52.29.262.546 I slot print_timing: id 3 | task 63816 | n_decoded = 434, tg = 22.20 t/s, tg_3s = 21.98 t/s 52.32.271.810 I slot print_timing: id 3 | task 63816 | n_decoded = 501, tg = 22.20 t/s, tg_3s = 22.26 t/s 52.35.280.679 I slot print_timing: id 3 | task 63816 | n_decoded = 568, tg = 22.21 t/s, tg_3s = 22.27 t/s 52.38.305.560 I slot print_timing: id 3 | task 63816 | n_decoded = 635, tg = 22.21 t/s, tg_3s = 22.15 t/s 52.41.336.456 I slot print_timing: id 3 | task 63816 | n_decoded = 702, tg = 22.20 t/s, tg_3s = 22.11 t/s 52.44.360.280 I slot print_timing: id 3 | task 63816 | n_decoded = 768, tg = 22.16 t/s, tg_3s = 21.83 t/s 52.47.374.514 I slot print_timing: id 3 | task 63816 | n_decoded = 834, tg = 22.14 t/s, tg_3s = 21.90 t/s 52.50.412.906 I slot print_timing: id 3 | task 63816 | n_decoded = 901, tg = 22.14 t/s, tg_3s = 22.05 t/s 52.53.422.345 I slot print_timing: id 3 | task 63816 | n_decoded = 967, tg = 22.12 t/s, tg_3s = 21.93 t/s 52.56.425.003 I slot print_timing: id 3 | task 63816 | n_decoded = 1033, tg = 22.11 t/s, tg_3s = 21.98 t/s 52.59.432.424 I slot print_timing: id 3 | task 63816 | n_decoded = 1099, tg = 22.10 t/s, tg_3s = 21.95 t/s 53.02.460.514 I slot print_timing: id 3 | task 63816 | n_decoded = 1165, tg = 22.08 t/s, tg_3s = 21.80 t/s 53.05.477.088 I slot print_timing: id 3 | task 63816 | n_decoded = 1231, tg = 22.07 t/s, tg_3s = 21.88 t/s 53.08.484.852 I slot print_timing: id 3 | task 63816 | n_decoded = 1297, tg = 22.07 t/s, tg_3s = 21.94 t/s 53.11.501.350 I slot print_timing: id 3 | task 63816 | n_decoded = 1363, tg = 22.06 t/s, tg_3s = 21.88 t/s 53.14.504.254 I slot print_timing: id 3 | task 63816 | n_decoded = 1428, tg = 22.04 t/s, tg_3s = 21.65 t/s 53.17.541.220 I slot print_timing: id 3 | task 63816 | n_decoded = 1494, tg = 22.02 t/s, tg_3s = 21.73 t/s 53.20.559.353 I slot print_timing: id 3 | task 63816 | n_decoded = 1560, tg = 22.02 t/s, tg_3s = 21.87 t/s 53.23.590.129 I slot print_timing: id 3 | task 63816 | n_decoded = 1625, tg = 21.99 t/s, tg_3s = 21.45 t/s 53.26.624.428 I slot print_timing: id 3 | task 63816 | n_decoded = 1691, tg = 21.99 t/s, tg_3s = 21.75 t/s 53.29.650.513 I slot print_timing: id 3 | task 63816 | n_decoded = 1754, tg = 21.94 t/s, tg_3s = 20.82 t/s 53.32.659.358 I slot print_timing: id 3 | task 63816 | n_decoded = 1816, tg = 21.89 t/s, tg_3s = 20.61 t/s 53.35.691.242 I slot print_timing: id 3 | task 63816 | n_decoded = 1879, tg = 21.85 t/s, tg_3s = 20.78 t/s 53.38.708.159 I slot print_timing: id 3 | task 63816 | n_decoded = 1942, tg = 21.82 t/s, tg_3s = 20.88 t/s 53.41.709.430 I slot print_timing: id 3 | task 63816 | n_decoded = 2004, tg = 21.78 t/s, tg_3s = 20.66 t/s 53.44.741.901 I slot print_timing: id 3 | task 63816 | n_decoded = 2067, tg = 21.75 t/s, tg_3s = 20.78 t/s 53.47.745.850 I slot print_timing: id 3 | task 63816 | n_decoded = 2129, tg = 21.72 t/s, tg_3s = 20.64 t/s 53.50.765.467 I slot print_timing: id 3 | task 63816 | n_decoded = 2191, tg = 21.68 t/s, tg_3s = 20.53 t/s 53.53.788.500 I slot print_timing: id 3 | task 63816 | n_decoded = 2253, tg = 21.65 t/s, tg_3s = 20.51 t/s 53.56.824.123 I slot print_timing: id 3 | task 63816 | n_decoded = 2315, tg = 21.61 t/s, tg_3s = 20.42 t/s 53.59.851.602 I slot print_timing: id 3 | task 63816 | n_decoded = 2377, tg = 21.58 t/s, tg_3s = 20.48 t/s 54.02.871.606 I slot print_timing: id 3 | task 63816 | n_decoded = 2439, tg = 21.55 t/s, tg_3s = 20.53 t/s 54.05.878.314 I slot print_timing: id 3 | task 63816 | n_decoded = 2500, tg = 21.52 t/s, tg_3s = 20.29 t/s 54.08.888.494 I slot print_timing: id 3 | task 63816 | n_decoded = 2561, tg = 21.49 t/s, tg_3s = 20.26 t/s 54.11.898.318 I slot print_timing: id 3 | task 63816 | n_decoded = 2622, tg = 21.46 t/s, tg_3s = 20.27 t/s 54.14.945.368 I slot print_timing: id 3 | task 63816 | n_decoded = 2684, tg = 21.43 t/s, tg_3s = 20.35 t/s 54.17.981.337 I slot print_timing: id 3 | task 63816 | n_decoded = 2745, tg = 21.40 t/s, tg_3s = 20.09 t/s 54.21.021.365 I slot print_timing: id 3 | task 63816 | n_decoded = 2806, tg = 21.37 t/s, tg_3s = 20.07 t/s 54.24.059.915 I slot print_timing: id 3 | task 63816 | n_decoded = 2867, tg = 21.34 t/s, tg_3s = 20.08 t/s 54.27.089.559 I slot print_timing: id 3 | task 63816 | n_decoded = 2928, tg = 21.31 t/s, tg_3s = 20.13 t/s 54.30.102.524 I slot print_timing: id 3 | task 63816 | n_decoded = 2988, tg = 21.28 t/s, tg_3s = 19.91 t/s 54.33.116.079 I slot print_timing: id 3 | task 63816 | n_decoded = 3048, tg = 21.25 t/s, tg_3s = 19.91 t/s 54.36.116.271 I slot print_timing: id 3 | task 63816 | n_decoded = 3108, tg = 21.23 t/s, tg_3s = 20.00 t/s 54.39.166.299 I slot print_timing: id 3 | task 63816 | n_decoded = 3169, tg = 21.20 t/s, tg_3s = 20.00 t/s 54.42.177.463 I slot print_timing: id 3 | task 63816 | n_decoded = 3229, tg = 21.18 t/s, tg_3s = 19.93 t/s 54.45.180.415 I slot print_timing: id 3 | task 63816 | n_decoded = 3289, tg = 21.15 t/s, tg_3s = 19.98 t/s 54.48.182.490 I slot print_timing: id 3 | task 63816 | n_decoded = 3349, tg = 21.13 t/s, tg_3s = 19.99 t/s 54.51.183.607 I slot print_timing: id 3 | task 63816 | n_decoded = 3409, tg = 21.11 t/s, tg_3s = 19.99 t/s 54.54.188.582 I slot print_timing: id 3 | task 63816 | n_decoded = 3469, tg = 21.09 t/s, tg_3s = 19.97 t/s 54.57.193.153 I slot print_timing: id 3 | task 63816 | n_decoded = 3528, tg = 21.06 t/s, tg_3s = 19.64 t/s 55.00.203.679 I slot print_timing: id 3 | task 63816 | n_decoded = 3587, tg = 21.04 t/s, tg_3s = 19.60 t/s 55.03.205.926 I slot print_timing: id 3 | task 63816 | n_decoded = 3646, tg = 21.01 t/s, tg_3s = 19.65 t/s 55.06.212.613 I slot print_timing: id 3 | task 63816 | n_decoded = 3705, tg = 20.99 t/s, tg_3s = 19.62 t/s 55.09.242.019 I slot print_timing: id 3 | task 63816 | n_decoded = 3764, tg = 20.97 t/s, tg_3s = 19.48 t/s 55.12.250.211 I slot print_timing: id 3 | task 63816 | n_decoded = 3823, tg = 20.94 t/s, tg_3s = 19.61 t/s 55.15.256.830 I slot print_timing: id 3 | task 63816 | n_decoded = 3882, tg = 20.92 t/s, tg_3s = 19.62 t/s 55.18.266.100 I slot print_timing: id 3 | task 63816 | n_decoded = 3941, tg = 20.90 t/s, tg_3s = 19.61 t/s 55.21.287.167 I slot print_timing: id 3 | task 63816 | n_decoded = 4000, tg = 20.88 t/s, tg_3s = 19.53 t/s 55.24.303.418 I slot print_timing: id 3 | task 63816 | n_decoded = 4059, tg = 20.86 t/s, tg_3s = 19.56 t/s 55.26.208.370 I slot print_timing: id 3 | task 63816 | prompt eval time = 938.23 ms / 31 tokens ( 30.27 ms per token, 33.04 tokens per second) 55.26.208.375 I slot print_timing: id 3 | task 63816 | eval time = 196499.74 ms / 4096 tokens ( 47.97 ms per token, 20.84 tokens per second) 55.26.208.376 I slot print_timing: id 3 | task 63816 | total time = 197437.98 ms / 4127 tokens 55.26.208.376 I slot print_timing: id 3 | task 63816 | graphs reused = 67586 55.26.208.410 I slot release: id 3 | task 63816 | stop processing: n_tokens = 4200, truncated = 0 55.26.208.442 I srv update_slots: all slots are idle 55.26.209.588 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 55.26.229.046 I srv operator(): Chat format: peg-native 55.26.234.757 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.673 (> 0.100 thold), f_keep = 0.018 55.26.234.759 I srv get_availabl: updating prompt cache 55.26.234.889 W srv prompt_save: - saving prompt with length 4200, total state size = 1017.237 MiB (draft: 0.000 MiB) 55.26.854.629 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.673 55.26.854.638 W srv update: - cache size limit reached, removing oldest entry (size = 634.805 MiB) 55.26.917.406 I srv update: - cache state: 9 prompts, 7626.858 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 55.26.917.410 I srv update: - prompt 0x561d90a1b120: 4225 tokens, checkpoints: 0, 1023.292 MiB 55.26.917.411 I srv update: - prompt 0x561d90b66b80: 4213 tokens, checkpoints: 0, 1020.386 MiB 55.26.917.412 I srv update: - prompt 0x561d90a2a790: 1197 tokens, checkpoints: 0, 289.914 MiB 55.26.917.412 I srv update: - prompt 0x561d90e0e700: 4207 tokens, checkpoints: 0, 1018.932 MiB 55.26.917.413 I srv update: - prompt 0x561d90aa1a40: 4210 tokens, checkpoints: 0, 1019.659 MiB 55.26.917.414 I srv update: - prompt 0x561d90ad2ae0: 4200 tokens, checkpoints: 0, 1017.237 MiB 55.26.917.414 I srv update: - prompt 0x561d90afb3d0: 830 tokens, checkpoints: 0, 201.027 MiB 55.26.917.415 I srv update: - prompt 0x561d90d37f60: 4208 tokens, checkpoints: 0, 1019.175 MiB 55.26.917.415 I srv update: - prompt 0x561d97e923e0: 4200 tokens, checkpoints: 0, 1017.237 MiB 55.26.917.417 I srv get_availabl: prompt cache update took 682.66 ms 55.26.919.422 I slot launch_slot_: id 3 | task 67913 | processing task, is_child = 0 55.26.919.424 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 55.26.919.425 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 55.26.919.450 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 55.26.919.450 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 55.26.919.465 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 55.26.919.466 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 55.32.398.176 I slot print_timing: id 3 | task 67913 | n_decoded = 100, tg = 22.55 t/s, tg_3s = 22.55 t/s 55.35.418.627 I slot print_timing: id 3 | task 67913 | n_decoded = 167, tg = 22.40 t/s, tg_3s = 22.18 t/s 55.38.458.473 I slot print_timing: id 3 | task 67913 | n_decoded = 234, tg = 22.30 t/s, tg_3s = 22.04 t/s 55.41.499.475 I slot print_timing: id 3 | task 67913 | n_decoded = 301, tg = 22.24 t/s, tg_3s = 22.03 t/s 55.44.511.730 I slot print_timing: id 3 | task 67913 | n_decoded = 368, tg = 22.24 t/s, tg_3s = 22.24 t/s 55.47.519.660 I slot print_timing: id 3 | task 67913 | n_decoded = 435, tg = 22.24 t/s, tg_3s = 22.27 t/s 55.50.522.775 I slot print_timing: id 3 | task 67913 | n_decoded = 502, tg = 22.25 t/s, tg_3s = 22.31 t/s 55.53.525.739 I slot print_timing: id 3 | task 67913 | n_decoded = 569, tg = 22.26 t/s, tg_3s = 22.31 t/s 55.56.555.924 I slot print_timing: id 3 | task 67913 | n_decoded = 636, tg = 22.24 t/s, tg_3s = 22.11 t/s 55.59.577.508 I slot print_timing: id 3 | task 67913 | n_decoded = 702, tg = 22.21 t/s, tg_3s = 21.84 t/s 56.02.612.500 I slot print_timing: id 3 | task 67913 | n_decoded = 769, tg = 22.19 t/s, tg_3s = 22.08 t/s 56.05.649.779 I slot print_timing: id 3 | task 67913 | n_decoded = 836, tg = 22.18 t/s, tg_3s = 22.06 t/s 56.08.687.188 I slot print_timing: id 3 | task 67913 | n_decoded = 903, tg = 22.17 t/s, tg_3s = 22.06 t/s 56.11.703.805 I slot print_timing: id 3 | task 67913 | n_decoded = 969, tg = 22.15 t/s, tg_3s = 21.88 t/s 56.14.705.894 I slot print_timing: id 3 | task 67913 | n_decoded = 1035, tg = 22.14 t/s, tg_3s = 21.98 t/s 56.17.723.176 I slot print_timing: id 3 | task 67913 | n_decoded = 1101, tg = 22.13 t/s, tg_3s = 21.87 t/s 56.20.740.040 I slot print_timing: id 3 | task 67913 | n_decoded = 1166, tg = 22.09 t/s, tg_3s = 21.55 t/s 56.23.765.070 I slot print_timing: id 3 | task 67913 | n_decoded = 1232, tg = 22.08 t/s, tg_3s = 21.82 t/s 56.26.792.523 I slot print_timing: id 3 | task 67913 | n_decoded = 1298, tg = 22.06 t/s, tg_3s = 21.80 t/s 56.29.826.108 I slot print_timing: id 3 | task 67913 | n_decoded = 1364, tg = 22.05 t/s, tg_3s = 21.76 t/s 56.32.853.603 I slot print_timing: id 3 | task 67913 | n_decoded = 1430, tg = 22.04 t/s, tg_3s = 21.80 t/s 56.35.872.261 I slot print_timing: id 3 | task 67913 | n_decoded = 1496, tg = 22.03 t/s, tg_3s = 21.86 t/s 56.38.893.392 I slot print_timing: id 3 | task 67913 | n_decoded = 1562, tg = 22.02 t/s, tg_3s = 21.85 t/s 56.41.912.204 I slot print_timing: id 3 | task 67913 | n_decoded = 1628, tg = 22.02 t/s, tg_3s = 21.86 t/s 56.44.958.019 I slot print_timing: id 3 | task 67913 | n_decoded = 1693, tg = 21.99 t/s, tg_3s = 21.34 t/s 56.48.005.643 I slot print_timing: id 3 | task 67913 | n_decoded = 1756, tg = 21.94 t/s, tg_3s = 20.67 t/s 56.51.045.568 I slot print_timing: id 3 | task 67913 | n_decoded = 1819, tg = 21.89 t/s, tg_3s = 20.72 t/s 56.54.079.295 I slot print_timing: id 3 | task 67913 | n_decoded = 1882, tg = 21.85 t/s, tg_3s = 20.77 t/s 56.57.119.551 I slot print_timing: id 3 | task 67913 | n_decoded = 1945, tg = 21.82 t/s, tg_3s = 20.72 t/s 57.00.128.433 I slot print_timing: id 3 | task 67913 | n_decoded = 2007, tg = 21.78 t/s, tg_3s = 20.61 t/s 57.03.164.540 I slot print_timing: id 3 | task 67913 | n_decoded = 2070, tg = 21.74 t/s, tg_3s = 20.75 t/s 57.06.201.070 I slot print_timing: id 3 | task 67913 | n_decoded = 2133, tg = 21.71 t/s, tg_3s = 20.75 t/s 57.09.240.535 I slot print_timing: id 3 | task 67913 | n_decoded = 2196, tg = 21.68 t/s, tg_3s = 20.73 t/s 57.12.272.050 I slot print_timing: id 3 | task 67913 | n_decoded = 2258, tg = 21.65 t/s, tg_3s = 20.45 t/s 57.15.296.048 I slot print_timing: id 3 | task 67913 | n_decoded = 2320, tg = 21.62 t/s, tg_3s = 20.50 t/s 57.18.314.809 I slot print_timing: id 3 | task 67913 | n_decoded = 2382, tg = 21.59 t/s, tg_3s = 20.54 t/s 57.21.357.382 I slot print_timing: id 3 | task 67913 | n_decoded = 2444, tg = 21.55 t/s, tg_3s = 20.38 t/s 57.24.377.006 I slot print_timing: id 3 | task 67913 | n_decoded = 2505, tg = 21.52 t/s, tg_3s = 20.20 t/s 57.27.397.765 I slot print_timing: id 3 | task 67913 | n_decoded = 2566, tg = 21.48 t/s, tg_3s = 20.19 t/s 57.30.402.874 I slot print_timing: id 3 | task 67913 | n_decoded = 2627, tg = 21.46 t/s, tg_3s = 20.30 t/s 57.33.436.389 I slot print_timing: id 3 | task 67913 | n_decoded = 2688, tg = 21.42 t/s, tg_3s = 20.11 t/s 57.36.452.426 I slot print_timing: id 3 | task 67913 | n_decoded = 2748, tg = 21.39 t/s, tg_3s = 19.89 t/s 57.39.475.754 I slot print_timing: id 3 | task 67913 | n_decoded = 2809, tg = 21.36 t/s, tg_3s = 20.18 t/s 57.42.497.041 I slot print_timing: id 3 | task 67913 | n_decoded = 2870, tg = 21.33 t/s, tg_3s = 20.19 t/s 57.45.531.200 I slot print_timing: id 3 | task 67913 | n_decoded = 2931, tg = 21.31 t/s, tg_3s = 20.10 t/s 57.48.580.913 I slot print_timing: id 3 | task 67913 | n_decoded = 2992, tg = 21.28 t/s, tg_3s = 20.00 t/s 57.51.629.302 I slot print_timing: id 3 | task 67913 | n_decoded = 3053, tg = 21.25 t/s, tg_3s = 20.01 t/s 57.54.630.536 I slot print_timing: id 3 | task 67913 | n_decoded = 3113, tg = 21.23 t/s, tg_3s = 19.99 t/s 57.57.634.254 I slot print_timing: id 3 | task 67913 | n_decoded = 3173, tg = 21.20 t/s, tg_3s = 19.98 t/s 58.00.651.860 I slot print_timing: id 3 | task 67913 | n_decoded = 3233, tg = 21.17 t/s, tg_3s = 19.88 t/s 58.03.685.338 I slot print_timing: id 3 | task 67913 | n_decoded = 3293, tg = 21.15 t/s, tg_3s = 19.78 t/s 58.06.688.321 I slot print_timing: id 3 | task 67913 | n_decoded = 3353, tg = 21.12 t/s, tg_3s = 19.98 t/s 58.09.694.375 I slot print_timing: id 3 | task 67913 | n_decoded = 3413, tg = 21.10 t/s, tg_3s = 19.96 t/s 58.12.711.195 I slot print_timing: id 3 | task 67913 | n_decoded = 3473, tg = 21.08 t/s, tg_3s = 19.89 t/s 58.15.723.170 I slot print_timing: id 3 | task 67913 | n_decoded = 3532, tg = 21.05 t/s, tg_3s = 19.59 t/s 58.18.738.303 I slot print_timing: id 3 | task 67913 | n_decoded = 3591, tg = 21.03 t/s, tg_3s = 19.57 t/s 58.21.738.788 I slot print_timing: id 3 | task 67913 | n_decoded = 3650, tg = 21.00 t/s, tg_3s = 19.66 t/s 58.24.789.519 I slot print_timing: id 3 | task 67913 | n_decoded = 3710, tg = 20.98 t/s, tg_3s = 19.67 t/s 58.27.812.180 I slot print_timing: id 3 | task 67913 | n_decoded = 3769, tg = 20.96 t/s, tg_3s = 19.52 t/s 58.30.826.925 I slot print_timing: id 3 | task 67913 | n_decoded = 3828, tg = 20.93 t/s, tg_3s = 19.57 t/s 58.33.828.179 I slot print_timing: id 3 | task 67913 | n_decoded = 3887, tg = 20.91 t/s, tg_3s = 19.66 t/s 58.36.835.102 I slot print_timing: id 3 | task 67913 | n_decoded = 3946, tg = 20.89 t/s, tg_3s = 19.62 t/s 58.39.872.327 I slot print_timing: id 3 | task 67913 | n_decoded = 4005, tg = 20.87 t/s, tg_3s = 19.43 t/s 58.42.914.785 I slot print_timing: id 3 | task 67913 | n_decoded = 4064, tg = 20.85 t/s, tg_3s = 19.39 t/s 58.44.556.524 I slot print_timing: id 3 | task 67913 | prompt eval time = 1044.59 ms / 36 tokens ( 29.02 ms per token, 34.46 tokens per second) 58.44.556.528 I slot print_timing: id 3 | task 67913 | eval time = 196592.44 ms / 4096 tokens ( 48.00 ms per token, 20.83 tokens per second) 58.44.556.528 I slot print_timing: id 3 | task 67913 | total time = 197637.02 ms / 4132 tokens 58.44.556.530 I slot print_timing: id 3 | task 67913 | graphs reused = 71664 58.44.556.561 I slot release: id 3 | task 67913 | stop processing: n_tokens = 4205, truncated = 0 58.44.556.599 I srv update_slots: all slots are idle 58.44.557.868 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 58.44.576.807 I srv operator(): Chat format: peg-native 58.44.583.073 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.622 (> 0.100 thold), f_keep = 0.018 58.44.583.075 I srv get_availabl: updating prompt cache 58.44.583.242 W srv prompt_save: - saving prompt with length 4205, total state size = 1018.448 MiB (draft: 0.000 MiB) 58.45.237.090 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.622 58.45.237.099 W srv update: - cache size limit reached, removing oldest entry (size = 1023.292 MiB) 58.45.321.068 I srv update: - cache state: 9 prompts, 7622.014 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 58.45.321.073 I srv update: - prompt 0x561d90b66b80: 4213 tokens, checkpoints: 0, 1020.386 MiB 58.45.321.074 I srv update: - prompt 0x561d90a2a790: 1197 tokens, checkpoints: 0, 289.914 MiB 58.45.321.074 I srv update: - prompt 0x561d90e0e700: 4207 tokens, checkpoints: 0, 1018.932 MiB 58.45.321.075 I srv update: - prompt 0x561d90aa1a40: 4210 tokens, checkpoints: 0, 1019.659 MiB 58.45.321.076 I srv update: - prompt 0x561d90ad2ae0: 4200 tokens, checkpoints: 0, 1017.237 MiB 58.45.321.076 I srv update: - prompt 0x561d90afb3d0: 830 tokens, checkpoints: 0, 201.027 MiB 58.45.321.077 I srv update: - prompt 0x561d90d37f60: 4208 tokens, checkpoints: 0, 1019.175 MiB 58.45.321.078 I srv update: - prompt 0x561d97e923e0: 4200 tokens, checkpoints: 0, 1017.237 MiB 58.45.321.078 I srv update: - prompt 0x561d90afd360: 4205 tokens, checkpoints: 0, 1018.448 MiB 58.45.321.080 I srv get_availabl: prompt cache update took 738.00 ms 58.45.322.702 I slot launch_slot_: id 3 | task 72010 | processing task, is_child = 0 58.45.322.705 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 58.45.322.706 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 58.45.322.728 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 58.45.322.729 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 58.45.322.740 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 58.45.322.740 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 58.50.973.691 I slot print_timing: id 3 | task 72010 | n_decoded = 100, tg = 22.65 t/s, tg_3s = 22.65 t/s 58.54.014.989 I slot print_timing: id 3 | task 72010 | n_decoded = 168, tg = 22.53 t/s, tg_3s = 22.36 t/s 58.57.048.312 I slot print_timing: id 3 | task 72010 | n_decoded = 236, tg = 22.50 t/s, tg_3s = 22.42 t/s 59.00.085.831 I slot print_timing: id 3 | task 72010 | n_decoded = 304, tg = 22.47 t/s, tg_3s = 22.39 t/s 59.03.124.951 I slot print_timing: id 3 | task 72010 | n_decoded = 372, tg = 22.46 t/s, tg_3s = 22.37 t/s 59.06.169.576 I slot print_timing: id 3 | task 72010 | n_decoded = 439, tg = 22.39 t/s, tg_3s = 22.01 t/s 59.09.214.626 I slot print_timing: id 3 | task 72010 | n_decoded = 506, tg = 22.33 t/s, tg_3s = 22.00 t/s 59.12.231.567 I slot print_timing: id 3 | task 72010 | n_decoded = 572, tg = 22.28 t/s, tg_3s = 21.88 t/s 59.15.244.594 I slot print_timing: id 3 | task 72010 | n_decoded = 639, tg = 22.28 t/s, tg_3s = 22.24 t/s 59.18.254.171 I slot print_timing: id 3 | task 72010 | n_decoded = 705, tg = 22.24 t/s, tg_3s = 21.93 t/s 59.21.261.695 I slot print_timing: id 3 | task 72010 | n_decoded = 771, tg = 22.22 t/s, tg_3s = 21.94 t/s 59.24.306.110 I slot print_timing: id 3 | task 72010 | n_decoded = 838, tg = 22.20 t/s, tg_3s = 22.01 t/s 59.27.347.949 I slot print_timing: id 3 | task 72010 | n_decoded = 905, tg = 22.19 t/s, tg_3s = 22.03 t/s 59.30.372.777 I slot print_timing: id 3 | task 72010 | n_decoded = 971, tg = 22.16 t/s, tg_3s = 21.82 t/s 59.33.405.694 I slot print_timing: id 3 | task 72010 | n_decoded = 1036, tg = 22.11 t/s, tg_3s = 21.43 t/s 59.36.441.233 I slot print_timing: id 3 | task 72010 | n_decoded = 1102, tg = 22.09 t/s, tg_3s = 21.74 t/s 59.39.454.373 I slot print_timing: id 3 | task 72010 | n_decoded = 1168, tg = 22.08 t/s, tg_3s = 21.90 t/s 59.42.462.942 I slot print_timing: id 3 | task 72010 | n_decoded = 1234, tg = 22.07 t/s, tg_3s = 21.94 t/s 59.45.470.383 I slot print_timing: id 3 | task 72010 | n_decoded = 1300, tg = 22.07 t/s, tg_3s = 21.95 t/s 59.48.479.107 I slot print_timing: id 3 | task 72010 | n_decoded = 1366, tg = 22.06 t/s, tg_3s = 21.94 t/s 59.51.505.809 I slot print_timing: id 3 | task 72010 | n_decoded = 1432, tg = 22.05 t/s, tg_3s = 21.81 t/s 59.54.525.206 I slot print_timing: id 3 | task 72010 | n_decoded = 1498, tg = 22.04 t/s, tg_3s = 21.86 t/s 59.57.533.720 I slot print_timing: id 3 | task 72010 | n_decoded = 1563, tg = 22.02 t/s, tg_3s = 21.61 t/s 60.00.577.036 I slot print_timing: id 3 | task 72010 | n_decoded = 1629, tg = 22.01 t/s, tg_3s = 21.69 t/s 60.03.608.172 I slot print_timing: id 3 | task 72010 | n_decoded = 1694, tg = 21.99 t/s, tg_3s = 21.44 t/s 60.06.631.137 I slot print_timing: id 3 | task 72010 | n_decoded = 1757, tg = 21.94 t/s, tg_3s = 20.84 t/s 60.09.655.275 I slot print_timing: id 3 | task 72010 | n_decoded = 1820, tg = 21.90 t/s, tg_3s = 20.83 t/s 60.12.672.402 I slot print_timing: id 3 | task 72010 | n_decoded = 1883, tg = 21.87 t/s, tg_3s = 20.88 t/s 60.15.704.694 I slot print_timing: id 3 | task 72010 | n_decoded = 1946, tg = 21.83 t/s, tg_3s = 20.78 t/s 60.18.739.473 I slot print_timing: id 3 | task 72010 | n_decoded = 2009, tg = 21.79 t/s, tg_3s = 20.76 t/s 60.21.752.742 I slot print_timing: id 3 | task 72010 | n_decoded = 2071, tg = 21.76 t/s, tg_3s = 20.58 t/s 60.24.780.791 I slot print_timing: id 3 | task 72010 | n_decoded = 2134, tg = 21.73 t/s, tg_3s = 20.81 t/s 60.27.824.788 I slot print_timing: id 3 | task 72010 | n_decoded = 2197, tg = 21.70 t/s, tg_3s = 20.70 t/s 60.30.852.184 I slot print_timing: id 3 | task 72010 | n_decoded = 2259, tg = 21.66 t/s, tg_3s = 20.48 t/s 60.33.873.509 I slot print_timing: id 3 | task 72010 | n_decoded = 2321, tg = 21.63 t/s, tg_3s = 20.52 t/s 60.36.886.905 I slot print_timing: id 3 | task 72010 | n_decoded = 2383, tg = 21.60 t/s, tg_3s = 20.57 t/s 60.39.921.002 I slot print_timing: id 3 | task 72010 | n_decoded = 2445, tg = 21.57 t/s, tg_3s = 20.43 t/s 60.42.928.093 I slot print_timing: id 3 | task 72010 | n_decoded = 2506, tg = 21.53 t/s, tg_3s = 20.29 t/s 60.45.934.358 I slot print_timing: id 3 | task 72010 | n_decoded = 2567, tg = 21.50 t/s, tg_3s = 20.29 t/s 60.48.956.496 I slot print_timing: id 3 | task 72010 | n_decoded = 2628, tg = 21.47 t/s, tg_3s = 20.18 t/s 60.51.986.069 I slot print_timing: id 3 | task 72010 | n_decoded = 2689, tg = 21.44 t/s, tg_3s = 20.13 t/s 60.55.019.175 I slot print_timing: id 3 | task 72010 | n_decoded = 2750, tg = 21.41 t/s, tg_3s = 20.11 t/s 60.58.045.336 I slot print_timing: id 3 | task 72010 | n_decoded = 2811, tg = 21.38 t/s, tg_3s = 20.16 t/s 61.01.084.393 I slot print_timing: id 3 | task 72010 | n_decoded = 2872, tg = 21.35 t/s, tg_3s = 20.07 t/s 61.04.112.594 I slot print_timing: id 3 | task 72010 | n_decoded = 2933, tg = 21.32 t/s, tg_3s = 20.14 t/s 61.07.162.960 I slot print_timing: id 3 | task 72010 | n_decoded = 2993, tg = 21.29 t/s, tg_3s = 19.67 t/s 61.10.173.317 I slot print_timing: id 3 | task 72010 | n_decoded = 3053, tg = 21.26 t/s, tg_3s = 19.93 t/s 61.13.195.256 I slot print_timing: id 3 | task 72010 | n_decoded = 3113, tg = 21.23 t/s, tg_3s = 19.85 t/s 61.16.209.587 I slot print_timing: id 3 | task 72010 | n_decoded = 3173, tg = 21.20 t/s, tg_3s = 19.90 t/s 61.19.227.459 I slot print_timing: id 3 | task 72010 | n_decoded = 3233, tg = 21.18 t/s, tg_3s = 19.88 t/s 61.22.240.347 I slot print_timing: id 3 | task 72010 | n_decoded = 3293, tg = 21.15 t/s, tg_3s = 19.91 t/s 61.25.246.388 I slot print_timing: id 3 | task 72010 | n_decoded = 3353, tg = 21.13 t/s, tg_3s = 19.96 t/s 61.28.252.393 I slot print_timing: id 3 | task 72010 | n_decoded = 3413, tg = 21.11 t/s, tg_3s = 19.96 t/s 61.31.273.716 I slot print_timing: id 3 | task 72010 | n_decoded = 3473, tg = 21.08 t/s, tg_3s = 19.86 t/s 61.34.315.688 I slot print_timing: id 3 | task 72010 | n_decoded = 3533, tg = 21.06 t/s, tg_3s = 19.72 t/s 61.37.364.044 I slot print_timing: id 3 | task 72010 | n_decoded = 3593, tg = 21.04 t/s, tg_3s = 19.68 t/s 61.40.411.415 I slot print_timing: id 3 | task 72010 | n_decoded = 3653, tg = 21.01 t/s, tg_3s = 19.69 t/s 61.43.423.376 I slot print_timing: id 3 | task 72010 | n_decoded = 3712, tg = 20.99 t/s, tg_3s = 19.59 t/s 61.46.441.728 I slot print_timing: id 3 | task 72010 | n_decoded = 3771, tg = 20.96 t/s, tg_3s = 19.55 t/s 61.49.450.849 I slot print_timing: id 3 | task 72010 | n_decoded = 3830, tg = 20.94 t/s, tg_3s = 19.61 t/s 61.52.460.510 I slot print_timing: id 3 | task 72010 | n_decoded = 3889, tg = 20.92 t/s, tg_3s = 19.60 t/s 61.55.511.418 I slot print_timing: id 3 | task 72010 | n_decoded = 3949, tg = 20.90 t/s, tg_3s = 19.67 t/s 61.58.544.157 I slot print_timing: id 3 | task 72010 | n_decoded = 4008, tg = 20.88 t/s, tg_3s = 19.45 t/s 62.01.570.938 I slot print_timing: id 3 | task 72010 | n_decoded = 4067, tg = 20.86 t/s, tg_3s = 19.49 t/s 62.03.054.300 I slot print_timing: id 3 | task 72010 | prompt eval time = 1236.54 ms / 45 tokens ( 27.48 ms per token, 36.39 tokens per second) 62.03.054.307 I slot print_timing: id 3 | task 72010 | eval time = 196494.98 ms / 4096 tokens ( 47.97 ms per token, 20.85 tokens per second) 62.03.054.307 I slot print_timing: id 3 | task 72010 | total time = 197731.53 ms / 4141 tokens 62.03.054.308 I slot print_timing: id 3 | task 72010 | graphs reused = 75742 62.03.054.335 I slot release: id 3 | task 72010 | stop processing: n_tokens = 4214, truncated = 0 62.03.054.361 I srv update_slots: all slots are idle 62.03.055.548 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues. 62.03.072.750 I srv operator(): Chat format: peg-native 62.03.078.304 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.705 (> 0.100 thold), f_keep = 0.018 62.03.078.306 I srv get_availabl: updating prompt cache 62.03.078.431 W srv prompt_save: - saving prompt with length 4214, total state size = 1020.628 MiB (draft: 0.000 MiB) 62.03.707.552 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.705 62.03.707.559 W srv update: - cache size limit reached, removing oldest entry (size = 1020.386 MiB) 62.03.786.192 I srv update: - cache state: 9 prompts, 7622.256 MiB (limits: 8192.000 MiB, 16384 tokens, 33823 est) 62.03.786.196 I srv update: - prompt 0x561d90a2a790: 1197 tokens, checkpoints: 0, 289.914 MiB 62.03.786.196 I srv update: - prompt 0x561d90e0e700: 4207 tokens, checkpoints: 0, 1018.932 MiB 62.03.786.197 I srv update: - prompt 0x561d90aa1a40: 4210 tokens, checkpoints: 0, 1019.659 MiB 62.03.786.197 I srv update: - prompt 0x561d90ad2ae0: 4200 tokens, checkpoints: 0, 1017.237 MiB 62.03.786.198 I srv update: - prompt 0x561d90afb3d0: 830 tokens, checkpoints: 0, 201.027 MiB 62.03.786.199 I srv update: - prompt 0x561d90d37f60: 4208 tokens, checkpoints: 0, 1019.175 MiB 62.03.786.199 I srv update: - prompt 0x561d97e923e0: 4200 tokens, checkpoints: 0, 1017.237 MiB 62.03.786.199 I srv update: - prompt 0x561d90afd360: 4205 tokens, checkpoints: 0, 1018.448 MiB 62.03.786.200 I srv update: - prompt 0x561d9809cde0: 4214 tokens, checkpoints: 0, 1020.628 MiB 62.03.786.201 I srv get_availabl: prompt cache update took 707.89 ms 62.03.789.434 I slot launch_slot_: id 3 | task 76107 | processing task, is_child = 0 62.03.789.439 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache 62.03.789.440 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens 62.03.789.461 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache 62.03.789.462 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens 62.03.789.474 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache 62.03.789.474 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens 62.09.078.199 I slot print_timing: id 3 | task 76107 | n_decoded = 100, tg = 22.64 t/s, tg_3s = 22.64 t/s 62.12.114.493 I slot print_timing: id 3 | task 76107 | n_decoded = 168, tg = 22.54 t/s, tg_3s = 22.40 t/s 62.15.148.791 I slot print_timing: id 3 | task 76107 | n_decoded = 236, tg = 22.50 t/s, tg_3s = 22.41 t/s 62.18.167.384 I slot print_timing: id 3 | task 76107 | n_decoded = 303, tg = 22.43 t/s, tg_3s = 22.20 t/s 62.21.207.699 I slot print_timing: id 3 | task 76107 | n_decoded = 371, tg = 22.42 t/s, tg_3s = 22.37 t/s 62.24.219.304 I slot print_timing: id 3 | task 76107 | n_decoded = 438, tg = 22.39 t/s, tg_3s = 22.25 t/s 62.27.242.099 I slot print_timing: id 3 | task 76107 | n_decoded = 504, tg = 22.32 t/s, tg_3s = 21.83 t/s 62.30.243.969 I slot print_timing: id 3 | task 76107 | n_decoded = 570, tg = 22.28 t/s, tg_3s = 21.99 t/s 62.33.256.828 I slot print_timing: id 3 | task 76107 | n_decoded = 637, tg = 22.28 t/s, tg_3s = 22.24 t/s 62.36.292.198 I slot print_timing: id 3 | task 76107 | n_decoded = 703, tg = 22.22 t/s, tg_3s = 21.74 t/s 62.39.335.692 I slot print_timing: id 3 | task 76107 | n_decoded = 770, tg = 22.21 t/s, tg_3s = 22.01 t/s 62.42.376.534 I slot print_timing: id 3 | task 76107 | n_decoded = 837, tg = 22.19 t/s, tg_3s = 22.03 t/s 62.45.388.202 I slot print_timing: id 3 | task 76107 | n_decoded = 903, tg = 22.17 t/s, tg_3s = 21.91 t/s 62.48.401.941 I slot print_timing: id 3 | task 76107 | n_decoded = 969, tg = 22.15 t/s, tg_3s = 21.90 t/s 62.51.403.345 I slot print_timing: id 3 | task 76107 | n_decoded = 1035, tg = 22.14 t/s, tg_3s = 21.99 t/s 62.54.405.950 I slot print_timing: id 3 | task 76107 | n_decoded = 1101, tg = 22.13 t/s, tg_3s = 21.98 t/s 62.57.408.936 I slot print_timing: id 3 | task 76107 | n_decoded = 1167, tg = 22.12 t/s, tg_3s = 21.98 t/s 63.00.435.666 I slot print_timing: id 3 | task 76107 | n_decoded = 1233, tg = 22.11 t/s, tg_3s = 21.81 t/s 63.03.463.166 I slot print_timing: id 3 | task 76107 | n_decoded = 1299, tg = 22.09 t/s, tg_3s = 21.80 t/s 63.06.494.405 I slot print_timing: id 3 | task 76107 | n_decoded = 1364, tg = 22.06 t/s, tg_3s = 21.44 t/s 63.09.533.712 I slot print_timing: id 3 | task 76107 | n_decoded = 1430, tg = 22.04 t/s, tg_3s = 21.72 t/s 63.12.570.471 I slot print_timing: id 3 | task 76107 | n_decoded = 1496, tg = 22.03 t/s, tg_3s = 21.73 t/s 63.15.571.111 I slot print_timing: id 3 | task 76107 | n_decoded = 1561, tg = 22.01 t/s, tg_3s = 21.66 t/s 63.18.592.737 I slot print_timing: id 3 | task 76107 | n_decoded = 1627, tg = 22.01 t/s, tg_3s = 21.84 t/s 63.21.593.096 I slot print_timing: id 3 | task 76107 | n_decoded = 1692, tg = 21.99 t/s, tg_3s = 21.66 t/s 63.24.609.863 I slot print_timing: id 3 | task 76107 | n_decoded = 1755, tg = 21.95 t/s, tg_3s = 20.88 t/s 63.27.635.159 I slot print_timing: id 3 | task 76107 | n_decoded = 1818, tg = 21.91 t/s, tg_3s = 20.82 t/s 63.30.665.665 I slot print_timing: id 3 | task 76107 | n_decoded = 1881, tg = 21.87 t/s, tg_3s = 20.79 t/s 63.33.682.973 I slot print_timing: id 3 | task 76107 | n_decoded = 1944, tg = 21.84 t/s, tg_3s = 20.88 t/s 63.36.727.213 I slot print_timing: id 3 | task 76107 | n_decoded = 2006, tg = 21.79 t/s, tg_3s = 20.37 t/s 63.39.729.156 I slot print_timing: id 3 | task 76107 | n_decoded = 2068, tg = 21.75 t/s, tg_3s = 20.65 t/s 63.42.769.142 I slot print_timing: id 3 | task 76107 | n_decoded = 2131, tg = 21.72 t/s, tg_3s = 20.72 t/s 63.45.804.025 I slot print_timing: id 3 | task 76107 | n_decoded = 2194, tg = 21.69 t/s, tg_3s = 20.76 t/s 63.48.842.908 I slot print_timing: id 3 | task 76107 | n_decoded = 2255, tg = 21.64 t/s, tg_3s = 20.07 t/s 63.51.867.909 I slot print_timing: id 3 | task 76107 | n_decoded = 2317, tg = 21.61 t/s, tg_3s = 20.50 t/s 63.54.886.423 I slot print_timing: id 3 | task 76107 | n_decoded = 2379, tg = 21.58 t/s, tg_3s = 20.54 t/s 63.57.907.222 I slot print_timing: id 3 | task 76107 | n_decoded = 2441, tg = 21.55 t/s, tg_3s = 20.52 t/s 64.00.943.308 I slot print_timing: id 3 | task 76107 | n_decoded = 2502, tg = 21.52 t/s, tg_3s = 20.09 t/s 64.03.944.263 I slot print_timing: id 3 | task 76107 | n_decoded = 2563, tg = 21.49 t/s, tg_3s = 20.33 t/s 64.06.987.694 I slot print_timing: id 3 | task 76107 | n_decoded = 2625, tg = 21.46 t/s, tg_3s = 20.37 t/s 64.10.031.274 I slot print_timing: id 3 | task 76107 | n_decoded = 2687, tg = 21.43 t/s, tg_3s = 20.37 t/s 64.13.078.933 I slot print_timing: id 3 | task 76107 | n_decoded = 2748, tg = 21.40 t/s, tg_3s = 20.02 t/s 64.16.110.866 I slot print_timing: id 3 | task 76107 | n_decoded = 2809, tg = 21.37 t/s, tg_3s = 20.12 t/s 64.19.157.311 I slot print_timing: id 3 | task 76107 | n_decoded = 2870, tg = 21.34 t/s, tg_3s = 20.02 t/s 64.22.180.140 I slot print_timing: id 3 | task 76107 | n_decoded = 2931, tg = 21.31 t/s, tg_3s = 20.18 t/s 64.25.181.169 I slot print_timing: id 3 | task 76107 | n_decoded = 2991, tg = 21.29 t/s, tg_3s = 19.99 t/s 64.28.183.305 I slot print_timing: id 3 | task 76107 | n_decoded = 3051, tg = 21.26 t/s, tg_3s = 19.99 t/s 64.31.198.697 I slot print_timing: id 3 | task 76107 | n_decoded = 3111, tg = 21.23 t/s, tg_3s = 19.90 t/s 64.34.206.214 I slot print_timing: id 3 | task 76107 | n_decoded = 3171, tg = 21.20 t/s, tg_3s = 19.95 t/s 64.37.217.554 I slot print_timing: id 3 | task 76107 | n_decoded = 3231, tg = 21.18 t/s, tg_3s = 19.92 t/s 64.40.218.420 I slot print_timing: id 3 | task 76107 | n_decoded = 3290, tg = 21.15 t/s, tg_3s = 19.66 t/s 64.43.225.336 I slot print_timing: id 3 | task 76107 | n_decoded = 3350, tg = 21.13 t/s, tg_3s = 19.95 t/s 64.46.238.286 I slot print_timing: id 3 | task 76107 | n_decoded = 3410, tg = 21.10 t/s, tg_3s = 19.91 t/s 64.49.246.621 I slot print_timing: id 3 | task 76107 | n_decoded = 3470, tg = 21.08 t/s, tg_3s = 19.94 t/s 64.52.283.459 I slot print_timing: id 3 | task 76107 | n_decoded = 3529, tg = 21.05 t/s, tg_3s = 19.43 t/s 64.55.329.608 I slot print_timing: id 3 | task 76107 | n_decoded = 3589, tg = 21.03 t/s, tg_3s = 19.70 t/s 64.58.329.854 I slot print_timing: id 3 | task 76107 | n_decoded = 3648, tg = 21.01 t/s, tg_3s = 19.67 t/s 65.01.333.361 I slot print_timing: id 3 | task 76107 | n_decoded = 3707, tg = 20.98 t/s, tg_3s = 19.64 t/s 65.04.363.998 I slot print_timing: id 3 | task 76107 | n_decoded = 3766, tg = 20.96 t/s, tg_3s = 19.47 t/s 65.07.401.301 I slot print_timing: id 3 | task 76107 | n_decoded = 3825, tg = 20.93 t/s, tg_3s = 19.43 t/s 65.10.410.977 I slot print_timing: id 3 | task 76107 | n_decoded = 3884, tg = 20.91 t/s, tg_3s = 19.60 t/s 65.13.415.739 I slot print_timing: id 3 | task 76107 | n_decoded = 3943, tg = 20.89 t/s, tg_3s = 19.64 t/s 65.16.431.718 I slot print_timing: id 3 | task 76107 | n_decoded = 4002, tg = 20.87 t/s, tg_3s = 19.56 t/s 65.19.446.739 I slot print_timing: id 3 | task 76107 | n_decoded = 4061, tg = 20.85 t/s, tg_3s = 19.57 t/s 65.21.236.362 I slot print_timing: id 3 | task 76107 | prompt eval time = 871.19 ms / 31 tokens ( 28.10 ms per token, 35.58 tokens per second) 65.21.236.366 I slot print_timing: id 3 | task 76107 | eval time = 196575.67 ms / 4096 tokens ( 47.99 ms per token, 20.84 tokens per second) 65.21.236.366 I slot print_timing: id 3 | task 76107 | total time = 197446.86 ms / 4127 tokens 65.21.236.367 I slot print_timing: id 3 | task 76107 | graphs reused = 79820 65.21.236.397 I slot release: id 3 | task 76107 | stop processing: n_tokens = 4200, truncated = 0 65.21.236.432 I srv update_slots: all slots are idle 65.21.237.271 I srv operator(): operator(): cleaning up before exit...