qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results_hard/qwen3.6-35b-a3b.server.log

1117 lines
112 KiB
Plaintext
Raw Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.264.537 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.264.540 I device_info:
0.00.264.560 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.264.564 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.264.572 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.264.631 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.264.635 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.264.834 I srv init: running without SSL
0.00.265.024 I srv init: using 23 threads for HTTP server
0.00.265.343 I srv start: binding port with default address family
0.00.266.550 I srv llama_server: loading model
0.00.266.559 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf'
0.00.266.605 I common_init_result: fitting params to device memory ...
0.00.266.606 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.15.168.936 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.15.268.091 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.15.469.331 I srv load_model: initializing slots, n_slots = 4
0.15.530.210 W srv load_model: speculative decoding will use checkpoints
0.15.530.234 W common_speculative_init: no implementations specified for speculative decoding
0.15.530.235 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384
0.15.530.240 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384
0.15.530.240 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384
0.15.530.240 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384
0.15.530.288 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.15.530.288 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.15.530.289 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.15.530.289 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.15.530.307 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.15.548.989 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
<think>
'
0.15.562.767 I srv init: init: chat template, thinking = 1
0.15.562.805 I srv llama_server: model loaded
0.15.562.811 I srv llama_server: server is listening on http://127.0.0.1:8090
0.15.562.823 I srv update_slots: all slots are idle
0.19.932.665 I srv operator(): Chat format: peg-native
0.19.933.018 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.19.933.021 I srv get_availabl: updating prompt cache
0.19.933.026 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.19.933.030 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est)
0.19.933.031 I srv get_availabl: prompt cache update took 0.01 ms
0.19.933.080 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.19.933.080 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.19.933.083 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.19.933.098 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.19.933.099 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.19.933.110 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.19.933.110 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.20.644.449 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 70, pos_max = 70, n_tokens = 71, size = 62.813 MiB)
0.23.795.538 I slot print_timing: id 3 | task 0 | n_decoded = 153, tg = 50.97 t/s, tg_3s = 50.97 t/s
0.26.800.018 I slot print_timing: id 3 | task 0 | n_decoded = 305, tg = 50.78 t/s, tg_3s = 50.59 t/s
0.29.810.236 I slot print_timing: id 3 | task 0 | n_decoded = 457, tg = 50.68 t/s, tg_3s = 50.49 t/s
0.32.810.918 I slot print_timing: id 3 | task 0 | n_decoded = 609, tg = 50.68 t/s, tg_3s = 50.66 t/s
0.35.819.699 I slot print_timing: id 3 | task 0 | n_decoded = 760, tg = 50.58 t/s, tg_3s = 50.19 t/s
0.38.820.923 I slot print_timing: id 3 | task 0 | n_decoded = 911, tg = 50.53 t/s, tg_3s = 50.31 t/s
0.41.839.731 I slot print_timing: id 3 | task 0 | n_decoded = 1062, tg = 50.46 t/s, tg_3s = 50.02 t/s
0.44.842.882 I slot print_timing: id 3 | task 0 | n_decoded = 1212, tg = 50.40 t/s, tg_3s = 49.95 t/s
0.47.842.993 I slot print_timing: id 3 | task 0 | n_decoded = 1362, tg = 50.35 t/s, tg_3s = 50.00 t/s
0.50.852.975 I slot print_timing: id 3 | task 0 | n_decoded = 1512, tg = 50.30 t/s, tg_3s = 49.83 t/s
0.53.854.244 I slot print_timing: id 3 | task 0 | n_decoded = 1662, tg = 50.27 t/s, tg_3s = 49.98 t/s
0.56.865.055 I slot print_timing: id 3 | task 0 | n_decoded = 1812, tg = 50.23 t/s, tg_3s = 49.82 t/s
0.59.875.091 I slot print_timing: id 3 | task 0 | n_decoded = 1962, tg = 50.20 t/s, tg_3s = 49.83 t/s
1.02.875.225 I slot print_timing: id 3 | task 0 | n_decoded = 2112, tg = 50.19 t/s, tg_3s = 50.00 t/s
1.05.890.154 I slot print_timing: id 3 | task 0 | n_decoded = 2262, tg = 50.16 t/s, tg_3s = 49.75 t/s
1.08.900.332 I slot print_timing: id 3 | task 0 | n_decoded = 2412, tg = 50.14 t/s, tg_3s = 49.83 t/s
1.11.914.437 I slot print_timing: id 3 | task 0 | n_decoded = 2562, tg = 50.12 t/s, tg_3s = 49.77 t/s
1.14.933.022 I slot print_timing: id 3 | task 0 | n_decoded = 2713, tg = 50.11 t/s, tg_3s = 50.02 t/s
1.17.943.296 I slot print_timing: id 3 | task 0 | n_decoded = 2863, tg = 50.10 t/s, tg_3s = 49.83 t/s
1.18.303.109 I slot print_timing: id 3 | task 0 | prompt eval time = 860.55 ms / 100 tokens ( 8.61 ms per token, 116.20 tokens per second)
1.18.303.112 I slot print_timing: id 3 | task 0 | eval time = 57509.42 ms / 2881 tokens ( 19.96 ms per token, 50.10 tokens per second)
1.18.303.113 I slot print_timing: id 3 | task 0 | total time = 58369.97 ms / 2981 tokens
1.18.303.117 I slot print_timing: id 3 | task 0 | graphs reused = 2869
1.18.303.171 I slot release: id 3 | task 0 | stop processing: n_tokens = 2980, truncated = 0
1.18.303.188 I srv update_slots: all slots are idle
1.18.319.120 I srv operator(): Chat format: peg-native
1.18.319.422 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.787 (> 0.100 thold), f_keep = 0.025
1.18.319.425 I srv get_availabl: updating prompt cache
1.18.319.577 W srv prompt_save: - saving prompt with length 2980, total state size = 121.073 MiB (draft: 0.000 MiB)
1.18.484.544 I srv load: - looking for better prompt, base f_keep = 0.025, sim = 0.787
1.18.484.551 I srv update: - cache state: 1 prompts, 183.887 MiB (limits: 8192.000 MiB, 16384 tokens, 132756 est)
1.18.484.552 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
1.18.484.554 I srv get_availabl: prompt cache update took 165.13 ms
1.18.484.652 I slot launch_slot_: id 3 | task 2884 | processing task, is_child = 0
1.18.484.655 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.18.484.656 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.18.484.674 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.18.484.674 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.18.484.698 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.18.484.698 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.18.484.720 I slot update_slots: id 3 | task 2884 | Checking checkpoint with [70, 70] against 74...
1.18.500.485 W slot update_slots: id 3 | task 2884 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
1.21.634.969 I slot print_timing: id 3 | task 2884 | n_decoded = 153, tg = 50.83 t/s, tg_3s = 50.83 t/s
1.24.654.160 I slot print_timing: id 3 | task 2884 | n_decoded = 305, tg = 50.59 t/s, tg_3s = 50.34 t/s
1.27.659.732 I slot print_timing: id 3 | task 2884 | n_decoded = 456, tg = 50.47 t/s, tg_3s = 50.24 t/s
1.30.660.541 I slot print_timing: id 3 | task 2884 | n_decoded = 607, tg = 50.43 t/s, tg_3s = 50.32 t/s
1.33.679.311 I slot print_timing: id 3 | task 2884 | n_decoded = 758, tg = 50.35 t/s, tg_3s = 50.02 t/s
1.36.689.163 I slot print_timing: id 3 | task 2884 | n_decoded = 909, tg = 50.32 t/s, tg_3s = 50.17 t/s
1.39.702.383 I slot print_timing: id 3 | task 2884 | n_decoded = 1060, tg = 50.29 t/s, tg_3s = 50.11 t/s
1.42.707.096 I slot print_timing: id 3 | task 2884 | n_decoded = 1210, tg = 50.24 t/s, tg_3s = 49.92 t/s
1.45.707.757 I slot print_timing: id 3 | task 2884 | n_decoded = 1360, tg = 50.22 t/s, tg_3s = 49.99 t/s
1.48.720.157 I slot print_timing: id 3 | task 2884 | n_decoded = 1510, tg = 50.17 t/s, tg_3s = 49.79 t/s
1.51.723.664 I slot print_timing: id 3 | task 2884 | n_decoded = 1660, tg = 50.15 t/s, tg_3s = 49.94 t/s
1.54.604.082 I slot print_timing: id 3 | task 2884 | prompt eval time = 140.02 ms / 23 tokens ( 6.09 ms per token, 164.27 tokens per second)
1.54.604.086 I slot print_timing: id 3 | task 2884 | eval time = 35979.34 ms / 1803 tokens ( 19.96 ms per token, 50.11 tokens per second)
1.54.604.086 I slot print_timing: id 3 | task 2884 | total time = 36119.36 ms / 1826 tokens
1.54.604.087 I slot print_timing: id 3 | task 2884 | graphs reused = 4663
1.54.604.123 I slot release: id 3 | task 2884 | stop processing: n_tokens = 1896, truncated = 0
1.54.604.138 I srv update_slots: all slots are idle
1.54.620.250 I srv operator(): Chat format: peg-native
1.54.620.614 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.787 (> 0.100 thold), f_keep = 0.039
1.54.620.616 I srv get_availabl: updating prompt cache
1.54.620.709 W srv prompt_save: - saving prompt with length 1896, total state size = 99.881 MiB (draft: 0.000 MiB)
1.54.786.329 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.787
1.54.786.335 I srv update: - cache state: 2 prompts, 346.581 MiB (limits: 8192.000 MiB, 16384 tokens, 115252 est)
1.54.786.336 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
1.54.786.336 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
1.54.786.338 I srv get_availabl: prompt cache update took 165.72 ms
1.54.786.408 I slot launch_slot_: id 3 | task 4689 | processing task, is_child = 0
1.54.786.410 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.54.786.411 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.54.786.426 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.54.786.426 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.54.786.435 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.54.786.435 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.54.786.453 I slot update_slots: id 3 | task 4689 | Checking checkpoint with [70, 70] against 74...
1.54.802.234 W slot update_slots: id 3 | task 4689 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
1.57.943.145 I slot print_timing: id 3 | task 4689 | n_decoded = 152, tg = 50.35 t/s, tg_3s = 50.35 t/s
2.00.948.840 I slot print_timing: id 3 | task 4689 | n_decoded = 304, tg = 50.46 t/s, tg_3s = 50.57 t/s
2.03.968.376 I slot print_timing: id 3 | task 4689 | n_decoded = 457, tg = 50.53 t/s, tg_3s = 50.67 t/s
2.06.986.074 I slot print_timing: id 3 | task 4689 | n_decoded = 610, tg = 50.57 t/s, tg_3s = 50.70 t/s
2.10.000.298 I slot print_timing: id 3 | task 4689 | n_decoded = 762, tg = 50.54 t/s, tg_3s = 50.43 t/s
2.13.010.566 I slot print_timing: id 3 | task 4689 | n_decoded = 914, tg = 50.54 t/s, tg_3s = 50.49 t/s
2.13.714.350 I slot print_timing: id 3 | task 4689 | prompt eval time = 137.81 ms / 23 tokens ( 5.99 ms per token, 166.90 tokens per second)
2.13.714.353 I slot print_timing: id 3 | task 4689 | eval time = 18790.08 ms / 949 tokens ( 19.80 ms per token, 50.51 tokens per second)
2.13.714.354 I slot print_timing: id 3 | task 4689 | total time = 18927.89 ms / 972 tokens
2.13.714.355 I slot print_timing: id 3 | task 4689 | graphs reused = 5606
2.13.714.387 I slot release: id 3 | task 4689 | stop processing: n_tokens = 1042, truncated = 0
2.13.714.398 I srv update_slots: all slots are idle
2.13.730.835 I srv operator(): Chat format: peg-native
2.13.731.131 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.796 (> 0.100 thold), f_keep = 0.071
2.13.731.133 I srv get_availabl: updating prompt cache
2.13.731.207 W srv prompt_save: - saving prompt with length 1042, total state size = 83.185 MiB (draft: 0.000 MiB)
2.13.844.156 I srv load: - looking for better prompt, base f_keep = 0.071, sim = 0.796
2.13.844.162 I srv update: - cache state: 3 prompts, 492.579 MiB (limits: 8192.000 MiB, 16384 tokens, 98421 est)
2.13.844.163 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
2.13.844.163 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
2.13.844.164 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
2.13.844.165 I srv get_availabl: prompt cache update took 113.03 ms
2.13.844.247 I slot launch_slot_: id 3 | task 5640 | processing task, is_child = 0
2.13.844.249 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.13.844.250 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.13.844.265 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.13.844.266 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.13.844.278 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.13.844.279 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.13.844.301 I slot update_slots: id 3 | task 5640 | Checking checkpoint with [70, 70] against 74...
2.13.860.001 W slot update_slots: id 3 | task 5640 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
2.16.995.884 I slot print_timing: id 3 | task 5640 | n_decoded = 153, tg = 50.77 t/s, tg_3s = 50.77 t/s
2.19.998.662 I slot print_timing: id 3 | task 5640 | n_decoded = 304, tg = 50.53 t/s, tg_3s = 50.29 t/s
2.23.005.799 I slot print_timing: id 3 | task 5640 | n_decoded = 455, tg = 50.42 t/s, tg_3s = 50.21 t/s
2.26.017.380 I slot print_timing: id 3 | task 5640 | n_decoded = 606, tg = 50.35 t/s, tg_3s = 50.14 t/s
2.29.025.856 I slot print_timing: id 3 | task 5640 | n_decoded = 756, tg = 50.25 t/s, tg_3s = 49.86 t/s
2.32.044.931 I slot print_timing: id 3 | task 5640 | n_decoded = 907, tg = 50.21 t/s, tg_3s = 50.02 t/s
2.35.054.866 I slot print_timing: id 3 | task 5640 | n_decoded = 1057, tg = 50.16 t/s, tg_3s = 49.83 t/s
2.38.063.850 I slot print_timing: id 3 | task 5640 | n_decoded = 1207, tg = 50.12 t/s, tg_3s = 49.85 t/s
2.41.069.150 I slot print_timing: id 3 | task 5640 | n_decoded = 1357, tg = 50.10 t/s, tg_3s = 49.91 t/s
2.44.079.128 I slot print_timing: id 3 | task 5640 | n_decoded = 1507, tg = 50.07 t/s, tg_3s = 49.83 t/s
2.47.079.175 I slot print_timing: id 3 | task 5640 | n_decoded = 1657, tg = 50.07 t/s, tg_3s = 50.00 t/s
2.50.090.148 I slot print_timing: id 3 | task 5640 | n_decoded = 1807, tg = 50.04 t/s, tg_3s = 49.82 t/s
2.53.117.937 I slot print_timing: id 3 | task 5640 | n_decoded = 1958, tg = 50.03 t/s, tg_3s = 49.87 t/s
2.56.118.151 I slot print_timing: id 3 | task 5640 | n_decoded = 2108, tg = 50.03 t/s, tg_3s = 50.00 t/s
2.57.717.983 I slot print_timing: id 3 | task 5640 | prompt eval time = 138.19 ms / 22 tokens ( 6.28 ms per token, 159.20 tokens per second)
2.57.717.986 I slot print_timing: id 3 | task 5640 | eval time = 43735.49 ms / 2188 tokens ( 19.99 ms per token, 50.03 tokens per second)
2.57.717.987 I slot print_timing: id 3 | task 5640 | total time = 43873.68 ms / 2210 tokens
2.57.717.988 I slot print_timing: id 3 | task 5640 | graphs reused = 7784
2.57.718.025 I slot release: id 3 | task 5640 | stop processing: n_tokens = 2280, truncated = 0
2.57.718.042 I srv update_slots: all slots are idle
2.57.734.000 I srv operator(): Chat format: peg-native
2.57.734.295 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.698 (> 0.100 thold), f_keep = 0.032
2.57.734.297 I srv get_availabl: updating prompt cache
2.57.734.415 W srv prompt_save: - saving prompt with length 2280, total state size = 107.388 MiB (draft: 0.000 MiB)
2.57.877.871 I srv load: - looking for better prompt, base f_keep = 0.032, sim = 0.698
2.57.877.878 I srv update: - cache state: 4 prompts, 662.780 MiB (limits: 8192.000 MiB, 16384 tokens, 101327 est)
2.57.877.879 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
2.57.877.880 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
2.57.877.881 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
2.57.877.882 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
2.57.877.883 I srv get_availabl: prompt cache update took 143.59 ms
2.57.877.961 I slot launch_slot_: id 3 | task 7830 | processing task, is_child = 0
2.57.877.962 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.57.877.963 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.57.877.982 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.57.877.982 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.57.877.995 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.57.877.996 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.57.878.020 I slot update_slots: id 3 | task 7830 | Checking checkpoint with [70, 70] against 74...
2.57.893.847 W slot update_slots: id 3 | task 7830 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
3.01.047.640 I slot print_timing: id 3 | task 7830 | n_decoded = 152, tg = 50.65 t/s, tg_3s = 50.65 t/s
3.04.055.252 I slot print_timing: id 3 | task 7830 | n_decoded = 303, tg = 50.43 t/s, tg_3s = 50.21 t/s
3.07.064.970 I slot print_timing: id 3 | task 7830 | n_decoded = 454, tg = 50.34 t/s, tg_3s = 50.17 t/s
3.10.069.535 I slot print_timing: id 3 | task 7830 | n_decoded = 605, tg = 50.32 t/s, tg_3s = 50.26 t/s
3.13.089.143 I slot print_timing: id 3 | task 7830 | n_decoded = 756, tg = 50.26 t/s, tg_3s = 50.01 t/s
3.16.108.251 I slot print_timing: id 3 | task 7830 | n_decoded = 907, tg = 50.22 t/s, tg_3s = 50.01 t/s
3.19.115.364 I slot print_timing: id 3 | task 7830 | n_decoded = 1057, tg = 50.17 t/s, tg_3s = 49.88 t/s
3.22.123.311 I slot print_timing: id 3 | task 7830 | n_decoded = 1207, tg = 50.13 t/s, tg_3s = 49.87 t/s
3.25.125.025 I slot print_timing: id 3 | task 7830 | n_decoded = 1357, tg = 50.11 t/s, tg_3s = 49.97 t/s
3.28.125.445 I slot print_timing: id 3 | task 7830 | n_decoded = 1506, tg = 50.07 t/s, tg_3s = 49.66 t/s
3.31.130.396 I slot print_timing: id 3 | task 7830 | n_decoded = 1656, tg = 50.05 t/s, tg_3s = 49.92 t/s
3.34.142.381 I slot print_timing: id 3 | task 7830 | n_decoded = 1806, tg = 50.03 t/s, tg_3s = 49.80 t/s
3.37.157.495 I slot print_timing: id 3 | task 7830 | n_decoded = 1956, tg = 50.01 t/s, tg_3s = 49.75 t/s
3.39.923.077 I slot print_timing: id 3 | task 7830 | prompt eval time = 168.63 ms / 35 tokens ( 4.82 ms per token, 207.55 tokens per second)
3.39.923.080 I slot print_timing: id 3 | task 7830 | eval time = 41876.42 ms / 2095 tokens ( 19.99 ms per token, 50.03 tokens per second)
3.39.923.081 I slot print_timing: id 3 | task 7830 | total time = 42045.05 ms / 2130 tokens
3.39.923.082 I slot print_timing: id 3 | task 7830 | graphs reused = 9869
3.39.923.117 I slot release: id 3 | task 7830 | stop processing: n_tokens = 2200, truncated = 0
3.39.923.132 I srv update_slots: all slots are idle
3.39.939.025 I srv operator(): Chat format: peg-native
3.39.939.321 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.779 (> 0.100 thold), f_keep = 0.034
3.39.939.323 I srv get_availabl: updating prompt cache
3.39.939.408 W srv prompt_save: - saving prompt with length 2200, total state size = 105.824 MiB (draft: 0.000 MiB)
3.40.054.999 I srv load: - looking for better prompt, base f_keep = 0.034, sim = 0.779
3.40.055.007 I srv update: - cache state: 5 prompts, 831.418 MiB (limits: 8192.000 MiB, 16384 tokens, 102452 est)
3.40.055.008 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
3.40.055.009 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
3.40.055.012 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
3.40.055.013 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
3.40.055.013 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
3.40.055.015 I srv get_availabl: prompt cache update took 115.69 ms
3.40.055.093 I slot launch_slot_: id 3 | task 9927 | processing task, is_child = 0
3.40.055.094 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.40.055.095 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.40.055.109 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.40.055.110 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.40.055.119 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.40.055.119 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.40.055.138 I slot update_slots: id 3 | task 9927 | Checking checkpoint with [70, 70] against 74...
3.40.070.829 W slot update_slots: id 3 | task 9927 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
3.43.208.605 I slot print_timing: id 3 | task 9927 | n_decoded = 154, tg = 51.14 t/s, tg_3s = 51.14 t/s
3.46.223.716 I slot print_timing: id 3 | task 9927 | n_decoded = 307, tg = 50.94 t/s, tg_3s = 50.74 t/s
3.49.242.573 I slot print_timing: id 3 | task 9927 | n_decoded = 460, tg = 50.85 t/s, tg_3s = 50.68 t/s
3.52.260.156 I slot print_timing: id 3 | task 9927 | n_decoded = 613, tg = 50.82 t/s, tg_3s = 50.70 t/s
3.55.269.621 I slot print_timing: id 3 | task 9927 | n_decoded = 765, tg = 50.76 t/s, tg_3s = 50.51 t/s
3.58.279.717 I slot print_timing: id 3 | task 9927 | n_decoded = 917, tg = 50.71 t/s, tg_3s = 50.50 t/s
4.01.287.611 I slot print_timing: id 3 | task 9927 | n_decoded = 1068, tg = 50.64 t/s, tg_3s = 50.20 t/s
4.04.301.661 I slot print_timing: id 3 | task 9927 | n_decoded = 1219, tg = 50.57 t/s, tg_3s = 50.10 t/s
4.07.313.624 I slot print_timing: id 3 | task 9927 | n_decoded = 1370, tg = 50.52 t/s, tg_3s = 50.13 t/s
4.10.317.875 I slot print_timing: id 3 | task 9927 | n_decoded = 1520, tg = 50.46 t/s, tg_3s = 49.93 t/s
4.13.337.943 I slot print_timing: id 3 | task 9927 | n_decoded = 1671, tg = 50.42 t/s, tg_3s = 50.00 t/s
4.15.087.849 I slot print_timing: id 3 | task 9927 | prompt eval time = 142.07 ms / 24 tokens ( 5.92 ms per token, 168.93 tokens per second)
4.15.087.852 I slot print_timing: id 3 | task 9927 | eval time = 34890.63 ms / 1758 tokens ( 19.85 ms per token, 50.39 tokens per second)
4.15.087.853 I slot print_timing: id 3 | task 9927 | total time = 35032.71 ms / 1782 tokens
4.15.087.854 I slot print_timing: id 3 | task 9927 | graphs reused = 11618
4.15.087.891 I slot release: id 3 | task 9927 | stop processing: n_tokens = 1852, truncated = 0
4.15.087.904 I srv update_slots: all slots are idle
4.15.103.637 I srv operator(): Chat format: peg-native
4.15.103.960 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.824 (> 0.100 thold), f_keep = 0.040
4.15.103.963 I srv get_availabl: updating prompt cache
4.15.104.035 W srv prompt_save: - saving prompt with length 1852, total state size = 99.021 MiB (draft: 0.000 MiB)
4.15.229.760 I srv load: - looking for better prompt, base f_keep = 0.040, sim = 0.824
4.15.229.768 I srv update: - cache state: 6 prompts, 993.251 MiB (limits: 8192.000 MiB, 16384 tokens, 101033 est)
4.15.229.770 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
4.15.229.771 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
4.15.229.771 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
4.15.229.772 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
4.15.229.772 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
4.15.229.773 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
4.15.229.774 I srv get_availabl: prompt cache update took 125.81 ms
4.15.229.864 I slot launch_slot_: id 3 | task 11687 | processing task, is_child = 0
4.15.229.865 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.15.229.865 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.15.229.881 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.15.229.881 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.15.229.890 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.15.229.890 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.15.229.907 I slot update_slots: id 3 | task 11687 | Checking checkpoint with [70, 70] against 75...
4.15.245.832 W slot update_slots: id 3 | task 11687 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
4.18.354.850 I slot print_timing: id 3 | task 11687 | n_decoded = 152, tg = 50.38 t/s, tg_3s = 50.38 t/s
4.21.360.713 I slot print_timing: id 3 | task 11687 | n_decoded = 302, tg = 50.14 t/s, tg_3s = 49.90 t/s
4.24.368.897 I slot print_timing: id 3 | task 11687 | n_decoded = 452, tg = 50.05 t/s, tg_3s = 49.86 t/s
4.27.387.921 I slot print_timing: id 3 | task 11687 | n_decoded = 603, tg = 50.04 t/s, tg_3s = 50.02 t/s
4.30.395.974 I slot print_timing: id 3 | task 11687 | n_decoded = 753, tg = 50.01 t/s, tg_3s = 49.87 t/s
4.33.406.865 I slot print_timing: id 3 | task 11687 | n_decoded = 904, tg = 50.03 t/s, tg_3s = 50.15 t/s
4.36.414.787 I slot print_timing: id 3 | task 11687 | n_decoded = 1055, tg = 50.05 t/s, tg_3s = 50.20 t/s
4.36.812.783 I slot print_timing: id 3 | task 11687 | prompt eval time = 107.62 ms / 20 tokens ( 5.38 ms per token, 185.83 tokens per second)
4.36.812.786 I slot print_timing: id 3 | task 11687 | eval time = 21475.24 ms / 1075 tokens ( 19.98 ms per token, 50.06 tokens per second)
4.36.812.787 I slot print_timing: id 3 | task 11687 | total time = 21582.87 ms / 1095 tokens
4.36.812.788 I slot print_timing: id 3 | task 11687 | graphs reused = 12687
4.36.812.836 I slot release: id 3 | task 11687 | stop processing: n_tokens = 1165, truncated = 0
4.36.812.850 I srv update_slots: all slots are idle
4.36.828.427 I srv operator(): Chat format: peg-native
4.36.828.719 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.806 (> 0.100 thold), f_keep = 0.064
4.36.828.721 I srv get_availabl: updating prompt cache
4.36.828.782 W srv prompt_save: - saving prompt with length 1165, total state size = 85.590 MiB (draft: 0.000 MiB)
4.36.924.706 I srv load: - looking for better prompt, base f_keep = 0.064, sim = 0.806
4.36.924.718 I srv update: - cache state: 7 prompts, 1141.654 MiB (limits: 8192.000 MiB, 16384 tokens, 96260 est)
4.36.924.719 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
4.36.924.720 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
4.36.924.720 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
4.36.924.721 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
4.36.924.722 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
4.36.924.723 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
4.36.924.724 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
4.36.924.725 I srv get_availabl: prompt cache update took 96.00 ms
4.36.924.793 I slot launch_slot_: id 3 | task 12764 | processing task, is_child = 0
4.36.924.794 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.36.924.795 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.36.924.809 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.36.924.809 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.36.924.825 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.36.924.825 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.36.924.842 I slot update_slots: id 3 | task 12764 | Checking checkpoint with [70, 70] against 75...
4.36.940.781 W slot update_slots: id 3 | task 12764 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
4.40.071.938 I slot print_timing: id 3 | task 12764 | n_decoded = 152, tg = 50.55 t/s, tg_3s = 50.54 t/s
4.43.081.646 I slot print_timing: id 3 | task 12764 | n_decoded = 303, tg = 50.36 t/s, tg_3s = 50.17 t/s
4.46.092.433 I slot print_timing: id 3 | task 12764 | n_decoded = 454, tg = 50.29 t/s, tg_3s = 50.15 t/s
4.49.104.015 I slot print_timing: id 3 | task 12764 | n_decoded = 605, tg = 50.25 t/s, tg_3s = 50.14 t/s
4.52.122.903 I slot print_timing: id 3 | task 12764 | n_decoded = 756, tg = 50.21 t/s, tg_3s = 50.02 t/s
4.55.138.150 I slot print_timing: id 3 | task 12764 | n_decoded = 907, tg = 50.18 t/s, tg_3s = 50.08 t/s
4.58.145.184 I slot print_timing: id 3 | task 12764 | n_decoded = 1057, tg = 50.14 t/s, tg_3s = 49.88 t/s
5.01.154.335 I slot print_timing: id 3 | task 12764 | n_decoded = 1207, tg = 50.10 t/s, tg_3s = 49.85 t/s
5.04.174.209 I slot print_timing: id 3 | task 12764 | n_decoded = 1358, tg = 50.09 t/s, tg_3s = 50.00 t/s
5.07.183.182 I slot print_timing: id 3 | task 12764 | n_decoded = 1508, tg = 50.07 t/s, tg_3s = 49.85 t/s
5.09.123.485 I slot print_timing: id 3 | task 12764 | prompt eval time = 139.87 ms / 22 tokens ( 6.36 ms per token, 157.29 tokens per second)
5.09.123.489 I slot print_timing: id 3 | task 12764 | eval time = 32058.77 ms / 1605 tokens ( 19.97 ms per token, 50.06 tokens per second)
5.09.123.489 I slot print_timing: id 3 | task 12764 | total time = 32198.64 ms / 1627 tokens
5.09.123.490 I slot print_timing: id 3 | task 12764 | graphs reused = 14284
5.09.123.529 I slot release: id 3 | task 12764 | stop processing: n_tokens = 1697, truncated = 0
5.09.123.541 I srv update_slots: all slots are idle
5.09.139.807 I srv operator(): Chat format: peg-native
5.09.140.111 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.612 (> 0.100 thold), f_keep = 0.044
5.09.140.113 I srv get_availabl: updating prompt cache
5.09.140.180 W srv prompt_save: - saving prompt with length 1697, total state size = 95.990 MiB (draft: 0.000 MiB)
5.09.240.830 I srv load: - looking for better prompt, base f_keep = 0.044, sim = 0.612
5.09.240.842 I srv update: - cache state: 8 prompts, 1300.458 MiB (limits: 8192.000 MiB, 16384 tokens, 95195 est)
5.09.240.843 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
5.09.240.847 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
5.09.240.848 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
5.09.240.849 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
5.09.240.849 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
5.09.240.851 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
5.09.240.852 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
5.09.240.853 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
5.09.240.854 I srv get_availabl: prompt cache update took 100.74 ms
5.09.240.933 I slot launch_slot_: id 3 | task 14371 | processing task, is_child = 0
5.09.240.934 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.09.240.935 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.09.240.946 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.09.240.947 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.09.240.967 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.09.240.967 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.09.240.995 I slot update_slots: id 3 | task 14371 | Checking checkpoint with [70, 70] against 74...
5.09.256.763 W slot update_slots: id 3 | task 14371 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
5.12.429.284 I slot print_timing: id 3 | task 14371 | n_decoded = 151, tg = 50.30 t/s, tg_3s = 50.30 t/s
5.15.440.242 I slot print_timing: id 3 | task 14371 | n_decoded = 302, tg = 50.22 t/s, tg_3s = 50.15 t/s
5.18.448.164 I slot print_timing: id 3 | task 14371 | n_decoded = 452, tg = 50.10 t/s, tg_3s = 49.87 t/s
5.21.459.620 I slot print_timing: id 3 | task 14371 | n_decoded = 605, tg = 50.28 t/s, tg_3s = 50.81 t/s
5.24.466.641 I slot print_timing: id 3 | task 14371 | n_decoded = 757, tg = 50.33 t/s, tg_3s = 50.55 t/s
5.27.484.236 I slot print_timing: id 3 | task 14371 | n_decoded = 909, tg = 50.34 t/s, tg_3s = 50.37 t/s
5.30.491.452 I slot print_timing: id 3 | task 14371 | n_decoded = 1061, tg = 50.37 t/s, tg_3s = 50.55 t/s
5.33.494.398 I slot print_timing: id 3 | task 14371 | n_decoded = 1212, tg = 50.36 t/s, tg_3s = 50.28 t/s
5.36.501.051 I slot print_timing: id 3 | task 14371 | n_decoded = 1363, tg = 50.34 t/s, tg_3s = 50.22 t/s
5.39.503.603 I slot print_timing: id 3 | task 14371 | n_decoded = 1513, tg = 50.30 t/s, tg_3s = 49.96 t/s
5.42.522.480 I slot print_timing: id 3 | task 14371 | n_decoded = 1664, tg = 50.28 t/s, tg_3s = 50.02 t/s
5.45.293.919 I slot print_timing: id 3 | task 14371 | prompt eval time = 186.05 ms / 50 tokens ( 3.72 ms per token, 268.75 tokens per second)
5.45.293.923 I slot print_timing: id 3 | task 14371 | eval time = 35866.87 ms / 1802 tokens ( 19.90 ms per token, 50.24 tokens per second)
5.45.293.925 I slot print_timing: id 3 | task 14371 | total time = 36052.92 ms / 1852 tokens
5.45.293.925 I slot print_timing: id 3 | task 14371 | graphs reused = 16077
5.45.293.973 I slot release: id 3 | task 14371 | stop processing: n_tokens = 1922, truncated = 0
5.45.293.989 I srv update_slots: all slots are idle
5.45.310.170 I srv operator(): Chat format: peg-native
5.45.310.464 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.532 (> 0.100 thold), f_keep = 0.039
5.45.310.466 I srv get_availabl: updating prompt cache
5.45.310.557 W srv prompt_save: - saving prompt with length 1922, total state size = 100.389 MiB (draft: 0.000 MiB)
5.45.426.936 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.532
5.45.426.947 I srv update: - cache state: 9 prompts, 1463.660 MiB (limits: 8192.000 MiB, 16384 tokens, 95338 est)
5.45.426.948 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
5.45.426.952 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
5.45.426.953 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
5.45.426.954 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
5.45.426.955 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
5.45.426.958 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
5.45.426.959 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
5.45.426.960 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
5.45.426.960 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
5.45.426.962 I srv get_availabl: prompt cache update took 116.49 ms
5.45.427.047 I slot launch_slot_: id 3 | task 16175 | processing task, is_child = 0
5.45.427.049 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.45.427.049 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.45.427.066 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.45.427.067 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.45.427.078 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.45.427.079 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.45.427.099 I slot update_slots: id 3 | task 16175 | Checking checkpoint with [70, 70] against 75...
5.45.443.121 W slot update_slots: id 3 | task 16175 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
5.48.662.966 I slot print_timing: id 3 | task 16175 | n_decoded = 151, tg = 50.22 t/s, tg_3s = 50.22 t/s
5.51.676.540 I slot print_timing: id 3 | task 16175 | n_decoded = 303, tg = 50.33 t/s, tg_3s = 50.44 t/s
5.54.685.566 I slot print_timing: id 3 | task 16175 | n_decoded = 454, tg = 50.28 t/s, tg_3s = 50.18 t/s
5.57.687.510 I slot print_timing: id 3 | task 16175 | n_decoded = 605, tg = 50.29 t/s, tg_3s = 50.30 t/s
6.00.687.559 I slot print_timing: id 3 | task 16175 | n_decoded = 755, tg = 50.23 t/s, tg_3s = 50.00 t/s
6.03.696.746 I slot print_timing: id 3 | task 16175 | n_decoded = 905, tg = 50.17 t/s, tg_3s = 49.85 t/s
6.06.712.861 I slot print_timing: id 3 | task 16175 | n_decoded = 1056, tg = 50.15 t/s, tg_3s = 50.06 t/s
6.09.725.214 I slot print_timing: id 3 | task 16175 | n_decoded = 1206, tg = 50.11 t/s, tg_3s = 49.79 t/s
6.12.743.037 I slot print_timing: id 3 | task 16175 | n_decoded = 1357, tg = 50.10 t/s, tg_3s = 50.04 t/s
6.15.745.833 I slot print_timing: id 3 | task 16175 | n_decoded = 1507, tg = 50.08 t/s, tg_3s = 49.95 t/s
6.18.755.835 I slot print_timing: id 3 | task 16175 | n_decoded = 1657, tg = 50.06 t/s, tg_3s = 49.83 t/s
6.21.056.132 I slot print_timing: id 3 | task 16175 | prompt eval time = 229.28 ms / 70 tokens ( 3.28 ms per token, 305.30 tokens per second)
6.21.056.138 I slot print_timing: id 3 | task 16175 | eval time = 35399.74 ms / 1772 tokens ( 19.98 ms per token, 50.06 tokens per second)
6.21.056.139 I slot print_timing: id 3 | task 16175 | total time = 35629.03 ms / 1842 tokens
6.21.056.140 I slot print_timing: id 3 | task 16175 | graphs reused = 17840
6.21.056.186 I slot release: id 3 | task 16175 | stop processing: n_tokens = 1912, truncated = 0
6.21.056.207 I srv update_slots: all slots are idle
6.21.072.158 I srv operator(): Chat format: peg-native
6.21.072.463 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.643 (> 0.100 thold), f_keep = 0.039
6.21.072.466 I srv get_availabl: updating prompt cache
6.21.072.561 W srv prompt_save: - saving prompt with length 1912, total state size = 100.194 MiB (draft: 0.000 MiB)
6.21.178.498 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.643
6.21.178.508 I srv update: - cache state: 10 prompts, 1626.667 MiB (limits: 8192.000 MiB, 16384 tokens, 95413 est)
6.21.178.509 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
6.21.178.510 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
6.21.178.511 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
6.21.178.515 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
6.21.178.515 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
6.21.178.516 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
6.21.178.516 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
6.21.178.517 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
6.21.178.518 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
6.21.178.519 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
6.21.178.520 I srv get_availabl: prompt cache update took 106.05 ms
6.21.178.588 I slot launch_slot_: id 3 | task 17949 | processing task, is_child = 0
6.21.178.588 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
6.21.178.589 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
6.21.178.603 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
6.21.178.604 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
6.21.178.613 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
6.21.178.613 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
6.21.178.632 I slot update_slots: id 3 | task 17949 | Checking checkpoint with [70, 70] against 74...
6.21.194.431 W slot update_slots: id 3 | task 17949 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
6.24.383.107 I slot print_timing: id 3 | task 17949 | n_decoded = 152, tg = 50.49 t/s, tg_3s = 50.49 t/s
6.27.400.029 I slot print_timing: id 3 | task 17949 | n_decoded = 304, tg = 50.43 t/s, tg_3s = 50.38 t/s
6.30.404.309 I slot print_timing: id 3 | task 17949 | n_decoded = 455, tg = 50.38 t/s, tg_3s = 50.26 t/s
6.33.406.975 I slot print_timing: id 3 | task 17949 | n_decoded = 606, tg = 50.35 t/s, tg_3s = 50.29 t/s
6.36.410.587 I slot print_timing: id 3 | task 17949 | n_decoded = 756, tg = 50.27 t/s, tg_3s = 49.94 t/s
6.39.428.085 I slot print_timing: id 3 | task 17949 | n_decoded = 907, tg = 50.23 t/s, tg_3s = 50.04 t/s
6.42.439.483 I slot print_timing: id 3 | task 17949 | n_decoded = 1057, tg = 50.17 t/s, tg_3s = 49.81 t/s
6.45.449.231 I slot print_timing: id 3 | task 17949 | n_decoded = 1207, tg = 50.13 t/s, tg_3s = 49.84 t/s
6.48.451.701 I slot print_timing: id 3 | task 17949 | n_decoded = 1357, tg = 50.11 t/s, tg_3s = 49.96 t/s
6.51.467.707 I slot print_timing: id 3 | task 17949 | n_decoded = 1507, tg = 50.07 t/s, tg_3s = 49.73 t/s
6.54.470.771 I slot print_timing: id 3 | task 17949 | n_decoded = 1657, tg = 50.06 t/s, tg_3s = 49.95 t/s
6.57.473.271 I slot print_timing: id 3 | task 17949 | n_decoded = 1806, tg = 50.03 t/s, tg_3s = 49.63 t/s
7.00.486.886 I slot print_timing: id 3 | task 17949 | n_decoded = 1957, tg = 50.03 t/s, tg_3s = 50.11 t/s
7.03.489.442 I slot print_timing: id 3 | task 17949 | n_decoded = 2108, tg = 50.05 t/s, tg_3s = 50.29 t/s
7.04.331.070 I slot print_timing: id 3 | task 17949 | prompt eval time = 193.72 ms / 44 tokens ( 4.40 ms per token, 227.14 tokens per second)
7.04.331.073 I slot print_timing: id 3 | task 17949 | eval time = 42958.72 ms / 2150 tokens ( 19.98 ms per token, 50.05 tokens per second)
7.04.331.074 I slot print_timing: id 3 | task 17949 | total time = 43152.44 ms / 2194 tokens
7.04.331.075 I slot print_timing: id 3 | task 17949 | graphs reused = 19980
7.04.331.109 I slot release: id 3 | task 17949 | stop processing: n_tokens = 2264, truncated = 0
7.04.331.126 I srv update_slots: all slots are idle
7.04.347.221 I srv operator(): Chat format: peg-native
7.04.347.511 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.525 (> 0.100 thold), f_keep = 0.033
7.04.347.513 I srv get_availabl: updating prompt cache
7.04.347.613 W srv prompt_save: - saving prompt with length 2264, total state size = 107.075 MiB (draft: 0.000 MiB)
7.04.465.406 I srv load: - looking for better prompt, base f_keep = 0.033, sim = 0.525
7.04.465.415 I srv update: - cache state: 11 prompts, 1796.556 MiB (limits: 8192.000 MiB, 16384 tokens, 96714 est)
7.04.465.416 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
7.04.465.417 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
7.04.465.418 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
7.04.465.421 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
7.04.465.421 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
7.04.465.423 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
7.04.465.423 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
7.04.465.431 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
7.04.465.432 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
7.04.465.432 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
7.04.465.433 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
7.04.465.435 I srv get_availabl: prompt cache update took 117.92 ms
7.04.465.517 I slot launch_slot_: id 3 | task 20101 | processing task, is_child = 0
7.04.465.517 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
7.04.465.518 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
7.04.465.533 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
7.04.465.533 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
7.04.465.542 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
7.04.465.543 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
7.04.465.559 I slot update_slots: id 3 | task 20101 | Checking checkpoint with [70, 70] against 74...
7.04.481.614 W slot update_slots: id 3 | task 20101 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
7.07.708.276 I slot print_timing: id 3 | task 20101 | n_decoded = 153, tg = 50.78 t/s, tg_3s = 50.78 t/s
7.10.720.121 I slot print_timing: id 3 | task 20101 | n_decoded = 306, tg = 50.79 t/s, tg_3s = 50.80 t/s
7.13.728.160 I slot print_timing: id 3 | task 20101 | n_decoded = 458, tg = 50.70 t/s, tg_3s = 50.53 t/s
7.16.746.272 I slot print_timing: id 3 | task 20101 | n_decoded = 611, tg = 50.70 t/s, tg_3s = 50.69 t/s
7.19.747.746 I slot print_timing: id 3 | task 20101 | n_decoded = 762, tg = 50.62 t/s, tg_3s = 50.31 t/s
7.22.753.523 I slot print_timing: id 3 | task 20101 | n_decoded = 913, tg = 50.56 t/s, tg_3s = 50.24 t/s
7.25.760.442 I slot print_timing: id 3 | task 20101 | n_decoded = 1064, tg = 50.51 t/s, tg_3s = 50.22 t/s
7.28.766.610 I slot print_timing: id 3 | task 20101 | n_decoded = 1214, tg = 50.43 t/s, tg_3s = 49.90 t/s
7.31.773.591 I slot print_timing: id 3 | task 20101 | n_decoded = 1364, tg = 50.37 t/s, tg_3s = 49.88 t/s
7.34.787.602 I slot print_timing: id 3 | task 20101 | n_decoded = 1514, tg = 50.31 t/s, tg_3s = 49.77 t/s
7.37.800.835 I slot print_timing: id 3 | task 20101 | n_decoded = 1664, tg = 50.26 t/s, tg_3s = 49.78 t/s
7.38.861.928 I slot print_timing: id 3 | task 20101 | prompt eval time = 229.85 ms / 70 tokens ( 3.28 ms per token, 304.55 tokens per second)
7.38.861.932 I slot print_timing: id 3 | task 20101 | eval time = 34166.52 ms / 1717 tokens ( 19.90 ms per token, 50.25 tokens per second)
7.38.861.932 I slot print_timing: id 3 | task 20101 | total time = 34396.36 ms / 1787 tokens
7.38.861.933 I slot print_timing: id 3 | task 20101 | graphs reused = 21688
7.38.861.972 I slot release: id 3 | task 20101 | stop processing: n_tokens = 1857, truncated = 0
7.38.861.986 I srv update_slots: all slots are idle
7.38.877.721 I srv operator(): Chat format: peg-native
7.38.878.022 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.552 (> 0.100 thold), f_keep = 0.040
7.38.878.024 I srv get_availabl: updating prompt cache
7.38.878.101 W srv prompt_save: - saving prompt with length 1857, total state size = 99.118 MiB (draft: 0.000 MiB)
7.38.986.680 I srv load: - looking for better prompt, base f_keep = 0.040, sim = 0.552
7.38.986.692 I srv update: - cache state: 12 prompts, 1958.487 MiB (limits: 8192.000 MiB, 16384 tokens, 96485 est)
7.38.986.695 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
7.38.986.695 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
7.38.986.696 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
7.38.986.697 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
7.38.986.697 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
7.38.986.700 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
7.38.986.701 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
7.38.986.701 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
7.38.986.702 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
7.38.986.702 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
7.38.986.704 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
7.38.986.705 I srv update: - prompt 0x64b5df404d90: 1857 tokens, checkpoints: 1, 161.932 MiB
7.38.986.706 I srv get_availabl: prompt cache update took 108.68 ms
7.38.986.777 I slot launch_slot_: id 3 | task 21820 | processing task, is_child = 0
7.38.986.778 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
7.38.986.778 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
7.38.986.793 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
7.38.986.793 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
7.38.986.802 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
7.38.986.803 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
7.38.987.036 I slot update_slots: id 3 | task 21820 | Checking checkpoint with [70, 70] against 74...
7.39.002.922 W slot update_slots: id 3 | task 21820 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
7.42.240.539 I slot print_timing: id 3 | task 21820 | n_decoded = 151, tg = 50.13 t/s, tg_3s = 50.13 t/s
7.45.256.676 I slot print_timing: id 3 | task 21820 | n_decoded = 302, tg = 50.10 t/s, tg_3s = 50.06 t/s
7.48.274.276 I slot print_timing: id 3 | task 21820 | n_decoded = 453, tg = 50.08 t/s, tg_3s = 50.04 t/s
7.51.290.757 I slot print_timing: id 3 | task 21820 | n_decoded = 604, tg = 50.07 t/s, tg_3s = 50.06 t/s
7.54.303.711 I slot print_timing: id 3 | task 21820 | n_decoded = 755, tg = 50.08 t/s, tg_3s = 50.12 t/s
7.57.307.607 I slot print_timing: id 3 | task 21820 | n_decoded = 906, tg = 50.11 t/s, tg_3s = 50.27 t/s
8.00.318.949 I slot print_timing: id 3 | task 21820 | n_decoded = 1058, tg = 50.16 t/s, tg_3s = 50.48 t/s
8.03.319.536 I slot print_timing: id 3 | task 21820 | n_decoded = 1208, tg = 50.14 t/s, tg_3s = 49.99 t/s
8.06.334.519 I slot print_timing: id 3 | task 21820 | n_decoded = 1359, tg = 50.14 t/s, tg_3s = 50.08 t/s
8.09.343.934 I slot print_timing: id 3 | task 21820 | n_decoded = 1509, tg = 50.11 t/s, tg_3s = 49.84 t/s
8.12.365.232 I slot print_timing: id 3 | task 21820 | n_decoded = 1660, tg = 50.10 t/s, tg_3s = 49.98 t/s
8.14.713.153 I slot print_timing: id 3 | task 21820 | prompt eval time = 241.26 ms / 63 tokens ( 3.83 ms per token, 261.13 tokens per second)
8.14.713.157 I slot print_timing: id 3 | task 21820 | eval time = 35484.85 ms / 1777 tokens ( 19.97 ms per token, 50.08 tokens per second)
8.14.713.158 I slot print_timing: id 3 | task 21820 | total time = 35726.11 ms / 1840 tokens
8.14.713.158 I slot print_timing: id 3 | task 21820 | graphs reused = 23456
8.14.713.200 I slot release: id 3 | task 21820 | stop processing: n_tokens = 1910, truncated = 0
8.14.713.212 I srv update_slots: all slots are idle
8.14.729.005 I srv operator(): Chat format: peg-native
8.14.729.296 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.615 (> 0.100 thold), f_keep = 0.039
8.14.729.298 I srv get_availabl: updating prompt cache
8.14.729.383 W srv prompt_save: - saving prompt with length 1910, total state size = 100.155 MiB (draft: 0.000 MiB)
8.14.838.385 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.615
8.14.838.397 I srv update: - cache state: 13 prompts, 2121.455 MiB (limits: 8192.000 MiB, 16384 tokens, 96448 est)
8.14.838.398 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
8.14.838.399 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
8.14.838.400 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
8.14.838.401 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
8.14.838.401 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
8.14.838.402 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
8.14.838.407 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
8.14.838.408 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
8.14.838.409 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
8.14.838.409 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
8.14.838.410 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
8.14.838.414 I srv update: - prompt 0x64b5df404d90: 1857 tokens, checkpoints: 1, 161.932 MiB
8.14.838.414 I srv update: - prompt 0x64b5df47e7d0: 1910 tokens, checkpoints: 1, 162.968 MiB
8.14.838.416 I srv get_availabl: prompt cache update took 109.12 ms
8.14.838.501 I slot launch_slot_: id 3 | task 23599 | processing task, is_child = 0
8.14.838.501 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
8.14.838.502 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
8.14.838.518 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
8.14.838.518 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
8.14.838.541 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
8.14.838.541 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
8.14.838.562 I slot update_slots: id 3 | task 23599 | Checking checkpoint with [70, 70] against 75...
8.14.854.546 W slot update_slots: id 3 | task 23599 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
8.18.028.785 I slot print_timing: id 3 | task 23599 | n_decoded = 151, tg = 50.26 t/s, tg_3s = 50.26 t/s
8.21.032.486 I slot print_timing: id 3 | task 23599 | n_decoded = 302, tg = 50.26 t/s, tg_3s = 50.27 t/s
8.24.052.118 I slot print_timing: id 3 | task 23599 | n_decoded = 453, tg = 50.18 t/s, tg_3s = 50.01 t/s
8.27.066.189 I slot print_timing: id 3 | task 23599 | n_decoded = 604, tg = 50.16 t/s, tg_3s = 50.10 t/s
8.30.073.134 I slot print_timing: id 3 | task 23599 | n_decoded = 754, tg = 50.10 t/s, tg_3s = 49.88 t/s
8.33.100.249 I slot print_timing: id 3 | task 23599 | n_decoded = 905, tg = 50.07 t/s, tg_3s = 49.88 t/s
8.36.105.453 I slot print_timing: id 3 | task 23599 | n_decoded = 1055, tg = 50.04 t/s, tg_3s = 49.91 t/s
8.39.107.327 I slot print_timing: id 3 | task 23599 | n_decoded = 1205, tg = 50.04 t/s, tg_3s = 49.97 t/s
8.42.113.977 I slot print_timing: id 3 | task 23599 | n_decoded = 1357, tg = 50.09 t/s, tg_3s = 50.55 t/s
8.45.118.884 I slot print_timing: id 3 | task 23599 | n_decoded = 1508, tg = 50.11 t/s, tg_3s = 50.25 t/s
8.48.137.549 I slot print_timing: id 3 | task 23599 | n_decoded = 1660, tg = 50.13 t/s, tg_3s = 50.35 t/s
8.51.139.319 I slot print_timing: id 3 | task 23599 | n_decoded = 1810, tg = 50.12 t/s, tg_3s = 49.97 t/s
8.54.148.359 I slot print_timing: id 3 | task 23599 | n_decoded = 1960, tg = 50.10 t/s, tg_3s = 49.85 t/s
8.57.151.386 I slot print_timing: id 3 | task 23599 | n_decoded = 2110, tg = 50.09 t/s, tg_3s = 49.95 t/s
8.58.980.529 I slot print_timing: id 3 | task 23599 | prompt eval time = 185.65 ms / 51 tokens ( 3.64 ms per token, 274.71 tokens per second)
8.58.980.533 I slot print_timing: id 3 | task 23599 | eval time = 43956.31 ms / 2201 tokens ( 19.97 ms per token, 50.07 tokens per second)
8.58.980.533 I slot print_timing: id 3 | task 23599 | total time = 44141.96 ms / 2252 tokens
8.58.980.534 I slot print_timing: id 3 | task 23599 | graphs reused = 25646
8.58.980.569 I slot release: id 3 | task 23599 | stop processing: n_tokens = 2322, truncated = 0
8.58.980.584 I srv update_slots: all slots are idle
8.58.996.320 I srv operator(): Chat format: peg-native
8.58.996.612 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.587 (> 0.100 thold), f_keep = 0.032
8.58.996.613 I srv get_availabl: updating prompt cache
8.58.996.712 W srv prompt_save: - saving prompt with length 2322, total state size = 108.209 MiB (draft: 0.000 MiB)
8.59.107.545 I srv load: - looking for better prompt, base f_keep = 0.032, sim = 0.587
8.59.107.559 I srv update: - cache state: 14 prompts, 2292.478 MiB (limits: 8192.000 MiB, 16384 tokens, 97550 est)
8.59.107.561 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
8.59.107.562 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
8.59.107.563 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
8.59.107.564 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
8.59.107.564 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
8.59.107.565 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
8.59.107.566 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
8.59.107.566 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
8.59.107.567 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
8.59.107.567 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
8.59.107.568 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
8.59.107.569 I srv update: - prompt 0x64b5df404d90: 1857 tokens, checkpoints: 1, 161.932 MiB
8.59.107.569 I srv update: - prompt 0x64b5df47e7d0: 1910 tokens, checkpoints: 1, 162.968 MiB
8.59.107.570 I srv update: - prompt 0x64b5d59d2980: 2322 tokens, checkpoints: 1, 171.023 MiB
8.59.107.571 I srv get_availabl: prompt cache update took 110.96 ms
8.59.107.655 I slot launch_slot_: id 3 | task 25802 | processing task, is_child = 0
8.59.107.656 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
8.59.107.656 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
8.59.107.672 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
8.59.107.672 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
8.59.107.695 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
8.59.107.696 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
8.59.107.729 I slot update_slots: id 3 | task 25802 | Checking checkpoint with [70, 70] against 74...
8.59.123.725 W slot update_slots: id 3 | task 25802 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
9.02.333.133 I slot print_timing: id 3 | task 25802 | n_decoded = 152, tg = 50.49 t/s, tg_3s = 50.49 t/s
9.05.337.913 I slot print_timing: id 3 | task 25802 | n_decoded = 303, tg = 50.37 t/s, tg_3s = 50.25 t/s
9.08.347.419 I slot print_timing: id 3 | task 25802 | n_decoded = 454, tg = 50.31 t/s, tg_3s = 50.17 t/s
9.11.354.764 I slot print_timing: id 3 | task 25802 | n_decoded = 605, tg = 50.28 t/s, tg_3s = 50.21 t/s
9.14.365.475 I slot print_timing: id 3 | task 25802 | n_decoded = 755, tg = 50.19 t/s, tg_3s = 49.82 t/s
9.17.378.531 I slot print_timing: id 3 | task 25802 | n_decoded = 905, tg = 50.12 t/s, tg_3s = 49.78 t/s
9.20.381.685 I slot print_timing: id 3 | task 25802 | n_decoded = 1055, tg = 50.10 t/s, tg_3s = 49.95 t/s
9.23.392.582 I slot print_timing: id 3 | task 25802 | n_decoded = 1205, tg = 50.06 t/s, tg_3s = 49.82 t/s
9.26.392.630 I slot print_timing: id 3 | task 25802 | n_decoded = 1355, tg = 50.06 t/s, tg_3s = 50.00 t/s
9.29.402.599 I slot print_timing: id 3 | task 25802 | n_decoded = 1505, tg = 50.03 t/s, tg_3s = 49.83 t/s
9.32.421.604 I slot print_timing: id 3 | task 25802 | n_decoded = 1656, tg = 50.03 t/s, tg_3s = 50.02 t/s
9.35.423.755 I slot print_timing: id 3 | task 25802 | n_decoded = 1806, tg = 50.03 t/s, tg_3s = 49.96 t/s
9.38.429.705 I slot print_timing: id 3 | task 25802 | n_decoded = 1956, tg = 50.02 t/s, tg_3s = 49.90 t/s
9.41.448.607 I slot print_timing: id 3 | task 25802 | n_decoded = 2107, tg = 50.02 t/s, tg_3s = 50.02 t/s
9.44.465.876 I slot print_timing: id 3 | task 25802 | n_decoded = 2257, tg = 50.00 t/s, tg_3s = 49.71 t/s
9.47.470.697 I slot print_timing: id 3 | task 25802 | n_decoded = 2407, tg = 49.99 t/s, tg_3s = 49.92 t/s
9.50.480.069 I slot print_timing: id 3 | task 25802 | n_decoded = 2557, tg = 49.98 t/s, tg_3s = 49.84 t/s
9.53.491.997 I slot print_timing: id 3 | task 25802 | n_decoded = 2707, tg = 49.97 t/s, tg_3s = 49.80 t/s
9.53.972.856 I slot print_timing: id 3 | task 25802 | prompt eval time = 214.92 ms / 55 tokens ( 3.91 ms per token, 255.91 tokens per second)
9.53.972.859 I slot print_timing: id 3 | task 25802 | eval time = 54650.20 ms / 2731 tokens ( 20.01 ms per token, 49.97 tokens per second)
9.53.972.860 I slot print_timing: id 3 | task 25802 | total time = 54865.12 ms / 2786 tokens
9.53.972.861 I slot print_timing: id 3 | task 25802 | graphs reused = 28364
9.53.972.900 I slot release: id 3 | task 25802 | stop processing: n_tokens = 2856, truncated = 0
9.53.972.916 I srv update_slots: all slots are idle
9.53.988.837 I srv operator(): Chat format: peg-native
9.53.989.130 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.638 (> 0.100 thold), f_keep = 0.026
9.53.989.132 I srv get_availabl: updating prompt cache
9.53.989.247 W srv prompt_save: - saving prompt with length 2856, total state size = 118.649 MiB (draft: 0.000 MiB)
9.54.106.354 I srv load: - looking for better prompt, base f_keep = 0.026, sim = 0.638
9.54.106.365 I srv update: - cache state: 15 prompts, 2473.940 MiB (limits: 8192.000 MiB, 16384 tokens, 99852 est)
9.54.106.366 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
9.54.106.367 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
9.54.106.367 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
9.54.106.368 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
9.54.106.368 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
9.54.106.369 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
9.54.106.369 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
9.54.106.370 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
9.54.106.370 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
9.54.106.371 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
9.54.106.371 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
9.54.106.372 I srv update: - prompt 0x64b5df404d90: 1857 tokens, checkpoints: 1, 161.932 MiB
9.54.106.372 I srv update: - prompt 0x64b5df47e7d0: 1910 tokens, checkpoints: 1, 162.968 MiB
9.54.106.373 I srv update: - prompt 0x64b5d59d2980: 2322 tokens, checkpoints: 1, 171.023 MiB
9.54.106.373 I srv update: - prompt 0x64b5df7e5630: 2856 tokens, checkpoints: 1, 181.462 MiB
9.54.106.374 I srv get_availabl: prompt cache update took 117.24 ms
9.54.106.441 I slot launch_slot_: id 3 | task 28535 | processing task, is_child = 0
9.54.106.442 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
9.54.106.443 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
9.54.106.456 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
9.54.106.456 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
9.54.106.466 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
9.54.106.466 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
9.54.106.485 I slot update_slots: id 3 | task 28535 | Checking checkpoint with [70, 70] against 74...
9.54.122.207 W slot update_slots: id 3 | task 28535 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
9.57.297.679 I slot print_timing: id 3 | task 28535 | n_decoded = 152, tg = 50.50 t/s, tg_3s = 50.50 t/s
10.00.312.067 I slot print_timing: id 3 | task 28535 | n_decoded = 304, tg = 50.46 t/s, tg_3s = 50.42 t/s
10.03.328.880 I slot print_timing: id 3 | task 28535 | n_decoded = 455, tg = 50.33 t/s, tg_3s = 50.05 t/s
10.06.343.790 I slot print_timing: id 3 | task 28535 | n_decoded = 606, tg = 50.27 t/s, tg_3s = 50.08 t/s
10.09.352.937 I slot print_timing: id 3 | task 28535 | n_decoded = 756, tg = 50.18 t/s, tg_3s = 49.85 t/s
10.12.368.210 I slot print_timing: id 3 | task 28535 | n_decoded = 907, tg = 50.17 t/s, tg_3s = 50.08 t/s
10.15.379.751 I slot print_timing: id 3 | task 28535 | n_decoded = 1057, tg = 50.11 t/s, tg_3s = 49.81 t/s
10.18.388.776 I slot print_timing: id 3 | task 28535 | n_decoded = 1207, tg = 50.08 t/s, tg_3s = 49.85 t/s
10.21.401.736 I slot print_timing: id 3 | task 28535 | n_decoded = 1359, tg = 50.12 t/s, tg_3s = 50.45 t/s
10.24.411.480 I slot print_timing: id 3 | task 28535 | n_decoded = 1510, tg = 50.13 t/s, tg_3s = 50.17 t/s
10.27.419.217 I slot print_timing: id 3 | task 28535 | n_decoded = 1661, tg = 50.13 t/s, tg_3s = 50.20 t/s
10.30.426.611 I slot print_timing: id 3 | task 28535 | n_decoded = 1811, tg = 50.11 t/s, tg_3s = 49.88 t/s
10.33.439.274 I slot print_timing: id 3 | task 28535 | n_decoded = 1961, tg = 50.09 t/s, tg_3s = 49.79 t/s
10.36.444.379 I slot print_timing: id 3 | task 28535 | n_decoded = 2111, tg = 50.08 t/s, tg_3s = 49.92 t/s
10.39.454.385 I slot print_timing: id 3 | task 28535 | n_decoded = 2261, tg = 50.06 t/s, tg_3s = 49.83 t/s
10.42.462.537 I slot print_timing: id 3 | task 28535 | n_decoded = 2411, tg = 50.05 t/s, tg_3s = 49.86 t/s
10.45.480.452 I slot print_timing: id 3 | task 28535 | n_decoded = 2561, tg = 50.03 t/s, tg_3s = 49.70 t/s
10.47.548.540 I slot print_timing: id 3 | task 28535 | prompt eval time = 181.53 ms / 45 tokens ( 4.03 ms per token, 247.89 tokens per second)
10.47.548.544 I slot print_timing: id 3 | task 28535 | eval time = 53260.52 ms / 2664 tokens ( 19.99 ms per token, 50.02 tokens per second)
10.47.548.545 I slot print_timing: id 3 | task 28535 | total time = 53442.05 ms / 2709 tokens
10.47.548.545 I slot print_timing: id 3 | task 28535 | graphs reused = 31016
10.47.548.586 I slot release: id 3 | task 28535 | stop processing: n_tokens = 2779, truncated = 0
10.47.548.604 I srv update_slots: all slots are idle
10.47.564.731 I srv operator(): Chat format: peg-native
10.47.565.032 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.643 (> 0.100 thold), f_keep = 0.027
10.47.565.034 I srv get_availabl: updating prompt cache
10.47.565.149 W srv prompt_save: - saving prompt with length 2779, total state size = 117.144 MiB (draft: 0.000 MiB)
10.47.678.524 I srv load: - looking for better prompt, base f_keep = 0.027, sim = 0.643
10.47.678.537 I srv update: - cache state: 16 prompts, 2653.897 MiB (limits: 8192.000 MiB, 16384 tokens, 101660 est)
10.47.678.538 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
10.47.678.539 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
10.47.678.540 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
10.47.678.541 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
10.47.678.541 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
10.47.678.542 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
10.47.678.543 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
10.47.678.543 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
10.47.678.544 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
10.47.678.545 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
10.47.678.545 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
10.47.678.546 I srv update: - prompt 0x64b5df404d90: 1857 tokens, checkpoints: 1, 161.932 MiB
10.47.678.547 I srv update: - prompt 0x64b5df47e7d0: 1910 tokens, checkpoints: 1, 162.968 MiB
10.47.678.547 I srv update: - prompt 0x64b5d59d2980: 2322 tokens, checkpoints: 1, 171.023 MiB
10.47.678.548 I srv update: - prompt 0x64b5df7e5630: 2856 tokens, checkpoints: 1, 181.462 MiB
10.47.678.549 I srv update: - prompt 0x64b5df42df00: 2779 tokens, checkpoints: 1, 179.957 MiB
10.47.678.550 I srv get_availabl: prompt cache update took 113.52 ms
10.47.678.636 I slot launch_slot_: id 3 | task 31201 | processing task, is_child = 0
10.47.678.638 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
10.47.678.639 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
10.47.678.659 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
10.47.678.660 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
10.47.678.678 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
10.47.678.678 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
10.47.678.703 I slot update_slots: id 3 | task 31201 | Checking checkpoint with [70, 70] against 74...
10.47.694.567 W slot update_slots: id 3 | task 31201 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
10.50.887.539 I slot print_timing: id 3 | task 31201 | n_decoded = 151, tg = 50.07 t/s, tg_3s = 50.07 t/s
10.53.904.429 I slot print_timing: id 3 | task 31201 | n_decoded = 302, tg = 50.06 t/s, tg_3s = 50.05 t/s
10.56.916.309 I slot print_timing: id 3 | task 31201 | n_decoded = 452, tg = 49.98 t/s, tg_3s = 49.80 t/s
10.59.916.323 I slot print_timing: id 3 | task 31201 | n_decoded = 602, tg = 49.98 t/s, tg_3s = 50.00 t/s
11.02.927.507 I slot print_timing: id 3 | task 31201 | n_decoded = 752, tg = 49.95 t/s, tg_3s = 49.81 t/s
11.05.927.615 I slot print_timing: id 3 | task 31201 | n_decoded = 902, tg = 49.96 t/s, tg_3s = 50.00 t/s
11.08.937.568 I slot print_timing: id 3 | task 31201 | n_decoded = 1052, tg = 49.94 t/s, tg_3s = 49.83 t/s
11.11.952.712 I slot print_timing: id 3 | task 31201 | n_decoded = 1202, tg = 49.92 t/s, tg_3s = 49.75 t/s
11.14.967.504 I slot print_timing: id 3 | task 31201 | n_decoded = 1353, tg = 49.93 t/s, tg_3s = 50.09 t/s
11.17.974.624 I slot print_timing: id 3 | task 31201 | n_decoded = 1503, tg = 49.93 t/s, tg_3s = 49.88 t/s
11.20.992.830 I slot print_timing: id 3 | task 31201 | n_decoded = 1654, tg = 49.94 t/s, tg_3s = 50.03 t/s
11.24.001.508 I slot print_timing: id 3 | task 31201 | n_decoded = 1804, tg = 49.93 t/s, tg_3s = 49.86 t/s
11.27.010.574 I slot print_timing: id 3 | task 31201 | n_decoded = 1954, tg = 49.93 t/s, tg_3s = 49.85 t/s
11.30.012.721 I slot print_timing: id 3 | task 31201 | n_decoded = 2104, tg = 49.93 t/s, tg_3s = 49.96 t/s
11.33.028.013 I slot print_timing: id 3 | task 31201 | n_decoded = 2254, tg = 49.92 t/s, tg_3s = 49.75 t/s
11.36.033.978 I slot print_timing: id 3 | task 31201 | n_decoded = 2404, tg = 49.91 t/s, tg_3s = 49.90 t/s
11.39.050.507 I slot print_timing: id 3 | task 31201 | n_decoded = 2554, tg = 49.90 t/s, tg_3s = 49.73 t/s
11.42.074.403 I slot print_timing: id 3 | task 31201 | n_decoded = 2704, tg = 49.89 t/s, tg_3s = 49.60 t/s
11.45.086.646 I slot print_timing: id 3 | task 31201 | n_decoded = 2854, tg = 49.88 t/s, tg_3s = 49.80 t/s
11.48.099.858 I slot print_timing: id 3 | task 31201 | n_decoded = 3003, tg = 49.86 t/s, tg_3s = 49.45 t/s
11.49.850.392 I slot print_timing: id 3 | task 31201 | prompt eval time = 193.17 ms / 44 tokens ( 4.39 ms per token, 227.77 tokens per second)
11.49.850.395 I slot print_timing: id 3 | task 31201 | eval time = 61978.51 ms / 3090 tokens ( 20.06 ms per token, 49.86 tokens per second)
11.49.850.396 I slot print_timing: id 3 | task 31201 | total time = 62171.68 ms / 3134 tokens
11.49.850.397 I slot print_timing: id 3 | task 31201 | graphs reused = 34092
11.49.850.436 I slot release: id 3 | task 31201 | stop processing: n_tokens = 3204, truncated = 0
11.49.850.454 I srv update_slots: all slots are idle
11.49.866.448 I srv operator(): Chat format: peg-native
11.49.866.740 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.692 (> 0.100 thold), f_keep = 0.023
11.49.866.742 I srv get_availabl: updating prompt cache
11.49.866.875 W srv prompt_save: - saving prompt with length 3204, total state size = 125.453 MiB (draft: 0.000 MiB)
11.49.984.831 I srv load: - looking for better prompt, base f_keep = 0.023, sim = 0.692
11.49.984.844 I srv update: - cache state: 17 prompts, 2842.163 MiB (limits: 8192.000 MiB, 16384 tokens, 104160 est)
11.49.984.845 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
11.49.984.846 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
11.49.984.846 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
11.49.984.847 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
11.49.984.847 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
11.49.984.848 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
11.49.984.848 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
11.49.984.849 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
11.49.984.849 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
11.49.984.850 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
11.49.984.850 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
11.49.984.851 I srv update: - prompt 0x64b5df404d90: 1857 tokens, checkpoints: 1, 161.932 MiB
11.49.984.851 I srv update: - prompt 0x64b5df47e7d0: 1910 tokens, checkpoints: 1, 162.968 MiB
11.49.984.852 I srv update: - prompt 0x64b5d59d2980: 2322 tokens, checkpoints: 1, 171.023 MiB
11.49.984.853 I srv update: - prompt 0x64b5df7e5630: 2856 tokens, checkpoints: 1, 181.462 MiB
11.49.984.853 I srv update: - prompt 0x64b5df42df00: 2779 tokens, checkpoints: 1, 179.957 MiB
11.49.984.854 I srv update: - prompt 0x64b5df77c3c0: 3204 tokens, checkpoints: 1, 188.266 MiB
11.49.984.855 I srv get_availabl: prompt cache update took 118.11 ms
11.49.984.924 I slot launch_slot_: id 3 | task 34293 | processing task, is_child = 0
11.49.984.924 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
11.49.984.925 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
11.49.984.938 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
11.49.984.938 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
11.49.984.949 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
11.49.984.949 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
11.49.984.968 I slot update_slots: id 3 | task 34293 | Checking checkpoint with [70, 70] against 74...
11.50.000.640 W slot update_slots: id 3 | task 34293 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
11.53.192.226 I slot print_timing: id 3 | task 34293 | n_decoded = 152, tg = 50.20 t/s, tg_3s = 50.20 t/s
11.56.206.010 I slot print_timing: id 3 | task 34293 | n_decoded = 303, tg = 50.15 t/s, tg_3s = 50.10 t/s
11.59.216.704 I slot print_timing: id 3 | task 34293 | n_decoded = 454, tg = 50.15 t/s, tg_3s = 50.15 t/s
12.02.222.542 I slot print_timing: id 3 | task 34293 | n_decoded = 607, tg = 50.34 t/s, tg_3s = 50.90 t/s
12.05.234.604 I slot print_timing: id 3 | task 34293 | n_decoded = 759, tg = 50.36 t/s, tg_3s = 50.46 t/s
12.08.239.545 I slot print_timing: id 3 | task 34293 | n_decoded = 911, tg = 50.40 t/s, tg_3s = 50.58 t/s
12.11.241.162 I slot print_timing: id 3 | task 34293 | n_decoded = 1062, tg = 50.39 t/s, tg_3s = 50.31 t/s
12.13.807.152 I slot print_timing: id 3 | task 34293 | prompt eval time = 179.46 ms / 36 tokens ( 4.98 ms per token, 200.61 tokens per second)
12.13.807.155 I slot print_timing: id 3 | task 34293 | eval time = 23642.72 ms / 1191 tokens ( 19.85 ms per token, 50.37 tokens per second)
12.13.807.156 I slot print_timing: id 3 | task 34293 | total time = 23822.18 ms / 1227 tokens
12.13.807.156 I slot print_timing: id 3 | task 34293 | graphs reused = 35276
12.13.807.195 I slot release: id 3 | task 34293 | stop processing: n_tokens = 1297, truncated = 0
12.13.807.208 I srv update_slots: all slots are idle
12.13.823.741 I srv operator(): Chat format: peg-native
12.13.824.050 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.763 (> 0.100 thold), f_keep = 0.057
12.13.824.052 I srv get_availabl: updating prompt cache
12.13.824.112 W srv prompt_save: - saving prompt with length 1297, total state size = 88.170 MiB (draft: 0.000 MiB)
12.13.918.241 I srv load: - looking for better prompt, base f_keep = 0.057, sim = 0.763
12.13.918.256 I srv update: - cache state: 18 prompts, 2993.146 MiB (limits: 8192.000 MiB, 16384 tokens, 102456 est)
12.13.918.257 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
12.13.918.257 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
12.13.918.258 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
12.13.918.259 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
12.13.918.259 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
12.13.918.260 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
12.13.918.260 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
12.13.918.261 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
12.13.918.261 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
12.13.918.262 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
12.13.918.262 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
12.13.918.263 I srv update: - prompt 0x64b5df404d90: 1857 tokens, checkpoints: 1, 161.932 MiB
12.13.918.263 I srv update: - prompt 0x64b5df47e7d0: 1910 tokens, checkpoints: 1, 162.968 MiB
12.13.918.264 I srv update: - prompt 0x64b5d59d2980: 2322 tokens, checkpoints: 1, 171.023 MiB
12.13.918.264 I srv update: - prompt 0x64b5df7e5630: 2856 tokens, checkpoints: 1, 181.462 MiB
12.13.918.266 I srv update: - prompt 0x64b5df42df00: 2779 tokens, checkpoints: 1, 179.957 MiB
12.13.918.266 I srv update: - prompt 0x64b5df77c3c0: 3204 tokens, checkpoints: 1, 188.266 MiB
12.13.918.267 I srv update: - prompt 0x64b5df923f90: 1297 tokens, checkpoints: 1, 150.983 MiB
12.13.918.269 I srv get_availabl: prompt cache update took 94.22 ms
12.13.918.345 I slot launch_slot_: id 3 | task 35486 | processing task, is_child = 0
12.13.918.346 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
12.13.918.346 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
12.13.918.358 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
12.13.918.358 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
12.13.918.367 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
12.13.918.367 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
12.13.918.384 I slot update_slots: id 3 | task 35486 | Checking checkpoint with [70, 70] against 74...
12.13.934.334 W slot update_slots: id 3 | task 35486 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
12.17.061.097 I slot print_timing: id 3 | task 35486 | n_decoded = 152, tg = 50.65 t/s, tg_3s = 50.65 t/s
12.20.064.341 I slot print_timing: id 3 | task 35486 | n_decoded = 303, tg = 50.46 t/s, tg_3s = 50.28 t/s
12.23.083.822 I slot print_timing: id 3 | task 35486 | n_decoded = 454, tg = 50.31 t/s, tg_3s = 50.01 t/s
12.26.096.150 I slot print_timing: id 3 | task 35486 | n_decoded = 605, tg = 50.26 t/s, tg_3s = 50.13 t/s
12.29.107.314 I slot print_timing: id 3 | task 35486 | n_decoded = 755, tg = 50.17 t/s, tg_3s = 49.81 t/s
12.32.107.347 I slot print_timing: id 3 | task 35486 | n_decoded = 905, tg = 50.15 t/s, tg_3s = 50.00 t/s
12.35.117.373 I slot print_timing: id 3 | task 35486 | n_decoded = 1055, tg = 50.10 t/s, tg_3s = 49.83 t/s
12.38.126.354 I slot print_timing: id 3 | task 35486 | n_decoded = 1205, tg = 50.07 t/s, tg_3s = 49.85 t/s
12.39.546.379 I slot print_timing: id 3 | task 35486 | prompt eval time = 141.43 ms / 26 tokens ( 5.44 ms per token, 183.84 tokens per second)
12.39.546.382 I slot print_timing: id 3 | task 35486 | eval time = 25486.56 ms / 1276 tokens ( 19.97 ms per token, 50.07 tokens per second)
12.39.546.382 I slot print_timing: id 3 | task 35486 | total time = 25627.99 ms / 1302 tokens
12.39.546.383 I slot print_timing: id 3 | task 35486 | graphs reused = 36545
12.39.546.422 I slot release: id 3 | task 35486 | stop processing: n_tokens = 1372, truncated = 0
12.39.546.433 I srv update_slots: all slots are idle
12.39.562.692 I srv operator(): Chat format: peg-native
12.39.563.005 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.627 (> 0.100 thold), f_keep = 0.054
12.39.563.008 I srv get_availabl: updating prompt cache
12.39.563.082 W srv prompt_save: - saving prompt with length 1372, total state size = 89.637 MiB (draft: 0.000 MiB)
12.39.659.571 I srv load: - looking for better prompt, base f_keep = 0.054, sim = 0.627
12.39.659.583 I srv update: - cache state: 19 prompts, 3145.596 MiB (limits: 8192.000 MiB, 16384 tokens, 101064 est)
12.39.659.584 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
12.39.659.584 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
12.39.659.585 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
12.39.659.586 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
12.39.659.586 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
12.39.659.587 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
12.39.659.587 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
12.39.659.588 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
12.39.659.588 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
12.39.659.589 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
12.39.659.589 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
12.39.659.590 I srv update: - prompt 0x64b5df404d90: 1857 tokens, checkpoints: 1, 161.932 MiB
12.39.659.590 I srv update: - prompt 0x64b5df47e7d0: 1910 tokens, checkpoints: 1, 162.968 MiB
12.39.659.591 I srv update: - prompt 0x64b5d59d2980: 2322 tokens, checkpoints: 1, 171.023 MiB
12.39.659.591 I srv update: - prompt 0x64b5df7e5630: 2856 tokens, checkpoints: 1, 181.462 MiB
12.39.659.592 I srv update: - prompt 0x64b5df42df00: 2779 tokens, checkpoints: 1, 179.957 MiB
12.39.659.592 I srv update: - prompt 0x64b5df77c3c0: 3204 tokens, checkpoints: 1, 188.266 MiB
12.39.659.593 I srv update: - prompt 0x64b5df923f90: 1297 tokens, checkpoints: 1, 150.983 MiB
12.39.659.593 I srv update: - prompt 0x64b5e6bf50f0: 1372 tokens, checkpoints: 1, 152.450 MiB
12.39.659.594 I srv get_availabl: prompt cache update took 96.58 ms
12.39.659.671 I slot launch_slot_: id 3 | task 36764 | processing task, is_child = 0
12.39.659.672 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
12.39.659.673 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
12.39.659.684 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
12.39.659.684 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
12.39.659.693 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
12.39.659.694 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
12.39.659.709 I slot update_slots: id 3 | task 36764 | Checking checkpoint with [70, 70] against 74...
12.39.675.537 W slot update_slots: id 3 | task 36764 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
12.42.851.468 I slot print_timing: id 3 | task 36764 | n_decoded = 150, tg = 49.91 t/s, tg_3s = 49.91 t/s
12.45.857.415 I slot print_timing: id 3 | task 36764 | n_decoded = 300, tg = 49.91 t/s, tg_3s = 49.90 t/s
12.48.870.542 I slot print_timing: id 3 | task 36764 | n_decoded = 450, tg = 49.87 t/s, tg_3s = 49.78 t/s
12.51.870.563 I slot print_timing: id 3 | task 36764 | n_decoded = 600, tg = 49.90 t/s, tg_3s = 50.00 t/s
12.54.883.677 I slot print_timing: id 3 | task 36764 | n_decoded = 751, tg = 49.94 t/s, tg_3s = 50.11 t/s
12.56.239.412 I slot print_timing: id 3 | task 36764 | prompt eval time = 186.54 ms / 47 tokens ( 3.97 ms per token, 251.95 tokens per second)
12.56.239.416 I slot print_timing: id 3 | task 36764 | eval time = 16393.15 ms / 819 tokens ( 20.02 ms per token, 49.96 tokens per second)
12.56.239.417 I slot print_timing: id 3 | task 36764 | total time = 16579.70 ms / 866 tokens
12.56.239.417 I slot print_timing: id 3 | task 36764 | graphs reused = 37359
12.56.239.457 I slot release: id 3 | task 36764 | stop processing: n_tokens = 936, truncated = 0
12.56.239.470 I srv update_slots: all slots are idle
12.56.255.146 I srv operator(): Chat format: peg-native
12.56.255.439 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.705 (> 0.100 thold), f_keep = 0.079
12.56.255.442 I srv get_availabl: updating prompt cache
12.56.255.492 W srv prompt_save: - saving prompt with length 936, total state size = 81.113 MiB (draft: 0.000 MiB)
12.56.344.504 I srv load: - looking for better prompt, base f_keep = 0.079, sim = 0.705
12.56.344.520 I srv update: - cache state: 20 prompts, 3289.522 MiB (limits: 8192.000 MiB, 16384 tokens, 98973 est)
12.56.344.521 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
12.56.344.521 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
12.56.344.522 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
12.56.344.523 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
12.56.344.523 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
12.56.344.524 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
12.56.344.524 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
12.56.344.525 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
12.56.344.525 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
12.56.344.526 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
12.56.344.526 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
12.56.344.527 I srv update: - prompt 0x64b5df404d90: 1857 tokens, checkpoints: 1, 161.932 MiB
12.56.344.527 I srv update: - prompt 0x64b5df47e7d0: 1910 tokens, checkpoints: 1, 162.968 MiB
12.56.344.528 I srv update: - prompt 0x64b5d59d2980: 2322 tokens, checkpoints: 1, 171.023 MiB
12.56.344.534 I srv update: - prompt 0x64b5df7e5630: 2856 tokens, checkpoints: 1, 181.462 MiB
12.56.344.534 I srv update: - prompt 0x64b5df42df00: 2779 tokens, checkpoints: 1, 179.957 MiB
12.56.344.535 I srv update: - prompt 0x64b5df77c3c0: 3204 tokens, checkpoints: 1, 188.266 MiB
12.56.344.535 I srv update: - prompt 0x64b5df923f90: 1297 tokens, checkpoints: 1, 150.983 MiB
12.56.344.536 I srv update: - prompt 0x64b5e6bf50f0: 1372 tokens, checkpoints: 1, 152.450 MiB
12.56.344.536 I srv update: - prompt 0x64b5df47de60: 936 tokens, checkpoints: 1, 143.926 MiB
12.56.344.538 I srv get_availabl: prompt cache update took 89.10 ms
12.56.344.615 I slot launch_slot_: id 3 | task 37585 | processing task, is_child = 0
12.56.344.616 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
12.56.344.616 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
12.56.344.630 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
12.56.344.630 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
12.56.344.639 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
12.56.344.639 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
12.56.344.659 I slot update_slots: id 3 | task 37585 | Checking checkpoint with [70, 70] against 74...
12.56.360.548 W slot update_slots: id 3 | task 37585 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
12.59.519.420 I slot print_timing: id 3 | task 37585 | n_decoded = 152, tg = 50.59 t/s, tg_3s = 50.58 t/s
13.02.532.387 I slot print_timing: id 3 | task 37585 | n_decoded = 303, tg = 50.35 t/s, tg_3s = 50.12 t/s
13.05.547.715 I slot print_timing: id 3 | task 37585 | n_decoded = 454, tg = 50.26 t/s, tg_3s = 50.08 t/s
13.08.562.600 I slot print_timing: id 3 | task 37585 | n_decoded = 605, tg = 50.22 t/s, tg_3s = 50.08 t/s
13.11.566.516 I slot print_timing: id 3 | task 37585 | n_decoded = 755, tg = 50.16 t/s, tg_3s = 49.93 t/s
13.14.579.516 I slot print_timing: id 3 | task 37585 | n_decoded = 906, tg = 50.15 t/s, tg_3s = 50.12 t/s
13.17.592.446 I slot print_timing: id 3 | task 37585 | n_decoded = 1056, tg = 50.10 t/s, tg_3s = 49.79 t/s
13.20.603.632 I slot print_timing: id 3 | task 37585 | n_decoded = 1206, tg = 50.06 t/s, tg_3s = 49.81 t/s
13.20.843.777 I slot print_timing: id 3 | task 37585 | prompt eval time = 169.91 ms / 34 tokens ( 5.00 ms per token, 200.10 tokens per second)
13.20.843.780 I slot print_timing: id 3 | task 37585 | eval time = 24329.20 ms / 1218 tokens ( 19.97 ms per token, 50.06 tokens per second)
13.20.843.781 I slot print_timing: id 3 | task 37585 | total time = 24499.12 ms / 1252 tokens
13.20.843.781 I slot print_timing: id 3 | task 37585 | graphs reused = 38570
13.20.843.823 I slot release: id 3 | task 37585 | stop processing: n_tokens = 1322, truncated = 0
13.20.843.836 I srv update_slots: all slots are idle
13.20.860.004 I srv operator(): Chat format: peg-native
13.20.860.301 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.661 (> 0.100 thold), f_keep = 0.056
13.20.860.304 I srv get_availabl: updating prompt cache
13.20.860.369 W srv prompt_save: - saving prompt with length 1322, total state size = 88.659 MiB (draft: 0.000 MiB)
13.20.955.048 I srv load: - looking for better prompt, base f_keep = 0.056, sim = 0.661
13.20.955.062 I srv update: - cache state: 21 prompts, 3440.994 MiB (limits: 8192.000 MiB, 16384 tokens, 97763 est)
13.20.955.063 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
13.20.955.063 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
13.20.955.064 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
13.20.955.065 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
13.20.955.065 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
13.20.955.066 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
13.20.955.066 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
13.20.955.067 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
13.20.955.067 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
13.20.955.068 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
13.20.955.068 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
13.20.955.069 I srv update: - prompt 0x64b5df404d90: 1857 tokens, checkpoints: 1, 161.932 MiB
13.20.955.069 I srv update: - prompt 0x64b5df47e7d0: 1910 tokens, checkpoints: 1, 162.968 MiB
13.20.955.070 I srv update: - prompt 0x64b5d59d2980: 2322 tokens, checkpoints: 1, 171.023 MiB
13.20.955.070 I srv update: - prompt 0x64b5df7e5630: 2856 tokens, checkpoints: 1, 181.462 MiB
13.20.955.071 I srv update: - prompt 0x64b5df42df00: 2779 tokens, checkpoints: 1, 179.957 MiB
13.20.955.071 I srv update: - prompt 0x64b5df77c3c0: 3204 tokens, checkpoints: 1, 188.266 MiB
13.20.955.072 I srv update: - prompt 0x64b5df923f90: 1297 tokens, checkpoints: 1, 150.983 MiB
13.20.955.073 I srv update: - prompt 0x64b5e6bf50f0: 1372 tokens, checkpoints: 1, 152.450 MiB
13.20.955.074 I srv update: - prompt 0x64b5df47de60: 936 tokens, checkpoints: 1, 143.926 MiB
13.20.955.074 I srv update: - prompt 0x64b5df784ca0: 1322 tokens, checkpoints: 1, 151.472 MiB
13.20.955.076 I srv get_availabl: prompt cache update took 94.77 ms
13.20.955.145 I slot launch_slot_: id 3 | task 38805 | processing task, is_child = 0
13.20.955.145 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
13.20.955.146 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
13.20.955.158 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
13.20.955.158 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
13.20.955.167 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
13.20.955.168 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
13.20.955.184 I slot update_slots: id 3 | task 38805 | Checking checkpoint with [70, 70] against 74...
13.20.971.010 W slot update_slots: id 3 | task 38805 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
13.24.151.828 I slot print_timing: id 3 | task 38805 | n_decoded = 151, tg = 50.25 t/s, tg_3s = 50.25 t/s
13.27.164.251 I slot print_timing: id 3 | task 38805 | n_decoded = 301, tg = 50.02 t/s, tg_3s = 49.79 t/s
13.30.183.006 I slot print_timing: id 3 | task 38805 | n_decoded = 452, tg = 50.02 t/s, tg_3s = 50.02 t/s
13.33.198.864 I slot print_timing: id 3 | task 38805 | n_decoded = 603, tg = 50.03 t/s, tg_3s = 50.07 t/s
13.36.204.272 I slot print_timing: id 3 | task 38805 | n_decoded = 753, tg = 50.01 t/s, tg_3s = 49.91 t/s
13.39.220.000 I slot print_timing: id 3 | task 38805 | n_decoded = 904, tg = 50.02 t/s, tg_3s = 50.07 t/s
13.42.234.119 I slot print_timing: id 3 | task 38805 | n_decoded = 1056, tg = 50.08 t/s, tg_3s = 50.43 t/s
13.45.249.456 I slot print_timing: id 3 | task 38805 | n_decoded = 1208, tg = 50.12 t/s, tg_3s = 50.41 t/s
13.48.261.200 I slot print_timing: id 3 | task 38805 | n_decoded = 1360, tg = 50.16 t/s, tg_3s = 50.47 t/s
13.51.280.101 I slot print_timing: id 3 | task 38805 | n_decoded = 1512, tg = 50.18 t/s, tg_3s = 50.35 t/s
13.54.294.935 I slot print_timing: id 3 | task 38805 | n_decoded = 1664, tg = 50.20 t/s, tg_3s = 50.42 t/s
13.56.354.033 I slot print_timing: id 3 | task 38805 | prompt eval time = 191.55 ms / 41 tokens ( 4.67 ms per token, 214.05 tokens per second)
13.56.354.037 I slot print_timing: id 3 | task 38805 | eval time = 35207.30 ms / 1767 tokens ( 19.92 ms per token, 50.19 tokens per second)
13.56.354.038 I slot print_timing: id 3 | task 38805 | total time = 35398.84 ms / 1808 tokens
13.56.354.039 I slot print_timing: id 3 | task 38805 | graphs reused = 40328
13.56.354.078 I slot release: id 3 | task 38805 | stop processing: n_tokens = 1878, truncated = 0
13.56.354.092 I srv update_slots: all slots are idle
13.56.369.909 I srv operator(): Chat format: peg-native
13.56.370.201 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.617 (> 0.100 thold), f_keep = 0.039
13.56.370.203 I srv get_availabl: updating prompt cache
13.56.370.281 W srv prompt_save: - saving prompt with length 1878, total state size = 99.529 MiB (draft: 0.000 MiB)
13.56.471.390 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.617
13.56.471.404 I srv update: - cache state: 22 prompts, 3603.336 MiB (limits: 8192.000 MiB, 16384 tokens, 97628 est)
13.56.471.405 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
13.56.471.406 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
13.56.471.406 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
13.56.471.407 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
13.56.471.407 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
13.56.471.408 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
13.56.471.409 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
13.56.471.409 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
13.56.471.410 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
13.56.471.410 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
13.56.471.411 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
13.56.471.411 I srv update: - prompt 0x64b5df404d90: 1857 tokens, checkpoints: 1, 161.932 MiB
13.56.471.412 I srv update: - prompt 0x64b5df47e7d0: 1910 tokens, checkpoints: 1, 162.968 MiB
13.56.471.412 I srv update: - prompt 0x64b5d59d2980: 2322 tokens, checkpoints: 1, 171.023 MiB
13.56.471.413 I srv update: - prompt 0x64b5df7e5630: 2856 tokens, checkpoints: 1, 181.462 MiB
13.56.471.413 I srv update: - prompt 0x64b5df42df00: 2779 tokens, checkpoints: 1, 179.957 MiB
13.56.471.414 I srv update: - prompt 0x64b5df77c3c0: 3204 tokens, checkpoints: 1, 188.266 MiB
13.56.471.414 I srv update: - prompt 0x64b5df923f90: 1297 tokens, checkpoints: 1, 150.983 MiB
13.56.471.415 I srv update: - prompt 0x64b5e6bf50f0: 1372 tokens, checkpoints: 1, 152.450 MiB
13.56.471.416 I srv update: - prompt 0x64b5df47de60: 936 tokens, checkpoints: 1, 143.926 MiB
13.56.471.417 I srv update: - prompt 0x64b5df784ca0: 1322 tokens, checkpoints: 1, 151.472 MiB
13.56.471.418 I srv update: - prompt 0x64b5df446570: 1878 tokens, checkpoints: 1, 162.342 MiB
13.56.471.419 I srv get_availabl: prompt cache update took 101.21 ms
13.56.471.487 I slot launch_slot_: id 3 | task 40574 | processing task, is_child = 0
13.56.471.488 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
13.56.471.489 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
13.56.471.501 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
13.56.471.501 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
13.56.471.511 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
13.56.471.511 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
13.56.471.528 I slot update_slots: id 3 | task 40574 | Checking checkpoint with [70, 70] against 74...
13.56.487.295 W slot update_slots: id 3 | task 40574 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
13.59.660.115 I slot print_timing: id 3 | task 40574 | n_decoded = 152, tg = 50.66 t/s, tg_3s = 50.66 t/s
14.02.665.125 I slot print_timing: id 3 | task 40574 | n_decoded = 304, tg = 50.62 t/s, tg_3s = 50.58 t/s
14.05.683.690 I slot print_timing: id 3 | task 40574 | n_decoded = 456, tg = 50.53 t/s, tg_3s = 50.36 t/s
14.08.688.235 I slot print_timing: id 3 | task 40574 | n_decoded = 607, tg = 50.46 t/s, tg_3s = 50.26 t/s
14.11.690.270 I slot print_timing: id 3 | task 40574 | n_decoded = 757, tg = 50.36 t/s, tg_3s = 49.97 t/s
14.14.693.348 I slot print_timing: id 3 | task 40574 | n_decoded = 907, tg = 50.30 t/s, tg_3s = 49.95 t/s
14.15.811.384 I slot print_timing: id 3 | task 40574 | prompt eval time = 188.37 ms / 49 tokens ( 3.84 ms per token, 260.13 tokens per second)
14.15.811.388 I slot print_timing: id 3 | task 40574 | eval time = 19151.49 ms / 963 tokens ( 19.89 ms per token, 50.28 tokens per second)
14.15.811.389 I slot print_timing: id 3 | task 40574 | total time = 19339.85 ms / 1012 tokens
14.15.811.390 I slot print_timing: id 3 | task 40574 | graphs reused = 41285
14.15.811.429 I slot release: id 3 | task 40574 | stop processing: n_tokens = 1082, truncated = 0
14.15.811.441 I srv update_slots: all slots are idle
14.15.826.957 I srv operator(): Chat format: peg-native
14.15.827.262 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.712 (> 0.100 thold), f_keep = 0.068
14.15.827.264 I srv get_availabl: updating prompt cache
14.15.827.312 W srv prompt_save: - saving prompt with length 1082, total state size = 83.967 MiB (draft: 0.000 MiB)
14.15.922.412 I srv load: - looking for better prompt, base f_keep = 0.068, sim = 0.712
14.15.922.427 I srv update: - cache state: 23 prompts, 3750.116 MiB (limits: 8192.000 MiB, 16384 tokens, 96171 est)
14.15.922.428 I srv update: - prompt 0x64b5e6bf6d20: 2980 tokens, checkpoints: 1, 183.887 MiB
14.15.922.428 I srv update: - prompt 0x64b5e6cfe340: 1896 tokens, checkpoints: 1, 162.694 MiB
14.15.922.429 I srv update: - prompt 0x64b5df5a5890: 1042 tokens, checkpoints: 1, 145.998 MiB
14.15.922.429 I srv update: - prompt 0x64b5df04af20: 2280 tokens, checkpoints: 1, 170.201 MiB
14.15.922.430 I srv update: - prompt 0x64b5df410220: 2200 tokens, checkpoints: 1, 168.637 MiB
14.15.922.430 I srv update: - prompt 0x64b5df7f0460: 1852 tokens, checkpoints: 1, 161.834 MiB
14.15.922.431 I srv update: - prompt 0x64b5e669e6b0: 1165 tokens, checkpoints: 1, 148.403 MiB
14.15.922.431 I srv update: - prompt 0x64b5e6dd5630: 1697 tokens, checkpoints: 1, 158.804 MiB
14.15.922.432 I srv update: - prompt 0x64b5e6c41eb0: 1922 tokens, checkpoints: 1, 163.202 MiB
14.15.922.432 I srv update: - prompt 0x64b5e6c023c0: 1912 tokens, checkpoints: 1, 163.007 MiB
14.15.922.433 I srv update: - prompt 0x64b5e6dc2a10: 2264 tokens, checkpoints: 1, 169.889 MiB
14.15.922.434 I srv update: - prompt 0x64b5df404d90: 1857 tokens, checkpoints: 1, 161.932 MiB
14.15.922.434 I srv update: - prompt 0x64b5df47e7d0: 1910 tokens, checkpoints: 1, 162.968 MiB
14.15.922.435 I srv update: - prompt 0x64b5d59d2980: 2322 tokens, checkpoints: 1, 171.023 MiB
14.15.922.435 I srv update: - prompt 0x64b5df7e5630: 2856 tokens, checkpoints: 1, 181.462 MiB
14.15.922.436 I srv update: - prompt 0x64b5df42df00: 2779 tokens, checkpoints: 1, 179.957 MiB
14.15.922.436 I srv update: - prompt 0x64b5df77c3c0: 3204 tokens, checkpoints: 1, 188.266 MiB
14.15.922.437 I srv update: - prompt 0x64b5df923f90: 1297 tokens, checkpoints: 1, 150.983 MiB
14.15.922.437 I srv update: - prompt 0x64b5e6bf50f0: 1372 tokens, checkpoints: 1, 152.450 MiB
14.15.922.438 I srv update: - prompt 0x64b5df47de60: 936 tokens, checkpoints: 1, 143.926 MiB
14.15.922.438 I srv update: - prompt 0x64b5df784ca0: 1322 tokens, checkpoints: 1, 151.472 MiB
14.15.922.439 I srv update: - prompt 0x64b5df446570: 1878 tokens, checkpoints: 1, 162.342 MiB
14.15.922.439 I srv update: - prompt 0x64b5e6d2ded0: 1082 tokens, checkpoints: 1, 146.780 MiB
14.15.922.440 I srv get_availabl: prompt cache update took 95.17 ms
14.15.922.516 I slot launch_slot_: id 3 | task 41539 | processing task, is_child = 0
14.15.922.517 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
14.15.922.517 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
14.15.922.530 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
14.15.922.530 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
14.15.922.551 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
14.15.922.551 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
14.15.922.570 I slot update_slots: id 3 | task 41539 | Checking checkpoint with [70, 70] against 74...
14.15.938.537 W slot update_slots: id 3 | task 41539 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 62.813 MiB)
14.19.105.516 I slot print_timing: id 3 | task 41539 | n_decoded = 152, tg = 50.41 t/s, tg_3s = 50.41 t/s
14.22.106.423 I slot print_timing: id 3 | task 41539 | n_decoded = 302, tg = 50.20 t/s, tg_3s = 49.98 t/s
14.25.110.492 I slot print_timing: id 3 | task 41539 | n_decoded = 452, tg = 50.11 t/s, tg_3s = 49.93 t/s
14.28.128.262 I slot print_timing: id 3 | task 41539 | n_decoded = 603, tg = 50.09 t/s, tg_3s = 50.04 t/s
14.31.142.074 I slot print_timing: id 3 | task 41539 | n_decoded = 753, tg = 50.03 t/s, tg_3s = 49.77 t/s
14.34.145.443 I slot print_timing: id 3 | task 41539 | n_decoded = 904, tg = 50.07 t/s, tg_3s = 50.28 t/s
14.37.149.466 I slot print_timing: id 3 | task 41539 | n_decoded = 1054, tg = 50.05 t/s, tg_3s = 49.93 t/s
14.39.598.528 I slot print_timing: id 3 | task 41539 | prompt eval time = 167.58 ms / 33 tokens ( 5.08 ms per token, 196.92 tokens per second)
14.39.598.531 I slot print_timing: id 3 | task 41539 | eval time = 23508.37 ms / 1177 tokens ( 19.97 ms per token, 50.07 tokens per second)
14.39.598.531 I slot print_timing: id 3 | task 41539 | total time = 23675.95 ms / 1210 tokens
14.39.598.532 I slot print_timing: id 3 | task 41539 | graphs reused = 42456
14.39.598.571 I slot release: id 3 | task 41539 | stop processing: n_tokens = 1280, truncated = 0
14.39.598.583 I srv update_slots: all slots are idle
14.39.599.218 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink