experiments-07-07-12-local-.../results_hard/nemotron3-super-120b.server.log

0.00.067.543 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.067.546 I device_info:
0.00.067.566 I   - ROCm0   : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.067.570 I   - ROCm1   : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.067.577 I   - CPU     : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.067.639 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
0.00.067.643 I srv  llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.067.688 I srv          init: running without SSL
0.00.067.761 I srv          init: using 23 threads for HTTP server
0.00.068.109 I srv         start: binding port with default address family
0.00.069.319 I srv  llama_server: loading model
0.00.069.327 I srv    load_model: loading model '/home/qrf/.cache/llama.cpp/nvidia_Nemotron-3-Super-120B-A12B-Q4_K_M/nvidia_Nemotron-3-Super-120B-A12B-Q4_K_M-00001-of-00003.gguf'
0.00.069.395 I common_init_result: fitting params to device memory ...
0.00.069.397 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.07.715.403 W llama_model_loader: tensor overrides to CPU are used with mmap enabled - consider using --no-mmap for better performance
0.55.307.093 W llama_context: n_ctx_seq (16384) < n_ctx_train (1048576) -- the full capacity of the model will not be utilized
0.55.523.165 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.55.824.859 I srv    load_model: initializing slots, n_slots = 4
0.55.970.515 W srv    load_model: speculative decoding will use checkpoints
0.55.970.536 W common_speculative_init: no implementations specified for speculative decoding
0.55.970.537 I slot   load_model: id  0 | task -1 | new slot, n_ctx = 16384
0.55.970.543 I slot   load_model: id  1 | task -1 | new slot, n_ctx = 16384
0.55.970.543 I slot   load_model: id  2 | task -1 | new slot, n_ctx = 16384
0.55.970.543 I slot   load_model: id  3 | task -1 | new slot, n_ctx = 16384
0.55.970.648 I srv    load_model: prompt cache is enabled, size limit: 8192 MiB
0.55.970.649 I srv    load_model: use `--cache-ram 0` to disable the prompt cache
0.55.970.650 I srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.55.970.650 I srv    load_model: context checkpoints enabled, max = 32, min spacing = 256
0.55.970.691 I srv          init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.56.009.177 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
<think></think>Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
<think>
'
0.56.038.626 I srv          init: init: chat template, thinking = 1
0.56.038.662 I srv  llama_server: model loaded
0.56.038.667 I srv  llama_server: server is listening on http://127.0.0.1:8090
0.56.038.677 I srv  update_slots: all slots are idle
1.00.334.465 I srv    operator(): Chat format: peg-native
1.00.334.771 I slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
1.00.334.773 I srv  get_availabl: updating prompt cache
1.00.334.781 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
1.00.334.786 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est)
1.00.334.786 I srv  get_availabl: prompt cache update took 0.01 ms
1.00.334.897 I slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
1.00.334.899 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.00.334.900 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
1.00.334.916 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
1.00.334.917 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
1.00.334.927 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
1.00.334.927 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
1.07.042.551 I slot print_timing: id  3 | task 0 | prompt processing, n_tokens =     71, progress = 0.71, t =   6.71 s / 10.59 tokens per second
1.07.186.967 I slot create_check: id  3 | task 0 | created context checkpoint 1 of 32 (pos_min = 70, pos_max = 70, n_tokens = 71, size = 164.688 MiB)
1.07.830.379 I slot print_timing: id  3 | task 0 | prompt processing, n_tokens =     96, progress = 0.96, t =   7.50 s / 12.81 tokens per second
1.14.657.042 I slot print_timing: id  3 | task 0 | n_decoded =    100, tg =  15.01 t/s, tg_3s =  15.01 t/s
1.17.702.570 I slot print_timing: id  3 | task 0 | n_decoded =    146, tg =  15.04 t/s, tg_3s =  15.10 t/s
1.20.707.592 I slot print_timing: id  3 | task 0 | n_decoded =    191, tg =  15.03 t/s, tg_3s =  14.97 t/s
1.23.729.429 I slot print_timing: id  3 | task 0 | n_decoded =    237, tg =  15.06 t/s, tg_3s =  15.22 t/s
1.26.746.220 I slot print_timing: id  3 | task 0 | n_decoded =    283, tg =  15.09 t/s, tg_3s =  15.25 t/s
1.29.780.128 I slot print_timing: id  3 | task 0 | n_decoded =    329, tg =  15.10 t/s, tg_3s =  15.16 t/s
1.32.843.054 I slot print_timing: id  3 | task 0 | n_decoded =    376, tg =  15.13 t/s, tg_3s =  15.34 t/s
1.35.860.025 I slot print_timing: id  3 | task 0 | n_decoded =    422, tg =  15.15 t/s, tg_3s =  15.25 t/s
1.38.906.890 I slot print_timing: id  3 | task 0 | n_decoded =    468, tg =  15.14 t/s, tg_3s =  15.10 t/s
1.41.951.888 I slot print_timing: id  3 | task 0 | n_decoded =    514, tg =  15.14 t/s, tg_3s =  15.11 t/s
1.44.992.687 I slot print_timing: id  3 | task 0 | n_decoded =    560, tg =  15.14 t/s, tg_3s =  15.13 t/s
1.48.012.707 I slot print_timing: id  3 | task 0 | n_decoded =    606, tg =  15.14 t/s, tg_3s =  15.23 t/s
1.51.035.697 I slot print_timing: id  3 | task 0 | n_decoded =    652, tg =  15.15 t/s, tg_3s =  15.22 t/s
1.54.042.926 I slot print_timing: id  3 | task 0 | n_decoded =    697, tg =  15.14 t/s, tg_3s =  14.96 t/s
1.57.093.971 I slot print_timing: id  3 | task 0 | n_decoded =    743, tg =  15.13 t/s, tg_3s =  15.08 t/s
2.00.100.965 I slot print_timing: id  3 | task 0 | n_decoded =    788, tg =  15.12 t/s, tg_3s =  14.97 t/s
2.03.143.158 I slot print_timing: id  3 | task 0 | n_decoded =    834, tg =  15.12 t/s, tg_3s =  15.12 t/s
2.06.192.180 I slot print_timing: id  3 | task 0 | n_decoded =    880, tg =  15.12 t/s, tg_3s =  15.09 t/s
2.09.245.106 I slot print_timing: id  3 | task 0 | n_decoded =    926, tg =  15.12 t/s, tg_3s =  15.07 t/s
2.12.306.863 I slot print_timing: id  3 | task 0 | n_decoded =    972, tg =  15.11 t/s, tg_3s =  15.02 t/s
2.15.311.747 I slot print_timing: id  3 | task 0 | n_decoded =   1017, tg =  15.11 t/s, tg_3s =  14.98 t/s
2.18.328.872 I slot print_timing: id  3 | task 0 | n_decoded =   1062, tg =  15.10 t/s, tg_3s =  14.91 t/s
2.21.380.057 I slot print_timing: id  3 | task 0 | n_decoded =   1107, tg =  15.09 t/s, tg_3s =  14.75 t/s
2.24.387.272 I slot print_timing: id  3 | task 0 | n_decoded =   1152, tg =  15.08 t/s, tg_3s =  14.96 t/s
2.27.406.272 I slot print_timing: id  3 | task 0 | n_decoded =   1197, tg =  15.07 t/s, tg_3s =  14.91 t/s
2.30.451.263 I slot print_timing: id  3 | task 0 | n_decoded =   1243, tg =  15.07 t/s, tg_3s =  15.11 t/s
2.33.513.963 I slot print_timing: id  3 | task 0 | n_decoded =   1289, tg =  15.07 t/s, tg_3s =  15.02 t/s
2.36.555.268 I slot print_timing: id  3 | task 0 | n_decoded =   1334, tg =  15.06 t/s, tg_3s =  14.80 t/s
2.37.907.990 I slot print_timing: id  3 | task 0 | prompt eval time =    7661.60 ms /   100 tokens (   76.62 ms per token,    13.05 tokens per second)
2.37.907.994 I slot print_timing: id  3 | task 0 |        eval time =   89911.42 ms /  1354 tokens (   66.40 ms per token,    15.06 tokens per second)
2.37.907.994 I slot print_timing: id  3 | task 0 |       total time =   97573.02 ms /  1454 tokens
2.37.907.999 I slot print_timing: id  3 | task 0 |    graphs reused =       1348
2.37.908.053 I slot      release: id  3 | task 0 | stop processing: n_tokens = 1453, truncated = 0
2.37.908.068 I srv  update_slots: all slots are idle
2.37.939.990 I srv    operator(): Chat format: peg-native
2.37.940.242 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.779 (> 0.100 thold), f_keep = 0.051
2.37.940.244 I srv  get_availabl: updating prompt cache
2.37.940.337 W srv   prompt_save:  - saving prompt with length 1453, total state size = 176.057 MiB (draft: 0.000 MiB)
2.38.198.796 I srv          load:  - looking for better prompt, base f_keep = 0.051, sim = 0.779
2.38.198.802 I srv        update:  - cache state: 1 prompts, 340.745 MiB (limits: 8192.000 MiB, 16384 tokens, 34932 est)
2.38.198.803 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
2.38.198.805 I srv  get_availabl: prompt cache update took 258.56 ms
2.38.198.926 I slot launch_slot_: id  3 | task 1357 | processing task, is_child = 0
2.38.198.928 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
2.38.198.929 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
2.38.198.945 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
2.38.198.946 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
2.38.198.968 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
2.38.198.968 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
2.38.198.989 I slot update_slots: id  3 | task 1357 | Checking checkpoint with [70, 70] against 74...
2.38.230.005 W slot update_slots: id  3 | task 1357 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
2.45.563.242 I slot print_timing: id  3 | task 1357 | n_decoded =    100, tg =  15.07 t/s, tg_3s =  15.07 t/s
2.48.622.256 I slot print_timing: id  3 | task 1357 | n_decoded =    146, tg =  15.06 t/s, tg_3s =  15.04 t/s
2.51.688.409 I slot print_timing: id  3 | task 1357 | n_decoded =    192, tg =  15.05 t/s, tg_3s =  15.00 t/s
2.54.750.168 I slot print_timing: id  3 | task 1357 | n_decoded =    238, tg =  15.04 t/s, tg_3s =  15.02 t/s
2.57.799.997 I slot print_timing: id  3 | task 1357 | n_decoded =    284, tg =  15.05 t/s, tg_3s =  15.08 t/s
3.00.860.457 I slot print_timing: id  3 | task 1357 | n_decoded =    330, tg =  15.05 t/s, tg_3s =  15.03 t/s
3.03.891.092 I slot print_timing: id  3 | task 1357 | n_decoded =    376, tg =  15.06 t/s, tg_3s =  15.18 t/s
3.06.911.080 I slot print_timing: id  3 | task 1357 | n_decoded =    421, tg =  15.04 t/s, tg_3s =  14.90 t/s
3.09.930.821 I slot print_timing: id  3 | task 1357 | n_decoded =    467, tg =  15.06 t/s, tg_3s =  15.23 t/s
3.10.510.472 I slot print_timing: id  3 | task 1357 | prompt eval time =     729.26 ms /    24 tokens (   30.39 ms per token,    32.91 tokens per second)
3.10.510.477 I slot print_timing: id  3 | task 1357 |        eval time =   31582.21 ms /   476 tokens (   66.35 ms per token,    15.07 tokens per second)
3.10.510.478 I slot print_timing: id  3 | task 1357 |       total time =   32311.47 ms /   500 tokens
3.10.510.478 I slot print_timing: id  3 | task 1357 |    graphs reused =       1820
3.10.510.510 I slot      release: id  3 | task 1357 | stop processing: n_tokens = 570, truncated = 0
3.10.510.528 I srv  update_slots: all slots are idle
3.10.541.535 I srv    operator(): Chat format: peg-native
3.10.541.769 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.779 (> 0.100 thold), f_keep = 0.130
3.10.541.771 I srv  get_availabl: updating prompt cache
3.10.541.820 W srv   prompt_save:  - saving prompt with length 570, total state size = 169.148 MiB (draft: 0.000 MiB)
3.10.732.047 I srv          load:  - looking for better prompt, base f_keep = 0.130, sim = 0.779
3.10.732.054 I srv        update:  - cache state: 2 prompts, 674.582 MiB (limits: 8192.000 MiB, 16384 tokens, 24566 est)
3.10.732.055 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
3.10.732.055 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
3.10.732.057 I srv  get_availabl: prompt cache update took 190.28 ms
3.10.732.162 I slot launch_slot_: id  3 | task 1835 | processing task, is_child = 0
3.10.732.165 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
3.10.732.166 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
3.10.732.183 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
3.10.732.183 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
3.10.732.207 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
3.10.732.207 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
3.10.732.226 I slot update_slots: id  3 | task 1835 | Checking checkpoint with [70, 70] against 74...
3.10.763.225 W slot update_slots: id  3 | task 1835 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
3.17.954.748 I slot print_timing: id  3 | task 1835 | n_decoded =    100, tg =  15.37 t/s, tg_3s =  15.37 t/s
3.20.986.618 I slot print_timing: id  3 | task 1835 | n_decoded =    146, tg =  15.30 t/s, tg_3s =  15.17 t/s
3.23.990.178 I slot print_timing: id  3 | task 1835 | n_decoded =    191, tg =  15.23 t/s, tg_3s =  14.98 t/s
3.27.041.824 I slot print_timing: id  3 | task 1835 | n_decoded =    237, tg =  15.20 t/s, tg_3s =  15.07 t/s
3.30.103.822 I slot print_timing: id  3 | task 1835 | n_decoded =    283, tg =  15.17 t/s, tg_3s =  15.02 t/s
3.31.489.881 I slot print_timing: id  3 | task 1835 | prompt eval time =     714.57 ms /    24 tokens (   29.77 ms per token,    33.59 tokens per second)
3.31.489.886 I slot print_timing: id  3 | task 1835 |        eval time =   20043.08 ms /   304 tokens (   65.93 ms per token,    15.17 tokens per second)
3.31.489.886 I slot print_timing: id  3 | task 1835 |       total time =   20757.64 ms /   328 tokens
3.31.489.887 I slot print_timing: id  3 | task 1835 |    graphs reused =       2121
3.31.489.918 I slot      release: id  3 | task 1835 | stop processing: n_tokens = 398, truncated = 0
3.31.489.933 I srv  update_slots: all slots are idle
3.31.521.966 I srv    operator(): Chat format: peg-native
3.31.522.218 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.787 (> 0.100 thold), f_keep = 0.186
3.31.522.220 I srv  get_availabl: updating prompt cache
3.31.522.273 W srv   prompt_save:  - saving prompt with length 398, total state size = 167.803 MiB (draft: 0.000 MiB)
3.31.704.773 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 0.787
3.31.704.782 I srv        update:  - cache state: 3 prompts, 1007.073 MiB (limits: 8192.000 MiB, 16384 tokens, 19693 est)
3.31.704.783 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
3.31.704.785 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
3.31.704.786 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
3.31.704.788 I srv  get_availabl: prompt cache update took 182.57 ms
3.31.704.897 I slot launch_slot_: id  3 | task 2141 | processing task, is_child = 0
3.31.704.899 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
3.31.704.900 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
3.31.704.914 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
3.31.704.915 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
3.31.704.924 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
3.31.704.925 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
3.31.704.943 I slot update_slots: id  3 | task 2141 | Checking checkpoint with [70, 70] against 74...
3.31.735.581 W slot update_slots: id  3 | task 2141 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
3.38.973.041 I slot print_timing: id  3 | task 2141 | n_decoded =    100, tg =  15.22 t/s, tg_3s =  15.22 t/s
3.42.017.057 I slot print_timing: id  3 | task 2141 | n_decoded =    146, tg =  15.19 t/s, tg_3s =  15.11 t/s
3.45.067.071 I slot print_timing: id  3 | task 2141 | n_decoded =    192, tg =  15.16 t/s, tg_3s =  15.08 t/s
3.48.125.705 I slot print_timing: id  3 | task 2141 | n_decoded =    238, tg =  15.14 t/s, tg_3s =  15.04 t/s
3.51.190.750 I slot print_timing: id  3 | task 2141 | n_decoded =    284, tg =  15.12 t/s, tg_3s =  15.01 t/s
3.54.200.626 I slot print_timing: id  3 | task 2141 | n_decoded =    329, tg =  15.09 t/s, tg_3s =  14.95 t/s
3.57.219.007 I slot print_timing: id  3 | task 2141 | n_decoded =    374, tg =  15.07 t/s, tg_3s =  14.91 t/s
4.00.228.730 I slot print_timing: id  3 | task 2141 | n_decoded =    419, tg =  15.06 t/s, tg_3s =  14.95 t/s
4.03.247.960 I slot print_timing: id  3 | task 2141 | n_decoded =    464, tg =  15.04 t/s, tg_3s =  14.90 t/s
4.06.265.765 I slot print_timing: id  3 | task 2141 | n_decoded =    509, tg =  15.03 t/s, tg_3s =  14.91 t/s
4.09.284.201 I slot print_timing: id  3 | task 2141 | n_decoded =    554, tg =  15.02 t/s, tg_3s =  14.91 t/s
4.12.286.784 I slot print_timing: id  3 | task 2141 | n_decoded =    599, tg =  15.02 t/s, tg_3s =  14.99 t/s
4.15.348.719 I slot print_timing: id  3 | task 2141 | n_decoded =    645, tg =  15.02 t/s, tg_3s =  15.02 t/s
4.18.408.888 I slot print_timing: id  3 | task 2141 | n_decoded =    691, tg =  15.02 t/s, tg_3s =  15.03 t/s
4.21.415.779 I slot print_timing: id  3 | task 2141 | n_decoded =    736, tg =  15.02 t/s, tg_3s =  14.97 t/s
4.24.476.870 I slot print_timing: id  3 | task 2141 | n_decoded =    782, tg =  15.02 t/s, tg_3s =  15.03 t/s
4.24.542.983 I slot print_timing: id  3 | task 2141 | prompt eval time =     699.88 ms /    23 tokens (   30.43 ms per token,    32.86 tokens per second)
4.24.542.987 I slot print_timing: id  3 | task 2141 |        eval time =   52138.15 ms /   783 tokens (   66.59 ms per token,    15.02 tokens per second)
4.24.542.988 I slot print_timing: id  3 | task 2141 |       total time =   52838.03 ms /   806 tokens
4.24.542.988 I slot print_timing: id  3 | task 2141 |    graphs reused =       2899
4.24.543.021 I slot      release: id  3 | task 2141 | stop processing: n_tokens = 876, truncated = 0
4.24.543.039 I srv  update_slots: all slots are idle
4.24.573.863 I srv    operator(): Chat format: peg-native
4.24.574.099 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.692 (> 0.100 thold), f_keep = 0.084
4.24.574.101 I srv  get_availabl: updating prompt cache
4.24.574.175 W srv   prompt_save:  - saving prompt with length 876, total state size = 171.542 MiB (draft: 0.000 MiB)
4.24.770.302 I srv          load:  - looking for better prompt, base f_keep = 0.084, sim = 0.692
4.24.770.312 I srv        update:  - cache state: 4 prompts, 1343.304 MiB (limits: 8192.000 MiB, 16384 tokens, 20106 est)
4.24.770.314 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
4.24.770.315 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
4.24.770.316 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
4.24.770.316 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
4.24.770.318 I srv  get_availabl: prompt cache update took 196.22 ms
4.24.770.436 I slot launch_slot_: id  3 | task 2926 | processing task, is_child = 0
4.24.770.438 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
4.24.770.439 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
4.24.770.454 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
4.24.770.455 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
4.24.770.469 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
4.24.770.469 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
4.24.770.493 I slot update_slots: id  3 | task 2926 | Checking checkpoint with [70, 70] against 74...
4.24.801.305 W slot update_slots: id  3 | task 2926 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
4.28.864.697 I slot print_timing: id  3 | task 2926 | prompt processing, n_tokens =     32, progress = 0.96, t =   4.09 s / 7.82 tokens per second
4.35.577.862 I slot print_timing: id  3 | task 2926 | n_decoded =    100, tg =  15.27 t/s, tg_3s =  15.27 t/s
4.38.605.315 I slot print_timing: id  3 | task 2926 | n_decoded =    146, tg =  15.24 t/s, tg_3s =  15.19 t/s
4.41.647.180 I slot print_timing: id  3 | task 2926 | n_decoded =    192, tg =  15.21 t/s, tg_3s =  15.12 t/s
4.44.668.162 I slot print_timing: id  3 | task 2926 | n_decoded =    238, tg =  15.22 t/s, tg_3s =  15.23 t/s
4.47.670.891 I slot print_timing: id  3 | task 2926 | n_decoded =    284, tg =  15.23 t/s, tg_3s =  15.32 t/s
4.50.723.316 I slot print_timing: id  3 | task 2926 | n_decoded =    331, tg =  15.26 t/s, tg_3s =  15.40 t/s
4.53.758.697 I slot print_timing: id  3 | task 2926 | n_decoded =    378, tg =  15.28 t/s, tg_3s =  15.48 t/s
4.56.815.462 I slot print_timing: id  3 | task 2926 | n_decoded =    424, tg =  15.26 t/s, tg_3s =  15.05 t/s
4.59.853.636 I slot print_timing: id  3 | task 2926 | n_decoded =    470, tg =  15.25 t/s, tg_3s =  15.14 t/s
5.02.896.624 I slot print_timing: id  3 | task 2926 | n_decoded =    516, tg =  15.23 t/s, tg_3s =  15.12 t/s
5.05.935.414 I slot print_timing: id  3 | task 2926 | n_decoded =    562, tg =  15.23 t/s, tg_3s =  15.14 t/s
5.08.970.757 I slot print_timing: id  3 | task 2926 | n_decoded =    608, tg =  15.22 t/s, tg_3s =  15.15 t/s
5.12.010.670 I slot print_timing: id  3 | task 2926 | n_decoded =    654, tg =  15.22 t/s, tg_3s =  15.13 t/s
5.15.013.034 I slot print_timing: id  3 | task 2926 | n_decoded =    699, tg =  15.20 t/s, tg_3s =  14.99 t/s
5.18.060.909 I slot print_timing: id  3 | task 2926 | n_decoded =    745, tg =  15.19 t/s, tg_3s =  15.09 t/s
5.21.097.929 I slot print_timing: id  3 | task 2926 | n_decoded =    791, tg =  15.19 t/s, tg_3s =  15.15 t/s
5.24.139.993 I slot print_timing: id  3 | task 2926 | n_decoded =    837, tg =  15.19 t/s, tg_3s =  15.12 t/s
5.27.204.997 I slot print_timing: id  3 | task 2926 | n_decoded =    883, tg =  15.18 t/s, tg_3s =  15.01 t/s
5.30.231.603 I slot print_timing: id  3 | task 2926 | n_decoded =    928, tg =  15.16 t/s, tg_3s =  14.87 t/s
5.33.243.800 I slot print_timing: id  3 | task 2926 | n_decoded =    973, tg =  15.15 t/s, tg_3s =  14.94 t/s
5.36.258.505 I slot print_timing: id  3 | task 2926 | n_decoded =   1018, tg =  15.14 t/s, tg_3s =  14.93 t/s
5.39.279.846 I slot print_timing: id  3 | task 2926 | n_decoded =   1063, tg =  15.13 t/s, tg_3s =  14.89 t/s
5.42.296.563 I slot print_timing: id  3 | task 2926 | n_decoded =   1108, tg =  15.12 t/s, tg_3s =  14.92 t/s
5.45.338.846 I slot print_timing: id  3 | task 2926 | n_decoded =   1153, tg =  15.11 t/s, tg_3s =  14.79 t/s
5.48.405.648 I slot print_timing: id  3 | task 2926 | n_decoded =   1198, tg =  15.09 t/s, tg_3s =  14.67 t/s
5.51.414.638 I slot print_timing: id  3 | task 2926 | n_decoded =   1243, tg =  15.09 t/s, tg_3s =  14.96 t/s
5.54.480.665 I slot print_timing: id  3 | task 2926 | n_decoded =   1289, tg =  15.08 t/s, tg_3s =  15.00 t/s
5.57.485.991 I slot print_timing: id  3 | task 2926 | n_decoded =   1334, tg =  15.08 t/s, tg_3s =  14.97 t/s
6.00.544.636 I slot print_timing: id  3 | task 2926 | n_decoded =   1380, tg =  15.08 t/s, tg_3s =  15.04 t/s
6.03.600.994 I slot print_timing: id  3 | task 2926 | n_decoded =   1426, tg =  15.08 t/s, tg_3s =  15.05 t/s
6.06.659.843 I slot print_timing: id  3 | task 2926 | n_decoded =   1472, tg =  15.08 t/s, tg_3s =  15.04 t/s
6.09.697.119 I slot print_timing: id  3 | task 2926 | n_decoded =   1518, tg =  15.08 t/s, tg_3s =  15.15 t/s
6.12.727.669 I slot print_timing: id  3 | task 2926 | n_decoded =   1564, tg =  15.08 t/s, tg_3s =  15.18 t/s
6.15.749.934 I slot print_timing: id  3 | task 2926 | n_decoded =   1610, tg =  15.09 t/s, tg_3s =  15.22 t/s
6.18.767.751 I slot print_timing: id  3 | task 2926 | n_decoded =   1656, tg =  15.09 t/s, tg_3s =  15.24 t/s
6.20.209.753 I slot print_timing: id  3 | task 2926 | prompt eval time =    4256.71 ms /    36 tokens (  118.24 ms per token,     8.46 tokens per second)
6.20.209.760 I slot print_timing: id  3 | task 2926 |        eval time =  111182.54 ms /  1678 tokens (   66.26 ms per token,    15.09 tokens per second)
6.20.209.761 I slot print_timing: id  3 | task 2926 |       total time =  115439.25 ms /  1714 tokens
6.20.209.761 I slot print_timing: id  3 | task 2926 |    graphs reused =       4569
6.20.209.797 I slot      release: id  3 | task 2926 | stop processing: n_tokens = 1784, truncated = 0
6.20.209.821 I srv  update_slots: all slots are idle
6.20.242.060 I srv    operator(): Chat format: peg-native
6.20.242.377 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.771 (> 0.100 thold), f_keep = 0.041
6.20.242.379 I srv  get_availabl: updating prompt cache
6.20.242.471 W srv   prompt_save:  - saving prompt with length 1784, total state size = 178.647 MiB (draft: 0.000 MiB)
6.20.429.765 I srv          load:  - looking for better prompt, base f_keep = 0.041, sim = 0.771
6.20.429.773 I srv        update:  - cache state: 5 prompts, 1686.639 MiB (limits: 8192.000 MiB, 16384 tokens, 24678 est)
6.20.429.774 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
6.20.429.774 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
6.20.429.775 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
6.20.429.775 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
6.20.429.776 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
6.20.429.777 I srv  get_availabl: prompt cache update took 187.40 ms
6.20.429.877 I slot launch_slot_: id  3 | task 4606 | processing task, is_child = 0
6.20.429.878 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
6.20.429.878 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
6.20.429.893 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
6.20.429.894 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
6.20.429.902 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
6.20.429.902 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
6.20.429.919 I slot update_slots: id  3 | task 4606 | Checking checkpoint with [70, 70] against 74...
6.20.460.731 W slot update_slots: id  3 | task 4606 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
6.27.644.159 I slot print_timing: id  3 | task 4606 | n_decoded =    100, tg =  15.39 t/s, tg_3s =  15.39 t/s
6.30.698.646 I slot print_timing: id  3 | task 4606 | n_decoded =    146, tg =  15.29 t/s, tg_3s =  15.06 t/s
6.33.734.683 I slot print_timing: id  3 | task 4606 | n_decoded =    193, tg =  15.33 t/s, tg_3s =  15.48 t/s
6.36.763.330 I slot print_timing: id  3 | task 4606 | n_decoded =    239, tg =  15.31 t/s, tg_3s =  15.19 t/s
6.39.799.531 I slot print_timing: id  3 | task 4606 | n_decoded =    285, tg =  15.28 t/s, tg_3s =  15.15 t/s
6.42.839.614 I slot print_timing: id  3 | task 4606 | n_decoded =    331, tg =  15.26 t/s, tg_3s =  15.13 t/s
6.45.886.528 I slot print_timing: id  3 | task 4606 | n_decoded =    377, tg =  15.24 t/s, tg_3s =  15.10 t/s
6.48.943.576 I slot print_timing: id  3 | task 4606 | n_decoded =    423, tg =  15.22 t/s, tg_3s =  15.05 t/s
6.51.981.558 I slot print_timing: id  3 | task 4606 | n_decoded =    469, tg =  15.21 t/s, tg_3s =  15.14 t/s
6.55.024.547 I slot print_timing: id  3 | task 4606 | n_decoded =    515, tg =  15.20 t/s, tg_3s =  15.12 t/s
6.58.090.772 I slot print_timing: id  3 | task 4606 | n_decoded =    561, tg =  15.19 t/s, tg_3s =  15.00 t/s
7.01.090.820 I slot print_timing: id  3 | task 4606 | n_decoded =    606, tg =  15.17 t/s, tg_3s =  15.00 t/s
7.04.151.605 I slot print_timing: id  3 | task 4606 | n_decoded =    652, tg =  15.16 t/s, tg_3s =  15.03 t/s
7.07.174.888 I slot print_timing: id  3 | task 4606 | n_decoded =    697, tg =  15.14 t/s, tg_3s =  14.88 t/s
7.10.204.490 I slot print_timing: id  3 | task 4606 | n_decoded =    742, tg =  15.13 t/s, tg_3s =  14.85 t/s
7.13.257.704 I slot print_timing: id  3 | task 4606 | n_decoded =    787, tg =  15.10 t/s, tg_3s =  14.74 t/s
7.16.287.403 I slot print_timing: id  3 | task 4606 | n_decoded =    832, tg =  15.09 t/s, tg_3s =  14.85 t/s
7.19.296.871 I slot print_timing: id  3 | task 4606 | n_decoded =    877, tg =  15.08 t/s, tg_3s =  14.95 t/s
7.22.302.745 I slot print_timing: id  3 | task 4606 | n_decoded =    922, tg =  15.08 t/s, tg_3s =  14.97 t/s
7.25.351.530 I slot print_timing: id  3 | task 4606 | n_decoded =    967, tg =  15.06 t/s, tg_3s =  14.76 t/s
7.28.358.511 I slot print_timing: id  3 | task 4606 | n_decoded =   1012, tg =  15.06 t/s, tg_3s =  14.97 t/s
7.31.369.451 I slot print_timing: id  3 | task 4606 | n_decoded =   1057, tg =  15.05 t/s, tg_3s =  14.95 t/s
7.34.429.463 I slot print_timing: id  3 | task 4606 | n_decoded =   1103, tg =  15.05 t/s, tg_3s =  15.03 t/s
7.37.431.898 I slot print_timing: id  3 | task 4606 | n_decoded =   1148, tg =  15.05 t/s, tg_3s =  14.99 t/s
7.40.442.590 I slot print_timing: id  3 | task 4606 | n_decoded =   1193, tg =  15.05 t/s, tg_3s =  14.95 t/s
7.43.457.637 I slot print_timing: id  3 | task 4606 | n_decoded =   1238, tg =  15.04 t/s, tg_3s =  14.93 t/s
7.46.513.716 I slot print_timing: id  3 | task 4606 | n_decoded =   1284, tg =  15.04 t/s, tg_3s =  15.05 t/s
7.49.566.702 I slot print_timing: id  3 | task 4606 | n_decoded =   1330, tg =  15.04 t/s, tg_3s =  15.07 t/s
7.52.629.121 I slot print_timing: id  3 | task 4606 | n_decoded =   1376, tg =  15.04 t/s, tg_3s =  15.02 t/s
7.54.570.093 I slot print_timing: id  3 | task 4606 | prompt eval time =     718.30 ms /    25 tokens (   28.73 ms per token,    34.80 tokens per second)
7.54.570.098 I slot print_timing: id  3 | task 4606 |        eval time =   93421.86 ms /  1405 tokens (   66.49 ms per token,    15.04 tokens per second)
7.54.570.099 I slot print_timing: id  3 | task 4606 |       total time =   94140.16 ms /  1430 tokens
7.54.570.100 I slot print_timing: id  3 | task 4606 |    graphs reused =       5967
7.54.570.136 I slot      release: id  3 | task 4606 | stop processing: n_tokens = 1500, truncated = 0
7.54.570.156 I srv  update_slots: all slots are idle
7.54.602.760 I srv    operator(): Chat format: peg-native
7.54.603.024 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.815 (> 0.100 thold), f_keep = 0.050
7.54.603.026 I srv  get_availabl: updating prompt cache
7.54.603.099 W srv   prompt_save:  - saving prompt with length 1500, total state size = 176.425 MiB (draft: 0.000 MiB)
7.54.799.933 I srv          load:  - looking for better prompt, base f_keep = 0.050, sim = 0.815
7.54.799.942 I srv        update:  - cache state: 6 prompts, 2027.752 MiB (limits: 8192.000 MiB, 16384 tokens, 26586 est)
7.54.799.943 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
7.54.799.944 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
7.54.799.944 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
7.54.799.945 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
7.54.799.946 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
7.54.799.946 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
7.54.799.948 I srv  get_availabl: prompt cache update took 196.92 ms
7.54.800.064 I slot launch_slot_: id  3 | task 6013 | processing task, is_child = 0
7.54.800.066 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
7.54.800.067 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
7.54.800.089 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
7.54.800.089 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
7.54.800.113 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
7.54.800.117 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
7.54.800.139 I slot update_slots: id  3 | task 6013 | Checking checkpoint with [70, 70] against 75...
7.54.830.898 W slot update_slots: id  3 | task 6013 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
8.02.029.628 I slot print_timing: id  3 | task 6013 | n_decoded =    100, tg =  15.30 t/s, tg_3s =  15.30 t/s
8.05.046.863 I slot print_timing: id  3 | task 6013 | n_decoded =    146, tg =  15.28 t/s, tg_3s =  15.25 t/s
8.08.055.566 I slot print_timing: id  3 | task 6013 | n_decoded =    192, tg =  15.28 t/s, tg_3s =  15.29 t/s
8.11.099.502 I slot print_timing: id  3 | task 6013 | n_decoded =    239, tg =  15.31 t/s, tg_3s =  15.44 t/s
8.14.159.236 I slot print_timing: id  3 | task 6013 | n_decoded =    286, tg =  15.32 t/s, tg_3s =  15.36 t/s
8.17.204.394 I slot print_timing: id  3 | task 6013 | n_decoded =    332, tg =  15.29 t/s, tg_3s =  15.11 t/s
8.20.243.389 I slot print_timing: id  3 | task 6013 | n_decoded =    378, tg =  15.27 t/s, tg_3s =  15.14 t/s
8.23.289.407 I slot print_timing: id  3 | task 6013 | n_decoded =    424, tg =  15.25 t/s, tg_3s =  15.10 t/s
8.26.327.438 I slot print_timing: id  3 | task 6013 | n_decoded =    470, tg =  15.24 t/s, tg_3s =  15.14 t/s
8.29.382.541 I slot print_timing: id  3 | task 6013 | n_decoded =    516, tg =  15.23 t/s, tg_3s =  15.06 t/s
8.32.439.731 I slot print_timing: id  3 | task 6013 | n_decoded =    562, tg =  15.21 t/s, tg_3s =  15.05 t/s
8.35.479.656 I slot print_timing: id  3 | task 6013 | n_decoded =    608, tg =  15.20 t/s, tg_3s =  15.13 t/s
8.36.073.502 I slot print_timing: id  3 | task 6013 | prompt eval time =     691.99 ms /    21 tokens (   32.95 ms per token,    30.35 tokens per second)
8.36.073.508 I slot print_timing: id  3 | task 6013 |        eval time =   40581.36 ms /   617 tokens (   65.77 ms per token,    15.20 tokens per second)
8.36.073.509 I slot print_timing: id  3 | task 6013 |       total time =   41273.35 ms /   638 tokens
8.36.073.510 I slot print_timing: id  3 | task 6013 |    graphs reused =       6580
8.36.073.542 I slot      release: id  3 | task 6013 | stop processing: n_tokens = 708, truncated = 0
8.36.073.558 I srv  update_slots: all slots are idle
8.36.105.707 I srv    operator(): Chat format: peg-native
8.36.105.968 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.798 (> 0.100 thold), f_keep = 0.106
8.36.105.970 I srv  get_availabl: updating prompt cache
8.36.106.020 W srv   prompt_save:  - saving prompt with length 708, total state size = 170.228 MiB (draft: 0.000 MiB)
8.36.282.364 I srv          load:  - looking for better prompt, base f_keep = 0.106, sim = 0.798
8.36.282.375 I srv        update:  - cache state: 7 prompts, 2362.668 MiB (limits: 8192.000 MiB, 16384 tokens, 25272 est)
8.36.282.376 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
8.36.282.378 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
8.36.282.379 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
8.36.282.380 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
8.36.282.381 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
8.36.282.382 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
8.36.282.383 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
8.36.282.384 I srv  get_availabl: prompt cache update took 176.41 ms
8.36.282.510 I slot launch_slot_: id  3 | task 6632 | processing task, is_child = 0
8.36.282.511 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
8.36.282.511 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
8.36.282.528 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
8.36.282.529 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
8.36.282.540 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
8.36.282.542 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
8.36.282.566 I slot update_slots: id  3 | task 6632 | Checking checkpoint with [70, 70] against 75...
8.36.313.553 W slot update_slots: id  3 | task 6632 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
8.43.623.340 I slot print_timing: id  3 | task 6632 | n_decoded =    100, tg =  15.05 t/s, tg_3s =  15.05 t/s
8.46.670.304 I slot print_timing: id  3 | task 6632 | n_decoded =    146, tg =  15.07 t/s, tg_3s =  15.10 t/s
8.49.726.760 I slot print_timing: id  3 | task 6632 | n_decoded =    192, tg =  15.06 t/s, tg_3s =  15.05 t/s
8.52.776.601 I slot print_timing: id  3 | task 6632 | n_decoded =    238, tg =  15.07 t/s, tg_3s =  15.08 t/s
8.55.788.319 I slot print_timing: id  3 | task 6632 | n_decoded =    283, tg =  15.05 t/s, tg_3s =  14.94 t/s
8.58.797.522 I slot print_timing: id  3 | task 6632 | n_decoded =    328, tg =  15.03 t/s, tg_3s =  14.95 t/s
9.01.805.538 I slot print_timing: id  3 | task 6632 | n_decoded =    373, tg =  15.03 t/s, tg_3s =  14.96 t/s
9.04.820.351 I slot print_timing: id  3 | task 6632 | n_decoded =    418, tg =  15.01 t/s, tg_3s =  14.93 t/s
9.07.872.648 I slot print_timing: id  3 | task 6632 | n_decoded =    463, tg =  14.99 t/s, tg_3s =  14.74 t/s
9.10.876.394 I slot print_timing: id  3 | task 6632 | n_decoded =    508, tg =  14.99 t/s, tg_3s =  14.98 t/s
9.13.882.732 I slot print_timing: id  3 | task 6632 | n_decoded =    553, tg =  14.99 t/s, tg_3s =  14.97 t/s
9.16.947.453 I slot print_timing: id  3 | task 6632 | n_decoded =    599, tg =  14.99 t/s, tg_3s =  15.01 t/s
9.17.813.365 I slot print_timing: id  3 | task 6632 | prompt eval time =     698.00 ms /    23 tokens (   30.35 ms per token,    32.95 tokens per second)
9.17.813.370 I slot print_timing: id  3 | task 6632 |        eval time =   40832.79 ms /   612 tokens (   66.72 ms per token,    14.99 tokens per second)
9.17.813.370 I slot print_timing: id  3 | task 6632 |       total time =   41530.79 ms /   635 tokens
9.17.813.371 I slot print_timing: id  3 | task 6632 |    graphs reused =       7188
9.17.813.403 I slot      release: id  3 | task 6632 | stop processing: n_tokens = 705, truncated = 0
9.17.813.420 I srv  update_slots: all slots are idle
9.17.843.326 I srv    operator(): Chat format: peg-native
9.17.843.562 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.607 (> 0.100 thold), f_keep = 0.105
9.17.843.563 I srv  get_availabl: updating prompt cache
9.17.843.607 W srv   prompt_save:  - saving prompt with length 705, total state size = 170.205 MiB (draft: 0.000 MiB)
9.18.018.169 I srv          load:  - looking for better prompt, base f_keep = 0.105, sim = 0.607
9.18.018.179 I srv        update:  - cache state: 8 prompts, 2697.561 MiB (limits: 8192.000 MiB, 16384 tokens, 24276 est)
9.18.018.180 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
9.18.018.180 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
9.18.018.182 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
9.18.018.182 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
9.18.018.184 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
9.18.018.184 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
9.18.018.185 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
9.18.018.185 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
9.18.018.186 I srv  get_availabl: prompt cache update took 174.62 ms
9.18.018.276 I slot launch_slot_: id  3 | task 7246 | processing task, is_child = 0
9.18.018.280 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
9.18.018.281 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
9.18.018.295 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
9.18.018.295 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
9.18.018.303 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
9.18.018.304 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
9.18.018.321 I slot update_slots: id  3 | task 7246 | Checking checkpoint with [70, 70] against 74...
9.18.049.307 W slot update_slots: id  3 | task 7246 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
9.23.000.095 I slot print_timing: id  3 | task 7246 | prompt processing, n_tokens =     47, progress = 0.97, t =   4.98 s / 9.43 tokens per second
9.29.741.637 I slot print_timing: id  3 | task 7246 | n_decoded =    100, tg =  15.19 t/s, tg_3s =  15.19 t/s
9.32.759.727 I slot print_timing: id  3 | task 7246 | n_decoded =    145, tg =  15.10 t/s, tg_3s =  14.91 t/s
9.35.787.431 I slot print_timing: id  3 | task 7246 | n_decoded =    191, tg =  15.13 t/s, tg_3s =  15.19 t/s
9.38.796.648 I slot print_timing: id  3 | task 7246 | n_decoded =    237, tg =  15.16 t/s, tg_3s =  15.29 t/s
9.41.837.891 I slot print_timing: id  3 | task 7246 | n_decoded =    283, tg =  15.15 t/s, tg_3s =  15.13 t/s
9.44.858.007 I slot print_timing: id  3 | task 7246 | n_decoded =    329, tg =  15.16 t/s, tg_3s =  15.23 t/s
9.47.860.537 I slot print_timing: id  3 | task 7246 | n_decoded =    375, tg =  15.18 t/s, tg_3s =  15.32 t/s
9.50.912.985 I slot print_timing: id  3 | task 7246 | n_decoded =    422, tg =  15.21 t/s, tg_3s =  15.40 t/s
9.53.961.044 I slot print_timing: id  3 | task 7246 | n_decoded =    469, tg =  15.23 t/s, tg_3s =  15.42 t/s
9.57.019.294 I slot print_timing: id  3 | task 7246 | n_decoded =    515, tg =  15.21 t/s, tg_3s =  15.04 t/s
10.00.068.531 I slot print_timing: id  3 | task 7246 | n_decoded =    561, tg =  15.20 t/s, tg_3s =  15.09 t/s
10.01.530.251 I slot print_timing: id  3 | task 7246 | prompt eval time =    5141.55 ms /    51 tokens (  100.81 ms per token,     9.92 tokens per second)
10.01.530.258 I slot print_timing: id  3 | task 7246 |        eval time =   38370.37 ms /   583 tokens (   65.82 ms per token,    15.19 tokens per second)
10.01.530.258 I slot print_timing: id  3 | task 7246 |       total time =   43511.92 ms /   634 tokens
10.01.530.259 I slot print_timing: id  3 | task 7246 |    graphs reused =       7767
10.01.530.298 I slot      release: id  3 | task 7246 | stop processing: n_tokens = 704, truncated = 0
10.01.530.316 I srv  update_slots: all slots are idle
10.01.562.894 I srv    operator(): Chat format: peg-native
10.01.563.143 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.528 (> 0.100 thold), f_keep = 0.107
10.01.563.146 I srv  get_availabl: updating prompt cache
10.01.563.203 W srv   prompt_save:  - saving prompt with length 704, total state size = 170.197 MiB (draft: 0.000 MiB)
10.01.738.556 I srv          load:  - looking for better prompt, base f_keep = 0.107, sim = 0.528
10.01.738.566 I srv        update:  - cache state: 9 prompts, 3032.446 MiB (limits: 8192.000 MiB, 16384 tokens, 23497 est)
10.01.738.568 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
10.01.738.571 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
10.01.738.572 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
10.01.738.572 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
10.01.738.573 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
10.01.738.574 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
10.01.738.577 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
10.01.738.577 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
10.01.738.578 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
10.01.738.579 I srv  get_availabl: prompt cache update took 175.43 ms
10.01.738.681 I slot launch_slot_: id  3 | task 7831 | processing task, is_child = 0
10.01.738.682 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
10.01.738.683 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
10.01.738.700 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
10.01.738.700 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
10.01.738.710 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
10.01.738.710 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
10.01.738.728 I slot update_slots: id  3 | task 7831 | Checking checkpoint with [70, 70] against 75...
10.01.769.949 W slot update_slots: id  3 | task 7831 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
10.07.098.790 I slot print_timing: id  3 | task 7831 | prompt processing, n_tokens =     67, progress = 0.97, t =   5.36 s / 12.50 tokens per second
10.13.817.310 I slot print_timing: id  3 | task 7831 | n_decoded =    100, tg =  15.25 t/s, tg_3s =  15.25 t/s
10.16.870.585 I slot print_timing: id  3 | task 7831 | n_decoded =    146, tg =  15.19 t/s, tg_3s =  15.07 t/s
10.19.910.548 I slot print_timing: id  3 | task 7831 | n_decoded =    192, tg =  15.18 t/s, tg_3s =  15.13 t/s
10.22.950.539 I slot print_timing: id  3 | task 7831 | n_decoded =    238, tg =  15.17 t/s, tg_3s =  15.13 t/s
10.25.997.582 I slot print_timing: id  3 | task 7831 | n_decoded =    284, tg =  15.16 t/s, tg_3s =  15.10 t/s
10.29.005.309 I slot print_timing: id  3 | task 7831 | n_decoded =    329, tg =  15.13 t/s, tg_3s =  14.96 t/s
10.32.008.284 I slot print_timing: id  3 | task 7831 | n_decoded =    374, tg =  15.11 t/s, tg_3s =  14.99 t/s
10.35.018.128 I slot print_timing: id  3 | task 7831 | n_decoded =    419, tg =  15.09 t/s, tg_3s =  14.95 t/s
10.38.058.390 I slot print_timing: id  3 | task 7831 | n_decoded =    464, tg =  15.07 t/s, tg_3s =  14.80 t/s
10.41.092.505 I slot print_timing: id  3 | task 7831 | n_decoded =    509, tg =  15.04 t/s, tg_3s =  14.83 t/s
10.44.109.193 I slot print_timing: id  3 | task 7831 | n_decoded =    554, tg =  15.03 t/s, tg_3s =  14.92 t/s
10.44.641.184 I slot print_timing: id  3 | task 7831 | prompt eval time =    5521.12 ms /    71 tokens (   77.76 ms per token,    12.86 tokens per second)
10.44.641.189 I slot print_timing: id  3 | task 7831 |        eval time =   37381.32 ms /   562 tokens (   66.51 ms per token,    15.03 tokens per second)
10.44.641.190 I slot print_timing: id  3 | task 7831 |       total time =   42902.44 ms /   633 tokens
10.44.641.191 I slot print_timing: id  3 | task 7831 |    graphs reused =       8325
10.44.641.235 I slot      release: id  3 | task 7831 | stop processing: n_tokens = 703, truncated = 0
10.44.641.253 I srv  update_slots: all slots are idle
10.44.672.783 I srv    operator(): Chat format: peg-native
10.44.673.126 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.632 (> 0.100 thold), f_keep = 0.105
10.44.673.128 I srv  get_availabl: updating prompt cache
10.44.673.180 W srv   prompt_save:  - saving prompt with length 703, total state size = 170.189 MiB (draft: 0.000 MiB)
10.44.846.836 I srv          load:  - looking for better prompt, base f_keep = 0.105, sim = 0.632
10.44.846.847 I srv        update:  - cache state: 10 prompts, 3367.324 MiB (limits: 8192.000 MiB, 16384 tokens, 22870 est)
10.44.846.848 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
10.44.846.850 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
10.44.846.854 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
10.44.846.854 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
10.44.846.855 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
10.44.846.856 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
10.44.846.856 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
10.44.846.858 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
10.44.846.859 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
10.44.846.859 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
10.44.846.860 I srv  get_availabl: prompt cache update took 173.73 ms
10.44.846.970 I slot launch_slot_: id  3 | task 8395 | processing task, is_child = 0
10.44.846.971 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
10.44.846.972 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
10.44.846.987 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
10.44.846.987 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
10.44.847.009 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
10.44.847.009 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
10.44.847.028 I slot update_slots: id  3 | task 8395 | Checking checkpoint with [70, 70] against 74...
10.44.878.000 W slot update_slots: id  3 | task 8395 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
10.49.480.094 I slot print_timing: id  3 | task 8395 | prompt processing, n_tokens =     42, progress = 0.97, t =   4.63 s / 9.07 tokens per second
10.56.271.632 I slot print_timing: id  3 | task 8395 | n_decoded =    100, tg =  15.08 t/s, tg_3s =  15.08 t/s
10.59.272.337 I slot print_timing: id  3 | task 8395 | n_decoded =    145, tg =  15.05 t/s, tg_3s =  15.00 t/s
11.02.276.359 I slot print_timing: id  3 | task 8395 | n_decoded =    190, tg =  15.03 t/s, tg_3s =  14.98 t/s
11.05.278.616 I slot print_timing: id  3 | task 8395 | n_decoded =    235, tg =  15.03 t/s, tg_3s =  14.99 t/s
11.08.293.524 I slot print_timing: id  3 | task 8395 | n_decoded =    280, tg =  15.01 t/s, tg_3s =  14.93 t/s
11.11.332.306 I slot print_timing: id  3 | task 8395 | n_decoded =    326, tg =  15.03 t/s, tg_3s =  15.14 t/s
11.14.373.701 I slot print_timing: id  3 | task 8395 | n_decoded =    372, tg =  15.04 t/s, tg_3s =  15.12 t/s
11.17.403.461 I slot print_timing: id  3 | task 8395 | n_decoded =    418, tg =  15.05 t/s, tg_3s =  15.18 t/s
11.20.424.549 I slot print_timing: id  3 | task 8395 | n_decoded =    464, tg =  15.07 t/s, tg_3s =  15.23 t/s
11.23.472.409 I slot print_timing: id  3 | task 8395 | n_decoded =    510, tg =  15.07 t/s, tg_3s =  15.09 t/s
11.26.534.737 I slot print_timing: id  3 | task 8395 | n_decoded =    556, tg =  15.07 t/s, tg_3s =  15.02 t/s
11.29.588.823 I slot print_timing: id  3 | task 8395 | n_decoded =    603, tg =  15.09 t/s, tg_3s =  15.39 t/s
11.30.441.377 I slot print_timing: id  3 | task 8395 | prompt eval time =    4791.53 ms /    46 tokens (  104.16 ms per token,     9.60 tokens per second)
11.30.441.382 I slot print_timing: id  3 | task 8395 |        eval time =   40802.80 ms /   616 tokens (   66.24 ms per token,    15.10 tokens per second)
11.30.441.383 I slot print_timing: id  3 | task 8395 |       total time =   45594.34 ms /   662 tokens
11.30.441.384 I slot print_timing: id  3 | task 8395 |    graphs reused =       8937
11.30.441.424 I slot      release: id  3 | task 8395 | stop processing: n_tokens = 732, truncated = 0
11.30.441.442 I srv  update_slots: all slots are idle
11.30.470.461 I srv    operator(): Chat format: peg-native
11.30.470.708 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.525 (> 0.100 thold), f_keep = 0.101
11.30.470.711 I srv  get_availabl: updating prompt cache
11.30.470.765 W srv   prompt_save:  - saving prompt with length 732, total state size = 170.416 MiB (draft: 0.000 MiB)
11.30.648.747 I srv          load:  - looking for better prompt, base f_keep = 0.101, sim = 0.525
11.30.648.757 I srv        update:  - cache state: 11 prompts, 3702.428 MiB (limits: 8192.000 MiB, 16384 tokens, 22420 est)
11.30.648.759 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
11.30.648.759 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
11.30.648.771 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
11.30.648.772 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
11.30.648.773 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
11.30.648.774 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
11.30.648.775 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
11.30.648.775 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
11.30.648.776 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
11.30.648.776 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
11.30.648.777 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
11.30.648.778 I srv  get_availabl: prompt cache update took 178.07 ms
11.30.648.898 I slot launch_slot_: id  3 | task 9013 | processing task, is_child = 0
11.30.648.902 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
11.30.648.902 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
11.30.648.918 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
11.30.648.919 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
11.30.648.929 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
11.30.648.930 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
11.30.648.950 I slot update_slots: id  3 | task 9013 | Checking checkpoint with [70, 70] against 74...
11.30.679.703 W slot update_slots: id  3 | task 9013 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
11.35.894.800 I slot print_timing: id  3 | task 9013 | prompt processing, n_tokens =     66, progress = 0.97, t =   5.25 s / 12.58 tokens per second
11.42.568.924 I slot print_timing: id  3 | task 9013 | n_decoded =    100, tg =  15.35 t/s, tg_3s =  15.35 t/s
11.45.598.178 I slot print_timing: id  3 | task 9013 | n_decoded =    146, tg =  15.29 t/s, tg_3s =  15.19 t/s
11.48.640.255 I slot print_timing: id  3 | task 9013 | n_decoded =    192, tg =  15.25 t/s, tg_3s =  15.12 t/s
11.51.676.211 I slot print_timing: id  3 | task 9013 | n_decoded =    238, tg =  15.23 t/s, tg_3s =  15.15 t/s
11.54.714.249 I slot print_timing: id  3 | task 9013 | n_decoded =    284, tg =  15.22 t/s, tg_3s =  15.14 t/s
11.57.765.240 I slot print_timing: id  3 | task 9013 | n_decoded =    330, tg =  15.20 t/s, tg_3s =  15.08 t/s
12.00.830.557 I slot print_timing: id  3 | task 9013 | n_decoded =    376, tg =  15.17 t/s, tg_3s =  15.01 t/s
12.02.423.420 I slot print_timing: id  3 | task 9013 | prompt eval time =    5403.30 ms /    70 tokens (   77.19 ms per token,    12.96 tokens per second)
12.02.423.424 I slot print_timing: id  3 | task 9013 |        eval time =   26371.15 ms /   400 tokens (   65.93 ms per token,    15.17 tokens per second)
12.02.423.425 I slot print_timing: id  3 | task 9013 |       total time =   31774.45 ms /   470 tokens
12.02.423.425 I slot print_timing: id  3 | task 9013 |    graphs reused =       9333
12.02.423.459 I slot      release: id  3 | task 9013 | stop processing: n_tokens = 540, truncated = 0
12.02.423.473 I srv  update_slots: all slots are idle
12.02.452.191 I srv    operator(): Chat format: peg-native
12.02.452.435 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.552 (> 0.100 thold), f_keep = 0.137
12.02.452.437 I srv  get_availabl: updating prompt cache
12.02.452.475 W srv   prompt_save:  - saving prompt with length 540, total state size = 168.914 MiB (draft: 0.000 MiB)
12.02.629.940 I srv          load:  - looking for better prompt, base f_keep = 0.137, sim = 0.552
12.02.629.955 I srv        update:  - cache state: 12 prompts, 4036.030 MiB (limits: 8192.000 MiB, 16384 tokens, 21663 est)
12.02.629.957 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
12.02.629.958 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
12.02.629.959 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
12.02.629.960 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
12.02.629.960 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
12.02.629.961 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
12.02.629.962 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
12.02.629.963 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
12.02.629.963 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
12.02.629.964 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
12.02.629.965 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
12.02.629.966 I srv        update:    - prompt 0x5a83af787180:     540 tokens, checkpoints:  1,   333.602 MiB
12.02.629.967 I srv  get_availabl: prompt cache update took 177.53 ms
12.02.630.095 I slot launch_slot_: id  3 | task 9415 | processing task, is_child = 0
12.02.630.097 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
12.02.630.098 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
12.02.630.119 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
12.02.630.120 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
12.02.630.134 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
12.02.630.135 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
12.02.630.155 I slot update_slots: id  3 | task 9415 | Checking checkpoint with [70, 70] against 74...
12.02.660.960 W slot update_slots: id  3 | task 9415 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
12.07.873.359 I slot print_timing: id  3 | task 9415 | prompt processing, n_tokens =     59, progress = 0.97, t =   5.24 s / 11.25 tokens per second
12.14.664.363 I slot print_timing: id  3 | task 9415 | n_decoded =    100, tg =  15.07 t/s, tg_3s =  15.07 t/s
12.17.685.031 I slot print_timing: id  3 | task 9415 | n_decoded =    145, tg =  15.02 t/s, tg_3s =  14.90 t/s
12.20.719.365 I slot print_timing: id  3 | task 9415 | n_decoded =    190, tg =  14.97 t/s, tg_3s =  14.83 t/s
12.23.728.296 I slot print_timing: id  3 | task 9415 | n_decoded =    235, tg =  14.97 t/s, tg_3s =  14.96 t/s
12.26.736.124 I slot print_timing: id  3 | task 9415 | n_decoded =    280, tg =  14.97 t/s, tg_3s =  14.96 t/s
12.29.749.566 I slot print_timing: id  3 | task 9415 | n_decoded =    325, tg =  14.96 t/s, tg_3s =  14.93 t/s
12.32.777.387 I slot print_timing: id  3 | task 9415 | n_decoded =    370, tg =  14.95 t/s, tg_3s =  14.86 t/s
12.35.822.197 I slot print_timing: id  3 | task 9415 | n_decoded =    415, tg =  14.93 t/s, tg_3s =  14.78 t/s
12.38.836.205 I slot print_timing: id  3 | task 9415 | n_decoded =    460, tg =  14.93 t/s, tg_3s =  14.93 t/s
12.38.970.223 I slot print_timing: id  3 | task 9415 | prompt eval time =    5400.34 ms /    63 tokens (   85.72 ms per token,    11.67 tokens per second)
12.38.970.228 I slot print_timing: id  3 | task 9415 |        eval time =   30939.72 ms /   462 tokens (   66.97 ms per token,    14.93 tokens per second)
12.38.970.229 I slot print_timing: id  3 | task 9415 |       total time =   36340.06 ms /   525 tokens
12.38.970.229 I slot print_timing: id  3 | task 9415 |    graphs reused =       9791
12.38.970.270 I slot      release: id  3 | task 9415 | stop processing: n_tokens = 595, truncated = 0
12.38.970.286 I srv  update_slots: all slots are idle
12.38.999.282 I srv    operator(): Chat format: peg-native
12.38.999.527 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.605 (> 0.100 thold), f_keep = 0.126
12.38.999.529 I srv  get_availabl: updating prompt cache
12.38.999.588 W srv   prompt_save:  - saving prompt with length 595, total state size = 169.344 MiB (draft: 0.000 MiB)
12.39.176.453 I srv          load:  - looking for better prompt, base f_keep = 0.126, sim = 0.605
12.39.176.466 I srv        update:  - cache state: 13 prompts, 4370.062 MiB (limits: 8192.000 MiB, 16384 tokens, 21122 est)
12.39.176.470 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
12.39.176.471 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
12.39.176.472 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
12.39.176.473 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
12.39.176.473 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
12.39.176.474 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
12.39.176.475 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
12.39.176.478 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
12.39.176.478 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
12.39.176.479 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
12.39.176.480 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
12.39.176.481 I srv        update:    - prompt 0x5a83af787180:     540 tokens, checkpoints:  1,   333.602 MiB
12.39.176.481 I srv        update:    - prompt 0x5a83afa46850:     595 tokens, checkpoints:  1,   334.032 MiB
12.39.176.483 I srv  get_availabl: prompt cache update took 176.95 ms
12.39.176.609 I slot launch_slot_: id  3 | task 9879 | processing task, is_child = 0
12.39.176.611 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
12.39.176.611 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
12.39.176.633 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
12.39.176.633 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
12.39.176.658 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
12.39.176.658 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
12.39.176.690 I slot update_slots: id  3 | task 9879 | Checking checkpoint with [70, 70] against 75...
12.39.207.510 W slot update_slots: id  3 | task 9879 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
12.44.010.829 I slot print_timing: id  3 | task 9879 | prompt processing, n_tokens =     49, progress = 0.97, t =   4.83 s / 10.14 tokens per second
12.50.817.159 I slot print_timing: id  3 | task 9879 | n_decoded =    100, tg =  15.04 t/s, tg_3s =  15.04 t/s
12.53.866.409 I slot print_timing: id  3 | task 9879 | n_decoded =    146, tg =  15.06 t/s, tg_3s =  15.09 t/s
12.56.898.689 I slot print_timing: id  3 | task 9879 | n_decoded =    192, tg =  15.08 t/s, tg_3s =  15.17 t/s
12.59.950.529 I slot print_timing: id  3 | task 9879 | n_decoded =    238, tg =  15.08 t/s, tg_3s =  15.07 t/s
13.02.963.481 I slot print_timing: id  3 | task 9879 | n_decoded =    283, tg =  15.06 t/s, tg_3s =  14.94 t/s
13.05.995.536 I slot print_timing: id  3 | task 9879 | n_decoded =    329, tg =  15.07 t/s, tg_3s =  15.17 t/s
13.09.022.192 I slot print_timing: id  3 | task 9879 | n_decoded =    375, tg =  15.09 t/s, tg_3s =  15.20 t/s
13.12.034.375 I slot print_timing: id  3 | task 9879 | n_decoded =    421, tg =  15.11 t/s, tg_3s =  15.27 t/s
13.14.596.833 I slot print_timing: id  3 | task 9879 | prompt eval time =    4993.02 ms /    53 tokens (   94.21 ms per token,    10.61 tokens per second)
13.14.596.838 I slot print_timing: id  3 | task 9879 |        eval time =   30427.11 ms /   460 tokens (   66.15 ms per token,    15.12 tokens per second)
13.14.596.839 I slot print_timing: id  3 | task 9879 |       total time =   35420.13 ms /   513 tokens
13.14.596.840 I slot print_timing: id  3 | task 9879 |    graphs reused =      10247
13.14.596.876 I slot      release: id  3 | task 9879 | stop processing: n_tokens = 583, truncated = 0
13.14.596.891 I srv  update_slots: all slots are idle
13.14.629.505 I srv    operator(): Chat format: peg-native
13.14.629.757 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.569 (> 0.100 thold), f_keep = 0.127
13.14.629.759 I srv  get_availabl: updating prompt cache
13.14.629.813 W srv   prompt_save:  - saving prompt with length 583, total state size = 169.250 MiB (draft: 0.000 MiB)
13.14.818.404 I srv          load:  - looking for better prompt, base f_keep = 0.127, sim = 0.569
13.14.818.415 I srv        update:  - cache state: 14 prompts, 4704.001 MiB (limits: 8192.000 MiB, 16384 tokens, 20638 est)
13.14.818.416 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
13.14.818.416 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
13.14.818.417 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
13.14.818.418 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
13.14.818.418 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
13.14.818.419 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
13.14.818.419 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
13.14.818.420 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
13.14.818.420 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
13.14.818.421 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
13.14.818.421 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
13.14.818.422 I srv        update:    - prompt 0x5a83af787180:     540 tokens, checkpoints:  1,   333.602 MiB
13.14.818.422 I srv        update:    - prompt 0x5a83afa46850:     595 tokens, checkpoints:  1,   334.032 MiB
13.14.818.423 I srv        update:    - prompt 0x5a83af78b470:     583 tokens, checkpoints:  1,   333.938 MiB
13.14.818.424 I srv  get_availabl: prompt cache update took 188.66 ms
13.14.818.520 I slot launch_slot_: id  3 | task 10341 | processing task, is_child = 0
13.14.818.521 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
13.14.818.521 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
13.14.818.533 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
13.14.818.534 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
13.14.818.554 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
13.14.818.554 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
13.14.818.571 I slot update_slots: id  3 | task 10341 | Checking checkpoint with [70, 70] against 74...
13.14.849.525 W slot update_slots: id  3 | task 10341 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
13.20.188.287 I slot print_timing: id  3 | task 10341 | prompt processing, n_tokens =     55, progress = 0.97, t =   5.37 s / 10.24 tokens per second
13.26.908.066 I slot print_timing: id  3 | task 10341 | n_decoded =    100, tg =  15.25 t/s, tg_3s =  15.25 t/s
13.29.968.063 I slot print_timing: id  3 | task 10341 | n_decoded =    146, tg =  15.18 t/s, tg_3s =  15.03 t/s
13.32.974.260 I slot print_timing: id  3 | task 10341 | n_decoded =    191, tg =  15.13 t/s, tg_3s =  14.97 t/s
13.36.010.282 I slot print_timing: id  3 | task 10341 | n_decoded =    237, tg =  15.14 t/s, tg_3s =  15.15 t/s
13.39.046.356 I slot print_timing: id  3 | task 10341 | n_decoded =    283, tg =  15.14 t/s, tg_3s =  15.15 t/s
13.42.092.165 I slot print_timing: id  3 | task 10341 | n_decoded =    329, tg =  15.13 t/s, tg_3s =  15.10 t/s
13.45.106.241 I slot print_timing: id  3 | task 10341 | n_decoded =    374, tg =  15.11 t/s, tg_3s =  14.93 t/s
13.48.113.383 I slot print_timing: id  3 | task 10341 | n_decoded =    419, tg =  15.09 t/s, tg_3s =  14.96 t/s
13.51.113.393 I slot print_timing: id  3 | task 10341 | n_decoded =    464, tg =  15.08 t/s, tg_3s =  15.00 t/s
13.54.122.121 I slot print_timing: id  3 | task 10341 | n_decoded =    509, tg =  15.07 t/s, tg_3s =  14.96 t/s
13.57.133.277 I slot print_timing: id  3 | task 10341 | n_decoded =    554, tg =  15.06 t/s, tg_3s =  14.94 t/s
14.00.135.343 I slot print_timing: id  3 | task 10341 | n_decoded =    598, tg =  15.03 t/s, tg_3s =  14.66 t/s
14.03.156.960 I slot print_timing: id  3 | task 10341 | n_decoded =    643, tg =  15.02 t/s, tg_3s =  14.89 t/s
14.06.158.002 I slot print_timing: id  3 | task 10341 | n_decoded =    688, tg =  15.02 t/s, tg_3s =  14.99 t/s
14.09.214.452 I slot print_timing: id  3 | task 10341 | n_decoded =    734, tg =  15.02 t/s, tg_3s =  15.05 t/s
14.12.216.969 I slot print_timing: id  3 | task 10341 | n_decoded =    779, tg =  15.02 t/s, tg_3s =  14.99 t/s
14.13.486.032 I slot print_timing: id  3 | task 10341 | prompt eval time =    5533.58 ms /    59 tokens (   93.79 ms per token,    10.66 tokens per second)
14.13.486.037 I slot print_timing: id  3 | task 10341 |        eval time =   53133.87 ms /   798 tokens (   66.58 ms per token,    15.02 tokens per second)
14.13.486.038 I slot print_timing: id  3 | task 10341 |       total time =   58667.45 ms /   857 tokens
14.13.486.039 I slot print_timing: id  3 | task 10341 |    graphs reused =      11040
14.13.486.077 I slot      release: id  3 | task 10341 | stop processing: n_tokens = 927, truncated = 0
14.13.486.096 I srv  update_slots: all slots are idle
14.13.515.688 I srv    operator(): Chat format: peg-native
14.13.515.931 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.638 (> 0.100 thold), f_keep = 0.080
14.13.515.933 I srv  get_availabl: updating prompt cache
14.13.516.003 W srv   prompt_save:  - saving prompt with length 927, total state size = 171.941 MiB (draft: 0.000 MiB)
14.13.696.193 I srv          load:  - looking for better prompt, base f_keep = 0.080, sim = 0.638
14.13.696.207 I srv        update:  - cache state: 15 prompts, 5040.631 MiB (limits: 8192.000 MiB, 16384 tokens, 20766 est)
14.13.696.208 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
14.13.696.209 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
14.13.696.210 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
14.13.696.211 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
14.13.696.212 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
14.13.696.213 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
14.13.696.213 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
14.13.696.214 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
14.13.696.215 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
14.13.696.215 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
14.13.696.216 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
14.13.696.217 I srv        update:    - prompt 0x5a83af787180:     540 tokens, checkpoints:  1,   333.602 MiB
14.13.696.218 I srv        update:    - prompt 0x5a83afa46850:     595 tokens, checkpoints:  1,   334.032 MiB
14.13.696.218 I srv        update:    - prompt 0x5a83af78b470:     583 tokens, checkpoints:  1,   333.938 MiB
14.13.696.219 I srv        update:    - prompt 0x5a83af682c20:     927 tokens, checkpoints:  1,   336.630 MiB
14.13.696.221 I srv  get_availabl: prompt cache update took 180.29 ms
14.13.696.348 I slot launch_slot_: id  3 | task 11141 | processing task, is_child = 0
14.13.696.350 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
14.13.696.351 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
14.13.696.370 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
14.13.696.370 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
14.13.696.382 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
14.13.696.382 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
14.13.696.403 I slot update_slots: id  3 | task 11141 | Checking checkpoint with [70, 70] against 74...
14.13.727.412 W slot update_slots: id  3 | task 11141 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
14.18.132.325 I slot print_timing: id  3 | task 11141 | prompt processing, n_tokens =     41, progress = 0.97, t =   4.44 s / 9.24 tokens per second
14.24.932.148 I slot print_timing: id  3 | task 11141 | n_decoded =    100, tg =  15.06 t/s, tg_3s =  15.06 t/s
14.27.934.183 I slot print_timing: id  3 | task 11141 | n_decoded =    145, tg =  15.04 t/s, tg_3s =  14.99 t/s
14.30.981.086 I slot print_timing: id  3 | task 11141 | n_decoded =    191, tg =  15.05 t/s, tg_3s =  15.10 t/s
14.34.044.172 I slot print_timing: id  3 | task 11141 | n_decoded =    237, tg =  15.05 t/s, tg_3s =  15.02 t/s
14.37.076.300 I slot print_timing: id  3 | task 11141 | n_decoded =    283, tg =  15.07 t/s, tg_3s =  15.17 t/s
14.40.104.204 I slot print_timing: id  3 | task 11141 | n_decoded =    329, tg =  15.08 t/s, tg_3s =  15.19 t/s
14.43.126.573 I slot print_timing: id  3 | task 11141 | n_decoded =    375, tg =  15.10 t/s, tg_3s =  15.22 t/s
14.46.151.237 I slot print_timing: id  3 | task 11141 | n_decoded =    420, tg =  15.08 t/s, tg_3s =  14.88 t/s
14.49.210.994 I slot print_timing: id  3 | task 11141 | n_decoded =    467, tg =  15.10 t/s, tg_3s =  15.36 t/s
14.52.240.012 I slot print_timing: id  3 | task 11141 | n_decoded =    514, tg =  15.14 t/s, tg_3s =  15.52 t/s
14.55.241.709 I slot print_timing: id  3 | task 11141 | n_decoded =    560, tg =  15.16 t/s, tg_3s =  15.32 t/s
14.58.266.033 I slot print_timing: id  3 | task 11141 | n_decoded =    606, tg =  15.16 t/s, tg_3s =  15.21 t/s
15.01.314.655 I slot print_timing: id  3 | task 11141 | n_decoded =    652, tg =  15.15 t/s, tg_3s =  15.09 t/s
15.04.361.749 I slot print_timing: id  3 | task 11141 | n_decoded =    698, tg =  15.15 t/s, tg_3s =  15.10 t/s
15.07.406.939 I slot print_timing: id  3 | task 11141 | n_decoded =    744, tg =  15.15 t/s, tg_3s =  15.11 t/s
15.10.455.990 I slot print_timing: id  3 | task 11141 | n_decoded =    790, tg =  15.14 t/s, tg_3s =  15.09 t/s
15.13.503.983 I slot print_timing: id  3 | task 11141 | n_decoded =    836, tg =  15.14 t/s, tg_3s =  15.09 t/s
15.16.559.938 I slot print_timing: id  3 | task 11141 | n_decoded =    882, tg =  15.14 t/s, tg_3s =  15.05 t/s
15.19.623.155 I slot print_timing: id  3 | task 11141 | n_decoded =    928, tg =  15.13 t/s, tg_3s =  15.02 t/s
15.22.680.218 I slot print_timing: id  3 | task 11141 | n_decoded =    974, tg =  15.13 t/s, tg_3s =  15.05 t/s
15.25.698.214 I slot print_timing: id  3 | task 11141 | n_decoded =   1019, tg =  15.12 t/s, tg_3s =  14.91 t/s
15.28.752.848 I slot print_timing: id  3 | task 11141 | n_decoded =   1065, tg =  15.11 t/s, tg_3s =  15.06 t/s
15.30.617.098 I slot print_timing: id  3 | task 11141 | prompt eval time =    4595.19 ms /    45 tokens (  102.12 ms per token,     9.79 tokens per second)
15.30.617.103 I slot print_timing: id  3 | task 11141 |        eval time =   72325.49 ms /  1093 tokens (   66.17 ms per token,    15.11 tokens per second)
15.30.617.103 I slot print_timing: id  3 | task 11141 |       total time =   76920.68 ms /  1138 tokens
15.30.617.104 I slot print_timing: id  3 | task 11141 |    graphs reused =      12127
15.30.617.138 I slot      release: id  3 | task 11141 | stop processing: n_tokens = 1208, truncated = 0
15.30.617.159 I srv  update_slots: all slots are idle
15.30.649.420 I srv    operator(): Chat format: peg-native
15.30.649.672 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.638 (> 0.100 thold), f_keep = 0.061
15.30.649.674 I srv  get_availabl: updating prompt cache
15.30.649.748 W srv   prompt_save:  - saving prompt with length 1208, total state size = 174.140 MiB (draft: 0.000 MiB)
15.30.828.779 I srv          load:  - looking for better prompt, base f_keep = 0.061, sim = 0.638
15.30.828.790 I srv        update:  - cache state: 16 prompts, 5379.459 MiB (limits: 8192.000 MiB, 16384 tokens, 21298 est)
15.30.828.792 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
15.30.828.793 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
15.30.828.793 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
15.30.828.794 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
15.30.828.794 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
15.30.828.795 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
15.30.828.795 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
15.30.828.796 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
15.30.828.797 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
15.30.828.798 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
15.30.828.798 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
15.30.828.799 I srv        update:    - prompt 0x5a83af787180:     540 tokens, checkpoints:  1,   333.602 MiB
15.30.828.799 I srv        update:    - prompt 0x5a83afa46850:     595 tokens, checkpoints:  1,   334.032 MiB
15.30.828.800 I srv        update:    - prompt 0x5a83af78b470:     583 tokens, checkpoints:  1,   333.938 MiB
15.30.828.800 I srv        update:    - prompt 0x5a83af682c20:     927 tokens, checkpoints:  1,   336.630 MiB
15.30.828.800 I srv        update:    - prompt 0x5a83af78aa30:    1208 tokens, checkpoints:  1,   338.828 MiB
15.30.828.802 I srv  get_availabl: prompt cache update took 179.13 ms
15.30.828.906 I slot launch_slot_: id  3 | task 12236 | processing task, is_child = 0
15.30.828.907 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
15.30.828.907 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
15.30.828.921 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
15.30.828.921 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
15.30.828.929 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
15.30.828.929 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
15.30.828.946 I slot update_slots: id  3 | task 12236 | Checking checkpoint with [70, 70] against 74...
15.30.859.781 W slot update_slots: id  3 | task 12236 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
15.35.350.815 I slot print_timing: id  3 | task 12236 | prompt processing, n_tokens =     41, progress = 0.97, t =   4.52 s / 9.07 tokens per second
15.42.168.302 I slot print_timing: id  3 | task 12236 | n_decoded =    100, tg =  15.02 t/s, tg_3s =  15.02 t/s
15.45.181.313 I slot print_timing: id  3 | task 12236 | n_decoded =    145, tg =  15.00 t/s, tg_3s =  14.94 t/s
15.48.244.330 I slot print_timing: id  3 | task 12236 | n_decoded =    191, tg =  15.00 t/s, tg_3s =  15.02 t/s
15.51.288.345 I slot print_timing: id  3 | task 12236 | n_decoded =    237, tg =  15.02 t/s, tg_3s =  15.11 t/s
15.54.354.245 I slot print_timing: id  3 | task 12236 | n_decoded =    283, tg =  15.02 t/s, tg_3s =  15.00 t/s
15.57.374.878 I slot print_timing: id  3 | task 12236 | n_decoded =    328, tg =  15.00 t/s, tg_3s =  14.90 t/s
16.00.433.922 I slot print_timing: id  3 | task 12236 | n_decoded =    374, tg =  15.01 t/s, tg_3s =  15.04 t/s
16.03.443.392 I slot print_timing: id  3 | task 12236 | n_decoded =    419, tg =  15.00 t/s, tg_3s =  14.95 t/s
16.06.455.039 I slot print_timing: id  3 | task 12236 | n_decoded =    464, tg =  15.00 t/s, tg_3s =  14.94 t/s
16.09.468.967 I slot print_timing: id  3 | task 12236 | n_decoded =    509, tg =  14.99 t/s, tg_3s =  14.93 t/s
16.10.143.404 I slot print_timing: id  3 | task 12236 | prompt eval time =    4682.68 ms /    45 tokens (  104.06 ms per token,     9.61 tokens per second)
16.10.143.408 I slot print_timing: id  3 | task 12236 |        eval time =   34631.76 ms /   519 tokens (   66.73 ms per token,    14.99 tokens per second)
16.10.143.409 I slot print_timing: id  3 | task 12236 |       total time =   39314.44 ms /   564 tokens
16.10.143.410 I slot print_timing: id  3 | task 12236 |    graphs reused =      12642
16.10.143.450 I slot      release: id  3 | task 12236 | stop processing: n_tokens = 634, truncated = 0
16.10.143.464 I srv  update_slots: all slots are idle
16.10.172.780 I srv    operator(): Chat format: peg-native
16.10.173.021 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.685 (> 0.100 thold), f_keep = 0.117
16.10.173.023 I srv  get_availabl: updating prompt cache
16.10.173.076 W srv   prompt_save:  - saving prompt with length 634, total state size = 169.649 MiB (draft: 0.000 MiB)
16.10.350.038 I srv          load:  - looking for better prompt, base f_keep = 0.117, sim = 0.685
16.10.350.050 I srv        update:  - cache state: 17 prompts, 5713.796 MiB (limits: 8192.000 MiB, 16384 tokens, 20961 est)
16.10.350.051 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
16.10.350.052 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
16.10.350.052 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
16.10.350.053 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
16.10.350.053 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
16.10.350.054 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
16.10.350.054 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
16.10.350.055 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
16.10.350.055 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
16.10.350.056 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
16.10.350.056 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
16.10.350.056 I srv        update:    - prompt 0x5a83af787180:     540 tokens, checkpoints:  1,   333.602 MiB
16.10.350.057 I srv        update:    - prompt 0x5a83afa46850:     595 tokens, checkpoints:  1,   334.032 MiB
16.10.350.057 I srv        update:    - prompt 0x5a83af78b470:     583 tokens, checkpoints:  1,   333.938 MiB
16.10.350.058 I srv        update:    - prompt 0x5a83af682c20:     927 tokens, checkpoints:  1,   336.630 MiB
16.10.350.058 I srv        update:    - prompt 0x5a83af78aa30:    1208 tokens, checkpoints:  1,   338.828 MiB
16.10.350.059 I srv        update:    - prompt 0x5a83af9ffb00:     634 tokens, checkpoints:  1,   334.337 MiB
16.10.350.060 I srv  get_availabl: prompt cache update took 177.04 ms
16.10.350.159 I slot launch_slot_: id  3 | task 12757 | processing task, is_child = 0
16.10.350.159 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
16.10.350.160 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
16.10.350.173 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
16.10.350.174 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
16.10.350.182 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
16.10.350.182 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
16.10.350.199 I slot update_slots: id  3 | task 12757 | Checking checkpoint with [70, 70] against 74...
16.10.381.225 W slot update_slots: id  3 | task 12757 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
16.14.572.536 I slot print_timing: id  3 | task 12757 | prompt processing, n_tokens =     33, progress = 0.96, t =   4.22 s / 7.82 tokens per second
16.21.283.488 I slot print_timing: id  3 | task 12757 | n_decoded =    100, tg =  15.27 t/s, tg_3s =  15.27 t/s
16.24.318.513 I slot print_timing: id  3 | task 12757 | n_decoded =    146, tg =  15.23 t/s, tg_3s =  15.16 t/s
16.27.336.876 I slot print_timing: id  3 | task 12757 | n_decoded =    192, tg =  15.23 t/s, tg_3s =  15.24 t/s
16.30.385.872 I slot print_timing: id  3 | task 12757 | n_decoded =    239, tg =  15.27 t/s, tg_3s =  15.41 t/s
16.33.420.230 I slot print_timing: id  3 | task 12757 | n_decoded =    286, tg =  15.30 t/s, tg_3s =  15.49 t/s
16.36.450.588 I slot print_timing: id  3 | task 12757 | n_decoded =    332, tg =  15.29 t/s, tg_3s =  15.18 t/s
16.39.500.634 I slot print_timing: id  3 | task 12757 | n_decoded =    378, tg =  15.26 t/s, tg_3s =  15.08 t/s
16.42.511.994 I slot print_timing: id  3 | task 12757 | n_decoded =    423, tg =  15.23 t/s, tg_3s =  14.94 t/s
16.45.557.776 I slot print_timing: id  3 | task 12757 | n_decoded =    469, tg =  15.21 t/s, tg_3s =  15.10 t/s
16.48.597.612 I slot print_timing: id  3 | task 12757 | n_decoded =    515, tg =  15.21 t/s, tg_3s =  15.13 t/s
16.51.658.699 I slot print_timing: id  3 | task 12757 | n_decoded =    561, tg =  15.19 t/s, tg_3s =  15.03 t/s
16.54.714.839 I slot print_timing: id  3 | task 12757 | n_decoded =    607, tg =  15.18 t/s, tg_3s =  15.05 t/s
16.57.743.876 I slot print_timing: id  3 | task 12757 | n_decoded =    652, tg =  15.16 t/s, tg_3s =  14.86 t/s
17.00.785.129 I slot print_timing: id  3 | task 12757 | n_decoded =    697, tg =  15.13 t/s, tg_3s =  14.80 t/s
17.03.795.980 I slot print_timing: id  3 | task 12757 | n_decoded =    742, tg =  15.12 t/s, tg_3s =  14.95 t/s
17.06.851.099 I slot print_timing: id  3 | task 12757 | n_decoded =    788, tg =  15.12 t/s, tg_3s =  15.06 t/s
17.09.916.133 I slot print_timing: id  3 | task 12757 | n_decoded =    834, tg =  15.11 t/s, tg_3s =  15.01 t/s
17.12.921.164 I slot print_timing: id  3 | task 12757 | n_decoded =    879, tg =  15.11 t/s, tg_3s =  14.97 t/s
17.15.943.640 I slot print_timing: id  3 | task 12757 | n_decoded =    924, tg =  15.10 t/s, tg_3s =  14.89 t/s
17.18.958.663 I slot print_timing: id  3 | task 12757 | n_decoded =    969, tg =  15.09 t/s, tg_3s =  14.93 t/s
17.21.971.782 I slot print_timing: id  3 | task 12757 | n_decoded =   1014, tg =  15.08 t/s, tg_3s =  14.93 t/s
17.24.992.798 I slot print_timing: id  3 | task 12757 | n_decoded =   1059, tg =  15.07 t/s, tg_3s =  14.90 t/s
17.28.027.761 I slot print_timing: id  3 | task 12757 | n_decoded =   1104, tg =  15.06 t/s, tg_3s =  14.83 t/s
17.30.595.778 I slot print_timing: id  3 | task 12757 | prompt eval time =    4382.50 ms /    37 tokens (  118.45 ms per token,     8.44 tokens per second)
17.30.595.784 I slot print_timing: id  3 | task 12757 |        eval time =   75863.06 ms /  1142 tokens (   66.43 ms per token,    15.05 tokens per second)
17.30.595.785 I slot print_timing: id  3 | task 12757 |       total time =   80245.56 ms /  1179 tokens
17.30.595.786 I slot print_timing: id  3 | task 12757 |    graphs reused =      13778
17.30.595.837 I slot      release: id  3 | task 12757 | stop processing: n_tokens = 1249, truncated = 0
17.30.595.858 I srv  update_slots: all slots are idle
17.30.625.832 I srv    operator(): Chat format: peg-native
17.30.626.082 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.763 (> 0.100 thold), f_keep = 0.059
17.30.626.084 I srv  get_availabl: updating prompt cache
17.30.626.160 W srv   prompt_save:  - saving prompt with length 1249, total state size = 174.461 MiB (draft: 0.000 MiB)
17.30.803.965 I srv          load:  - looking for better prompt, base f_keep = 0.059, sim = 0.763
17.30.803.977 I srv        update:  - cache state: 18 prompts, 6052.946 MiB (limits: 8192.000 MiB, 16384 tokens, 21476 est)
17.30.803.979 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
17.30.803.979 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
17.30.803.980 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
17.30.803.980 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
17.30.803.981 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
17.30.803.981 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
17.30.803.982 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
17.30.803.982 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
17.30.803.983 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
17.30.803.983 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
17.30.803.984 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
17.30.803.987 I srv        update:    - prompt 0x5a83af787180:     540 tokens, checkpoints:  1,   333.602 MiB
17.30.803.987 I srv        update:    - prompt 0x5a83afa46850:     595 tokens, checkpoints:  1,   334.032 MiB
17.30.803.988 I srv        update:    - prompt 0x5a83af78b470:     583 tokens, checkpoints:  1,   333.938 MiB
17.30.803.988 I srv        update:    - prompt 0x5a83af682c20:     927 tokens, checkpoints:  1,   336.630 MiB
17.30.803.989 I srv        update:    - prompt 0x5a83af78aa30:    1208 tokens, checkpoints:  1,   338.828 MiB
17.30.803.989 I srv        update:    - prompt 0x5a83af9ffb00:     634 tokens, checkpoints:  1,   334.337 MiB
17.30.803.989 I srv        update:    - prompt 0x5a83af81d240:    1249 tokens, checkpoints:  1,   339.149 MiB
17.30.803.991 I srv  get_availabl: prompt cache update took 177.91 ms
17.30.804.092 I slot launch_slot_: id  3 | task 13901 | processing task, is_child = 0
17.30.804.093 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
17.30.804.093 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
17.30.804.108 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
17.30.804.109 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
17.30.804.127 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
17.30.804.127 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
17.30.804.155 I slot update_slots: id  3 | task 13901 | Checking checkpoint with [70, 70] against 74...
17.30.835.024 W slot update_slots: id  3 | task 13901 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
17.38.163.069 I slot print_timing: id  3 | task 13901 | n_decoded =    100, tg =  15.14 t/s, tg_3s =  15.14 t/s
17.41.187.937 I slot print_timing: id  3 | task 13901 | n_decoded =    145, tg =  15.05 t/s, tg_3s =  14.88 t/s
17.44.254.230 I slot print_timing: id  3 | task 13901 | n_decoded =    191, tg =  15.04 t/s, tg_3s =  15.00 t/s
17.47.311.963 I slot print_timing: id  3 | task 13901 | n_decoded =    237, tg =  15.04 t/s, tg_3s =  15.04 t/s
17.50.356.019 I slot print_timing: id  3 | task 13901 | n_decoded =    283, tg =  15.05 t/s, tg_3s =  15.11 t/s
17.53.368.959 I slot print_timing: id  3 | task 13901 | n_decoded =    329, tg =  15.08 t/s, tg_3s =  15.27 t/s
17.56.376.863 I slot print_timing: id  3 | task 13901 | n_decoded =    375, tg =  15.11 t/s, tg_3s =  15.29 t/s
17.59.417.860 I slot print_timing: id  3 | task 13901 | n_decoded =    421, tg =  15.11 t/s, tg_3s =  15.13 t/s
18.02.472.376 I slot print_timing: id  3 | task 13901 | n_decoded =    467, tg =  15.11 t/s, tg_3s =  15.06 t/s
18.05.519.436 I slot print_timing: id  3 | task 13901 | n_decoded =    513, tg =  15.10 t/s, tg_3s =  15.10 t/s
18.08.561.832 I slot print_timing: id  3 | task 13901 | n_decoded =    560, tg =  15.13 t/s, tg_3s =  15.45 t/s
18.11.525.734 I slot print_timing: id  3 | task 13901 | prompt eval time =     752.04 ms /    26 tokens (   28.92 ms per token,    34.57 tokens per second)
18.11.525.740 I slot print_timing: id  3 | task 13901 |        eval time =   39969.52 ms /   606 tokens (   65.96 ms per token,    15.16 tokens per second)
18.11.525.740 I slot print_timing: id  3 | task 13901 |       total time =   40721.57 ms /   632 tokens
18.11.525.741 I slot print_timing: id  3 | task 13901 |    graphs reused =      14380
18.11.525.773 I slot      release: id  3 | task 13901 | stop processing: n_tokens = 702, truncated = 0
18.11.525.791 I srv  update_slots: all slots are idle
18.11.556.946 I srv    operator(): Chat format: peg-native
18.11.557.177 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.622 (> 0.100 thold), f_keep = 0.105
18.11.557.180 I srv  get_availabl: updating prompt cache
18.11.557.237 W srv   prompt_save:  - saving prompt with length 702, total state size = 170.181 MiB (draft: 0.000 MiB)
18.11.737.983 I srv          load:  - looking for better prompt, base f_keep = 0.105, sim = 0.622
18.11.737.995 I srv        update:  - cache state: 19 prompts, 6387.815 MiB (limits: 8192.000 MiB, 16384 tokens, 21251 est)
18.11.737.998 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
18.11.737.999 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
18.11.737.999 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
18.11.738.000 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
18.11.738.000 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
18.11.738.001 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
18.11.738.004 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
18.11.738.004 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
18.11.738.005 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
18.11.738.005 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
18.11.738.006 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
18.11.738.006 I srv        update:    - prompt 0x5a83af787180:     540 tokens, checkpoints:  1,   333.602 MiB
18.11.738.007 I srv        update:    - prompt 0x5a83afa46850:     595 tokens, checkpoints:  1,   334.032 MiB
18.11.738.016 I srv        update:    - prompt 0x5a83af78b470:     583 tokens, checkpoints:  1,   333.938 MiB
18.11.738.017 I srv        update:    - prompt 0x5a83af682c20:     927 tokens, checkpoints:  1,   336.630 MiB
18.11.738.017 I srv        update:    - prompt 0x5a83af78aa30:    1208 tokens, checkpoints:  1,   338.828 MiB
18.11.738.017 I srv        update:    - prompt 0x5a83af9ffb00:     634 tokens, checkpoints:  1,   334.337 MiB
18.11.738.018 I srv        update:    - prompt 0x5a83af81d240:    1249 tokens, checkpoints:  1,   339.149 MiB
18.11.738.018 I srv        update:    - prompt 0x5a83af61b670:     702 tokens, checkpoints:  1,   334.869 MiB
18.11.738.019 I srv  get_availabl: prompt cache update took 180.84 ms
18.11.738.119 I slot launch_slot_: id  3 | task 14509 | processing task, is_child = 0
18.11.738.120 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
18.11.738.121 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
18.11.738.134 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
18.11.738.134 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
18.11.738.142 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
18.11.738.143 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
18.11.738.159 I slot update_slots: id  3 | task 14509 | Checking checkpoint with [70, 70] against 74...
18.11.769.081 W slot update_slots: id  3 | task 14509 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
18.16.515.731 I slot print_timing: id  3 | task 14509 | prompt processing, n_tokens =     44, progress = 0.97, t =   4.78 s / 9.21 tokens per second
18.23.158.963 I slot print_timing: id  3 | task 14509 | n_decoded =    100, tg =  15.41 t/s, tg_3s =  15.41 t/s
18.26.191.427 I slot print_timing: id  3 | task 14509 | n_decoded =    146, tg =  15.34 t/s, tg_3s =  15.17 t/s
18.29.218.458 I slot print_timing: id  3 | task 14509 | n_decoded =    192, tg =  15.30 t/s, tg_3s =  15.20 t/s
18.32.256.757 I slot print_timing: id  3 | task 14509 | n_decoded =    238, tg =  15.27 t/s, tg_3s =  15.14 t/s
18.35.311.770 I slot print_timing: id  3 | task 14509 | n_decoded =    284, tg =  15.24 t/s, tg_3s =  15.06 t/s
18.38.352.786 I slot print_timing: id  3 | task 14509 | n_decoded =    330, tg =  15.22 t/s, tg_3s =  15.13 t/s
18.41.416.991 I slot print_timing: id  3 | task 14509 | n_decoded =    376, tg =  15.19 t/s, tg_3s =  15.01 t/s
18.44.481.957 I slot print_timing: id  3 | task 14509 | n_decoded =    422, tg =  15.17 t/s, tg_3s =  15.01 t/s
18.44.747.993 I slot print_timing: id  3 | task 14509 | prompt eval time =    4933.07 ms /    48 tokens (  102.77 ms per token,     9.73 tokens per second)
18.44.747.997 I slot print_timing: id  3 | task 14509 |        eval time =   28076.75 ms /   426 tokens (   65.91 ms per token,    15.17 tokens per second)
18.44.747.998 I slot print_timing: id  3 | task 14509 |       total time =   33009.82 ms /   474 tokens
18.44.747.999 I slot print_timing: id  3 | task 14509 |    graphs reused =      14802
18.44.748.032 I slot      release: id  3 | task 14509 | stop processing: n_tokens = 544, truncated = 0
18.44.748.050 I srv  update_slots: all slots are idle
18.44.780.119 I srv    operator(): Chat format: peg-native
18.44.780.370 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.692 (> 0.100 thold), f_keep = 0.136
18.44.780.372 I srv  get_availabl: updating prompt cache
18.44.780.416 W srv   prompt_save:  - saving prompt with length 544, total state size = 168.945 MiB (draft: 0.000 MiB)
18.44.954.725 I srv          load:  - looking for better prompt, base f_keep = 0.136, sim = 0.692
18.44.954.737 I srv        update:  - cache state: 20 prompts, 6721.448 MiB (limits: 8192.000 MiB, 16384 tokens, 20859 est)
18.44.954.745 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
18.44.954.746 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
18.44.954.747 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
18.44.954.747 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
18.44.954.748 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
18.44.954.748 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
18.44.954.749 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
18.44.954.749 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
18.44.954.750 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
18.44.954.750 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
18.44.954.750 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
18.44.954.751 I srv        update:    - prompt 0x5a83af787180:     540 tokens, checkpoints:  1,   333.602 MiB
18.44.954.751 I srv        update:    - prompt 0x5a83afa46850:     595 tokens, checkpoints:  1,   334.032 MiB
18.44.954.752 I srv        update:    - prompt 0x5a83af78b470:     583 tokens, checkpoints:  1,   333.938 MiB
18.44.954.752 I srv        update:    - prompt 0x5a83af682c20:     927 tokens, checkpoints:  1,   336.630 MiB
18.44.954.753 I srv        update:    - prompt 0x5a83af78aa30:    1208 tokens, checkpoints:  1,   338.828 MiB
18.44.954.753 I srv        update:    - prompt 0x5a83af9ffb00:     634 tokens, checkpoints:  1,   334.337 MiB
18.44.954.754 I srv        update:    - prompt 0x5a83af81d240:    1249 tokens, checkpoints:  1,   339.149 MiB
18.44.954.754 I srv        update:    - prompt 0x5a83af61b670:     702 tokens, checkpoints:  1,   334.869 MiB
18.44.954.755 I srv        update:    - prompt 0x5a83af96bc00:     544 tokens, checkpoints:  1,   333.633 MiB
18.44.954.756 I srv  get_availabl: prompt cache update took 174.38 ms
18.44.954.859 I slot launch_slot_: id  3 | task 14937 | processing task, is_child = 0
18.44.954.859 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
18.44.954.860 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
18.44.954.875 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
18.44.954.875 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
18.44.954.884 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
18.44.954.884 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
18.44.954.900 I slot update_slots: id  3 | task 14937 | Checking checkpoint with [70, 70] against 74...
18.44.985.786 W slot update_slots: id  3 | task 14937 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
18.49.307.208 I slot print_timing: id  3 | task 14937 | prompt processing, n_tokens =     32, progress = 0.96, t =   4.35 s / 7.35 tokens per second
18.56.121.763 I slot print_timing: id  3 | task 14937 | n_decoded =    100, tg =  15.03 t/s, tg_3s =  15.03 t/s
18.59.130.010 I slot print_timing: id  3 | task 14937 | n_decoded =    145, tg =  15.01 t/s, tg_3s =  14.96 t/s
19.02.145.528 I slot print_timing: id  3 | task 14937 | n_decoded =    190, tg =  14.99 t/s, tg_3s =  14.92 t/s
19.05.158.585 I slot print_timing: id  3 | task 14937 | n_decoded =    235, tg =  14.98 t/s, tg_3s =  14.94 t/s
19.08.188.841 I slot print_timing: id  3 | task 14937 | n_decoded =    280, tg =  14.96 t/s, tg_3s =  14.85 t/s
19.11.200.672 I slot print_timing: id  3 | task 14937 | n_decoded =    325, tg =  14.96 t/s, tg_3s =  14.94 t/s
19.14.266.580 I slot print_timing: id  3 | task 14937 | n_decoded =    371, tg =  14.96 t/s, tg_3s =  15.00 t/s
19.17.288.740 I slot print_timing: id  3 | task 14937 | n_decoded =    416, tg =  14.95 t/s, tg_3s =  14.89 t/s
19.20.313.736 I slot print_timing: id  3 | task 14937 | n_decoded =    461, tg =  14.95 t/s, tg_3s =  14.88 t/s
19.23.323.762 I slot print_timing: id  3 | task 14937 | n_decoded =    506, tg =  14.95 t/s, tg_3s =  14.95 t/s
19.26.377.759 I slot print_timing: id  3 | task 14937 | n_decoded =    552, tg =  14.96 t/s, tg_3s =  15.06 t/s
19.29.441.034 I slot print_timing: id  3 | task 14937 | n_decoded =    598, tg =  14.96 t/s, tg_3s =  15.02 t/s
19.32.473.145 I slot print_timing: id  3 | task 14937 | n_decoded =    644, tg =  14.98 t/s, tg_3s =  15.17 t/s
19.35.503.899 I slot print_timing: id  3 | task 14937 | n_decoded =    690, tg =  14.99 t/s, tg_3s =  15.18 t/s
19.38.555.823 I slot print_timing: id  3 | task 14937 | n_decoded =    736, tg =  14.99 t/s, tg_3s =  15.07 t/s
19.41.586.255 I slot print_timing: id  3 | task 14937 | n_decoded =    782, tg =  15.00 t/s, tg_3s =  15.18 t/s
19.44.613.288 I slot print_timing: id  3 | task 14937 | n_decoded =    828, tg =  15.02 t/s, tg_3s =  15.20 t/s
19.47.637.663 I slot print_timing: id  3 | task 14937 | n_decoded =    874, tg =  15.03 t/s, tg_3s =  15.21 t/s
19.50.690.985 I slot print_timing: id  3 | task 14937 | n_decoded =    920, tg =  15.03 t/s, tg_3s =  15.07 t/s
19.53.704.673 I slot print_timing: id  3 | task 14937 | n_decoded =    966, tg =  15.04 t/s, tg_3s =  15.26 t/s
19.56.738.888 I slot print_timing: id  3 | task 14937 | n_decoded =   1012, tg =  15.04 t/s, tg_3s =  15.16 t/s
19.59.791.351 I slot print_timing: id  3 | task 14937 | n_decoded =   1058, tg =  15.05 t/s, tg_3s =  15.07 t/s
20.02.828.515 I slot print_timing: id  3 | task 14937 | n_decoded =   1104, tg =  15.05 t/s, tg_3s =  15.15 t/s
20.05.869.297 I slot print_timing: id  3 | task 14937 | n_decoded =   1150, tg =  15.05 t/s, tg_3s =  15.13 t/s
20.08.870.618 I slot print_timing: id  3 | task 14937 | n_decoded =   1195, tg =  15.05 t/s, tg_3s =  14.99 t/s
20.11.923.658 I slot print_timing: id  3 | task 14937 | n_decoded =   1241, tg =  15.05 t/s, tg_3s =  15.07 t/s
20.14.970.840 I slot print_timing: id  3 | task 14937 | n_decoded =   1287, tg =  15.05 t/s, tg_3s =  15.10 t/s
20.17.972.356 I slot print_timing: id  3 | task 14937 | n_decoded =   1332, tg =  15.05 t/s, tg_3s =  14.99 t/s
20.20.978.819 I slot print_timing: id  3 | task 14937 | n_decoded =   1377, tg =  15.05 t/s, tg_3s =  14.97 t/s
20.21.583.812 I slot print_timing: id  3 | task 14937 | prompt eval time =    4515.23 ms /    36 tokens (  125.42 ms per token,     7.97 tokens per second)
20.21.583.819 I slot print_timing: id  3 | task 14937 |        eval time =   92113.66 ms /  1386 tokens (   66.46 ms per token,    15.05 tokens per second)
20.21.583.819 I slot print_timing: id  3 | task 14937 |       total time =   96628.89 ms /  1422 tokens
20.21.583.820 I slot print_timing: id  3 | task 14937 |    graphs reused =      16181
20.21.583.857 I slot      release: id  3 | task 14937 | stop processing: n_tokens = 1492, truncated = 0
20.21.583.879 I srv  update_slots: all slots are idle
20.21.613.697 I srv    operator(): Chat format: peg-native
20.21.613.946 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.643 (> 0.100 thold), f_keep = 0.050
20.21.613.948 I srv  get_availabl: updating prompt cache
20.21.614.030 W srv   prompt_save:  - saving prompt with length 1492, total state size = 176.362 MiB (draft: 0.000 MiB)
20.21.794.978 I srv          load:  - looking for better prompt, base f_keep = 0.050, sim = 0.643
20.21.794.991 I srv        update:  - cache state: 21 prompts, 7062.499 MiB (limits: 8192.000 MiB, 16384 tokens, 21582 est)
20.21.794.995 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
20.21.794.995 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
20.21.794.996 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
20.21.794.997 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
20.21.794.997 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
20.21.794.998 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
20.21.794.998 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
20.21.794.999 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
20.21.794.999 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
20.21.794.999 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
20.21.795.000 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
20.21.795.001 I srv        update:    - prompt 0x5a83af787180:     540 tokens, checkpoints:  1,   333.602 MiB
20.21.795.004 I srv        update:    - prompt 0x5a83afa46850:     595 tokens, checkpoints:  1,   334.032 MiB
20.21.795.004 I srv        update:    - prompt 0x5a83af78b470:     583 tokens, checkpoints:  1,   333.938 MiB
20.21.795.005 I srv        update:    - prompt 0x5a83af682c20:     927 tokens, checkpoints:  1,   336.630 MiB
20.21.795.005 I srv        update:    - prompt 0x5a83af78aa30:    1208 tokens, checkpoints:  1,   338.828 MiB
20.21.795.006 I srv        update:    - prompt 0x5a83af9ffb00:     634 tokens, checkpoints:  1,   334.337 MiB
20.21.795.006 I srv        update:    - prompt 0x5a83af81d240:    1249 tokens, checkpoints:  1,   339.149 MiB
20.21.795.007 I srv        update:    - prompt 0x5a83af61b670:     702 tokens, checkpoints:  1,   334.869 MiB
20.21.795.007 I srv        update:    - prompt 0x5a83af96bc00:     544 tokens, checkpoints:  1,   333.633 MiB
20.21.795.008 I srv        update:    - prompt 0x5a83af791cd0:    1492 tokens, checkpoints:  1,   341.050 MiB
20.21.795.009 I srv  get_availabl: prompt cache update took 181.06 ms
20.21.795.110 I slot launch_slot_: id  3 | task 16325 | processing task, is_child = 0
20.21.795.111 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
20.21.795.112 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
20.21.795.127 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
20.21.795.127 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
20.21.795.136 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
20.21.795.136 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
20.21.795.152 I slot update_slots: id  3 | task 16325 | Checking checkpoint with [70, 70] against 74...
20.21.825.981 W slot update_slots: id  3 | task 16325 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
20.26.134.780 I slot print_timing: id  3 | task 16325 | prompt processing, n_tokens =     40, progress = 0.97, t =   4.34 s / 9.22 tokens per second
20.32.928.876 I slot print_timing: id  3 | task 16325 | n_decoded =    100, tg =  15.08 t/s, tg_3s =  15.08 t/s
20.35.936.384 I slot print_timing: id  3 | task 16325 | n_decoded =    145, tg =  15.04 t/s, tg_3s =  14.96 t/s
20.38.940.634 I slot print_timing: id  3 | task 16325 | n_decoded =    190, tg =  15.03 t/s, tg_3s =  14.98 t/s
20.42.003.492 I slot print_timing: id  3 | task 16325 | n_decoded =    236, tg =  15.03 t/s, tg_3s =  15.02 t/s
20.45.012.775 I slot print_timing: id  3 | task 16325 | n_decoded =    281, tg =  15.01 t/s, tg_3s =  14.95 t/s
20.48.067.678 I slot print_timing: id  3 | task 16325 | n_decoded =    327, tg =  15.02 t/s, tg_3s =  15.06 t/s
20.51.086.464 I slot print_timing: id  3 | task 16325 | n_decoded =    372, tg =  15.01 t/s, tg_3s =  14.91 t/s
20.54.088.852 I slot print_timing: id  3 | task 16325 | n_decoded =    417, tg =  15.00 t/s, tg_3s =  14.99 t/s
20.57.137.768 I slot print_timing: id  3 | task 16325 | n_decoded =    463, tg =  15.01 t/s, tg_3s =  15.09 t/s
21.00.184.769 I slot print_timing: id  3 | task 16325 | n_decoded =    509, tg =  15.02 t/s, tg_3s =  15.10 t/s
21.03.242.868 I slot print_timing: id  3 | task 16325 | n_decoded =    555, tg =  15.02 t/s, tg_3s =  15.04 t/s
21.06.265.536 I slot print_timing: id  3 | task 16325 | n_decoded =    600, tg =  15.01 t/s, tg_3s =  14.89 t/s
21.09.322.658 I slot print_timing: id  3 | task 16325 | n_decoded =    646, tg =  15.01 t/s, tg_3s =  15.05 t/s
21.12.366.015 I slot print_timing: id  3 | task 16325 | prompt eval time =    4501.53 ms /    44 tokens (  102.31 ms per token,     9.77 tokens per second)
21.12.366.023 I slot print_timing: id  3 | task 16325 |        eval time =   46069.32 ms /   692 tokens (   66.57 ms per token,    15.02 tokens per second)
21.12.366.024 I slot print_timing: id  3 | task 16325 |       total time =   50570.85 ms /   736 tokens
21.12.366.024 I slot print_timing: id  3 | task 16325 |    graphs reused =      16868
21.12.366.060 I slot      release: id  3 | task 16325 | stop processing: n_tokens = 806, truncated = 0
21.12.366.076 I srv  update_slots: all slots are idle
21.12.397.295 I srv    operator(): Chat format: peg-native
21.12.397.531 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.612 (> 0.100 thold), f_keep = 0.092
21.12.397.533 I srv  get_availabl: updating prompt cache
21.12.397.577 W srv   prompt_save:  - saving prompt with length 806, total state size = 170.995 MiB (draft: 0.000 MiB)
21.12.574.473 I srv          load:  - looking for better prompt, base f_keep = 0.092, sim = 0.612
21.12.574.487 I srv        update:  - cache state: 22 prompts, 7398.182 MiB (limits: 8192.000 MiB, 16384 tokens, 21495 est)
21.12.574.488 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
21.12.574.489 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
21.12.574.490 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
21.12.574.490 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
21.12.574.491 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
21.12.574.492 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
21.12.574.492 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
21.12.574.493 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
21.12.574.493 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
21.12.574.494 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
21.12.574.495 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
21.12.574.495 I srv        update:    - prompt 0x5a83af787180:     540 tokens, checkpoints:  1,   333.602 MiB
21.12.574.496 I srv        update:    - prompt 0x5a83afa46850:     595 tokens, checkpoints:  1,   334.032 MiB
21.12.574.496 I srv        update:    - prompt 0x5a83af78b470:     583 tokens, checkpoints:  1,   333.938 MiB
21.12.574.497 I srv        update:    - prompt 0x5a83af682c20:     927 tokens, checkpoints:  1,   336.630 MiB
21.12.574.498 I srv        update:    - prompt 0x5a83af78aa30:    1208 tokens, checkpoints:  1,   338.828 MiB
21.12.574.498 I srv        update:    - prompt 0x5a83af9ffb00:     634 tokens, checkpoints:  1,   334.337 MiB
21.12.574.499 I srv        update:    - prompt 0x5a83af81d240:    1249 tokens, checkpoints:  1,   339.149 MiB
21.12.574.499 I srv        update:    - prompt 0x5a83af61b670:     702 tokens, checkpoints:  1,   334.869 MiB
21.12.574.500 I srv        update:    - prompt 0x5a83af96bc00:     544 tokens, checkpoints:  1,   333.633 MiB
21.12.574.500 I srv        update:    - prompt 0x5a83af791cd0:    1492 tokens, checkpoints:  1,   341.050 MiB
21.12.574.504 I srv        update:    - prompt 0x5a83afa087f0:     806 tokens, checkpoints:  1,   335.683 MiB
21.12.574.505 I srv  get_availabl: prompt cache update took 176.97 ms
21.12.574.610 I slot launch_slot_: id  3 | task 17019 | processing task, is_child = 0
21.12.574.611 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
21.12.574.612 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
21.12.574.628 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
21.12.574.628 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
21.12.574.638 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
21.12.574.638 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
21.12.574.658 I slot update_slots: id  3 | task 17019 | Checking checkpoint with [70, 70] against 74...
21.12.605.699 W slot update_slots: id  3 | task 17019 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
21.17.456.149 I slot print_timing: id  3 | task 17019 | prompt processing, n_tokens =     46, progress = 0.97, t =   4.88 s / 9.42 tokens per second
21.24.126.672 I slot print_timing: id  3 | task 17019 | n_decoded =    100, tg =  15.37 t/s, tg_3s =  15.37 t/s
21.27.143.519 I slot print_timing: id  3 | task 17019 | n_decoded =    146, tg =  15.33 t/s, tg_3s =  15.25 t/s
21.30.177.610 I slot print_timing: id  3 | task 17019 | n_decoded =    193, tg =  15.37 t/s, tg_3s =  15.49 t/s
21.33.216.493 I slot print_timing: id  3 | task 17019 | n_decoded =    240, tg =  15.39 t/s, tg_3s =  15.47 t/s
21.36.235.329 I slot print_timing: id  3 | task 17019 | n_decoded =    286, tg =  15.36 t/s, tg_3s =  15.24 t/s
21.37.689.195 I slot print_timing: id  3 | task 17019 | prompt eval time =    5044.29 ms /    50 tokens (  100.89 ms per token,     9.91 tokens per second)
21.37.689.200 I slot print_timing: id  3 | task 17019 |        eval time =   20070.24 ms /   308 tokens (   65.16 ms per token,    15.35 tokens per second)
21.37.689.201 I slot print_timing: id  3 | task 17019 |       total time =   25114.52 ms /   358 tokens
21.37.689.201 I slot print_timing: id  3 | task 17019 |    graphs reused =      17173
21.37.689.236 I slot      release: id  3 | task 17019 | stop processing: n_tokens = 428, truncated = 0
21.37.689.252 I srv  update_slots: all slots are idle
21.37.719.585 I srv    operator(): Chat format: peg-native
21.37.719.830 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.698 (> 0.100 thold), f_keep = 0.173
21.37.719.832 I srv  get_availabl: updating prompt cache
21.37.719.867 W srv   prompt_save:  - saving prompt with length 428, total state size = 168.037 MiB (draft: 0.000 MiB)
21.37.909.408 I srv          load:  - looking for better prompt, base f_keep = 0.173, sim = 0.698
21.37.909.423 I srv        update:  - cache state: 23 prompts, 7730.908 MiB (limits: 8192.000 MiB, 16384 tokens, 21024 est)
21.37.909.424 I srv        update:    - prompt 0x5a83afb27b00:    1453 tokens, checkpoints:  1,   340.745 MiB
21.37.909.425 I srv        update:    - prompt 0x5a83af6d4800:     570 tokens, checkpoints:  1,   333.837 MiB
21.37.909.425 I srv        update:    - prompt 0x5a83af82c720:     398 tokens, checkpoints:  1,   332.491 MiB
21.37.909.426 I srv        update:    - prompt 0x5a83af814110:     876 tokens, checkpoints:  1,   336.231 MiB
21.37.909.427 I srv        update:    - prompt 0x5a83af739070:    1784 tokens, checkpoints:  1,   343.335 MiB
21.37.909.427 I srv        update:    - prompt 0x5a83b70e3f20:    1500 tokens, checkpoints:  1,   341.113 MiB
21.37.909.428 I srv        update:    - prompt 0x5a83af72ee20:     708 tokens, checkpoints:  1,   334.916 MiB
21.37.909.428 I srv        update:    - prompt 0x5a83af5dba30:     705 tokens, checkpoints:  1,   334.893 MiB
21.37.909.429 I srv        update:    - prompt 0x5a83afa99610:     704 tokens, checkpoints:  1,   334.885 MiB
21.37.909.429 I srv        update:    - prompt 0x5a83af9578e0:     703 tokens, checkpoints:  1,   334.877 MiB
21.37.909.430 I srv        update:    - prompt 0x5a83afb34d80:     732 tokens, checkpoints:  1,   335.104 MiB
21.37.909.430 I srv        update:    - prompt 0x5a83af787180:     540 tokens, checkpoints:  1,   333.602 MiB
21.37.909.431 I srv        update:    - prompt 0x5a83afa46850:     595 tokens, checkpoints:  1,   334.032 MiB
21.37.909.431 I srv        update:    - prompt 0x5a83af78b470:     583 tokens, checkpoints:  1,   333.938 MiB
21.37.909.432 I srv        update:    - prompt 0x5a83af682c20:     927 tokens, checkpoints:  1,   336.630 MiB
21.37.909.432 I srv        update:    - prompt 0x5a83af78aa30:    1208 tokens, checkpoints:  1,   338.828 MiB
21.37.909.433 I srv        update:    - prompt 0x5a83af9ffb00:     634 tokens, checkpoints:  1,   334.337 MiB
21.37.909.433 I srv        update:    - prompt 0x5a83af81d240:    1249 tokens, checkpoints:  1,   339.149 MiB
21.37.909.434 I srv        update:    - prompt 0x5a83af61b670:     702 tokens, checkpoints:  1,   334.869 MiB
21.37.909.434 I srv        update:    - prompt 0x5a83af96bc00:     544 tokens, checkpoints:  1,   333.633 MiB
21.37.909.435 I srv        update:    - prompt 0x5a83af791cd0:    1492 tokens, checkpoints:  1,   341.050 MiB
21.37.909.435 I srv        update:    - prompt 0x5a83afa087f0:     806 tokens, checkpoints:  1,   335.683 MiB
21.37.909.436 I srv        update:    - prompt 0x5a83af8c2ea0:     428 tokens, checkpoints:  1,   332.726 MiB
21.37.909.437 I srv  get_availabl: prompt cache update took 189.60 ms
21.37.909.532 I slot launch_slot_: id  3 | task 17329 | processing task, is_child = 0
21.37.909.533 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
21.37.909.534 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
21.37.909.546 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
21.37.909.547 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
21.37.909.555 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
21.37.909.556 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
21.37.909.573 I slot update_slots: id  3 | task 17329 | Checking checkpoint with [70, 70] against 74...
21.37.940.369 W slot update_slots: id  3 | task 17329 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 164.688 MiB)
21.45.432.146 I slot print_timing: id  3 | task 17329 | n_decoded =    100, tg =  15.17 t/s, tg_3s =  15.17 t/s
21.48.466.477 I slot print_timing: id  3 | task 17329 | n_decoded =    146, tg =  15.17 t/s, tg_3s =  15.16 t/s
21.51.481.348 I slot print_timing: id  3 | task 17329 | n_decoded =    191, tg =  15.11 t/s, tg_3s =  14.93 t/s
21.54.518.799 I slot print_timing: id  3 | task 17329 | n_decoded =    237, tg =  15.12 t/s, tg_3s =  15.14 t/s
21.57.569.485 I slot print_timing: id  3 | task 17329 | n_decoded =    283, tg =  15.11 t/s, tg_3s =  15.08 t/s
22.00.612.493 I slot print_timing: id  3 | task 17329 | n_decoded =    329, tg =  15.11 t/s, tg_3s =  15.12 t/s
22.03.621.698 I slot print_timing: id  3 | task 17329 | n_decoded =    374, tg =  15.09 t/s, tg_3s =  14.95 t/s
22.06.678.321 I slot print_timing: id  3 | task 17329 | n_decoded =    420, tg =  15.09 t/s, tg_3s =  15.05 t/s
22.09.728.705 I slot print_timing: id  3 | task 17329 | n_decoded =    466, tg =  15.09 t/s, tg_3s =  15.08 t/s
22.12.744.743 I slot print_timing: id  3 | task 17329 | n_decoded =    511, tg =  15.07 t/s, tg_3s =  14.92 t/s
22.15.749.558 I slot print_timing: id  3 | task 17329 | n_decoded =    556, tg =  15.06 t/s, tg_3s =  14.98 t/s
22.18.772.653 I slot print_timing: id  3 | task 17329 | n_decoded =    601, tg =  15.05 t/s, tg_3s =  14.89 t/s
22.21.777.501 I slot print_timing: id  3 | task 17329 | n_decoded =    646, tg =  15.05 t/s, tg_3s =  14.98 t/s
22.24.835.558 I slot print_timing: id  3 | task 17329 | n_decoded =    691, tg =  15.02 t/s, tg_3s =  14.72 t/s
22.27.851.826 I slot print_timing: id  3 | task 17329 | n_decoded =    736, tg =  15.02 t/s, tg_3s =  14.92 t/s
22.30.900.793 I slot print_timing: id  3 | task 17329 | n_decoded =    782, tg =  15.02 t/s, tg_3s =  15.09 t/s
22.33.963.430 I slot print_timing: id  3 | task 17329 | n_decoded =    828, tg =  15.02 t/s, tg_3s =  15.02 t/s
22.36.968.656 I slot print_timing: id  3 | task 17329 | n_decoded =    873, tg =  15.02 t/s, tg_3s =  14.97 t/s
22.40.008.458 I slot print_timing: id  3 | task 17329 | n_decoded =    918, tg =  15.01 t/s, tg_3s =  14.80 t/s
22.43.037.694 I slot print_timing: id  3 | task 17329 | n_decoded =    963, tg =  15.00 t/s, tg_3s =  14.86 t/s
22.46.096.694 I slot print_timing: id  3 | task 17329 | n_decoded =   1009, tg =  15.00 t/s, tg_3s =  15.04 t/s
22.49.148.429 I slot print_timing: id  3 | task 17329 | n_decoded =   1055, tg =  15.01 t/s, tg_3s =  15.07 t/s
22.52.193.395 I slot print_timing: id  3 | task 17329 | n_decoded =   1101, tg =  15.01 t/s, tg_3s =  15.11 t/s
22.55.222.684 I slot print_timing: id  3 | task 17329 | n_decoded =   1147, tg =  15.02 t/s, tg_3s =  15.19 t/s
22.58.271.575 I slot print_timing: id  3 | task 17329 | n_decoded =   1193, tg =  15.02 t/s, tg_3s =  15.09 t/s
23.01.295.536 I slot print_timing: id  3 | task 17329 | n_decoded =   1239, tg =  15.03 t/s, tg_3s =  15.21 t/s
23.04.328.555 I slot print_timing: id  3 | task 17329 | n_decoded =   1285, tg =  15.03 t/s, tg_3s =  15.17 t/s
23.04.394.592 I slot print_timing: id  3 | task 17329 | prompt eval time =     931.88 ms /    35 tokens (   26.63 ms per token,    37.56 tokens per second)
23.04.394.597 I slot print_timing: id  3 | task 17329 |        eval time =   85553.13 ms /  1286 tokens (   66.53 ms per token,    15.03 tokens per second)
23.04.394.598 I slot print_timing: id  3 | task 17329 |       total time =   86485.00 ms /  1321 tokens
23.04.394.599 I slot print_timing: id  3 | task 17329 |    graphs reused =      18452
23.04.394.636 I slot      release: id  3 | task 17329 | stop processing: n_tokens = 1391, truncated = 0
23.04.394.656 I srv  update_slots: all slots are idle
23.04.395.352 I srv    operator(): operator(): cleaning up before exit...