0.00.380.854 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.380.858 I device_info:
0.00.380.880 I   - ROCm0   : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.380.884 I   - ROCm1   : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.380.892 I   - CPU     : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.380.953 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
0.00.380.956 I srv  llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.381.142 I srv          init: running without SSL
0.00.381.206 I srv          init: using 23 threads for HTTP server
0.00.381.531 I srv         start: binding port with default address family
0.00.382.730 I srv  llama_server: loading model
0.00.382.737 I srv    load_model: loading model '/home/qrf/.cache/llama.cpp/nvidia_Nemotron-3-Super-120B-A12B-Q4_K_M/nvidia_Nemotron-3-Super-120B-A12B-Q4_K_M-00001-of-00003.gguf'
0.00.382.791 I common_init_result: fitting params to device memory ...
0.00.382.793 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.08.325.060 W llama_model_loader: tensor overrides to CPU are used with mmap enabled - consider using --no-mmap for better performance
0.59.260.030 W llama_context: n_ctx_seq (24576) < n_ctx_train (1048576) -- the full capacity of the model will not be utilized
0.59.447.338 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.59.750.927 I srv    load_model: initializing slots, n_slots = 4
0.59.915.406 W srv    load_model: speculative decoding will use checkpoints
0.59.915.420 W common_speculative_init: no implementations specified for speculative decoding
0.59.915.422 I slot   load_model: id  0 | task -1 | new slot, n_ctx = 24576
0.59.915.426 I slot   load_model: id  1 | task -1 | new slot, n_ctx = 24576
0.59.915.427 I slot   load_model: id  2 | task -1 | new slot, n_ctx = 24576
0.59.915.452 I slot   load_model: id  3 | task -1 | new slot, n_ctx = 24576
0.59.915.535 I srv    load_model: prompt cache is enabled, size limit: 8192 MiB
0.59.915.537 I srv    load_model: use `--cache-ram 0` to disable the prompt cache
0.59.915.537 I srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.59.915.537 I srv    load_model: context checkpoints enabled, max = 32, min spacing = 256
0.59.916.438 I srv          init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.59.954.687 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
<think></think>Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
<think>
'
0.59.984.342 I srv          init: init: chat template, thinking = 1
0.59.984.394 I srv  llama_server: model loaded
0.59.984.399 I srv  llama_server: server is listening on http://127.0.0.1:8090
0.59.984.407 I srv  update_slots: all slots are idle
1.04.963.635 I srv    operator(): Chat format: peg-native
1.04.965.729 I slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
1.04.965.731 I srv  get_availabl: updating prompt cache
1.04.965.736 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
1.04.965.755 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 24576 tokens, 8589934592 est)
1.04.965.757 I srv  get_availabl: prompt cache update took 0.03 ms
1.04.965.944 I slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
1.04.965.946 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.04.965.947 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
1.04.965.985 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
1.04.965.985 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
1.04.965.999 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
1.04.965.999 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
1.10.879.626 I slot print_timing: id  3 | task 0 | prompt processing, n_tokens =     42, progress = 0.61, t =   5.91 s / 7.10 tokens per second
1.11.060.030 I slot create_check: id  3 | task 0 | created context checkpoint 1 of 32 (pos_min = 41, pos_max = 41, n_tokens = 42, size = 164.688 MiB)
1.11.649.014 I slot print_timing: id  3 | task 0 | prompt processing, n_tokens =     65, progress = 0.94, t =   6.68 s / 9.73 tokens per second
1.18.410.969 I slot print_timing: id  3 | task 0 | n_decoded =    100, tg =  15.16 t/s, tg_3s =  15.16 t/s
1.20.460.675 I slot print_timing: id  3 | task 0 | prompt eval time =    6850.01 ms /    69 tokens (   99.28 ms per token,    10.07 tokens per second)
1.20.460.683 I slot print_timing: id  3 | task 0 |        eval time =    8644.58 ms /   131 tokens (   65.99 ms per token,    15.15 tokens per second)
1.20.460.690 I slot print_timing: id  3 | task 0 |       total time =   15494.59 ms /   200 tokens
1.20.460.695 I slot print_timing: id  3 | task 0 |    graphs reused =        130
1.20.461.168 I slot      release: id  3 | task 0 | stop processing: n_tokens = 199, truncated = 0
1.20.461.180 I srv  update_slots: all slots are idle
1.20.493.438 I srv    operator(): Chat format: peg-native
1.20.493.668 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.625 (> 0.100 thold), f_keep = 0.226
1.20.493.670 I srv  get_availabl: updating prompt cache
1.20.493.706 W srv   prompt_save:  - saving prompt with length 199, total state size = 166.246 MiB (draft: 0.000 MiB)
1.20.673.790 I srv          load:  - looking for better prompt, base f_keep = 0.226, sim = 0.625
1.20.673.797 I srv        update:  - cache state: 1 prompts, 330.934 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
1.20.673.798 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
1.20.673.799 I srv  get_availabl: prompt cache update took 180.13 ms
1.20.673.907 I slot launch_slot_: id  3 | task 134 | processing task, is_child = 0
1.20.673.909 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.20.673.910 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
1.20.673.926 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
1.20.673.937 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
1.20.673.949 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
1.20.673.950 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
1.20.673.969 I slot update_slots: id  3 | task 134 | Checking checkpoint with [41, 41] against 45...
1.20.704.871 W slot update_slots: id  3 | task 134 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 164.688 MiB)
1.23.407.585 I slot print_timing: id  3 | task 134 | prompt eval time =     853.27 ms /    30 tokens (   28.44 ms per token,    35.16 tokens per second)
1.23.407.589 I slot print_timing: id  3 | task 134 |        eval time =    1880.33 ms /    29 tokens (   64.84 ms per token,    15.42 tokens per second)
1.23.407.590 I slot print_timing: id  3 | task 134 |       total time =    2733.60 ms /    59 tokens
1.23.407.590 I slot print_timing: id  3 | task 134 |    graphs reused =        157
1.23.407.756 I slot      release: id  3 | task 134 | stop processing: n_tokens = 100, truncated = 0
1.23.407.771 I srv  update_slots: all slots are idle
1.23.441.593 I srv    operator(): Chat format: peg-native
1.23.441.910 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.489 (> 0.100 thold), f_keep = 0.450
1.23.441.913 I srv  get_availabl: updating prompt cache
1.23.441.943 W srv   prompt_save:  - saving prompt with length 100, total state size = 165.471 MiB (draft: 0.000 MiB)
1.23.653.173 I srv          load:  - looking for better prompt, base f_keep = 0.450, sim = 0.489
1.23.653.180 I srv        update:  - cache state: 2 prompts, 661.094 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
1.23.653.181 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
1.23.653.181 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
1.23.653.183 I srv  get_availabl: prompt cache update took 211.27 ms
1.23.653.274 I slot launch_slot_: id  3 | task 165 | processing task, is_child = 0
1.23.653.276 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.23.653.276 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
1.23.653.291 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
1.23.653.291 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
1.23.653.303 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
1.23.653.303 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
1.23.653.323 I slot update_slots: id  3 | task 165 | Checking checkpoint with [41, 41] against 45...
1.23.684.189 W slot update_slots: id  3 | task 165 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 164.688 MiB)
1.28.890.820 I slot print_timing: id  3 | task 165 | prompt processing, n_tokens =     46, progress = 0.96, t =   5.24 s / 8.78 tokens per second
1.35.626.135 I slot print_timing: id  3 | task 165 | n_decoded =    100, tg =  15.21 t/s, tg_3s =  15.21 t/s
1.38.659.121 I slot print_timing: id  3 | task 165 | n_decoded =    146, tg =  15.20 t/s, tg_3s =  15.17 t/s
1.41.705.738 I slot print_timing: id  3 | task 165 | n_decoded =    192, tg =  15.17 t/s, tg_3s =  15.10 t/s
1.44.741.795 I slot print_timing: id  3 | task 165 | n_decoded =    238, tg =  15.17 t/s, tg_3s =  15.15 t/s
1.47.773.201 I slot print_timing: id  3 | task 165 | n_decoded =    284, tg =  15.17 t/s, tg_3s =  15.17 t/s
1.50.797.127 I slot print_timing: id  3 | task 165 | n_decoded =    330, tg =  15.18 t/s, tg_3s =  15.21 t/s
1.51.782.263 I slot print_timing: id  3 | task 165 | prompt eval time =    5397.73 ms /    50 tokens (  107.95 ms per token,     9.26 tokens per second)
1.51.782.269 I slot print_timing: id  3 | task 165 |        eval time =   22731.19 ms /   345 tokens (   65.89 ms per token,    15.18 tokens per second)
1.51.782.270 I slot print_timing: id  3 | task 165 |       total time =   28128.93 ms /   395 tokens
1.51.782.270 I slot print_timing: id  3 | task 165 |    graphs reused =        499
1.51.782.301 I slot      release: id  3 | task 165 | stop processing: n_tokens = 436, truncated = 0
1.51.782.316 I srv  update_slots: all slots are idle
1.51.816.305 I srv    operator(): Chat format: peg-native
1.51.816.530 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.608 (> 0.100 thold), f_keep = 0.103
1.51.816.532 I srv  get_availabl: updating prompt cache
1.51.816.591 W srv   prompt_save:  - saving prompt with length 436, total state size = 168.100 MiB (draft: 0.000 MiB)
1.52.010.503 I srv          load:  - looking for better prompt, base f_keep = 0.103, sim = 0.608
1.52.010.511 I srv        update:  - cache state: 3 prompts, 993.882 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
1.52.010.512 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
1.52.010.513 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
1.52.010.513 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
1.52.010.514 I srv  get_availabl: prompt cache update took 193.98 ms
1.52.010.607 I slot launch_slot_: id  3 | task 512 | processing task, is_child = 0
1.52.010.609 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.52.010.609 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
1.52.010.624 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
1.52.010.624 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
1.52.010.636 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
1.52.010.636 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
1.52.010.656 I slot update_slots: id  3 | task 512 | Checking checkpoint with [41, 41] against 45...
1.52.041.541 W slot update_slots: id  3 | task 512 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 164.688 MiB)
1.59.483.831 I slot print_timing: id  3 | task 512 | n_decoded =    100, tg =  15.19 t/s, tg_3s =  15.19 t/s
2.02.496.976 I slot print_timing: id  3 | task 512 | n_decoded =    146, tg =  15.22 t/s, tg_3s =  15.27 t/s
2.05.500.419 I slot print_timing: id  3 | task 512 | n_decoded =    192, tg =  15.24 t/s, tg_3s =  15.32 t/s
2.08.554.964 I slot print_timing: id  3 | task 512 | n_decoded =    239, tg =  15.27 t/s, tg_3s =  15.39 t/s
2.11.573.725 I slot print_timing: id  3 | task 512 | n_decoded =    286, tg =  15.32 t/s, tg_3s =  15.57 t/s
2.13.506.747 I slot print_timing: id  3 | task 512 | prompt eval time =     890.54 ms /    32 tokens (   27.83 ms per token,    35.93 tokens per second)
2.13.506.801 I slot print_timing: id  3 | task 512 |        eval time =   20605.54 ms /   316 tokens (   65.21 ms per token,    15.34 tokens per second)
2.13.506.803 I slot print_timing: id  3 | task 512 |       total time =   21496.08 ms /   348 tokens
2.13.506.804 I slot print_timing: id  3 | task 512 |    graphs reused =        812
2.13.506.835 I slot      release: id  3 | task 512 | stop processing: n_tokens = 389, truncated = 0
2.13.506.854 I srv  update_slots: all slots are idle
2.13.539.729 I srv    operator(): Chat format: peg-native
2.13.539.966 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.643 (> 0.100 thold), f_keep = 0.116
2.13.539.969 I srv  get_availabl: updating prompt cache
2.13.540.035 W srv   prompt_save:  - saving prompt with length 389, total state size = 167.732 MiB (draft: 0.000 MiB)
2.13.752.022 I srv          load:  - looking for better prompt, base f_keep = 0.116, sim = 0.643
2.13.752.029 I srv        update:  - cache state: 4 prompts, 1326.302 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
2.13.752.030 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
2.13.752.031 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
2.13.752.032 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
2.13.752.032 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
2.13.752.033 I srv  get_availabl: prompt cache update took 212.06 ms
2.13.752.137 I slot launch_slot_: id  3 | task 830 | processing task, is_child = 0
2.13.752.140 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
2.13.752.141 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
2.13.752.157 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
2.13.752.157 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
2.13.752.170 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
2.13.752.170 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
2.13.752.213 I slot update_slots: id  3 | task 830 | Checking checkpoint with [41, 41] against 45...
2.13.782.960 W slot update_slots: id  3 | task 830 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 164.688 MiB)
2.19.962.584 I slot print_timing: id  3 | task 830 | prompt eval time =     774.02 ms /    28 tokens (   27.64 ms per token,    36.17 tokens per second)
2.19.962.591 I slot print_timing: id  3 | task 830 |        eval time =    5436.36 ms /    83 tokens (   65.50 ms per token,    15.27 tokens per second)
2.19.962.592 I slot print_timing: id  3 | task 830 |       total time =    6210.38 ms /   111 tokens
2.19.962.593 I slot print_timing: id  3 | task 830 |    graphs reused =        893
2.19.962.621 I slot      release: id  3 | task 830 | stop processing: n_tokens = 152, truncated = 0
2.19.962.639 I srv  update_slots: all slots are idle
2.19.997.149 I srv    operator(): Chat format: peg-native
2.19.997.372 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.500 (> 0.100 thold), f_keep = 0.296
2.19.997.374 I srv  get_availabl: updating prompt cache
2.19.997.408 W srv   prompt_save:  - saving prompt with length 152, total state size = 165.878 MiB (draft: 0.000 MiB)
2.20.184.660 I srv          load:  - looking for better prompt, base f_keep = 0.296, sim = 0.500
2.20.184.667 I srv        update:  - cache state: 5 prompts, 1656.869 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
2.20.184.668 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
2.20.184.687 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
2.20.184.689 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
2.20.184.689 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
2.20.184.690 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
2.20.184.691 I srv  get_availabl: prompt cache update took 187.32 ms
2.20.184.780 I slot launch_slot_: id  3 | task 915 | processing task, is_child = 0
2.20.184.782 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
2.20.184.783 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
2.20.184.799 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
2.20.184.800 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
2.20.184.811 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
2.20.184.811 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
2.20.184.830 I slot update_slots: id  3 | task 915 | Checking checkpoint with [41, 41] against 45...
2.20.215.616 W slot update_slots: id  3 | task 915 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 164.688 MiB)
2.25.557.071 I slot print_timing: id  3 | task 915 | prompt processing, n_tokens =     44, progress = 0.96, t =   5.37 s / 8.19 tokens per second
2.32.267.746 I slot print_timing: id  3 | task 915 | n_decoded =    100, tg =  15.26 t/s, tg_3s =  15.26 t/s
2.33.719.676 I slot print_timing: id  3 | task 915 | prompt eval time =    5530.84 ms /    48 tokens (  115.23 ms per token,     8.68 tokens per second)
2.33.719.684 I slot print_timing: id  3 | task 915 |        eval time =    8003.99 ms /   122 tokens (   65.61 ms per token,    15.24 tokens per second)
2.33.719.685 I slot print_timing: id  3 | task 915 |       total time =   13534.83 ms /   170 tokens
2.33.719.686 I slot print_timing: id  3 | task 915 |    graphs reused =       1013
2.33.719.837 I slot      release: id  3 | task 915 | stop processing: n_tokens = 211, truncated = 0
2.33.719.857 I srv  update_slots: all slots are idle
2.33.752.531 I srv    operator(): Chat format: peg-native
2.33.752.793 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.523 (> 0.100 thold), f_keep = 0.213
2.33.752.795 I srv  get_availabl: updating prompt cache
2.33.752.829 W srv   prompt_save:  - saving prompt with length 211, total state size = 166.339 MiB (draft: 0.000 MiB)
2.33.930.921 I srv          load:  - looking for better prompt, base f_keep = 0.213, sim = 0.523
2.33.930.929 I srv        update:  - cache state: 6 prompts, 1987.897 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
2.33.930.930 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
2.33.930.930 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
2.33.930.953 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
2.33.930.955 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
2.33.930.955 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
2.33.930.956 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
2.33.930.957 I srv  get_availabl: prompt cache update took 178.16 ms
2.33.931.040 I slot launch_slot_: id  3 | task 1039 | processing task, is_child = 0
2.33.931.041 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
2.33.931.041 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
2.33.931.057 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
2.33.931.057 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
2.33.931.068 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
2.33.931.069 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
2.33.931.087 I slot update_slots: id  3 | task 1039 | Checking checkpoint with [41, 41] against 45...
2.33.961.866 W slot update_slots: id  3 | task 1039 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 164.688 MiB)
2.39.131.266 I slot print_timing: id  3 | task 1039 | prompt processing, n_tokens =     40, progress = 0.95, t =   5.20 s / 7.69 tokens per second
2.45.863.956 I slot print_timing: id  3 | task 1039 | n_decoded =    100, tg =  15.21 t/s, tg_3s =  15.21 t/s
2.48.904.712 I slot print_timing: id  3 | task 1039 | n_decoded =    146, tg =  15.19 t/s, tg_3s =  15.13 t/s
2.51.968.472 I slot print_timing: id  3 | task 1039 | n_decoded =    192, tg =  15.15 t/s, tg_3s =  15.01 t/s
2.54.987.778 I slot print_timing: id  3 | task 1039 | n_decoded =    237, tg =  15.10 t/s, tg_3s =  14.90 t/s
2.56.122.697 I slot print_timing: id  3 | task 1039 | prompt eval time =    5360.10 ms /    44 tokens (  121.82 ms per token,     8.21 tokens per second)
2.56.122.705 I slot print_timing: id  3 | task 1039 |        eval time =   16831.49 ms /   254 tokens (   66.27 ms per token,    15.09 tokens per second)
2.56.122.705 I slot print_timing: id  3 | task 1039 |       total time =   22191.60 ms /   298 tokens
2.56.122.706 I slot print_timing: id  3 | task 1039 |    graphs reused =       1264
2.56.122.807 I slot      release: id  3 | task 1039 | stop processing: n_tokens = 339, truncated = 0
2.56.122.822 I srv  update_slots: all slots are idle
2.56.154.631 I srv    operator(): Chat format: peg-native
2.56.154.882 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.625 (> 0.100 thold), f_keep = 0.133
2.56.154.884 I srv  get_availabl: updating prompt cache
2.56.154.929 W srv   prompt_save:  - saving prompt with length 339, total state size = 167.341 MiB (draft: 0.000 MiB)
2.56.331.845 I srv          load:  - looking for better prompt, base f_keep = 0.133, sim = 0.625
2.56.331.877 I srv        update:  - cache state: 7 prompts, 2319.926 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
2.56.331.879 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
2.56.331.880 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
2.56.331.880 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
2.56.331.881 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
2.56.331.881 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
2.56.331.882 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
2.56.331.882 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
2.56.331.883 I srv  get_availabl: prompt cache update took 177.00 ms
2.56.331.969 I slot launch_slot_: id  3 | task 1295 | processing task, is_child = 0
2.56.331.969 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
2.56.331.970 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
2.56.331.985 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
2.56.331.986 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
2.56.332.013 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
2.56.332.020 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
2.56.332.040 I slot update_slots: id  3 | task 1295 | Checking checkpoint with [41, 41] against 45...
2.56.362.894 W slot update_slots: id  3 | task 1295 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 164.688 MiB)
3.03.797.004 I slot print_timing: id  3 | task 1295 | n_decoded =    100, tg =  15.11 t/s, tg_3s =  15.11 t/s
3.06.799.997 I slot print_timing: id  3 | task 1295 | n_decoded =    145, tg =  15.07 t/s, tg_3s =  14.99 t/s
3.08.327.562 I slot print_timing: id  3 | task 1295 | prompt eval time =     847.27 ms /    30 tokens (   28.24 ms per token,    35.41 tokens per second)
3.08.327.567 I slot print_timing: id  3 | task 1295 |        eval time =   11148.24 ms /   168 tokens (   66.36 ms per token,    15.07 tokens per second)
3.08.327.568 I slot print_timing: id  3 | task 1295 |       total time =   11995.51 ms /   198 tokens
3.08.327.569 I slot print_timing: id  3 | task 1295 |    graphs reused =       1430
3.08.327.598 I slot      release: id  3 | task 1295 | stop processing: n_tokens = 239, truncated = 0
3.08.327.613 I srv  update_slots: all slots are idle
3.08.361.383 I srv    operator(): Chat format: peg-native
3.08.361.653 I slot get_availabl: id  2 | task -1 | selected slot by LRU, t_last = -1
3.08.361.655 I srv  get_availabl: updating prompt cache
3.08.361.657 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
3.08.361.692 I srv        update:  - cache state: 7 prompts, 2319.926 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
3.08.361.695 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
3.08.361.696 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
3.08.361.696 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
3.08.361.697 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
3.08.361.697 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
3.08.361.698 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
3.08.361.699 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
3.08.361.699 I srv  get_availabl: prompt cache update took 0.04 ms
3.08.361.774 I slot launch_slot_: id  2 | task 1465 | processing task, is_child = 0
3.08.361.774 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
3.08.361.775 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
3.08.361.795 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
3.08.361.795 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
3.08.361.810 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
3.08.361.858 W srv   prompt_save:  - saving prompt with length 239, total state size = 166.559 MiB (draft: 0.000 MiB)
3.08.539.791 I srv        update:  - cache state: 8 prompts, 2651.173 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
3.08.539.794 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
3.08.539.795 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
3.08.539.796 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
3.08.539.796 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
3.08.539.797 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
3.08.539.797 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
3.08.539.798 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
3.08.539.798 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
3.08.539.799 I slot prompt_clear: id  3 | task -1 | clearing prompt with 239 tokens
3.14.203.422 I slot print_timing: id  2 | task 1465 | prompt processing, n_tokens =     60, progress = 0.59, t =   5.66 s / 10.59 tokens per second
3.14.308.879 I slot create_check: id  2 | task 1465 | created context checkpoint 1 of 32 (pos_min = 59, pos_max = 59, n_tokens = 60, size = 164.688 MiB)
3.19.101.585 I slot print_timing: id  2 | task 1465 | prompt processing, n_tokens =     97, progress = 0.96, t =  10.56 s / 9.18 tokens per second
3.25.876.130 I slot print_timing: id  2 | task 1465 | n_decoded =    100, tg =  15.12 t/s, tg_3s =  15.12 t/s
3.28.935.003 I slot print_timing: id  2 | task 1465 | n_decoded =    146, tg =  15.09 t/s, tg_3s =  15.04 t/s
3.31.951.613 I slot print_timing: id  2 | task 1465 | n_decoded =    191, tg =  15.05 t/s, tg_3s =  14.92 t/s
3.34.987.815 I slot print_timing: id  2 | task 1465 | n_decoded =    237, tg =  15.07 t/s, tg_3s =  15.15 t/s
3.38.014.989 I slot print_timing: id  2 | task 1465 | n_decoded =    283, tg =  15.09 t/s, tg_3s =  15.20 t/s
3.41.036.202 I slot print_timing: id  2 | task 1465 | n_decoded =    329, tg =  15.11 t/s, tg_3s =  15.23 t/s
3.44.054.792 I slot print_timing: id  2 | task 1465 | n_decoded =    375, tg =  15.13 t/s, tg_3s =  15.24 t/s
3.47.093.908 I slot print_timing: id  2 | task 1465 | n_decoded =    421, tg =  15.13 t/s, tg_3s =  15.14 t/s
3.50.155.582 I slot print_timing: id  2 | task 1465 | n_decoded =    468, tg =  15.15 t/s, tg_3s =  15.35 t/s
3.53.205.580 I slot print_timing: id  2 | task 1465 | n_decoded =    515, tg =  15.17 t/s, tg_3s =  15.41 t/s
3.56.266.227 I slot print_timing: id  2 | task 1465 | n_decoded =    562, tg =  15.19 t/s, tg_3s =  15.36 t/s
3.59.300.543 I slot print_timing: id  2 | task 1465 | n_decoded =    608, tg =  15.19 t/s, tg_3s =  15.16 t/s
4.02.337.458 I slot print_timing: id  2 | task 1465 | n_decoded =    654, tg =  15.18 t/s, tg_3s =  15.15 t/s
4.05.383.477 I slot print_timing: id  2 | task 1465 | n_decoded =    700, tg =  15.18 t/s, tg_3s =  15.10 t/s
4.08.423.536 I slot print_timing: id  2 | task 1465 | n_decoded =    746, tg =  15.17 t/s, tg_3s =  15.13 t/s
4.11.461.474 I slot print_timing: id  2 | task 1465 | n_decoded =    792, tg =  15.17 t/s, tg_3s =  15.14 t/s
4.14.506.583 I slot print_timing: id  2 | task 1465 | n_decoded =    838, tg =  15.17 t/s, tg_3s =  15.11 t/s
4.17.542.483 I slot print_timing: id  2 | task 1465 | n_decoded =    884, tg =  15.17 t/s, tg_3s =  15.15 t/s
4.20.579.342 I slot print_timing: id  2 | task 1465 | n_decoded =    929, tg =  15.15 t/s, tg_3s =  14.82 t/s
4.21.661.516 I slot print_timing: id  2 | task 1465 | prompt eval time =   10722.05 ms /   101 tokens (  106.16 ms per token,     9.42 tokens per second)
4.21.661.521 I slot print_timing: id  2 | task 1465 |        eval time =   62399.57 ms /   945 tokens (   66.03 ms per token,    15.14 tokens per second)
4.21.661.522 I slot print_timing: id  2 | task 1465 |       total time =   73121.62 ms /  1046 tokens
4.21.661.523 I slot print_timing: id  2 | task 1465 |    graphs reused =       2369
4.21.661.899 I slot      release: id  2 | task 1465 | stop processing: n_tokens = 1045, truncated = 0
4.21.661.914 I srv  update_slots: all slots are idle
4.21.695.502 I srv    operator(): Chat format: peg-native
4.21.695.755 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 0.500 (> 0.100 thold), f_keep = 0.060
4.21.695.756 I srv  get_availabl: updating prompt cache
4.21.695.811 W srv   prompt_save:  - saving prompt with length 1045, total state size = 172.865 MiB (draft: 0.000 MiB)
4.21.885.040 I srv          load:  - looking for better prompt, base f_keep = 0.060, sim = 0.500
4.21.885.051 I srv        update:  - cache state: 9 prompts, 2988.726 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
4.21.885.052 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
4.21.885.082 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
4.21.885.083 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
4.21.885.084 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
4.21.885.085 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
4.21.885.085 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
4.21.885.086 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
4.21.885.086 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
4.21.885.087 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
4.21.885.091 I srv  get_availabl: prompt cache update took 189.33 ms
4.21.885.193 I slot launch_slot_: id  2 | task 2413 | processing task, is_child = 0
4.21.885.193 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
4.21.885.194 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
4.21.885.212 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
4.21.885.212 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
4.21.885.225 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
4.21.885.231 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
4.21.885.254 I slot update_slots: id  2 | task 2413 | Checking checkpoint with [59, 59] against 63...
4.21.916.150 W slot update_slots: id  2 | task 2413 | restored context checkpoint (pos_min = 59, pos_max = 59, n_tokens = 60, n_past = 60, size = 164.688 MiB)
4.28.003.401 I slot print_timing: id  2 | task 2413 | prompt processing, n_tokens =     62, progress = 0.97, t =   6.12 s / 10.13 tokens per second
4.34.864.288 I slot print_timing: id  2 | task 2413 | n_decoded =    100, tg =  14.92 t/s, tg_3s =  14.92 t/s
4.37.884.780 I slot print_timing: id  2 | task 2413 | n_decoded =    145, tg =  14.91 t/s, tg_3s =  14.90 t/s
4.40.888.652 I slot print_timing: id  2 | task 2413 | n_decoded =    190, tg =  14.93 t/s, tg_3s =  14.98 t/s
4.43.888.818 I slot print_timing: id  2 | task 2413 | n_decoded =    235, tg =  14.94 t/s, tg_3s =  15.00 t/s
4.46.955.388 I slot print_timing: id  2 | task 2413 | n_decoded =    281, tg =  14.95 t/s, tg_3s =  15.00 t/s
4.48.155.437 I slot print_timing: id  2 | task 2413 | prompt eval time =    6275.37 ms /    66 tokens (   95.08 ms per token,    10.52 tokens per second)
4.48.155.444 I slot print_timing: id  2 | task 2413 |        eval time =   19994.80 ms /   299 tokens (   66.87 ms per token,    14.95 tokens per second)
4.48.155.445 I slot print_timing: id  2 | task 2413 |       total time =   26270.17 ms /   365 tokens
4.48.155.445 I slot print_timing: id  2 | task 2413 |    graphs reused =       2665
4.48.155.556 I slot      release: id  2 | task 2413 | stop processing: n_tokens = 424, truncated = 0
4.48.155.573 I srv  update_slots: all slots are idle
4.48.188.567 I srv    operator(): Chat format: peg-native
4.48.188.868 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 0.508 (> 0.100 thold), f_keep = 0.149
4.48.188.871 I srv  get_availabl: updating prompt cache
4.48.188.926 W srv   prompt_save:  - saving prompt with length 424, total state size = 168.006 MiB (draft: 0.000 MiB)
4.48.375.779 I srv          load:  - looking for better prompt, base f_keep = 0.149, sim = 0.508
4.48.375.787 I srv        update:  - cache state: 10 prompts, 3321.421 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
4.48.375.788 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
4.48.375.789 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
4.48.375.789 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
4.48.375.790 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
4.48.375.791 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
4.48.375.795 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
4.48.375.795 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
4.48.375.796 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
4.48.375.796 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
4.48.375.818 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
4.48.375.820 I srv  get_availabl: prompt cache update took 186.95 ms
4.48.375.923 I slot launch_slot_: id  2 | task 2714 | processing task, is_child = 0
4.48.375.924 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
4.48.375.924 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
4.48.375.945 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
4.48.375.946 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
4.48.375.979 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
4.48.375.979 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
4.48.376.005 I slot update_slots: id  2 | task 2714 | Checking checkpoint with [59, 59] against 63...
4.48.406.785 W slot update_slots: id  2 | task 2714 | restored context checkpoint (pos_min = 59, pos_max = 59, n_tokens = 60, n_past = 60, size = 164.688 MiB)
4.54.339.679 I slot print_timing: id  2 | task 2714 | prompt processing, n_tokens =     60, progress = 0.97, t =   5.96 s / 10.06 tokens per second
5.01.120.503 I slot print_timing: id  2 | task 2714 | n_decoded =    100, tg =  15.11 t/s, tg_3s =  15.11 t/s
5.04.168.558 I slot print_timing: id  2 | task 2714 | n_decoded =    145, tg =  15.00 t/s, tg_3s =  14.76 t/s
5.07.191.541 I slot print_timing: id  2 | task 2714 | n_decoded =    190, tg =  14.97 t/s, tg_3s =  14.89 t/s
5.10.229.579 I slot print_timing: id  2 | task 2714 | n_decoded =    236, tg =  15.01 t/s, tg_3s =  15.14 t/s
5.13.258.693 I slot print_timing: id  2 | task 2714 | n_decoded =    282, tg =  15.04 t/s, tg_3s =  15.19 t/s
5.16.284.632 I slot print_timing: id  2 | task 2714 | n_decoded =    328, tg =  15.06 t/s, tg_3s =  15.20 t/s
5.19.305.527 I slot print_timing: id  2 | task 2714 | n_decoded =    374, tg =  15.08 t/s, tg_3s =  15.23 t/s
5.22.335.772 I slot print_timing: id  2 | task 2714 | n_decoded =    420, tg =  15.09 t/s, tg_3s =  15.18 t/s
5.25.355.435 I slot print_timing: id  2 | task 2714 | n_decoded =    466, tg =  15.10 t/s, tg_3s =  15.23 t/s
5.28.361.059 I slot print_timing: id  2 | task 2714 | n_decoded =    512, tg =  15.12 t/s, tg_3s =  15.30 t/s
5.31.420.436 I slot print_timing: id  2 | task 2714 | n_decoded =    559, tg =  15.14 t/s, tg_3s =  15.36 t/s
5.34.466.288 I slot print_timing: id  2 | task 2714 | n_decoded =    606, tg =  15.16 t/s, tg_3s =  15.43 t/s
5.37.513.095 I slot print_timing: id  2 | task 2714 | n_decoded =    652, tg =  15.16 t/s, tg_3s =  15.10 t/s
5.40.546.259 I slot print_timing: id  2 | task 2714 | n_decoded =    698, tg =  15.16 t/s, tg_3s =  15.17 t/s
5.43.578.370 I slot print_timing: id  2 | task 2714 | n_decoded =    744, tg =  15.16 t/s, tg_3s =  15.17 t/s
5.46.615.262 I slot print_timing: id  2 | task 2714 | n_decoded =    790, tg =  15.16 t/s, tg_3s =  15.15 t/s
5.49.658.586 I slot print_timing: id  2 | task 2714 | n_decoded =    836, tg =  15.16 t/s, tg_3s =  15.12 t/s
5.52.689.127 I slot print_timing: id  2 | task 2714 | n_decoded =    882, tg =  15.16 t/s, tg_3s =  15.18 t/s
5.55.736.355 I slot print_timing: id  2 | task 2714 | n_decoded =    928, tg =  15.16 t/s, tg_3s =  15.10 t/s
5.58.802.648 I slot print_timing: id  2 | task 2714 | n_decoded =    974, tg =  15.15 t/s, tg_3s =  15.00 t/s
6.01.846.591 I slot print_timing: id  2 | task 2714 | n_decoded =   1020, tg =  15.15 t/s, tg_3s =  15.11 t/s
6.04.898.635 I slot print_timing: id  2 | task 2714 | n_decoded =   1066, tg =  15.14 t/s, tg_3s =  15.07 t/s
6.06.363.224 I slot print_timing: id  2 | task 2714 | prompt eval time =    6126.61 ms /    64 tokens (   95.73 ms per token,    10.45 tokens per second)
6.06.363.230 I slot print_timing: id  2 | task 2714 |        eval time =   71860.59 ms /  1088 tokens (   66.05 ms per token,    15.14 tokens per second)
6.06.363.231 I slot print_timing: id  2 | task 2714 |       total time =   77987.21 ms /  1152 tokens
6.06.363.232 I slot print_timing: id  2 | task 2714 |    graphs reused =       3747
6.06.363.274 I slot      release: id  2 | task 2714 | stop processing: n_tokens = 1211, truncated = 0
6.06.363.295 I srv  update_slots: all slots are idle
6.06.394.232 I srv    operator(): Chat format: peg-native
6.06.394.482 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 0.470 (> 0.100 thold), f_keep = 0.052
6.06.394.484 I srv  get_availabl: updating prompt cache
6.06.394.553 W srv   prompt_save:  - saving prompt with length 1211, total state size = 174.163 MiB (draft: 0.000 MiB)
6.06.588.519 I srv          load:  - looking for better prompt, base f_keep = 0.052, sim = 0.470
6.06.588.548 I srv        update:  - cache state: 11 prompts, 3660.273 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
6.06.588.550 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
6.06.588.551 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
6.06.588.552 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
6.06.588.553 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
6.06.588.558 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
6.06.588.559 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
6.06.588.560 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
6.06.588.561 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
6.06.588.562 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
6.06.588.565 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
6.06.588.566 I srv        update:    - prompt 0x5ae370019d10:    1211 tokens, checkpoints:  1,   338.852 MiB
6.06.588.567 I srv  get_availabl: prompt cache update took 194.08 ms
6.06.588.723 I slot launch_slot_: id  2 | task 3804 | processing task, is_child = 0
6.06.588.727 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
6.06.588.747 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
6.06.588.769 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
6.06.588.770 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
6.06.588.785 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
6.06.588.785 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
6.06.588.811 I slot update_slots: id  2 | task 3804 | Checking checkpoint with [59, 59] against 63...
6.06.619.627 W slot update_slots: id  2 | task 3804 | restored context checkpoint (pos_min = 59, pos_max = 59, n_tokens = 60, n_past = 60, size = 164.688 MiB)
6.12.199.183 I slot print_timing: id  2 | task 3804 | prompt processing, n_tokens =     70, progress = 0.97, t =   5.61 s / 12.48 tokens per second
6.18.995.515 I slot print_timing: id  2 | task 3804 | n_decoded =    100, tg =  15.06 t/s, tg_3s =  15.06 t/s
6.22.018.546 I slot print_timing: id  2 | task 3804 | n_decoded =    145, tg =  15.01 t/s, tg_3s =  14.89 t/s
6.25.029.271 I slot print_timing: id  2 | task 3804 | n_decoded =    190, tg =  14.99 t/s, tg_3s =  14.95 t/s
6.28.062.336 I slot print_timing: id  2 | task 3804 | n_decoded =    235, tg =  14.96 t/s, tg_3s =  14.84 t/s
6.31.123.706 I slot print_timing: id  2 | task 3804 | n_decoded =    281, tg =  14.97 t/s, tg_3s =  15.03 t/s
6.34.169.245 I slot print_timing: id  2 | task 3804 | n_decoded =    327, tg =  14.99 t/s, tg_3s =  15.10 t/s
6.37.225.269 I slot print_timing: id  2 | task 3804 | n_decoded =    373, tg =  15.00 t/s, tg_3s =  15.05 t/s
6.40.289.750 I slot print_timing: id  2 | task 3804 | n_decoded =    419, tg =  15.00 t/s, tg_3s =  15.01 t/s
6.43.331.321 I slot print_timing: id  2 | task 3804 | n_decoded =    465, tg =  15.01 t/s, tg_3s =  15.12 t/s
6.46.391.435 I slot print_timing: id  2 | task 3804 | n_decoded =    511, tg =  15.01 t/s, tg_3s =  15.03 t/s
6.49.436.395 I slot print_timing: id  2 | task 3804 | n_decoded =    557, tg =  15.02 t/s, tg_3s =  15.11 t/s
6.52.470.483 I slot print_timing: id  2 | task 3804 | n_decoded =    603, tg =  15.03 t/s, tg_3s =  15.16 t/s
6.55.496.407 I slot print_timing: id  2 | task 3804 | n_decoded =    649, tg =  15.04 t/s, tg_3s =  15.20 t/s
6.58.504.485 I slot print_timing: id  2 | task 3804 | n_decoded =    695, tg =  15.06 t/s, tg_3s =  15.29 t/s
7.01.526.561 I slot print_timing: id  2 | task 3804 | n_decoded =    741, tg =  15.07 t/s, tg_3s =  15.22 t/s
7.04.539.601 I slot print_timing: id  2 | task 3804 | n_decoded =    787, tg =  15.08 t/s, tg_3s =  15.27 t/s
7.04.736.434 I slot print_timing: id  2 | task 3804 | prompt eval time =    5766.81 ms /    74 tokens (   77.93 ms per token,    12.83 tokens per second)
7.04.736.440 I slot print_timing: id  2 | task 3804 |        eval time =   52380.80 ms /   790 tokens (   66.30 ms per token,    15.08 tokens per second)
7.04.736.441 I slot print_timing: id  2 | task 3804 |       total time =   58147.61 ms /   864 tokens
7.04.736.442 I slot print_timing: id  2 | task 3804 |    graphs reused =       4532
7.04.736.557 I slot      release: id  2 | task 3804 | stop processing: n_tokens = 923, truncated = 0
7.04.736.579 I srv  update_slots: all slots are idle
7.04.770.977 I srv    operator(): Chat format: peg-native
7.04.771.228 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 0.600 (> 0.100 thold), f_keep = 0.068
7.04.771.230 I srv  get_availabl: updating prompt cache
7.04.771.315 W srv   prompt_save:  - saving prompt with length 923, total state size = 171.910 MiB (draft: 0.000 MiB)
7.04.948.117 I srv          load:  - looking for better prompt, base f_keep = 0.068, sim = 0.600
7.04.948.127 I srv        update:  - cache state: 12 prompts, 3996.871 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
7.04.948.128 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
7.04.948.128 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
7.04.948.129 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
7.04.948.133 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
7.04.948.134 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
7.04.948.135 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
7.04.948.135 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
7.04.948.136 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
7.04.948.138 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
7.04.948.170 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
7.04.948.170 I srv        update:    - prompt 0x5ae370019d10:    1211 tokens, checkpoints:  1,   338.852 MiB
7.04.948.192 I srv        update:    - prompt 0x5ae3701b33d0:     923 tokens, checkpoints:  1,   336.599 MiB
7.04.948.193 I srv  get_availabl: prompt cache update took 176.96 ms
7.04.948.301 I slot launch_slot_: id  2 | task 4596 | processing task, is_child = 0
7.04.948.302 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
7.04.948.303 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
7.04.948.322 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
7.04.948.322 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
7.04.948.337 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
7.04.948.337 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
7.04.948.360 I slot update_slots: id  2 | task 4596 | Checking checkpoint with [59, 59] against 63...
7.04.979.271 W slot update_slots: id  2 | task 4596 | restored context checkpoint (pos_min = 59, pos_max = 59, n_tokens = 60, n_past = 60, size = 164.688 MiB)
7.09.879.812 I slot print_timing: id  2 | task 4596 | prompt processing, n_tokens =     41, progress = 0.96, t =   4.93 s / 8.31 tokens per second
7.16.432.348 I slot print_timing: id  2 | task 4596 | n_decoded =    100, tg =  15.64 t/s, tg_3s =  15.64 t/s
7.19.452.231 I slot print_timing: id  2 | task 4596 | n_decoded =    146, tg =  15.51 t/s, tg_3s =  15.23 t/s
7.22.485.079 I slot print_timing: id  2 | task 4596 | n_decoded =    192, tg =  15.43 t/s, tg_3s =  15.17 t/s
7.25.522.022 I slot print_timing: id  2 | task 4596 | n_decoded =    238, tg =  15.37 t/s, tg_3s =  15.15 t/s
7.28.555.237 I slot print_timing: id  2 | task 4596 | n_decoded =    284, tg =  15.34 t/s, tg_3s =  15.17 t/s
7.31.593.581 I slot print_timing: id  2 | task 4596 | n_decoded =    330, tg =  15.31 t/s, tg_3s =  15.14 t/s
7.34.634.402 I slot print_timing: id  2 | task 4596 | n_decoded =    376, tg =  15.29 t/s, tg_3s =  15.13 t/s
7.37.685.522 I slot print_timing: id  2 | task 4596 | n_decoded =    422, tg =  15.26 t/s, tg_3s =  15.08 t/s
7.40.746.164 I slot print_timing: id  2 | task 4596 | n_decoded =    468, tg =  15.24 t/s, tg_3s =  15.03 t/s
7.43.797.538 I slot print_timing: id  2 | task 4596 | n_decoded =    514, tg =  15.23 t/s, tg_3s =  15.08 t/s
7.46.851.579 I slot print_timing: id  2 | task 4596 | n_decoded =    560, tg =  15.21 t/s, tg_3s =  15.06 t/s
7.49.916.598 I slot print_timing: id  2 | task 4596 | n_decoded =    606, tg =  15.20 t/s, tg_3s =  15.01 t/s
7.52.929.387 I slot print_timing: id  2 | task 4596 | n_decoded =    651, tg =  15.18 t/s, tg_3s =  14.94 t/s
7.55.949.425 I slot print_timing: id  2 | task 4596 | n_decoded =    696, tg =  15.16 t/s, tg_3s =  14.90 t/s
7.58.977.449 I slot print_timing: id  2 | task 4596 | n_decoded =    741, tg =  15.14 t/s, tg_3s =  14.86 t/s
8.02.042.167 I slot print_timing: id  2 | task 4596 | n_decoded =    787, tg =  15.13 t/s, tg_3s =  15.01 t/s
8.05.043.212 I slot print_timing: id  2 | task 4596 | n_decoded =    832, tg =  15.13 t/s, tg_3s =  14.99 t/s
8.05.643.202 I slot print_timing: id  2 | task 4596 | prompt eval time =    5090.39 ms /    45 tokens (  113.12 ms per token,     8.84 tokens per second)
8.05.643.208 I slot print_timing: id  2 | task 4596 |        eval time =   55604.44 ms /   841 tokens (   66.12 ms per token,    15.12 tokens per second)
8.05.643.209 I slot print_timing: id  2 | task 4596 |       total time =   60694.83 ms /   886 tokens
8.05.643.210 I slot print_timing: id  2 | task 4596 |    graphs reused =       5368
8.05.643.313 I slot      release: id  2 | task 4596 | stop processing: n_tokens = 945, truncated = 0
8.05.643.333 I srv  update_slots: all slots are idle
8.05.676.014 I srv    operator(): Chat format: peg-native
8.05.676.265 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 0.525 (> 0.100 thold), f_keep = 0.067
8.05.676.267 I srv  get_availabl: updating prompt cache
8.05.676.342 W srv   prompt_save:  - saving prompt with length 945, total state size = 172.082 MiB (draft: 0.000 MiB)
8.05.854.190 I srv          load:  - looking for better prompt, base f_keep = 0.067, sim = 0.525
8.05.854.203 I srv        update:  - cache state: 13 prompts, 4333.642 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
8.05.854.206 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
8.05.854.207 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
8.05.854.208 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
8.05.854.208 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
8.05.854.211 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
8.05.854.212 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
8.05.854.213 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
8.05.854.214 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
8.05.854.215 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
8.05.854.218 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
8.05.854.218 I srv        update:    - prompt 0x5ae370019d10:    1211 tokens, checkpoints:  1,   338.852 MiB
8.05.854.219 I srv        update:    - prompt 0x5ae3701b33d0:     923 tokens, checkpoints:  1,   336.599 MiB
8.05.854.220 I srv        update:    - prompt 0x5ae370030910:     945 tokens, checkpoints:  1,   336.771 MiB
8.05.854.222 I srv  get_availabl: prompt cache update took 177.95 ms
8.05.854.338 I slot launch_slot_: id  2 | task 5439 | processing task, is_child = 0
8.05.854.339 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
8.05.854.340 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
8.05.854.371 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
8.05.854.372 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
8.05.854.408 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
8.05.854.410 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
8.05.854.433 I slot update_slots: id  2 | task 5439 | Checking checkpoint with [59, 59] against 63...
8.05.885.548 W slot update_slots: id  2 | task 5439 | restored context checkpoint (pos_min = 59, pos_max = 59, n_tokens = 60, n_past = 60, size = 164.688 MiB)
8.11.650.024 I slot print_timing: id  2 | task 5439 | prompt processing, n_tokens =     56, progress = 0.97, t =   5.80 s / 9.66 tokens per second
8.18.424.239 I slot print_timing: id  2 | task 5439 | n_decoded =    100, tg =  15.12 t/s, tg_3s =  15.12 t/s
8.21.466.284 I slot print_timing: id  2 | task 5439 | n_decoded =    145, tg =  15.02 t/s, tg_3s =  14.79 t/s
8.24.471.619 I slot print_timing: id  2 | task 5439 | n_decoded =    190, tg =  15.01 t/s, tg_3s =  14.97 t/s
8.27.529.726 I slot print_timing: id  2 | task 5439 | n_decoded =    236, tg =  15.01 t/s, tg_3s =  15.04 t/s
8.30.576.478 I slot print_timing: id  2 | task 5439 | n_decoded =    282, tg =  15.03 t/s, tg_3s =  15.10 t/s
8.33.594.510 I slot print_timing: id  2 | task 5439 | n_decoded =    327, tg =  15.01 t/s, tg_3s =  14.91 t/s
8.36.629.372 I slot print_timing: id  2 | task 5439 | n_decoded =    373, tg =  15.03 t/s, tg_3s =  15.16 t/s
8.39.676.582 I slot print_timing: id  2 | task 5439 | n_decoded =    419, tg =  15.04 t/s, tg_3s =  15.10 t/s
8.41.326.725 I slot print_timing: id  2 | task 5439 | prompt eval time =    5955.18 ms /    60 tokens (   99.25 ms per token,    10.08 tokens per second)
8.41.326.731 I slot print_timing: id  2 | task 5439 |        eval time =   29517.10 ms /   444 tokens (   66.48 ms per token,    15.04 tokens per second)
8.41.326.732 I slot print_timing: id  2 | task 5439 |       total time =   35472.28 ms /   504 tokens
8.41.326.732 I slot print_timing: id  2 | task 5439 |    graphs reused =       5808
8.41.326.854 I slot      release: id  2 | task 5439 | stop processing: n_tokens = 563, truncated = 0
8.41.326.874 I srv  update_slots: all slots are idle
8.41.362.153 I srv    operator(): Chat format: peg-native
8.41.362.417 I slot get_availabl: id  1 | task -1 | selected slot by LRU, t_last = -1
8.41.362.420 I srv  get_availabl: updating prompt cache
8.41.362.422 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
8.41.362.430 I srv        update:  - cache state: 13 prompts, 4333.642 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
8.41.362.432 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
8.41.362.433 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
8.41.362.434 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
8.41.362.435 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
8.41.362.436 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
8.41.362.437 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
8.41.362.438 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
8.41.362.439 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
8.41.362.440 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
8.41.362.441 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
8.41.362.442 I srv        update:    - prompt 0x5ae370019d10:    1211 tokens, checkpoints:  1,   338.852 MiB
8.41.362.442 I srv        update:    - prompt 0x5ae3701b33d0:     923 tokens, checkpoints:  1,   336.599 MiB
8.41.362.443 I srv        update:    - prompt 0x5ae370030910:     945 tokens, checkpoints:  1,   336.771 MiB
8.41.362.444 I srv  get_availabl: prompt cache update took 0.02 ms
8.41.362.513 I slot launch_slot_: id  1 | task 5885 | processing task, is_child = 0
8.41.362.514 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
8.41.362.515 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
8.41.362.539 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
8.41.362.612 W srv   prompt_save:  - saving prompt with length 563, total state size = 169.094 MiB (draft: 0.000 MiB)
8.41.551.512 I srv        update:  - cache state: 14 prompts, 4667.424 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
8.41.551.516 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
8.41.551.517 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
8.41.551.517 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
8.41.551.518 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
8.41.551.519 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
8.41.551.519 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
8.41.551.520 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
8.41.551.521 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
8.41.551.521 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
8.41.551.522 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
8.41.551.522 I srv        update:    - prompt 0x5ae370019d10:    1211 tokens, checkpoints:  1,   338.852 MiB
8.41.551.523 I srv        update:    - prompt 0x5ae3701b33d0:     923 tokens, checkpoints:  1,   336.599 MiB
8.41.551.524 I srv        update:    - prompt 0x5ae370030910:     945 tokens, checkpoints:  1,   336.771 MiB
8.41.551.528 I srv        update:    - prompt 0x5ae3769bd230:     563 tokens, checkpoints:  1,   333.782 MiB
8.41.551.529 I slot prompt_clear: id  2 | task -1 | clearing prompt with 563 tokens
8.41.551.649 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
8.41.551.650 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
8.46.904.088 I slot print_timing: id  1 | task 5885 | prompt processing, n_tokens =     56, progress = 0.41, t =   5.35 s / 10.46 tokens per second
8.47.001.080 I slot create_check: id  1 | task 5885 | created context checkpoint 1 of 32 (pos_min = 55, pos_max = 55, n_tokens = 56, size = 164.688 MiB)
8.53.015.835 I slot print_timing: id  1 | task 5885 | prompt processing, n_tokens =    132, progress = 0.97, t =  11.46 s / 11.51 tokens per second
8.59.703.982 I slot print_timing: id  1 | task 5885 | n_decoded =    100, tg =  15.32 t/s, tg_3s =  15.32 t/s
9.02.754.326 I slot print_timing: id  1 | task 5885 | n_decoded =    146, tg =  15.24 t/s, tg_3s =  15.08 t/s
9.05.790.978 I slot print_timing: id  1 | task 5885 | n_decoded =    192, tg =  15.22 t/s, tg_3s =  15.15 t/s
9.07.844.168 I slot print_timing: id  1 | task 5885 | prompt eval time =   11622.89 ms /   136 tokens (   85.46 ms per token,    11.70 tokens per second)
9.07.844.172 I slot print_timing: id  1 | task 5885 |        eval time =   14669.54 ms /   223 tokens (   65.78 ms per token,    15.20 tokens per second)
9.07.844.173 I slot print_timing: id  1 | task 5885 |       total time =   26292.43 ms /   359 tokens
9.07.844.174 I slot print_timing: id  1 | task 5885 |    graphs reused =       6028
9.07.844.304 I slot      release: id  1 | task 5885 | stop processing: n_tokens = 358, truncated = 0
9.07.844.317 I srv  update_slots: all slots are idle
9.07.942.774 I srv    operator(): Chat format: peg-native
9.07.943.030 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.179
9.07.943.033 I srv  get_availabl: updating prompt cache
9.07.943.081 W srv   prompt_save:  - saving prompt with length 358, total state size = 167.490 MiB (draft: 0.000 MiB)
9.08.115.302 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 0.667
9.08.115.311 I srv        update:  - cache state: 15 prompts, 4999.602 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
9.08.115.312 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
9.08.115.316 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
9.08.115.317 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
9.08.115.317 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
9.08.115.317 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
9.08.115.318 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
9.08.115.318 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
9.08.115.319 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
9.08.115.320 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
9.08.115.320 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
9.08.115.320 I srv        update:    - prompt 0x5ae370019d10:    1211 tokens, checkpoints:  1,   338.852 MiB
9.08.115.321 I srv        update:    - prompt 0x5ae3701b33d0:     923 tokens, checkpoints:  1,   336.599 MiB
9.08.115.325 I srv        update:    - prompt 0x5ae370030910:     945 tokens, checkpoints:  1,   336.771 MiB
9.08.115.325 I srv        update:    - prompt 0x5ae3769bd230:     563 tokens, checkpoints:  1,   333.782 MiB
9.08.115.326 I srv        update:    - prompt 0x5ae37040ecb0:     358 tokens, checkpoints:  1,   332.178 MiB
9.08.115.327 I srv  get_availabl: prompt cache update took 172.29 ms
9.08.115.419 I slot launch_slot_: id  1 | task 6111 | processing task, is_child = 0
9.08.115.420 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
9.08.115.421 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
9.08.115.436 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
9.08.115.437 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
9.08.115.448 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
9.08.115.448 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
9.08.115.469 I slot update_slots: id  1 | task 6111 | Checking checkpoint with [55, 55] against 64...
9.08.146.229 W slot update_slots: id  1 | task 6111 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 164.688 MiB)
9.12.581.176 I slot print_timing: id  1 | task 6111 | prompt processing, n_tokens =     36, progress = 0.96, t =   4.47 s / 8.06 tokens per second
9.19.220.456 I slot print_timing: id  1 | task 6111 | prompt eval time =    4620.89 ms /    40 tokens (  115.52 ms per token,     8.66 tokens per second)
9.19.220.462 I slot print_timing: id  1 | task 6111 |        eval time =    6484.08 ms /    99 tokens (   65.50 ms per token,    15.27 tokens per second)
9.19.220.463 I slot print_timing: id  1 | task 6111 |       total time =   11104.97 ms /   139 tokens
9.19.220.463 I slot print_timing: id  1 | task 6111 |    graphs reused =       6125
9.19.220.493 I slot      release: id  1 | task 6111 | stop processing: n_tokens = 194, truncated = 0
9.19.220.506 I srv  update_slots: all slots are idle
9.19.294.544 I srv    operator(): Chat format: peg-native
9.19.294.786 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 0.615 (> 0.100 thold), f_keep = 0.330
9.19.294.788 I srv  get_availabl: updating prompt cache
9.19.294.824 W srv   prompt_save:  - saving prompt with length 194, total state size = 166.206 MiB (draft: 0.000 MiB)
9.19.479.198 I srv          load:  - looking for better prompt, base f_keep = 0.330, sim = 0.615
9.19.479.209 I srv        update:  - cache state: 16 prompts, 5330.497 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
9.19.479.210 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
9.19.479.213 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
9.19.479.214 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
9.19.479.214 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
9.19.479.215 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
9.19.479.215 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
9.19.479.216 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
9.19.479.217 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
9.19.479.218 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
9.19.479.219 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
9.19.479.219 I srv        update:    - prompt 0x5ae370019d10:    1211 tokens, checkpoints:  1,   338.852 MiB
9.19.479.220 I srv        update:    - prompt 0x5ae3701b33d0:     923 tokens, checkpoints:  1,   336.599 MiB
9.19.479.220 I srv        update:    - prompt 0x5ae370030910:     945 tokens, checkpoints:  1,   336.771 MiB
9.19.479.220 I srv        update:    - prompt 0x5ae3769bd230:     563 tokens, checkpoints:  1,   333.782 MiB
9.19.479.221 I srv        update:    - prompt 0x5ae37040ecb0:     358 tokens, checkpoints:  1,   332.178 MiB
9.19.479.221 I srv        update:    - prompt 0x5ae37018ab00:     194 tokens, checkpoints:  1,   330.895 MiB
9.19.479.223 I srv  get_availabl: prompt cache update took 184.43 ms
9.19.479.320 I slot launch_slot_: id  1 | task 6212 | processing task, is_child = 0
9.19.479.320 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
9.19.479.321 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
9.19.479.337 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
9.19.479.337 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
9.19.479.349 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
9.19.479.349 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
9.19.479.368 I slot update_slots: id  1 | task 6212 | Checking checkpoint with [55, 55] against 64...
9.19.509.998 W slot update_slots: id  1 | task 6212 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 164.688 MiB)
9.24.495.246 I slot print_timing: id  1 | task 6212 | prompt processing, n_tokens =     44, progress = 0.96, t =   5.02 s / 8.77 tokens per second
9.31.260.124 I slot print_timing: id  1 | task 6212 | n_decoded =    100, tg =  15.13 t/s, tg_3s =  15.13 t/s
9.34.280.336 I slot print_timing: id  1 | task 6212 | n_decoded =    145, tg =  15.06 t/s, tg_3s =  14.90 t/s
9.34.483.333 I slot print_timing: id  1 | task 6212 | prompt eval time =    5172.03 ms /    48 tokens (  107.75 ms per token,     9.28 tokens per second)
9.34.483.339 I slot print_timing: id  1 | task 6212 |        eval time =    9831.92 ms /   148 tokens (   66.43 ms per token,    15.05 tokens per second)
9.34.483.339 I slot print_timing: id  1 | task 6212 |       total time =   15003.95 ms /   196 tokens
9.34.483.340 I slot print_timing: id  1 | task 6212 |    graphs reused =       6271
9.34.483.381 I slot      release: id  1 | task 6212 | stop processing: n_tokens = 251, truncated = 0
9.34.483.399 I srv  update_slots: all slots are idle
9.34.583.129 I srv    operator(): Chat format: peg-native
9.34.583.387 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 0.474 (> 0.100 thold), f_keep = 0.255
9.34.583.389 I srv  get_availabl: updating prompt cache
9.34.583.427 W srv   prompt_save:  - saving prompt with length 251, total state size = 166.652 MiB (draft: 0.000 MiB)
9.34.758.185 I srv          load:  - looking for better prompt, base f_keep = 0.255, sim = 0.474
9.34.758.195 I srv        update:  - cache state: 17 prompts, 5661.838 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
9.34.758.198 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
9.34.758.199 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
9.34.758.200 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
9.34.758.200 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
9.34.758.201 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
9.34.758.202 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
9.34.758.203 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
9.34.758.203 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
9.34.758.204 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
9.34.758.204 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
9.34.758.205 I srv        update:    - prompt 0x5ae370019d10:    1211 tokens, checkpoints:  1,   338.852 MiB
9.34.758.205 I srv        update:    - prompt 0x5ae3701b33d0:     923 tokens, checkpoints:  1,   336.599 MiB
9.34.758.206 I srv        update:    - prompt 0x5ae370030910:     945 tokens, checkpoints:  1,   336.771 MiB
9.34.758.206 I srv        update:    - prompt 0x5ae3769bd230:     563 tokens, checkpoints:  1,   333.782 MiB
9.34.758.206 I srv        update:    - prompt 0x5ae37040ecb0:     358 tokens, checkpoints:  1,   332.178 MiB
9.34.758.207 I srv        update:    - prompt 0x5ae37018ab00:     194 tokens, checkpoints:  1,   330.895 MiB
9.34.758.207 I srv        update:    - prompt 0x5ae370219650:     251 tokens, checkpoints:  1,   331.341 MiB
9.34.758.209 I srv  get_availabl: prompt cache update took 174.82 ms
9.34.758.309 I slot launch_slot_: id  1 | task 6362 | processing task, is_child = 0
9.34.758.310 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
9.34.758.311 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
9.34.758.327 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
9.34.758.328 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
9.34.758.356 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
9.34.758.356 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
9.34.758.374 I slot update_slots: id  1 | task 6362 | Checking checkpoint with [55, 55] against 64...
9.34.789.151 W slot update_slots: id  1 | task 6362 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 164.688 MiB)
9.40.711.635 I slot print_timing: id  1 | task 6362 | prompt processing, n_tokens =     75, progress = 0.97, t =   5.95 s / 12.60 tokens per second
9.47.537.450 I slot print_timing: id  1 | task 6362 | n_decoded =    100, tg =  15.00 t/s, tg_3s =  15.00 t/s
9.50.547.519 I slot print_timing: id  1 | task 6362 | n_decoded =    145, tg =  14.98 t/s, tg_3s =  14.95 t/s
9.53.604.540 I slot print_timing: id  1 | task 6362 | n_decoded =    191, tg =  15.00 t/s, tg_3s =  15.05 t/s
9.56.667.539 I slot print_timing: id  1 | task 6362 | n_decoded =    237, tg =  15.00 t/s, tg_3s =  15.02 t/s
9.59.703.324 I slot print_timing: id  1 | task 6362 | n_decoded =    282, tg =  14.97 t/s, tg_3s =  14.82 t/s
10.02.756.182 I slot print_timing: id  1 | task 6362 | n_decoded =    328, tg =  14.99 t/s, tg_3s =  15.07 t/s
10.05.814.620 I slot print_timing: id  1 | task 6362 | n_decoded =    374, tg =  14.99 t/s, tg_3s =  15.04 t/s
10.08.857.651 I slot print_timing: id  1 | task 6362 | n_decoded =    420, tg =  15.01 t/s, tg_3s =  15.12 t/s
10.11.885.358 I slot print_timing: id  1 | task 6362 | n_decoded =    466, tg =  15.02 t/s, tg_3s =  15.19 t/s
10.14.903.642 I slot print_timing: id  1 | task 6362 | n_decoded =    512, tg =  15.04 t/s, tg_3s =  15.24 t/s
10.17.925.413 I slot print_timing: id  1 | task 6362 | n_decoded =    558, tg =  15.06 t/s, tg_3s =  15.22 t/s
10.20.929.758 I slot print_timing: id  1 | task 6362 | n_decoded =    604, tg =  15.08 t/s, tg_3s =  15.31 t/s
10.23.945.169 I slot print_timing: id  1 | task 6362 | n_decoded =    650, tg =  15.09 t/s, tg_3s =  15.25 t/s
10.26.946.284 I slot print_timing: id  1 | task 6362 | n_decoded =    695, tg =  15.08 t/s, tg_3s =  14.99 t/s
10.29.950.228 I slot print_timing: id  1 | task 6362 | n_decoded =    741, tg =  15.10 t/s, tg_3s =  15.31 t/s
10.30.144.780 I slot print_timing: id  1 | task 6362 | prompt eval time =    6111.12 ms /    79 tokens (   77.36 ms per token,    12.93 tokens per second)
10.30.144.785 I slot print_timing: id  1 | task 6362 |        eval time =   49275.27 ms /   744 tokens (   66.23 ms per token,    15.10 tokens per second)
10.30.144.786 I slot print_timing: id  1 | task 6362 |       total time =   55386.39 ms /   823 tokens
10.30.144.787 I slot print_timing: id  1 | task 6362 |    graphs reused =       7010
10.30.144.895 I slot      release: id  1 | task 6362 | stop processing: n_tokens = 878, truncated = 0
10.30.144.913 I srv  update_slots: all slots are idle
10.30.247.760 I srv    operator(): Chat format: peg-native
10.30.248.021 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 0.587 (> 0.100 thold), f_keep = 0.073
10.30.248.023 I srv  get_availabl: updating prompt cache
10.30.248.072 W srv   prompt_save:  - saving prompt with length 878, total state size = 171.558 MiB (draft: 0.000 MiB)
10.30.425.211 I srv          load:  - looking for better prompt, base f_keep = 0.073, sim = 0.587
10.30.425.221 I srv        update:  - cache state: 18 prompts, 5998.084 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
10.30.425.222 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
10.30.425.226 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
10.30.425.227 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
10.30.425.228 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
10.30.425.228 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
10.30.425.229 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
10.30.425.229 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
10.30.425.230 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
10.30.425.231 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
10.30.425.231 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
10.30.425.232 I srv        update:    - prompt 0x5ae370019d10:    1211 tokens, checkpoints:  1,   338.852 MiB
10.30.425.235 I srv        update:    - prompt 0x5ae3701b33d0:     923 tokens, checkpoints:  1,   336.599 MiB
10.30.425.235 I srv        update:    - prompt 0x5ae370030910:     945 tokens, checkpoints:  1,   336.771 MiB
10.30.425.236 I srv        update:    - prompt 0x5ae3769bd230:     563 tokens, checkpoints:  1,   333.782 MiB
10.30.425.237 I srv        update:    - prompt 0x5ae37040ecb0:     358 tokens, checkpoints:  1,   332.178 MiB
10.30.425.237 I srv        update:    - prompt 0x5ae37018ab00:     194 tokens, checkpoints:  1,   330.895 MiB
10.30.425.238 I srv        update:    - prompt 0x5ae370219650:     251 tokens, checkpoints:  1,   331.341 MiB
10.30.425.238 I srv        update:    - prompt 0x5ae3703b31a0:     878 tokens, checkpoints:  1,   336.247 MiB
10.30.425.240 I srv  get_availabl: prompt cache update took 177.22 ms
10.30.425.346 I slot launch_slot_: id  1 | task 7108 | processing task, is_child = 0
10.30.425.347 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
10.30.425.347 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
10.30.425.367 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
10.30.425.367 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
10.30.425.402 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
10.30.425.402 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
10.30.425.427 I slot update_slots: id  1 | task 7108 | Checking checkpoint with [55, 55] against 64...
10.30.456.346 W slot update_slots: id  1 | task 7108 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 164.688 MiB)
10.35.917.013 I slot print_timing: id  1 | task 7108 | prompt processing, n_tokens =     49, progress = 0.96, t =   5.49 s / 8.92 tokens per second
10.42.609.861 I slot print_timing: id  1 | task 7108 | n_decoded =    100, tg =  15.30 t/s, tg_3s =  15.30 t/s
10.45.656.114 I slot print_timing: id  1 | task 7108 | n_decoded =    146, tg =  15.24 t/s, tg_3s =  15.10 t/s
10.48.707.126 I slot print_timing: id  1 | task 7108 | n_decoded =    192, tg =  15.20 t/s, tg_3s =  15.08 t/s
10.51.744.158 I slot print_timing: id  1 | task 7108 | n_decoded =    238, tg =  15.19 t/s, tg_3s =  15.15 t/s
10.54.781.137 I slot print_timing: id  1 | task 7108 | n_decoded =    284, tg =  15.18 t/s, tg_3s =  15.15 t/s
10.55.771.167 I slot print_timing: id  1 | task 7108 | prompt eval time =    5648.00 ms /    53 tokens (  106.57 ms per token,     9.38 tokens per second)
10.55.771.172 I slot print_timing: id  1 | task 7108 |        eval time =   19697.73 ms /   299 tokens (   65.88 ms per token,    15.18 tokens per second)
10.55.771.172 I slot print_timing: id  1 | task 7108 |       total time =   25345.73 ms /   352 tokens
10.55.771.173 I slot print_timing: id  1 | task 7108 |    graphs reused =       7306
10.55.771.203 I slot      release: id  1 | task 7108 | stop processing: n_tokens = 407, truncated = 0
10.55.771.218 I srv  update_slots: all slots are idle
10.55.845.719 I srv    operator(): Chat format: peg-native
10.55.845.977 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 0.509 (> 0.100 thold), f_keep = 0.145
10.55.845.980 I srv  get_availabl: updating prompt cache
10.55.846.029 W srv   prompt_save:  - saving prompt with length 407, total state size = 167.873 MiB (draft: 0.000 MiB)
10.56.020.207 I srv          load:  - looking for better prompt, base f_keep = 0.145, sim = 0.509
10.56.020.218 I srv        update:  - cache state: 19 prompts, 6330.646 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
10.56.020.219 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
10.56.020.220 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
10.56.020.220 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
10.56.020.221 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
10.56.020.221 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
10.56.020.222 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
10.56.020.223 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
10.56.020.223 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
10.56.020.224 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
10.56.020.224 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
10.56.020.224 I srv        update:    - prompt 0x5ae370019d10:    1211 tokens, checkpoints:  1,   338.852 MiB
10.56.020.225 I srv        update:    - prompt 0x5ae3701b33d0:     923 tokens, checkpoints:  1,   336.599 MiB
10.56.020.225 I srv        update:    - prompt 0x5ae370030910:     945 tokens, checkpoints:  1,   336.771 MiB
10.56.020.226 I srv        update:    - prompt 0x5ae3769bd230:     563 tokens, checkpoints:  1,   333.782 MiB
10.56.020.226 I srv        update:    - prompt 0x5ae37040ecb0:     358 tokens, checkpoints:  1,   332.178 MiB
10.56.020.227 I srv        update:    - prompt 0x5ae37018ab00:     194 tokens, checkpoints:  1,   330.895 MiB
10.56.020.227 I srv        update:    - prompt 0x5ae370219650:     251 tokens, checkpoints:  1,   331.341 MiB
10.56.020.228 I srv        update:    - prompt 0x5ae3703b31a0:     878 tokens, checkpoints:  1,   336.247 MiB
10.56.020.228 I srv        update:    - prompt 0x5ae3705163d0:     407 tokens, checkpoints:  1,   332.561 MiB
10.56.020.230 I srv  get_availabl: prompt cache update took 174.25 ms
10.56.020.324 I slot launch_slot_: id  1 | task 7409 | processing task, is_child = 0
10.56.020.324 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
10.56.020.325 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
10.56.020.342 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
10.56.020.342 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
10.56.020.370 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
10.56.020.371 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
10.56.020.391 I slot update_slots: id  1 | task 7409 | Checking checkpoint with [55, 55] against 59...
10.56.051.056 W slot update_slots: id  1 | task 7409 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 164.688 MiB)
11.01.642.178 I slot print_timing: id  1 | task 7409 | prompt processing, n_tokens =     56, progress = 0.97, t =   5.62 s / 9.96 tokens per second
11.06.699.029 I slot print_timing: id  1 | task 7409 | prompt eval time =    5776.28 ms /    60 tokens (   96.27 ms per token,    10.39 tokens per second)
11.06.699.037 I slot print_timing: id  1 | task 7409 |        eval time =    4902.34 ms /    75 tokens (   65.36 ms per token,    15.30 tokens per second)
11.06.699.038 I slot print_timing: id  1 | task 7409 |       total time =   10678.62 ms /   135 tokens
11.06.699.039 I slot print_timing: id  1 | task 7409 |    graphs reused =       7379
11.06.699.147 I slot      release: id  1 | task 7409 | stop processing: n_tokens = 190, truncated = 0
11.06.699.162 I srv  update_slots: all slots are idle
11.06.733.181 I srv    operator(): Chat format: peg-native
11.06.733.438 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 0.567 (> 0.100 thold), f_keep = 0.311
11.06.733.440 I srv  get_availabl: updating prompt cache
11.06.733.471 W srv   prompt_save:  - saving prompt with length 190, total state size = 166.175 MiB (draft: 0.000 MiB)
11.06.918.569 I srv          load:  - looking for better prompt, base f_keep = 0.311, sim = 0.567
11.06.918.577 I srv          load:  - found better prompt with f_keep = 0.330, sim = 0.615
11.06.985.044 I srv        update:  - cache state: 19 prompts, 6330.614 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
11.06.985.047 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
11.06.985.048 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
11.06.985.049 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
11.06.985.049 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
11.06.985.050 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
11.06.985.050 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
11.06.985.051 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
11.06.985.052 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
11.06.985.052 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
11.06.985.053 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
11.06.985.053 I srv        update:    - prompt 0x5ae370019d10:    1211 tokens, checkpoints:  1,   338.852 MiB
11.06.985.054 I srv        update:    - prompt 0x5ae3701b33d0:     923 tokens, checkpoints:  1,   336.599 MiB
11.06.985.054 I srv        update:    - prompt 0x5ae370030910:     945 tokens, checkpoints:  1,   336.771 MiB
11.06.985.055 I srv        update:    - prompt 0x5ae3769bd230:     563 tokens, checkpoints:  1,   333.782 MiB
11.06.985.057 I srv        update:    - prompt 0x5ae37040ecb0:     358 tokens, checkpoints:  1,   332.178 MiB
11.06.985.058 I srv        update:    - prompt 0x5ae370219650:     251 tokens, checkpoints:  1,   331.341 MiB
11.06.985.058 I srv        update:    - prompt 0x5ae3703b31a0:     878 tokens, checkpoints:  1,   336.247 MiB
11.06.985.059 I srv        update:    - prompt 0x5ae3705163d0:     407 tokens, checkpoints:  1,   332.561 MiB
11.06.985.059 I srv        update:    - prompt 0x5ae3769f1170:     190 tokens, checkpoints:  1,   330.864 MiB
11.06.985.060 I srv  get_availabl: prompt cache update took 251.62 ms
11.06.985.159 I slot launch_slot_: id  1 | task 7486 | processing task, is_child = 0
11.06.985.159 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
11.06.985.160 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
11.06.985.177 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
11.06.985.177 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
11.06.985.190 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
11.06.985.190 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
11.06.985.211 I slot update_slots: id  1 | task 7486 | Checking checkpoint with [55, 55] against 64...
11.07.027.008 W slot update_slots: id  1 | task 7486 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 164.688 MiB)
11.11.706.540 I slot print_timing: id  1 | task 7486 | prompt processing, n_tokens =     44, progress = 0.96, t =   4.72 s / 9.32 tokens per second
11.18.457.429 I slot print_timing: id  1 | task 7486 | n_decoded =    100, tg =  15.16 t/s, tg_3s =  15.16 t/s
11.21.457.995 I slot print_timing: id  1 | task 7486 | n_decoded =    145, tg =  15.11 t/s, tg_3s =  15.00 t/s
11.24.466.462 I slot print_timing: id  1 | task 7486 | n_decoded =    190, tg =  15.07 t/s, tg_3s =  14.96 t/s
11.27.517.252 I slot print_timing: id  1 | task 7486 | n_decoded =    236, tg =  15.08 t/s, tg_3s =  15.08 t/s
11.30.573.087 I slot print_timing: id  1 | task 7486 | n_decoded =    282, tg =  15.07 t/s, tg_3s =  15.05 t/s
11.30.902.318 I slot print_timing: id  1 | task 7486 | prompt eval time =    4877.00 ms /    48 tokens (  101.60 ms per token,     9.84 tokens per second)
11.30.902.323 I slot print_timing: id  1 | task 7486 |        eval time =   19040.09 ms /   287 tokens (   66.34 ms per token,    15.07 tokens per second)
11.30.902.323 I slot print_timing: id  1 | task 7486 |       total time =   23917.09 ms /   335 tokens
11.30.902.324 I slot print_timing: id  1 | task 7486 |    graphs reused =       7663
11.30.902.461 I slot      release: id  1 | task 7486 | stop processing: n_tokens = 390, truncated = 0
11.30.902.476 I srv  update_slots: all slots are idle
11.30.993.107 I srv    operator(): Chat format: peg-native
11.30.993.369 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 0.615 (> 0.100 thold), f_keep = 0.164
11.30.993.371 I srv  get_availabl: updating prompt cache
11.30.993.423 W srv   prompt_save:  - saving prompt with length 390, total state size = 167.740 MiB (draft: 0.000 MiB)
11.31.168.043 I srv          load:  - looking for better prompt, base f_keep = 0.164, sim = 0.615
11.31.168.055 I srv        update:  - cache state: 20 prompts, 6663.043 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
11.31.168.056 I srv        update:    - prompt 0x5ae3704be4a0:     199 tokens, checkpoints:  1,   330.934 MiB
11.31.168.058 I srv        update:    - prompt 0x5ae3704da3c0:     100 tokens, checkpoints:  1,   330.159 MiB
11.31.168.061 I srv        update:    - prompt 0x5ae3703cec70:     436 tokens, checkpoints:  1,   332.788 MiB
11.31.168.062 I srv        update:    - prompt 0x5ae37048c5a0:     389 tokens, checkpoints:  1,   332.421 MiB
11.31.168.063 I srv        update:    - prompt 0x5ae3704b1500:     152 tokens, checkpoints:  1,   330.566 MiB
11.31.168.063 I srv        update:    - prompt 0x5ae370053970:     211 tokens, checkpoints:  1,   331.028 MiB
11.31.168.064 I srv        update:    - prompt 0x5ae3701f9230:     339 tokens, checkpoints:  1,   332.029 MiB
11.31.168.065 I srv        update:    - prompt 0x5ae3703c7a20:     239 tokens, checkpoints:  1,   331.247 MiB
11.31.168.065 I srv        update:    - prompt 0x5ae37055b2f0:    1045 tokens, checkpoints:  1,   337.553 MiB
11.31.168.066 I srv        update:    - prompt 0x5ae370521490:     424 tokens, checkpoints:  1,   332.694 MiB
11.31.168.066 I srv        update:    - prompt 0x5ae370019d10:    1211 tokens, checkpoints:  1,   338.852 MiB
11.31.168.068 I srv        update:    - prompt 0x5ae3701b33d0:     923 tokens, checkpoints:  1,   336.599 MiB
11.31.168.070 I srv        update:    - prompt 0x5ae370030910:     945 tokens, checkpoints:  1,   336.771 MiB
11.31.168.070 I srv        update:    - prompt 0x5ae3769bd230:     563 tokens, checkpoints:  1,   333.782 MiB
11.31.168.071 I srv        update:    - prompt 0x5ae37040ecb0:     358 tokens, checkpoints:  1,   332.178 MiB
11.31.168.071 I srv        update:    - prompt 0x5ae370219650:     251 tokens, checkpoints:  1,   331.341 MiB
11.31.168.072 I srv        update:    - prompt 0x5ae3703b31a0:     878 tokens, checkpoints:  1,   336.247 MiB
11.31.168.072 I srv        update:    - prompt 0x5ae3705163d0:     407 tokens, checkpoints:  1,   332.561 MiB
11.31.168.074 I srv        update:    - prompt 0x5ae3769f1170:     190 tokens, checkpoints:  1,   330.864 MiB
11.31.168.074 I srv        update:    - prompt 0x5ae370431270:     390 tokens, checkpoints:  1,   332.428 MiB
11.31.168.075 I srv  get_availabl: prompt cache update took 174.70 ms
11.31.168.178 I slot launch_slot_: id  1 | task 7775 | processing task, is_child = 0
11.31.168.179 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
11.31.168.180 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
11.31.168.198 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
11.31.168.199 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
11.31.168.214 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
11.31.168.214 I slot prompt_clear: id  3 | task -1 | clearing prompt with 0 tokens
11.31.168.237 I slot update_slots: id  1 | task 7775 | Checking checkpoint with [55, 55] against 64...
11.31.199.154 W slot update_slots: id  1 | task 7775 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 164.688 MiB)
11.36.125.400 I slot print_timing: id  1 | task 7775 | prompt processing, n_tokens =     44, progress = 0.96, t =   4.96 s / 8.88 tokens per second
11.42.839.526 I slot print_timing: id  1 | task 7775 | n_decoded =    100, tg =  15.25 t/s, tg_3s =  15.25 t/s
11.45.876.291 I slot print_timing: id  1 | task 7775 | n_decoded =    146, tg =  15.22 t/s, tg_3s =  15.15 t/s
11.46.205.552 I slot print_timing: id  1 | task 7775 | prompt eval time =    5114.49 ms /    48 tokens (  106.55 ms per token,     9.39 tokens per second)
11.46.205.557 I slot print_timing: id  1 | task 7775 |        eval time =    9922.81 ms /   151 tokens (   65.71 ms per token,    15.22 tokens per second)
11.46.205.558 I slot print_timing: id  1 | task 7775 |       total time =   15037.30 ms /   199 tokens
11.46.205.559 I slot print_timing: id  1 | task 7775 |    graphs reused =       7812
11.46.205.594 I slot      release: id  1 | task 7775 | stop processing: n_tokens = 254, truncated = 0
11.46.205.610 I srv  update_slots: all slots are idle
11.46.266.564 I srv    operator(): operator(): cleaning up before exit...