0.00.064.449 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.064.453 I device_info:
0.00.064.474 I   - ROCm0   : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.064.478 I   - ROCm1   : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.064.485 I   - CPU     : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.064.542 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
0.00.064.546 I srv  llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.064.586 I srv          init: running without SSL
0.00.064.651 I srv          init: using 23 threads for HTTP server
0.00.064.977 I srv         start: binding port with default address family
0.00.066.171 I srv  llama_server: loading model
0.00.066.178 I srv    load_model: loading model '/home/qrf/.cache/llama.cpp/nvidia_Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf'
0.00.066.227 I common_init_result: fitting params to device memory ...
0.00.066.228 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.16.433.630 W llama_context: n_ctx_seq (16384) < n_ctx_train (1048576) -- the full capacity of the model will not be utilized
0.16.542.772 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.16.731.252 I srv    load_model: initializing slots, n_slots = 4
0.16.763.694 W srv    load_model: speculative decoding will use checkpoints
0.16.763.714 W common_speculative_init: no implementations specified for speculative decoding
0.16.763.715 I slot   load_model: id  0 | task -1 | new slot, n_ctx = 16384
0.16.763.719 I slot   load_model: id  1 | task -1 | new slot, n_ctx = 16384
0.16.763.719 I slot   load_model: id  2 | task -1 | new slot, n_ctx = 16384
0.16.763.720 I slot   load_model: id  3 | task -1 | new slot, n_ctx = 16384
0.16.763.792 I srv    load_model: prompt cache is enabled, size limit: 8192 MiB
0.16.763.792 I srv    load_model: use `--cache-ram 0` to disable the prompt cache
0.16.763.793 I srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.16.763.793 I srv    load_model: context checkpoints enabled, max = 32, min spacing = 256
0.16.763.816 I srv          init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.16.799.188 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
<think></think>Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
<think>
'
0.16.828.637 I srv          init: init: chat template, thinking = 1
0.16.828.673 I srv  llama_server: model loaded
0.16.828.676 I srv  llama_server: server is listening on http://127.0.0.1:8090
0.16.828.681 I srv  update_slots: all slots are idle
0.20.017.197 I srv    operator(): Chat format: peg-native
0.20.020.131 I slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
0.20.020.133 I srv  get_availabl: updating prompt cache
0.20.020.139 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.20.020.144 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est)
0.20.020.145 I srv  get_availabl: prompt cache update took 0.01 ms
0.20.021.055 I slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
0.20.021.057 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.20.021.059 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
0.20.021.073 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.20.021.074 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
0.20.021.082 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.20.021.082 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
0.20.776.522 I slot create_check: id  3 | task 0 | created context checkpoint 1 of 32 (pos_min = 70, pos_max = 70, n_tokens = 71, size = 47.618 MiB)
0.23.935.789 I slot print_timing: id  3 | task 0 | n_decoded =    237, tg =  78.99 t/s, tg_3s =  78.99 t/s
0.26.939.179 I slot print_timing: id  3 | task 0 | n_decoded =    472, tg =  78.62 t/s, tg_3s =  78.24 t/s
0.29.944.634 I slot print_timing: id  3 | task 0 | n_decoded =    704, tg =  78.14 t/s, tg_3s =  77.19 t/s
0.32.952.683 I slot print_timing: id  3 | task 0 | n_decoded =    935, tg =  77.81 t/s, tg_3s =  76.79 t/s
0.35.955.700 I slot print_timing: id  3 | task 0 | n_decoded =   1166, tg =  77.63 t/s, tg_3s =  76.92 t/s
0.38.963.740 I slot print_timing: id  3 | task 0 | n_decoded =   1397, tg =  77.49 t/s, tg_3s =  76.79 t/s
0.41.972.768 I slot print_timing: id  3 | task 0 | n_decoded =   1628, tg =  77.39 t/s, tg_3s =  76.77 t/s
0.44.979.836 I slot print_timing: id  3 | task 0 | n_decoded =   1859, tg =  77.32 t/s, tg_3s =  76.82 t/s
0.47.987.891 I slot print_timing: id  3 | task 0 | n_decoded =   2090, tg =  77.26 t/s, tg_3s =  76.79 t/s
0.50.990.787 I slot print_timing: id  3 | task 0 | n_decoded =   2327, tg =  77.42 t/s, tg_3s =  78.92 t/s
0.54.001.249 I slot print_timing: id  3 | task 0 | n_decoded =   2567, tg =  77.63 t/s, tg_3s =  79.72 t/s
0.57.008.104 I slot print_timing: id  3 | task 0 | n_decoded =   2805, tg =  77.76 t/s, tg_3s =  79.15 t/s
0.57.489.311 I slot print_timing: id  3 | task 0 | prompt eval time =     914.38 ms /   100 tokens (    9.14 ms per token,   109.36 tokens per second)
0.57.489.315 I slot print_timing: id  3 | task 0 |        eval time =   36553.83 ms /  2843 tokens (   12.86 ms per token,    77.78 tokens per second)
0.57.489.315 I slot print_timing: id  3 | task 0 |       total time =   37468.20 ms /  2943 tokens
0.57.489.320 I slot print_timing: id  3 | task 0 |    graphs reused =       2831
0.57.489.369 I slot      release: id  3 | task 0 | stop processing: n_tokens = 2942, truncated = 0
0.57.489.383 I srv  update_slots: all slots are idle
0.57.523.273 I srv    operator(): Chat format: peg-native
0.57.526.184 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.779 (> 0.100 thold), f_keep = 0.025
0.57.526.186 I srv  get_availabl: updating prompt cache
0.57.526.305 W srv   prompt_save:  - saving prompt with length 2942, total state size = 64.890 MiB (draft: 0.000 MiB)
0.57.617.139 I srv          load:  - looking for better prompt, base f_keep = 0.025, sim = 0.779
0.57.617.145 I srv        update:  - cache state: 1 prompts, 112.508 MiB (limits: 8192.000 MiB, 16384 tokens, 214215 est)
0.57.617.146 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
0.57.617.147 I srv  get_availabl: prompt cache update took 90.96 ms
0.57.618.029 I slot launch_slot_: id  3 | task 2846 | processing task, is_child = 0
0.57.618.030 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.57.618.032 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
0.57.618.044 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.57.618.044 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
0.57.618.053 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.57.618.054 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
0.57.618.070 I slot update_slots: id  3 | task 2846 | Checking checkpoint with [70, 70] against 74...
0.57.629.944 W slot update_slots: id  3 | task 2846 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
1.00.775.198 I slot print_timing: id  3 | task 2846 | n_decoded =    241, tg =  80.07 t/s, tg_3s =  80.07 t/s
1.03.781.804 I slot print_timing: id  3 | task 2846 | n_decoded =    481, tg =  79.95 t/s, tg_3s =  79.82 t/s
1.06.794.105 I slot print_timing: id  3 | task 2846 | n_decoded =    721, tg =  79.86 t/s, tg_3s =  79.67 t/s
1.09.802.132 I slot print_timing: id  3 | task 2846 | n_decoded =    959, tg =  79.67 t/s, tg_3s =  79.12 t/s
1.12.811.904 I slot print_timing: id  3 | task 2846 | n_decoded =   1196, tg =  79.49 t/s, tg_3s =  78.74 t/s
1.15.813.817 I slot print_timing: id  3 | task 2846 | n_decoded =   1431, tg =  79.29 t/s, tg_3s =  78.28 t/s
1.18.819.801 I slot print_timing: id  3 | task 2846 | n_decoded =   1664, tg =  79.03 t/s, tg_3s =  77.51 t/s
1.19.065.825 I slot print_timing: id  3 | task 2846 | prompt eval time =     147.33 ms /    24 tokens (    6.14 ms per token,   162.90 tokens per second)
1.19.065.829 I slot print_timing: id  3 | task 2846 |        eval time =   21300.42 ms /  1683 tokens (   12.66 ms per token,    79.01 tokens per second)
1.19.065.829 I slot print_timing: id  3 | task 2846 |       total time =   21447.75 ms /  1707 tokens
1.19.065.830 I slot print_timing: id  3 | task 2846 |    graphs reused =       4506
1.19.065.867 I slot      release: id  3 | task 2846 | stop processing: n_tokens = 1777, truncated = 0
1.19.065.885 I srv  update_slots: all slots are idle
1.19.099.640 I srv    operator(): Chat format: peg-native
1.19.102.529 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.779 (> 0.100 thold), f_keep = 0.042
1.19.102.531 I srv  get_availabl: updating prompt cache
1.19.102.609 W srv   prompt_save:  - saving prompt with length 1777, total state size = 58.050 MiB (draft: 0.000 MiB)
1.19.207.295 I srv          load:  - looking for better prompt, base f_keep = 0.042, sim = 0.779
1.19.207.301 I srv        update:  - cache state: 2 prompts, 218.176 MiB (limits: 8192.000 MiB, 16384 tokens, 177187 est)
1.19.207.302 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
1.19.207.303 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
1.19.207.304 I srv  get_availabl: prompt cache update took 104.77 ms
1.19.208.201 I slot launch_slot_: id  3 | task 4531 | processing task, is_child = 0
1.19.208.203 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.19.208.204 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
1.19.208.220 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
1.19.208.221 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
1.19.208.230 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
1.19.208.230 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
1.19.208.246 I slot update_slots: id  3 | task 4531 | Checking checkpoint with [70, 70] against 74...
1.19.220.360 W slot update_slots: id  3 | task 4531 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
1.22.361.851 I slot print_timing: id  3 | task 4531 | n_decoded =    236, tg =  78.44 t/s, tg_3s =  78.44 t/s
1.25.370.850 I slot print_timing: id  3 | task 4531 | n_decoded =    469, tg =  77.94 t/s, tg_3s =  77.43 t/s
1.28.377.896 I slot print_timing: id  3 | task 4531 | n_decoded =    700, tg =  77.56 t/s, tg_3s =  76.82 t/s
1.30.717.901 I slot print_timing: id  3 | task 4531 | prompt eval time =     144.90 ms /    24 tokens (    6.04 ms per token,   165.63 tokens per second)
1.30.717.904 I slot print_timing: id  3 | task 4531 |        eval time =   11364.75 ms /   880 tokens (   12.91 ms per token,    77.43 tokens per second)
1.30.717.905 I slot print_timing: id  3 | task 4531 |       total time =   11509.65 ms /   904 tokens
1.30.717.905 I slot print_timing: id  3 | task 4531 |    graphs reused =       5381
1.30.717.933 I slot      release: id  3 | task 4531 | stop processing: n_tokens = 974, truncated = 0
1.30.717.944 I srv  update_slots: all slots are idle
1.30.753.017 I srv    operator(): Chat format: peg-native
1.30.756.089 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.787 (> 0.100 thold), f_keep = 0.076
1.30.756.091 I srv  get_availabl: updating prompt cache
1.30.756.142 W srv   prompt_save:  - saving prompt with length 974, total state size = 53.336 MiB (draft: 0.000 MiB)
1.30.834.442 I srv          load:  - looking for better prompt, base f_keep = 0.076, sim = 0.787
1.30.834.449 I srv        update:  - cache state: 3 prompts, 319.129 MiB (limits: 8192.000 MiB, 16384 tokens, 146138 est)
1.30.834.451 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
1.30.834.451 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
1.30.834.452 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
1.30.834.453 I srv  get_availabl: prompt cache update took 78.36 ms
1.30.835.329 I slot launch_slot_: id  3 | task 5413 | processing task, is_child = 0
1.30.835.331 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.30.835.332 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
1.30.835.344 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
1.30.835.344 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
1.30.835.353 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
1.30.835.353 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
1.30.835.368 I slot update_slots: id  3 | task 5413 | Checking checkpoint with [70, 70] against 74...
1.30.847.368 W slot update_slots: id  3 | task 5413 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
1.33.988.906 I slot print_timing: id  3 | task 5413 | n_decoded =    232, tg =  77.02 t/s, tg_3s =  77.02 t/s
1.36.991.512 I slot print_timing: id  3 | task 5413 | n_decoded =    468, tg =  77.81 t/s, tg_3s =  78.60 t/s
1.39.996.565 I slot print_timing: id  3 | task 5413 | n_decoded =    708, tg =  78.49 t/s, tg_3s =  79.87 t/s
1.43.000.599 I slot print_timing: id  3 | task 5413 | n_decoded =    947, tg =  78.76 t/s, tg_3s =  79.56 t/s
1.46.007.498 I slot print_timing: id  3 | task 5413 | n_decoded =   1185, tg =  78.84 t/s, tg_3s =  79.15 t/s
1.49.016.495 I slot print_timing: id  3 | task 5413 | n_decoded =   1422, tg =  78.83 t/s, tg_3s =  78.76 t/s
1.49.338.471 I slot print_timing: id  3 | task 5413 | prompt eval time =     141.47 ms /    23 tokens (    6.15 ms per token,   162.58 tokens per second)
1.49.338.475 I slot print_timing: id  3 | task 5413 |        eval time =   18361.63 ms /  1447 tokens (   12.69 ms per token,    78.81 tokens per second)
1.49.338.476 I slot print_timing: id  3 | task 5413 |       total time =   18503.10 ms /  1470 tokens
1.49.338.476 I slot print_timing: id  3 | task 5413 |    graphs reused =       6820
1.49.338.507 I slot      release: id  3 | task 5413 | stop processing: n_tokens = 1540, truncated = 0
1.49.338.524 I srv  update_slots: all slots are idle
1.49.371.744 I srv    operator(): Chat format: peg-native
1.49.374.619 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.692 (> 0.100 thold), f_keep = 0.048
1.49.374.620 I srv  get_availabl: updating prompt cache
1.49.374.681 W srv   prompt_save:  - saving prompt with length 1540, total state size = 56.659 MiB (draft: 0.000 MiB)
1.49.439.735 I srv          load:  - looking for better prompt, base f_keep = 0.048, sim = 0.692
1.49.439.743 I srv        update:  - cache state: 4 prompts, 423.406 MiB (limits: 8192.000 MiB, 16384 tokens, 139943 est)
1.49.439.744 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
1.49.439.745 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
1.49.439.745 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
1.49.439.747 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
1.49.439.749 I srv  get_availabl: prompt cache update took 65.13 ms
1.49.440.631 I slot launch_slot_: id  3 | task 6862 | processing task, is_child = 0
1.49.440.633 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.49.440.634 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
1.49.440.650 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
1.49.440.650 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
1.49.440.659 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
1.49.440.660 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
1.49.440.677 I slot update_slots: id  3 | task 6862 | Checking checkpoint with [70, 70] against 74...
1.49.452.601 W slot update_slots: id  3 | task 6862 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
1.52.638.499 I slot print_timing: id  3 | task 6862 | n_decoded =    239, tg =  79.33 t/s, tg_3s =  79.33 t/s
1.55.647.031 I slot print_timing: id  3 | task 6862 | n_decoded =    475, tg =  78.89 t/s, tg_3s =  78.44 t/s
1.58.659.541 I slot print_timing: id  3 | task 6862 | n_decoded =    710, tg =  78.59 t/s, tg_3s =  78.01 t/s
2.01.669.547 I slot print_timing: id  3 | task 6862 | n_decoded =    941, tg =  78.13 t/s, tg_3s =  76.74 t/s
2.04.672.572 I slot print_timing: id  3 | task 6862 | n_decoded =   1172, tg =  77.89 t/s, tg_3s =  76.92 t/s
2.07.681.639 I slot print_timing: id  3 | task 6862 | n_decoded =   1403, tg =  77.70 t/s, tg_3s =  76.77 t/s
2.10.689.681 I slot print_timing: id  3 | task 6862 | n_decoded =   1634, tg =  77.57 t/s, tg_3s =  76.79 t/s
2.13.697.714 I slot print_timing: id  3 | task 6862 | n_decoded =   1865, tg =  77.48 t/s, tg_3s =  76.79 t/s
2.16.705.725 I slot print_timing: id  3 | task 6862 | n_decoded =   2096, tg =  77.40 t/s, tg_3s =  76.79 t/s
2.19.713.753 I slot print_timing: id  3 | task 6862 | n_decoded =   2327, tg =  77.34 t/s, tg_3s =  76.79 t/s
2.21.325.808 I slot print_timing: id  3 | task 6862 | prompt eval time =     185.10 ms /    36 tokens (    5.14 ms per token,   194.49 tokens per second)
2.21.325.811 I slot print_timing: id  3 | task 6862 |        eval time =   31700.03 ms /  2451 tokens (   12.93 ms per token,    77.32 tokens per second)
2.21.325.812 I slot print_timing: id  3 | task 6862 |       total time =   31885.13 ms /  2487 tokens
2.21.325.812 I slot print_timing: id  3 | task 6862 |    graphs reused =       9260
2.21.325.846 I slot      release: id  3 | task 6862 | stop processing: n_tokens = 2557, truncated = 0
2.21.325.868 I srv  update_slots: all slots are idle
2.21.361.352 I srv    operator(): Chat format: peg-native
2.21.364.404 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.771 (> 0.100 thold), f_keep = 0.029
2.21.364.405 I srv  get_availabl: updating prompt cache
2.21.364.494 W srv   prompt_save:  - saving prompt with length 2557, total state size = 62.630 MiB (draft: 0.000 MiB)
2.21.474.063 I srv          load:  - looking for better prompt, base f_keep = 0.029, sim = 0.771
2.21.474.069 I srv        update:  - cache state: 5 prompts, 533.653 MiB (limits: 8192.000 MiB, 16384 tokens, 150284 est)
2.21.474.070 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
2.21.474.071 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
2.21.474.072 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
2.21.474.076 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
2.21.474.076 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
2.21.474.077 I srv  get_availabl: prompt cache update took 109.67 ms
2.21.474.945 I slot launch_slot_: id  3 | task 9315 | processing task, is_child = 0
2.21.474.947 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
2.21.474.947 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
2.21.474.961 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
2.21.474.961 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
2.21.474.970 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
2.21.474.971 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
2.21.474.986 I slot update_slots: id  3 | task 9315 | Checking checkpoint with [70, 70] against 74...
2.21.486.867 W slot update_slots: id  3 | task 9315 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
2.24.621.750 I slot print_timing: id  3 | task 9315 | n_decoded =    231, tg =  76.93 t/s, tg_3s =  76.93 t/s
2.27.629.795 I slot print_timing: id  3 | task 9315 | n_decoded =    462, tg =  76.86 t/s, tg_3s =  76.79 t/s
2.30.635.137 I slot print_timing: id  3 | task 9315 | n_decoded =    698, tg =  77.42 t/s, tg_3s =  78.53 t/s
2.33.639.707 I slot print_timing: id  3 | task 9315 | n_decoded =    938, tg =  78.03 t/s, tg_3s =  79.88 t/s
2.36.643.176 I slot print_timing: id  3 | task 9315 | n_decoded =   1178, tg =  78.41 t/s, tg_3s =  79.91 t/s
2.39.651.678 I slot print_timing: id  3 | task 9315 | n_decoded =   1416, tg =  78.52 t/s, tg_3s =  79.11 t/s
2.42.659.675 I slot print_timing: id  3 | task 9315 | n_decoded =   1652, tg =  78.52 t/s, tg_3s =  78.46 t/s
2.45.670.228 I slot print_timing: id  3 | task 9315 | n_decoded =   1886, tg =  78.42 t/s, tg_3s =  77.73 t/s
2.48.674.721 I slot print_timing: id  3 | task 9315 | n_decoded =   2117, tg =  78.25 t/s, tg_3s =  76.88 t/s
2.51.682.768 I slot print_timing: id  3 | task 9315 | n_decoded =   2348, tg =  78.10 t/s, tg_3s =  76.79 t/s
2.54.689.830 I slot print_timing: id  3 | task 9315 | n_decoded =   2579, tg =  77.98 t/s, tg_3s =  76.82 t/s
2.55.846.823 I slot print_timing: id  3 | task 9315 | prompt eval time =     144.17 ms /    25 tokens (    5.77 ms per token,   173.41 tokens per second)
2.55.846.826 I slot print_timing: id  3 | task 9315 |        eval time =   34227.67 ms /  2668 tokens (   12.83 ms per token,    77.95 tokens per second)
2.55.846.827 I slot print_timing: id  3 | task 9315 |       total time =   34371.83 ms /  2693 tokens
2.55.846.828 I slot print_timing: id  3 | task 9315 |    graphs reused =      11916
2.55.846.855 I slot      release: id  3 | task 9315 | stop processing: n_tokens = 2763, truncated = 0
2.55.846.877 I srv  update_slots: all slots are idle
2.55.880.165 I srv    operator(): Chat format: peg-native
2.55.883.037 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.815 (> 0.100 thold), f_keep = 0.027
2.55.883.039 I srv  get_availabl: updating prompt cache
2.55.883.156 W srv   prompt_save:  - saving prompt with length 2763, total state size = 63.839 MiB (draft: 0.000 MiB)
2.55.961.567 I srv          load:  - looking for better prompt, base f_keep = 0.027, sim = 0.815
2.55.961.575 I srv        update:  - cache state: 6 prompts, 645.110 MiB (limits: 8192.000 MiB, 16384 tokens, 159405 est)
2.55.961.576 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
2.55.961.577 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
2.55.961.578 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
2.55.961.578 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
2.55.961.579 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
2.55.961.579 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
2.55.961.580 I srv  get_availabl: prompt cache update took 78.54 ms
2.55.962.454 I slot launch_slot_: id  3 | task 11985 | processing task, is_child = 0
2.55.962.456 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
2.55.962.457 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
2.55.962.469 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
2.55.962.469 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
2.55.962.479 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
2.55.962.480 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
2.55.962.496 I slot update_slots: id  3 | task 11985 | Checking checkpoint with [70, 70] against 75...
2.55.974.515 W slot update_slots: id  3 | task 11985 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
2.59.109.722 I slot print_timing: id  3 | task 11985 | n_decoded =    232, tg =  77.15 t/s, tg_3s =  77.15 t/s
3.02.118.805 I slot print_timing: id  3 | task 11985 | n_decoded =    463, tg =  76.96 t/s, tg_3s =  76.77 t/s
3.05.126.822 I slot print_timing: id  3 | task 11985 | n_decoded =    694, tg =  76.90 t/s, tg_3s =  76.79 t/s
3.08.129.831 I slot print_timing: id  3 | task 11985 | n_decoded =    925, tg =  76.91 t/s, tg_3s =  76.92 t/s
3.11.137.888 I slot print_timing: id  3 | task 11985 | n_decoded =   1156, tg =  76.89 t/s, tg_3s =  76.79 t/s
3.12.610.939 I slot print_timing: id  3 | task 11985 | prompt eval time =     139.98 ms /    21 tokens (    6.67 ms per token,   150.02 tokens per second)
3.12.610.942 I slot print_timing: id  3 | task 11985 |        eval time =   16508.46 ms /  1269 tokens (   13.01 ms per token,    76.87 tokens per second)
3.12.610.943 I slot print_timing: id  3 | task 11985 |       total time =   16648.44 ms /  1290 tokens
3.12.610.944 I slot print_timing: id  3 | task 11985 |    graphs reused =      13178
3.12.610.973 I slot      release: id  3 | task 11985 | stop processing: n_tokens = 1360, truncated = 0
3.12.610.984 I srv  update_slots: all slots are idle
3.12.644.399 I srv    operator(): Chat format: peg-native
3.12.647.441 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.798 (> 0.100 thold), f_keep = 0.055
3.12.647.443 I srv  get_availabl: updating prompt cache
3.12.647.495 W srv   prompt_save:  - saving prompt with length 1360, total state size = 55.602 MiB (draft: 0.000 MiB)
3.12.729.955 I srv          load:  - looking for better prompt, base f_keep = 0.055, sim = 0.798
3.12.729.965 I srv        update:  - cache state: 7 prompts, 748.330 MiB (limits: 8192.000 MiB, 16384 tokens, 152306 est)
3.12.729.966 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
3.12.729.970 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
3.12.729.970 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
3.12.729.971 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
3.12.729.972 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
3.12.729.972 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
3.12.729.973 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
3.12.729.974 I srv  get_availabl: prompt cache update took 82.53 ms
3.12.730.867 I slot launch_slot_: id  3 | task 13256 | processing task, is_child = 0
3.12.730.869 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
3.12.730.870 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
3.12.730.887 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
3.12.730.887 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
3.12.730.898 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
3.12.730.899 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
3.12.730.915 I slot update_slots: id  3 | task 13256 | Checking checkpoint with [70, 70] against 75...
3.12.742.730 W slot update_slots: id  3 | task 13256 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
3.15.883.473 I slot print_timing: id  3 | task 13256 | n_decoded =    234, tg =  77.70 t/s, tg_3s =  77.70 t/s
3.18.889.942 I slot print_timing: id  3 | task 13256 | n_decoded =    474, tg =  78.76 t/s, tg_3s =  79.83 t/s
3.21.900.449 I slot print_timing: id  3 | task 13256 | n_decoded =    713, tg =  78.97 t/s, tg_3s =  79.39 t/s
3.24.908.491 I slot print_timing: id  3 | task 13256 | n_decoded =    951, tg =  79.01 t/s, tg_3s =  79.12 t/s
3.27.916.579 I slot print_timing: id  3 | task 13256 | n_decoded =   1188, tg =  78.96 t/s, tg_3s =  78.79 t/s
3.28.455.525 I slot print_timing: id  3 | task 13256 | prompt eval time =     140.94 ms /    23 tokens (    6.13 ms per token,   163.19 tokens per second)
3.28.455.528 I slot print_timing: id  3 | task 13256 |        eval time =   15583.66 ms /  1230 tokens (   12.67 ms per token,    78.93 tokens per second)
3.28.455.529 I slot print_timing: id  3 | task 13256 |       total time =   15724.61 ms /  1253 tokens
3.28.455.530 I slot print_timing: id  3 | task 13256 |    graphs reused =      14401
3.28.455.561 I slot      release: id  3 | task 13256 | stop processing: n_tokens = 1323, truncated = 0
3.28.455.573 I srv  update_slots: all slots are idle
3.28.489.778 I srv    operator(): Chat format: peg-native
3.28.492.674 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.607 (> 0.100 thold), f_keep = 0.056
3.28.492.676 I srv  get_availabl: updating prompt cache
3.28.492.732 W srv   prompt_save:  - saving prompt with length 1323, total state size = 55.385 MiB (draft: 0.000 MiB)
3.28.567.685 I srv          load:  - looking for better prompt, base f_keep = 0.056, sim = 0.607
3.28.567.694 I srv        update:  - cache state: 8 prompts, 851.333 MiB (limits: 8192.000 MiB, 16384 tokens, 146609 est)
3.28.567.695 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
3.28.567.696 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
3.28.567.697 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
3.28.567.697 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
3.28.567.698 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
3.28.567.699 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
3.28.567.700 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
3.28.567.701 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
3.28.567.702 I srv  get_availabl: prompt cache update took 75.03 ms
3.28.568.598 I slot launch_slot_: id  3 | task 14488 | processing task, is_child = 0
3.28.568.600 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
3.28.568.601 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
3.28.568.616 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
3.28.568.617 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
3.28.568.626 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
3.28.568.626 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
3.28.568.642 I slot update_slots: id  3 | task 14488 | Checking checkpoint with [70, 70] against 74...
3.28.580.678 W slot update_slots: id  3 | task 14488 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
3.31.835.526 I slot print_timing: id  3 | task 14488 | n_decoded =    237, tg =  78.82 t/s, tg_3s =  78.82 t/s
3.34.839.557 I slot print_timing: id  3 | task 14488 | n_decoded =    472, tg =  78.52 t/s, tg_3s =  78.23 t/s
3.37.851.532 I slot print_timing: id  3 | task 14488 | n_decoded =    707, tg =  78.36 t/s, tg_3s =  78.02 t/s
3.40.855.512 I slot print_timing: id  3 | task 14488 | n_decoded =    939, tg =  78.07 t/s, tg_3s =  77.23 t/s
3.40.894.501 I slot print_timing: id  3 | task 14488 | prompt eval time =     259.86 ms /    51 tokens (    5.10 ms per token,   196.26 tokens per second)
3.40.894.504 I slot print_timing: id  3 | task 14488 |        eval time =   12066.00 ms /   942 tokens (   12.81 ms per token,    78.07 tokens per second)
3.40.894.504 I slot print_timing: id  3 | task 14488 |       total time =   12325.86 ms /   993 tokens
3.40.894.505 I slot print_timing: id  3 | task 14488 |    graphs reused =      15337
3.40.894.537 I slot      release: id  3 | task 14488 | stop processing: n_tokens = 1063, truncated = 0
3.40.894.547 I srv  update_slots: all slots are idle
3.40.927.909 I srv    operator(): Chat format: peg-native
3.40.930.781 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.528 (> 0.100 thold), f_keep = 0.071
3.40.930.782 I srv  get_availabl: updating prompt cache
3.40.930.846 W srv   prompt_save:  - saving prompt with length 1063, total state size = 53.859 MiB (draft: 0.000 MiB)
3.40.994.700 I srv          load:  - looking for better prompt, base f_keep = 0.071, sim = 0.528
3.40.994.710 I srv        update:  - cache state: 9 prompts, 952.809 MiB (limits: 8192.000 MiB, 16384 tokens, 140134 est)
3.40.994.711 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
3.40.994.712 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
3.40.994.712 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
3.40.994.713 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
3.40.994.713 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
3.40.994.714 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
3.40.994.714 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
3.40.994.715 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
3.40.994.716 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
3.40.994.721 I srv  get_availabl: prompt cache update took 63.94 ms
3.40.995.640 I slot launch_slot_: id  3 | task 15432 | processing task, is_child = 0
3.40.995.642 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
3.40.995.643 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
3.40.995.657 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
3.40.995.657 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
3.40.995.666 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
3.40.995.666 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
3.40.995.682 I slot update_slots: id  3 | task 15432 | Checking checkpoint with [70, 70] against 75...
3.41.007.789 W slot update_slots: id  3 | task 15432 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
3.44.251.082 I slot print_timing: id  3 | task 15432 | n_decoded =    233, tg =  77.66 t/s, tg_3s =  77.66 t/s
3.47.259.556 I slot print_timing: id  3 | task 15432 | n_decoded =    464, tg =  77.22 t/s, tg_3s =  76.78 t/s
3.50.267.618 I slot print_timing: id  3 | task 15432 | n_decoded =    695, tg =  77.08 t/s, tg_3s =  76.79 t/s
3.53.274.651 I slot print_timing: id  3 | task 15432 | n_decoded =    926, tg =  77.01 t/s, tg_3s =  76.82 t/s
3.56.281.700 I slot print_timing: id  3 | task 15432 | n_decoded =   1157, tg =  76.97 t/s, tg_3s =  76.82 t/s
3.59.284.723 I slot print_timing: id  3 | task 15432 | n_decoded =   1388, tg =  76.97 t/s, tg_3s =  76.92 t/s
4.02.149.753 I slot print_timing: id  3 | task 15432 | prompt eval time =     255.12 ms /    71 tokens (    3.59 ms per token,   278.30 tokens per second)
4.02.149.757 I slot print_timing: id  3 | task 15432 |        eval time =   20898.95 ms /  1608 tokens (   13.00 ms per token,    76.94 tokens per second)
4.02.149.758 I slot print_timing: id  3 | task 15432 |       total time =   21154.07 ms /  1679 tokens
4.02.149.758 I slot print_timing: id  3 | task 15432 |    graphs reused =      16937
4.02.149.792 I slot      release: id  3 | task 15432 | stop processing: n_tokens = 1749, truncated = 0
4.02.149.805 I srv  update_slots: all slots are idle
4.02.183.677 I srv    operator(): Chat format: peg-native
4.02.186.582 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.632 (> 0.100 thold), f_keep = 0.042
4.02.186.584 I srv  get_availabl: updating prompt cache
4.02.186.671 W srv   prompt_save:  - saving prompt with length 1749, total state size = 57.886 MiB (draft: 0.000 MiB)
4.02.264.087 I srv          load:  - looking for better prompt, base f_keep = 0.042, sim = 0.632
4.02.264.098 I srv        update:  - cache state: 10 prompts, 1058.313 MiB (limits: 8192.000 MiB, 16384 tokens, 139702 est)
4.02.264.099 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
4.02.264.104 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
4.02.264.104 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
4.02.264.105 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
4.02.264.105 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
4.02.264.106 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
4.02.264.107 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
4.02.264.110 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
4.02.264.110 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
4.02.264.111 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
4.02.264.112 I srv  get_availabl: prompt cache update took 77.53 ms
4.02.265.067 I slot launch_slot_: id  3 | task 17042 | processing task, is_child = 0
4.02.265.069 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
4.02.265.069 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
4.02.265.087 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
4.02.265.087 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
4.02.265.096 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
4.02.265.096 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
4.02.265.113 I slot update_slots: id  3 | task 17042 | Checking checkpoint with [70, 70] against 74...
4.02.277.235 W slot update_slots: id  3 | task 17042 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
4.05.521.708 I slot print_timing: id  3 | task 17042 | n_decoded =    228, tg =  75.88 t/s, tg_3s =  75.88 t/s
4.08.529.775 I slot print_timing: id  3 | task 17042 | n_decoded =    384, tg =  63.86 t/s, tg_3s =  51.86 t/s
4.11.530.577 I slot print_timing: id  3 | task 17042 | n_decoded =    542, tg =  60.13 t/s, tg_3s =  52.65 t/s
4.14.596.510 I slot print_timing: id  3 | task 17042 | n_decoded =    684, tg =  56.63 t/s, tg_3s =  46.32 t/s
4.16.241.278 I slot print_timing: id  3 | task 17042 | prompt eval time =     251.91 ms /    46 tokens (    5.48 ms per token,   182.60 tokens per second)
4.16.241.281 I slot print_timing: id  3 | task 17042 |        eval time =   13724.24 ms /   779 tokens (   17.62 ms per token,    56.76 tokens per second)
4.16.241.282 I slot print_timing: id  3 | task 17042 |       total time =   13976.15 ms /   825 tokens
4.16.241.283 I slot print_timing: id  3 | task 17042 |    graphs reused =      17711
4.16.242.002 I slot      release: id  3 | task 17042 | stop processing: n_tokens = 895, truncated = 0
4.16.242.342 I srv  update_slots: all slots are idle
4.16.292.693 I srv    operator(): Chat format: peg-native
4.16.297.697 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.525 (> 0.100 thold), f_keep = 0.083
4.16.297.699 I srv  get_availabl: updating prompt cache
4.16.297.992 W srv   prompt_save:  - saving prompt with length 895, total state size = 52.872 MiB (draft: 0.000 MiB)
4.16.364.219 I srv          load:  - looking for better prompt, base f_keep = 0.083, sim = 0.525
4.16.364.228 I srv        update:  - cache state: 11 prompts, 1158.803 MiB (limits: 8192.000 MiB, 16384 tokens, 133914 est)
4.16.364.229 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
4.16.364.233 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
4.16.364.233 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
4.16.364.234 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
4.16.364.234 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
4.16.364.235 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
4.16.364.235 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
4.16.364.236 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
4.16.364.246 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
4.16.364.247 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
4.16.364.247 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
4.16.364.249 I srv  get_availabl: prompt cache update took 66.55 ms
4.16.365.229 I slot launch_slot_: id  3 | task 17823 | processing task, is_child = 0
4.16.365.231 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
4.16.365.232 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
4.16.365.244 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
4.16.365.244 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
4.16.365.252 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
4.16.365.252 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
4.16.365.267 I slot update_slots: id  3 | task 17823 | Checking checkpoint with [70, 70] against 74...
4.16.377.208 W slot update_slots: id  3 | task 17823 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
4.19.631.698 I slot print_timing: id  3 | task 17823 | n_decoded =    237, tg =  78.83 t/s, tg_3s =  78.83 t/s
4.22.637.659 I slot print_timing: id  3 | task 17823 | n_decoded =    472, tg =  78.50 t/s, tg_3s =  78.18 t/s
4.23.355.633 I slot print_timing: id  3 | task 17823 | prompt eval time =     259.90 ms /    70 tokens (    3.71 ms per token,   269.33 tokens per second)
4.23.355.637 I slot print_timing: id  3 | task 17823 |        eval time =    6730.45 ms /   528 tokens (   12.75 ms per token,    78.45 tokens per second)
4.23.355.637 I slot print_timing: id  3 | task 17823 |       total time =    6990.35 ms /   598 tokens
4.23.355.638 I slot print_timing: id  3 | task 17823 |    graphs reused =      18235
4.23.355.747 I slot      release: id  3 | task 17823 | stop processing: n_tokens = 668, truncated = 0
4.23.355.758 I srv  update_slots: all slots are idle
4.23.390.602 I srv    operator(): Chat format: peg-native
4.23.393.472 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.552 (> 0.100 thold), f_keep = 0.111
4.23.393.473 I srv  get_availabl: updating prompt cache
4.23.393.515 W srv   prompt_save:  - saving prompt with length 668, total state size = 51.540 MiB (draft: 0.000 MiB)
4.23.491.891 I srv          load:  - looking for better prompt, base f_keep = 0.111, sim = 0.552
4.23.491.902 I srv        update:  - cache state: 12 prompts, 1257.960 MiB (limits: 8192.000 MiB, 16384 tokens, 127709 est)
4.23.491.904 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
4.23.491.908 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
4.23.491.908 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
4.23.491.920 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
4.23.491.921 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
4.23.491.922 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
4.23.491.922 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
4.23.491.923 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
4.23.491.923 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
4.23.491.924 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
4.23.491.924 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
4.23.491.925 I srv        update:    - prompt 0x56c94da5c020:     668 tokens, checkpoints:  1,    99.157 MiB
4.23.491.926 I srv  get_availabl: prompt cache update took 98.45 ms
4.23.492.802 I slot launch_slot_: id  3 | task 18353 | processing task, is_child = 0
4.23.492.803 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
4.23.492.804 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
4.23.492.816 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
4.23.492.817 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
4.23.492.832 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
4.23.492.833 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
4.23.492.848 I slot update_slots: id  3 | task 18353 | Checking checkpoint with [70, 70] against 74...
4.23.504.820 W slot update_slots: id  3 | task 18353 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
4.26.854.128 I slot print_timing: id  3 | task 18353 | n_decoded =    232, tg =  77.25 t/s, tg_3s =  77.25 t/s
4.29.866.627 I slot print_timing: id  3 | task 18353 | n_decoded =    464, tg =  77.13 t/s, tg_3s =  77.01 t/s
4.32.875.705 I slot print_timing: id  3 | task 18353 | n_decoded =    695, tg =  77.01 t/s, tg_3s =  76.77 t/s
4.35.505.713 I slot print_timing: id  3 | task 18353 | prompt eval time =     358.06 ms /    63 tokens (    5.68 ms per token,   175.95 tokens per second)
4.35.505.717 I slot print_timing: id  3 | task 18353 |        eval time =   11654.80 ms /   897 tokens (   12.99 ms per token,    76.96 tokens per second)
4.35.505.718 I slot print_timing: id  3 | task 18353 |       total time =   12012.86 ms /   960 tokens
4.35.505.718 I slot print_timing: id  3 | task 18353 |    graphs reused =      19126
4.35.505.751 I slot      release: id  3 | task 18353 | stop processing: n_tokens = 1030, truncated = 0
4.35.505.761 I srv  update_slots: all slots are idle
4.35.541.048 I srv    operator(): Chat format: peg-native
4.35.543.903 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.605 (> 0.100 thold), f_keep = 0.073
4.35.543.916 I srv  get_availabl: updating prompt cache
4.35.543.975 W srv   prompt_save:  - saving prompt with length 1030, total state size = 53.665 MiB (draft: 0.000 MiB)
4.35.643.909 I srv          load:  - looking for better prompt, base f_keep = 0.073, sim = 0.605
4.35.643.920 I srv        update:  - cache state: 13 prompts, 1359.243 MiB (limits: 8192.000 MiB, 16384 tokens, 124400 est)
4.35.643.924 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
4.35.643.925 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
4.35.643.925 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
4.35.643.926 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
4.35.643.927 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
4.35.643.928 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
4.35.643.928 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
4.35.643.929 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
4.35.643.929 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
4.35.643.930 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
4.35.643.930 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
4.35.643.938 I srv        update:    - prompt 0x56c94da5c020:     668 tokens, checkpoints:  1,    99.157 MiB
4.35.643.938 I srv        update:    - prompt 0x56c94dab24f0:    1030 tokens, checkpoints:  1,   101.283 MiB
4.35.643.939 I srv  get_availabl: prompt cache update took 100.02 ms
4.35.644.824 I slot launch_slot_: id  3 | task 19252 | processing task, is_child = 0
4.35.644.826 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
4.35.644.827 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
4.35.644.840 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
4.35.644.840 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
4.35.644.849 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
4.35.644.849 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
4.35.644.870 I slot update_slots: id  3 | task 19252 | Checking checkpoint with [70, 70] against 75...
4.35.657.049 W slot update_slots: id  3 | task 19252 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
4.38.970.687 I slot print_timing: id  3 | task 19252 | n_decoded =    231, tg =  76.70 t/s, tg_3s =  76.70 t/s
4.41.980.729 I slot print_timing: id  3 | task 19252 | n_decoded =    462, tg =  76.72 t/s, tg_3s =  76.74 t/s
4.42.955.854 I slot print_timing: id  3 | task 19252 | prompt eval time =     314.26 ms /    53 tokens (    5.93 ms per token,   168.65 tokens per second)
4.42.955.864 I slot print_timing: id  3 | task 19252 |        eval time =    6996.72 ms /   537 tokens (   13.03 ms per token,    76.75 tokens per second)
4.42.955.865 I slot print_timing: id  3 | task 19252 |       total time =    7310.98 ms /   590 tokens
4.42.955.866 I slot print_timing: id  3 | task 19252 |    graphs reused =      19659
4.42.955.892 I slot      release: id  3 | task 19252 | stop processing: n_tokens = 660, truncated = 0
4.42.955.900 I srv  update_slots: all slots are idle
4.42.992.908 I srv    operator(): Chat format: peg-native
4.42.995.725 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.569 (> 0.100 thold), f_keep = 0.112
4.42.995.727 I srv  get_availabl: updating prompt cache
4.42.995.770 W srv   prompt_save:  - saving prompt with length 660, total state size = 51.493 MiB (draft: 0.000 MiB)
4.43.085.089 I srv          load:  - looking for better prompt, base f_keep = 0.112, sim = 0.569
4.43.085.101 I srv        update:  - cache state: 14 prompts, 1458.353 MiB (limits: 8192.000 MiB, 16384 tokens, 119654 est)
4.43.085.103 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
4.43.085.104 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
4.43.085.105 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
4.43.085.105 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
4.43.085.131 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
4.43.085.131 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
4.43.085.132 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
4.43.085.132 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
4.43.085.133 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
4.43.085.133 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
4.43.085.134 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
4.43.085.134 I srv        update:    - prompt 0x56c94da5c020:     668 tokens, checkpoints:  1,    99.157 MiB
4.43.085.135 I srv        update:    - prompt 0x56c94dab24f0:    1030 tokens, checkpoints:  1,   101.283 MiB
4.43.085.135 I srv        update:    - prompt 0x56c94da9db40:     660 tokens, checkpoints:  1,    99.110 MiB
4.43.085.137 I srv  get_availabl: prompt cache update took 89.41 ms
4.43.086.029 I slot launch_slot_: id  3 | task 19791 | processing task, is_child = 0
4.43.086.031 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
4.43.086.032 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
4.43.086.044 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
4.43.086.044 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
4.43.086.053 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
4.43.086.053 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
4.43.086.069 I slot update_slots: id  3 | task 19791 | Checking checkpoint with [70, 70] against 74...
4.43.098.053 W slot update_slots: id  3 | task 19791 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
4.46.427.734 I slot print_timing: id  3 | task 19791 | n_decoded =    232, tg =  77.00 t/s, tg_3s =  77.00 t/s
4.49.430.778 I slot print_timing: id  3 | task 19791 | n_decoded =    462, tg =  76.80 t/s, tg_3s =  76.59 t/s
4.52.438.828 I slot print_timing: id  3 | task 19791 | n_decoded =    693, tg =  76.80 t/s, tg_3s =  76.79 t/s
4.55.451.380 I slot print_timing: id  3 | task 19791 | n_decoded =    925, tg =  76.85 t/s, tg_3s =  77.01 t/s
4.58.458.343 I slot print_timing: id  3 | task 19791 | n_decoded =   1164, tg =  77.38 t/s, tg_3s =  79.48 t/s
5.01.320.692 I slot print_timing: id  3 | task 19791 | prompt eval time =     328.75 ms /    59 tokens (    5.57 ms per token,   179.47 tokens per second)
5.01.320.696 I slot print_timing: id  3 | task 19791 |        eval time =   17905.87 ms /  1393 tokens (   12.85 ms per token,    77.80 tokens per second)
5.01.320.697 I slot print_timing: id  3 | task 19791 |       total time =   18234.62 ms /  1452 tokens
5.01.320.697 I slot print_timing: id  3 | task 19791 |    graphs reused =      21045
5.01.320.731 I slot      release: id  3 | task 19791 | stop processing: n_tokens = 1522, truncated = 0
5.01.320.742 I srv  update_slots: all slots are idle
5.01.354.678 I srv    operator(): Chat format: peg-native
5.01.357.521 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.638 (> 0.100 thold), f_keep = 0.049
5.01.357.523 I srv  get_availabl: updating prompt cache
5.01.357.597 W srv   prompt_save:  - saving prompt with length 1522, total state size = 56.553 MiB (draft: 0.000 MiB)
5.01.427.175 I srv          load:  - looking for better prompt, base f_keep = 0.049, sim = 0.638
5.01.427.186 I srv        update:  - cache state: 15 prompts, 1562.524 MiB (limits: 8192.000 MiB, 16384 tokens, 119656 est)
5.01.427.189 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
5.01.427.190 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
5.01.427.191 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
5.01.427.191 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
5.01.427.192 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
5.01.427.192 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
5.01.427.215 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
5.01.427.216 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
5.01.427.230 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
5.01.427.232 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
5.01.427.232 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
5.01.427.233 I srv        update:    - prompt 0x56c94da5c020:     668 tokens, checkpoints:  1,    99.157 MiB
5.01.427.233 I srv        update:    - prompt 0x56c94dab24f0:    1030 tokens, checkpoints:  1,   101.283 MiB
5.01.427.234 I srv        update:    - prompt 0x56c94da9db40:     660 tokens, checkpoints:  1,    99.110 MiB
5.01.427.234 I srv        update:    - prompt 0x56c94d94dfa0:    1522 tokens, checkpoints:  1,   104.171 MiB
5.01.427.236 I srv  get_availabl: prompt cache update took 69.71 ms
5.01.428.121 I slot launch_slot_: id  3 | task 21186 | processing task, is_child = 0
5.01.428.123 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
5.01.428.123 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
5.01.428.138 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
5.01.428.138 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
5.01.428.147 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
5.01.428.148 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
5.01.428.163 I slot update_slots: id  3 | task 21186 | Checking checkpoint with [70, 70] against 74...
5.01.440.194 W slot update_slots: id  3 | task 21186 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
5.04.684.677 I slot print_timing: id  3 | task 21186 | n_decoded =    241, tg =  80.04 t/s, tg_3s =  80.04 t/s
5.07.685.443 I slot print_timing: id  3 | task 21186 | n_decoded =    480, tg =  79.84 t/s, tg_3s =  79.65 t/s
5.10.688.768 I slot print_timing: id  3 | task 21186 | n_decoded =    719, tg =  79.76 t/s, tg_3s =  79.58 t/s
5.13.694.823 I slot print_timing: id  3 | task 21186 | n_decoded =    957, tg =  79.61 t/s, tg_3s =  79.17 t/s
5.16.707.524 I slot print_timing: id  3 | task 21186 | n_decoded =   1195, tg =  79.49 t/s, tg_3s =  79.00 t/s
5.17.784.515 I slot print_timing: id  3 | task 21186 | prompt eval time =     245.60 ms /    45 tokens (    5.46 ms per token,   183.23 tokens per second)
5.17.784.519 I slot print_timing: id  3 | task 21186 |        eval time =   16110.75 ms /  1279 tokens (   12.60 ms per token,    79.39 tokens per second)
5.17.784.519 I slot print_timing: id  3 | task 21186 |       total time =   16356.35 ms /  1324 tokens
5.17.784.520 I slot print_timing: id  3 | task 21186 |    graphs reused =      22317
5.17.784.588 I slot      release: id  3 | task 21186 | stop processing: n_tokens = 1394, truncated = 0
5.17.784.602 I srv  update_slots: all slots are idle
5.17.819.210 I srv    operator(): Chat format: peg-native
5.17.822.106 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.638 (> 0.100 thold), f_keep = 0.053
5.17.822.108 I srv  get_availabl: updating prompt cache
5.17.822.192 W srv   prompt_save:  - saving prompt with length 1394, total state size = 55.802 MiB (draft: 0.000 MiB)
5.17.890.872 I srv          load:  - looking for better prompt, base f_keep = 0.053, sim = 0.638
5.17.890.883 I srv        update:  - cache state: 16 prompts, 1665.944 MiB (limits: 8192.000 MiB, 16384 tokens, 119083 est)
5.17.890.884 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
5.17.890.886 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
5.17.890.886 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
5.17.890.890 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
5.17.890.891 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
5.17.890.891 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
5.17.890.892 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
5.17.890.892 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
5.17.890.893 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
5.17.890.893 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
5.17.890.902 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
5.17.890.902 I srv        update:    - prompt 0x56c94da5c020:     668 tokens, checkpoints:  1,    99.157 MiB
5.17.890.941 I srv        update:    - prompt 0x56c94dab24f0:    1030 tokens, checkpoints:  1,   101.283 MiB
5.17.890.941 I srv        update:    - prompt 0x56c94da9db40:     660 tokens, checkpoints:  1,    99.110 MiB
5.17.890.942 I srv        update:    - prompt 0x56c94d94dfa0:    1522 tokens, checkpoints:  1,   104.171 MiB
5.17.890.942 I srv        update:    - prompt 0x56c94db1c940:    1394 tokens, checkpoints:  1,   103.420 MiB
5.17.890.944 I srv  get_availabl: prompt cache update took 68.83 ms
5.17.891.827 I slot launch_slot_: id  3 | task 22467 | processing task, is_child = 0
5.17.891.829 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
5.17.891.830 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
5.17.891.844 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
5.17.891.844 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
5.17.891.853 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
5.17.891.854 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
5.17.891.877 I slot update_slots: id  3 | task 22467 | Checking checkpoint with [70, 70] against 74...
5.17.903.944 W slot update_slots: id  3 | task 22467 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
5.21.151.578 I slot print_timing: id  3 | task 22467 | n_decoded =    238, tg =  79.14 t/s, tg_3s =  79.14 t/s
5.24.154.649 I slot print_timing: id  3 | task 22467 | n_decoded =    474, tg =  78.86 t/s, tg_3s =  78.59 t/s
5.27.161.692 I slot print_timing: id  3 | task 22467 | n_decoded =    709, tg =  78.62 t/s, tg_3s =  78.15 t/s
5.30.164.584 I slot print_timing: id  3 | task 22467 | n_decoded =    941, tg =  78.28 t/s, tg_3s =  77.26 t/s
5.33.173.713 I slot print_timing: id  3 | task 22467 | n_decoded =   1173, tg =  78.05 t/s, tg_3s =  77.10 t/s
5.33.589.630 I slot print_timing: id  3 | task 22467 | prompt eval time =     252.18 ms /    45 tokens (    5.60 ms per token,   178.44 tokens per second)
5.33.589.633 I slot print_timing: id  3 | task 22467 |        eval time =   15445.57 ms /  1205 tokens (   12.82 ms per token,    78.02 tokens per second)
5.33.589.634 I slot print_timing: id  3 | task 22467 |       total time =   15697.75 ms /  1250 tokens
5.33.589.634 I slot print_timing: id  3 | task 22467 |    graphs reused =      23515
5.33.589.666 I slot      release: id  3 | task 22467 | stop processing: n_tokens = 1320, truncated = 0
5.33.589.676 I srv  update_slots: all slots are idle
5.33.623.699 I srv    operator(): Chat format: peg-native
5.33.626.581 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.685 (> 0.100 thold), f_keep = 0.056
5.33.626.583 I srv  get_availabl: updating prompt cache
5.33.626.649 W srv   prompt_save:  - saving prompt with length 1320, total state size = 55.367 MiB (draft: 0.000 MiB)
5.33.706.334 I srv          load:  - looking for better prompt, base f_keep = 0.056, sim = 0.685
5.33.706.384 I srv        update:  - cache state: 17 prompts, 1768.929 MiB (limits: 8192.000 MiB, 16384 tokens, 118263 est)
5.33.706.386 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
5.33.706.387 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
5.33.706.387 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
5.33.706.388 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
5.33.706.388 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
5.33.706.389 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
5.33.706.402 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
5.33.706.403 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
5.33.706.404 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
5.33.706.404 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
5.33.706.405 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
5.33.706.405 I srv        update:    - prompt 0x56c94da5c020:     668 tokens, checkpoints:  1,    99.157 MiB
5.33.706.406 I srv        update:    - prompt 0x56c94dab24f0:    1030 tokens, checkpoints:  1,   101.283 MiB
5.33.706.406 I srv        update:    - prompt 0x56c94da9db40:     660 tokens, checkpoints:  1,    99.110 MiB
5.33.706.418 I srv        update:    - prompt 0x56c94d94dfa0:    1522 tokens, checkpoints:  1,   104.171 MiB
5.33.706.420 I srv        update:    - prompt 0x56c94db1c940:    1394 tokens, checkpoints:  1,   103.420 MiB
5.33.706.420 I srv        update:    - prompt 0x56c94daccff0:    1320 tokens, checkpoints:  1,   102.985 MiB
5.33.706.421 I srv  get_availabl: prompt cache update took 79.84 ms
5.33.707.324 I slot launch_slot_: id  3 | task 23674 | processing task, is_child = 0
5.33.707.326 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
5.33.707.327 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
5.33.707.341 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
5.33.707.342 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
5.33.707.350 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
5.33.707.351 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
5.33.707.367 I slot update_slots: id  3 | task 23674 | Checking checkpoint with [70, 70] against 74...
5.33.719.400 W slot update_slots: id  3 | task 23674 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
5.36.920.618 I slot print_timing: id  3 | task 23674 | n_decoded =    234, tg =  77.85 t/s, tg_3s =  77.85 t/s
5.39.923.675 I slot print_timing: id  3 | task 23674 | n_decoded =    465, tg =  77.39 t/s, tg_3s =  76.92 t/s
5.42.932.721 I slot print_timing: id  3 | task 23674 | n_decoded =    696, tg =  77.18 t/s, tg_3s =  76.77 t/s
5.45.940.768 I slot print_timing: id  3 | task 23674 | n_decoded =    927, tg =  77.08 t/s, tg_3s =  76.79 t/s
5.48.942.548 I slot print_timing: id  3 | task 23674 | n_decoded =   1158, tg =  77.06 t/s, tg_3s =  76.95 t/s
5.50.180.470 I slot print_timing: id  3 | task 23674 | prompt eval time =     207.58 ms /    37 tokens (    5.61 ms per token,   178.24 tokens per second)
5.50.180.473 I slot print_timing: id  3 | task 23674 |        eval time =   16265.52 ms /  1256 tokens (   12.95 ms per token,    77.22 tokens per second)
5.50.180.474 I slot print_timing: id  3 | task 23674 |       total time =   16473.10 ms /  1293 tokens
5.50.180.475 I slot print_timing: id  3 | task 23674 |    graphs reused =      24764
5.50.180.506 I slot      release: id  3 | task 23674 | stop processing: n_tokens = 1363, truncated = 0
5.50.180.518 I srv  update_slots: all slots are idle
5.50.214.617 I srv    operator(): Chat format: peg-native
5.50.217.495 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.763 (> 0.100 thold), f_keep = 0.054
5.50.217.496 I srv  get_availabl: updating prompt cache
5.50.217.556 W srv   prompt_save:  - saving prompt with length 1363, total state size = 55.620 MiB (draft: 0.000 MiB)
5.50.297.128 I srv          load:  - looking for better prompt, base f_keep = 0.054, sim = 0.763
5.50.297.140 I srv        update:  - cache state: 18 prompts, 1872.166 MiB (limits: 8192.000 MiB, 16384 tokens, 117705 est)
5.50.297.178 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
5.50.297.182 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
5.50.297.183 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
5.50.297.183 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
5.50.297.184 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
5.50.297.184 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
5.50.297.185 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
5.50.297.187 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
5.50.297.188 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
5.50.297.188 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
5.50.297.189 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
5.50.297.190 I srv        update:    - prompt 0x56c94da5c020:     668 tokens, checkpoints:  1,    99.157 MiB
5.50.297.197 I srv        update:    - prompt 0x56c94dab24f0:    1030 tokens, checkpoints:  1,   101.283 MiB
5.50.297.198 I srv        update:    - prompt 0x56c94da9db40:     660 tokens, checkpoints:  1,    99.110 MiB
5.50.297.198 I srv        update:    - prompt 0x56c94d94dfa0:    1522 tokens, checkpoints:  1,   104.171 MiB
5.50.297.210 I srv        update:    - prompt 0x56c94db1c940:    1394 tokens, checkpoints:  1,   103.420 MiB
5.50.297.211 I srv        update:    - prompt 0x56c94daccff0:    1320 tokens, checkpoints:  1,   102.985 MiB
5.50.297.212 I srv        update:    - prompt 0x56c953cc2a40:    1363 tokens, checkpoints:  1,   103.238 MiB
5.50.297.213 I srv  get_availabl: prompt cache update took 79.72 ms
5.50.298.094 I slot launch_slot_: id  3 | task 24932 | processing task, is_child = 0
5.50.298.096 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
5.50.298.097 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
5.50.298.112 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
5.50.298.113 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
5.50.298.122 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
5.50.298.122 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
5.50.298.138 I slot update_slots: id  3 | task 24932 | Checking checkpoint with [70, 70] against 74...
5.50.310.104 W slot update_slots: id  3 | task 24932 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
5.53.455.566 I slot print_timing: id  3 | task 24932 | n_decoded =    240, tg =  79.75 t/s, tg_3s =  79.75 t/s
5.56.460.562 I slot print_timing: id  3 | task 24932 | n_decoded =    477, tg =  79.31 t/s, tg_3s =  78.87 t/s
5.59.465.900 I slot print_timing: id  3 | task 24932 | n_decoded =    714, tg =  79.16 t/s, tg_3s =  78.86 t/s
6.02.466.899 I slot print_timing: id  3 | task 24932 | n_decoded =    949, tg =  78.95 t/s, tg_3s =  78.31 t/s
6.05.480.696 I slot print_timing: id  3 | task 24932 | n_decoded =   1185, tg =  78.82 t/s, tg_3s =  78.31 t/s
6.08.486.601 I slot print_timing: id  3 | task 24932 | n_decoded =   1420, tg =  78.71 t/s, tg_3s =  78.18 t/s
6.11.493.632 I slot print_timing: id  3 | task 24932 | n_decoded =   1651, tg =  78.44 t/s, tg_3s =  76.82 t/s
6.13.811.662 I slot print_timing: id  3 | task 24932 | prompt eval time =     148.13 ms /    26 tokens (    5.70 ms per token,   175.52 tokens per second)
6.13.811.665 I slot print_timing: id  3 | task 24932 |        eval time =   23365.39 ms /  1829 tokens (   12.77 ms per token,    78.28 tokens per second)
6.13.811.666 I slot print_timing: id  3 | task 24932 |       total time =   23513.52 ms /  1855 tokens
6.13.811.667 I slot print_timing: id  3 | task 24932 |    graphs reused =      26584
6.13.811.695 I slot      release: id  3 | task 24932 | stop processing: n_tokens = 1925, truncated = 0
6.13.811.709 I srv  update_slots: all slots are idle
6.13.847.010 I srv    operator(): Chat format: peg-native
6.13.849.915 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.622 (> 0.100 thold), f_keep = 0.038
6.13.849.916 I srv  get_availabl: updating prompt cache
6.13.850.004 W srv   prompt_save:  - saving prompt with length 1925, total state size = 58.919 MiB (draft: 0.000 MiB)
6.13.923.012 I srv          load:  - looking for better prompt, base f_keep = 0.038, sim = 0.622
6.13.923.026 I srv        update:  - cache state: 19 prompts, 1978.703 MiB (limits: 8192.000 MiB, 16384 tokens, 119337 est)
6.13.923.042 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
6.13.923.043 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
6.13.923.044 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
6.13.923.045 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
6.13.923.045 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
6.13.923.046 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
6.13.923.046 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
6.13.923.047 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
6.13.923.047 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
6.13.923.050 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
6.13.923.051 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
6.13.923.051 I srv        update:    - prompt 0x56c94da5c020:     668 tokens, checkpoints:  1,    99.157 MiB
6.13.923.052 I srv        update:    - prompt 0x56c94dab24f0:    1030 tokens, checkpoints:  1,   101.283 MiB
6.13.923.063 I srv        update:    - prompt 0x56c94da9db40:     660 tokens, checkpoints:  1,    99.110 MiB
6.13.923.064 I srv        update:    - prompt 0x56c94d94dfa0:    1522 tokens, checkpoints:  1,   104.171 MiB
6.13.923.065 I srv        update:    - prompt 0x56c94db1c940:    1394 tokens, checkpoints:  1,   103.420 MiB
6.13.923.065 I srv        update:    - prompt 0x56c94daccff0:    1320 tokens, checkpoints:  1,   102.985 MiB
6.13.923.066 I srv        update:    - prompt 0x56c953cc2a40:    1363 tokens, checkpoints:  1,   103.238 MiB
6.13.923.066 I srv        update:    - prompt 0x56c94d964d00:    1925 tokens, checkpoints:  1,   106.537 MiB
6.13.923.068 I srv  get_availabl: prompt cache update took 73.15 ms
6.13.923.968 I slot launch_slot_: id  3 | task 26763 | processing task, is_child = 0
6.13.923.970 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
6.13.923.971 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
6.13.923.987 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
6.13.923.987 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
6.13.923.997 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
6.13.923.997 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
6.13.924.013 I slot update_slots: id  3 | task 26763 | Checking checkpoint with [70, 70] against 74...
6.13.936.072 W slot update_slots: id  3 | task 26763 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
6.17.183.636 I slot print_timing: id  3 | task 26763 | n_decoded =    234, tg =  77.80 t/s, tg_3s =  77.80 t/s
6.20.190.672 I slot print_timing: id  3 | task 26763 | n_decoded =    465, tg =  77.31 t/s, tg_3s =  76.82 t/s
6.23.199.726 I slot print_timing: id  3 | task 26763 | n_decoded =    696, tg =  77.13 t/s, tg_3s =  76.77 t/s
6.26.207.797 I slot print_timing: id  3 | task 26763 | n_decoded =    927, tg =  77.05 t/s, tg_3s =  76.79 t/s
6.28.651.769 I slot print_timing: id  3 | task 26763 | prompt eval time =     251.88 ms /    48 tokens (    5.25 ms per token,   190.56 tokens per second)
6.28.651.772 I slot print_timing: id  3 | task 26763 |        eval time =   14475.87 ms /  1115 tokens (   12.98 ms per token,    77.02 tokens per second)
6.28.651.773 I slot print_timing: id  3 | task 26763 |       total time =   14727.75 ms /  1163 tokens
6.28.651.773 I slot print_timing: id  3 | task 26763 |    graphs reused =      27693
6.28.651.839 I slot      release: id  3 | task 26763 | stop processing: n_tokens = 1233, truncated = 0
6.28.651.853 I srv  update_slots: all slots are idle
6.28.686.735 I srv    operator(): Chat format: peg-native
6.28.689.608 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.692 (> 0.100 thold), f_keep = 0.060
6.28.689.609 I srv  get_availabl: updating prompt cache
6.28.689.675 W srv   prompt_save:  - saving prompt with length 1233, total state size = 54.857 MiB (draft: 0.000 MiB)
6.28.754.559 I srv          load:  - looking for better prompt, base f_keep = 0.060, sim = 0.692
6.28.754.571 I srv        update:  - cache state: 20 prompts, 2081.178 MiB (limits: 8192.000 MiB, 16384 tokens, 118315 est)
6.28.754.573 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
6.28.754.574 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
6.28.754.575 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
6.28.754.575 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
6.28.754.576 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
6.28.754.576 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
6.28.754.577 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
6.28.754.577 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
6.28.754.578 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
6.28.754.578 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
6.28.754.579 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
6.28.754.579 I srv        update:    - prompt 0x56c94da5c020:     668 tokens, checkpoints:  1,    99.157 MiB
6.28.754.580 I srv        update:    - prompt 0x56c94dab24f0:    1030 tokens, checkpoints:  1,   101.283 MiB
6.28.754.592 I srv        update:    - prompt 0x56c94da9db40:     660 tokens, checkpoints:  1,    99.110 MiB
6.28.754.593 I srv        update:    - prompt 0x56c94d94dfa0:    1522 tokens, checkpoints:  1,   104.171 MiB
6.28.754.593 I srv        update:    - prompt 0x56c94db1c940:    1394 tokens, checkpoints:  1,   103.420 MiB
6.28.754.594 I srv        update:    - prompt 0x56c94daccff0:    1320 tokens, checkpoints:  1,   102.985 MiB
6.28.754.595 I srv        update:    - prompt 0x56c953cc2a40:    1363 tokens, checkpoints:  1,   103.238 MiB
6.28.754.596 I srv        update:    - prompt 0x56c94d964d00:    1925 tokens, checkpoints:  1,   106.537 MiB
6.28.754.596 I srv        update:    - prompt 0x56c94d98a240:    1233 tokens, checkpoints:  1,   102.474 MiB
6.28.754.597 I srv  get_availabl: prompt cache update took 64.99 ms
6.28.755.545 I slot launch_slot_: id  3 | task 27880 | processing task, is_child = 0
6.28.755.548 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
6.28.755.549 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
6.28.755.564 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
6.28.755.564 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
6.28.755.576 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
6.28.755.577 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
6.28.755.616 I slot update_slots: id  3 | task 27880 | Checking checkpoint with [70, 70] against 74...
6.28.767.651 W slot update_slots: id  3 | task 27880 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
6.31.959.773 I slot print_timing: id  3 | task 27880 | n_decoded =    232, tg =  77.04 t/s, tg_3s =  77.04 t/s
6.34.967.801 I slot print_timing: id  3 | task 27880 | n_decoded =    463, tg =  76.92 t/s, tg_3s =  76.79 t/s
6.37.971.631 I slot print_timing: id  3 | task 27880 | n_decoded =    704, tg =  78.02 t/s, tg_3s =  80.23 t/s
6.40.973.412 I slot print_timing: id  3 | task 27880 | n_decoded =    944, tg =  78.50 t/s, tg_3s =  79.95 t/s
6.43.982.691 I slot print_timing: id  3 | task 27880 | n_decoded =   1184, tg =  78.75 t/s, tg_3s =  79.75 t/s
6.46.985.477 I slot print_timing: id  3 | task 27880 | n_decoded =   1424, tg =  78.95 t/s, tg_3s =  79.93 t/s
6.47.266.776 I slot print_timing: id  3 | task 27880 | prompt eval time =     192.59 ms /    36 tokens (    5.35 ms per token,   186.93 tokens per second)
6.47.266.780 I slot print_timing: id  3 | task 27880 |        eval time =   18318.59 ms /  1446 tokens (   12.67 ms per token,    78.94 tokens per second)
6.47.266.780 I slot print_timing: id  3 | task 27880 |       total time =   18511.17 ms /  1482 tokens
6.47.266.781 I slot print_timing: id  3 | task 27880 |    graphs reused =      29131
6.47.266.812 I slot      release: id  3 | task 27880 | stop processing: n_tokens = 1552, truncated = 0
6.47.266.828 I srv  update_slots: all slots are idle
6.47.301.678 I srv    operator(): Chat format: peg-native
6.47.304.574 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.643 (> 0.100 thold), f_keep = 0.048
6.47.304.577 I srv  get_availabl: updating prompt cache
6.47.304.666 W srv   prompt_save:  - saving prompt with length 1552, total state size = 56.729 MiB (draft: 0.000 MiB)
6.47.377.778 I srv          load:  - looking for better prompt, base f_keep = 0.048, sim = 0.643
6.47.377.791 I srv        update:  - cache state: 21 prompts, 2185.525 MiB (limits: 8192.000 MiB, 16384 tokens, 118483 est)
6.47.377.795 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
6.47.377.796 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
6.47.377.796 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
6.47.377.797 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
6.47.377.797 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
6.47.377.799 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
6.47.377.800 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
6.47.377.800 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
6.47.377.801 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
6.47.377.801 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
6.47.377.802 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
6.47.377.844 I srv        update:    - prompt 0x56c94da5c020:     668 tokens, checkpoints:  1,    99.157 MiB
6.47.377.845 I srv        update:    - prompt 0x56c94dab24f0:    1030 tokens, checkpoints:  1,   101.283 MiB
6.47.377.846 I srv        update:    - prompt 0x56c94da9db40:     660 tokens, checkpoints:  1,    99.110 MiB
6.47.377.846 I srv        update:    - prompt 0x56c94d94dfa0:    1522 tokens, checkpoints:  1,   104.171 MiB
6.47.377.847 I srv        update:    - prompt 0x56c94db1c940:    1394 tokens, checkpoints:  1,   103.420 MiB
6.47.377.848 I srv        update:    - prompt 0x56c94daccff0:    1320 tokens, checkpoints:  1,   102.985 MiB
6.47.377.848 I srv        update:    - prompt 0x56c953cc2a40:    1363 tokens, checkpoints:  1,   103.238 MiB
6.47.377.849 I srv        update:    - prompt 0x56c94d964d00:    1925 tokens, checkpoints:  1,   106.537 MiB
6.47.377.850 I srv        update:    - prompt 0x56c94d98a240:    1233 tokens, checkpoints:  1,   102.474 MiB
6.47.377.850 I srv        update:    - prompt 0x56c94d984e80:    1552 tokens, checkpoints:  1,   104.347 MiB
6.47.377.851 I srv  get_availabl: prompt cache update took 73.27 ms
6.47.378.762 I slot launch_slot_: id  3 | task 29328 | processing task, is_child = 0
6.47.378.764 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
6.47.378.765 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
6.47.378.782 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
6.47.378.800 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
6.47.378.816 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
6.47.378.816 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
6.47.378.837 I slot update_slots: id  3 | task 29328 | Checking checkpoint with [70, 70] against 74...
6.47.391.049 W slot update_slots: id  3 | task 29328 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
6.50.599.705 I slot print_timing: id  3 | task 29328 | n_decoded =    241, tg =  80.13 t/s, tg_3s =  80.13 t/s
6.53.609.764 I slot print_timing: id  3 | task 29328 | n_decoded =    481, tg =  79.93 t/s, tg_3s =  79.73 t/s
6.56.621.484 I slot print_timing: id  3 | task 29328 | n_decoded =    721, tg =  79.85 t/s, tg_3s =  79.69 t/s
6.59.629.691 I slot print_timing: id  3 | task 29328 | n_decoded =    959, tg =  79.67 t/s, tg_3s =  79.12 t/s
7.02.640.231 I slot print_timing: id  3 | task 29328 | n_decoded =   1196, tg =  79.48 t/s, tg_3s =  78.72 t/s
7.05.645.763 I slot print_timing: id  3 | task 29328 | n_decoded =   1432, tg =  79.32 t/s, tg_3s =  78.52 t/s
7.08.657.796 I slot print_timing: id  3 | task 29328 | n_decoded =   1669, tg =  79.23 t/s, tg_3s =  78.68 t/s
7.11.658.806 I slot print_timing: id  3 | task 29328 | n_decoded =   1903, tg =  79.07 t/s, tg_3s =  77.97 t/s
7.13.204.288 I slot print_timing: id  3 | task 29328 | prompt eval time =     213.09 ms /    44 tokens (    4.84 ms per token,   206.48 tokens per second)
7.13.204.312 I slot print_timing: id  3 | task 29328 |        eval time =   25612.35 ms /  2023 tokens (   12.66 ms per token,    78.99 tokens per second)
7.13.204.314 I slot print_timing: id  3 | task 29328 |       total time =   25825.44 ms /  2067 tokens
7.13.204.314 I slot print_timing: id  3 | task 29328 |    graphs reused =      31144
7.13.204.347 I slot      release: id  3 | task 29328 | stop processing: n_tokens = 2137, truncated = 0
7.13.204.361 I srv  update_slots: all slots are idle
7.13.238.229 I srv    operator(): Chat format: peg-native
7.13.241.089 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.612 (> 0.100 thold), f_keep = 0.035
7.13.241.091 I srv  get_availabl: updating prompt cache
7.13.241.188 W srv   prompt_save:  - saving prompt with length 2137, total state size = 60.164 MiB (draft: 0.000 MiB)
7.13.311.086 I srv          load:  - looking for better prompt, base f_keep = 0.035, sim = 0.612
7.13.311.098 I srv        update:  - cache state: 22 prompts, 2293.306 MiB (limits: 8192.000 MiB, 16384 tokens, 120548 est)
7.13.311.099 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
7.13.311.100 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
7.13.311.101 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
7.13.311.101 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
7.13.311.102 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
7.13.311.138 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
7.13.311.140 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
7.13.311.141 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
7.13.311.141 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
7.13.311.142 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
7.13.311.142 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
7.13.311.143 I srv        update:    - prompt 0x56c94da5c020:     668 tokens, checkpoints:  1,    99.157 MiB
7.13.311.143 I srv        update:    - prompt 0x56c94dab24f0:    1030 tokens, checkpoints:  1,   101.283 MiB
7.13.311.144 I srv        update:    - prompt 0x56c94da9db40:     660 tokens, checkpoints:  1,    99.110 MiB
7.13.311.144 I srv        update:    - prompt 0x56c94d94dfa0:    1522 tokens, checkpoints:  1,   104.171 MiB
7.13.311.146 I srv        update:    - prompt 0x56c94db1c940:    1394 tokens, checkpoints:  1,   103.420 MiB
7.13.311.149 I srv        update:    - prompt 0x56c94daccff0:    1320 tokens, checkpoints:  1,   102.985 MiB
7.13.311.149 I srv        update:    - prompt 0x56c953cc2a40:    1363 tokens, checkpoints:  1,   103.238 MiB
7.13.311.150 I srv        update:    - prompt 0x56c94d964d00:    1925 tokens, checkpoints:  1,   106.537 MiB
7.13.311.150 I srv        update:    - prompt 0x56c94d98a240:    1233 tokens, checkpoints:  1,   102.474 MiB
7.13.311.151 I srv        update:    - prompt 0x56c94d984e80:    1552 tokens, checkpoints:  1,   104.347 MiB
7.13.311.151 I srv        update:    - prompt 0x56c94d947d40:    2137 tokens, checkpoints:  1,   107.782 MiB
7.13.311.152 I srv  get_availabl: prompt cache update took 70.06 ms
7.13.312.042 I slot launch_slot_: id  3 | task 31353 | processing task, is_child = 0
7.13.312.044 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
7.13.312.044 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
7.13.312.056 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
7.13.312.057 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
7.13.312.066 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
7.13.312.066 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
7.13.312.084 I slot update_slots: id  3 | task 31353 | Checking checkpoint with [70, 70] against 74...
7.13.324.057 W slot update_slots: id  3 | task 31353 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
7.16.575.805 I slot print_timing: id  3 | task 31353 | n_decoded =    237, tg =  78.75 t/s, tg_3s =  78.75 t/s
7.19.583.603 I slot print_timing: id  3 | task 31353 | n_decoded =    472, tg =  78.44 t/s, tg_3s =  78.13 t/s
7.22.223.299 I slot print_timing: id  3 | task 31353 | prompt eval time =     254.17 ms /    50 tokens (    5.08 ms per token,   196.72 tokens per second)
7.22.223.303 I slot print_timing: id  3 | task 31353 |        eval time =    8657.04 ms /   677 tokens (   12.79 ms per token,    78.20 tokens per second)
7.22.223.304 I slot print_timing: id  3 | task 31353 |       total time =    8911.21 ms /   727 tokens
7.22.223.305 I slot print_timing: id  3 | task 31353 |    graphs reused =      31816
7.22.223.340 I slot      release: id  3 | task 31353 | stop processing: n_tokens = 797, truncated = 0
7.22.223.349 I srv  update_slots: all slots are idle
7.22.257.928 I srv    operator(): Chat format: peg-native
7.22.260.810 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.698 (> 0.100 thold), f_keep = 0.093
7.22.260.813 I srv  get_availabl: updating prompt cache
7.22.260.884 W srv   prompt_save:  - saving prompt with length 797, total state size = 52.297 MiB (draft: 0.000 MiB)
7.22.328.972 I srv          load:  - looking for better prompt, base f_keep = 0.093, sim = 0.698
7.22.328.986 I srv        update:  - cache state: 23 prompts, 2393.221 MiB (limits: 8192.000 MiB, 16384 tokens, 118244 est)
7.22.328.989 I srv        update:    - prompt 0x56c94da9e380:    2942 tokens, checkpoints:  1,   112.508 MiB
7.22.328.989 I srv        update:    - prompt 0x56c94d9496f0:    1777 tokens, checkpoints:  1,   105.668 MiB
7.22.328.990 I srv        update:    - prompt 0x56c94da8dd60:     974 tokens, checkpoints:  1,   100.954 MiB
7.22.328.991 I srv        update:    - prompt 0x56c953dc6a40:    1540 tokens, checkpoints:  1,   104.277 MiB
7.22.328.992 I srv        update:    - prompt 0x56c953e012b0:    2557 tokens, checkpoints:  1,   110.247 MiB
7.22.328.993 I srv        update:    - prompt 0x56c953dd4620:    2763 tokens, checkpoints:  1,   111.457 MiB
7.22.328.993 I srv        update:    - prompt 0x56c94dbbd9f0:    1360 tokens, checkpoints:  1,   103.220 MiB
7.22.328.994 I srv        update:    - prompt 0x56c94da2e120:    1323 tokens, checkpoints:  1,   103.003 MiB
7.22.328.995 I srv        update:    - prompt 0x56c94d93bae0:    1063 tokens, checkpoints:  1,   101.476 MiB
7.22.328.995 I srv        update:    - prompt 0x56c953ddbe00:    1749 tokens, checkpoints:  1,   105.504 MiB
7.22.328.996 I srv        update:    - prompt 0x56c94da69260:     895 tokens, checkpoints:  1,   100.490 MiB
7.22.328.996 I srv        update:    - prompt 0x56c94da5c020:     668 tokens, checkpoints:  1,    99.157 MiB
7.22.328.997 I srv        update:    - prompt 0x56c94dab24f0:    1030 tokens, checkpoints:  1,   101.283 MiB
7.22.328.997 I srv        update:    - prompt 0x56c94da9db40:     660 tokens, checkpoints:  1,    99.110 MiB
7.22.328.998 I srv        update:    - prompt 0x56c94d94dfa0:    1522 tokens, checkpoints:  1,   104.171 MiB
7.22.328.998 I srv        update:    - prompt 0x56c94db1c940:    1394 tokens, checkpoints:  1,   103.420 MiB
7.22.328.999 I srv        update:    - prompt 0x56c94daccff0:    1320 tokens, checkpoints:  1,   102.985 MiB
7.22.329.000 I srv        update:    - prompt 0x56c953cc2a40:    1363 tokens, checkpoints:  1,   103.238 MiB
7.22.329.000 I srv        update:    - prompt 0x56c94d964d00:    1925 tokens, checkpoints:  1,   106.537 MiB
7.22.329.000 I srv        update:    - prompt 0x56c94d98a240:    1233 tokens, checkpoints:  1,   102.474 MiB
7.22.329.001 I srv        update:    - prompt 0x56c94d984e80:    1552 tokens, checkpoints:  1,   104.347 MiB
7.22.329.001 I srv        update:    - prompt 0x56c94d947d40:    2137 tokens, checkpoints:  1,   107.782 MiB
7.22.329.002 I srv        update:    - prompt 0x56c953db7200:     797 tokens, checkpoints:  1,    99.915 MiB
7.22.329.003 I srv  get_availabl: prompt cache update took 68.19 ms
7.22.329.875 I slot launch_slot_: id  3 | task 32032 | processing task, is_child = 0
7.22.329.877 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
7.22.329.878 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
7.22.329.890 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
7.22.329.890 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
7.22.329.899 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
7.22.329.899 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
7.22.329.914 I slot update_slots: id  3 | task 32032 | Checking checkpoint with [70, 70] against 74...
7.22.342.020 W slot update_slots: id  3 | task 32032 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 47.618 MiB)
7.25.517.283 I slot print_timing: id  3 | task 32032 | n_decoded =    235, tg =  78.30 t/s, tg_3s =  78.30 t/s
7.28.524.313 I slot print_timing: id  3 | task 32032 | n_decoded =    468, tg =  77.89 t/s, tg_3s =  77.49 t/s
7.31.532.745 I slot print_timing: id  3 | task 32032 | n_decoded =    707, tg =  78.41 t/s, tg_3s =  79.44 t/s
7.34.543.784 I slot print_timing: id  3 | task 32032 | n_decoded =    947, tg =  78.74 t/s, tg_3s =  79.71 t/s
7.37.546.833 I slot print_timing: id  3 | task 32032 | n_decoded =   1186, tg =  78.91 t/s, tg_3s =  79.59 t/s
7.40.552.620 I slot print_timing: id  3 | task 32032 | n_decoded =   1425, tg =  79.01 t/s, tg_3s =  79.51 t/s
7.42.590.533 I slot print_timing: id  3 | task 32032 | prompt eval time =     186.24 ms /    35 tokens (    5.32 ms per token,   187.93 tokens per second)
7.42.590.537 I slot print_timing: id  3 | task 32032 |        eval time =   20074.38 ms /  1586 tokens (   12.66 ms per token,    79.01 tokens per second)
7.42.590.537 I slot print_timing: id  3 | task 32032 |       total time =   20260.62 ms /  1621 tokens
7.42.590.538 I slot print_timing: id  3 | task 32032 |    graphs reused =      33394
7.42.590.569 I slot      release: id  3 | task 32032 | stop processing: n_tokens = 1691, truncated = 0
7.42.590.583 I srv  update_slots: all slots are idle
7.42.591.585 I srv    operator(): operator(): cleaning up before exit...