experiments-07-07-12-local-llm-knowledge-2/results_hard/glm4.7-flash.server.log

0.00.065.276 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.065.279 I device_info:
0.00.065.299 I   - ROCm0   : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.303 I   - ROCm1   : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.310 I   - CPU     : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.065.364 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
0.00.065.368 I srv  llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.065.408 I srv          init: running without SSL
0.00.065.473 I srv          init: using 23 threads for HTTP server
0.00.065.798 I srv         start: binding port with default address family
0.00.067.000 I srv  llama_server: loading model
0.00.067.009 I srv    load_model: loading model '/home/qrf/.cache/llama.cpp/zai-org_GLM-4.7-Flash-Q5_K_M.gguf'
0.00.067.068 I common_init_result: fitting params to device memory ...
0.00.067.070 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.01.172.880 W load: special_eot_id is not in special_eog_ids - the tokenizer config may be incorrect
0.01.172.887 W load: special_eom_id is not in special_eog_ids - the tokenizer config may be incorrect
0.14.477.848 W llama_context: n_ctx_seq (16384) < n_ctx_train (202752) -- the full capacity of the model will not be utilized
0.14.576.566 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.14.760.255 I srv    load_model: initializing slots, n_slots = 4
0.14.810.782 W common_speculative_init: no implementations specified for speculative decoding
0.14.810.787 I slot   load_model: id  0 | task -1 | new slot, n_ctx = 16384
0.14.810.792 I slot   load_model: id  1 | task -1 | new slot, n_ctx = 16384
0.14.810.792 I slot   load_model: id  2 | task -1 | new slot, n_ctx = 16384
0.14.810.793 I slot   load_model: id  3 | task -1 | new slot, n_ctx = 16384
0.14.810.833 I srv    load_model: prompt cache is enabled, size limit: 8192 MiB
0.14.810.834 I srv    load_model: use `--cache-ram 0` to disable the prompt cache
0.14.810.835 I srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.14.810.835 I srv    load_model: context checkpoints enabled, max = 32, min spacing = 256
0.14.810.853 I srv          init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.14.822.971 I init: chat template, example_format: '[gMASK]<sop><|system|>You are a helpful assistant<|user|>Hello<|assistant|></think>Hi there<|user|>How are you?<|assistant|><think>'
0.14.832.242 I srv          init: init: chat template, thinking = 1
0.14.832.276 I srv  llama_server: model loaded
0.14.832.279 I srv  llama_server: server is listening on http://127.0.0.1:8090
0.14.832.285 I srv  update_slots: all slots are idle
0.14.975.167 I srv    operator(): Chat format: peg-native
0.14.975.419 I slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
0.14.975.421 I srv  get_availabl: updating prompt cache
0.14.975.425 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.14.975.429 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est)
0.14.975.430 I srv  get_availabl: prompt cache update took 0.01 ms
0.14.975.449 I slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
0.14.975.450 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.14.975.450 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
0.14.975.462 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.14.975.462 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
0.14.975.471 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.14.975.471 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
0.18.774.933 I slot print_timing: id  3 | task 0 | n_decoded =    137, tg =  45.54 t/s, tg_3s =  45.54 t/s
0.21.785.772 I slot print_timing: id  3 | task 0 | n_decoded =    273, tg =  45.36 t/s, tg_3s =  45.17 t/s
0.24.790.037 I slot print_timing: id  3 | task 0 | n_decoded =    409, tg =  45.33 t/s, tg_3s =  45.27 t/s
0.27.792.122 I slot print_timing: id  3 | task 0 | n_decoded =    543, tg =  45.15 t/s, tg_3s =  44.64 t/s
0.30.808.668 I slot print_timing: id  3 | task 0 | n_decoded =    678, tg =  45.07 t/s, tg_3s =  44.75 t/s
0.33.831.004 I slot print_timing: id  3 | task 0 | n_decoded =    812, tg =  44.95 t/s, tg_3s =  44.34 t/s
0.36.834.227 I slot print_timing: id  3 | task 0 | n_decoded =    945, tg =  44.86 t/s, tg_3s =  44.29 t/s
0.39.834.712 I slot print_timing: id  3 | task 0 | n_decoded =   1078, tg =  44.79 t/s, tg_3s =  44.33 t/s
0.42.849.674 I slot print_timing: id  3 | task 0 | n_decoded =   1211, tg =  44.71 t/s, tg_3s =  44.11 t/s
0.45.858.881 I slot print_timing: id  3 | task 0 | n_decoded =   1345, tg =  44.70 t/s, tg_3s =  44.53 t/s
0.48.878.637 I slot print_timing: id  3 | task 0 | n_decoded =   1479, tg =  44.67 t/s, tg_3s =  44.37 t/s
0.51.893.740 I slot print_timing: id  3 | task 0 | n_decoded =   1613, tg =  44.65 t/s, tg_3s =  44.44 t/s
0.54.904.620 I slot print_timing: id  3 | task 0 | n_decoded =   1746, tg =  44.61 t/s, tg_3s =  44.17 t/s
0.55.334.887 I slot print_timing: id  3 | task 0 | prompt eval time =     791.24 ms /    91 tokens (    8.69 ms per token,   115.01 tokens per second)
0.55.334.891 I slot print_timing: id  3 | task 0 |        eval time =   39568.15 ms /  1765 tokens (   22.42 ms per token,    44.61 tokens per second)
0.55.334.892 I slot print_timing: id  3 | task 0 |       total time =   40359.39 ms /  1856 tokens
0.55.334.896 I slot print_timing: id  3 | task 0 |    graphs reused =       1757
0.55.334.954 I slot      release: id  3 | task 0 | stop processing: n_tokens = 1855, truncated = 0
0.55.334.966 I srv  update_slots: all slots are idle
0.55.346.647 I srv    operator(): Chat format: peg-native
0.55.346.876 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.824 (> 0.100 thold), f_keep = 0.038
0.55.346.878 I srv  get_availabl: updating prompt cache
0.55.346.952 W srv   prompt_save:  - saving prompt with length 1855, total state size = 95.806 MiB (draft: 0.000 MiB)
0.55.452.002 I srv          load:  - looking for better prompt, base f_keep = 0.038, sim = 0.824
0.55.452.007 I srv        update:  - cache state: 1 prompts, 95.806 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
0.55.452.008 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
0.55.452.009 I srv  get_availabl: prompt cache update took 105.13 ms
0.55.452.057 I slot launch_slot_: id  3 | task 1767 | processing task, is_child = 0
0.55.452.057 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.55.452.058 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
0.55.452.069 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.55.452.069 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
0.55.452.078 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.55.452.079 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
0.58.613.096 I slot print_timing: id  3 | task 1767 | n_decoded =    137, tg =  45.56 t/s, tg_3s =  45.56 t/s
1.01.620.453 I slot print_timing: id  3 | task 1767 | n_decoded =    273, tg =  45.39 t/s, tg_3s =  45.22 t/s
1.04.636.499 I slot print_timing: id  3 | task 1767 | n_decoded =    410, tg =  45.40 t/s, tg_3s =  45.42 t/s
1.07.657.633 I slot print_timing: id  3 | task 1767 | n_decoded =    545, tg =  45.22 t/s, tg_3s =  44.69 t/s
1.10.671.523 I slot print_timing: id  3 | task 1767 | n_decoded =    680, tg =  45.14 t/s, tg_3s =  44.79 t/s
1.13.686.674 I slot print_timing: id  3 | task 1767 | n_decoded =    814, tg =  45.02 t/s, tg_3s =  44.44 t/s
1.16.701.281 I slot print_timing: id  3 | task 1767 | n_decoded =    948, tg =  44.94 t/s, tg_3s =  44.45 t/s
1.19.718.347 I slot print_timing: id  3 | task 1767 | n_decoded =   1082, tg =  44.87 t/s, tg_3s =  44.41 t/s
1.21.340.981 I slot print_timing: id  3 | task 1767 | prompt eval time =     153.66 ms /    15 tokens (   10.24 ms per token,    97.62 tokens per second)
1.21.340.984 I slot print_timing: id  3 | task 1767 |        eval time =   25735.22 ms /  1154 tokens (   22.30 ms per token,    44.84 tokens per second)
1.21.340.985 I slot print_timing: id  3 | task 1767 |       total time =   25888.88 ms /  1169 tokens
1.21.340.985 I slot print_timing: id  3 | task 1767 |    graphs reused =       2905
1.21.341.031 I slot      release: id  3 | task 1767 | stop processing: n_tokens = 1238, truncated = 0
1.21.341.043 I srv  update_slots: all slots are idle
1.21.352.430 I srv    operator(): Chat format: peg-native
1.21.352.656 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.814 (> 0.100 thold), f_keep = 0.057
1.21.352.658 I srv  get_availabl: updating prompt cache
1.21.352.718 W srv   prompt_save:  - saving prompt with length 1238, total state size = 63.940 MiB (draft: 0.000 MiB)
1.21.441.960 I srv          load:  - looking for better prompt, base f_keep = 0.057, sim = 0.814
1.21.441.966 I srv        update:  - cache state: 2 prompts, 159.746 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
1.21.441.967 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
1.21.441.968 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
1.21.441.969 I srv  get_availabl: prompt cache update took 89.31 ms
1.21.442.008 I slot launch_slot_: id  3 | task 2922 | processing task, is_child = 0
1.21.442.009 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.21.442.009 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
1.21.442.024 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
1.21.442.024 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
1.21.442.033 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
1.21.442.034 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
1.24.597.561 I slot print_timing: id  3 | task 2922 | n_decoded =    136, tg =  45.27 t/s, tg_3s =  45.27 t/s
1.27.609.883 I slot print_timing: id  3 | task 2922 | n_decoded =    271, tg =  45.04 t/s, tg_3s =  44.82 t/s
1.30.616.397 I slot print_timing: id  3 | task 2922 | n_decoded =    406, tg =  45.00 t/s, tg_3s =  44.90 t/s
1.33.623.969 I slot print_timing: id  3 | task 2922 | n_decoded =    540, tg =  44.89 t/s, tg_3s =  44.55 t/s
1.36.594.805 I slot print_timing: id  3 | task 2922 | prompt eval time =     151.31 ms /    16 tokens (    9.46 ms per token,   105.75 tokens per second)
1.36.594.808 I slot print_timing: id  3 | task 2922 |        eval time =   15001.44 ms /   673 tokens (   22.29 ms per token,    44.86 tokens per second)
1.36.594.809 I slot print_timing: id  3 | task 2922 |       total time =   15152.75 ms /   689 tokens
1.36.594.810 I slot print_timing: id  3 | task 2922 |    graphs reused =       3574
1.36.594.848 I slot      release: id  3 | task 2922 | stop processing: n_tokens = 758, truncated = 0
1.36.594.858 I srv  update_slots: all slots are idle
1.36.606.066 I srv    operator(): Chat format: peg-native
1.36.606.296 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.092
1.36.606.298 I srv  get_availabl: updating prompt cache
1.36.606.342 W srv   prompt_save:  - saving prompt with length 758, total state size = 39.149 MiB (draft: 0.000 MiB)
1.36.636.126 I srv          load:  - looking for better prompt, base f_keep = 0.092, sim = 0.833
1.36.636.136 I srv        update:  - cache state: 3 prompts, 198.895 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
1.36.636.137 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
1.36.636.138 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
1.36.636.139 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
1.36.636.140 I srv  get_availabl: prompt cache update took 29.84 ms
1.36.636.192 I slot launch_slot_: id  3 | task 3596 | processing task, is_child = 0
1.36.636.192 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.36.636.193 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
1.36.636.206 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
1.36.636.207 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
1.36.636.215 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
1.36.636.216 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
1.39.790.700 I slot print_timing: id  3 | task 3596 | n_decoded =    137, tg =  45.62 t/s, tg_3s =  45.62 t/s
1.42.792.812 I slot print_timing: id  3 | task 3596 | n_decoded =    273, tg =  45.46 t/s, tg_3s =  45.30 t/s
1.45.793.908 I slot print_timing: id  3 | task 3596 | n_decoded =    409, tg =  45.41 t/s, tg_3s =  45.32 t/s
1.48.808.322 I slot print_timing: id  3 | task 3596 | n_decoded =    544, tg =  45.26 t/s, tg_3s =  44.78 t/s
1.51.813.477 I slot print_timing: id  3 | task 3596 | n_decoded =    679, tg =  45.19 t/s, tg_3s =  44.92 t/s
1.54.817.026 I slot print_timing: id  3 | task 3596 | n_decoded =    813, tg =  45.09 t/s, tg_3s =  44.61 t/s
1.57.820.585 I slot print_timing: id  3 | task 3596 | n_decoded =    947, tg =  45.02 t/s, tg_3s =  44.61 t/s
2.00.829.366 I slot print_timing: id  3 | task 3596 | n_decoded =   1081, tg =  44.96 t/s, tg_3s =  44.54 t/s
2.03.832.952 I slot print_timing: id  3 | task 3596 | n_decoded =   1214, tg =  44.89 t/s, tg_3s =  44.28 t/s
2.06.833.830 I slot print_timing: id  3 | task 3596 | n_decoded =   1347, tg =  44.83 t/s, tg_3s =  44.32 t/s
2.07.082.131 I slot print_timing: id  3 | task 3596 | prompt eval time =     151.33 ms /    14 tokens (   10.81 ms per token,    92.51 tokens per second)
2.07.082.134 I slot print_timing: id  3 | task 3596 |        eval time =   30294.56 ms /  1358 tokens (   22.31 ms per token,    44.83 tokens per second)
2.07.082.135 I slot print_timing: id  3 | task 3596 |       total time =   30445.89 ms /  1372 tokens
2.07.082.135 I slot print_timing: id  3 | task 3596 |    graphs reused =       4925
2.07.082.172 I slot      release: id  3 | task 3596 | stop processing: n_tokens = 1441, truncated = 0
2.07.082.186 I srv  update_slots: all slots are idle
2.07.093.406 I srv    operator(): Chat format: peg-native
2.07.093.637 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.722 (> 0.100 thold), f_keep = 0.049
2.07.093.639 I srv  get_availabl: updating prompt cache
2.07.093.712 W srv   prompt_save:  - saving prompt with length 1441, total state size = 74.424 MiB (draft: 0.000 MiB)
2.07.156.606 I srv          load:  - looking for better prompt, base f_keep = 0.049, sim = 0.722
2.07.156.612 I srv        update:  - cache state: 4 prompts, 273.319 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
2.07.156.613 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
2.07.156.614 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
2.07.156.614 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
2.07.156.615 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
2.07.156.616 I srv  get_availabl: prompt cache update took 62.98 ms
2.07.156.647 I slot launch_slot_: id  3 | task 4955 | processing task, is_child = 0
2.07.156.648 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
2.07.156.649 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
2.07.156.660 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
2.07.156.660 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
2.07.156.669 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
2.07.156.670 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
2.10.402.446 I slot print_timing: id  3 | task 4955 | n_decoded =    137, tg =  45.56 t/s, tg_3s =  45.56 t/s
2.13.407.861 I slot print_timing: id  3 | task 4955 | n_decoded =    272, tg =  45.24 t/s, tg_3s =  44.92 t/s
2.16.428.585 I slot print_timing: id  3 | task 4955 | n_decoded =    408, tg =  45.17 t/s, tg_3s =  45.02 t/s
2.19.438.808 I slot print_timing: id  3 | task 4955 | n_decoded =    542, tg =  45.00 t/s, tg_3s =  44.51 t/s
2.22.453.430 I slot print_timing: id  3 | task 4955 | n_decoded =    676, tg =  44.89 t/s, tg_3s =  44.45 t/s
2.25.459.794 I slot print_timing: id  3 | task 4955 | n_decoded =    811, tg =  44.89 t/s, tg_3s =  44.90 t/s
2.28.471.594 I slot print_timing: id  3 | task 4955 | n_decoded =    946, tg =  44.88 t/s, tg_3s =  44.82 t/s
2.31.490.448 I slot print_timing: id  3 | task 4955 | n_decoded =   1081, tg =  44.86 t/s, tg_3s =  44.72 t/s
2.34.505.203 I slot print_timing: id  3 | task 4955 | n_decoded =   1215, tg =  44.82 t/s, tg_3s =  44.45 t/s
2.37.519.604 I slot print_timing: id  3 | task 4955 | n_decoded =   1349, tg =  44.78 t/s, tg_3s =  44.45 t/s
2.40.523.479 I slot print_timing: id  3 | task 4955 | n_decoded =   1482, tg =  44.74 t/s, tg_3s =  44.28 t/s
2.43.540.896 I slot print_timing: id  3 | task 4955 | n_decoded =   1616, tg =  44.71 t/s, tg_3s =  44.41 t/s
2.46.552.591 I slot print_timing: id  3 | task 4955 | n_decoded =   1749, tg =  44.67 t/s, tg_3s =  44.16 t/s
2.49.565.980 I slot print_timing: id  3 | task 4955 | n_decoded =   1882, tg =  44.63 t/s, tg_3s =  44.14 t/s
2.49.611.523 I slot print_timing: id  3 | task 4955 | prompt eval time =     238.70 ms /    27 tokens (    8.84 ms per token,   113.11 tokens per second)
2.49.611.526 I slot print_timing: id  3 | task 4955 |        eval time =   42216.13 ms /  1884 tokens (   22.41 ms per token,    44.63 tokens per second)
2.49.611.527 I slot print_timing: id  3 | task 4955 |       total time =   42454.84 ms /  1911 tokens
2.49.611.527 I slot print_timing: id  3 | task 4955 |    graphs reused =       6800
2.49.611.566 I slot      release: id  3 | task 4955 | stop processing: n_tokens = 1980, truncated = 0
2.49.611.581 I srv  update_slots: all slots are idle
2.49.623.172 I srv    operator(): Chat format: peg-native
2.49.623.404 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.805 (> 0.100 thold), f_keep = 0.035
2.49.623.405 I srv  get_availabl: updating prompt cache
2.49.623.480 W srv   prompt_save:  - saving prompt with length 1980, total state size = 102.262 MiB (draft: 0.000 MiB)
2.49.753.953 I srv          load:  - looking for better prompt, base f_keep = 0.035, sim = 0.805
2.49.753.962 I srv        update:  - cache state: 5 prompts, 375.581 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
2.49.753.963 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
2.49.753.964 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
2.49.753.965 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
2.49.753.965 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
2.49.753.966 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
2.49.753.967 I srv  get_availabl: prompt cache update took 130.56 ms
2.49.754.009 I slot launch_slot_: id  3 | task 6840 | processing task, is_child = 0
2.49.754.010 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
2.49.754.011 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
2.49.754.031 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
2.49.754.032 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
2.49.754.048 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
2.49.754.048 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
2.52.947.453 I slot print_timing: id  3 | task 6840 | n_decoded =    137, tg =  45.54 t/s, tg_3s =  45.54 t/s
2.55.956.574 I slot print_timing: id  3 | task 6840 | n_decoded =    273, tg =  45.37 t/s, tg_3s =  45.20 t/s
2.58.957.305 I slot print_timing: id  3 | task 6840 | n_decoded =    408, tg =  45.24 t/s, tg_3s =  44.99 t/s
3.01.965.188 I slot print_timing: id  3 | task 6840 | n_decoded =    542, tg =  45.07 t/s, tg_3s =  44.55 t/s
3.04.972.532 I slot print_timing: id  3 | task 6840 | n_decoded =    676, tg =  44.97 t/s, tg_3s =  44.56 t/s
3.07.973.765 I slot print_timing: id  3 | task 6840 | n_decoded =    809, tg =  44.86 t/s, tg_3s =  44.32 t/s
3.10.978.300 I slot print_timing: id  3 | task 6840 | n_decoded =    942, tg =  44.77 t/s, tg_3s =  44.27 t/s
3.13.997.366 I slot print_timing: id  3 | task 6840 | n_decoded =   1076, tg =  44.73 t/s, tg_3s =  44.38 t/s
3.15.954.807 I slot print_timing: id  3 | task 6840 | prompt eval time =     185.36 ms /    17 tokens (   10.90 ms per token,    91.71 tokens per second)
3.15.954.810 I slot print_timing: id  3 | task 6840 |        eval time =   26015.37 ms /  1163 tokens (   22.37 ms per token,    44.70 tokens per second)
3.15.954.811 I slot print_timing: id  3 | task 6840 |       total time =   26200.73 ms /  1180 tokens
3.15.954.812 I slot print_timing: id  3 | task 6840 |    graphs reused =       7957
3.15.954.849 I slot      release: id  3 | task 6840 | stop processing: n_tokens = 1249, truncated = 0
3.15.954.861 I srv  update_slots: all slots are idle
3.15.966.855 I srv    operator(): Chat format: peg-native
3.15.967.100 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.866 (> 0.100 thold), f_keep = 0.057
3.15.967.102 I srv  get_availabl: updating prompt cache
3.15.967.157 W srv   prompt_save:  - saving prompt with length 1249, total state size = 64.508 MiB (draft: 0.000 MiB)
3.16.021.086 I srv          load:  - looking for better prompt, base f_keep = 0.057, sim = 0.866
3.16.021.094 I srv        update:  - cache state: 6 prompts, 440.089 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
3.16.021.095 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
3.16.021.096 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
3.16.021.097 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
3.16.021.097 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
3.16.021.098 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
3.16.021.098 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
3.16.021.099 I srv  get_availabl: prompt cache update took 54.00 ms
3.16.021.141 I slot launch_slot_: id  3 | task 8004 | processing task, is_child = 0
3.16.021.143 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
3.16.021.143 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
3.16.021.154 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
3.16.021.154 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
3.16.021.163 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
3.16.021.163 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
3.19.180.056 I slot print_timing: id  3 | task 8004 | n_decoded =    137, tg =  45.48 t/s, tg_3s =  45.48 t/s
3.22.186.812 I slot print_timing: id  3 | task 8004 | n_decoded =    273, tg =  45.35 t/s, tg_3s =  45.23 t/s
3.25.201.385 I slot print_timing: id  3 | task 8004 | n_decoded =    410, tg =  45.38 t/s, tg_3s =  45.45 t/s
3.28.214.783 I slot print_timing: id  3 | task 8004 | n_decoded =    545, tg =  45.24 t/s, tg_3s =  44.80 t/s
3.31.222.566 I slot print_timing: id  3 | task 8004 | n_decoded =    680, tg =  45.17 t/s, tg_3s =  44.88 t/s
3.34.223.588 I slot print_timing: id  3 | task 8004 | n_decoded =    814, tg =  45.08 t/s, tg_3s =  44.65 t/s
3.37.226.627 I slot print_timing: id  3 | task 8004 | n_decoded =    948, tg =  45.02 t/s, tg_3s =  44.62 t/s
3.40.227.785 I slot print_timing: id  3 | task 8004 | n_decoded =   1082, tg =  44.97 t/s, tg_3s =  44.65 t/s
3.41.863.744 I slot print_timing: id  3 | task 8004 | prompt eval time =     146.29 ms /    11 tokens (   13.30 ms per token,    75.19 tokens per second)
3.41.863.747 I slot print_timing: id  3 | task 8004 |        eval time =   25696.27 ms /  1155 tokens (   22.25 ms per token,    44.95 tokens per second)
3.41.863.748 I slot print_timing: id  3 | task 8004 |       total time =   25842.56 ms /  1166 tokens
3.41.863.749 I slot print_timing: id  3 | task 8004 |    graphs reused =       9106
3.41.863.788 I slot      release: id  3 | task 8004 | stop processing: n_tokens = 1236, truncated = 0
3.41.863.801 I srv  update_slots: all slots are idle
3.41.875.140 I srv    operator(): Chat format: peg-native
3.41.875.365 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.845 (> 0.100 thold), f_keep = 0.057
3.41.875.367 I srv  get_availabl: updating prompt cache
3.41.875.418 W srv   prompt_save:  - saving prompt with length 1236, total state size = 63.836 MiB (draft: 0.000 MiB)
3.41.945.128 I srv          load:  - looking for better prompt, base f_keep = 0.057, sim = 0.845
3.41.945.136 I srv        update:  - cache state: 7 prompts, 503.926 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
3.41.945.137 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
3.41.945.138 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
3.41.945.140 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
3.41.945.140 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
3.41.945.141 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
3.41.945.142 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
3.41.945.142 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
3.41.945.143 I srv  get_availabl: prompt cache update took 69.78 ms
3.41.945.187 I slot launch_slot_: id  3 | task 9160 | processing task, is_child = 0
3.41.945.187 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
3.41.945.188 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
3.41.945.198 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
3.41.945.198 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
3.41.945.207 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
3.41.945.207 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
3.45.105.513 I slot print_timing: id  3 | task 9160 | n_decoded =    137, tg =  45.53 t/s, tg_3s =  45.53 t/s
3.48.125.555 I slot print_timing: id  3 | task 9160 | n_decoded =    273, tg =  45.28 t/s, tg_3s =  45.03 t/s
3.51.139.613 I slot print_timing: id  3 | task 9160 | n_decoded =    409, tg =  45.23 t/s, tg_3s =  45.12 t/s
3.54.149.800 I slot print_timing: id  3 | task 9160 | n_decoded =    543, tg =  45.05 t/s, tg_3s =  44.52 t/s
3.57.154.584 I slot print_timing: id  3 | task 9160 | n_decoded =    677, tg =  44.96 t/s, tg_3s =  44.60 t/s
4.00.156.767 I slot print_timing: id  3 | task 9160 | n_decoded =    810, tg =  44.85 t/s, tg_3s =  44.30 t/s
4.03.160.463 I slot print_timing: id  3 | task 9160 | n_decoded =    943, tg =  44.77 t/s, tg_3s =  44.28 t/s
4.06.164.703 I slot print_timing: id  3 | task 9160 | n_decoded =   1078, tg =  44.79 t/s, tg_3s =  44.94 t/s
4.09.166.624 I slot print_timing: id  3 | task 9160 | n_decoded =   1212, tg =  44.77 t/s, tg_3s =  44.64 t/s
4.11.379.350 I slot print_timing: id  3 | task 9160 | prompt eval time =     151.53 ms /    13 tokens (   11.66 ms per token,    85.79 tokens per second)
4.11.379.354 I slot print_timing: id  3 | task 9160 |        eval time =   29282.59 ms /  1311 tokens (   22.34 ms per token,    44.77 tokens per second)
4.11.379.355 I slot print_timing: id  3 | task 9160 |       total time =   29434.12 ms /  1324 tokens
4.11.379.355 I slot print_timing: id  3 | task 9160 |    graphs reused =      10410
4.11.379.398 I slot      release: id  3 | task 9160 | stop processing: n_tokens = 1394, truncated = 0
4.11.379.411 I srv  update_slots: all slots are idle
4.11.391.171 I srv    operator(): Chat format: peg-native
4.11.391.406 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.631 (> 0.100 thold), f_keep = 0.050
4.11.391.407 I srv  get_availabl: updating prompt cache
4.11.391.474 W srv   prompt_save:  - saving prompt with length 1394, total state size = 71.997 MiB (draft: 0.000 MiB)
4.11.479.748 I srv          load:  - looking for better prompt, base f_keep = 0.050, sim = 0.631
4.11.479.756 I srv        update:  - cache state: 8 prompts, 575.922 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
4.11.479.757 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
4.11.479.758 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
4.11.479.759 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
4.11.479.759 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
4.11.479.760 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
4.11.479.761 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
4.11.479.762 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
4.11.479.763 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
4.11.479.765 I srv  get_availabl: prompt cache update took 88.36 ms
4.11.479.805 I slot launch_slot_: id  3 | task 10472 | processing task, is_child = 0
4.11.479.805 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
4.11.479.806 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
4.11.479.818 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
4.11.479.818 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
4.11.479.828 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
4.11.479.828 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
4.14.720.277 I slot print_timing: id  3 | task 10472 | n_decoded =    137, tg =  45.62 t/s, tg_3s =  45.62 t/s
4.17.726.512 I slot print_timing: id  3 | task 10472 | n_decoded =    273, tg =  45.43 t/s, tg_3s =  45.24 t/s
4.20.731.297 I slot print_timing: id  3 | task 10472 | n_decoded =    409, tg =  45.37 t/s, tg_3s =  45.26 t/s
4.23.738.322 I slot print_timing: id  3 | task 10472 | n_decoded =    544, tg =  45.25 t/s, tg_3s =  44.89 t/s
4.26.759.781 I slot print_timing: id  3 | task 10472 | n_decoded =    679, tg =  45.14 t/s, tg_3s =  44.68 t/s
4.29.778.103 I slot print_timing: id  3 | task 10472 | n_decoded =    814, tg =  45.07 t/s, tg_3s =  44.73 t/s
4.32.796.815 I slot print_timing: id  3 | task 10472 | n_decoded =    948, tg =  44.97 t/s, tg_3s =  44.39 t/s
4.35.799.373 I slot print_timing: id  3 | task 10472 | n_decoded =   1082, tg =  44.93 t/s, tg_3s =  44.63 t/s
4.38.821.636 I slot print_timing: id  3 | task 10472 | n_decoded =   1216, tg =  44.86 t/s, tg_3s =  44.34 t/s
4.41.842.500 I slot print_timing: id  3 | task 10472 | n_decoded =   1350, tg =  44.81 t/s, tg_3s =  44.36 t/s
4.43.331.234 I slot print_timing: id  3 | task 10472 | prompt eval time =     237.51 ms /    41 tokens (    5.79 ms per token,   172.63 tokens per second)
4.43.331.238 I slot print_timing: id  3 | task 10472 |        eval time =   31613.88 ms /  1416 tokens (   22.33 ms per token,    44.79 tokens per second)
4.43.331.238 I slot print_timing: id  3 | task 10472 |       total time =   31851.38 ms /  1457 tokens
4.43.331.239 I slot print_timing: id  3 | task 10472 |    graphs reused =      11819
4.43.331.282 I slot      release: id  3 | task 10472 | stop processing: n_tokens = 1526, truncated = 0
4.43.331.300 I srv  update_slots: all slots are idle
4.43.342.822 I srv    operator(): Chat format: peg-native
4.43.343.061 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.550 (> 0.100 thold), f_keep = 0.047
4.43.343.062 I srv  get_availabl: updating prompt cache
4.43.343.125 W srv   prompt_save:  - saving prompt with length 1526, total state size = 78.814 MiB (draft: 0.000 MiB)
4.43.424.267 I srv          load:  - looking for better prompt, base f_keep = 0.047, sim = 0.550
4.43.424.276 I srv        update:  - cache state: 9 prompts, 654.736 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
4.43.424.277 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
4.43.424.277 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
4.43.424.278 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
4.43.424.278 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
4.43.424.279 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
4.43.424.279 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
4.43.424.280 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
4.43.424.285 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
4.43.424.285 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
4.43.424.286 I srv  get_availabl: prompt cache update took 81.22 ms
4.43.424.329 I slot launch_slot_: id  3 | task 11889 | processing task, is_child = 0
4.43.424.330 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
4.43.424.331 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
4.43.424.342 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
4.43.424.342 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
4.43.424.362 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
4.43.424.362 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
4.46.803.690 I slot print_timing: id  3 | task 11889 | n_decoded =    136, tg =  45.04 t/s, tg_3s =  45.04 t/s
4.49.814.396 I slot print_timing: id  3 | task 11889 | n_decoded =    271, tg =  44.94 t/s, tg_3s =  44.84 t/s
4.52.829.844 I slot print_timing: id  3 | task 11889 | n_decoded =    406, tg =  44.88 t/s, tg_3s =  44.77 t/s
4.55.850.051 I slot print_timing: id  3 | task 11889 | n_decoded =    541, tg =  44.84 t/s, tg_3s =  44.70 t/s
4.58.857.533 I slot print_timing: id  3 | task 11889 | n_decoded =    676, tg =  44.85 t/s, tg_3s =  44.89 t/s
5.01.866.208 I slot print_timing: id  3 | task 11889 | n_decoded =    811, tg =  44.85 t/s, tg_3s =  44.87 t/s
5.04.873.271 I slot print_timing: id  3 | task 11889 | n_decoded =    945, tg =  44.81 t/s, tg_3s =  44.56 t/s
5.07.889.287 I slot print_timing: id  3 | task 11889 | n_decoded =   1080, tg =  44.80 t/s, tg_3s =  44.76 t/s
5.10.903.442 I slot print_timing: id  3 | task 11889 | n_decoded =   1214, tg =  44.77 t/s, tg_3s =  44.46 t/s
5.13.918.840 I slot print_timing: id  3 | task 11889 | n_decoded =   1348, tg =  44.73 t/s, tg_3s =  44.44 t/s
5.16.929.906 I slot print_timing: id  3 | task 11889 | n_decoded =   1481, tg =  44.68 t/s, tg_3s =  44.17 t/s
5.19.931.612 I slot print_timing: id  3 | task 11889 | n_decoded =   1614, tg =  44.65 t/s, tg_3s =  44.31 t/s
5.22.933.674 I slot print_timing: id  3 | task 11889 | n_decoded =   1746, tg =  44.60 t/s, tg_3s =  43.97 t/s
5.23.160.348 I slot print_timing: id  3 | task 11889 | prompt eval time =     359.80 ms /    58 tokens (    6.20 ms per token,   161.20 tokens per second)
5.23.160.352 I slot print_timing: id  3 | task 11889 |        eval time =   39376.15 ms /  1756 tokens (   22.42 ms per token,    44.60 tokens per second)
5.23.160.352 I slot print_timing: id  3 | task 11889 |       total time =   39735.95 ms /  1814 tokens
5.23.160.353 I slot print_timing: id  3 | task 11889 |    graphs reused =      13566
5.23.160.396 I slot      release: id  3 | task 11889 | stop processing: n_tokens = 1884, truncated = 0
5.23.160.410 I srv  update_slots: all slots are idle
5.23.172.078 I srv    operator(): Chat format: peg-native
5.23.172.317 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.037
5.23.172.318 I srv  get_availabl: updating prompt cache
5.23.172.393 W srv   prompt_save:  - saving prompt with length 1884, total state size = 97.304 MiB (draft: 0.000 MiB)
5.23.272.370 I srv          load:  - looking for better prompt, base f_keep = 0.037, sim = 0.667
5.23.272.381 I srv        update:  - cache state: 10 prompts, 752.040 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
5.23.272.384 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
5.23.272.385 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
5.23.272.386 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
5.23.272.388 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
5.23.272.389 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
5.23.272.390 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
5.23.272.390 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
5.23.272.391 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
5.23.272.392 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
5.23.272.392 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
5.23.272.393 I srv  get_availabl: prompt cache update took 100.07 ms
5.23.272.440 I slot launch_slot_: id  3 | task 13646 | processing task, is_child = 0
5.23.272.441 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
5.23.272.442 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
5.23.272.463 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
5.23.272.463 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
5.23.272.486 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
5.23.272.486 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
5.26.489.545 I slot print_timing: id  3 | task 13646 | n_decoded =    137, tg =  45.60 t/s, tg_3s =  45.60 t/s
5.29.510.467 I slot print_timing: id  3 | task 13646 | n_decoded =    273, tg =  45.31 t/s, tg_3s =  45.02 t/s
5.32.519.042 I slot print_timing: id  3 | task 13646 | n_decoded =    409, tg =  45.27 t/s, tg_3s =  45.20 t/s
5.35.521.280 I slot print_timing: id  3 | task 13646 | n_decoded =    543, tg =  45.11 t/s, tg_3s =  44.63 t/s
5.38.522.822 I slot print_timing: id  3 | task 13646 | n_decoded =    677, tg =  45.02 t/s, tg_3s =  44.64 t/s
5.41.529.722 I slot print_timing: id  3 | task 13646 | n_decoded =    812, tg =  45.00 t/s, tg_3s =  44.90 t/s
5.44.534.997 I slot print_timing: id  3 | task 13646 | n_decoded =    946, tg =  44.94 t/s, tg_3s =  44.59 t/s
5.47.536.754 I slot print_timing: id  3 | task 13646 | n_decoded =   1081, tg =  44.95 t/s, tg_3s =  44.97 t/s
5.50.538.100 I slot print_timing: id  3 | task 13646 | n_decoded =   1215, tg =  44.91 t/s, tg_3s =  44.65 t/s
5.53.545.020 I slot print_timing: id  3 | task 13646 | n_decoded =   1349, tg =  44.88 t/s, tg_3s =  44.56 t/s
5.56.567.080 I slot print_timing: id  3 | task 13646 | n_decoded =   1483, tg =  44.83 t/s, tg_3s =  44.34 t/s
5.59.582.174 I slot print_timing: id  3 | task 13646 | n_decoded =   1617, tg =  44.80 t/s, tg_3s =  44.44 t/s
6.02.588.295 I slot print_timing: id  3 | task 13646 | n_decoded =   1750, tg =  44.75 t/s, tg_3s =  44.24 t/s
6.04.347.773 I slot print_timing: id  3 | task 13646 | prompt eval time =     212.78 ms /    35 tokens (    6.08 ms per token,   164.49 tokens per second)
6.04.347.777 I slot print_timing: id  3 | task 13646 |        eval time =   40862.48 ms /  1828 tokens (   22.35 ms per token,    44.74 tokens per second)
6.04.347.777 I slot print_timing: id  3 | task 13646 |       total time =   41075.26 ms /  1863 tokens
6.04.347.778 I slot print_timing: id  3 | task 13646 |    graphs reused =      15385
6.04.347.817 I slot      release: id  3 | task 13646 | stop processing: n_tokens = 1932, truncated = 0
6.04.347.832 I srv  update_slots: all slots are idle
6.04.359.211 I srv    operator(): Chat format: peg-native
6.04.359.443 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.583 (> 0.100 thold), f_keep = 0.036
6.04.359.444 I srv  get_availabl: updating prompt cache
6.04.359.527 W srv   prompt_save:  - saving prompt with length 1932, total state size = 99.783 MiB (draft: 0.000 MiB)
6.04.461.867 I srv          load:  - looking for better prompt, base f_keep = 0.036, sim = 0.583
6.04.461.875 I srv        update:  - cache state: 11 prompts, 851.823 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
6.04.461.876 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
6.04.461.876 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
6.04.461.877 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
6.04.461.878 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
6.04.461.878 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
6.04.461.879 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
6.04.461.879 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
6.04.461.880 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
6.04.461.881 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
6.04.461.885 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
6.04.461.886 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
6.04.461.887 I srv  get_availabl: prompt cache update took 102.44 ms
6.04.461.924 I slot launch_slot_: id  3 | task 15475 | processing task, is_child = 0
6.04.461.925 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
6.04.461.925 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
6.04.461.936 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
6.04.461.937 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
6.04.461.946 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
6.04.461.946 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
6.07.803.789 I slot print_timing: id  3 | task 15475 | n_decoded =    137, tg =  45.35 t/s, tg_3s =  45.35 t/s
6.10.815.593 I slot print_timing: id  3 | task 15475 | n_decoded =    273, tg =  45.25 t/s, tg_3s =  45.16 t/s
6.13.831.040 I slot print_timing: id  3 | task 15475 | n_decoded =    408, tg =  45.09 t/s, tg_3s =  44.77 t/s
6.16.835.006 I slot print_timing: id  3 | task 15475 | n_decoded =    542, tg =  44.97 t/s, tg_3s =  44.61 t/s
6.19.849.293 I slot print_timing: id  3 | task 15475 | n_decoded =    676, tg =  44.87 t/s, tg_3s =  44.45 t/s
6.22.869.540 I slot print_timing: id  3 | task 15475 | n_decoded =    810, tg =  44.78 t/s, tg_3s =  44.37 t/s
6.25.876.696 I slot print_timing: id  3 | task 15475 | n_decoded =    943, tg =  44.70 t/s, tg_3s =  44.23 t/s
6.27.677.400 I slot print_timing: id  3 | task 15475 | prompt eval time =     320.74 ms /    50 tokens (    6.41 ms per token,   155.89 tokens per second)
6.27.677.403 I slot print_timing: id  3 | task 15475 |        eval time =   22894.70 ms /  1023 tokens (   22.38 ms per token,    44.68 tokens per second)
6.27.677.404 I slot print_timing: id  3 | task 15475 |       total time =   23215.43 ms /  1073 tokens
6.27.677.405 I slot print_timing: id  3 | task 15475 |    graphs reused =      16402
6.27.677.447 I slot      release: id  3 | task 15475 | stop processing: n_tokens = 1142, truncated = 0
6.27.677.457 I srv  update_slots: all slots are idle
6.27.689.146 I srv    operator(): Chat format: peg-native
6.27.689.376 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.574 (> 0.100 thold), f_keep = 0.061
6.27.689.378 I srv  get_availabl: updating prompt cache
6.27.689.435 W srv   prompt_save:  - saving prompt with length 1142, total state size = 58.982 MiB (draft: 0.000 MiB)
6.27.745.908 I srv          load:  - looking for better prompt, base f_keep = 0.061, sim = 0.574
6.27.745.920 I srv        update:  - cache state: 12 prompts, 910.805 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
6.27.745.921 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
6.27.745.922 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
6.27.745.926 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
6.27.745.927 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
6.27.745.927 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
6.27.745.928 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
6.27.745.928 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
6.27.745.929 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
6.27.745.932 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
6.27.745.933 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
6.27.745.933 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
6.27.745.934 I srv        update:    - prompt 0x63c2ea969620:    1142 tokens, checkpoints:  0,    58.982 MiB
6.27.745.935 I srv  get_availabl: prompt cache update took 56.56 ms
6.27.745.974 I slot launch_slot_: id  3 | task 16499 | processing task, is_child = 0
6.27.745.975 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
6.27.745.975 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
6.27.745.985 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
6.27.745.986 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
6.27.745.995 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
6.27.745.995 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
6.31.081.196 I slot print_timing: id  3 | task 16499 | n_decoded =    136, tg =  45.19 t/s, tg_3s =  45.19 t/s
6.34.083.080 I slot print_timing: id  3 | task 16499 | n_decoded =    270, tg =  44.92 t/s, tg_3s =  44.64 t/s
6.37.085.339 I slot print_timing: id  3 | task 16499 | n_decoded =    404, tg =  44.82 t/s, tg_3s =  44.63 t/s
6.40.104.537 I slot print_timing: id  3 | task 16499 | n_decoded =    541, tg =  44.96 t/s, tg_3s =  45.38 t/s
6.43.118.727 I slot print_timing: id  3 | task 16499 | n_decoded =    677, tg =  44.99 t/s, tg_3s =  45.12 t/s
6.46.138.723 I slot print_timing: id  3 | task 16499 | n_decoded =    813, tg =  45.00 t/s, tg_3s =  45.03 t/s
6.49.155.336 I slot print_timing: id  3 | task 16499 | n_decoded =    948, tg =  44.96 t/s, tg_3s =  44.75 t/s
6.52.168.738 I slot print_timing: id  3 | task 16499 | n_decoded =   1083, tg =  44.94 t/s, tg_3s =  44.80 t/s
6.54.777.722 I slot print_timing: id  3 | task 16499 | prompt eval time =     325.79 ms /    52 tokens (    6.27 ms per token,   159.61 tokens per second)
6.54.777.725 I slot print_timing: id  3 | task 16499 |        eval time =   26705.92 ms /  1199 tokens (   22.27 ms per token,    44.90 tokens per second)
6.54.777.726 I slot print_timing: id  3 | task 16499 |       total time =   27031.71 ms /  1251 tokens
6.54.777.726 I slot print_timing: id  3 | task 16499 |    graphs reused =      17594
6.54.777.772 I slot      release: id  3 | task 16499 | stop processing: n_tokens = 1320, truncated = 0
6.54.777.785 I srv  update_slots: all slots are idle
6.54.789.275 I srv    operator(): Chat format: peg-native
6.54.789.502 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.651 (> 0.100 thold), f_keep = 0.054
6.54.789.504 I srv  get_availabl: updating prompt cache
6.54.789.557 W srv   prompt_save:  - saving prompt with length 1320, total state size = 68.175 MiB (draft: 0.000 MiB)
6.54.853.468 I srv          load:  - looking for better prompt, base f_keep = 0.054, sim = 0.651
6.54.853.477 I srv        update:  - cache state: 13 prompts, 978.980 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
6.54.853.479 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
6.54.853.482 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
6.54.853.483 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
6.54.853.483 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
6.54.853.484 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
6.54.853.485 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
6.54.853.485 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
6.54.853.488 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
6.54.853.489 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
6.54.853.489 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
6.54.853.490 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
6.54.853.490 I srv        update:    - prompt 0x63c2ea969620:    1142 tokens, checkpoints:  0,    58.982 MiB
6.54.853.491 I srv        update:    - prompt 0x63c2eaf04ba0:    1320 tokens, checkpoints:  0,    68.175 MiB
6.54.853.494 I srv  get_availabl: prompt cache update took 63.99 ms
6.54.853.536 I slot launch_slot_: id  3 | task 17699 | processing task, is_child = 0
6.54.853.537 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
6.54.853.538 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
6.54.853.548 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
6.54.853.548 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
6.54.853.557 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
6.54.853.557 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
6.58.068.405 I slot print_timing: id  3 | task 17699 | n_decoded =    137, tg =  45.63 t/s, tg_3s =  45.63 t/s
7.01.076.615 I slot print_timing: id  3 | task 17699 | n_decoded =    273, tg =  45.42 t/s, tg_3s =  45.21 t/s
7.04.088.770 I slot print_timing: id  3 | task 17699 | n_decoded =    409, tg =  45.33 t/s, tg_3s =  45.15 t/s
7.07.096.062 I slot print_timing: id  3 | task 17699 | n_decoded =    544, tg =  45.22 t/s, tg_3s =  44.89 t/s
7.10.097.166 I slot print_timing: id  3 | task 17699 | n_decoded =    678, tg =  45.11 t/s, tg_3s =  44.65 t/s
7.13.103.610 I slot print_timing: id  3 | task 17699 | n_decoded =    812, tg =  45.02 t/s, tg_3s =  44.57 t/s
7.16.105.204 I slot print_timing: id  3 | task 17699 | n_decoded =    945, tg =  44.92 t/s, tg_3s =  44.31 t/s
7.19.121.180 I slot print_timing: id  3 | task 17699 | n_decoded =   1079, tg =  44.85 t/s, tg_3s =  44.43 t/s
7.22.127.689 I slot print_timing: id  3 | task 17699 | n_decoded =   1212, tg =  44.79 t/s, tg_3s =  44.24 t/s
7.25.140.041 I slot print_timing: id  3 | task 17699 | n_decoded =   1346, tg =  44.76 t/s, tg_3s =  44.48 t/s
7.28.141.442 I slot print_timing: id  3 | task 17699 | n_decoded =   1479, tg =  44.72 t/s, tg_3s =  44.31 t/s
7.31.156.481 I slot print_timing: id  3 | task 17699 | n_decoded =   1613, tg =  44.69 t/s, tg_3s =  44.44 t/s
7.34.161.310 I slot print_timing: id  3 | task 17699 | n_decoded =   1746, tg =  44.66 t/s, tg_3s =  44.26 t/s
7.37.164.658 I slot print_timing: id  3 | task 17699 | n_decoded =   1879, tg =  44.63 t/s, tg_3s =  44.28 t/s
7.40.186.409 I slot print_timing: id  3 | task 17699 | n_decoded =   2012, tg =  44.59 t/s, tg_3s =  44.01 t/s
7.43.198.444 I slot print_timing: id  3 | task 17699 | n_decoded =   2144, tg =  44.54 t/s, tg_3s =  43.82 t/s
7.46.209.542 I slot print_timing: id  3 | task 17699 | n_decoded =   2275, tg =  44.48 t/s, tg_3s =  43.51 t/s
7.49.231.761 I slot print_timing: id  3 | task 17699 | n_decoded =   2407, tg =  44.44 t/s, tg_3s =  43.68 t/s
7.52.251.517 I slot print_timing: id  3 | task 17699 | n_decoded =   2538, tg =  44.38 t/s, tg_3s =  43.38 t/s
7.55.264.540 I slot print_timing: id  3 | task 17699 | n_decoded =   2669, tg =  44.34 t/s, tg_3s =  43.48 t/s
7.58.268.320 I slot print_timing: id  3 | task 17699 | n_decoded =   2798, tg =  44.27 t/s, tg_3s =  42.95 t/s
8.01.268.838 I slot print_timing: id  3 | task 17699 | n_decoded =   2927, tg =  44.21 t/s, tg_3s =  42.99 t/s
8.04.271.082 I slot print_timing: id  3 | task 17699 | n_decoded =   3050, tg =  44.07 t/s, tg_3s =  40.97 t/s
8.07.291.459 I slot print_timing: id  3 | task 17699 | n_decoded =   3172, tg =  43.92 t/s, tg_3s =  40.39 t/s
8.09.684.188 I slot print_timing: id  3 | task 17699 | prompt eval time =     212.27 ms /    38 tokens (    5.59 ms per token,   179.02 tokens per second)
8.09.684.191 I slot print_timing: id  3 | task 17699 |        eval time =   74618.34 ms /  3268 tokens (   22.83 ms per token,    43.80 tokens per second)
8.09.684.191 I slot print_timing: id  3 | task 17699 |       total time =   74830.61 ms /  3306 tokens
8.09.684.192 I slot print_timing: id  3 | task 17699 |    graphs reused =      20847
8.09.684.229 I slot      release: id  3 | task 17699 | stop processing: n_tokens = 3376, truncated = 0
8.09.684.247 I srv  update_slots: all slots are idle
8.09.695.719 I srv    operator(): Chat format: peg-native
8.09.695.948 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.609 (> 0.100 thold), f_keep = 0.021
8.09.695.950 I srv  get_availabl: updating prompt cache
8.09.696.077 W srv   prompt_save:  - saving prompt with length 3376, total state size = 174.361 MiB (draft: 0.000 MiB)
8.09.855.089 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.609
8.09.855.100 I srv        update:  - cache state: 14 prompts, 1153.341 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
8.09.855.101 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
8.09.855.102 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
8.09.855.103 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
8.09.855.103 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
8.09.855.104 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
8.09.855.105 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
8.09.855.106 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
8.09.855.106 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
8.09.855.107 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
8.09.855.107 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
8.09.855.108 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
8.09.855.108 I srv        update:    - prompt 0x63c2ea969620:    1142 tokens, checkpoints:  0,    58.982 MiB
8.09.855.109 I srv        update:    - prompt 0x63c2eaf04ba0:    1320 tokens, checkpoints:  0,    68.175 MiB
8.09.855.109 I srv        update:    - prompt 0x63c2f1f09fb0:    3376 tokens, checkpoints:  0,   174.361 MiB
8.09.855.110 I srv  get_availabl: prompt cache update took 159.16 ms
8.09.855.152 I slot launch_slot_: id  3 | task 20968 | processing task, is_child = 0
8.09.855.152 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
8.09.855.153 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
8.09.855.168 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
8.09.855.168 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
8.09.855.178 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
8.09.855.178 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
8.13.102.368 I slot print_timing: id  3 | task 20968 | n_decoded =    136, tg =  45.30 t/s, tg_3s =  45.30 t/s
8.16.117.874 I slot print_timing: id  3 | task 20968 | n_decoded =    271, tg =  45.03 t/s, tg_3s =  44.77 t/s
8.19.131.574 I slot print_timing: id  3 | task 20968 | n_decoded =    407, tg =  45.07 t/s, tg_3s =  45.13 t/s
8.22.132.583 I slot print_timing: id  3 | task 20968 | n_decoded =    543, tg =  45.13 t/s, tg_3s =  45.32 t/s
8.25.143.932 I slot print_timing: id  3 | task 20968 | n_decoded =    679, tg =  45.14 t/s, tg_3s =  45.16 t/s
8.28.146.616 I slot print_timing: id  3 | task 20968 | n_decoded =    814, tg =  45.11 t/s, tg_3s =  44.96 t/s
8.31.168.006 I slot print_timing: id  3 | task 20968 | n_decoded =    949, tg =  45.05 t/s, tg_3s =  44.68 t/s
8.34.183.068 I slot print_timing: id  3 | task 20968 | n_decoded =   1084, tg =  45.01 t/s, tg_3s =  44.78 t/s
8.37.196.676 I slot print_timing: id  3 | task 20968 | n_decoded =   1218, tg =  44.95 t/s, tg_3s =  44.46 t/s
8.40.211.718 I slot print_timing: id  3 | task 20968 | n_decoded =   1352, tg =  44.90 t/s, tg_3s =  44.44 t/s
8.43.220.848 I slot print_timing: id  3 | task 20968 | n_decoded =   1485, tg =  44.84 t/s, tg_3s =  44.20 t/s
8.44.061.677 I slot print_timing: id  3 | task 20968 | prompt eval time =     245.03 ms /    45 tokens (    5.45 ms per token,   183.65 tokens per second)
8.44.061.680 I slot print_timing: id  3 | task 20968 |        eval time =   33961.45 ms /  1522 tokens (   22.31 ms per token,    44.82 tokens per second)
8.44.061.681 I slot print_timing: id  3 | task 20968 |       total time =   34206.48 ms /  1567 tokens
8.44.061.682 I slot print_timing: id  3 | task 20968 |    graphs reused =      22361
8.44.061.727 I slot      release: id  3 | task 20968 | stop processing: n_tokens = 1636, truncated = 0
8.44.061.741 I srv  update_slots: all slots are idle
8.44.073.144 I srv    operator(): Chat format: peg-native
8.44.073.372 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.673 (> 0.100 thold), f_keep = 0.043
8.44.073.373 I srv  get_availabl: updating prompt cache
8.44.073.438 W srv   prompt_save:  - saving prompt with length 1636, total state size = 84.495 MiB (draft: 0.000 MiB)
8.44.147.491 I srv          load:  - looking for better prompt, base f_keep = 0.043, sim = 0.673
8.44.147.502 I srv        update:  - cache state: 15 prompts, 1237.837 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
8.44.147.506 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
8.44.147.507 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
8.44.147.507 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
8.44.147.508 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
8.44.147.508 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
8.44.147.511 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
8.44.147.512 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
8.44.147.512 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
8.44.147.513 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
8.44.147.513 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
8.44.147.514 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
8.44.147.515 I srv        update:    - prompt 0x63c2ea969620:    1142 tokens, checkpoints:  0,    58.982 MiB
8.44.147.516 I srv        update:    - prompt 0x63c2eaf04ba0:    1320 tokens, checkpoints:  0,    68.175 MiB
8.44.147.516 I srv        update:    - prompt 0x63c2f1f09fb0:    3376 tokens, checkpoints:  0,   174.361 MiB
8.44.147.517 I srv        update:    - prompt 0x63c2ea962870:    1636 tokens, checkpoints:  0,    84.495 MiB
8.44.147.517 I srv  get_availabl: prompt cache update took 74.14 ms
8.44.147.556 I slot launch_slot_: id  3 | task 22491 | processing task, is_child = 0
8.44.147.557 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
8.44.147.558 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
8.44.147.568 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
8.44.147.569 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
8.44.147.589 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
8.44.147.589 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
8.47.362.420 I slot print_timing: id  3 | task 22491 | n_decoded =    137, tg =  45.54 t/s, tg_3s =  45.54 t/s
8.50.377.002 I slot print_timing: id  3 | task 22491 | n_decoded =    272, tg =  45.16 t/s, tg_3s =  44.78 t/s
8.53.386.920 I slot print_timing: id  3 | task 22491 | n_decoded =    407, tg =  45.06 t/s, tg_3s =  44.85 t/s
8.56.401.774 I slot print_timing: id  3 | task 22491 | n_decoded =    541, tg =  44.90 t/s, tg_3s =  44.45 t/s
8.59.419.611 I slot print_timing: id  3 | task 22491 | n_decoded =    675, tg =  44.80 t/s, tg_3s =  44.40 t/s
9.02.434.596 I slot print_timing: id  3 | task 22491 | n_decoded =    809, tg =  44.74 t/s, tg_3s =  44.44 t/s
9.05.441.327 I slot print_timing: id  3 | task 22491 | n_decoded =    943, tg =  44.72 t/s, tg_3s =  44.57 t/s
9.08.442.352 I slot print_timing: id  3 | task 22491 | n_decoded =   1077, tg =  44.71 t/s, tg_3s =  44.65 t/s
9.11.464.459 I slot print_timing: id  3 | task 22491 | n_decoded =   1211, tg =  44.67 t/s, tg_3s =  44.34 t/s
9.14.479.576 I slot print_timing: id  3 | task 22491 | n_decoded =   1345, tg =  44.65 t/s, tg_3s =  44.44 t/s
9.17.482.300 I slot print_timing: id  3 | task 22491 | n_decoded =   1478, tg =  44.61 t/s, tg_3s =  44.29 t/s
9.20.501.874 I slot print_timing: id  3 | task 22491 | n_decoded =   1612, tg =  44.59 t/s, tg_3s =  44.38 t/s
9.23.522.661 I slot print_timing: id  3 | task 22491 | n_decoded =   1745, tg =  44.55 t/s, tg_3s =  44.03 t/s
9.26.539.938 I slot print_timing: id  3 | task 22491 | n_decoded =   1878, tg =  44.52 t/s, tg_3s =  44.08 t/s
9.29.539.985 I slot print_timing: id  3 | task 22491 | n_decoded =   2009, tg =  44.46 t/s, tg_3s =  43.67 t/s
9.32.559.761 I slot print_timing: id  3 | task 22491 | n_decoded =   2141, tg =  44.41 t/s, tg_3s =  43.71 t/s
9.35.578.171 I slot print_timing: id  3 | task 22491 | n_decoded =   2272, tg =  44.35 t/s, tg_3s =  43.40 t/s
9.38.597.335 I slot print_timing: id  3 | task 22491 | n_decoded =   2403, tg =  44.30 t/s, tg_3s =  43.39 t/s
9.41.323.499 I slot print_timing: id  3 | task 22491 | prompt eval time =     206.34 ms /    34 tokens (    6.07 ms per token,   164.77 tokens per second)
9.41.323.503 I slot print_timing: id  3 | task 22491 |        eval time =   56969.53 ms /  2521 tokens (   22.60 ms per token,    44.25 tokens per second)
9.41.323.503 I slot print_timing: id  3 | task 22491 |       total time =   57175.88 ms /  2555 tokens
9.41.323.504 I slot print_timing: id  3 | task 22491 |    graphs reused =      24870
9.41.323.543 I slot      release: id  3 | task 22491 | stop processing: n_tokens = 2624, truncated = 0
9.41.323.559 I srv  update_slots: all slots are idle
9.41.334.934 I srv    operator(): Chat format: peg-native
9.41.335.173 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.660 (> 0.100 thold), f_keep = 0.027
9.41.335.175 I srv  get_availabl: updating prompt cache
9.41.335.276 W srv   prompt_save:  - saving prompt with length 2624, total state size = 135.523 MiB (draft: 0.000 MiB)
9.41.451.753 I srv          load:  - looking for better prompt, base f_keep = 0.027, sim = 0.660
9.41.451.764 I srv        update:  - cache state: 16 prompts, 1373.359 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
9.41.451.768 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
9.41.451.769 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
9.41.451.769 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
9.41.451.770 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
9.41.451.773 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
9.41.451.773 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
9.41.451.774 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
9.41.451.774 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
9.41.451.775 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
9.41.451.775 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
9.41.451.778 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
9.41.451.778 I srv        update:    - prompt 0x63c2ea969620:    1142 tokens, checkpoints:  0,    58.982 MiB
9.41.451.779 I srv        update:    - prompt 0x63c2eaf04ba0:    1320 tokens, checkpoints:  0,    68.175 MiB
9.41.451.779 I srv        update:    - prompt 0x63c2f1f09fb0:    3376 tokens, checkpoints:  0,   174.361 MiB
9.41.451.780 I srv        update:    - prompt 0x63c2ea962870:    1636 tokens, checkpoints:  0,    84.495 MiB
9.41.451.780 I srv        update:    - prompt 0x63c2f1efda40:    2624 tokens, checkpoints:  0,   135.523 MiB
9.41.451.782 I srv  get_availabl: prompt cache update took 116.61 ms
9.41.451.824 I slot launch_slot_: id  3 | task 25013 | processing task, is_child = 0
9.41.451.824 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
9.41.451.825 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
9.41.451.837 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
9.41.451.837 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
9.41.451.857 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
9.41.451.857 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
9.44.678.427 I slot print_timing: id  3 | task 25013 | n_decoded =    137, tg =  45.43 t/s, tg_3s =  45.43 t/s
9.47.695.749 I slot print_timing: id  3 | task 25013 | n_decoded =    272, tg =  45.08 t/s, tg_3s =  44.74 t/s
9.50.697.122 I slot print_timing: id  3 | task 25013 | n_decoded =    407, tg =  45.05 t/s, tg_3s =  44.98 t/s
9.53.709.906 I slot print_timing: id  3 | task 25013 | n_decoded =    541, tg =  44.91 t/s, tg_3s =  44.48 t/s
9.56.722.593 I slot print_timing: id  3 | task 25013 | n_decoded =    675, tg =  44.82 t/s, tg_3s =  44.48 t/s
9.59.727.192 I slot print_timing: id  3 | task 25013 | n_decoded =    810, tg =  44.84 t/s, tg_3s =  44.93 t/s
10.02.747.893 I slot print_timing: id  3 | task 25013 | n_decoded =    945, tg =  44.82 t/s, tg_3s =  44.69 t/s
10.05.764.949 I slot print_timing: id  3 | task 25013 | n_decoded =   1080, tg =  44.81 t/s, tg_3s =  44.75 t/s
10.08.776.572 I slot print_timing: id  3 | task 25013 | n_decoded =   1214, tg =  44.77 t/s, tg_3s =  44.49 t/s
10.11.790.641 I slot print_timing: id  3 | task 25013 | n_decoded =   1348, tg =  44.74 t/s, tg_3s =  44.46 t/s
10.14.804.020 I slot print_timing: id  3 | task 25013 | n_decoded =   1481, tg =  44.69 t/s, tg_3s =  44.14 t/s
10.17.818.030 I slot print_timing: id  3 | task 25013 | n_decoded =   1614, tg =  44.64 t/s, tg_3s =  44.13 t/s
10.20.824.058 I slot print_timing: id  3 | task 25013 | n_decoded =   1746, tg =  44.58 t/s, tg_3s =  43.91 t/s
10.23.844.232 I slot print_timing: id  3 | task 25013 | n_decoded =   1879, tg =  44.55 t/s, tg_3s =  44.04 t/s
10.26.847.149 I slot print_timing: id  3 | task 25013 | n_decoded =   2010, tg =  44.48 t/s, tg_3s =  43.62 t/s
10.29.864.191 I slot print_timing: id  3 | task 25013 | n_decoded =   2141, tg =  44.42 t/s, tg_3s =  43.42 t/s
10.32.867.384 I slot print_timing: id  3 | task 25013 | n_decoded =   2271, tg =  44.35 t/s, tg_3s =  43.29 t/s
10.35.880.543 I slot print_timing: id  3 | task 25013 | n_decoded =   2402, tg =  44.30 t/s, tg_3s =  43.48 t/s
10.38.881.749 I slot print_timing: id  3 | task 25013 | n_decoded =   2532, tg =  44.25 t/s, tg_3s =  43.32 t/s
10.41.896.781 I slot print_timing: id  3 | task 25013 | n_decoded =   2663, tg =  44.21 t/s, tg_3s =  43.45 t/s
10.43.681.533 I slot print_timing: id  3 | task 25013 | prompt eval time =     210.68 ms /    36 tokens (    5.85 ms per token,   170.88 tokens per second)
10.43.681.537 I slot print_timing: id  3 | task 25013 |        eval time =   62018.97 ms /  2740 tokens (   22.63 ms per token,    44.18 tokens per second)
10.43.681.538 I slot print_timing: id  3 | task 25013 |       total time =   62229.64 ms /  2776 tokens
10.43.681.538 I slot print_timing: id  3 | task 25013 |    graphs reused =      27597
10.43.681.578 I slot      release: id  3 | task 25013 | stop processing: n_tokens = 2845, truncated = 0
10.43.681.595 I srv  update_slots: all slots are idle
10.43.693.143 I srv    operator(): Chat format: peg-native
10.43.693.380 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.729 (> 0.100 thold), f_keep = 0.025
10.43.693.382 I srv  get_availabl: updating prompt cache
10.43.693.479 W srv   prompt_save:  - saving prompt with length 2845, total state size = 146.937 MiB (draft: 0.000 MiB)
10.43.821.188 I srv          load:  - looking for better prompt, base f_keep = 0.025, sim = 0.729
10.43.821.199 I srv        update:  - cache state: 17 prompts, 1520.296 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
10.43.821.200 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
10.43.821.200 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
10.43.821.201 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
10.43.821.202 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
10.43.821.206 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
10.43.821.207 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
10.43.821.207 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
10.43.821.208 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
10.43.821.208 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
10.43.821.209 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
10.43.821.209 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
10.43.821.212 I srv        update:    - prompt 0x63c2ea969620:    1142 tokens, checkpoints:  0,    58.982 MiB
10.43.821.213 I srv        update:    - prompt 0x63c2eaf04ba0:    1320 tokens, checkpoints:  0,    68.175 MiB
10.43.821.213 I srv        update:    - prompt 0x63c2f1f09fb0:    3376 tokens, checkpoints:  0,   174.361 MiB
10.43.821.214 I srv        update:    - prompt 0x63c2ea962870:    1636 tokens, checkpoints:  0,    84.495 MiB
10.43.821.214 I srv        update:    - prompt 0x63c2f1efda40:    2624 tokens, checkpoints:  0,   135.523 MiB
10.43.821.215 I srv        update:    - prompt 0x63c2f1e7c4d0:    2845 tokens, checkpoints:  0,   146.937 MiB
10.43.821.216 I srv  get_availabl: prompt cache update took 127.83 ms
10.43.821.256 I slot launch_slot_: id  3 | task 27754 | processing task, is_child = 0
10.43.821.256 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
10.43.821.257 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
10.43.821.269 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
10.43.821.270 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
10.43.821.289 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
10.43.821.289 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
10.47.069.654 I slot print_timing: id  3 | task 27754 | n_decoded =    137, tg =  45.59 t/s, tg_3s =  45.59 t/s
10.50.076.799 I slot print_timing: id  3 | task 27754 | n_decoded =    273, tg =  45.41 t/s, tg_3s =  45.23 t/s
10.53.090.861 I slot print_timing: id  3 | task 27754 | n_decoded =    410, tg =  45.42 t/s, tg_3s =  45.45 t/s
10.56.096.112 I slot print_timing: id  3 | task 27754 | n_decoded =    545, tg =  45.30 t/s, tg_3s =  44.92 t/s
10.59.105.403 I slot print_timing: id  3 | task 27754 | n_decoded =    680, tg =  45.21 t/s, tg_3s =  44.86 t/s
11.02.117.969 I slot print_timing: id  3 | task 27754 | n_decoded =    815, tg =  45.14 t/s, tg_3s =  44.81 t/s
11.05.129.156 I slot print_timing: id  3 | task 27754 | n_decoded =    949, tg =  45.05 t/s, tg_3s =  44.50 t/s
11.08.129.194 I slot print_timing: id  3 | task 27754 | n_decoded =   1083, tg =  45.00 t/s, tg_3s =  44.67 t/s
11.11.135.956 I slot print_timing: id  3 | task 27754 | n_decoded =   1216, tg =  44.92 t/s, tg_3s =  44.23 t/s
11.14.157.284 I slot print_timing: id  3 | task 27754 | n_decoded =   1350, tg =  44.86 t/s, tg_3s =  44.35 t/s
11.17.172.466 I slot print_timing: id  3 | task 27754 | n_decoded =   1483, tg =  44.79 t/s, tg_3s =  44.11 t/s
11.17.851.371 I slot print_timing: id  3 | task 27754 | prompt eval time =     243.06 ms /    26 tokens (    9.35 ms per token,   106.97 tokens per second)
11.17.851.374 I slot print_timing: id  3 | task 27754 |        eval time =   33786.99 ms /  1513 tokens (   22.33 ms per token,    44.78 tokens per second)
11.17.851.375 I slot print_timing: id  3 | task 27754 |       total time =   34030.05 ms /  1539 tokens
11.17.851.376 I slot print_timing: id  3 | task 27754 |    graphs reused =      29102
11.17.851.422 I slot      release: id  3 | task 27754 | stop processing: n_tokens = 1608, truncated = 0
11.17.851.435 I srv  update_slots: all slots are idle
11.17.862.796 I srv    operator(): Chat format: peg-native
11.17.863.032 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.795 (> 0.100 thold), f_keep = 0.044
11.17.863.034 I srv  get_availabl: updating prompt cache
11.17.863.095 W srv   prompt_save:  - saving prompt with length 1608, total state size = 83.049 MiB (draft: 0.000 MiB)
11.17.923.459 I srv          load:  - looking for better prompt, base f_keep = 0.044, sim = 0.795
11.17.923.472 I srv        update:  - cache state: 18 prompts, 1603.345 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
11.17.923.473 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
11.17.923.474 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
11.17.923.475 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
11.17.923.475 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
11.17.923.476 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
11.17.923.476 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
11.17.923.479 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
11.17.923.480 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
11.17.923.480 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
11.17.923.481 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
11.17.923.481 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
11.17.923.481 I srv        update:    - prompt 0x63c2ea969620:    1142 tokens, checkpoints:  0,    58.982 MiB
11.17.923.483 I srv        update:    - prompt 0x63c2eaf04ba0:    1320 tokens, checkpoints:  0,    68.175 MiB
11.17.923.484 I srv        update:    - prompt 0x63c2f1f09fb0:    3376 tokens, checkpoints:  0,   174.361 MiB
11.17.923.484 I srv        update:    - prompt 0x63c2ea962870:    1636 tokens, checkpoints:  0,    84.495 MiB
11.17.923.485 I srv        update:    - prompt 0x63c2f1efda40:    2624 tokens, checkpoints:  0,   135.523 MiB
11.17.923.485 I srv        update:    - prompt 0x63c2f1e7c4d0:    2845 tokens, checkpoints:  0,   146.937 MiB
11.17.923.486 I srv        update:    - prompt 0x63c2eaaf4520:    1608 tokens, checkpoints:  0,    83.049 MiB
11.17.923.487 I srv  get_availabl: prompt cache update took 60.45 ms
11.17.923.530 I slot launch_slot_: id  3 | task 29268 | processing task, is_child = 0
11.17.923.531 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
11.17.923.532 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
11.17.923.542 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
11.17.923.543 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
11.17.923.552 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
11.17.923.552 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
11.21.123.021 I slot print_timing: id  3 | task 29268 | n_decoded =    137, tg =  45.62 t/s, tg_3s =  45.62 t/s
11.24.129.795 I slot print_timing: id  3 | task 29268 | n_decoded =    272, tg =  45.26 t/s, tg_3s =  44.90 t/s
11.27.134.464 I slot print_timing: id  3 | task 29268 | n_decoded =    407, tg =  45.15 t/s, tg_3s =  44.93 t/s
11.30.144.523 I slot print_timing: id  3 | task 29268 | n_decoded =    541, tg =  44.99 t/s, tg_3s =  44.52 t/s
11.33.153.790 I slot print_timing: id  3 | task 29268 | n_decoded =    675, tg =  44.90 t/s, tg_3s =  44.53 t/s
11.33.370.337 I slot print_timing: id  3 | task 29268 | prompt eval time =     196.05 ms /    18 tokens (   10.89 ms per token,    91.81 tokens per second)
11.33.370.342 I slot print_timing: id  3 | task 29268 |        eval time =   15250.71 ms /   684 tokens (   22.30 ms per token,    44.85 tokens per second)
11.33.370.342 I slot print_timing: id  3 | task 29268 |       total time =   15446.76 ms /   702 tokens
11.33.370.343 I slot print_timing: id  3 | task 29268 |    graphs reused =      29781
11.33.370.383 I slot      release: id  3 | task 29268 | stop processing: n_tokens = 771, truncated = 0
11.33.370.394 I srv  update_slots: all slots are idle
11.33.381.781 I srv    operator(): Chat format: peg-native
11.33.382.029 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.654 (> 0.100 thold), f_keep = 0.091
11.33.382.031 I srv  get_availabl: updating prompt cache
11.33.382.083 W srv   prompt_save:  - saving prompt with length 771, total state size = 39.821 MiB (draft: 0.000 MiB)
11.33.412.028 I srv          load:  - looking for better prompt, base f_keep = 0.091, sim = 0.654
11.33.412.043 I srv        update:  - cache state: 19 prompts, 1643.166 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
11.33.412.043 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
11.33.412.044 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
11.33.412.045 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
11.33.412.045 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
11.33.412.046 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
11.33.412.047 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
11.33.412.047 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
11.33.412.048 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
11.33.412.052 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
11.33.412.053 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
11.33.412.053 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
11.33.412.054 I srv        update:    - prompt 0x63c2ea969620:    1142 tokens, checkpoints:  0,    58.982 MiB
11.33.412.054 I srv        update:    - prompt 0x63c2eaf04ba0:    1320 tokens, checkpoints:  0,    68.175 MiB
11.33.412.055 I srv        update:    - prompt 0x63c2f1f09fb0:    3376 tokens, checkpoints:  0,   174.361 MiB
11.33.412.055 I srv        update:    - prompt 0x63c2ea962870:    1636 tokens, checkpoints:  0,    84.495 MiB
11.33.412.058 I srv        update:    - prompt 0x63c2f1efda40:    2624 tokens, checkpoints:  0,   135.523 MiB
11.33.412.059 I srv        update:    - prompt 0x63c2f1e7c4d0:    2845 tokens, checkpoints:  0,   146.937 MiB
11.33.412.059 I srv        update:    - prompt 0x63c2eaaf4520:    1608 tokens, checkpoints:  0,    83.049 MiB
11.33.412.060 I srv        update:    - prompt 0x63c2f1e62ea0:     771 tokens, checkpoints:  0,    39.821 MiB
11.33.412.061 I srv  get_availabl: prompt cache update took 30.03 ms
11.33.412.121 I slot launch_slot_: id  3 | task 29953 | processing task, is_child = 0
11.33.412.122 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
11.33.412.123 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
11.33.412.135 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
11.33.412.136 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
11.33.412.145 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
11.33.412.145 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
11.36.637.324 I slot print_timing: id  3 | task 29953 | n_decoded =    136, tg =  45.23 t/s, tg_3s =  45.23 t/s
11.39.642.338 I slot print_timing: id  3 | task 29953 | n_decoded =    272, tg =  45.24 t/s, tg_3s =  45.26 t/s
11.42.648.871 I slot print_timing: id  3 | task 29953 | n_decoded =    408, tg =  45.24 t/s, tg_3s =  45.23 t/s
11.45.651.502 I slot print_timing: id  3 | task 29953 | n_decoded =    543, tg =  45.17 t/s, tg_3s =  44.96 t/s
11.48.667.889 I slot print_timing: id  3 | task 29953 | n_decoded =    678, tg =  45.09 t/s, tg_3s =  44.76 t/s
11.51.681.237 I slot print_timing: id  3 | task 29953 | n_decoded =    813, tg =  45.04 t/s, tg_3s =  44.80 t/s
11.54.692.970 I slot print_timing: id  3 | task 29953 | n_decoded =    947, tg =  44.96 t/s, tg_3s =  44.49 t/s
11.57.693.245 I slot print_timing: id  3 | task 29953 | n_decoded =   1081, tg =  44.92 t/s, tg_3s =  44.66 t/s
11.58.790.061 I slot print_timing: id  3 | task 29953 | prompt eval time =     218.40 ms /    37 tokens (    5.90 ms per token,   169.42 tokens per second)
11.58.790.064 I slot print_timing: id  3 | task 29953 |        eval time =   25159.50 ms /  1130 tokens (   22.27 ms per token,    44.91 tokens per second)
11.58.790.065 I slot print_timing: id  3 | task 29953 |       total time =   25377.89 ms /  1167 tokens
11.58.790.065 I slot print_timing: id  3 | task 29953 |    graphs reused =      30905
11.58.790.109 I slot      release: id  3 | task 29953 | stop processing: n_tokens = 1236, truncated = 0
11.58.790.122 I srv  update_slots: all slots are idle
11.58.801.209 I srv    operator(): Chat format: peg-native
11.58.801.446 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.729 (> 0.100 thold), f_keep = 0.057
11.58.801.448 I srv  get_availabl: updating prompt cache
11.58.801.502 W srv   prompt_save:  - saving prompt with length 1236, total state size = 63.836 MiB (draft: 0.000 MiB)
11.58.853.165 I srv          load:  - looking for better prompt, base f_keep = 0.057, sim = 0.729
11.58.853.176 I srv        update:  - cache state: 20 prompts, 1707.002 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
11.58.853.178 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
11.58.853.179 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
11.58.853.182 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
11.58.853.183 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
11.58.853.183 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
11.58.853.184 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
11.58.853.185 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
11.58.853.185 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
11.58.853.188 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
11.58.853.188 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
11.58.853.189 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
11.58.853.190 I srv        update:    - prompt 0x63c2ea969620:    1142 tokens, checkpoints:  0,    58.982 MiB
11.58.853.190 I srv        update:    - prompt 0x63c2eaf04ba0:    1320 tokens, checkpoints:  0,    68.175 MiB
11.58.853.193 I srv        update:    - prompt 0x63c2f1f09fb0:    3376 tokens, checkpoints:  0,   174.361 MiB
11.58.853.194 I srv        update:    - prompt 0x63c2ea962870:    1636 tokens, checkpoints:  0,    84.495 MiB
11.58.853.194 I srv        update:    - prompt 0x63c2f1efda40:    2624 tokens, checkpoints:  0,   135.523 MiB
11.58.853.195 I srv        update:    - prompt 0x63c2f1e7c4d0:    2845 tokens, checkpoints:  0,   146.937 MiB
11.58.853.195 I srv        update:    - prompt 0x63c2eaaf4520:    1608 tokens, checkpoints:  0,    83.049 MiB
11.58.853.197 I srv        update:    - prompt 0x63c2f1e62ea0:     771 tokens, checkpoints:  0,    39.821 MiB
11.58.853.198 I srv        update:    - prompt 0x63c2eaa9f9b0:    1236 tokens, checkpoints:  0,    63.836 MiB
11.58.853.199 I srv  get_availabl: prompt cache update took 51.75 ms
11.58.853.238 I slot launch_slot_: id  3 | task 31084 | processing task, is_child = 0
11.58.853.239 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
11.58.853.240 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
11.58.853.250 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
11.58.853.250 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
11.58.853.259 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
11.58.853.259 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
12.02.110.072 I slot print_timing: id  3 | task 31084 | n_decoded =    137, tg =  45.42 t/s, tg_3s =  45.42 t/s
12.05.131.116 I slot print_timing: id  3 | task 31084 | n_decoded =    273, tg =  45.22 t/s, tg_3s =  45.02 t/s
12.08.145.404 I slot print_timing: id  3 | task 31084 | n_decoded =    409, tg =  45.18 t/s, tg_3s =  45.12 t/s
12.11.150.566 I slot print_timing: id  3 | task 31084 | n_decoded =    543, tg =  45.04 t/s, tg_3s =  44.59 t/s
12.14.157.381 I slot print_timing: id  3 | task 31084 | n_decoded =    677, tg =  44.94 t/s, tg_3s =  44.57 t/s
12.17.173.934 I slot print_timing: id  3 | task 31084 | n_decoded =    811, tg =  44.86 t/s, tg_3s =  44.42 t/s
12.20.179.147 I slot print_timing: id  3 | task 31084 | n_decoded =    944, tg =  44.77 t/s, tg_3s =  44.26 t/s
12.23.181.754 I slot print_timing: id  3 | task 31084 | n_decoded =   1077, tg =  44.71 t/s, tg_3s =  44.29 t/s
12.25.119.972 I slot print_timing: id  3 | task 31084 | prompt eval time =     240.22 ms /    26 tokens (    9.24 ms per token,   108.24 tokens per second)
12.25.119.976 I slot print_timing: id  3 | task 31084 |        eval time =   26026.48 ms /  1164 tokens (   22.36 ms per token,    44.72 tokens per second)
12.25.119.976 I slot print_timing: id  3 | task 31084 |       total time =   26266.69 ms /  1190 tokens
12.25.119.977 I slot print_timing: id  3 | task 31084 |    graphs reused =      32063
12.25.120.029 I slot      release: id  3 | task 31084 | stop processing: n_tokens = 1259, truncated = 0
12.25.120.041 I srv  update_slots: all slots are idle
12.25.131.527 I srv    operator(): Chat format: peg-native
12.25.131.765 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.693 (> 0.100 thold), f_keep = 0.056
12.25.131.766 I srv  get_availabl: updating prompt cache
12.25.131.823 W srv   prompt_save:  - saving prompt with length 1259, total state size = 65.024 MiB (draft: 0.000 MiB)
12.25.185.900 I srv          load:  - looking for better prompt, base f_keep = 0.056, sim = 0.693
12.25.185.913 I srv        update:  - cache state: 21 prompts, 1772.027 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
12.25.185.915 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
12.25.185.915 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
12.25.185.916 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
12.25.185.916 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
12.25.185.917 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
12.25.185.917 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
12.25.185.918 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
12.25.185.921 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
12.25.185.921 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
12.25.185.922 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
12.25.185.922 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
12.25.185.923 I srv        update:    - prompt 0x63c2ea969620:    1142 tokens, checkpoints:  0,    58.982 MiB
12.25.185.923 I srv        update:    - prompt 0x63c2eaf04ba0:    1320 tokens, checkpoints:  0,    68.175 MiB
12.25.185.924 I srv        update:    - prompt 0x63c2f1f09fb0:    3376 tokens, checkpoints:  0,   174.361 MiB
12.25.185.925 I srv        update:    - prompt 0x63c2ea962870:    1636 tokens, checkpoints:  0,    84.495 MiB
12.25.185.926 I srv        update:    - prompt 0x63c2f1efda40:    2624 tokens, checkpoints:  0,   135.523 MiB
12.25.185.926 I srv        update:    - prompt 0x63c2f1e7c4d0:    2845 tokens, checkpoints:  0,   146.937 MiB
12.25.185.927 I srv        update:    - prompt 0x63c2eaaf4520:    1608 tokens, checkpoints:  0,    83.049 MiB
12.25.185.928 I srv        update:    - prompt 0x63c2f1e62ea0:     771 tokens, checkpoints:  0,    39.821 MiB
12.25.185.928 I srv        update:    - prompt 0x63c2eaa9f9b0:    1236 tokens, checkpoints:  0,    63.836 MiB
12.25.185.928 I srv        update:    - prompt 0x63c2f1dc6ec0:    1259 tokens, checkpoints:  0,    65.024 MiB
12.25.185.930 I srv  get_availabl: prompt cache update took 54.16 ms
12.25.185.975 I slot launch_slot_: id  3 | task 32249 | processing task, is_child = 0
12.25.185.976 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
12.25.185.976 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
12.25.185.990 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
12.25.185.990 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
12.25.185.999 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
12.25.185.999 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
12.28.441.788 I slot print_timing: id  3 | task 32249 | n_decoded =    137, tg =  45.58 t/s, tg_3s =  45.58 t/s
12.31.451.825 I slot print_timing: id  3 | task 32249 | n_decoded =    273, tg =  45.38 t/s, tg_3s =  45.18 t/s
12.34.467.842 I slot print_timing: id  3 | task 32249 | n_decoded =    410, tg =  45.39 t/s, tg_3s =  45.42 t/s
12.37.480.026 I slot print_timing: id  3 | task 32249 | n_decoded =    545, tg =  45.25 t/s, tg_3s =  44.82 t/s
12.40.495.761 I slot print_timing: id  3 | task 32249 | n_decoded =    680, tg =  45.15 t/s, tg_3s =  44.77 t/s
12.43.515.314 I slot print_timing: id  3 | task 32249 | n_decoded =    815, tg =  45.08 t/s, tg_3s =  44.71 t/s
12.46.232.327 I slot print_timing: id  3 | task 32249 | prompt eval time =     249.88 ms /    31 tokens (    8.06 ms per token,   124.06 tokens per second)
12.46.232.331 I slot print_timing: id  3 | task 32249 |        eval time =   20796.42 ms /   936 tokens (   22.22 ms per token,    45.01 tokens per second)
12.46.232.332 I slot print_timing: id  3 | task 32249 |       total time =   21046.30 ms /   967 tokens
12.46.232.332 I slot print_timing: id  3 | task 32249 |    graphs reused =      32993
12.46.232.371 I slot      release: id  3 | task 32249 | stop processing: n_tokens = 1036, truncated = 0
12.46.232.383 I srv  update_slots: all slots are idle
12.46.243.736 I srv    operator(): Chat format: peg-native
12.46.243.973 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.636 (> 0.100 thold), f_keep = 0.068
12.46.243.976 I srv  get_availabl: updating prompt cache
12.46.244.053 W srv   prompt_save:  - saving prompt with length 1036, total state size = 53.507 MiB (draft: 0.000 MiB)
12.46.292.886 I srv          load:  - looking for better prompt, base f_keep = 0.068, sim = 0.636
12.46.292.899 I srv        update:  - cache state: 22 prompts, 1825.534 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
12.46.292.900 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
12.46.292.901 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
12.46.292.902 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
12.46.292.902 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
12.46.292.903 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
12.46.292.903 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
12.46.292.904 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
12.46.292.904 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
12.46.292.905 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
12.46.292.905 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
12.46.292.906 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
12.46.292.906 I srv        update:    - prompt 0x63c2ea969620:    1142 tokens, checkpoints:  0,    58.982 MiB
12.46.292.907 I srv        update:    - prompt 0x63c2eaf04ba0:    1320 tokens, checkpoints:  0,    68.175 MiB
12.46.292.907 I srv        update:    - prompt 0x63c2f1f09fb0:    3376 tokens, checkpoints:  0,   174.361 MiB
12.46.292.908 I srv        update:    - prompt 0x63c2ea962870:    1636 tokens, checkpoints:  0,    84.495 MiB
12.46.292.908 I srv        update:    - prompt 0x63c2f1efda40:    2624 tokens, checkpoints:  0,   135.523 MiB
12.46.292.909 I srv        update:    - prompt 0x63c2f1e7c4d0:    2845 tokens, checkpoints:  0,   146.937 MiB
12.46.292.909 I srv        update:    - prompt 0x63c2eaaf4520:    1608 tokens, checkpoints:  0,    83.049 MiB
12.46.292.910 I srv        update:    - prompt 0x63c2f1e62ea0:     771 tokens, checkpoints:  0,    39.821 MiB
12.46.292.910 I srv        update:    - prompt 0x63c2eaa9f9b0:    1236 tokens, checkpoints:  0,    63.836 MiB
12.46.292.911 I srv        update:    - prompt 0x63c2f1dc6ec0:    1259 tokens, checkpoints:  0,    65.024 MiB
12.46.292.911 I srv        update:    - prompt 0x63c2f1f48c20:    1036 tokens, checkpoints:  0,    53.507 MiB
12.46.292.912 I srv  get_availabl: prompt cache update took 48.94 ms
12.46.292.953 I slot launch_slot_: id  3 | task 33186 | processing task, is_child = 0
12.46.292.953 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
12.46.292.954 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
12.46.292.964 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
12.46.292.964 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
12.46.292.973 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
12.46.292.973 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
12.49.516.969 I slot print_timing: id  3 | task 33186 | n_decoded =    137, tg =  45.62 t/s, tg_3s =  45.62 t/s
12.52.518.120 I slot print_timing: id  3 | task 33186 | n_decoded =    272, tg =  45.30 t/s, tg_3s =  44.98 t/s
12.55.533.993 I slot print_timing: id  3 | task 33186 | n_decoded =    408, tg =  45.23 t/s, tg_3s =  45.09 t/s
12.58.554.101 I slot print_timing: id  3 | task 33186 | n_decoded =    543, tg =  45.10 t/s, tg_3s =  44.70 t/s
13.01.572.332 I slot print_timing: id  3 | task 33186 | n_decoded =    677, tg =  44.96 t/s, tg_3s =  44.40 t/s
13.04.594.484 I slot print_timing: id  3 | task 33186 | n_decoded =    812, tg =  44.91 t/s, tg_3s =  44.67 t/s
13.06.182.922 I slot print_timing: id  3 | task 33186 | prompt eval time =     220.69 ms /    40 tokens (    5.52 ms per token,   181.25 tokens per second)
13.06.182.927 I slot print_timing: id  3 | task 33186 |        eval time =   19669.24 ms /   883 tokens (   22.28 ms per token,    44.89 tokens per second)
13.06.182.928 I slot print_timing: id  3 | task 33186 |       total time =   19889.93 ms /   923 tokens
13.06.182.929 I slot print_timing: id  3 | task 33186 |    graphs reused =      33871
13.06.182.982 I slot      release: id  3 | task 33186 | stop processing: n_tokens = 992, truncated = 0
13.06.182.994 I srv  update_slots: all slots are idle
13.06.193.968 I srv    operator(): Chat format: peg-native
13.06.194.217 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.737 (> 0.100 thold), f_keep = 0.071
13.06.194.219 I srv  get_availabl: updating prompt cache
13.06.194.290 W srv   prompt_save:  - saving prompt with length 992, total state size = 51.235 MiB (draft: 0.000 MiB)
13.06.236.830 I srv          load:  - looking for better prompt, base f_keep = 0.071, sim = 0.737
13.06.236.845 I srv        update:  - cache state: 23 prompts, 1876.768 MiB (limits: 8192.000 MiB, 16384 tokens, 158613 est)
13.06.236.850 I srv        update:    - prompt 0x63c2eae7a0e0:    1855 tokens, checkpoints:  0,    95.806 MiB
13.06.236.851 I srv        update:    - prompt 0x63c2f1995a20:    1238 tokens, checkpoints:  0,    63.940 MiB
13.06.236.852 I srv        update:    - prompt 0x63c2ea8de9c0:     758 tokens, checkpoints:  0,    39.149 MiB
13.06.236.853 I srv        update:    - prompt 0x63c2f1951910:    1441 tokens, checkpoints:  0,    74.424 MiB
13.06.236.857 I srv        update:    - prompt 0x63c2f1db0040:    1980 tokens, checkpoints:  0,   102.262 MiB
13.06.236.857 I srv        update:    - prompt 0x63c2f1dbd710:    1249 tokens, checkpoints:  0,    64.508 MiB
13.06.236.858 I srv        update:    - prompt 0x63c2f1df9610:    1236 tokens, checkpoints:  0,    63.836 MiB
13.06.236.859 I srv        update:    - prompt 0x63c2eae7e5d0:    1394 tokens, checkpoints:  0,    71.997 MiB
13.06.236.859 I srv        update:    - prompt 0x63c2ea8d81a0:    1526 tokens, checkpoints:  0,    78.814 MiB
13.06.236.861 I srv        update:    - prompt 0x63c2f1dbb070:    1884 tokens, checkpoints:  0,    97.304 MiB
13.06.236.862 I srv        update:    - prompt 0x63c2f1e6a330:    1932 tokens, checkpoints:  0,    99.783 MiB
13.06.236.862 I srv        update:    - prompt 0x63c2ea969620:    1142 tokens, checkpoints:  0,    58.982 MiB
13.06.236.863 I srv        update:    - prompt 0x63c2eaf04ba0:    1320 tokens, checkpoints:  0,    68.175 MiB
13.06.236.864 I srv        update:    - prompt 0x63c2f1f09fb0:    3376 tokens, checkpoints:  0,   174.361 MiB
13.06.236.864 I srv        update:    - prompt 0x63c2ea962870:    1636 tokens, checkpoints:  0,    84.495 MiB
13.06.236.865 I srv        update:    - prompt 0x63c2f1efda40:    2624 tokens, checkpoints:  0,   135.523 MiB
13.06.236.865 I srv        update:    - prompt 0x63c2f1e7c4d0:    2845 tokens, checkpoints:  0,   146.937 MiB
13.06.236.866 I srv        update:    - prompt 0x63c2eaaf4520:    1608 tokens, checkpoints:  0,    83.049 MiB
13.06.236.866 I srv        update:    - prompt 0x63c2f1e62ea0:     771 tokens, checkpoints:  0,    39.821 MiB
13.06.236.867 I srv        update:    - prompt 0x63c2eaa9f9b0:    1236 tokens, checkpoints:  0,    63.836 MiB
13.06.236.868 I srv        update:    - prompt 0x63c2f1dc6ec0:    1259 tokens, checkpoints:  0,    65.024 MiB
13.06.236.868 I srv        update:    - prompt 0x63c2f1f48c20:    1036 tokens, checkpoints:  0,    53.507 MiB
13.06.236.869 I srv        update:    - prompt 0x63c2eaabfd80:     992 tokens, checkpoints:  0,    51.235 MiB
13.06.236.870 I srv  get_availabl: prompt cache update took 42.65 ms
13.06.236.919 I slot launch_slot_: id  3 | task 34070 | processing task, is_child = 0
13.06.236.920 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
13.06.236.921 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
13.06.236.932 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
13.06.236.933 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
13.06.236.956 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
13.06.236.957 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
13.09.490.194 I slot print_timing: id  3 | task 34070 | n_decoded =    137, tg =  45.33 t/s, tg_3s =  45.33 t/s
13.12.500.669 I slot print_timing: id  3 | task 34070 | n_decoded =    272, tg =  45.09 t/s, tg_3s =  44.84 t/s
13.15.501.680 I slot print_timing: id  3 | task 34070 | n_decoded =    407, tg =  45.05 t/s, tg_3s =  44.98 t/s
13.18.510.052 I slot print_timing: id  3 | task 34070 | n_decoded =    542, tg =  45.01 t/s, tg_3s =  44.87 t/s
13.21.530.237 I slot print_timing: id  3 | task 34070 | n_decoded =    678, tg =  45.01 t/s, tg_3s =  45.03 t/s
13.24.537.244 I slot print_timing: id  3 | task 34070 | n_decoded =    813, tg =  44.99 t/s, tg_3s =  44.90 t/s
13.27.559.286 I slot print_timing: id  3 | task 34070 | n_decoded =    948, tg =  44.95 t/s, tg_3s =  44.67 t/s
13.30.577.153 I slot print_timing: id  3 | task 34070 | n_decoded =   1083, tg =  44.92 t/s, tg_3s =  44.73 t/s
13.33.588.275 I slot print_timing: id  3 | task 34070 | n_decoded =   1217, tg =  44.87 t/s, tg_3s =  44.50 t/s
13.34.914.083 I slot print_timing: id  3 | task 34070 | prompt eval time =     231.20 ms /    25 tokens (    9.25 ms per token,   108.13 tokens per second)
13.34.914.087 I slot print_timing: id  3 | task 34070 |        eval time =   28445.89 ms /  1276 tokens (   22.29 ms per token,    44.86 tokens per second)
13.34.914.088 I slot print_timing: id  3 | task 34070 |       total time =   28677.09 ms /  1301 tokens
13.34.914.089 I slot print_timing: id  3 | task 34070 |    graphs reused =      35140
13.34.914.133 I slot      release: id  3 | task 34070 | stop processing: n_tokens = 1370, truncated = 0
13.34.914.148 I srv  update_slots: all slots are idle
13.34.914.741 I srv    operator(): operator(): cleaning up before exit...