qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results_hard/granite4-micro.server.log

787 lines
76 KiB
Plaintext
Raw Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.065.456 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.065.459 I device_info:
0.00.065.480 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.484 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.492 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.065.547 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.065.550 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.065.589 I srv init: running without SSL
0.00.065.657 I srv init: using 23 threads for HTTP server
0.00.065.982 I srv start: binding port with default address family
0.00.067.185 I srv llama_server: loading model
0.00.067.193 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/granite-4.0-micro-Q4_K_M.gguf'
0.00.067.245 I common_init_result: fitting params to device memory ...
0.00.067.249 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.02.095.087 W llama_context: n_ctx_seq (16384) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
0.02.129.849 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.02.255.990 I srv load_model: initializing slots, n_slots = 4
0.02.300.355 W common_speculative_init: no implementations specified for speculative decoding
0.02.300.361 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384
0.02.300.368 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384
0.02.300.368 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384
0.02.300.369 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384
0.02.300.452 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.02.300.455 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.02.300.455 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.02.300.456 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.02.300.477 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.02.308.825 I init: chat template, example_format: '<|start_of_role|>system<|end_of_role|>You are a helpful assistant<|end_of_text|>
<|start_of_role|>user<|end_of_role|>Hello<|end_of_text|>
<|start_of_role|>assistant<|end_of_role|>Hi there<|end_of_text|>
<|start_of_role|>user<|end_of_role|>How are you?<|end_of_text|>
<|start_of_role|>assistant<|end_of_role|>'
0.02.313.793 I srv init: init: chat template, thinking = 0
0.02.313.830 I srv llama_server: model loaded
0.02.313.834 I srv llama_server: server is listening on http://127.0.0.1:8090
0.02.313.839 I srv update_slots: all slots are idle
0.04.968.929 I srv operator(): Chat format: peg-native
0.04.969.180 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.04.969.182 I srv get_availabl: updating prompt cache
0.04.969.189 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.04.969.194 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est)
0.04.969.194 I srv get_availabl: prompt cache update took 0.01 ms
0.04.969.264 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.04.969.265 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.04.969.266 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.04.969.280 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.04.969.280 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.04.969.289 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.04.969.289 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.08.151.151 I slot print_timing: id 3 | task 0 | n_decoded = 229, tg = 76.22 t/s, tg_3s = 76.22 t/s
0.08.818.006 I slot print_timing: id 3 | task 0 | prompt eval time = 177.33 ms / 98 tokens ( 1.81 ms per token, 552.63 tokens per second)
0.08.818.009 I slot print_timing: id 3 | task 0 | eval time = 3671.35 ms / 279 tokens ( 13.16 ms per token, 75.99 tokens per second)
0.08.818.010 I slot print_timing: id 3 | task 0 | total time = 3848.69 ms / 377 tokens
0.08.818.014 I slot print_timing: id 3 | task 0 | graphs reused = 277
0.08.818.061 I slot release: id 3 | task 0 | stop processing: n_tokens = 376, truncated = 0
0.08.818.066 I srv update_slots: all slots are idle
0.08.825.289 I srv operator(): Chat format: peg-native
0.08.825.490 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.804 (> 0.100 thold), f_keep = 0.197
0.08.825.492 I srv get_availabl: updating prompt cache
0.08.825.530 W srv prompt_save: - saving prompt with length 376, total state size = 29.380 MiB (draft: 0.000 MiB)
0.08.850.778 I srv load: - looking for better prompt, base f_keep = 0.197, sim = 0.804
0.08.850.785 I srv update: - cache state: 1 prompts, 29.380 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.08.850.786 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.08.850.788 I srv get_availabl: prompt cache update took 25.30 ms
0.08.850.894 I slot launch_slot_: id 3 | task 281 | processing task, is_child = 0
0.08.850.897 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.08.850.898 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.08.850.912 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.08.850.913 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.08.850.923 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.08.850.923 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.10.462.896 I slot print_timing: id 3 | task 281 | prompt eval time = 69.38 ms / 18 tokens ( 3.85 ms per token, 259.44 tokens per second)
0.10.462.899 I slot print_timing: id 3 | task 281 | eval time = 1542.57 ms / 119 tokens ( 12.96 ms per token, 77.14 tokens per second)
0.10.462.900 I slot print_timing: id 3 | task 281 | total time = 1611.95 ms / 137 tokens
0.10.462.901 I slot print_timing: id 3 | task 281 | graphs reused = 394
0.10.462.931 I slot release: id 3 | task 281 | stop processing: n_tokens = 210, truncated = 0
0.10.462.936 I srv update_slots: all slots are idle
0.10.469.473 I srv operator(): Chat format: peg-native
0.10.469.682 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.796 (> 0.100 thold), f_keep = 0.352
0.10.469.684 I srv get_availabl: updating prompt cache
0.10.469.709 W srv prompt_save: - saving prompt with length 210, total state size = 16.410 MiB (draft: 0.000 MiB)
0.10.484.336 I srv load: - looking for better prompt, base f_keep = 0.352, sim = 0.796
0.10.484.344 I srv update: - cache state: 2 prompts, 45.790 MiB (limits: 8192.000 MiB, 16384 tokens, 104837 est)
0.10.484.345 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.10.484.345 I srv update: - prompt 0x638b5baa7990: 210 tokens, checkpoints: 0, 16.410 MiB
0.10.484.346 I srv get_availabl: prompt cache update took 14.66 ms
0.10.484.417 I slot launch_slot_: id 3 | task 401 | processing task, is_child = 0
0.10.484.418 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.10.484.418 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.10.484.430 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.10.484.431 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.10.484.439 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.10.484.440 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.13.524.171 I slot print_timing: id 3 | task 401 | prompt eval time = 69.58 ms / 19 tokens ( 3.66 ms per token, 273.06 tokens per second)
0.13.524.175 I slot print_timing: id 3 | task 401 | eval time = 2970.13 ms / 227 tokens ( 13.08 ms per token, 76.43 tokens per second)
0.13.524.176 I slot print_timing: id 3 | task 401 | total time = 3039.71 ms / 246 tokens
0.13.524.177 I slot print_timing: id 3 | task 401 | graphs reused = 618
0.13.524.205 I slot release: id 3 | task 401 | stop processing: n_tokens = 319, truncated = 0
0.13.524.211 I srv update_slots: all slots are idle
0.13.531.007 I srv operator(): Chat format: peg-native
0.13.531.212 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.813 (> 0.100 thold), f_keep = 0.232
0.13.531.214 I srv get_availabl: updating prompt cache
0.13.531.248 W srv prompt_save: - saving prompt with length 319, total state size = 24.926 MiB (draft: 0.000 MiB)
0.13.552.336 I srv load: - looking for better prompt, base f_keep = 0.232, sim = 0.813
0.13.552.344 I srv update: - cache state: 3 prompts, 70.716 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.13.552.345 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.13.552.346 I srv update: - prompt 0x638b5baa7990: 210 tokens, checkpoints: 0, 16.410 MiB
0.13.552.346 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.13.552.347 I srv get_availabl: prompt cache update took 21.13 ms
0.13.552.435 I slot launch_slot_: id 3 | task 629 | processing task, is_child = 0
0.13.552.436 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.13.552.441 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.13.552.455 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.13.552.456 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.13.552.464 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.13.552.464 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.16.627.747 I slot print_timing: id 3 | task 629 | n_decoded = 230, tg = 76.52 t/s, tg_3s = 76.52 t/s
0.17.726.773 I slot print_timing: id 3 | task 629 | prompt eval time = 69.42 ms / 17 tokens ( 4.08 ms per token, 244.89 tokens per second)
0.17.726.777 I slot print_timing: id 3 | task 629 | eval time = 4104.87 ms / 313 tokens ( 13.11 ms per token, 76.25 tokens per second)
0.17.726.778 I slot print_timing: id 3 | task 629 | total time = 4174.29 ms / 330 tokens
0.17.726.779 I slot print_timing: id 3 | task 629 | graphs reused = 928
0.17.726.809 I slot release: id 3 | task 629 | stop processing: n_tokens = 403, truncated = 0
0.17.726.817 I srv update_slots: all slots are idle
0.17.733.654 I srv operator(): Chat format: peg-native
0.17.733.874 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.712 (> 0.100 thold), f_keep = 0.184
0.17.733.876 I srv get_availabl: updating prompt cache
0.17.733.907 W srv prompt_save: - saving prompt with length 403, total state size = 31.490 MiB (draft: 0.000 MiB)
0.17.760.678 I srv load: - looking for better prompt, base f_keep = 0.184, sim = 0.712
0.17.760.683 I srv load: - found better prompt with f_keep = 0.357, sim = 0.721
0.17.769.704 I srv update: - cache state: 3 prompts, 85.797 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.17.769.708 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.17.769.708 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.17.769.709 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.17.769.710 I srv get_availabl: prompt cache update took 35.83 ms
0.17.769.782 I slot launch_slot_: id 3 | task 943 | processing task, is_child = 0
0.17.769.783 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.17.769.784 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.17.769.800 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.17.769.801 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.17.769.812 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.17.769.813 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.20.862.167 I slot print_timing: id 3 | task 943 | n_decoded = 230, tg = 76.38 t/s, tg_3s = 76.38 t/s
0.22.447.768 I slot print_timing: id 3 | task 943 | prompt eval time = 80.99 ms / 29 tokens ( 2.79 ms per token, 358.05 tokens per second)
0.22.447.771 I slot print_timing: id 3 | task 943 | eval time = 4596.94 ms / 350 tokens ( 13.13 ms per token, 76.14 tokens per second)
0.22.447.772 I slot print_timing: id 3 | task 943 | total time = 4677.93 ms / 379 tokens
0.22.447.773 I slot print_timing: id 3 | task 943 | graphs reused = 1275
0.22.447.804 I slot release: id 3 | task 943 | stop processing: n_tokens = 453, truncated = 0
0.22.447.810 I srv update_slots: all slots are idle
0.22.454.251 I srv operator(): Chat format: peg-native
0.22.454.464 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.787 (> 0.100 thold), f_keep = 0.163
0.22.454.465 I srv get_availabl: updating prompt cache
0.22.454.501 W srv prompt_save: - saving prompt with length 453, total state size = 35.397 MiB (draft: 0.000 MiB)
0.22.484.116 I srv load: - looking for better prompt, base f_keep = 0.163, sim = 0.787
0.22.484.124 I srv update: - cache state: 4 prompts, 121.193 MiB (limits: 8192.000 MiB, 16384 tokens, 104839 est)
0.22.484.125 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.22.484.126 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.22.484.126 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.22.484.127 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
0.22.484.128 I srv get_availabl: prompt cache update took 29.66 ms
0.22.484.219 I slot launch_slot_: id 3 | task 1294 | processing task, is_child = 0
0.22.484.221 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.22.484.221 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.22.484.235 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.22.484.235 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.22.484.243 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.22.484.244 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.25.555.855 I slot print_timing: id 3 | task 1294 | n_decoded = 231, tg = 76.94 t/s, tg_3s = 76.94 t/s
0.26.751.869 I slot print_timing: id 3 | task 1294 | prompt eval time = 69.28 ms / 20 tokens ( 3.46 ms per token, 288.68 tokens per second)
0.26.751.872 I slot print_timing: id 3 | task 1294 | eval time = 4198.32 ms / 323 tokens ( 13.00 ms per token, 76.94 tokens per second)
0.26.751.873 I slot print_timing: id 3 | task 1294 | total time = 4267.60 ms / 343 tokens
0.26.751.874 I slot print_timing: id 3 | task 1294 | graphs reused = 1595
0.26.751.908 I slot release: id 3 | task 1294 | stop processing: n_tokens = 416, truncated = 0
0.26.751.917 I srv update_slots: all slots are idle
0.26.758.485 I srv operator(): Chat format: peg-native
0.26.758.696 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.843 (> 0.100 thold), f_keep = 0.180
0.26.758.698 I srv get_availabl: updating prompt cache
0.26.758.742 W srv prompt_save: - saving prompt with length 416, total state size = 32.506 MiB (draft: 0.000 MiB)
0.26.785.613 I srv load: - looking for better prompt, base f_keep = 0.180, sim = 0.843
0.26.785.622 I srv update: - cache state: 5 prompts, 153.699 MiB (limits: 8192.000 MiB, 16384 tokens, 104839 est)
0.26.785.623 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.26.785.624 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.26.785.625 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.26.785.626 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
0.26.785.630 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
0.26.785.631 I srv get_availabl: prompt cache update took 26.93 ms
0.26.785.720 I slot launch_slot_: id 3 | task 1618 | processing task, is_child = 0
0.26.785.721 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.26.785.722 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.26.785.737 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.26.785.737 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.26.785.747 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.26.785.748 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.28.835.461 I slot print_timing: id 3 | task 1618 | prompt eval time = 42.06 ms / 14 tokens ( 3.00 ms per token, 332.87 tokens per second)
0.28.835.465 I slot print_timing: id 3 | task 1618 | eval time = 2007.63 ms / 155 tokens ( 12.95 ms per token, 77.21 tokens per second)
0.28.835.466 I slot print_timing: id 3 | task 1618 | total time = 2049.69 ms / 169 tokens
0.28.835.467 I slot print_timing: id 3 | task 1618 | graphs reused = 1748
0.28.835.501 I slot release: id 3 | task 1618 | stop processing: n_tokens = 243, truncated = 0
0.28.835.510 I srv update_slots: all slots are idle
0.28.842.037 I srv operator(): Chat format: peg-native
0.28.842.242 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.824 (> 0.100 thold), f_keep = 0.309
0.28.842.244 I srv get_availabl: updating prompt cache
0.28.842.279 W srv prompt_save: - saving prompt with length 243, total state size = 18.988 MiB (draft: 0.000 MiB)
0.28.859.214 I srv load: - looking for better prompt, base f_keep = 0.309, sim = 0.824
0.28.859.223 I srv update: - cache state: 6 prompts, 172.687 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.28.859.224 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.28.859.225 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.28.859.225 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.28.859.227 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
0.28.859.228 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
0.28.859.229 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
0.28.859.230 I srv get_availabl: prompt cache update took 16.99 ms
0.28.859.314 I slot launch_slot_: id 3 | task 1774 | processing task, is_child = 0
0.28.859.315 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.28.859.315 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.28.859.328 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.28.859.329 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.28.859.339 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.28.859.339 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.31.905.888 I slot print_timing: id 3 | task 1774 | n_decoded = 231, tg = 76.92 t/s, tg_3s = 76.92 t/s
0.32.841.855 I slot print_timing: id 3 | task 1774 | prompt eval time = 43.39 ms / 16 tokens ( 2.71 ms per token, 368.77 tokens per second)
0.32.841.858 I slot print_timing: id 3 | task 1774 | eval time = 3939.11 ms / 303 tokens ( 13.00 ms per token, 76.92 tokens per second)
0.32.841.858 I slot print_timing: id 3 | task 1774 | total time = 3982.49 ms / 319 tokens
0.32.841.859 I slot print_timing: id 3 | task 1774 | graphs reused = 2048
0.32.841.888 I slot release: id 3 | task 1774 | stop processing: n_tokens = 393, truncated = 0
0.32.841.895 I srv update_slots: all slots are idle
0.32.848.585 I srv operator(): Chat format: peg-native
0.32.848.810 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.627 (> 0.100 thold), f_keep = 0.188
0.32.848.811 I srv get_availabl: updating prompt cache
0.32.848.849 W srv prompt_save: - saving prompt with length 393, total state size = 30.709 MiB (draft: 0.000 MiB)
0.32.874.079 I srv load: - looking for better prompt, base f_keep = 0.188, sim = 0.627
0.32.874.089 I srv update: - cache state: 7 prompts, 203.396 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.32.874.090 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.32.874.090 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.32.874.091 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.32.874.091 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
0.32.874.092 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
0.32.874.092 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
0.32.874.093 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
0.32.874.094 I srv get_availabl: prompt cache update took 25.28 ms
0.32.874.193 I slot launch_slot_: id 3 | task 2078 | processing task, is_child = 0
0.32.874.195 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.32.874.196 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.32.874.212 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.32.874.212 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.32.874.224 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.32.874.225 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.35.797.883 I slot print_timing: id 3 | task 2078 | prompt eval time = 80.20 ms / 44 tokens ( 1.82 ms per token, 548.66 tokens per second)
0.35.797.887 I slot print_timing: id 3 | task 2078 | eval time = 2843.44 ms / 219 tokens ( 12.98 ms per token, 77.02 tokens per second)
0.35.797.887 I slot print_timing: id 3 | task 2078 | total time = 2923.63 ms / 263 tokens
0.35.797.888 I slot print_timing: id 3 | task 2078 | graphs reused = 2264
0.35.797.919 I slot release: id 3 | task 2078 | stop processing: n_tokens = 336, truncated = 0
0.35.797.926 I srv update_slots: all slots are idle
0.35.804.627 I srv operator(): Chat format: peg-native
0.35.804.839 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.560 (> 0.100 thold), f_keep = 0.223
0.35.804.841 I srv get_availabl: updating prompt cache
0.35.804.870 W srv prompt_save: - saving prompt with length 336, total state size = 26.255 MiB (draft: 0.000 MiB)
0.35.826.576 I srv load: - looking for better prompt, base f_keep = 0.223, sim = 0.560
0.35.826.586 I srv update: - cache state: 8 prompts, 229.651 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.35.826.587 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.35.826.587 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.35.826.588 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.35.826.591 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
0.35.826.597 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
0.35.826.597 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
0.35.826.598 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
0.35.826.598 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
0.35.826.600 I srv get_availabl: prompt cache update took 21.76 ms
0.35.826.686 I slot launch_slot_: id 3 | task 2298 | processing task, is_child = 0
0.35.826.687 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.35.826.688 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.35.826.701 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.35.826.701 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.35.826.710 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.35.826.710 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.38.928.946 I slot print_timing: id 3 | task 2298 | n_decoded = 231, tg = 76.91 t/s, tg_3s = 76.91 t/s
0.39.994.976 I slot print_timing: id 3 | task 2298 | prompt eval time = 98.89 ms / 59 tokens ( 1.68 ms per token, 596.64 tokens per second)
0.39.994.979 I slot print_timing: id 3 | task 2298 | eval time = 4069.36 ms / 313 tokens ( 13.00 ms per token, 76.92 tokens per second)
0.39.994.980 I slot print_timing: id 3 | task 2298 | total time = 4168.25 ms / 372 tokens
0.39.994.981 I slot print_timing: id 3 | task 2298 | graphs reused = 2574
0.39.995.014 I slot release: id 3 | task 2298 | stop processing: n_tokens = 446, truncated = 0
0.39.995.021 I srv update_slots: all slots are idle
0.40.001.502 I srv operator(): Chat format: peg-native
0.40.001.713 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.661 (> 0.100 thold), f_keep = 0.166
0.40.001.715 I srv get_availabl: updating prompt cache
0.40.001.758 W srv prompt_save: - saving prompt with length 446, total state size = 34.850 MiB (draft: 0.000 MiB)
0.40.030.359 I srv load: - looking for better prompt, base f_keep = 0.166, sim = 0.661
0.40.030.368 I srv update: - cache state: 9 prompts, 264.500 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.40.030.369 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.40.030.369 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.40.030.370 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.40.030.370 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
0.40.030.371 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
0.40.030.372 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
0.40.030.372 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
0.40.030.373 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
0.40.030.373 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
0.40.030.374 I srv get_availabl: prompt cache update took 28.66 ms
0.40.030.464 I slot launch_slot_: id 3 | task 2612 | processing task, is_child = 0
0.40.030.465 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.40.030.466 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.40.030.479 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.40.030.479 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.40.030.488 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.40.030.488 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.43.122.974 I slot print_timing: id 3 | task 2612 | n_decoded = 231, tg = 76.98 t/s, tg_3s = 76.98 t/s
0.43.655.960 I slot print_timing: id 3 | task 2612 | prompt eval time = 91.84 ms / 38 tokens ( 2.42 ms per token, 413.75 tokens per second)
0.43.655.963 I slot print_timing: id 3 | task 2612 | eval time = 3533.61 ms / 272 tokens ( 12.99 ms per token, 76.98 tokens per second)
0.43.655.964 I slot print_timing: id 3 | task 2612 | total time = 3625.45 ms / 310 tokens
0.43.655.964 I slot print_timing: id 3 | task 2612 | graphs reused = 2843
0.43.655.995 I slot release: id 3 | task 2612 | stop processing: n_tokens = 383, truncated = 0
0.43.656.004 I srv update_slots: all slots are idle
0.43.662.403 I srv operator(): Chat format: peg-native
0.43.662.627 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.583 (> 0.100 thold), f_keep = 0.193
0.43.662.629 I srv get_availabl: updating prompt cache
0.43.662.656 W srv prompt_save: - saving prompt with length 383, total state size = 29.927 MiB (draft: 0.000 MiB)
0.43.687.331 I srv load: - looking for better prompt, base f_keep = 0.193, sim = 0.583
0.43.687.341 I srv update: - cache state: 10 prompts, 294.428 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.43.687.342 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.43.687.343 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.43.687.343 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.43.687.345 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
0.43.687.348 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
0.43.687.349 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
0.43.687.349 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
0.43.687.350 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
0.43.687.350 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
0.43.687.350 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
0.43.687.352 I srv get_availabl: prompt cache update took 24.72 ms
0.43.687.441 I slot launch_slot_: id 3 | task 2885 | processing task, is_child = 0
0.43.687.441 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.43.687.442 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.43.687.455 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.43.687.455 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.43.687.463 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.43.687.464 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.46.776.980 I slot print_timing: id 3 | task 2885 | n_decoded = 231, tg = 77.00 t/s, tg_3s = 77.00 t/s
0.47.426.983 I slot print_timing: id 3 | task 2885 | prompt eval time = 89.44 ms / 53 tokens ( 1.69 ms per token, 592.59 tokens per second)
0.47.426.985 I slot print_timing: id 3 | task 2885 | eval time = 3650.06 ms / 281 tokens ( 12.99 ms per token, 76.98 tokens per second)
0.47.426.986 I slot print_timing: id 3 | task 2885 | total time = 3739.50 ms / 334 tokens
0.47.426.987 I slot print_timing: id 3 | task 2885 | graphs reused = 3121
0.47.427.020 I slot release: id 3 | task 2885 | stop processing: n_tokens = 407, truncated = 0
0.47.427.029 I srv update_slots: all slots are idle
0.47.433.637 I srv operator(): Chat format: peg-native
0.47.433.844 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.574 (> 0.100 thold), f_keep = 0.182
0.47.433.846 I srv get_availabl: updating prompt cache
0.47.433.874 W srv prompt_save: - saving prompt with length 407, total state size = 31.802 MiB (draft: 0.000 MiB)
0.47.459.840 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 0.574
0.47.459.852 I srv update: - cache state: 11 prompts, 326.230 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.47.459.853 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.47.459.853 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.47.459.854 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.47.459.854 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
0.47.459.855 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
0.47.459.855 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
0.47.459.856 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
0.47.459.856 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
0.47.459.857 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
0.47.459.858 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
0.47.459.862 I srv update: - prompt 0x638b5b8a99b0: 407 tokens, checkpoints: 0, 31.802 MiB
0.47.459.863 I srv get_availabl: prompt cache update took 26.02 ms
0.47.459.942 I slot launch_slot_: id 3 | task 3167 | processing task, is_child = 0
0.47.459.943 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.47.459.944 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.47.459.956 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.47.459.957 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.47.459.965 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.47.459.965 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.50.042.027 I slot print_timing: id 3 | task 3167 | prompt eval time = 88.96 ms / 55 tokens ( 1.62 ms per token, 618.23 tokens per second)
0.50.042.030 I slot print_timing: id 3 | task 3167 | eval time = 2493.08 ms / 192 tokens ( 12.98 ms per token, 77.01 tokens per second)
0.50.042.031 I slot print_timing: id 3 | task 3167 | total time = 2582.04 ms / 247 tokens
0.50.042.032 I slot print_timing: id 3 | task 3167 | graphs reused = 3310
0.50.042.065 I slot release: id 3 | task 3167 | stop processing: n_tokens = 320, truncated = 0
0.50.042.071 I srv update_slots: all slots are idle
0.50.048.747 I srv operator(): Chat format: peg-native
0.50.048.963 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.647 (> 0.100 thold), f_keep = 0.234
0.50.048.966 I srv get_availabl: updating prompt cache
0.50.049.009 W srv prompt_save: - saving prompt with length 320, total state size = 25.005 MiB (draft: 0.000 MiB)
0.50.070.155 I srv load: - looking for better prompt, base f_keep = 0.234, sim = 0.647
0.50.070.167 I srv update: - cache state: 12 prompts, 351.235 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.50.070.168 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.50.070.168 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.50.070.169 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.50.070.169 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
0.50.070.170 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
0.50.070.171 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
0.50.070.176 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
0.50.070.177 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
0.50.070.177 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
0.50.070.178 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
0.50.070.178 I srv update: - prompt 0x638b5b8a99b0: 407 tokens, checkpoints: 0, 31.802 MiB
0.50.070.179 I srv update: - prompt 0x638b5ba41a40: 320 tokens, checkpoints: 0, 25.005 MiB
0.50.070.181 I srv get_availabl: prompt cache update took 21.21 ms
0.50.070.281 I slot launch_slot_: id 3 | task 3360 | processing task, is_child = 0
0.50.070.282 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.50.070.283 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.50.070.301 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.50.070.301 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.50.070.313 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.50.070.313 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.53.159.077 I slot print_timing: id 3 | task 3360 | n_decoded = 231, tg = 76.77 t/s, tg_3s = 76.77 t/s
0.53.419.080 I slot print_timing: id 3 | task 3360 | prompt eval time = 79.79 ms / 41 tokens ( 1.95 ms per token, 513.87 tokens per second)
0.53.419.084 I slot print_timing: id 3 | task 3360 | eval time = 3268.96 ms / 251 tokens ( 13.02 ms per token, 76.78 tokens per second)
0.53.419.085 I slot print_timing: id 3 | task 3360 | total time = 3348.74 ms / 292 tokens
0.53.419.085 I slot print_timing: id 3 | task 3360 | graphs reused = 3558
0.53.419.125 I slot release: id 3 | task 3360 | stop processing: n_tokens = 366, truncated = 0
0.53.419.133 I srv update_slots: all slots are idle
0.53.425.766 I srv operator(): Chat format: peg-native
0.53.425.971 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.607 (> 0.100 thold), f_keep = 0.202
0.53.425.974 I srv get_availabl: updating prompt cache
0.53.426.020 W srv prompt_save: - saving prompt with length 366, total state size = 28.599 MiB (draft: 0.000 MiB)
0.53.449.668 I srv load: - looking for better prompt, base f_keep = 0.202, sim = 0.607
0.53.449.678 I srv update: - cache state: 13 prompts, 379.833 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.53.449.679 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.53.449.680 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.53.449.681 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.53.449.682 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
0.53.449.684 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
0.53.449.685 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
0.53.449.685 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
0.53.449.686 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
0.53.449.687 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
0.53.449.687 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
0.53.449.688 I srv update: - prompt 0x638b5b8a99b0: 407 tokens, checkpoints: 0, 31.802 MiB
0.53.449.688 I srv update: - prompt 0x638b5ba41a40: 320 tokens, checkpoints: 0, 25.005 MiB
0.53.449.689 I srv update: - prompt 0x638b5a6cdca0: 366 tokens, checkpoints: 0, 28.599 MiB
0.53.449.690 I srv get_availabl: prompt cache update took 23.72 ms
0.53.449.772 I slot launch_slot_: id 3 | task 3612 | processing task, is_child = 0
0.53.449.773 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.53.449.774 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.53.449.789 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.53.449.794 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.53.449.804 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.53.449.804 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.56.318.073 I slot print_timing: id 3 | task 3612 | prompt eval time = 80.33 ms / 48 tokens ( 1.67 ms per token, 597.52 tokens per second)
0.56.318.077 I slot print_timing: id 3 | task 3612 | eval time = 2787.91 ms / 214 tokens ( 13.03 ms per token, 76.76 tokens per second)
0.56.318.077 I slot print_timing: id 3 | task 3612 | total time = 2868.25 ms / 262 tokens
0.56.318.078 I slot print_timing: id 3 | task 3612 | graphs reused = 3769
0.56.318.110 I slot release: id 3 | task 3612 | stop processing: n_tokens = 335, truncated = 0
0.56.318.117 I srv update_slots: all slots are idle
0.56.324.990 I srv operator(): Chat format: peg-native
0.56.325.195 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.221
0.56.325.197 I srv get_availabl: updating prompt cache
0.56.325.231 W srv prompt_save: - saving prompt with length 335, total state size = 26.177 MiB (draft: 0.000 MiB)
0.56.346.872 I srv load: - looking for better prompt, base f_keep = 0.221, sim = 0.667
0.56.346.884 I srv update: - cache state: 14 prompts, 406.010 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.56.346.885 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.56.346.885 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.56.346.886 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.56.346.886 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
0.56.346.887 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
0.56.346.887 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
0.56.346.888 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
0.56.346.889 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
0.56.346.890 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
0.56.346.891 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
0.56.346.891 I srv update: - prompt 0x638b5b8a99b0: 407 tokens, checkpoints: 0, 31.802 MiB
0.56.346.892 I srv update: - prompt 0x638b5ba41a40: 320 tokens, checkpoints: 0, 25.005 MiB
0.56.346.892 I srv update: - prompt 0x638b5a6cdca0: 366 tokens, checkpoints: 0, 28.599 MiB
0.56.346.893 I srv update: - prompt 0x638b5ba61000: 335 tokens, checkpoints: 0, 26.177 MiB
0.56.346.894 I srv get_availabl: prompt cache update took 21.70 ms
0.56.346.973 I slot launch_slot_: id 3 | task 3827 | processing task, is_child = 0
0.56.346.973 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.56.346.974 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.56.346.986 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.56.346.986 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.56.346.995 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.56.346.995 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.59.443.022 I slot print_timing: id 3 | task 3827 | n_decoded = 230, tg = 76.55 t/s, tg_3s = 76.55 t/s
0.59.602.021 I slot print_timing: id 3 | task 3827 | prompt eval time = 91.52 ms / 37 tokens ( 2.47 ms per token, 404.27 tokens per second)
0.59.602.024 I slot print_timing: id 3 | task 3827 | eval time = 3163.48 ms / 242 tokens ( 13.07 ms per token, 76.50 tokens per second)
0.59.602.025 I slot print_timing: id 3 | task 3827 | total time = 3255.01 ms / 279 tokens
0.59.602.025 I slot print_timing: id 3 | task 3827 | graphs reused = 4008
0.59.602.054 I slot release: id 3 | task 3827 | stop processing: n_tokens = 352, truncated = 0
0.59.602.061 I srv update_slots: all slots are idle
0.59.608.770 I srv operator(): Chat format: peg-native
0.59.608.974 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.655 (> 0.100 thold), f_keep = 0.210
0.59.608.976 I srv get_availabl: updating prompt cache
0.59.609.010 W srv prompt_save: - saving prompt with length 352, total state size = 27.505 MiB (draft: 0.000 MiB)
0.59.632.029 I srv load: - looking for better prompt, base f_keep = 0.210, sim = 0.655
0.59.632.041 I srv update: - cache state: 15 prompts, 433.515 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
0.59.632.042 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
0.59.632.043 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
0.59.632.044 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
0.59.632.044 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
0.59.632.045 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
0.59.632.045 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
0.59.632.046 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
0.59.632.046 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
0.59.632.046 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
0.59.632.047 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
0.59.632.049 I srv update: - prompt 0x638b5b8a99b0: 407 tokens, checkpoints: 0, 31.802 MiB
0.59.632.049 I srv update: - prompt 0x638b5ba41a40: 320 tokens, checkpoints: 0, 25.005 MiB
0.59.632.050 I srv update: - prompt 0x638b5a6cdca0: 366 tokens, checkpoints: 0, 28.599 MiB
0.59.632.050 I srv update: - prompt 0x638b5ba61000: 335 tokens, checkpoints: 0, 26.177 MiB
0.59.632.051 I srv update: - prompt 0x638b5b6d0f60: 352 tokens, checkpoints: 0, 27.505 MiB
0.59.632.052 I srv get_availabl: prompt cache update took 23.08 ms
0.59.632.135 I slot launch_slot_: id 3 | task 4070 | processing task, is_child = 0
0.59.632.136 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.59.632.137 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.59.632.151 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.59.632.151 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.59.632.160 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.59.632.160 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.02.724.450 I slot print_timing: id 3 | task 4070 | n_decoded = 230, tg = 76.65 t/s, tg_3s = 76.65 t/s
1.02.764.090 I slot print_timing: id 3 | task 4070 | prompt eval time = 91.65 ms / 39 tokens ( 2.35 ms per token, 425.52 tokens per second)
1.02.764.093 I slot print_timing: id 3 | task 4070 | eval time = 3040.26 ms / 233 tokens ( 13.05 ms per token, 76.64 tokens per second)
1.02.764.094 I slot print_timing: id 3 | task 4070 | total time = 3131.91 ms / 272 tokens
1.02.764.094 I slot print_timing: id 3 | task 4070 | graphs reused = 4238
1.02.764.131 I slot release: id 3 | task 4070 | stop processing: n_tokens = 345, truncated = 0
1.02.764.139 I srv update_slots: all slots are idle
1.02.770.891 I srv operator(): Chat format: peg-native
1.02.771.094 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.718 (> 0.100 thold), f_keep = 0.214
1.02.771.095 I srv get_availabl: updating prompt cache
1.02.771.129 W srv prompt_save: - saving prompt with length 345, total state size = 26.958 MiB (draft: 0.000 MiB)
1.02.793.506 I srv load: - looking for better prompt, base f_keep = 0.214, sim = 0.718
1.02.793.520 I srv update: - cache state: 16 prompts, 460.473 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
1.02.793.520 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
1.02.793.521 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
1.02.793.522 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
1.02.793.522 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
1.02.793.523 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
1.02.793.523 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
1.02.793.524 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
1.02.793.524 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
1.02.793.524 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
1.02.793.525 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
1.02.793.525 I srv update: - prompt 0x638b5b8a99b0: 407 tokens, checkpoints: 0, 31.802 MiB
1.02.793.526 I srv update: - prompt 0x638b5ba41a40: 320 tokens, checkpoints: 0, 25.005 MiB
1.02.793.526 I srv update: - prompt 0x638b5a6cdca0: 366 tokens, checkpoints: 0, 28.599 MiB
1.02.793.527 I srv update: - prompt 0x638b5ba61000: 335 tokens, checkpoints: 0, 26.177 MiB
1.02.793.527 I srv update: - prompt 0x638b5b6d0f60: 352 tokens, checkpoints: 0, 27.505 MiB
1.02.793.528 I srv update: - prompt 0x638b5baa7990: 345 tokens, checkpoints: 0, 26.958 MiB
1.02.793.529 I srv get_availabl: prompt cache update took 22.43 ms
1.02.793.624 I slot launch_slot_: id 3 | task 4304 | processing task, is_child = 0
1.02.793.625 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.02.793.627 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.02.793.643 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.02.793.644 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.02.793.656 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.02.793.657 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.04.114.772 I slot print_timing: id 3 | task 4304 | prompt eval time = 81.01 ms / 29 tokens ( 2.79 ms per token, 357.98 tokens per second)
1.04.114.775 I slot print_timing: id 3 | task 4304 | eval time = 1240.08 ms / 96 tokens ( 12.92 ms per token, 77.41 tokens per second)
1.04.114.776 I slot print_timing: id 3 | task 4304 | total time = 1321.09 ms / 125 tokens
1.04.114.776 I slot print_timing: id 3 | task 4304 | graphs reused = 4332
1.04.114.807 I slot release: id 3 | task 4304 | stop processing: n_tokens = 198, truncated = 0
1.04.114.813 I srv update_slots: all slots are idle
1.04.121.363 I srv operator(): Chat format: peg-native
1.04.121.565 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.779 (> 0.100 thold), f_keep = 0.374
1.04.121.567 I srv get_availabl: updating prompt cache
1.04.121.591 W srv prompt_save: - saving prompt with length 198, total state size = 15.472 MiB (draft: 0.000 MiB)
1.04.135.364 I srv load: - looking for better prompt, base f_keep = 0.374, sim = 0.779
1.04.135.375 I srv update: - cache state: 17 prompts, 475.945 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
1.04.135.377 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
1.04.135.377 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
1.04.135.378 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
1.04.135.379 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
1.04.135.379 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
1.04.135.380 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
1.04.135.383 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
1.04.135.383 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
1.04.135.384 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
1.04.135.384 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
1.04.135.385 I srv update: - prompt 0x638b5b8a99b0: 407 tokens, checkpoints: 0, 31.802 MiB
1.04.135.385 I srv update: - prompt 0x638b5ba41a40: 320 tokens, checkpoints: 0, 25.005 MiB
1.04.135.385 I srv update: - prompt 0x638b5a6cdca0: 366 tokens, checkpoints: 0, 28.599 MiB
1.04.135.388 I srv update: - prompt 0x638b5ba61000: 335 tokens, checkpoints: 0, 26.177 MiB
1.04.135.389 I srv update: - prompt 0x638b5b6d0f60: 352 tokens, checkpoints: 0, 27.505 MiB
1.04.135.389 I srv update: - prompt 0x638b5baa7990: 345 tokens, checkpoints: 0, 26.958 MiB
1.04.135.390 I srv update: - prompt 0x638b5b942ae0: 198 tokens, checkpoints: 0, 15.472 MiB
1.04.135.391 I srv get_availabl: prompt cache update took 13.82 ms
1.04.135.468 I slot launch_slot_: id 3 | task 4401 | processing task, is_child = 0
1.04.135.470 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.04.135.471 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.04.135.483 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.04.135.483 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.04.135.492 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.04.135.492 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.06.840.047 I slot print_timing: id 3 | task 4401 | prompt eval time = 69.70 ms / 21 tokens ( 3.32 ms per token, 301.29 tokens per second)
1.06.840.050 I slot print_timing: id 3 | task 4401 | eval time = 2634.84 ms / 202 tokens ( 13.04 ms per token, 76.67 tokens per second)
1.06.840.051 I slot print_timing: id 3 | task 4401 | total time = 2704.54 ms / 223 tokens
1.06.840.051 I slot print_timing: id 3 | task 4401 | graphs reused = 4531
1.06.840.081 I slot release: id 3 | task 4401 | stop processing: n_tokens = 296, truncated = 0
1.06.840.087 I srv update_slots: all slots are idle
1.06.846.588 I srv operator(): Chat format: peg-native
1.06.846.809 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.649 (> 0.100 thold), f_keep = 0.250
1.06.846.810 I srv get_availabl: updating prompt cache
1.06.846.835 W srv prompt_save: - saving prompt with length 296, total state size = 23.129 MiB (draft: 0.000 MiB)
1.06.866.702 I srv load: - looking for better prompt, base f_keep = 0.250, sim = 0.649
1.06.866.714 I srv update: - cache state: 18 prompts, 499.074 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
1.06.866.715 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
1.06.866.716 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
1.06.866.719 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
1.06.866.720 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
1.06.866.720 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
1.06.866.721 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
1.06.866.721 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
1.06.866.722 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
1.06.866.725 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
1.06.866.725 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
1.06.866.725 I srv update: - prompt 0x638b5b8a99b0: 407 tokens, checkpoints: 0, 31.802 MiB
1.06.866.726 I srv update: - prompt 0x638b5ba41a40: 320 tokens, checkpoints: 0, 25.005 MiB
1.06.866.726 I srv update: - prompt 0x638b5a6cdca0: 366 tokens, checkpoints: 0, 28.599 MiB
1.06.866.727 I srv update: - prompt 0x638b5ba61000: 335 tokens, checkpoints: 0, 26.177 MiB
1.06.866.727 I srv update: - prompt 0x638b5b6d0f60: 352 tokens, checkpoints: 0, 27.505 MiB
1.06.866.728 I srv update: - prompt 0x638b5baa7990: 345 tokens, checkpoints: 0, 26.958 MiB
1.06.866.729 I srv update: - prompt 0x638b5b942ae0: 198 tokens, checkpoints: 0, 15.472 MiB
1.06.866.730 I srv update: - prompt 0x638b5b92a750: 296 tokens, checkpoints: 0, 23.129 MiB
1.06.866.731 I srv get_availabl: prompt cache update took 19.92 ms
1.06.866.813 I slot launch_slot_: id 3 | task 4604 | processing task, is_child = 0
1.06.866.814 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.06.866.815 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.06.866.826 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.06.866.827 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.06.866.835 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.06.866.835 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.09.525.863 I slot print_timing: id 3 | task 4604 | prompt eval time = 91.90 ms / 40 tokens ( 2.30 ms per token, 435.25 tokens per second)
1.09.525.866 I slot print_timing: id 3 | task 4604 | eval time = 2567.10 ms / 197 tokens ( 13.03 ms per token, 76.74 tokens per second)
1.09.525.867 I slot print_timing: id 3 | task 4604 | total time = 2659.01 ms / 237 tokens
1.09.525.868 I slot print_timing: id 3 | task 4604 | graphs reused = 4725
1.09.525.897 I slot release: id 3 | task 4604 | stop processing: n_tokens = 310, truncated = 0
1.09.525.907 I srv update_slots: all slots are idle
1.09.532.273 I srv operator(): Chat format: peg-native
1.09.532.485 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.718 (> 0.100 thold), f_keep = 0.239
1.09.532.486 I srv get_availabl: updating prompt cache
1.09.532.512 W srv prompt_save: - saving prompt with length 310, total state size = 24.223 MiB (draft: 0.000 MiB)
1.09.553.236 I srv load: - looking for better prompt, base f_keep = 0.239, sim = 0.718
1.09.553.250 I srv update: - cache state: 19 prompts, 523.298 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
1.09.553.251 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
1.09.553.255 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
1.09.553.256 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
1.09.553.256 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
1.09.553.257 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
1.09.553.257 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
1.09.553.258 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
1.09.553.271 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
1.09.553.272 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
1.09.553.272 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
1.09.553.273 I srv update: - prompt 0x638b5b8a99b0: 407 tokens, checkpoints: 0, 31.802 MiB
1.09.553.273 I srv update: - prompt 0x638b5ba41a40: 320 tokens, checkpoints: 0, 25.005 MiB
1.09.553.274 I srv update: - prompt 0x638b5a6cdca0: 366 tokens, checkpoints: 0, 28.599 MiB
1.09.553.274 I srv update: - prompt 0x638b5ba61000: 335 tokens, checkpoints: 0, 26.177 MiB
1.09.553.275 I srv update: - prompt 0x638b5b6d0f60: 352 tokens, checkpoints: 0, 27.505 MiB
1.09.553.275 I srv update: - prompt 0x638b5baa7990: 345 tokens, checkpoints: 0, 26.958 MiB
1.09.553.276 I srv update: - prompt 0x638b5b942ae0: 198 tokens, checkpoints: 0, 15.472 MiB
1.09.553.276 I srv update: - prompt 0x638b5b92a750: 296 tokens, checkpoints: 0, 23.129 MiB
1.09.553.277 I srv update: - prompt 0x638b5c724840: 310 tokens, checkpoints: 0, 24.223 MiB
1.09.553.278 I srv get_availabl: prompt cache update took 20.79 ms
1.09.553.364 I slot launch_slot_: id 3 | task 4802 | processing task, is_child = 0
1.09.553.365 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.09.553.366 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.09.553.378 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.09.553.378 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.09.553.387 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.09.553.388 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.12.645.944 I slot print_timing: id 3 | task 4802 | n_decoded = 231, tg = 76.71 t/s, tg_3s = 76.71 t/s
1.15.073.188 I slot print_timing: id 3 | task 4802 | prompt eval time = 81.08 ms / 29 tokens ( 2.80 ms per token, 357.68 tokens per second)
1.15.073.191 I slot print_timing: id 3 | task 4802 | eval time = 5438.70 ms / 417 tokens ( 13.04 ms per token, 76.67 tokens per second)
1.15.073.192 I slot print_timing: id 3 | task 4802 | total time = 5519.78 ms / 446 tokens
1.15.073.192 I slot print_timing: id 3 | task 4802 | graphs reused = 5138
1.15.073.221 I slot release: id 3 | task 4802 | stop processing: n_tokens = 519, truncated = 0
1.15.073.230 I srv update_slots: all slots are idle
1.15.079.687 I srv operator(): Chat format: peg-native
1.15.079.900 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.685 (> 0.100 thold), f_keep = 0.143
1.15.079.901 I srv get_availabl: updating prompt cache
1.15.079.935 W srv prompt_save: - saving prompt with length 519, total state size = 40.554 MiB (draft: 0.000 MiB)
1.15.112.015 I srv load: - looking for better prompt, base f_keep = 0.143, sim = 0.685
1.15.112.032 I srv update: - cache state: 20 prompts, 563.851 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
1.15.112.033 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
1.15.112.033 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
1.15.112.034 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
1.15.112.034 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
1.15.112.035 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
1.15.112.035 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
1.15.112.036 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
1.15.112.036 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
1.15.112.037 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
1.15.112.037 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
1.15.112.038 I srv update: - prompt 0x638b5b8a99b0: 407 tokens, checkpoints: 0, 31.802 MiB
1.15.112.038 I srv update: - prompt 0x638b5ba41a40: 320 tokens, checkpoints: 0, 25.005 MiB
1.15.112.039 I srv update: - prompt 0x638b5a6cdca0: 366 tokens, checkpoints: 0, 28.599 MiB
1.15.112.039 I srv update: - prompt 0x638b5ba61000: 335 tokens, checkpoints: 0, 26.177 MiB
1.15.112.039 I srv update: - prompt 0x638b5b6d0f60: 352 tokens, checkpoints: 0, 27.505 MiB
1.15.112.040 I srv update: - prompt 0x638b5baa7990: 345 tokens, checkpoints: 0, 26.958 MiB
1.15.112.040 I srv update: - prompt 0x638b5b942ae0: 198 tokens, checkpoints: 0, 15.472 MiB
1.15.112.041 I srv update: - prompt 0x638b5b92a750: 296 tokens, checkpoints: 0, 23.129 MiB
1.15.112.041 I srv update: - prompt 0x638b5c724840: 310 tokens, checkpoints: 0, 24.223 MiB
1.15.112.042 I srv update: - prompt 0x638b5ba45c70: 519 tokens, checkpoints: 0, 40.554 MiB
1.15.112.043 I srv get_availabl: prompt cache update took 32.14 ms
1.15.112.137 I slot launch_slot_: id 3 | task 5220 | processing task, is_child = 0
1.15.112.138 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.15.112.139 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.15.112.153 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.15.112.153 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.15.112.162 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.15.112.162 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.18.021.979 I slot print_timing: id 3 | task 5220 | prompt eval time = 91.31 ms / 34 tokens ( 2.69 ms per token, 372.37 tokens per second)
1.18.021.983 I slot print_timing: id 3 | task 5220 | eval time = 2818.49 ms / 217 tokens ( 12.99 ms per token, 76.99 tokens per second)
1.18.021.983 I slot print_timing: id 3 | task 5220 | total time = 2909.79 ms / 251 tokens
1.18.021.984 I slot print_timing: id 3 | task 5220 | graphs reused = 5352
1.18.022.017 I slot release: id 3 | task 5220 | stop processing: n_tokens = 324, truncated = 0
1.18.022.024 I srv update_slots: all slots are idle
1.18.028.678 I srv operator(): Chat format: peg-native
1.18.028.955 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.632 (> 0.100 thold), f_keep = 0.228
1.18.028.957 I srv get_availabl: updating prompt cache
1.18.028.988 W srv prompt_save: - saving prompt with length 324, total state size = 25.317 MiB (draft: 0.000 MiB)
1.18.050.437 I srv load: - looking for better prompt, base f_keep = 0.228, sim = 0.632
1.18.050.448 I srv load: - found better prompt with f_keep = 0.384, sim = 0.650
1.18.057.783 I srv update: - cache state: 20 prompts, 573.697 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
1.18.057.785 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
1.18.057.786 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
1.18.057.786 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
1.18.057.787 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
1.18.057.788 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
1.18.057.788 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
1.18.057.789 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
1.18.057.789 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
1.18.057.790 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
1.18.057.790 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
1.18.057.790 I srv update: - prompt 0x638b5b8a99b0: 407 tokens, checkpoints: 0, 31.802 MiB
1.18.057.791 I srv update: - prompt 0x638b5ba41a40: 320 tokens, checkpoints: 0, 25.005 MiB
1.18.057.791 I srv update: - prompt 0x638b5a6cdca0: 366 tokens, checkpoints: 0, 28.599 MiB
1.18.057.792 I srv update: - prompt 0x638b5ba61000: 335 tokens, checkpoints: 0, 26.177 MiB
1.18.057.793 I srv update: - prompt 0x638b5b6d0f60: 352 tokens, checkpoints: 0, 27.505 MiB
1.18.057.796 I srv update: - prompt 0x638b5baa7990: 345 tokens, checkpoints: 0, 26.958 MiB
1.18.057.797 I srv update: - prompt 0x638b5b92a750: 296 tokens, checkpoints: 0, 23.129 MiB
1.18.057.797 I srv update: - prompt 0x638b5c724840: 310 tokens, checkpoints: 0, 24.223 MiB
1.18.057.798 I srv update: - prompt 0x638b5ba45c70: 519 tokens, checkpoints: 0, 40.554 MiB
1.18.057.798 I srv update: - prompt 0x638b5bab0cd0: 324 tokens, checkpoints: 0, 25.317 MiB
1.18.057.799 I srv get_availabl: prompt cache update took 28.84 ms
1.18.057.866 I slot launch_slot_: id 3 | task 5438 | processing task, is_child = 0
1.18.057.866 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.18.057.867 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.18.057.880 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.18.057.880 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.18.057.888 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.18.057.888 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.21.138.958 I slot print_timing: id 3 | task 5438 | n_decoded = 231, tg = 76.96 t/s, tg_3s = 76.96 t/s
1.21.736.967 I slot print_timing: id 3 | task 5438 | prompt eval time = 79.44 ms / 41 tokens ( 1.94 ms per token, 516.13 tokens per second)
1.21.736.970 I slot print_timing: id 3 | task 5438 | eval time = 3599.62 ms / 277 tokens ( 13.00 ms per token, 76.95 tokens per second)
1.21.736.970 I slot print_timing: id 3 | task 5438 | total time = 3679.06 ms / 318 tokens
1.21.736.971 I slot print_timing: id 3 | task 5438 | graphs reused = 5626
1.21.737.005 I slot release: id 3 | task 5438 | stop processing: n_tokens = 393, truncated = 0
1.21.737.012 I srv update_slots: all slots are idle
1.21.743.717 I srv operator(): Chat format: peg-native
1.21.743.922 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.725 (> 0.100 thold), f_keep = 0.188
1.21.743.924 I srv get_availabl: updating prompt cache
1.21.743.961 W srv prompt_save: - saving prompt with length 393, total state size = 30.709 MiB (draft: 0.000 MiB)
1.21.768.704 I srv load: - looking for better prompt, base f_keep = 0.188, sim = 0.725
1.21.768.719 I srv update: - cache state: 21 prompts, 604.405 MiB (limits: 8192.000 MiB, 16384 tokens, 104838 est)
1.21.768.720 I srv update: - prompt 0x638b5b8e7550: 376 tokens, checkpoints: 0, 29.380 MiB
1.21.768.720 I srv update: - prompt 0x638b5996c780: 319 tokens, checkpoints: 0, 24.926 MiB
1.21.768.721 I srv update: - prompt 0x638b5b9fee00: 403 tokens, checkpoints: 0, 31.490 MiB
1.21.768.721 I srv update: - prompt 0x638b5c7248f0: 453 tokens, checkpoints: 0, 35.397 MiB
1.21.768.722 I srv update: - prompt 0x638b5b9b0870: 416 tokens, checkpoints: 0, 32.506 MiB
1.21.768.722 I srv update: - prompt 0x638b5b851c90: 243 tokens, checkpoints: 0, 18.988 MiB
1.21.768.723 I srv update: - prompt 0x638b5b8ad4e0: 393 tokens, checkpoints: 0, 30.709 MiB
1.21.768.723 I srv update: - prompt 0x638b5b927ca0: 336 tokens, checkpoints: 0, 26.255 MiB
1.21.768.724 I srv update: - prompt 0x638b5b942600: 446 tokens, checkpoints: 0, 34.850 MiB
1.21.768.724 I srv update: - prompt 0x638b58937ce0: 383 tokens, checkpoints: 0, 29.927 MiB
1.21.768.725 I srv update: - prompt 0x638b5b8a99b0: 407 tokens, checkpoints: 0, 31.802 MiB
1.21.768.725 I srv update: - prompt 0x638b5ba41a40: 320 tokens, checkpoints: 0, 25.005 MiB
1.21.768.726 I srv update: - prompt 0x638b5a6cdca0: 366 tokens, checkpoints: 0, 28.599 MiB
1.21.768.726 I srv update: - prompt 0x638b5ba61000: 335 tokens, checkpoints: 0, 26.177 MiB
1.21.768.727 I srv update: - prompt 0x638b5b6d0f60: 352 tokens, checkpoints: 0, 27.505 MiB
1.21.768.727 I srv update: - prompt 0x638b5baa7990: 345 tokens, checkpoints: 0, 26.958 MiB
1.21.768.728 I srv update: - prompt 0x638b5b92a750: 296 tokens, checkpoints: 0, 23.129 MiB
1.21.768.728 I srv update: - prompt 0x638b5c724840: 310 tokens, checkpoints: 0, 24.223 MiB
1.21.768.729 I srv update: - prompt 0x638b5ba45c70: 519 tokens, checkpoints: 0, 40.554 MiB
1.21.768.729 I srv update: - prompt 0x638b5bab0cd0: 324 tokens, checkpoints: 0, 25.317 MiB
1.21.768.730 I srv update: - prompt 0x638b5a6cde80: 393 tokens, checkpoints: 0, 30.709 MiB
1.21.768.731 I srv get_availabl: prompt cache update took 24.81 ms
1.21.768.809 I slot launch_slot_: id 3 | task 5716 | processing task, is_child = 0
1.21.768.810 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.21.768.811 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.21.768.823 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.21.768.823 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.21.768.832 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.21.768.832 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.24.850.017 I slot print_timing: id 3 | task 5716 | n_decoded = 231, tg = 76.99 t/s, tg_3s = 76.99 t/s
1.25.890.015 I slot print_timing: id 3 | task 5716 | prompt eval time = 80.70 ms / 28 tokens ( 2.88 ms per token, 346.96 tokens per second)
1.25.890.018 I slot print_timing: id 3 | task 5716 | eval time = 4040.46 ms / 311 tokens ( 12.99 ms per token, 76.97 tokens per second)
1.25.890.019 I slot print_timing: id 3 | task 5716 | total time = 4121.16 ms / 339 tokens
1.25.890.019 I slot print_timing: id 3 | task 5716 | graphs reused = 5934
1.25.890.050 I slot release: id 3 | task 5716 | stop processing: n_tokens = 412, truncated = 0
1.25.890.057 I srv update_slots: all slots are idle
1.25.890.953 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink