qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results/devstral-small2-24b.server.log
G gael 5cb7da3855 readme report
2026-07-13 01:35:10 +00:00

854 lines
80 KiB
Plaintext
Raw Permalink Blame History

0.00.065.082 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.065.085 I device_info:
0.00.065.105 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.110 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.117 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.065.175 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.065.179 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.065.214 I srv init: running without SSL
0.00.065.274 I srv init: using 23 threads for HTTP server
0.00.065.599 I srv start: binding port with default address family
0.00.066.812 I srv llama_server: loading model
0.00.066.819 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/mistralai_Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf'
0.00.066.859 I common_init_result: fitting params to device memory ...
0.00.066.861 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.07.675.285 W llama_context: setting new yarn_attn_factor = 1.0000 (mscale == 1.0, mscale_all_dim = 1.0)
0.07.675.293 W llama_context: n_ctx_seq (8192) < n_ctx_train (393216) -- the full capacity of the model will not be utilized
0.07.741.408 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.07.903.825 I srv load_model: initializing slots, n_slots = 4
0.07.964.793 W common_speculative_init: no implementations specified for speculative decoding
0.07.964.796 I slot load_model: id 0 | task -1 | new slot, n_ctx = 8192
0.07.964.799 I slot load_model: id 1 | task -1 | new slot, n_ctx = 8192
0.07.964.799 I slot load_model: id 2 | task -1 | new slot, n_ctx = 8192
0.07.964.800 I slot load_model: id 3 | task -1 | new slot, n_ctx = 8192
0.07.964.907 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.07.964.907 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.07.964.908 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.07.964.908 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.07.964.923 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.07.967.873 I init: chat template, example_format: '[SYSTEM_PROMPT]You are a helpful assistant[/SYSTEM_PROMPT][INST]Hello[/INST]Hi there</s>[INST]How are you?[/INST]'
0.07.968.272 I srv init: init: chat template, thinking = 1
0.07.968.304 I srv llama_server: model loaded
0.07.968.307 I srv llama_server: server is listening on http://127.0.0.1:8090
0.07.968.312 I srv update_slots: all slots are idle
0.09.964.703 I srv operator(): Chat format: peg-native
0.09.964.884 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.09.964.887 I srv get_availabl: updating prompt cache
0.09.964.894 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.09.964.899 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.09.964.899 I srv get_availabl: prompt cache update took 0.01 ms
0.09.964.941 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.09.964.942 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.09.964.943 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.09.964.953 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.09.964.953 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.09.964.958 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.09.964.958 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.12.330.821 I slot print_timing: id 3 | task 0 | prompt eval time = 330.78 ms / 64 tokens ( 5.17 ms per token, 193.48 tokens per second)
0.12.330.825 I slot print_timing: id 3 | task 0 | eval time = 2035.06 ms / 50 tokens ( 40.70 ms per token, 24.57 tokens per second)
0.12.330.825 I slot print_timing: id 3 | task 0 | total time = 2365.84 ms / 114 tokens
0.12.330.829 I slot print_timing: id 3 | task 0 | graphs reused = 0
0.12.330.861 I slot release: id 3 | task 0 | stop processing: n_tokens = 113, truncated = 0
0.12.330.866 I srv update_slots: all slots are idle
0.12.333.637 I srv operator(): Chat format: peg-native
0.12.333.744 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.818 (> 0.100 thold), f_keep = 0.398
0.12.333.745 I srv get_availabl: updating prompt cache
0.12.333.765 W srv prompt_save: - saving prompt with length 113, total state size = 17.658 MiB (draft: 0.000 MiB)
0.12.349.568 I srv load: - looking for better prompt, base f_keep = 0.398, sim = 0.818
0.12.349.574 I srv update: - cache state: 1 prompts, 17.658 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.12.349.575 I srv update: - prompt 0x56ead493f460: 113 tokens, checkpoints: 0, 17.658 MiB
0.12.349.576 I srv get_availabl: prompt cache update took 15.83 ms
0.12.349.610 I slot launch_slot_: id 3 | task 51 | processing task, is_child = 0
0.12.349.612 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.12.349.613 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.12.349.620 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.12.349.620 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.12.349.625 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.12.349.625 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.13.055.537 I slot print_timing: id 3 | task 51 | prompt eval time = 122.98 ms / 10 tokens ( 12.30 ms per token, 81.31 tokens per second)
0.13.055.540 I slot print_timing: id 3 | task 51 | eval time = 582.91 ms / 15 tokens ( 38.86 ms per token, 25.73 tokens per second)
0.13.055.541 I slot print_timing: id 3 | task 51 | total time = 705.89 ms / 25 tokens
0.13.055.541 I slot print_timing: id 3 | task 51 | graphs reused = 0
0.13.055.557 I slot release: id 3 | task 51 | stop processing: n_tokens = 69, truncated = 0
0.13.055.564 I srv update_slots: all slots are idle
0.13.058.183 I srv operator(): Chat format: peg-native
0.13.058.285 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.652
0.13.058.304 I slot launch_slot_: id 3 | task 67 | processing task, is_child = 0
0.13.058.305 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.13.058.307 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.13.058.313 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.13.058.313 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.13.058.318 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.13.058.318 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.14.672.617 I slot print_timing: id 3 | task 67 | prompt eval time = 123.27 ms / 13 tokens ( 9.48 ms per token, 105.46 tokens per second)
0.14.672.620 I slot print_timing: id 3 | task 67 | eval time = 1491.01 ms / 37 tokens ( 40.30 ms per token, 24.82 tokens per second)
0.14.672.621 I slot print_timing: id 3 | task 67 | total time = 1614.28 ms / 50 tokens
0.14.672.622 I slot print_timing: id 3 | task 67 | graphs reused = 0
0.14.672.638 I slot release: id 3 | task 67 | stop processing: n_tokens = 94, truncated = 0
0.14.672.644 I srv update_slots: all slots are idle
0.14.675.059 I srv operator(): Chat format: peg-native
0.14.675.181 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.807 (> 0.100 thold), f_keep = 0.489
0.14.675.183 I srv get_availabl: updating prompt cache
0.14.675.199 W srv prompt_save: - saving prompt with length 94, total state size = 14.690 MiB (draft: 0.000 MiB)
0.14.689.166 I srv load: - looking for better prompt, base f_keep = 0.489, sim = 0.807
0.14.689.172 I srv update: - cache state: 2 prompts, 32.348 MiB (limits: 8192.000 MiB, 8192 tokens, 52421 est)
0.14.689.173 I srv update: - prompt 0x56ead493f460: 113 tokens, checkpoints: 0, 17.658 MiB
0.14.689.173 I srv update: - prompt 0x56ead42ba0a0: 94 tokens, checkpoints: 0, 14.690 MiB
0.14.689.174 I srv get_availabl: prompt cache update took 13.99 ms
0.14.689.202 I slot launch_slot_: id 3 | task 105 | processing task, is_child = 0
0.14.689.202 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.14.689.203 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.14.689.210 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.14.689.210 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.14.689.215 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.14.689.215 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.17.939.153 I slot print_timing: id 3 | task 105 | prompt eval time = 122.96 ms / 11 tokens ( 11.18 ms per token, 89.46 tokens per second)
0.17.939.157 I slot print_timing: id 3 | task 105 | eval time = 3126.95 ms / 77 tokens ( 40.61 ms per token, 24.62 tokens per second)
0.17.939.157 I slot print_timing: id 3 | task 105 | total time = 3249.92 ms / 88 tokens
0.17.939.158 I slot print_timing: id 3 | task 105 | graphs reused = 0
0.17.939.173 I slot release: id 3 | task 105 | stop processing: n_tokens = 133, truncated = 0
0.17.939.180 I srv update_slots: all slots are idle
0.17.941.623 I srv operator(): Chat format: peg-native
0.17.941.724 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.780 (> 0.100 thold), f_keep = 0.346
0.17.941.725 I srv get_availabl: updating prompt cache
0.17.941.743 W srv prompt_save: - saving prompt with length 133, total state size = 20.784 MiB (draft: 0.000 MiB)
0.17.959.916 I srv load: - looking for better prompt, base f_keep = 0.346, sim = 0.780
0.17.959.922 I srv load: - found better prompt with f_keep = 0.425, sim = 0.814
0.17.969.202 I srv update: - cache state: 2 prompts, 35.473 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.17.969.205 I srv update: - prompt 0x56ead42ba0a0: 94 tokens, checkpoints: 0, 14.690 MiB
0.17.969.206 I srv update: - prompt 0x56ead176d1b0: 133 tokens, checkpoints: 0, 20.784 MiB
0.17.969.207 I srv get_availabl: prompt cache update took 27.48 ms
0.17.969.238 I slot launch_slot_: id 3 | task 183 | processing task, is_child = 0
0.17.969.239 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.17.969.240 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.17.969.249 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.17.969.249 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.17.969.253 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.17.969.254 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.18.954.172 I slot print_timing: id 3 | task 183 | prompt eval time = 123.08 ms / 11 tokens ( 11.19 ms per token, 89.37 tokens per second)
0.18.954.175 I slot print_timing: id 3 | task 183 | eval time = 861.82 ms / 22 tokens ( 39.17 ms per token, 25.53 tokens per second)
0.18.954.176 I slot print_timing: id 3 | task 183 | total time = 984.90 ms / 33 tokens
0.18.954.177 I slot print_timing: id 3 | task 183 | graphs reused = 0
0.18.954.195 I slot release: id 3 | task 183 | stop processing: n_tokens = 80, truncated = 0
0.18.954.202 I srv update_slots: all slots are idle
0.18.956.647 I srv operator(): Chat format: peg-native
0.18.956.751 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.562
0.18.956.769 I slot launch_slot_: id 3 | task 206 | processing task, is_child = 0
0.18.956.770 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.18.956.771 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.18.956.776 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.18.956.777 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.18.956.781 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.18.956.782 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.19.530.900 I slot print_timing: id 3 | task 206 | prompt eval time = 122.68 ms / 12 tokens ( 10.22 ms per token, 97.81 tokens per second)
0.19.530.904 I slot print_timing: id 3 | task 206 | eval time = 451.42 ms / 12 tokens ( 37.62 ms per token, 26.58 tokens per second)
0.19.530.904 I slot print_timing: id 3 | task 206 | total time = 574.11 ms / 24 tokens
0.19.530.905 I slot print_timing: id 3 | task 206 | graphs reused = 0
0.19.530.919 I slot release: id 3 | task 206 | stop processing: n_tokens = 68, truncated = 0
0.19.530.925 I srv update_slots: all slots are idle
0.19.533.550 I srv operator(): Chat format: peg-native
0.19.533.673 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.662
0.19.533.690 I slot launch_slot_: id 3 | task 219 | processing task, is_child = 0
0.19.533.691 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.19.533.691 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.19.533.696 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.19.533.697 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.19.533.701 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.19.533.701 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.20.569.171 I slot print_timing: id 3 | task 219 | prompt eval time = 212.79 ms / 18 tokens ( 11.82 ms per token, 84.59 tokens per second)
0.20.569.175 I slot print_timing: id 3 | task 219 | eval time = 822.66 ms / 21 tokens ( 39.17 ms per token, 25.53 tokens per second)
0.20.569.176 I slot print_timing: id 3 | task 219 | total time = 1035.45 ms / 39 tokens
0.20.569.176 I slot print_timing: id 3 | task 219 | graphs reused = 0
0.20.569.197 I slot release: id 3 | task 219 | stop processing: n_tokens = 83, truncated = 0
0.20.569.204 I srv update_slots: all slots are idle
0.20.571.573 I srv operator(): Chat format: peg-native
0.20.571.680 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.868 (> 0.100 thold), f_keep = 0.554
0.20.571.697 I slot launch_slot_: id 3 | task 241 | processing task, is_child = 0
0.20.571.698 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.20.571.699 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.20.571.706 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.20.571.706 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.20.571.710 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.20.571.711 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.22.643.117 I slot print_timing: id 3 | task 241 | prompt eval time = 179.34 ms / 7 tokens ( 25.62 ms per token, 39.03 tokens per second)
0.22.643.121 I slot print_timing: id 3 | task 241 | eval time = 1892.05 ms / 47 tokens ( 40.26 ms per token, 24.84 tokens per second)
0.22.643.122 I slot print_timing: id 3 | task 241 | total time = 2071.39 ms / 54 tokens
0.22.643.122 I slot print_timing: id 3 | task 241 | graphs reused = 0
0.22.643.141 I slot release: id 3 | task 241 | stop processing: n_tokens = 99, truncated = 0
0.22.643.149 I srv update_slots: all slots are idle
0.22.645.432 I srv operator(): Chat format: peg-native
0.22.645.528 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.455
0.22.645.529 I srv get_availabl: updating prompt cache
0.22.645.544 W srv prompt_save: - saving prompt with length 99, total state size = 15.471 MiB (draft: 0.000 MiB)
0.22.659.107 I srv load: - looking for better prompt, base f_keep = 0.455, sim = 0.714
0.22.659.114 I srv update: - cache state: 3 prompts, 50.944 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.22.659.115 I srv update: - prompt 0x56ead42ba0a0: 94 tokens, checkpoints: 0, 14.690 MiB
0.22.659.115 I srv update: - prompt 0x56ead176d1b0: 133 tokens, checkpoints: 0, 20.784 MiB
0.22.659.116 I srv update: - prompt 0x56ead42ad630: 99 tokens, checkpoints: 0, 15.471 MiB
0.22.659.117 I srv get_availabl: prompt cache update took 13.59 ms
0.22.659.146 I slot launch_slot_: id 3 | task 289 | processing task, is_child = 0
0.22.659.147 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.22.659.148 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.22.659.154 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.22.659.155 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.22.659.159 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.22.659.160 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.24.025.185 I slot print_timing: id 3 | task 289 | prompt eval time = 212.89 ms / 18 tokens ( 11.83 ms per token, 84.55 tokens per second)
0.24.025.188 I slot print_timing: id 3 | task 289 | eval time = 1153.12 ms / 29 tokens ( 39.76 ms per token, 25.15 tokens per second)
0.24.025.189 I slot print_timing: id 3 | task 289 | total time = 1366.01 ms / 47 tokens
0.24.025.190 I slot print_timing: id 3 | task 289 | graphs reused = 0
0.24.025.209 I slot release: id 3 | task 289 | stop processing: n_tokens = 91, truncated = 0
0.24.025.217 I srv update_slots: all slots are idle
0.24.027.577 I srv operator(): Chat format: peg-native
0.24.027.675 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.495
0.24.027.677 I srv get_availabl: updating prompt cache
0.24.027.695 W srv prompt_save: - saving prompt with length 91, total state size = 14.221 MiB (draft: 0.000 MiB)
0.24.040.855 I srv load: - looking for better prompt, base f_keep = 0.495, sim = 0.776
0.24.040.859 I srv load: - found better prompt with f_keep = 0.511, sim = 0.828
0.24.048.367 I srv update: - cache state: 3 prompts, 50.475 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.24.048.371 I srv update: - prompt 0x56ead176d1b0: 133 tokens, checkpoints: 0, 20.784 MiB
0.24.048.372 I srv update: - prompt 0x56ead42ad630: 99 tokens, checkpoints: 0, 15.471 MiB
0.24.048.372 I srv update: - prompt 0x56ead495d400: 91 tokens, checkpoints: 0, 14.221 MiB
0.24.048.374 I srv get_availabl: prompt cache update took 20.70 ms
0.24.048.403 I slot launch_slot_: id 3 | task 319 | processing task, is_child = 0
0.24.048.404 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.24.048.405 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.24.048.412 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.24.048.413 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.24.048.417 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.24.048.417 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.27.761.603 I slot print_timing: id 3 | task 319 | prompt eval time = 122.19 ms / 10 tokens ( 12.22 ms per token, 81.84 tokens per second)
0.27.761.608 I slot print_timing: id 3 | task 319 | eval time = 3590.97 ms / 88 tokens ( 40.81 ms per token, 24.51 tokens per second)
0.27.761.608 I slot print_timing: id 3 | task 319 | total time = 3713.16 ms / 98 tokens
0.27.761.609 I slot print_timing: id 3 | task 319 | graphs reused = 0
0.27.761.627 I slot release: id 3 | task 319 | stop processing: n_tokens = 145, truncated = 0
0.27.761.634 I srv update_slots: all slots are idle
0.27.764.380 I srv operator(): Chat format: peg-native
0.27.764.538 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.310
0.27.764.540 I srv get_availabl: updating prompt cache
0.27.764.555 W srv prompt_save: - saving prompt with length 145, total state size = 22.659 MiB (draft: 0.000 MiB)
0.27.783.688 I srv load: - looking for better prompt, base f_keep = 0.310, sim = 0.714
0.27.783.697 I srv update: - cache state: 4 prompts, 73.134 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.27.783.698 I srv update: - prompt 0x56ead176d1b0: 133 tokens, checkpoints: 0, 20.784 MiB
0.27.783.699 I srv update: - prompt 0x56ead42ad630: 99 tokens, checkpoints: 0, 15.471 MiB
0.27.783.699 I srv update: - prompt 0x56ead495d400: 91 tokens, checkpoints: 0, 14.221 MiB
0.27.783.700 I srv update: - prompt 0x56ead4a3fc50: 145 tokens, checkpoints: 0, 22.659 MiB
0.27.783.701 I srv get_availabl: prompt cache update took 19.16 ms
0.27.783.733 I slot launch_slot_: id 3 | task 408 | processing task, is_child = 0
0.27.783.734 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.27.783.735 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.27.783.743 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.27.783.744 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.27.783.748 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.27.783.749 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.29.689.186 I slot print_timing: id 3 | task 408 | prompt eval time = 212.24 ms / 18 tokens ( 11.79 ms per token, 84.81 tokens per second)
0.29.689.190 I slot print_timing: id 3 | task 408 | eval time = 1693.18 ms / 42 tokens ( 40.31 ms per token, 24.81 tokens per second)
0.29.689.191 I slot print_timing: id 3 | task 408 | total time = 1905.42 ms / 60 tokens
0.29.689.191 I slot print_timing: id 3 | task 408 | graphs reused = 0
0.29.689.211 I slot release: id 3 | task 408 | stop processing: n_tokens = 104, truncated = 0
0.29.689.217 I srv update_slots: all slots are idle
0.29.691.663 I srv operator(): Chat format: peg-native
0.29.691.767 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.433
0.29.691.769 I srv get_availabl: updating prompt cache
0.29.691.786 W srv prompt_save: - saving prompt with length 104, total state size = 16.252 MiB (draft: 0.000 MiB)
0.29.706.731 I srv load: - looking for better prompt, base f_keep = 0.433, sim = 0.776
0.29.706.737 I srv load: - found better prompt with f_keep = 0.465, sim = 0.793
0.29.713.446 I srv update: - cache state: 4 prompts, 73.915 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.29.713.450 I srv update: - prompt 0x56ead176d1b0: 133 tokens, checkpoints: 0, 20.784 MiB
0.29.713.451 I srv update: - prompt 0x56ead495d400: 91 tokens, checkpoints: 0, 14.221 MiB
0.29.713.451 I srv update: - prompt 0x56ead4a3fc50: 145 tokens, checkpoints: 0, 22.659 MiB
0.29.713.452 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
0.29.713.453 I srv get_availabl: prompt cache update took 21.68 ms
0.29.713.480 I slot launch_slot_: id 3 | task 451 | processing task, is_child = 0
0.29.713.480 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.713.481 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.713.489 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.713.489 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.713.493 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.713.494 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.31.902.881 I slot print_timing: id 3 | task 451 | prompt eval time = 122.26 ms / 12 tokens ( 10.19 ms per token, 98.15 tokens per second)
0.31.902.885 I slot print_timing: id 3 | task 451 | eval time = 2067.11 ms / 51 tokens ( 40.53 ms per token, 24.67 tokens per second)
0.31.902.885 I slot print_timing: id 3 | task 451 | total time = 2189.37 ms / 63 tokens
0.31.902.886 I slot print_timing: id 3 | task 451 | graphs reused = 0
0.31.902.902 I slot release: id 3 | task 451 | stop processing: n_tokens = 108, truncated = 0
0.31.902.909 I srv update_slots: all slots are idle
0.31.905.581 I srv operator(): Chat format: peg-native
0.31.905.687 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.750 (> 0.100 thold), f_keep = 0.417
0.31.905.689 I srv get_availabl: updating prompt cache
0.31.905.705 W srv prompt_save: - saving prompt with length 108, total state size = 16.877 MiB (draft: 0.000 MiB)
0.31.914.376 I srv load: - looking for better prompt, base f_keep = 0.417, sim = 0.750
0.31.914.384 I srv update: - cache state: 5 prompts, 90.793 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.31.914.385 I srv update: - prompt 0x56ead176d1b0: 133 tokens, checkpoints: 0, 20.784 MiB
0.31.914.385 I srv update: - prompt 0x56ead495d400: 91 tokens, checkpoints: 0, 14.221 MiB
0.31.914.386 I srv update: - prompt 0x56ead4a3fc50: 145 tokens, checkpoints: 0, 22.659 MiB
0.31.914.387 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
0.31.914.387 I srv update: - prompt 0x56ead47d1d90: 108 tokens, checkpoints: 0, 16.877 MiB
0.31.914.388 I srv get_availabl: prompt cache update took 8.70 ms
0.31.914.414 I slot launch_slot_: id 3 | task 503 | processing task, is_child = 0
0.31.914.414 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.31.914.415 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.31.914.421 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.31.914.422 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.31.914.426 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.31.914.426 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.32.698.654 I slot print_timing: id 3 | task 503 | prompt eval time = 122.91 ms / 15 tokens ( 8.19 ms per token, 122.04 tokens per second)
0.32.698.657 I slot print_timing: id 3 | task 503 | eval time = 661.30 ms / 17 tokens ( 38.90 ms per token, 25.71 tokens per second)
0.32.698.658 I slot print_timing: id 3 | task 503 | total time = 784.21 ms / 32 tokens
0.32.698.659 I slot print_timing: id 3 | task 503 | graphs reused = 0
0.32.698.676 I slot release: id 3 | task 503 | stop processing: n_tokens = 76, truncated = 0
0.32.698.683 I srv update_slots: all slots are idle
0.32.701.348 I srv operator(): Chat format: peg-native
0.32.701.456 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.763 (> 0.100 thold), f_keep = 0.592
0.32.701.476 I slot launch_slot_: id 3 | task 521 | processing task, is_child = 0
0.32.701.477 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.32.701.477 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.32.701.487 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.32.701.488 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.32.701.492 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.32.701.492 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.34.355.671 I slot print_timing: id 3 | task 521 | prompt eval time = 122.37 ms / 14 tokens ( 8.74 ms per token, 114.41 tokens per second)
0.34.355.674 I slot print_timing: id 3 | task 521 | eval time = 1531.80 ms / 38 tokens ( 40.31 ms per token, 24.81 tokens per second)
0.34.355.674 I slot print_timing: id 3 | task 521 | total time = 1654.16 ms / 52 tokens
0.34.355.675 I slot print_timing: id 3 | task 521 | graphs reused = 0
0.34.355.692 I slot release: id 3 | task 521 | stop processing: n_tokens = 96, truncated = 0
0.34.355.699 I srv update_slots: all slots are idle
0.34.358.331 I srv operator(): Chat format: peg-native
0.34.358.438 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.738 (> 0.100 thold), f_keep = 0.469
0.34.358.440 I srv get_availabl: updating prompt cache
0.34.358.458 W srv prompt_save: - saving prompt with length 96, total state size = 15.002 MiB (draft: 0.000 MiB)
0.34.372.418 I srv load: - looking for better prompt, base f_keep = 0.469, sim = 0.738
0.34.372.425 I srv update: - cache state: 6 prompts, 105.795 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.34.372.425 I srv update: - prompt 0x56ead176d1b0: 133 tokens, checkpoints: 0, 20.784 MiB
0.34.372.426 I srv update: - prompt 0x56ead495d400: 91 tokens, checkpoints: 0, 14.221 MiB
0.34.372.427 I srv update: - prompt 0x56ead4a3fc50: 145 tokens, checkpoints: 0, 22.659 MiB
0.34.372.427 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
0.34.372.428 I srv update: - prompt 0x56ead47d1d90: 108 tokens, checkpoints: 0, 16.877 MiB
0.34.372.428 I srv update: - prompt 0x56ead47d1960: 96 tokens, checkpoints: 0, 15.002 MiB
0.34.372.430 I srv get_availabl: prompt cache update took 13.99 ms
0.34.372.456 I slot launch_slot_: id 3 | task 560 | processing task, is_child = 0
0.34.372.458 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.34.372.458 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.34.372.465 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.34.372.465 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.34.372.470 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.34.372.470 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.35.281.872 I slot print_timing: id 3 | task 560 | prompt eval time = 123.42 ms / 16 tokens ( 7.71 ms per token, 129.64 tokens per second)
0.35.281.875 I slot print_timing: id 3 | task 560 | eval time = 785.97 ms / 20 tokens ( 39.30 ms per token, 25.45 tokens per second)
0.35.281.876 I slot print_timing: id 3 | task 560 | total time = 909.38 ms / 36 tokens
0.35.281.877 I slot print_timing: id 3 | task 560 | graphs reused = 0
0.35.281.895 I slot release: id 3 | task 560 | stop processing: n_tokens = 80, truncated = 0
0.35.281.903 I srv update_slots: all slots are idle
0.35.284.462 I srv operator(): Chat format: peg-native
0.35.284.564 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.562
0.35.284.581 I slot launch_slot_: id 3 | task 581 | processing task, is_child = 0
0.35.284.581 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.35.284.582 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.35.284.588 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.35.284.588 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.35.284.593 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.35.284.593 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.35.861.843 I slot print_timing: id 3 | task 581 | prompt eval time = 122.24 ms / 12 tokens ( 10.19 ms per token, 98.17 tokens per second)
0.35.861.847 I slot print_timing: id 3 | task 581 | eval time = 455.00 ms / 12 tokens ( 37.92 ms per token, 26.37 tokens per second)
0.35.861.847 I slot print_timing: id 3 | task 581 | total time = 577.24 ms / 24 tokens
0.35.861.848 I slot print_timing: id 3 | task 581 | graphs reused = 0
0.35.861.866 I slot release: id 3 | task 581 | stop processing: n_tokens = 68, truncated = 0
0.35.861.873 I srv update_slots: all slots are idle
0.35.864.420 I srv operator(): Chat format: peg-native
0.35.864.526 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.738 (> 0.100 thold), f_keep = 0.662
0.35.864.541 I slot launch_slot_: id 3 | task 594 | processing task, is_child = 0
0.35.864.542 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.35.864.542 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.35.864.547 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.35.864.548 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.35.864.552 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.35.864.552 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.37.394.861 I slot print_timing: id 3 | task 594 | prompt eval time = 123.29 ms / 16 tokens ( 7.71 ms per token, 129.77 tokens per second)
0.37.394.865 I slot print_timing: id 3 | task 594 | eval time = 1407.00 ms / 35 tokens ( 40.20 ms per token, 24.88 tokens per second)
0.37.394.865 I slot print_timing: id 3 | task 594 | total time = 1530.30 ms / 51 tokens
0.37.394.866 I slot print_timing: id 3 | task 594 | graphs reused = 0
0.37.394.882 I slot release: id 3 | task 594 | stop processing: n_tokens = 95, truncated = 0
0.37.394.889 I srv update_slots: all slots are idle
0.37.397.486 I srv operator(): Chat format: peg-native
0.37.397.589 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.804 (> 0.100 thold), f_keep = 0.474
0.37.397.590 I srv get_availabl: updating prompt cache
0.37.397.603 W srv prompt_save: - saving prompt with length 95, total state size = 14.846 MiB (draft: 0.000 MiB)
0.37.411.166 I srv load: - looking for better prompt, base f_keep = 0.474, sim = 0.804
0.37.411.170 I srv load: - found better prompt with f_keep = 0.505, sim = 0.821
0.37.417.573 I srv update: - cache state: 6 prompts, 106.420 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.37.417.576 I srv update: - prompt 0x56ead176d1b0: 133 tokens, checkpoints: 0, 20.784 MiB
0.37.417.577 I srv update: - prompt 0x56ead4a3fc50: 145 tokens, checkpoints: 0, 22.659 MiB
0.37.417.577 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
0.37.417.578 I srv update: - prompt 0x56ead47d1d90: 108 tokens, checkpoints: 0, 16.877 MiB
0.37.417.578 I srv update: - prompt 0x56ead47d1960: 96 tokens, checkpoints: 0, 15.002 MiB
0.37.417.579 I srv update: - prompt 0x56ead49229b0: 95 tokens, checkpoints: 0, 14.846 MiB
0.37.417.580 I srv get_availabl: prompt cache update took 19.99 ms
0.37.417.607 I slot launch_slot_: id 3 | task 630 | processing task, is_child = 0
0.37.417.608 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.37.417.609 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.37.417.616 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.37.417.617 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.37.417.621 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.37.417.621 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.38.739.657 I slot print_timing: id 3 | task 630 | prompt eval time = 122.18 ms / 10 tokens ( 12.22 ms per token, 81.84 tokens per second)
0.38.739.661 I slot print_timing: id 3 | task 630 | eval time = 1199.84 ms / 30 tokens ( 39.99 ms per token, 25.00 tokens per second)
0.38.739.662 I slot print_timing: id 3 | task 630 | total time = 1322.02 ms / 40 tokens
0.38.739.663 I slot print_timing: id 3 | task 630 | graphs reused = 0
0.38.739.679 I slot release: id 3 | task 630 | stop processing: n_tokens = 85, truncated = 0
0.38.739.685 I srv update_slots: all slots are idle
0.38.742.185 I srv operator(): Chat format: peg-native
0.38.742.308 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.818 (> 0.100 thold), f_keep = 0.529
0.38.742.325 I slot launch_slot_: id 3 | task 661 | processing task, is_child = 0
0.38.742.327 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.742.328 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.38.742.334 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.742.335 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.38.742.339 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.742.339 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.39.277.775 I slot print_timing: id 3 | task 661 | prompt eval time = 121.62 ms / 10 tokens ( 12.16 ms per token, 82.22 tokens per second)
0.39.277.778 I slot print_timing: id 3 | task 661 | eval time = 413.80 ms / 11 tokens ( 37.62 ms per token, 26.58 tokens per second)
0.39.277.778 I slot print_timing: id 3 | task 661 | total time = 535.42 ms / 21 tokens
0.39.277.779 I slot print_timing: id 3 | task 661 | graphs reused = 0
0.39.277.792 I slot release: id 3 | task 661 | stop processing: n_tokens = 65, truncated = 0
0.39.277.798 I srv update_slots: all slots are idle
0.39.280.283 I srv operator(): Chat format: peg-native
0.39.280.396 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.852 (> 0.100 thold), f_keep = 0.708
0.39.280.414 I slot launch_slot_: id 3 | task 673 | processing task, is_child = 0
0.39.280.415 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.39.280.416 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.39.280.422 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.39.280.422 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.39.280.426 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.39.280.427 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.39.932.900 I slot print_timing: id 3 | task 673 | prompt eval time = 197.66 ms / 8 tokens ( 24.71 ms per token, 40.47 tokens per second)
0.39.932.903 I slot print_timing: id 3 | task 673 | eval time = 454.80 ms / 12 tokens ( 37.90 ms per token, 26.39 tokens per second)
0.39.932.903 I slot print_timing: id 3 | task 673 | total time = 652.46 ms / 20 tokens
0.39.932.904 I slot print_timing: id 3 | task 673 | graphs reused = 0
0.39.932.916 I slot release: id 3 | task 673 | stop processing: n_tokens = 65, truncated = 0
0.39.932.922 I srv update_slots: all slots are idle
0.39.935.362 I srv operator(): Chat format: peg-native
0.39.935.467 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.849 (> 0.100 thold), f_keep = 0.692
0.39.935.486 I slot launch_slot_: id 3 | task 686 | processing task, is_child = 0
0.39.935.486 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.39.935.491 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.39.935.496 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.39.935.496 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.39.935.501 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.39.935.501 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.40.629.896 I slot print_timing: id 3 | task 686 | prompt eval time = 197.48 ms / 8 tokens ( 24.68 ms per token, 40.51 tokens per second)
0.40.629.898 I slot print_timing: id 3 | task 686 | eval time = 496.90 ms / 13 tokens ( 38.22 ms per token, 26.16 tokens per second)
0.40.629.899 I slot print_timing: id 3 | task 686 | total time = 694.38 ms / 21 tokens
0.40.629.900 I slot print_timing: id 3 | task 686 | graphs reused = 0
0.40.629.914 I slot release: id 3 | task 686 | stop processing: n_tokens = 65, truncated = 0
0.40.629.920 I srv update_slots: all slots are idle
0.40.632.357 I srv operator(): Chat format: peg-native
0.40.632.457 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.692
0.40.632.476 I slot launch_slot_: id 3 | task 700 | processing task, is_child = 0
0.40.632.480 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.40.632.481 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.40.632.487 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.40.632.487 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.40.632.492 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.40.632.492 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.42.907.836 I slot print_timing: id 3 | task 700 | prompt eval time = 122.30 ms / 13 tokens ( 9.41 ms per token, 106.29 tokens per second)
0.42.907.839 I slot print_timing: id 3 | task 700 | eval time = 2153.03 ms / 53 tokens ( 40.62 ms per token, 24.62 tokens per second)
0.42.907.840 I slot print_timing: id 3 | task 700 | total time = 2275.33 ms / 66 tokens
0.42.907.840 I slot print_timing: id 3 | task 700 | graphs reused = 0
0.42.907.858 I slot release: id 3 | task 700 | stop processing: n_tokens = 110, truncated = 0
0.42.907.863 I srv update_slots: all slots are idle
0.42.910.273 I srv operator(): Chat format: peg-native
0.42.910.370 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.409
0.42.910.371 I srv get_availabl: updating prompt cache
0.42.910.391 W srv prompt_save: - saving prompt with length 110, total state size = 17.190 MiB (draft: 0.000 MiB)
0.42.925.452 I srv load: - looking for better prompt, base f_keep = 0.409, sim = 0.789
0.42.925.458 I srv load: - found better prompt with f_keep = 0.505, sim = 0.842
0.42.931.924 I srv update: - cache state: 6 prompts, 108.764 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.42.931.926 I srv update: - prompt 0x56ead176d1b0: 133 tokens, checkpoints: 0, 20.784 MiB
0.42.931.927 I srv update: - prompt 0x56ead4a3fc50: 145 tokens, checkpoints: 0, 22.659 MiB
0.42.931.928 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
0.42.931.928 I srv update: - prompt 0x56ead47d1d90: 108 tokens, checkpoints: 0, 16.877 MiB
0.42.931.929 I srv update: - prompt 0x56ead47d1960: 96 tokens, checkpoints: 0, 15.002 MiB
0.42.931.929 I srv update: - prompt 0x56ead4964930: 110 tokens, checkpoints: 0, 17.190 MiB
0.42.931.930 I srv get_availabl: prompt cache update took 21.56 ms
0.42.931.959 I slot launch_slot_: id 3 | task 754 | processing task, is_child = 0
0.42.931.960 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.42.931.961 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.42.931.968 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.42.931.968 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.42.931.973 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.42.931.973 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.43.758.624 I slot print_timing: id 3 | task 754 | prompt eval time = 121.91 ms / 9 tokens ( 13.55 ms per token, 73.83 tokens per second)
0.43.758.627 I slot print_timing: id 3 | task 754 | eval time = 704.73 ms / 18 tokens ( 39.15 ms per token, 25.54 tokens per second)
0.43.758.628 I slot print_timing: id 3 | task 754 | total time = 826.63 ms / 27 tokens
0.43.758.629 I slot print_timing: id 3 | task 754 | graphs reused = 0
0.43.758.644 I slot release: id 3 | task 754 | stop processing: n_tokens = 74, truncated = 0
0.43.758.650 I srv update_slots: all slots are idle
0.43.760.877 I srv operator(): Chat format: peg-native
0.43.760.976 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.608
0.43.760.994 I slot launch_slot_: id 3 | task 773 | processing task, is_child = 0
0.43.760.994 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.43.760.995 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.43.761.000 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.43.761.001 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.43.761.005 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.43.761.005 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.44.926.323 I slot print_timing: id 3 | task 773 | prompt eval time = 212.34 ms / 18 tokens ( 11.80 ms per token, 84.77 tokens per second)
0.44.926.326 I slot print_timing: id 3 | task 773 | eval time = 952.96 ms / 24 tokens ( 39.71 ms per token, 25.18 tokens per second)
0.44.926.326 I slot print_timing: id 3 | task 773 | total time = 1165.30 ms / 42 tokens
0.44.926.327 I slot print_timing: id 3 | task 773 | graphs reused = 0
0.44.926.344 I slot release: id 3 | task 773 | stop processing: n_tokens = 86, truncated = 0
0.44.926.350 I srv update_slots: all slots are idle
0.44.929.019 I srv operator(): Chat format: peg-native
0.44.929.121 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.726 (> 0.100 thold), f_keep = 0.523
0.44.929.139 I slot launch_slot_: id 3 | task 798 | processing task, is_child = 0
0.44.929.140 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.44.929.140 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.44.929.146 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.44.929.146 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.44.929.151 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.44.929.151 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.45.929.020 I slot print_timing: id 3 | task 798 | prompt eval time = 211.99 ms / 17 tokens ( 12.47 ms per token, 80.19 tokens per second)
0.45.929.023 I slot print_timing: id 3 | task 798 | eval time = 787.86 ms / 20 tokens ( 39.39 ms per token, 25.39 tokens per second)
0.45.929.024 I slot print_timing: id 3 | task 798 | total time = 999.86 ms / 37 tokens
0.45.929.025 I slot print_timing: id 3 | task 798 | graphs reused = 0
0.45.929.043 I slot release: id 3 | task 798 | stop processing: n_tokens = 81, truncated = 0
0.45.929.049 I srv update_slots: all slots are idle
0.45.931.302 I srv operator(): Chat format: peg-native
0.45.931.398 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.849 (> 0.100 thold), f_keep = 0.556
0.45.931.418 I slot launch_slot_: id 3 | task 819 | processing task, is_child = 0
0.45.931.421 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.45.931.422 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.45.931.429 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.45.931.429 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.45.931.436 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.45.931.436 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.46.460.819 I slot print_timing: id 3 | task 819 | prompt eval time = 197.71 ms / 8 tokens ( 24.71 ms per token, 40.46 tokens per second)
0.46.460.823 I slot print_timing: id 3 | task 819 | eval time = 331.66 ms / 9 tokens ( 36.85 ms per token, 27.14 tokens per second)
0.46.460.823 I slot print_timing: id 3 | task 819 | total time = 529.37 ms / 17 tokens
0.46.460.824 I slot print_timing: id 3 | task 819 | graphs reused = 0
0.46.460.836 I slot release: id 3 | task 819 | stop processing: n_tokens = 61, truncated = 0
0.46.460.842 I srv update_slots: all slots are idle
0.46.463.262 I srv operator(): Chat format: peg-native
0.46.463.378 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.738
0.46.463.395 I slot launch_slot_: id 3 | task 829 | processing task, is_child = 0
0.46.463.396 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.46.463.397 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.46.463.404 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.46.463.404 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.46.463.410 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.46.463.410 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.47.206.938 I slot print_timing: id 3 | task 829 | prompt eval time = 121.79 ms / 12 tokens ( 10.15 ms per token, 98.53 tokens per second)
0.47.206.941 I slot print_timing: id 3 | task 829 | eval time = 621.72 ms / 16 tokens ( 38.86 ms per token, 25.73 tokens per second)
0.47.206.941 I slot print_timing: id 3 | task 829 | total time = 743.51 ms / 28 tokens
0.47.206.942 I slot print_timing: id 3 | task 829 | graphs reused = 0
0.47.206.956 I slot release: id 3 | task 829 | stop processing: n_tokens = 72, truncated = 0
0.47.206.962 I srv update_slots: all slots are idle
0.47.209.224 I srv operator(): Chat format: peg-native
0.47.209.321 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.763 (> 0.100 thold), f_keep = 0.625
0.47.209.342 I slot launch_slot_: id 3 | task 846 | processing task, is_child = 0
0.47.209.342 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.47.209.343 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.47.209.353 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.47.209.353 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.47.209.360 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.47.209.360 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.48.120.259 I slot print_timing: id 3 | task 846 | prompt eval time = 122.53 ms / 14 tokens ( 8.75 ms per token, 114.25 tokens per second)
0.48.120.262 I slot print_timing: id 3 | task 846 | eval time = 788.35 ms / 20 tokens ( 39.42 ms per token, 25.37 tokens per second)
0.48.120.262 I slot print_timing: id 3 | task 846 | total time = 910.88 ms / 34 tokens
0.48.120.263 I slot print_timing: id 3 | task 846 | graphs reused = 0
0.48.120.278 I slot release: id 3 | task 846 | stop processing: n_tokens = 78, truncated = 0
0.48.120.283 I srv update_slots: all slots are idle
0.48.122.630 I srv operator(): Chat format: peg-native
0.48.122.737 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.643 (> 0.100 thold), f_keep = 0.577
0.48.122.754 I slot launch_slot_: id 3 | task 867 | processing task, is_child = 0
0.48.122.755 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.48.122.756 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.48.122.763 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.48.122.763 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.48.122.769 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.48.122.770 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.52.112.070 I slot print_timing: id 3 | task 867 | prompt eval time = 250.21 ms / 25 tokens ( 10.01 ms per token, 99.91 tokens per second)
0.52.112.073 I slot print_timing: id 3 | task 867 | eval time = 3739.07 ms / 91 tokens ( 41.09 ms per token, 24.34 tokens per second)
0.52.112.074 I slot print_timing: id 3 | task 867 | total time = 3989.28 ms / 116 tokens
0.52.112.075 I slot print_timing: id 3 | task 867 | graphs reused = 0
0.52.112.096 I slot release: id 3 | task 867 | stop processing: n_tokens = 160, truncated = 0
0.52.112.104 I srv update_slots: all slots are idle
0.52.114.467 I srv operator(): Chat format: peg-native
0.52.114.574 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.754 (> 0.100 thold), f_keep = 0.287
0.52.114.577 I srv get_availabl: updating prompt cache
0.52.114.596 W srv prompt_save: - saving prompt with length 160, total state size = 25.003 MiB (draft: 0.000 MiB)
0.52.135.697 I srv load: - looking for better prompt, base f_keep = 0.287, sim = 0.754
0.52.135.706 I srv update: - cache state: 7 prompts, 133.766 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.52.135.707 I srv update: - prompt 0x56ead176d1b0: 133 tokens, checkpoints: 0, 20.784 MiB
0.52.135.708 I srv update: - prompt 0x56ead4a3fc50: 145 tokens, checkpoints: 0, 22.659 MiB
0.52.135.708 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
0.52.135.709 I srv update: - prompt 0x56ead47d1d90: 108 tokens, checkpoints: 0, 16.877 MiB
0.52.135.709 I srv update: - prompt 0x56ead47d1960: 96 tokens, checkpoints: 0, 15.002 MiB
0.52.135.710 I srv update: - prompt 0x56ead4964930: 110 tokens, checkpoints: 0, 17.190 MiB
0.52.135.710 I srv update: - prompt 0x56ead4a38e10: 160 tokens, checkpoints: 0, 25.003 MiB
0.52.135.711 I srv get_availabl: prompt cache update took 21.13 ms
0.52.135.747 I slot launch_slot_: id 3 | task 959 | processing task, is_child = 0
0.52.135.748 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.52.135.749 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.52.135.756 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.52.135.756 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.52.135.760 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.52.135.760 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.54.086.838 I slot print_timing: id 3 | task 959 | prompt eval time = 123.14 ms / 15 tokens ( 8.21 ms per token, 121.81 tokens per second)
0.54.086.842 I slot print_timing: id 3 | task 959 | eval time = 1827.92 ms / 45 tokens ( 40.62 ms per token, 24.62 tokens per second)
0.54.086.843 I slot print_timing: id 3 | task 959 | total time = 1951.06 ms / 60 tokens
0.54.086.843 I slot print_timing: id 3 | task 959 | graphs reused = 0
0.54.086.860 I slot release: id 3 | task 959 | stop processing: n_tokens = 105, truncated = 0
0.54.086.868 I srv update_slots: all slots are idle
0.54.089.123 I srv operator(): Chat format: peg-native
0.54.089.233 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.804 (> 0.100 thold), f_keep = 0.429
0.54.089.235 I srv get_availabl: updating prompt cache
0.54.089.251 W srv prompt_save: - saving prompt with length 105, total state size = 16.408 MiB (draft: 0.000 MiB)
0.54.104.036 I srv load: - looking for better prompt, base f_keep = 0.429, sim = 0.804
0.54.104.044 I srv update: - cache state: 8 prompts, 150.175 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.54.104.045 I srv update: - prompt 0x56ead176d1b0: 133 tokens, checkpoints: 0, 20.784 MiB
0.54.104.046 I srv update: - prompt 0x56ead4a3fc50: 145 tokens, checkpoints: 0, 22.659 MiB
0.54.104.046 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
0.54.104.047 I srv update: - prompt 0x56ead47d1d90: 108 tokens, checkpoints: 0, 16.877 MiB
0.54.104.047 I srv update: - prompt 0x56ead47d1960: 96 tokens, checkpoints: 0, 15.002 MiB
0.54.104.048 I srv update: - prompt 0x56ead4964930: 110 tokens, checkpoints: 0, 17.190 MiB
0.54.104.048 I srv update: - prompt 0x56ead4a38e10: 160 tokens, checkpoints: 0, 25.003 MiB
0.54.104.049 I srv update: - prompt 0x56ead4951cd0: 105 tokens, checkpoints: 0, 16.408 MiB
0.54.104.050 I srv get_availabl: prompt cache update took 14.81 ms
0.54.104.093 I slot launch_slot_: id 3 | task 1005 | processing task, is_child = 0
0.54.104.093 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.54.104.094 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.54.104.103 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.54.104.103 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.54.104.108 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.54.104.108 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.57.392.108 I slot print_timing: id 3 | task 1005 | prompt eval time = 122.09 ms / 11 tokens ( 11.10 ms per token, 90.10 tokens per second)
0.57.392.112 I slot print_timing: id 3 | task 1005 | eval time = 3165.88 ms / 77 tokens ( 41.12 ms per token, 24.32 tokens per second)
0.57.392.113 I slot print_timing: id 3 | task 1005 | total time = 3287.98 ms / 88 tokens
0.57.392.114 I slot print_timing: id 3 | task 1005 | graphs reused = 0
0.57.392.134 I slot release: id 3 | task 1005 | stop processing: n_tokens = 132, truncated = 0
0.57.392.146 I srv update_slots: all slots are idle
0.57.394.445 I srv operator(): Chat format: peg-native
0.57.394.560 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.738 (> 0.100 thold), f_keep = 0.341
0.57.394.562 I srv get_availabl: updating prompt cache
0.57.394.585 W srv prompt_save: - saving prompt with length 132, total state size = 20.627 MiB (draft: 0.000 MiB)
0.57.412.492 I srv load: - looking for better prompt, base f_keep = 0.341, sim = 0.738
0.57.412.499 I srv load: - found better prompt with f_keep = 0.346, sim = 0.754
0.57.422.575 I srv update: - cache state: 8 prompts, 150.018 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
0.57.422.579 I srv update: - prompt 0x56ead4a3fc50: 145 tokens, checkpoints: 0, 22.659 MiB
0.57.422.580 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
0.57.422.581 I srv update: - prompt 0x56ead47d1d90: 108 tokens, checkpoints: 0, 16.877 MiB
0.57.422.581 I srv update: - prompt 0x56ead47d1960: 96 tokens, checkpoints: 0, 15.002 MiB
0.57.422.582 I srv update: - prompt 0x56ead4964930: 110 tokens, checkpoints: 0, 17.190 MiB
0.57.422.582 I srv update: - prompt 0x56ead4a38e10: 160 tokens, checkpoints: 0, 25.003 MiB
0.57.422.583 I srv update: - prompt 0x56ead4951cd0: 105 tokens, checkpoints: 0, 16.408 MiB
0.57.422.583 I srv update: - prompt 0x56ead49538b0: 132 tokens, checkpoints: 0, 20.627 MiB
0.57.422.584 I srv get_availabl: prompt cache update took 28.02 ms
0.57.422.614 I slot launch_slot_: id 3 | task 1083 | processing task, is_child = 0
0.57.422.615 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.57.422.616 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.57.422.623 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.57.422.624 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.57.422.628 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.57.422.628 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.01.506.741 I slot print_timing: id 3 | task 1083 | prompt eval time = 123.19 ms / 15 tokens ( 8.21 ms per token, 121.76 tokens per second)
1.01.506.744 I slot print_timing: id 3 | task 1083 | eval time = 3960.90 ms / 96 tokens ( 41.26 ms per token, 24.24 tokens per second)
1.01.506.745 I slot print_timing: id 3 | task 1083 | total time = 4084.09 ms / 111 tokens
1.01.506.746 I slot print_timing: id 3 | task 1083 | graphs reused = 0
1.01.506.765 I slot release: id 3 | task 1083 | stop processing: n_tokens = 156, truncated = 0
1.01.506.772 I srv update_slots: all slots are idle
1.01.509.343 I srv operator(): Chat format: peg-native
1.01.509.451 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.754 (> 0.100 thold), f_keep = 0.295
1.01.509.453 I srv get_availabl: updating prompt cache
1.01.509.471 W srv prompt_save: - saving prompt with length 156, total state size = 24.378 MiB (draft: 0.000 MiB)
1.01.529.781 I srv load: - looking for better prompt, base f_keep = 0.295, sim = 0.754
1.01.529.790 I srv load: - found better prompt with f_keep = 0.331, sim = 0.787
1.01.540.696 I srv update: - cache state: 8 prompts, 151.737 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
1.01.540.699 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
1.01.540.699 I srv update: - prompt 0x56ead47d1d90: 108 tokens, checkpoints: 0, 16.877 MiB
1.01.540.700 I srv update: - prompt 0x56ead47d1960: 96 tokens, checkpoints: 0, 15.002 MiB
1.01.540.701 I srv update: - prompt 0x56ead4964930: 110 tokens, checkpoints: 0, 17.190 MiB
1.01.540.701 I srv update: - prompt 0x56ead4a38e10: 160 tokens, checkpoints: 0, 25.003 MiB
1.01.540.702 I srv update: - prompt 0x56ead4951cd0: 105 tokens, checkpoints: 0, 16.408 MiB
1.01.540.702 I srv update: - prompt 0x56ead49538b0: 132 tokens, checkpoints: 0, 20.627 MiB
1.01.540.703 I srv update: - prompt 0x56ead495dee0: 156 tokens, checkpoints: 0, 24.378 MiB
1.01.540.704 I srv get_availabl: prompt cache update took 31.25 ms
1.01.540.733 I slot launch_slot_: id 3 | task 1180 | processing task, is_child = 0
1.01.540.734 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.01.540.735 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.01.540.744 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.01.540.744 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.01.540.748 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.01.540.748 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.03.959.597 I slot print_timing: id 3 | task 1180 | prompt eval time = 122.28 ms / 13 tokens ( 9.41 ms per token, 106.32 tokens per second)
1.03.959.600 I slot print_timing: id 3 | task 1180 | eval time = 2296.55 ms / 57 tokens ( 40.29 ms per token, 24.82 tokens per second)
1.03.959.601 I slot print_timing: id 3 | task 1180 | total time = 2418.83 ms / 70 tokens
1.03.959.602 I slot print_timing: id 3 | task 1180 | graphs reused = 0
1.03.959.618 I slot release: id 3 | task 1180 | stop processing: n_tokens = 117, truncated = 0
1.03.959.626 I srv update_slots: all slots are idle
1.03.961.991 I srv operator(): Chat format: peg-native
1.03.962.106 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.818 (> 0.100 thold), f_keep = 0.385
1.03.962.107 I srv get_availabl: updating prompt cache
1.03.962.121 W srv prompt_save: - saving prompt with length 117, total state size = 18.284 MiB (draft: 0.000 MiB)
1.03.978.070 I srv load: - looking for better prompt, base f_keep = 0.385, sim = 0.818
1.03.978.081 I srv update: - cache state: 9 prompts, 170.021 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
1.03.978.082 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
1.03.978.082 I srv update: - prompt 0x56ead47d1d90: 108 tokens, checkpoints: 0, 16.877 MiB
1.03.978.083 I srv update: - prompt 0x56ead47d1960: 96 tokens, checkpoints: 0, 15.002 MiB
1.03.978.083 I srv update: - prompt 0x56ead4964930: 110 tokens, checkpoints: 0, 17.190 MiB
1.03.978.084 I srv update: - prompt 0x56ead4a38e10: 160 tokens, checkpoints: 0, 25.003 MiB
1.03.978.084 I srv update: - prompt 0x56ead4951cd0: 105 tokens, checkpoints: 0, 16.408 MiB
1.03.978.085 I srv update: - prompt 0x56ead49538b0: 132 tokens, checkpoints: 0, 20.627 MiB
1.03.978.085 I srv update: - prompt 0x56ead495dee0: 156 tokens, checkpoints: 0, 24.378 MiB
1.03.978.086 I srv update: - prompt 0x56ead18754e0: 117 tokens, checkpoints: 0, 18.284 MiB
1.03.978.088 I srv get_availabl: prompt cache update took 15.98 ms
1.03.978.120 I slot launch_slot_: id 3 | task 1238 | processing task, is_child = 0
1.03.978.121 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.03.978.122 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.03.978.130 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.03.978.130 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.03.978.135 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.03.978.135 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.05.453.691 I slot print_timing: id 3 | task 1238 | prompt eval time = 122.00 ms / 10 tokens ( 12.20 ms per token, 81.97 tokens per second)
1.05.453.694 I slot print_timing: id 3 | task 1238 | eval time = 1353.54 ms / 34 tokens ( 39.81 ms per token, 25.12 tokens per second)
1.05.453.694 I slot print_timing: id 3 | task 1238 | total time = 1475.54 ms / 44 tokens
1.05.453.695 I slot print_timing: id 3 | task 1238 | graphs reused = 0
1.05.453.711 I slot release: id 3 | task 1238 | stop processing: n_tokens = 88, truncated = 0
1.05.453.716 I srv update_slots: all slots are idle
1.05.456.113 I srv operator(): Chat format: peg-native
1.05.456.216 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.865 (> 0.100 thold), f_keep = 0.511
1.05.456.234 I slot launch_slot_: id 3 | task 1273 | processing task, is_child = 0
1.05.456.235 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.05.456.238 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.05.456.243 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.05.456.244 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.05.456.248 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.05.456.248 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.06.413.711 I slot print_timing: id 3 | task 1273 | prompt eval time = 178.05 ms / 7 tokens ( 25.44 ms per token, 39.31 tokens per second)
1.06.413.714 I slot print_timing: id 3 | task 1273 | eval time = 779.40 ms / 20 tokens ( 38.97 ms per token, 25.66 tokens per second)
1.06.413.715 I slot print_timing: id 3 | task 1273 | total time = 957.45 ms / 27 tokens
1.06.413.716 I slot print_timing: id 3 | task 1273 | graphs reused = 0
1.06.413.731 I slot release: id 3 | task 1273 | stop processing: n_tokens = 71, truncated = 0
1.06.413.738 I srv update_slots: all slots are idle
1.06.415.959 I srv operator(): Chat format: peg-native
1.06.416.064 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.634
1.06.416.079 I slot launch_slot_: id 3 | task 1294 | processing task, is_child = 0
1.06.416.080 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.06.416.083 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.06.416.089 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.06.416.090 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.06.416.094 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.06.416.094 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.09.706.314 I slot print_timing: id 3 | task 1294 | prompt eval time = 122.69 ms / 13 tokens ( 9.44 ms per token, 105.96 tokens per second)
1.09.706.319 I slot print_timing: id 3 | task 1294 | eval time = 3167.52 ms / 78 tokens ( 40.61 ms per token, 24.62 tokens per second)
1.09.706.319 I slot print_timing: id 3 | task 1294 | total time = 3290.21 ms / 91 tokens
1.09.706.320 I slot print_timing: id 3 | task 1294 | graphs reused = 0
1.09.706.343 I slot release: id 3 | task 1294 | stop processing: n_tokens = 135, truncated = 0
1.09.706.351 I srv update_slots: all slots are idle
1.09.708.660 I srv operator(): Chat format: peg-native
1.09.708.780 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.804 (> 0.100 thold), f_keep = 0.333
1.09.708.783 I srv get_availabl: updating prompt cache
1.09.708.806 W srv prompt_save: - saving prompt with length 135, total state size = 21.096 MiB (draft: 0.000 MiB)
1.09.726.831 I srv load: - looking for better prompt, base f_keep = 0.333, sim = 0.804
1.09.726.838 I srv load: - found better prompt with f_keep = 0.426, sim = 0.821
1.09.733.949 I srv update: - cache state: 9 prompts, 174.240 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
1.09.733.953 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
1.09.733.954 I srv update: - prompt 0x56ead47d1960: 96 tokens, checkpoints: 0, 15.002 MiB
1.09.733.954 I srv update: - prompt 0x56ead4964930: 110 tokens, checkpoints: 0, 17.190 MiB
1.09.733.955 I srv update: - prompt 0x56ead4a38e10: 160 tokens, checkpoints: 0, 25.003 MiB
1.09.733.955 I srv update: - prompt 0x56ead4951cd0: 105 tokens, checkpoints: 0, 16.408 MiB
1.09.733.956 I srv update: - prompt 0x56ead49538b0: 132 tokens, checkpoints: 0, 20.627 MiB
1.09.733.956 I srv update: - prompt 0x56ead495dee0: 156 tokens, checkpoints: 0, 24.378 MiB
1.09.733.957 I srv update: - prompt 0x56ead18754e0: 117 tokens, checkpoints: 0, 18.284 MiB
1.09.733.958 I srv update: - prompt 0x56ead4954f10: 135 tokens, checkpoints: 0, 21.096 MiB
1.09.733.959 I srv get_availabl: prompt cache update took 25.17 ms
1.09.733.987 I slot launch_slot_: id 3 | task 1373 | processing task, is_child = 0
1.09.733.988 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.09.733.989 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.09.733.997 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.09.733.998 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.09.734.002 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.09.734.002 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.13.563.379 I slot print_timing: id 3 | task 1373 | prompt eval time = 122.28 ms / 10 tokens ( 12.23 ms per token, 81.78 tokens per second)
1.13.563.383 I slot print_timing: id 3 | task 1373 | eval time = 3707.08 ms / 91 tokens ( 40.74 ms per token, 24.55 tokens per second)
1.13.563.384 I slot print_timing: id 3 | task 1373 | total time = 3829.36 ms / 101 tokens
1.13.563.385 I slot print_timing: id 3 | task 1373 | graphs reused = 0
1.13.563.402 I slot release: id 3 | task 1373 | stop processing: n_tokens = 146, truncated = 0
1.13.563.412 I srv update_slots: all slots are idle
1.13.565.808 I srv operator(): Chat format: peg-native
1.13.565.913 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.836 (> 0.100 thold), f_keep = 0.315
1.13.565.914 I srv get_availabl: updating prompt cache
1.13.565.936 W srv prompt_save: - saving prompt with length 146, total state size = 22.815 MiB (draft: 0.000 MiB)
1.13.577.300 I srv load: - looking for better prompt, base f_keep = 0.315, sim = 0.836
1.13.577.309 I srv update: - cache state: 10 prompts, 197.055 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
1.13.577.310 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
1.13.577.310 I srv update: - prompt 0x56ead47d1960: 96 tokens, checkpoints: 0, 15.002 MiB
1.13.577.311 I srv update: - prompt 0x56ead4964930: 110 tokens, checkpoints: 0, 17.190 MiB
1.13.577.313 I srv update: - prompt 0x56ead4a38e10: 160 tokens, checkpoints: 0, 25.003 MiB
1.13.577.314 I srv update: - prompt 0x56ead4951cd0: 105 tokens, checkpoints: 0, 16.408 MiB
1.13.577.314 I srv update: - prompt 0x56ead49538b0: 132 tokens, checkpoints: 0, 20.627 MiB
1.13.577.314 I srv update: - prompt 0x56ead495dee0: 156 tokens, checkpoints: 0, 24.378 MiB
1.13.577.315 I srv update: - prompt 0x56ead18754e0: 117 tokens, checkpoints: 0, 18.284 MiB
1.13.577.315 I srv update: - prompt 0x56ead4954f10: 135 tokens, checkpoints: 0, 21.096 MiB
1.13.577.316 I srv update: - prompt 0x56ead491f300: 146 tokens, checkpoints: 0, 22.815 MiB
1.13.577.317 I srv get_availabl: prompt cache update took 11.40 ms
1.13.577.350 I slot launch_slot_: id 3 | task 1465 | processing task, is_child = 0
1.13.577.350 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.13.577.351 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.13.577.359 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.13.577.359 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.13.577.363 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.13.577.364 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.16.168.652 I slot print_timing: id 3 | task 1465 | prompt eval time = 121.83 ms / 9 tokens ( 13.54 ms per token, 73.87 tokens per second)
1.16.168.655 I slot print_timing: id 3 | task 1465 | eval time = 2469.43 ms / 61 tokens ( 40.48 ms per token, 24.70 tokens per second)
1.16.168.655 I slot print_timing: id 3 | task 1465 | total time = 2591.27 ms / 70 tokens
1.16.168.656 I slot print_timing: id 3 | task 1465 | graphs reused = 0
1.16.168.675 I slot release: id 3 | task 1465 | stop processing: n_tokens = 115, truncated = 0
1.16.168.682 I srv update_slots: all slots are idle
1.16.170.977 I srv operator(): Chat format: peg-native
1.16.171.088 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.391
1.16.171.090 I srv get_availabl: updating prompt cache
1.16.171.107 W srv prompt_save: - saving prompt with length 115, total state size = 17.971 MiB (draft: 0.000 MiB)
1.16.187.042 I srv load: - looking for better prompt, base f_keep = 0.391, sim = 0.776
1.16.187.047 I srv load: - found better prompt with f_keep = 0.479, sim = 0.793
1.16.193.736 I srv update: - cache state: 10 prompts, 200.024 MiB (limits: 8192.000 MiB, 8192 tokens, 52422 est)
1.16.193.738 I srv update: - prompt 0x56ead18750a0: 104 tokens, checkpoints: 0, 16.252 MiB
1.16.193.739 I srv update: - prompt 0x56ead4964930: 110 tokens, checkpoints: 0, 17.190 MiB
1.16.193.740 I srv update: - prompt 0x56ead4a38e10: 160 tokens, checkpoints: 0, 25.003 MiB
1.16.193.740 I srv update: - prompt 0x56ead4951cd0: 105 tokens, checkpoints: 0, 16.408 MiB
1.16.193.741 I srv update: - prompt 0x56ead49538b0: 132 tokens, checkpoints: 0, 20.627 MiB
1.16.193.741 I srv update: - prompt 0x56ead495dee0: 156 tokens, checkpoints: 0, 24.378 MiB
1.16.193.741 I srv update: - prompt 0x56ead18754e0: 117 tokens, checkpoints: 0, 18.284 MiB
1.16.193.742 I srv update: - prompt 0x56ead4954f10: 135 tokens, checkpoints: 0, 21.096 MiB
1.16.193.742 I srv update: - prompt 0x56ead491f300: 146 tokens, checkpoints: 0, 22.815 MiB
1.16.193.743 I srv update: - prompt 0x56ead4a46b30: 115 tokens, checkpoints: 0, 17.971 MiB
1.16.193.744 I srv get_availabl: prompt cache update took 22.65 ms
1.16.193.774 I slot launch_slot_: id 3 | task 1527 | processing task, is_child = 0
1.16.193.774 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.16.193.775 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.16.193.783 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.16.193.784 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.16.193.788 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.16.193.788 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.17.139.409 I slot print_timing: id 3 | task 1527 | prompt eval time = 122.39 ms / 12 tokens ( 10.20 ms per token, 98.05 tokens per second)
1.17.139.412 I slot print_timing: id 3 | task 1527 | eval time = 823.22 ms / 21 tokens ( 39.20 ms per token, 25.51 tokens per second)
1.17.139.413 I slot print_timing: id 3 | task 1527 | total time = 945.60 ms / 33 tokens
1.17.139.413 I slot print_timing: id 3 | task 1527 | graphs reused = 0
1.17.139.432 I slot release: id 3 | task 1527 | stop processing: n_tokens = 78, truncated = 0
1.17.139.441 I srv update_slots: all slots are idle
1.17.141.859 I srv operator(): Chat format: peg-native
1.17.141.978 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.577
1.17.141.995 I slot launch_slot_: id 3 | task 1549 | processing task, is_child = 0
1.17.141.995 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.17.141.996 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.17.142.002 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.17.142.002 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.17.142.007 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.17.142.007 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.20.271.650 I slot print_timing: id 3 | task 1549 | prompt eval time = 121.43 ms / 9 tokens ( 13.49 ms per token, 74.12 tokens per second)
1.20.271.653 I slot print_timing: id 3 | task 1549 | eval time = 3008.20 ms / 74 tokens ( 40.65 ms per token, 24.60 tokens per second)
1.20.271.654 I slot print_timing: id 3 | task 1549 | total time = 3129.63 ms / 83 tokens
1.20.271.655 I slot print_timing: id 3 | task 1549 | graphs reused = 0
1.20.271.671 I slot release: id 3 | task 1549 | stop processing: n_tokens = 127, truncated = 0
1.20.271.679 I srv update_slots: all slots are idle
1.20.272.153 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue View Git Blame Copy Permalink