qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results_hard/qwen3next-80b-a3b.server.log

955 lines
94 KiB
Plaintext
Raw Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.066.404 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.066.407 I device_info:
0.00.066.426 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.066.430 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.066.437 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.066.491 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.066.495 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.066.532 I srv init: running without SSL
0.00.066.594 I srv init: using 23 threads for HTTP server
0.00.066.924 I srv start: binding port with default address family
0.00.068.142 I srv llama_server: loading model
0.00.068.150 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/Qwen3-Next-80B-A3B-Instruct-Q4_K_S.gguf'
0.00.068.204 I common_init_result: fitting params to device memory ...
0.00.068.205 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.940.279 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.28.229.930 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.28.424.733 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.28.633.664 I srv load_model: initializing slots, n_slots = 4
0.28.704.401 W srv load_model: speculative decoding will use checkpoints
0.28.704.416 W common_speculative_init: no implementations specified for speculative decoding
0.28.704.418 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384
0.28.704.423 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384
0.28.704.424 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384
0.28.704.424 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384
0.28.704.479 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.28.704.479 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.28.704.480 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.28.704.480 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.28.704.500 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.28.709.839 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
'
0.28.713.110 I srv init: init: chat template, thinking = 0
0.28.713.146 I srv llama_server: model loaded
0.28.713.149 I srv llama_server: server is listening on http://127.0.0.1:8090
0.28.713.154 I srv update_slots: all slots are idle
0.29.971.423 I srv operator(): Chat format: peg-native
0.29.971.660 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.29.971.662 I srv get_availabl: updating prompt cache
0.29.971.667 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.29.971.671 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est)
0.29.971.672 I srv get_availabl: prompt cache update took 0.01 ms
0.29.971.708 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.29.971.709 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.971.710 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.971.725 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.971.725 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.971.734 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.971.735 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.30.759.139 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 70, pos_max = 70, n_tokens = 71, size = 75.376 MiB)
0.33.970.469 I slot print_timing: id 3 | task 0 | n_decoded = 121, tg = 40.01 t/s, tg_3s = 40.01 t/s
0.36.984.665 I slot print_timing: id 3 | task 0 | n_decoded = 241, tg = 39.91 t/s, tg_3s = 39.81 t/s
0.39.985.784 I slot print_timing: id 3 | task 0 | n_decoded = 361, tg = 39.94 t/s, tg_3s = 39.99 t/s
0.43.002.742 I slot print_timing: id 3 | task 0 | n_decoded = 481, tg = 39.90 t/s, tg_3s = 39.78 t/s
0.46.003.859 I slot print_timing: id 3 | task 0 | n_decoded = 601, tg = 39.91 t/s, tg_3s = 39.99 t/s
0.49.021.900 I slot print_timing: id 3 | task 0 | n_decoded = 721, tg = 39.89 t/s, tg_3s = 39.76 t/s
0.52.033.912 I slot print_timing: id 3 | task 0 | n_decoded = 841, tg = 39.88 t/s, tg_3s = 39.84 t/s
0.54.171.992 I slot print_timing: id 3 | task 0 | prompt eval time = 974.82 ms / 98 tokens ( 9.95 ms per token, 100.53 tokens per second)
0.54.171.996 I slot print_timing: id 3 | task 0 | eval time = 23225.42 ms / 926 tokens ( 25.08 ms per token, 39.87 tokens per second)
0.54.171.996 I slot print_timing: id 3 | task 0 | total time = 24200.23 ms / 1024 tokens
0.54.172.001 I slot print_timing: id 3 | task 0 | graphs reused = 922
0.54.172.044 I slot release: id 3 | task 0 | stop processing: n_tokens = 1023, truncated = 0
0.54.172.061 I srv update_slots: all slots are idle
0.54.177.296 I srv operator(): Chat format: peg-native
0.54.177.504 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.804 (> 0.100 thold), f_keep = 0.072
0.54.177.506 I srv get_availabl: updating prompt cache
0.54.177.555 W srv prompt_save: - saving prompt with length 1023, total state size = 99.364 MiB (draft: 0.000 MiB)
0.54.334.635 I srv load: - looking for better prompt, base f_keep = 0.072, sim = 0.804
0.54.334.641 I srv update: - cache state: 1 prompts, 174.740 MiB (limits: 8192.000 MiB, 16384 tokens, 47959 est)
0.54.334.642 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
0.54.334.643 I srv get_availabl: prompt cache update took 157.14 ms
0.54.334.722 I slot launch_slot_: id 3 | task 929 | processing task, is_child = 0
0.54.334.724 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.54.334.725 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.54.334.741 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.54.334.741 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.54.334.761 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.54.334.762 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.54.334.790 I slot update_slots: id 3 | task 929 | Checking checkpoint with [70, 70] against 74...
0.54.353.833 W slot update_slots: id 3 | task 929 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
0.57.548.823 I slot print_timing: id 3 | task 929 | n_decoded = 121, tg = 40.09 t/s, tg_3s = 40.09 t/s
1.00.571.986 I slot print_timing: id 3 | task 929 | n_decoded = 241, tg = 39.89 t/s, tg_3s = 39.69 t/s
1.03.578.614 I slot print_timing: id 3 | task 929 | n_decoded = 361, tg = 39.90 t/s, tg_3s = 39.91 t/s
1.06.585.980 I slot print_timing: id 3 | task 929 | n_decoded = 480, tg = 39.82 t/s, tg_3s = 39.57 t/s
1.07.492.066 I slot print_timing: id 3 | task 929 | prompt eval time = 195.87 ms / 21 tokens ( 9.33 ms per token, 107.21 tokens per second)
1.07.492.070 I slot print_timing: id 3 | task 929 | eval time = 12961.40 ms / 516 tokens ( 25.12 ms per token, 39.81 tokens per second)
1.07.492.071 I slot print_timing: id 3 | task 929 | total time = 13157.27 ms / 537 tokens
1.07.492.072 I slot print_timing: id 3 | task 929 | graphs reused = 1434
1.07.492.100 I slot release: id 3 | task 929 | stop processing: n_tokens = 607, truncated = 0
1.07.492.108 I srv update_slots: all slots are idle
1.07.497.102 I srv operator(): Chat format: peg-native
1.07.497.313 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.796 (> 0.100 thold), f_keep = 0.122
1.07.497.315 I srv get_availabl: updating prompt cache
1.07.497.351 W srv prompt_save: - saving prompt with length 607, total state size = 89.610 MiB (draft: 0.000 MiB)
1.07.616.788 I srv load: - looking for better prompt, base f_keep = 0.122, sim = 0.796
1.07.616.795 I srv update: - cache state: 2 prompts, 339.726 MiB (limits: 8192.000 MiB, 16384 tokens, 39305 est)
1.07.616.796 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
1.07.616.796 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
1.07.616.798 I srv get_availabl: prompt cache update took 119.48 ms
1.07.616.874 I slot launch_slot_: id 3 | task 1447 | processing task, is_child = 0
1.07.616.875 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.07.616.876 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.07.616.889 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.07.616.890 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.07.616.899 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.07.616.899 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.07.616.916 I slot update_slots: id 3 | task 1447 | Checking checkpoint with [70, 70] against 74...
1.07.635.752 W slot update_slots: id 3 | task 1447 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
1.10.812.937 I slot print_timing: id 3 | task 1447 | n_decoded = 120, tg = 39.96 t/s, tg_3s = 39.96 t/s
1.13.818.058 I slot print_timing: id 3 | task 1447 | n_decoded = 239, tg = 39.78 t/s, tg_3s = 39.60 t/s
1.14.800.021 I slot print_timing: id 3 | task 1447 | prompt eval time = 192.96 ms / 22 tokens ( 8.77 ms per token, 114.01 tokens per second)
1.14.800.025 I slot print_timing: id 3 | task 1447 | eval time = 6990.14 ms / 278 tokens ( 25.14 ms per token, 39.77 tokens per second)
1.14.800.026 I slot print_timing: id 3 | task 1447 | total time = 7183.10 ms / 300 tokens
1.14.800.027 I slot print_timing: id 3 | task 1447 | graphs reused = 1709
1.14.800.063 I slot release: id 3 | task 1447 | stop processing: n_tokens = 370, truncated = 0
1.14.800.071 I srv update_slots: all slots are idle
1.14.804.937 I srv operator(): Chat format: peg-native
1.14.805.158 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.813 (> 0.100 thold), f_keep = 0.200
1.14.805.161 I srv get_availabl: updating prompt cache
1.14.805.199 W srv prompt_save: - saving prompt with length 370, total state size = 84.052 MiB (draft: 0.000 MiB)
1.14.906.736 I srv load: - looking for better prompt, base f_keep = 0.200, sim = 0.813
1.14.906.742 I srv update: - cache state: 3 prompts, 499.154 MiB (limits: 8192.000 MiB, 16384 tokens, 32823 est)
1.14.906.744 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
1.14.906.744 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
1.14.906.745 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
1.14.906.746 I srv get_availabl: prompt cache update took 101.58 ms
1.14.906.820 I slot launch_slot_: id 3 | task 1727 | processing task, is_child = 0
1.14.906.820 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.14.906.821 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.14.906.835 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.14.906.835 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.14.906.844 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.14.906.844 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.14.906.862 I slot update_slots: id 3 | task 1727 | Checking checkpoint with [70, 70] against 74...
1.14.925.689 W slot update_slots: id 3 | task 1727 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
1.18.070.642 I slot print_timing: id 3 | task 1727 | n_decoded = 120, tg = 39.94 t/s, tg_3s = 39.94 t/s
1.21.091.675 I slot print_timing: id 3 | task 1727 | n_decoded = 240, tg = 39.83 t/s, tg_3s = 39.72 t/s
1.24.093.749 I slot print_timing: id 3 | task 1727 | n_decoded = 360, tg = 39.88 t/s, tg_3s = 39.97 t/s
1.27.110.704 I slot print_timing: id 3 | task 1727 | n_decoded = 480, tg = 39.85 t/s, tg_3s = 39.78 t/s
1.28.512.178 I slot print_timing: id 3 | task 1727 | prompt eval time = 159.44 ms / 20 tokens ( 7.97 ms per token, 125.44 tokens per second)
1.28.512.181 I slot print_timing: id 3 | task 1727 | eval time = 13445.87 ms / 536 tokens ( 25.09 ms per token, 39.86 tokens per second)
1.28.512.182 I slot print_timing: id 3 | task 1727 | total time = 13605.31 ms / 556 tokens
1.28.512.183 I slot print_timing: id 3 | task 1727 | graphs reused = 2241
1.28.512.211 I slot release: id 3 | task 1727 | stop processing: n_tokens = 626, truncated = 0
1.28.512.220 I srv update_slots: all slots are idle
1.28.517.315 I srv operator(): Chat format: peg-native
1.28.517.523 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.712 (> 0.100 thold), f_keep = 0.118
1.28.517.525 I srv get_availabl: updating prompt cache
1.28.517.569 W srv prompt_save: - saving prompt with length 626, total state size = 90.055 MiB (draft: 0.000 MiB)
1.28.625.027 I srv load: - looking for better prompt, base f_keep = 0.118, sim = 0.712
1.28.625.035 I srv update: - cache state: 4 prompts, 664.585 MiB (limits: 8192.000 MiB, 16384 tokens, 32369 est)
1.28.625.036 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
1.28.625.037 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
1.28.625.038 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
1.28.625.039 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
1.28.625.043 I srv get_availabl: prompt cache update took 107.52 ms
1.28.625.126 I slot launch_slot_: id 3 | task 2265 | processing task, is_child = 0
1.28.625.127 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.28.625.127 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.28.625.139 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.28.625.139 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.28.625.148 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.28.625.148 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.28.625.165 I slot update_slots: id 3 | task 2265 | Checking checkpoint with [70, 70] against 74...
1.28.643.854 W slot update_slots: id 3 | task 2265 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
1.31.873.634 I slot print_timing: id 3 | task 2265 | n_decoded = 121, tg = 40.15 t/s, tg_3s = 40.15 t/s
1.34.887.859 I slot print_timing: id 3 | task 2265 | n_decoded = 241, tg = 39.98 t/s, tg_3s = 39.81 t/s
1.37.889.005 I slot print_timing: id 3 | task 2265 | n_decoded = 361, tg = 39.98 t/s, tg_3s = 39.98 t/s
1.40.904.921 I slot print_timing: id 3 | task 2265 | n_decoded = 481, tg = 39.93 t/s, tg_3s = 39.79 t/s
1.43.907.155 I slot print_timing: id 3 | task 2265 | n_decoded = 601, tg = 39.94 t/s, tg_3s = 39.97 t/s
1.46.908.107 I slot print_timing: id 3 | task 2265 | n_decoded = 720, tg = 39.89 t/s, tg_3s = 39.65 t/s
1.49.931.178 I slot print_timing: id 3 | task 2265 | n_decoded = 840, tg = 39.87 t/s, tg_3s = 39.69 t/s
1.52.946.146 I slot print_timing: id 3 | task 2265 | n_decoded = 959, tg = 39.82 t/s, tg_3s = 39.47 t/s
1.55.948.929 I slot print_timing: id 3 | task 2265 | n_decoded = 1078, tg = 39.80 t/s, tg_3s = 39.63 t/s
1.58.343.829 I slot print_timing: id 3 | task 2265 | prompt eval time = 235.00 ms / 33 tokens ( 7.12 ms per token, 140.42 tokens per second)
1.58.343.833 I slot print_timing: id 3 | task 2265 | eval time = 29483.65 ms / 1173 tokens ( 25.14 ms per token, 39.78 tokens per second)
1.58.343.833 I slot print_timing: id 3 | task 2265 | total time = 29718.66 ms / 1206 tokens
1.58.343.834 I slot print_timing: id 3 | task 2265 | graphs reused = 3408
1.58.343.868 I slot release: id 3 | task 2265 | stop processing: n_tokens = 1276, truncated = 0
1.58.343.879 I srv update_slots: all slots are idle
1.58.349.378 I srv operator(): Chat format: peg-native
1.58.349.597 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.787 (> 0.100 thold), f_keep = 0.058
1.58.349.599 I srv get_availabl: updating prompt cache
1.58.349.650 W srv prompt_save: - saving prompt with length 1276, total state size = 105.297 MiB (draft: 0.000 MiB)
1.58.465.859 I srv load: - looking for better prompt, base f_keep = 0.058, sim = 0.787
1.58.465.867 I srv update: - cache state: 5 prompts, 845.258 MiB (limits: 8192.000 MiB, 16384 tokens, 37817 est)
1.58.465.868 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
1.58.465.868 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
1.58.465.869 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
1.58.465.869 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
1.58.465.870 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
1.58.465.871 I srv get_availabl: prompt cache update took 116.27 ms
1.58.465.947 I slot launch_slot_: id 3 | task 3440 | processing task, is_child = 0
1.58.465.948 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.58.465.949 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.58.465.964 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.58.465.965 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.58.465.974 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.58.465.974 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.58.465.990 I slot update_slots: id 3 | task 3440 | Checking checkpoint with [70, 70] against 74...
1.58.485.049 W slot update_slots: id 3 | task 3440 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
2.01.664.899 I slot print_timing: id 3 | task 3440 | n_decoded = 120, tg = 39.97 t/s, tg_3s = 39.97 t/s
2.04.685.823 I slot print_timing: id 3 | task 3440 | n_decoded = 240, tg = 39.85 t/s, tg_3s = 39.72 t/s
2.07.685.892 I slot print_timing: id 3 | task 3440 | n_decoded = 360, tg = 39.90 t/s, tg_3s = 40.00 t/s
2.10.700.534 I slot print_timing: id 3 | task 3440 | n_decoded = 480, tg = 39.88 t/s, tg_3s = 39.81 t/s
2.12.501.882 I slot print_timing: id 3 | task 3440 | prompt eval time = 196.94 ms / 23 tokens ( 8.56 ms per token, 116.78 tokens per second)
2.12.501.885 I slot print_timing: id 3 | task 3440 | eval time = 13838.94 ms / 552 tokens ( 25.07 ms per token, 39.89 tokens per second)
2.12.501.886 I slot print_timing: id 3 | task 3440 | total time = 14035.89 ms / 575 tokens
2.12.501.887 I slot print_timing: id 3 | task 3440 | graphs reused = 3956
2.12.501.917 I slot release: id 3 | task 3440 | stop processing: n_tokens = 645, truncated = 0
2.12.501.926 I srv update_slots: all slots are idle
2.12.507.032 I srv operator(): Chat format: peg-native
2.12.507.248 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.843 (> 0.100 thold), f_keep = 0.116
2.12.507.251 I srv get_availabl: updating prompt cache
2.12.507.304 W srv prompt_save: - saving prompt with length 645, total state size = 90.501 MiB (draft: 0.000 MiB)
2.12.610.653 I srv load: - looking for better prompt, base f_keep = 0.116, sim = 0.843
2.12.610.661 I srv update: - cache state: 6 prompts, 1011.134 MiB (limits: 8192.000 MiB, 16384 tokens, 36838 est)
2.12.610.662 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
2.12.610.663 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
2.12.610.664 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
2.12.610.664 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
2.12.610.665 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
2.12.610.665 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
2.12.610.666 I srv get_availabl: prompt cache update took 103.42 ms
2.12.610.737 I slot launch_slot_: id 3 | task 3994 | processing task, is_child = 0
2.12.610.737 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.12.610.738 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.12.610.751 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.12.610.752 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.12.610.760 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.12.610.760 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.12.610.778 I slot update_slots: id 3 | task 3994 | Checking checkpoint with [70, 70] against 75...
2.12.629.605 W slot update_slots: id 3 | task 3994 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
2.15.777.536 I slot print_timing: id 3 | task 3994 | n_decoded = 121, tg = 40.14 t/s, tg_3s = 40.14 t/s
2.18.793.579 I slot print_timing: id 3 | task 3994 | n_decoded = 241, tg = 39.96 t/s, tg_3s = 39.79 t/s
2.21.796.745 I slot print_timing: id 3 | task 3994 | n_decoded = 361, tg = 39.96 t/s, tg_3s = 39.96 t/s
2.24.814.709 I slot print_timing: id 3 | task 3994 | n_decoded = 481, tg = 39.91 t/s, tg_3s = 39.76 t/s
2.25.515.844 I slot print_timing: id 3 | task 3994 | prompt eval time = 152.40 ms / 18 tokens ( 8.47 ms per token, 118.11 tokens per second)
2.25.515.847 I slot print_timing: id 3 | task 3994 | eval time = 12752.66 ms / 509 tokens ( 25.05 ms per token, 39.91 tokens per second)
2.25.515.848 I slot print_timing: id 3 | task 3994 | total time = 12905.06 ms / 527 tokens
2.25.515.849 I slot print_timing: id 3 | task 3994 | graphs reused = 4461
2.25.515.881 I slot release: id 3 | task 3994 | stop processing: n_tokens = 597, truncated = 0
2.25.515.888 I srv update_slots: all slots are idle
2.25.520.778 I srv operator(): Chat format: peg-native
2.25.520.985 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.824 (> 0.100 thold), f_keep = 0.126
2.25.520.987 I srv get_availabl: updating prompt cache
2.25.521.022 W srv prompt_save: - saving prompt with length 597, total state size = 89.375 MiB (draft: 0.000 MiB)
2.25.622.531 I srv load: - looking for better prompt, base f_keep = 0.126, sim = 0.824
2.25.622.542 I srv update: - cache state: 7 prompts, 1175.885 MiB (limits: 8192.000 MiB, 16384 tokens, 35836 est)
2.25.622.543 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
2.25.622.543 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
2.25.622.544 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
2.25.622.544 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
2.25.622.545 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
2.25.622.546 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
2.25.622.546 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
2.25.622.547 I srv get_availabl: prompt cache update took 101.56 ms
2.25.622.626 I slot launch_slot_: id 3 | task 4505 | processing task, is_child = 0
2.25.622.627 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.25.622.627 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.25.622.640 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.25.622.640 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.25.622.649 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.25.622.649 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.25.622.666 I slot update_slots: id 3 | task 4505 | Checking checkpoint with [70, 70] against 75...
2.25.641.436 W slot update_slots: id 3 | task 4505 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
2.28.796.303 I slot print_timing: id 3 | task 4505 | n_decoded = 121, tg = 40.13 t/s, tg_3s = 40.13 t/s
2.31.812.826 I slot print_timing: id 3 | task 4505 | n_decoded = 241, tg = 39.96 t/s, tg_3s = 39.78 t/s
2.34.818.771 I slot print_timing: id 3 | task 4505 | n_decoded = 361, tg = 39.94 t/s, tg_3s = 39.92 t/s
2.37.819.815 I slot print_timing: id 3 | task 4505 | n_decoded = 480, tg = 39.87 t/s, tg_3s = 39.65 t/s
2.40.826.956 I slot print_timing: id 3 | task 4505 | n_decoded = 600, tg = 39.88 t/s, tg_3s = 39.91 t/s
2.41.381.088 I slot print_timing: id 3 | task 4505 | prompt eval time = 158.38 ms / 20 tokens ( 7.92 ms per token, 126.28 tokens per second)
2.41.381.091 I slot print_timing: id 3 | task 4505 | eval time = 15600.03 ms / 622 tokens ( 25.08 ms per token, 39.87 tokens per second)
2.41.381.092 I slot print_timing: id 3 | task 4505 | total time = 15758.41 ms / 642 tokens
2.41.381.092 I slot print_timing: id 3 | task 4505 | graphs reused = 5079
2.41.381.167 I slot release: id 3 | task 4505 | stop processing: n_tokens = 712, truncated = 0
2.41.381.185 I srv update_slots: all slots are idle
2.41.386.221 I srv operator(): Chat format: peg-native
2.41.386.426 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.622 (> 0.100 thold), f_keep = 0.104
2.41.386.427 I srv get_availabl: updating prompt cache
2.41.386.466 W srv prompt_save: - saving prompt with length 712, total state size = 92.072 MiB (draft: 0.000 MiB)
2.41.488.395 I srv load: - looking for better prompt, base f_keep = 0.104, sim = 0.622
2.41.488.405 I srv update: - cache state: 8 prompts, 1343.333 MiB (limits: 8192.000 MiB, 16384 tokens, 35711 est)
2.41.488.406 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
2.41.488.407 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
2.41.488.407 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
2.41.488.408 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
2.41.488.409 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
2.41.488.409 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
2.41.488.410 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
2.41.488.410 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
2.41.488.411 I srv get_availabl: prompt cache update took 101.98 ms
2.41.488.483 I slot launch_slot_: id 3 | task 5129 | processing task, is_child = 0
2.41.488.485 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.41.488.485 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.41.488.498 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.41.488.498 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.41.488.519 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.41.488.519 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.41.488.548 I slot update_slots: id 3 | task 5129 | Checking checkpoint with [70, 70] against 74...
2.41.507.449 W slot update_slots: id 3 | task 5129 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
2.44.770.384 I slot print_timing: id 3 | task 5129 | n_decoded = 121, tg = 40.07 t/s, tg_3s = 40.07 t/s
2.47.792.727 I slot print_timing: id 3 | task 5129 | n_decoded = 241, tg = 39.89 t/s, tg_3s = 39.70 t/s
2.50.814.875 I slot print_timing: id 3 | task 5129 | n_decoded = 361, tg = 39.83 t/s, tg_3s = 39.71 t/s
2.53.831.493 I slot print_timing: id 3 | task 5129 | n_decoded = 480, tg = 39.73 t/s, tg_3s = 39.45 t/s
2.56.833.351 I slot print_timing: id 3 | task 5129 | n_decoded = 599, tg = 39.71 t/s, tg_3s = 39.64 t/s
2.58.791.880 I slot print_timing: id 3 | task 5129 | prompt eval time = 262.06 ms / 48 tokens ( 5.46 ms per token, 183.17 tokens per second)
2.58.791.884 I slot print_timing: id 3 | task 5129 | eval time = 17041.27 ms / 676 tokens ( 25.21 ms per token, 39.67 tokens per second)
2.58.791.884 I slot print_timing: id 3 | task 5129 | total time = 17303.33 ms / 724 tokens
2.58.791.885 I slot print_timing: id 3 | task 5129 | graphs reused = 5750
2.58.791.927 I slot release: id 3 | task 5129 | stop processing: n_tokens = 794, truncated = 0
2.58.791.936 I srv update_slots: all slots are idle
2.58.797.167 I srv operator(): Chat format: peg-native
2.58.797.377 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.540 (> 0.100 thold), f_keep = 0.094
2.58.797.379 I srv get_availabl: updating prompt cache
2.58.797.425 W srv prompt_save: - saving prompt with length 794, total state size = 93.995 MiB (draft: 0.000 MiB)
2.58.899.820 I srv load: - looking for better prompt, base f_keep = 0.094, sim = 0.540
2.58.899.830 I srv update: - cache state: 9 prompts, 1512.703 MiB (limits: 8192.000 MiB, 16384 tokens, 36012 est)
2.58.899.831 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
2.58.899.832 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
2.58.899.833 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
2.58.899.833 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
2.58.899.834 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
2.58.899.834 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
2.58.899.835 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
2.58.899.835 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
2.58.899.836 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
2.58.899.837 I srv get_availabl: prompt cache update took 102.46 ms
2.58.899.909 I slot launch_slot_: id 3 | task 5807 | processing task, is_child = 0
2.58.899.910 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.58.899.911 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.58.899.924 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.58.899.924 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.58.899.944 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.58.899.944 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.58.899.972 I slot update_slots: id 3 | task 5807 | Checking checkpoint with [70, 70] against 75...
2.58.918.667 W slot update_slots: id 3 | task 5807 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
3.02.254.631 I slot print_timing: id 3 | task 5807 | n_decoded = 120, tg = 39.96 t/s, tg_3s = 39.96 t/s
3.05.254.881 I slot print_timing: id 3 | task 5807 | n_decoded = 240, tg = 39.98 t/s, tg_3s = 40.00 t/s
3.08.261.864 I slot print_timing: id 3 | task 5807 | n_decoded = 360, tg = 39.95 t/s, tg_3s = 39.91 t/s
3.11.279.569 I slot print_timing: id 3 | task 5807 | n_decoded = 480, tg = 39.91 t/s, tg_3s = 39.77 t/s
3.13.634.047 I slot print_timing: id 3 | task 5807 | prompt eval time = 351.66 ms / 68 tokens ( 5.17 ms per token, 193.37 tokens per second)
3.13.634.051 I slot print_timing: id 3 | task 5807 | eval time = 14382.41 ms / 574 tokens ( 25.06 ms per token, 39.91 tokens per second)
3.13.634.056 I slot print_timing: id 3 | task 5807 | total time = 14734.07 ms / 642 tokens
3.13.634.056 I slot print_timing: id 3 | task 5807 | graphs reused = 6320
3.13.634.093 I slot release: id 3 | task 5807 | stop processing: n_tokens = 712, truncated = 0
3.13.634.101 I srv update_slots: all slots are idle
3.13.639.237 I srv operator(): Chat format: peg-native
3.13.639.444 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.655 (> 0.100 thold), f_keep = 0.104
3.13.639.446 I srv get_availabl: updating prompt cache
3.13.639.485 W srv prompt_save: - saving prompt with length 712, total state size = 92.072 MiB (draft: 0.000 MiB)
3.13.741.313 I srv load: - looking for better prompt, base f_keep = 0.104, sim = 0.655
3.13.741.323 I srv update: - cache state: 10 prompts, 1680.151 MiB (limits: 8192.000 MiB, 16384 tokens, 35895 est)
3.13.741.324 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
3.13.741.325 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
3.13.741.326 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
3.13.741.327 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
3.13.741.328 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
3.13.741.328 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
3.13.741.329 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
3.13.741.329 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
3.13.741.330 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
3.13.741.330 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
3.13.741.332 I srv get_availabl: prompt cache update took 101.89 ms
3.13.741.414 I slot launch_slot_: id 3 | task 6383 | processing task, is_child = 0
3.13.741.415 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.13.741.416 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.13.741.428 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.13.741.428 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.13.741.437 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.13.741.437 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.13.741.455 I slot update_slots: id 3 | task 6383 | Checking checkpoint with [70, 70] against 74...
3.13.760.711 W slot update_slots: id 3 | task 6383 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
3.17.022.916 I slot print_timing: id 3 | task 6383 | n_decoded = 121, tg = 40.09 t/s, tg_3s = 40.09 t/s
3.20.037.907 I slot print_timing: id 3 | task 6383 | n_decoded = 241, tg = 39.95 t/s, tg_3s = 39.80 t/s
3.23.046.002 I slot print_timing: id 3 | task 6383 | n_decoded = 361, tg = 39.93 t/s, tg_3s = 39.89 t/s
3.26.053.300 I slot print_timing: id 3 | task 6383 | n_decoded = 480, tg = 39.84 t/s, tg_3s = 39.57 t/s
3.28.471.962 I slot print_timing: id 3 | task 6383 | prompt eval time = 263.20 ms / 42 tokens ( 6.27 ms per token, 159.57 tokens per second)
3.28.471.966 I slot print_timing: id 3 | task 6383 | eval time = 14467.30 ms / 576 tokens ( 25.12 ms per token, 39.81 tokens per second)
3.28.471.967 I slot print_timing: id 3 | task 6383 | total time = 14730.50 ms / 618 tokens
3.28.471.968 I slot print_timing: id 3 | task 6383 | graphs reused = 6892
3.28.472.009 I slot release: id 3 | task 6383 | stop processing: n_tokens = 688, truncated = 0
3.28.472.019 I srv update_slots: all slots are idle
3.28.477.029 I srv operator(): Chat format: peg-native
3.28.477.249 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.532 (> 0.100 thold), f_keep = 0.108
3.28.477.252 I srv get_availabl: updating prompt cache
3.28.477.309 W srv prompt_save: - saving prompt with length 688, total state size = 91.509 MiB (draft: 0.000 MiB)
3.28.578.195 I srv load: - looking for better prompt, base f_keep = 0.108, sim = 0.532
3.28.578.206 I srv update: - cache state: 11 prompts, 1847.036 MiB (limits: 8192.000 MiB, 16384 tokens, 35703 est)
3.28.578.207 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
3.28.578.208 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
3.28.578.209 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
3.28.578.210 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
3.28.578.215 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
3.28.578.215 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
3.28.578.216 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
3.28.578.217 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
3.28.578.218 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
3.28.578.218 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
3.28.578.221 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
3.28.578.223 I srv get_availabl: prompt cache update took 100.97 ms
3.28.578.306 I slot launch_slot_: id 3 | task 6961 | processing task, is_child = 0
3.28.578.307 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.28.578.308 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.28.578.323 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.28.578.323 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.28.578.333 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.28.578.334 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.28.578.355 I slot update_slots: id 3 | task 6961 | Checking checkpoint with [70, 70] against 74...
3.28.597.332 W slot update_slots: id 3 | task 6961 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
3.31.948.822 I slot print_timing: id 3 | task 6961 | n_decoded = 120, tg = 39.72 t/s, tg_3s = 39.72 t/s
3.34.973.828 I slot print_timing: id 3 | task 6961 | n_decoded = 240, tg = 39.69 t/s, tg_3s = 39.67 t/s
3.37.998.398 I slot print_timing: id 3 | task 6961 | n_decoded = 360, tg = 39.69 t/s, tg_3s = 39.68 t/s
3.39.472.700 I slot print_timing: id 3 | task 6961 | prompt eval time = 349.16 ms / 68 tokens ( 5.13 ms per token, 194.75 tokens per second)
3.39.472.705 I slot print_timing: id 3 | task 6961 | eval time = 10545.18 ms / 418 tokens ( 25.23 ms per token, 39.64 tokens per second)
3.39.472.706 I slot print_timing: id 3 | task 6961 | total time = 10894.34 ms / 486 tokens
3.39.472.707 I slot print_timing: id 3 | task 6961 | graphs reused = 7306
3.39.472.747 I slot release: id 3 | task 6961 | stop processing: n_tokens = 556, truncated = 0
3.39.472.757 I srv update_slots: all slots are idle
3.39.477.784 I srv operator(): Chat format: peg-native
3.39.477.993 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.561 (> 0.100 thold), f_keep = 0.133
3.39.477.995 I srv get_availabl: updating prompt cache
3.39.478.042 W srv prompt_save: - saving prompt with length 556, total state size = 88.414 MiB (draft: 0.000 MiB)
3.39.576.820 I srv load: - looking for better prompt, base f_keep = 0.133, sim = 0.561
3.39.576.834 I srv update: - cache state: 12 prompts, 2010.826 MiB (limits: 8192.000 MiB, 16384 tokens, 35060 est)
3.39.576.835 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
3.39.576.839 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
3.39.576.840 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
3.39.576.840 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
3.39.576.841 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
3.39.576.841 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
3.39.576.842 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
3.39.576.842 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
3.39.576.846 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
3.39.576.846 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
3.39.576.847 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
3.39.576.847 I srv update: - prompt 0x6289821a7070: 556 tokens, checkpoints: 1, 163.790 MiB
3.39.576.848 I srv get_availabl: prompt cache update took 98.85 ms
3.39.576.922 I slot launch_slot_: id 3 | task 7381 | processing task, is_child = 0
3.39.576.922 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.39.576.923 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.39.576.937 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.39.576.938 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.39.576.958 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.39.576.958 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.39.576.987 I slot update_slots: id 3 | task 7381 | Checking checkpoint with [70, 70] against 74...
3.39.595.754 W slot update_slots: id 3 | task 7381 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
3.42.930.866 I slot print_timing: id 3 | task 7381 | n_decoded = 121, tg = 40.07 t/s, tg_3s = 40.07 t/s
3.45.954.236 I slot print_timing: id 3 | task 7381 | n_decoded = 241, tg = 39.88 t/s, tg_3s = 39.69 t/s
3.48.954.265 I slot print_timing: id 3 | task 7381 | n_decoded = 361, tg = 39.92 t/s, tg_3s = 40.00 t/s
3.49.693.245 I slot print_timing: id 3 | task 7381 | prompt eval time = 333.88 ms / 61 tokens ( 5.47 ms per token, 182.70 tokens per second)
3.49.693.248 I slot print_timing: id 3 | task 7381 | eval time = 9782.37 ms / 390 tokens ( 25.08 ms per token, 39.87 tokens per second)
3.49.693.249 I slot print_timing: id 3 | task 7381 | total time = 10116.25 ms / 451 tokens
3.49.693.250 I slot print_timing: id 3 | task 7381 | graphs reused = 7692
3.49.693.284 I slot release: id 3 | task 7381 | stop processing: n_tokens = 521, truncated = 0
3.49.693.291 I srv update_slots: all slots are idle
3.49.698.319 I srv operator(): Chat format: peg-native
3.49.698.530 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.625 (> 0.100 thold), f_keep = 0.144
3.49.698.532 I srv get_availabl: updating prompt cache
3.49.698.566 W srv prompt_save: - saving prompt with length 521, total state size = 87.593 MiB (draft: 0.000 MiB)
3.49.796.407 I srv load: - looking for better prompt, base f_keep = 0.144, sim = 0.625
3.49.796.417 I srv update: - cache state: 13 prompts, 2173.794 MiB (limits: 8192.000 MiB, 16384 tokens, 34395 est)
3.49.796.418 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
3.49.796.422 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
3.49.796.423 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
3.49.796.424 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
3.49.796.424 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
3.49.796.425 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
3.49.796.425 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
3.49.796.427 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
3.49.796.428 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
3.49.796.428 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
3.49.796.429 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
3.49.796.429 I srv update: - prompt 0x6289821a7070: 556 tokens, checkpoints: 1, 163.790 MiB
3.49.796.430 I srv update: - prompt 0x628989a2ae70: 521 tokens, checkpoints: 1, 162.969 MiB
3.49.796.431 I srv get_availabl: prompt cache update took 97.90 ms
3.49.796.506 I slot launch_slot_: id 3 | task 7773 | processing task, is_child = 0
3.49.796.507 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.49.796.507 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.49.796.519 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.49.796.519 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.49.796.528 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.49.796.528 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.49.796.546 I slot update_slots: id 3 | task 7773 | Checking checkpoint with [70, 70] against 75...
3.49.815.297 W slot update_slots: id 3 | task 7773 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
3.53.070.144 I slot print_timing: id 3 | task 7773 | n_decoded = 121, tg = 40.16 t/s, tg_3s = 40.16 t/s
3.56.086.639 I slot print_timing: id 3 | task 7773 | n_decoded = 241, tg = 39.97 t/s, tg_3s = 39.78 t/s
3.59.095.592 I slot print_timing: id 3 | task 7773 | n_decoded = 361, tg = 39.94 t/s, tg_3s = 39.88 t/s
4.02.117.583 I slot print_timing: id 3 | task 7773 | n_decoded = 481, tg = 39.88 t/s, tg_3s = 39.71 t/s
4.03.719.840 I slot print_timing: id 3 | task 7773 | prompt eval time = 260.73 ms / 49 tokens ( 5.32 ms per token, 187.93 tokens per second)
4.03.719.843 I slot print_timing: id 3 | task 7773 | eval time = 13662.56 ms / 545 tokens ( 25.07 ms per token, 39.89 tokens per second)
4.03.719.844 I slot print_timing: id 3 | task 7773 | total time = 13923.29 ms / 594 tokens
4.03.719.844 I slot print_timing: id 3 | task 7773 | graphs reused = 8233
4.03.719.883 I slot release: id 3 | task 7773 | stop processing: n_tokens = 664, truncated = 0
4.03.719.892 I srv update_slots: all slots are idle
4.03.725.153 I srv operator(): Chat format: peg-native
4.03.725.370 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.597 (> 0.100 thold), f_keep = 0.111
4.03.725.372 I srv get_availabl: updating prompt cache
4.03.725.414 W srv prompt_save: - saving prompt with length 664, total state size = 90.946 MiB (draft: 0.000 MiB)
4.03.825.915 I srv load: - looking for better prompt, base f_keep = 0.111, sim = 0.597
4.03.825.927 I srv update: - cache state: 14 prompts, 2340.116 MiB (limits: 8192.000 MiB, 16384 tokens, 34275 est)
4.03.825.929 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
4.03.825.930 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
4.03.825.931 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
4.03.825.932 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
4.03.825.932 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
4.03.825.933 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
4.03.825.933 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
4.03.825.934 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
4.03.825.935 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
4.03.825.935 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
4.03.825.936 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
4.03.825.936 I srv update: - prompt 0x6289821a7070: 556 tokens, checkpoints: 1, 163.790 MiB
4.03.825.937 I srv update: - prompt 0x628989a2ae70: 521 tokens, checkpoints: 1, 162.969 MiB
4.03.825.937 I srv update: - prompt 0x6289820b99b0: 664 tokens, checkpoints: 1, 166.322 MiB
4.03.825.939 I srv get_availabl: prompt cache update took 100.57 ms
4.03.826.013 I slot launch_slot_: id 3 | task 8320 | processing task, is_child = 0
4.03.826.014 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.03.826.015 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.03.826.029 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.03.826.029 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.03.826.038 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.03.826.038 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.03.826.063 I slot update_slots: id 3 | task 8320 | Checking checkpoint with [70, 70] against 74...
4.03.845.094 W slot update_slots: id 3 | task 8320 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
4.07.137.626 I slot print_timing: id 3 | task 8320 | n_decoded = 121, tg = 40.10 t/s, tg_3s = 40.10 t/s
4.10.155.596 I slot print_timing: id 3 | task 8320 | n_decoded = 241, tg = 39.93 t/s, tg_3s = 39.76 t/s
4.13.161.743 I slot print_timing: id 3 | task 8320 | n_decoded = 361, tg = 39.93 t/s, tg_3s = 39.92 t/s
4.16.163.631 I slot print_timing: id 3 | task 8320 | n_decoded = 480, tg = 39.86 t/s, tg_3s = 39.64 t/s
4.19.178.765 I slot print_timing: id 3 | task 8320 | n_decoded = 600, tg = 39.84 t/s, tg_3s = 39.80 t/s
4.22.188.777 I slot print_timing: id 3 | task 8320 | n_decoded = 719, tg = 39.79 t/s, tg_3s = 39.53 t/s
4.25.197.852 I slot print_timing: id 3 | task 8320 | n_decoded = 838, tg = 39.76 t/s, tg_3s = 39.55 t/s
4.28.218.479 I slot print_timing: id 3 | task 8320 | n_decoded = 957, tg = 39.71 t/s, tg_3s = 39.40 t/s
4.31.231.839 I slot print_timing: id 3 | task 8320 | n_decoded = 1076, tg = 39.69 t/s, tg_3s = 39.49 t/s
4.34.234.260 I slot print_timing: id 3 | task 8320 | n_decoded = 1194, tg = 39.65 t/s, tg_3s = 39.30 t/s
4.37.234.296 I slot print_timing: id 3 | task 8320 | n_decoded = 1312, tg = 39.62 t/s, tg_3s = 39.33 t/s
4.40.255.271 I slot print_timing: id 3 | task 8320 | n_decoded = 1431, tg = 39.60 t/s, tg_3s = 39.39 t/s
4.43.257.485 I slot print_timing: id 3 | task 8320 | n_decoded = 1550, tg = 39.60 t/s, tg_3s = 39.64 t/s
4.46.267.318 I slot print_timing: id 3 | task 8320 | n_decoded = 1669, tg = 39.60 t/s, tg_3s = 39.54 t/s
4.49.272.490 I slot print_timing: id 3 | task 8320 | n_decoded = 1787, tg = 39.58 t/s, tg_3s = 39.27 t/s
4.50.792.958 I slot print_timing: id 3 | task 8320 | prompt eval time = 294.30 ms / 53 tokens ( 5.55 ms per token, 180.09 tokens per second)
4.50.792.961 I slot print_timing: id 3 | task 8320 | eval time = 46672.59 ms / 1847 tokens ( 25.27 ms per token, 39.57 tokens per second)
4.50.792.962 I slot print_timing: id 3 | task 8320 | total time = 46966.89 ms / 1900 tokens
4.50.792.963 I slot print_timing: id 3 | task 8320 | graphs reused = 10071
4.50.793.003 I slot release: id 3 | task 8320 | stop processing: n_tokens = 1970, truncated = 0
4.50.793.016 I srv update_slots: all slots are idle
4.50.798.119 I srv operator(): Chat format: peg-native
4.50.798.330 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.649 (> 0.100 thold), f_keep = 0.038
4.50.798.332 I srv get_availabl: updating prompt cache
4.50.798.409 W srv prompt_save: - saving prompt with length 1970, total state size = 121.571 MiB (draft: 0.000 MiB)
4.50.920.666 I srv load: - looking for better prompt, base f_keep = 0.038, sim = 0.649
4.50.920.679 I srv update: - cache state: 15 prompts, 2537.063 MiB (limits: 8192.000 MiB, 16384 tokens, 37975 est)
4.50.920.680 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
4.50.920.681 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
4.50.920.682 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
4.50.920.682 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
4.50.920.683 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
4.50.920.683 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
4.50.920.684 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
4.50.920.684 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
4.50.920.685 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
4.50.920.685 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
4.50.920.686 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
4.50.920.686 I srv update: - prompt 0x6289821a7070: 556 tokens, checkpoints: 1, 163.790 MiB
4.50.920.687 I srv update: - prompt 0x628989a2ae70: 521 tokens, checkpoints: 1, 162.969 MiB
4.50.920.687 I srv update: - prompt 0x6289820b99b0: 664 tokens, checkpoints: 1, 166.322 MiB
4.50.920.688 I srv update: - prompt 0x6289822e0790: 1970 tokens, checkpoints: 1, 196.946 MiB
4.50.920.689 I srv get_availabl: prompt cache update took 122.36 ms
4.50.920.760 I slot launch_slot_: id 3 | task 10169 | processing task, is_child = 0
4.50.920.760 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.50.920.761 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.50.920.774 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.50.920.774 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.50.920.783 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.50.920.783 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.50.920.801 I slot update_slots: id 3 | task 10169 | Checking checkpoint with [70, 70] against 74...
4.50.939.540 W slot update_slots: id 3 | task 10169 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
4.54.205.877 I slot print_timing: id 3 | task 10169 | n_decoded = 121, tg = 40.07 t/s, tg_3s = 40.07 t/s
4.57.207.824 I slot print_timing: id 3 | task 10169 | n_decoded = 240, tg = 39.85 t/s, tg_3s = 39.64 t/s
5.00.232.788 I slot print_timing: id 3 | task 10169 | n_decoded = 360, tg = 39.79 t/s, tg_3s = 39.67 t/s
5.03.247.854 I slot print_timing: id 3 | task 10169 | n_decoded = 479, tg = 39.71 t/s, tg_3s = 39.47 t/s
5.06.271.912 I slot print_timing: id 3 | task 10169 | n_decoded = 599, tg = 39.71 t/s, tg_3s = 39.68 t/s
5.08.503.457 I slot print_timing: id 3 | task 10169 | prompt eval time = 265.08 ms / 43 tokens ( 6.16 ms per token, 162.21 tokens per second)
5.08.503.461 I slot print_timing: id 3 | task 10169 | eval time = 17317.57 ms / 687 tokens ( 25.21 ms per token, 39.67 tokens per second)
5.08.503.461 I slot print_timing: id 3 | task 10169 | total time = 17582.65 ms / 730 tokens
5.08.503.462 I slot print_timing: id 3 | task 10169 | graphs reused = 10753
5.08.503.501 I slot release: id 3 | task 10169 | stop processing: n_tokens = 800, truncated = 0
5.08.503.510 I srv update_slots: all slots are idle
5.08.508.483 I srv operator(): Chat format: peg-native
5.08.508.694 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.655 (> 0.100 thold), f_keep = 0.093
5.08.508.695 I srv get_availabl: updating prompt cache
5.08.508.740 W srv prompt_save: - saving prompt with length 800, total state size = 94.135 MiB (draft: 0.000 MiB)
5.08.610.831 I srv load: - looking for better prompt, base f_keep = 0.093, sim = 0.655
5.08.610.844 I srv update: - cache state: 16 prompts, 2706.574 MiB (limits: 8192.000 MiB, 16384 tokens, 38018 est)
5.08.610.845 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
5.08.610.846 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
5.08.610.846 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
5.08.610.847 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
5.08.610.847 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
5.08.610.848 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
5.08.610.848 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
5.08.610.849 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
5.08.610.849 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
5.08.610.850 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
5.08.610.850 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
5.08.610.851 I srv update: - prompt 0x6289821a7070: 556 tokens, checkpoints: 1, 163.790 MiB
5.08.610.851 I srv update: - prompt 0x628989a2ae70: 521 tokens, checkpoints: 1, 162.969 MiB
5.08.610.852 I srv update: - prompt 0x6289820b99b0: 664 tokens, checkpoints: 1, 166.322 MiB
5.08.610.852 I srv update: - prompt 0x6289822e0790: 1970 tokens, checkpoints: 1, 196.946 MiB
5.08.610.853 I srv update: - prompt 0x628981efcfc0: 800 tokens, checkpoints: 1, 169.511 MiB
5.08.610.854 I srv get_availabl: prompt cache update took 102.16 ms
5.08.610.932 I slot launch_slot_: id 3 | task 10858 | processing task, is_child = 0
5.08.610.933 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.08.610.934 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.08.610.948 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.08.610.948 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.08.610.968 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.08.610.968 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.08.610.997 I slot update_slots: id 3 | task 10858 | Checking checkpoint with [70, 70] against 74...
5.08.629.802 W slot update_slots: id 3 | task 10858 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
5.11.898.374 I slot print_timing: id 3 | task 10858 | n_decoded = 121, tg = 40.01 t/s, tg_3s = 40.01 t/s
5.14.907.209 I slot print_timing: id 3 | task 10858 | n_decoded = 240, tg = 39.78 t/s, tg_3s = 39.55 t/s
5.17.910.716 I slot print_timing: id 3 | task 10858 | n_decoded = 359, tg = 39.73 t/s, tg_3s = 39.62 t/s
5.20.623.342 I slot print_timing: id 3 | task 10858 | prompt eval time = 263.04 ms / 42 tokens ( 6.26 ms per token, 159.67 tokens per second)
5.20.623.345 I slot print_timing: id 3 | task 10858 | eval time = 11749.30 ms / 466 tokens ( 25.21 ms per token, 39.66 tokens per second)
5.20.623.346 I slot print_timing: id 3 | task 10858 | total time = 12012.34 ms / 508 tokens
5.20.623.347 I slot print_timing: id 3 | task 10858 | graphs reused = 11215
5.20.623.383 I slot release: id 3 | task 10858 | stop processing: n_tokens = 578, truncated = 0
5.20.623.394 I srv update_slots: all slots are idle
5.20.628.270 I srv operator(): Chat format: peg-native
5.20.628.481 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.705 (> 0.100 thold), f_keep = 0.128
5.20.628.483 I srv get_availabl: updating prompt cache
5.20.628.519 W srv prompt_save: - saving prompt with length 578, total state size = 88.930 MiB (draft: 0.000 MiB)
5.20.728.910 I srv load: - looking for better prompt, base f_keep = 0.128, sim = 0.705
5.20.728.923 I srv update: - cache state: 17 prompts, 2870.880 MiB (limits: 8192.000 MiB, 16384 tokens, 37491 est)
5.20.728.924 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
5.20.728.924 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
5.20.728.925 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
5.20.728.926 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
5.20.728.926 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
5.20.728.927 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
5.20.728.927 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
5.20.728.928 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
5.20.728.928 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
5.20.728.929 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
5.20.728.929 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
5.20.728.930 I srv update: - prompt 0x6289821a7070: 556 tokens, checkpoints: 1, 163.790 MiB
5.20.728.930 I srv update: - prompt 0x628989a2ae70: 521 tokens, checkpoints: 1, 162.969 MiB
5.20.728.931 I srv update: - prompt 0x6289820b99b0: 664 tokens, checkpoints: 1, 166.322 MiB
5.20.728.931 I srv update: - prompt 0x6289822e0790: 1970 tokens, checkpoints: 1, 196.946 MiB
5.20.728.932 I srv update: - prompt 0x628981efcfc0: 800 tokens, checkpoints: 1, 169.511 MiB
5.20.728.932 I srv update: - prompt 0x628989907d90: 578 tokens, checkpoints: 1, 164.305 MiB
5.20.728.933 I srv get_availabl: prompt cache update took 100.45 ms
5.20.729.016 I slot launch_slot_: id 3 | task 11326 | processing task, is_child = 0
5.20.729.017 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.20.729.017 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.20.729.029 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.20.729.029 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.20.729.049 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.20.729.050 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.20.729.078 I slot update_slots: id 3 | task 11326 | Checking checkpoint with [70, 70] against 74...
5.20.747.780 W slot update_slots: id 3 | task 11326 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
5.23.973.623 I slot print_timing: id 3 | task 11326 | n_decoded = 120, tg = 39.97 t/s, tg_3s = 39.97 t/s
5.26.992.213 I slot print_timing: id 3 | task 11326 | n_decoded = 240, tg = 39.86 t/s, tg_3s = 39.75 t/s
5.30.017.133 I slot print_timing: id 3 | task 11326 | n_decoded = 361, tg = 39.91 t/s, tg_3s = 40.00 t/s
5.33.009.214 I slot print_timing: id 3 | task 11326 | prompt eval time = 241.97 ms / 34 tokens ( 7.12 ms per token, 140.51 tokens per second)
5.33.009.218 I slot print_timing: id 3 | task 11326 | eval time = 12038.16 ms / 480 tokens ( 25.08 ms per token, 39.87 tokens per second)
5.33.009.218 I slot print_timing: id 3 | task 11326 | total time = 12280.13 ms / 514 tokens
5.33.009.219 I slot print_timing: id 3 | task 11326 | graphs reused = 11691
5.33.009.255 I slot release: id 3 | task 11326 | stop processing: n_tokens = 584, truncated = 0
5.33.009.264 I srv update_slots: all slots are idle
5.33.014.247 I srv operator(): Chat format: peg-native
5.33.014.467 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.779 (> 0.100 thold), f_keep = 0.127
5.33.014.469 I srv get_availabl: updating prompt cache
5.33.014.512 W srv prompt_save: - saving prompt with length 584, total state size = 89.070 MiB (draft: 0.000 MiB)
5.33.115.113 I srv load: - looking for better prompt, base f_keep = 0.127, sim = 0.779
5.33.115.128 I srv update: - cache state: 18 prompts, 3035.326 MiB (limits: 8192.000 MiB, 16384 tokens, 37036 est)
5.33.115.130 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
5.33.115.131 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
5.33.115.132 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
5.33.115.135 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
5.33.115.136 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
5.33.115.137 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
5.33.115.137 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
5.33.115.138 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
5.33.115.138 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
5.33.115.140 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
5.33.115.140 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
5.33.115.141 I srv update: - prompt 0x6289821a7070: 556 tokens, checkpoints: 1, 163.790 MiB
5.33.115.142 I srv update: - prompt 0x628989a2ae70: 521 tokens, checkpoints: 1, 162.969 MiB
5.33.115.142 I srv update: - prompt 0x6289820b99b0: 664 tokens, checkpoints: 1, 166.322 MiB
5.33.115.143 I srv update: - prompt 0x6289822e0790: 1970 tokens, checkpoints: 1, 196.946 MiB
5.33.115.143 I srv update: - prompt 0x628981efcfc0: 800 tokens, checkpoints: 1, 169.511 MiB
5.33.115.144 I srv update: - prompt 0x628989907d90: 578 tokens, checkpoints: 1, 164.305 MiB
5.33.115.144 I srv update: - prompt 0x62898244c5d0: 584 tokens, checkpoints: 1, 164.446 MiB
5.33.115.145 I srv get_availabl: prompt cache update took 100.67 ms
5.33.115.227 I slot launch_slot_: id 3 | task 11808 | processing task, is_child = 0
5.33.115.228 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.33.115.228 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.33.115.241 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.33.115.242 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.33.115.250 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.33.115.250 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.33.115.268 I slot update_slots: id 3 | task 11808 | Checking checkpoint with [70, 70] against 74...
5.33.134.170 W slot update_slots: id 3 | task 11808 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
5.36.325.214 I slot print_timing: id 3 | task 11808 | n_decoded = 121, tg = 40.15 t/s, tg_3s = 40.15 t/s
5.39.342.276 I slot print_timing: id 3 | task 11808 | n_decoded = 241, tg = 39.96 t/s, tg_3s = 39.77 t/s
5.42.342.290 I slot print_timing: id 3 | task 11808 | n_decoded = 361, tg = 39.97 t/s, tg_3s = 40.00 t/s
5.45.315.342 I slot print_timing: id 3 | task 11808 | prompt eval time = 195.91 ms / 24 tokens ( 8.16 ms per token, 122.51 tokens per second)
5.45.315.346 I slot print_timing: id 3 | task 11808 | eval time = 12004.17 ms / 479 tokens ( 25.06 ms per token, 39.90 tokens per second)
5.45.315.347 I slot print_timing: id 3 | task 11808 | total time = 12200.07 ms / 503 tokens
5.45.315.347 I slot print_timing: id 3 | task 11808 | graphs reused = 12166
5.45.315.383 I slot release: id 3 | task 11808 | stop processing: n_tokens = 573, truncated = 0
5.45.315.390 I srv update_slots: all slots are idle
5.45.320.584 I srv operator(): Chat format: peg-native
5.45.320.796 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.638 (> 0.100 thold), f_keep = 0.129
5.45.320.798 I srv get_availabl: updating prompt cache
5.45.320.839 W srv prompt_save: - saving prompt with length 573, total state size = 88.812 MiB (draft: 0.000 MiB)
5.45.421.666 I srv load: - looking for better prompt, base f_keep = 0.129, sim = 0.638
5.45.421.679 I srv update: - cache state: 19 prompts, 3199.514 MiB (limits: 8192.000 MiB, 16384 tokens, 36603 est)
5.45.421.682 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
5.45.421.683 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
5.45.421.683 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
5.45.421.684 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
5.45.421.685 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
5.45.421.685 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
5.45.421.686 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
5.45.421.686 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
5.45.421.687 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
5.45.421.687 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
5.45.421.688 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
5.45.421.688 I srv update: - prompt 0x6289821a7070: 556 tokens, checkpoints: 1, 163.790 MiB
5.45.421.689 I srv update: - prompt 0x628989a2ae70: 521 tokens, checkpoints: 1, 162.969 MiB
5.45.421.689 I srv update: - prompt 0x6289820b99b0: 664 tokens, checkpoints: 1, 166.322 MiB
5.45.421.690 I srv update: - prompt 0x6289822e0790: 1970 tokens, checkpoints: 1, 196.946 MiB
5.45.421.690 I srv update: - prompt 0x628981efcfc0: 800 tokens, checkpoints: 1, 169.511 MiB
5.45.421.691 I srv update: - prompt 0x628989907d90: 578 tokens, checkpoints: 1, 164.305 MiB
5.45.421.691 I srv update: - prompt 0x62898244c5d0: 584 tokens, checkpoints: 1, 164.446 MiB
5.45.421.692 I srv update: - prompt 0x628989a29d10: 573 tokens, checkpoints: 1, 164.188 MiB
5.45.421.693 I srv get_availabl: prompt cache update took 100.89 ms
5.45.421.777 I slot launch_slot_: id 3 | task 12289 | processing task, is_child = 0
5.45.421.777 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.45.421.778 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.45.421.790 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.45.421.791 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.45.421.799 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.45.421.799 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.45.421.816 I slot update_slots: id 3 | task 12289 | Checking checkpoint with [70, 70] against 74...
5.45.440.912 W slot update_slots: id 3 | task 12289 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
5.48.697.289 I slot print_timing: id 3 | task 12289 | n_decoded = 121, tg = 40.13 t/s, tg_3s = 40.13 t/s
5.51.699.274 I slot print_timing: id 3 | task 12289 | n_decoded = 240, tg = 39.89 t/s, tg_3s = 39.64 t/s
5.54.713.530 I slot print_timing: id 3 | task 12289 | n_decoded = 360, tg = 39.86 t/s, tg_3s = 39.81 t/s
5.55.542.684 I slot print_timing: id 3 | task 12289 | prompt eval time = 260.48 ms / 45 tokens ( 5.79 ms per token, 172.76 tokens per second)
5.55.542.687 I slot print_timing: id 3 | task 12289 | eval time = 9860.38 ms / 393 tokens ( 25.09 ms per token, 39.86 tokens per second)
5.55.542.688 I slot print_timing: id 3 | task 12289 | total time = 10120.86 ms / 438 tokens
5.55.542.689 I slot print_timing: id 3 | task 12289 | graphs reused = 12556
5.55.542.728 I slot release: id 3 | task 12289 | stop processing: n_tokens = 508, truncated = 0
5.55.542.737 I srv update_slots: all slots are idle
5.55.547.640 I srv operator(): Chat format: peg-native
5.55.547.852 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.718 (> 0.100 thold), f_keep = 0.146
5.55.547.854 I srv get_availabl: updating prompt cache
5.55.547.892 W srv prompt_save: - saving prompt with length 508, total state size = 87.288 MiB (draft: 0.000 MiB)
5.55.647.690 I srv load: - looking for better prompt, base f_keep = 0.146, sim = 0.718
5.55.647.706 I srv update: - cache state: 20 prompts, 3362.178 MiB (limits: 8192.000 MiB, 16384 tokens, 36070 est)
5.55.647.707 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
5.55.647.707 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
5.55.647.708 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
5.55.647.709 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
5.55.647.709 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
5.55.647.710 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
5.55.647.710 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
5.55.647.711 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
5.55.647.711 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
5.55.647.712 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
5.55.647.712 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
5.55.647.713 I srv update: - prompt 0x6289821a7070: 556 tokens, checkpoints: 1, 163.790 MiB
5.55.647.713 I srv update: - prompt 0x628989a2ae70: 521 tokens, checkpoints: 1, 162.969 MiB
5.55.647.714 I srv update: - prompt 0x6289820b99b0: 664 tokens, checkpoints: 1, 166.322 MiB
5.55.647.714 I srv update: - prompt 0x6289822e0790: 1970 tokens, checkpoints: 1, 196.946 MiB
5.55.647.715 I srv update: - prompt 0x628981efcfc0: 800 tokens, checkpoints: 1, 169.511 MiB
5.55.647.715 I srv update: - prompt 0x628989907d90: 578 tokens, checkpoints: 1, 164.305 MiB
5.55.647.716 I srv update: - prompt 0x62898244c5d0: 584 tokens, checkpoints: 1, 164.446 MiB
5.55.647.716 I srv update: - prompt 0x628989a29d10: 573 tokens, checkpoints: 1, 164.188 MiB
5.55.647.717 I srv update: - prompt 0x628982029ca0: 508 tokens, checkpoints: 1, 162.664 MiB
5.55.647.718 I srv get_availabl: prompt cache update took 99.86 ms
5.55.647.797 I slot launch_slot_: id 3 | task 12684 | processing task, is_child = 0
5.55.647.798 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.55.647.798 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.55.647.811 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.55.647.811 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.55.647.820 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.55.647.821 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.55.647.839 I slot update_slots: id 3 | task 12684 | Checking checkpoint with [70, 70] against 74...
5.55.666.718 W slot update_slots: id 3 | task 12684 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
5.58.888.384 I slot print_timing: id 3 | task 12684 | n_decoded = 120, tg = 39.95 t/s, tg_3s = 39.95 t/s
6.01.899.256 I slot print_timing: id 3 | task 12684 | n_decoded = 239, tg = 39.74 t/s, tg_3s = 39.52 t/s
6.04.923.532 I slot print_timing: id 3 | task 12684 | n_decoded = 359, tg = 39.72 t/s, tg_3s = 39.68 t/s
6.07.944.572 I slot print_timing: id 3 | task 12684 | n_decoded = 478, tg = 39.64 t/s, tg_3s = 39.39 t/s
6.08.448.615 I slot print_timing: id 3 | task 12684 | prompt eval time = 236.70 ms / 32 tokens ( 7.40 ms per token, 135.19 tokens per second)
6.08.448.619 I slot print_timing: id 3 | task 12684 | eval time = 12564.07 ms / 498 tokens ( 25.23 ms per token, 39.64 tokens per second)
6.08.448.620 I slot print_timing: id 3 | task 12684 | total time = 12800.77 ms / 530 tokens
6.08.448.620 I slot print_timing: id 3 | task 12684 | graphs reused = 13050
6.08.448.658 I slot release: id 3 | task 12684 | stop processing: n_tokens = 600, truncated = 0
6.08.448.665 I srv update_slots: all slots are idle
6.08.453.765 I srv operator(): Chat format: peg-native
6.08.453.972 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.673 (> 0.100 thold), f_keep = 0.123
6.08.453.973 I srv get_availabl: updating prompt cache
6.08.454.009 W srv prompt_save: - saving prompt with length 600, total state size = 89.446 MiB (draft: 0.000 MiB)
6.08.554.082 I srv load: - looking for better prompt, base f_keep = 0.123, sim = 0.673
6.08.554.096 I srv update: - cache state: 21 prompts, 3526.999 MiB (limits: 8192.000 MiB, 16384 tokens, 35778 est)
6.08.554.097 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
6.08.554.097 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
6.08.554.098 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
6.08.554.099 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
6.08.554.099 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
6.08.554.100 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
6.08.554.100 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
6.08.554.101 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
6.08.554.101 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
6.08.554.102 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
6.08.554.102 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
6.08.554.103 I srv update: - prompt 0x6289821a7070: 556 tokens, checkpoints: 1, 163.790 MiB
6.08.554.103 I srv update: - prompt 0x628989a2ae70: 521 tokens, checkpoints: 1, 162.969 MiB
6.08.554.104 I srv update: - prompt 0x6289820b99b0: 664 tokens, checkpoints: 1, 166.322 MiB
6.08.554.104 I srv update: - prompt 0x6289822e0790: 1970 tokens, checkpoints: 1, 196.946 MiB
6.08.554.105 I srv update: - prompt 0x628981efcfc0: 800 tokens, checkpoints: 1, 169.511 MiB
6.08.554.105 I srv update: - prompt 0x628989907d90: 578 tokens, checkpoints: 1, 164.305 MiB
6.08.554.106 I srv update: - prompt 0x62898244c5d0: 584 tokens, checkpoints: 1, 164.446 MiB
6.08.554.106 I srv update: - prompt 0x628989a29d10: 573 tokens, checkpoints: 1, 164.188 MiB
6.08.554.107 I srv update: - prompt 0x628982029ca0: 508 tokens, checkpoints: 1, 162.664 MiB
6.08.554.107 I srv update: - prompt 0x628981d31d30: 600 tokens, checkpoints: 1, 164.821 MiB
6.08.554.108 I srv get_availabl: prompt cache update took 100.13 ms
6.08.554.185 I slot launch_slot_: id 3 | task 13184 | processing task, is_child = 0
6.08.554.186 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
6.08.554.187 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
6.08.554.200 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
6.08.554.200 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
6.08.554.209 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
6.08.554.209 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
6.08.554.225 I slot update_slots: id 3 | task 13184 | Checking checkpoint with [70, 70] against 74...
6.08.573.003 W slot update_slots: id 3 | task 13184 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
6.11.815.110 I slot print_timing: id 3 | task 13184 | n_decoded = 120, tg = 39.97 t/s, tg_3s = 39.97 t/s
6.14.816.947 I slot print_timing: id 3 | task 13184 | n_decoded = 239, tg = 39.81 t/s, tg_3s = 39.64 t/s
6.17.836.255 I slot print_timing: id 3 | task 13184 | n_decoded = 359, tg = 39.79 t/s, tg_3s = 39.74 t/s
6.20.855.385 I slot print_timing: id 3 | task 13184 | n_decoded = 479, tg = 39.78 t/s, tg_3s = 39.75 t/s
6.20.955.340 I slot print_timing: id 3 | task 13184 | prompt eval time = 258.77 ms / 39 tokens ( 6.64 ms per token, 150.71 tokens per second)
6.20.955.343 I slot print_timing: id 3 | task 13184 | eval time = 12142.34 ms / 483 tokens ( 25.14 ms per token, 39.78 tokens per second)
6.20.955.344 I slot print_timing: id 3 | task 13184 | total time = 12401.11 ms / 522 tokens
6.20.955.345 I slot print_timing: id 3 | task 13184 | graphs reused = 13529
6.20.955.384 I slot release: id 3 | task 13184 | stop processing: n_tokens = 592, truncated = 0
6.20.955.392 I srv update_slots: all slots are idle
6.20.960.689 I srv operator(): Chat format: peg-native
6.20.960.899 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.627 (> 0.100 thold), f_keep = 0.125
6.20.960.901 I srv get_availabl: updating prompt cache
6.20.960.936 W srv prompt_save: - saving prompt with length 592, total state size = 89.258 MiB (draft: 0.000 MiB)
6.21.061.987 I srv load: - looking for better prompt, base f_keep = 0.125, sim = 0.627
6.21.062.002 I srv update: - cache state: 22 prompts, 3691.633 MiB (limits: 8192.000 MiB, 16384 tokens, 35496 est)
6.21.062.003 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
6.21.062.004 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
6.21.062.004 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
6.21.062.005 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
6.21.062.005 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
6.21.062.006 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
6.21.062.007 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
6.21.062.007 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
6.21.062.008 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
6.21.062.008 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
6.21.062.009 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
6.21.062.009 I srv update: - prompt 0x6289821a7070: 556 tokens, checkpoints: 1, 163.790 MiB
6.21.062.010 I srv update: - prompt 0x628989a2ae70: 521 tokens, checkpoints: 1, 162.969 MiB
6.21.062.010 I srv update: - prompt 0x6289820b99b0: 664 tokens, checkpoints: 1, 166.322 MiB
6.21.062.011 I srv update: - prompt 0x6289822e0790: 1970 tokens, checkpoints: 1, 196.946 MiB
6.21.062.011 I srv update: - prompt 0x628981efcfc0: 800 tokens, checkpoints: 1, 169.511 MiB
6.21.062.012 I srv update: - prompt 0x628989907d90: 578 tokens, checkpoints: 1, 164.305 MiB
6.21.062.012 I srv update: - prompt 0x62898244c5d0: 584 tokens, checkpoints: 1, 164.446 MiB
6.21.062.013 I srv update: - prompt 0x628989a29d10: 573 tokens, checkpoints: 1, 164.188 MiB
6.21.062.013 I srv update: - prompt 0x628982029ca0: 508 tokens, checkpoints: 1, 162.664 MiB
6.21.062.014 I srv update: - prompt 0x628981d31d30: 600 tokens, checkpoints: 1, 164.821 MiB
6.21.062.014 I srv update: - prompt 0x62898234bc40: 592 tokens, checkpoints: 1, 164.634 MiB
6.21.062.016 I srv get_availabl: prompt cache update took 101.11 ms
6.21.062.096 I slot launch_slot_: id 3 | task 13669 | processing task, is_child = 0
6.21.062.097 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
6.21.062.098 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
6.21.062.112 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
6.21.062.112 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
6.21.062.120 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
6.21.062.121 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
6.21.062.139 I slot update_slots: id 3 | task 13669 | Checking checkpoint with [70, 70] against 74...
6.21.080.955 W slot update_slots: id 3 | task 13669 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
6.24.339.557 I slot print_timing: id 3 | task 13669 | n_decoded = 121, tg = 40.15 t/s, tg_3s = 40.15 t/s
6.27.358.640 I slot print_timing: id 3 | task 13669 | n_decoded = 241, tg = 39.95 t/s, tg_3s = 39.75 t/s
6.30.363.130 I slot print_timing: id 3 | task 13669 | n_decoded = 361, tg = 39.94 t/s, tg_3s = 39.94 t/s
6.30.638.170 I slot print_timing: id 3 | task 13669 | prompt eval time = 263.46 ms / 47 tokens ( 5.61 ms per token, 178.39 tokens per second)
6.30.638.173 I slot print_timing: id 3 | task 13669 | eval time = 9312.57 ms / 372 tokens ( 25.03 ms per token, 39.95 tokens per second)
6.30.638.174 I slot print_timing: id 3 | task 13669 | total time = 9576.03 ms / 419 tokens
6.30.638.175 I slot print_timing: id 3 | task 13669 | graphs reused = 13898
6.30.638.210 I slot release: id 3 | task 13669 | stop processing: n_tokens = 489, truncated = 0
6.30.638.216 I srv update_slots: all slots are idle
6.30.643.046 I srv operator(): Chat format: peg-native
6.30.643.265 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.725 (> 0.100 thold), f_keep = 0.151
6.30.643.267 I srv get_availabl: updating prompt cache
6.30.643.305 W srv prompt_save: - saving prompt with length 489, total state size = 86.843 MiB (draft: 0.000 MiB)
6.30.742.110 I srv load: - looking for better prompt, base f_keep = 0.151, sim = 0.725
6.30.742.124 I srv update: - cache state: 23 prompts, 3853.852 MiB (limits: 8192.000 MiB, 16384 tokens, 35041 est)
6.30.742.125 I srv update: - prompt 0x6289822a9cd0: 1023 tokens, checkpoints: 1, 174.740 MiB
6.30.742.126 I srv update: - prompt 0x628989891af0: 607 tokens, checkpoints: 1, 164.986 MiB
6.30.742.126 I srv update: - prompt 0x628989868c20: 370 tokens, checkpoints: 1, 159.428 MiB
6.30.742.127 I srv update: - prompt 0x6289898cd960: 626 tokens, checkpoints: 1, 165.431 MiB
6.30.742.128 I srv update: - prompt 0x628981fb2100: 1276 tokens, checkpoints: 1, 180.673 MiB
6.30.742.128 I srv update: - prompt 0x628981f8bca0: 645 tokens, checkpoints: 1, 165.877 MiB
6.30.742.129 I srv update: - prompt 0x62898234c030: 597 tokens, checkpoints: 1, 164.751 MiB
6.30.742.129 I srv update: - prompt 0x628980bbb7b0: 712 tokens, checkpoints: 1, 167.448 MiB
6.30.742.130 I srv update: - prompt 0x628980bbba50: 794 tokens, checkpoints: 1, 169.370 MiB
6.30.742.130 I srv update: - prompt 0x628981d023d0: 712 tokens, checkpoints: 1, 167.448 MiB
6.30.742.131 I srv update: - prompt 0x6289822f2630: 688 tokens, checkpoints: 1, 166.885 MiB
6.30.742.131 I srv update: - prompt 0x6289821a7070: 556 tokens, checkpoints: 1, 163.790 MiB
6.30.742.132 I srv update: - prompt 0x628989a2ae70: 521 tokens, checkpoints: 1, 162.969 MiB
6.30.742.132 I srv update: - prompt 0x6289820b99b0: 664 tokens, checkpoints: 1, 166.322 MiB
6.30.742.133 I srv update: - prompt 0x6289822e0790: 1970 tokens, checkpoints: 1, 196.946 MiB
6.30.742.133 I srv update: - prompt 0x628981efcfc0: 800 tokens, checkpoints: 1, 169.511 MiB
6.30.742.134 I srv update: - prompt 0x628989907d90: 578 tokens, checkpoints: 1, 164.305 MiB
6.30.742.134 I srv update: - prompt 0x62898244c5d0: 584 tokens, checkpoints: 1, 164.446 MiB
6.30.742.135 I srv update: - prompt 0x628989a29d10: 573 tokens, checkpoints: 1, 164.188 MiB
6.30.742.135 I srv update: - prompt 0x628982029ca0: 508 tokens, checkpoints: 1, 162.664 MiB
6.30.742.136 I srv update: - prompt 0x628981d31d30: 600 tokens, checkpoints: 1, 164.821 MiB
6.30.742.136 I srv update: - prompt 0x62898234bc40: 592 tokens, checkpoints: 1, 164.634 MiB
6.30.742.137 I srv update: - prompt 0x6289823f3390: 489 tokens, checkpoints: 1, 162.219 MiB
6.30.742.138 I srv get_availabl: prompt cache update took 98.87 ms
6.30.742.219 I slot launch_slot_: id 3 | task 14043 | processing task, is_child = 0
6.30.742.219 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
6.30.742.220 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
6.30.742.236 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
6.30.742.236 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
6.30.742.256 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
6.30.742.257 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
6.30.742.286 I slot update_slots: id 3 | task 14043 | Checking checkpoint with [70, 70] against 74...
6.30.761.026 W slot update_slots: id 3 | task 14043 | restored context checkpoint (pos_min = 70, pos_max = 70, n_tokens = 71, n_past = 71, size = 75.376 MiB)
6.33.995.148 I slot print_timing: id 3 | task 14043 | n_decoded = 121, tg = 40.07 t/s, tg_3s = 40.07 t/s
6.37.015.408 I slot print_timing: id 3 | task 14043 | n_decoded = 241, tg = 39.90 t/s, tg_3s = 39.73 t/s
6.40.026.286 I slot print_timing: id 3 | task 14043 | n_decoded = 361, tg = 39.89 t/s, tg_3s = 39.86 t/s
6.42.649.479 I slot print_timing: id 3 | task 14043 | prompt eval time = 233.30 ms / 31 tokens ( 7.53 ms per token, 132.88 tokens per second)
6.42.649.482 I slot print_timing: id 3 | task 14043 | eval time = 11673.89 ms / 465 tokens ( 25.11 ms per token, 39.83 tokens per second)
6.42.649.483 I slot print_timing: id 3 | task 14043 | total time = 11907.19 ms / 496 tokens
6.42.649.483 I slot print_timing: id 3 | task 14043 | graphs reused = 14359
6.42.649.515 I slot release: id 3 | task 14043 | stop processing: n_tokens = 566, truncated = 0
6.42.649.523 I srv update_slots: all slots are idle
6.42.650.072 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink