qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results_roles/devstral2-123b-iq2.server.log

969 lines
97 KiB
Plaintext
Raw Permalink Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.065.883 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.065.886 I device_info:
0.00.065.906 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.910 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.916 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.065.970 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.065.973 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.066.011 I srv init: running without SSL
0.00.066.071 I srv init: using 23 threads for HTTP server
0.00.066.403 I srv start: binding port with default address family
0.00.067.608 I srv llama_server: loading model
0.00.067.616 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/Devstral-2-123B-Instruct-2512-UD-IQ2_M.gguf'
0.00.067.668 I common_init_result: fitting params to device memory ...
0.00.067.669 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.27.424.646 W llama_context: n_ctx_seq (24576) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.27.622.176 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.28.058.414 I srv load_model: initializing slots, n_slots = 4
0.28.427.503 W common_speculative_init: no implementations specified for speculative decoding
0.28.427.508 I slot load_model: id 0 | task -1 | new slot, n_ctx = 24576
0.28.427.511 I slot load_model: id 1 | task -1 | new slot, n_ctx = 24576
0.28.427.512 I slot load_model: id 2 | task -1 | new slot, n_ctx = 24576
0.28.427.512 I slot load_model: id 3 | task -1 | new slot, n_ctx = 24576
0.28.427.555 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.28.427.556 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.28.427.557 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.28.427.557 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.28.427.576 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.28.432.982 I init: chat template, example_format: '[SYSTEM_PROMPT]You are a helpful assistant[/SYSTEM_PROMPT][INST]Hello[/INST]Hi there</s>[INST]How are you?[/INST]'
0.28.433.680 I srv init: init: chat template, thinking = 1
0.28.433.810 I srv llama_server: model loaded
0.28.433.815 I srv llama_server: server is listening on http://127.0.0.1:8090
0.28.433.819 I srv update_slots: all slots are idle
0.29.969.410 I srv operator(): Chat format: peg-native
0.29.969.593 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.29.969.596 I srv get_availabl: updating prompt cache
0.29.969.603 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.29.969.607 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 24576 tokens, 8589934592 est)
0.29.969.608 I srv get_availabl: prompt cache update took 0.01 ms
0.29.969.638 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.29.969.639 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.969.641 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.969.660 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.969.660 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.969.674 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.969.674 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.46.394.918 I slot print_timing: id 3 | task 0 | prompt eval time = 1946.49 ms / 58 tokens ( 33.56 ms per token, 29.80 tokens per second)
0.46.394.922 I slot print_timing: id 3 | task 0 | eval time = 14478.71 ms / 50 tokens ( 289.57 ms per token, 3.45 tokens per second)
0.46.394.923 I slot print_timing: id 3 | task 0 | total time = 16425.21 ms / 108 tokens
0.46.394.928 I slot print_timing: id 3 | task 0 | graphs reused = 49
0.46.394.981 I slot release: id 3 | task 0 | stop processing: n_tokens = 107, truncated = 0
0.46.394.985 I srv update_slots: all slots are idle
0.46.397.856 I srv operator(): Chat format: peg-native
0.46.397.961 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.672 (> 0.100 thold), f_keep = 0.383
0.46.397.963 I srv get_availabl: updating prompt cache
0.46.398.005 W srv prompt_save: - saving prompt with length 107, total state size = 36.785 MiB (draft: 0.000 MiB)
0.46.429.256 I srv load: - looking for better prompt, base f_keep = 0.383, sim = 0.672
0.46.429.265 I srv update: - cache state: 1 prompts, 36.785 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
0.46.429.266 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
0.46.429.267 I srv get_availabl: prompt cache update took 31.30 ms
0.46.429.317 I slot launch_slot_: id 3 | task 51 | processing task, is_child = 0
0.46.429.318 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.46.429.318 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.46.429.337 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.46.429.337 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.46.429.350 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.46.429.350 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.47.862.206 I slot print_timing: id 3 | task 51 | prompt eval time = 1135.86 ms / 20 tokens ( 56.79 ms per token, 17.61 tokens per second)
0.47.862.210 I slot print_timing: id 3 | task 51 | eval time = 296.96 ms / 2 tokens ( 148.48 ms per token, 6.73 tokens per second)
0.47.862.211 I slot print_timing: id 3 | task 51 | total time = 1432.82 ms / 22 tokens
0.47.862.212 I slot print_timing: id 3 | task 51 | graphs reused = 49
0.47.862.240 I slot release: id 3 | task 51 | stop processing: n_tokens = 62, truncated = 0
0.47.862.251 I srv update_slots: all slots are idle
0.47.865.165 I srv operator(): Chat format: peg-native
0.47.865.291 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.506 (> 0.100 thold), f_keep = 0.661
0.47.865.309 I slot launch_slot_: id 3 | task 54 | processing task, is_child = 0
0.47.865.309 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.47.865.310 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.47.865.327 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.47.865.327 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.47.865.339 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.47.865.340 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.10.006.110 I slot print_timing: id 3 | task 54 | prompt eval time = 1429.98 ms / 40 tokens ( 35.75 ms per token, 27.97 tokens per second)
1.10.006.114 I slot print_timing: id 3 | task 54 | eval time = 20710.77 ms / 71 tokens ( 291.70 ms per token, 3.43 tokens per second)
1.10.006.114 I slot print_timing: id 3 | task 54 | total time = 22140.75 ms / 111 tokens
1.10.006.115 I slot print_timing: id 3 | task 54 | graphs reused = 118
1.10.006.143 I slot release: id 3 | task 54 | stop processing: n_tokens = 151, truncated = 0
1.10.006.151 I srv update_slots: all slots are idle
1.10.009.205 I srv operator(): Chat format: peg-native
1.10.009.309 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.651 (> 0.100 thold), f_keep = 0.272
1.10.009.311 I srv get_availabl: updating prompt cache
1.10.009.346 W srv prompt_save: - saving prompt with length 151, total state size = 51.910 MiB (draft: 0.000 MiB)
1.10.052.838 I srv load: - looking for better prompt, base f_keep = 0.272, sim = 0.651
1.10.052.845 I srv update: - cache state: 2 prompts, 88.695 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
1.10.052.846 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
1.10.052.847 I srv update: - prompt 0x5719f1d67110: 151 tokens, checkpoints: 0, 51.910 MiB
1.10.052.848 I srv get_availabl: prompt cache update took 43.54 ms
1.10.052.896 I slot launch_slot_: id 3 | task 126 | processing task, is_child = 0
1.10.052.897 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.10.052.898 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.10.052.917 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.10.052.917 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.10.052.930 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.10.052.930 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.40.538.588 I slot print_timing: id 3 | task 126 | n_decoded = 100, tg = 3.41 t/s, tg_3s = 3.41 t/s
1.43.802.021 I slot print_timing: id 3 | task 126 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s
1.47.063.443 I slot print_timing: id 3 | task 126 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s
1.50.327.933 I slot print_timing: id 3 | task 126 | n_decoded = 133, tg = 3.40 t/s, tg_3s = 3.37 t/s
1.53.591.549 I slot print_timing: id 3 | task 126 | n_decoded = 144, tg = 3.40 t/s, tg_3s = 3.37 t/s
1.56.858.800 I slot print_timing: id 3 | task 126 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.37 t/s
2.00.119.917 I slot print_timing: id 3 | task 126 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.37 t/s
2.03.381.149 I slot print_timing: id 3 | task 126 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.37 t/s
2.06.641.928 I slot print_timing: id 3 | task 126 | n_decoded = 188, tg = 3.39 t/s, tg_3s = 3.37 t/s
2.09.922.356 I slot print_timing: id 3 | task 126 | n_decoded = 199, tg = 3.39 t/s, tg_3s = 3.35 t/s
2.11.113.606 I slot print_timing: id 3 | task 126 | prompt eval time = 1137.37 ms / 22 tokens ( 51.70 ms per token, 19.34 tokens per second)
2.11.113.609 I slot print_timing: id 3 | task 126 | eval time = 59923.28 ms / 203 tokens ( 295.19 ms per token, 3.39 tokens per second)
2.11.113.610 I slot print_timing: id 3 | task 126 | total time = 61060.65 ms / 225 tokens
2.11.113.611 I slot print_timing: id 3 | task 126 | graphs reused = 318
2.11.113.642 I slot release: id 3 | task 126 | stop processing: n_tokens = 265, truncated = 0
2.11.113.652 I srv update_slots: all slots are idle
2.11.116.620 I srv operator(): Chat format: peg-native
2.11.116.732 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.695 (> 0.100 thold), f_keep = 0.155
2.11.116.734 I srv get_availabl: updating prompt cache
2.11.116.787 W srv prompt_save: - saving prompt with length 265, total state size = 91.099 MiB (draft: 0.000 MiB)
2.11.188.120 I srv load: - looking for better prompt, base f_keep = 0.155, sim = 0.695
2.11.188.127 I srv update: - cache state: 3 prompts, 179.793 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
2.11.188.128 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
2.11.188.129 I srv update: - prompt 0x5719f1d67110: 151 tokens, checkpoints: 0, 51.910 MiB
2.11.188.129 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
2.11.188.131 I srv get_availabl: prompt cache update took 71.40 ms
2.11.188.183 I slot launch_slot_: id 3 | task 330 | processing task, is_child = 0
2.11.188.184 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.11.188.184 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.11.188.209 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.11.188.209 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.11.188.225 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.11.188.225 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.23.306.861 I slot print_timing: id 3 | task 330 | prompt eval time = 1136.17 ms / 18 tokens ( 63.12 ms per token, 15.84 tokens per second)
2.23.306.864 I slot print_timing: id 3 | task 330 | eval time = 10982.44 ms / 38 tokens ( 289.01 ms per token, 3.46 tokens per second)
2.23.306.865 I slot print_timing: id 3 | task 330 | total time = 12118.61 ms / 56 tokens
2.23.306.866 I slot print_timing: id 3 | task 330 | graphs reused = 354
2.23.306.899 I slot release: id 3 | task 330 | stop processing: n_tokens = 96, truncated = 0
2.23.306.905 I srv update_slots: all slots are idle
2.23.309.845 I srv operator(): Chat format: peg-native
2.23.309.955 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.519 (> 0.100 thold), f_keep = 0.427
2.23.309.957 I srv get_availabl: updating prompt cache
2.23.309.993 W srv prompt_save: - saving prompt with length 96, total state size = 33.003 MiB (draft: 0.000 MiB)
2.23.339.998 I srv load: - looking for better prompt, base f_keep = 0.427, sim = 0.519
2.23.340.007 I srv update: - cache state: 4 prompts, 212.796 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
2.23.340.008 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
2.23.340.008 I srv update: - prompt 0x5719f1d67110: 151 tokens, checkpoints: 0, 51.910 MiB
2.23.340.009 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
2.23.340.010 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
2.23.340.011 I srv get_availabl: prompt cache update took 30.05 ms
2.23.340.055 I slot launch_slot_: id 3 | task 369 | processing task, is_child = 0
2.23.340.056 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.23.340.056 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.23.340.073 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.23.340.073 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.23.340.085 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.23.340.086 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.46.440.004 I slot print_timing: id 3 | task 369 | prompt eval time = 1431.69 ms / 38 tokens ( 37.68 ms per token, 26.54 tokens per second)
2.46.440.008 I slot print_timing: id 3 | task 369 | eval time = 21668.20 ms / 74 tokens ( 292.81 ms per token, 3.42 tokens per second)
2.46.440.009 I slot print_timing: id 3 | task 369 | total time = 23099.89 ms / 112 tokens
2.46.440.010 I slot print_timing: id 3 | task 369 | graphs reused = 426
2.46.440.045 I slot release: id 3 | task 369 | stop processing: n_tokens = 152, truncated = 0
2.46.440.053 I srv update_slots: all slots are idle
2.46.442.853 I srv operator(): Chat format: peg-native
2.46.442.959 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.547 (> 0.100 thold), f_keep = 0.270
2.46.442.961 I srv get_availabl: updating prompt cache
2.46.442.999 W srv prompt_save: - saving prompt with length 152, total state size = 52.254 MiB (draft: 0.000 MiB)
2.46.487.760 I srv load: - looking for better prompt, base f_keep = 0.270, sim = 0.547
2.46.487.768 I srv update: - cache state: 5 prompts, 265.050 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
2.46.487.769 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
2.46.487.769 I srv update: - prompt 0x5719f1d67110: 151 tokens, checkpoints: 0, 51.910 MiB
2.46.487.770 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
2.46.487.771 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
2.46.487.771 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
2.46.487.772 I srv get_availabl: prompt cache update took 44.81 ms
2.46.487.809 I slot launch_slot_: id 3 | task 444 | processing task, is_child = 0
2.46.487.809 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.46.487.810 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.46.487.826 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.46.487.826 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.46.487.839 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.46.487.839 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.07.520.284 I slot print_timing: id 3 | task 444 | prompt eval time = 1429.59 ms / 34 tokens ( 42.05 ms per token, 23.78 tokens per second)
3.07.520.287 I slot print_timing: id 3 | task 444 | eval time = 19602.83 ms / 67 tokens ( 292.58 ms per token, 3.42 tokens per second)
3.07.520.289 I slot print_timing: id 3 | task 444 | total time = 21032.42 ms / 101 tokens
3.07.520.289 I slot print_timing: id 3 | task 444 | graphs reused = 491
3.07.520.324 I slot release: id 3 | task 444 | stop processing: n_tokens = 141, truncated = 0
3.07.520.332 I srv update_slots: all slots are idle
3.07.523.088 I srv operator(): Chat format: peg-native
3.07.523.206 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.672 (> 0.100 thold), f_keep = 0.291
3.07.523.208 I srv get_availabl: updating prompt cache
3.07.523.251 W srv prompt_save: - saving prompt with length 141, total state size = 48.472 MiB (draft: 0.000 MiB)
3.07.564.609 I srv load: - looking for better prompt, base f_keep = 0.291, sim = 0.672
3.07.564.614 I srv load: - found better prompt with f_keep = 0.291, sim = 0.721
3.07.589.154 I srv update: - cache state: 5 prompts, 261.613 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
3.07.589.158 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
3.07.589.159 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
3.07.589.160 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
3.07.589.160 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
3.07.589.160 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
3.07.589.161 I srv get_availabl: prompt cache update took 65.95 ms
3.07.589.194 I slot launch_slot_: id 3 | task 512 | processing task, is_child = 0
3.07.589.195 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.07.589.196 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.07.589.224 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.07.589.224 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.07.589.236 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.07.589.237 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.14.968.866 I slot print_timing: id 3 | task 512 | prompt eval time = 1136.22 ms / 17 tokens ( 66.84 ms per token, 14.96 tokens per second)
3.14.968.869 I slot print_timing: id 3 | task 512 | eval time = 6243.38 ms / 22 tokens ( 283.79 ms per token, 3.52 tokens per second)
3.14.968.870 I slot print_timing: id 3 | task 512 | total time = 7379.60 ms / 39 tokens
3.14.968.870 I slot print_timing: id 3 | task 512 | graphs reused = 511
3.14.968.933 I slot release: id 3 | task 512 | stop processing: n_tokens = 82, truncated = 0
3.14.968.943 I srv update_slots: all slots are idle
3.14.971.724 I srv operator(): Chat format: peg-native
3.14.971.829 I slot get_availabl: id 2 | task -1 | selected slot by LRU, t_last = -1
3.14.971.831 I srv get_availabl: updating prompt cache
3.14.971.832 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
3.14.971.835 I srv update: - cache state: 5 prompts, 261.613 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
3.14.971.836 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
3.14.971.836 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
3.14.971.837 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
3.14.971.837 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
3.14.971.838 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
3.14.971.838 I srv get_availabl: prompt cache update took 0.01 ms
3.14.971.854 I slot launch_slot_: id 2 | task 535 | processing task, is_child = 0
3.14.971.854 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.14.971.855 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.14.971.874 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.14.971.875 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.14.971.905 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.14.971.935 W srv prompt_save: - saving prompt with length 82, total state size = 28.190 MiB (draft: 0.000 MiB)
3.14.998.094 I srv update: - cache state: 6 prompts, 289.803 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
3.14.998.100 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
3.14.998.101 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
3.14.998.102 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
3.14.998.102 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
3.14.998.103 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
3.14.998.103 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
3.14.998.104 I slot prompt_clear: id 3 | task -1 | clearing prompt with 82 tokens
3.47.450.151 I slot print_timing: id 2 | task 535 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s
3.50.714.263 I slot print_timing: id 2 | task 535 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s
3.53.981.239 I slot print_timing: id 2 | task 535 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s
3.57.247.606 I slot print_timing: id 2 | task 535 | n_decoded = 133, tg = 3.39 t/s, tg_3s = 3.37 t/s
4.00.514.961 I slot print_timing: id 2 | task 535 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s
4.03.779.855 I slot print_timing: id 2 | task 535 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.37 t/s
4.07.041.611 I slot print_timing: id 2 | task 535 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.37 t/s
4.10.327.058 I slot print_timing: id 2 | task 535 | n_decoded = 177, tg = 3.39 t/s, tg_3s = 3.35 t/s
4.13.599.902 I slot print_timing: id 2 | task 535 | n_decoded = 188, tg = 3.38 t/s, tg_3s = 3.36 t/s
4.16.879.541 I slot print_timing: id 2 | task 535 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.35 t/s
4.20.156.640 I slot print_timing: id 2 | task 535 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s
4.23.430.441 I slot print_timing: id 2 | task 535 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.36 t/s
4.26.709.606 I slot print_timing: id 2 | task 535 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.35 t/s
4.29.989.688 I slot print_timing: id 2 | task 535 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.35 t/s
4.33.267.057 I slot print_timing: id 2 | task 535 | n_decoded = 254, tg = 3.38 t/s, tg_3s = 3.36 t/s
4.36.542.048 I slot print_timing: id 2 | task 535 | n_decoded = 265, tg = 3.38 t/s, tg_3s = 3.36 t/s
4.39.823.096 I slot print_timing: id 2 | task 535 | n_decoded = 276, tg = 3.38 t/s, tg_3s = 3.35 t/s
4.43.096.085 I slot print_timing: id 2 | task 535 | n_decoded = 287, tg = 3.37 t/s, tg_3s = 3.36 t/s
4.46.373.062 I slot print_timing: id 2 | task 535 | n_decoded = 298, tg = 3.37 t/s, tg_3s = 3.36 t/s
4.49.656.313 I slot print_timing: id 2 | task 535 | n_decoded = 309, tg = 3.37 t/s, tg_3s = 3.35 t/s
4.52.936.606 I slot print_timing: id 2 | task 535 | n_decoded = 320, tg = 3.37 t/s, tg_3s = 3.35 t/s
4.56.221.500 I slot print_timing: id 2 | task 535 | n_decoded = 331, tg = 3.37 t/s, tg_3s = 3.35 t/s
4.59.503.163 I slot print_timing: id 2 | task 535 | n_decoded = 342, tg = 3.37 t/s, tg_3s = 3.35 t/s
5.02.785.092 I slot print_timing: id 2 | task 535 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.35 t/s
5.06.063.378 I slot print_timing: id 2 | task 535 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.36 t/s
5.09.345.217 I slot print_timing: id 2 | task 535 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.35 t/s
5.12.627.534 I slot print_timing: id 2 | task 535 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.35 t/s
5.15.905.496 I slot print_timing: id 2 | task 535 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.36 t/s
5.19.188.059 I slot print_timing: id 2 | task 535 | n_decoded = 408, tg = 3.37 t/s, tg_3s = 3.35 t/s
5.22.468.446 I slot print_timing: id 2 | task 535 | n_decoded = 419, tg = 3.37 t/s, tg_3s = 3.35 t/s
5.25.760.551 I slot print_timing: id 2 | task 535 | n_decoded = 430, tg = 3.37 t/s, tg_3s = 3.34 t/s
5.29.040.048 I slot print_timing: id 2 | task 535 | n_decoded = 441, tg = 3.37 t/s, tg_3s = 3.35 t/s
5.32.315.533 I slot print_timing: id 2 | task 535 | n_decoded = 452, tg = 3.37 t/s, tg_3s = 3.36 t/s
5.35.590.748 I slot print_timing: id 2 | task 535 | n_decoded = 463, tg = 3.37 t/s, tg_3s = 3.36 t/s
5.38.865.767 I slot print_timing: id 2 | task 535 | n_decoded = 474, tg = 3.37 t/s, tg_3s = 3.36 t/s
5.42.144.703 I slot print_timing: id 2 | task 535 | n_decoded = 485, tg = 3.37 t/s, tg_3s = 3.35 t/s
5.45.426.777 I slot print_timing: id 2 | task 535 | n_decoded = 496, tg = 3.37 t/s, tg_3s = 3.35 t/s
5.48.701.644 I slot print_timing: id 2 | task 535 | n_decoded = 507, tg = 3.37 t/s, tg_3s = 3.36 t/s
5.51.975.046 I slot print_timing: id 2 | task 535 | n_decoded = 518, tg = 3.37 t/s, tg_3s = 3.36 t/s
5.55.251.415 I slot print_timing: id 2 | task 535 | n_decoded = 529, tg = 3.37 t/s, tg_3s = 3.36 t/s
5.58.534.121 I slot print_timing: id 2 | task 535 | n_decoded = 540, tg = 3.36 t/s, tg_3s = 3.35 t/s
6.01.813.512 I slot print_timing: id 2 | task 535 | n_decoded = 551, tg = 3.36 t/s, tg_3s = 3.35 t/s
6.05.096.606 I slot print_timing: id 2 | task 535 | n_decoded = 562, tg = 3.36 t/s, tg_3s = 3.35 t/s
6.08.374.773 I slot print_timing: id 2 | task 535 | n_decoded = 573, tg = 3.36 t/s, tg_3s = 3.36 t/s
6.11.654.902 I slot print_timing: id 2 | task 535 | n_decoded = 584, tg = 3.36 t/s, tg_3s = 3.35 t/s
6.14.932.940 I slot print_timing: id 2 | task 535 | n_decoded = 595, tg = 3.36 t/s, tg_3s = 3.36 t/s
6.18.212.772 I slot print_timing: id 2 | task 535 | n_decoded = 606, tg = 3.36 t/s, tg_3s = 3.35 t/s
6.21.489.783 I slot print_timing: id 2 | task 535 | n_decoded = 617, tg = 3.36 t/s, tg_3s = 3.36 t/s
6.24.767.132 I slot print_timing: id 2 | task 535 | n_decoded = 628, tg = 3.36 t/s, tg_3s = 3.36 t/s
6.28.042.453 I slot print_timing: id 2 | task 535 | n_decoded = 639, tg = 3.36 t/s, tg_3s = 3.36 t/s
6.31.323.956 I slot print_timing: id 2 | task 535 | n_decoded = 650, tg = 3.36 t/s, tg_3s = 3.35 t/s
6.33.708.273 I slot print_timing: id 2 | task 535 | prompt eval time = 3053.58 ms / 90 tokens ( 33.93 ms per token, 29.47 tokens per second)
6.33.708.278 I slot print_timing: id 2 | task 535 | eval time = 195656.53 ms / 658 tokens ( 297.35 ms per token, 3.36 tokens per second)
6.33.708.279 I slot print_timing: id 2 | task 535 | total time = 198710.11 ms / 748 tokens
6.33.708.279 I slot print_timing: id 2 | task 535 | graphs reused = 1165
6.33.708.317 I slot release: id 2 | task 535 | stop processing: n_tokens = 747, truncated = 0
6.33.708.323 I srv update_slots: all slots are idle
6.33.711.246 I srv operator(): Chat format: peg-native
6.33.711.421 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.513 (> 0.100 thold), f_keep = 0.079
6.33.711.424 I srv get_availabl: updating prompt cache
6.33.711.482 W srv prompt_save: - saving prompt with length 747, total state size = 256.792 MiB (draft: 0.000 MiB)
6.34.031.411 I srv load: - looking for better prompt, base f_keep = 0.079, sim = 0.513
6.34.031.417 I srv update: - cache state: 7 prompts, 546.595 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
6.34.031.417 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
6.34.031.418 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
6.34.031.419 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
6.34.031.420 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
6.34.031.420 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
6.34.031.421 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
6.34.031.421 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
6.34.031.422 I srv get_availabl: prompt cache update took 320.00 ms
6.34.031.454 I slot launch_slot_: id 2 | task 1194 | processing task, is_child = 0
6.34.031.454 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
6.34.031.456 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
6.34.031.471 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
6.34.031.472 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
6.34.031.485 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
6.34.031.485 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
7.05.250.515 I slot print_timing: id 2 | task 1194 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s
7.08.520.853 I slot print_timing: id 2 | task 1194 | n_decoded = 111, tg = 3.39 t/s, tg_3s = 3.36 t/s
7.11.789.961 I slot print_timing: id 2 | task 1194 | n_decoded = 122, tg = 3.39 t/s, tg_3s = 3.36 t/s
7.15.059.942 I slot print_timing: id 2 | task 1194 | n_decoded = 133, tg = 3.39 t/s, tg_3s = 3.36 t/s
7.18.342.560 I slot print_timing: id 2 | task 1194 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.35 t/s
7.21.620.099 I slot print_timing: id 2 | task 1194 | n_decoded = 155, tg = 3.38 t/s, tg_3s = 3.36 t/s
7.24.893.860 I slot print_timing: id 2 | task 1194 | n_decoded = 166, tg = 3.38 t/s, tg_3s = 3.36 t/s
7.28.169.531 I slot print_timing: id 2 | task 1194 | n_decoded = 177, tg = 3.38 t/s, tg_3s = 3.36 t/s
7.31.449.064 I slot print_timing: id 2 | task 1194 | n_decoded = 188, tg = 3.38 t/s, tg_3s = 3.35 t/s
7.34.727.089 I slot print_timing: id 2 | task 1194 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.36 t/s
7.38.007.306 I slot print_timing: id 2 | task 1194 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.35 t/s
7.41.286.898 I slot print_timing: id 2 | task 1194 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.35 t/s
7.44.571.713 I slot print_timing: id 2 | task 1194 | n_decoded = 232, tg = 3.37 t/s, tg_3s = 3.35 t/s
7.47.851.676 I slot print_timing: id 2 | task 1194 | n_decoded = 243, tg = 3.37 t/s, tg_3s = 3.35 t/s
7.51.134.841 I slot print_timing: id 2 | task 1194 | n_decoded = 254, tg = 3.37 t/s, tg_3s = 3.35 t/s
7.54.417.627 I slot print_timing: id 2 | task 1194 | n_decoded = 265, tg = 3.37 t/s, tg_3s = 3.35 t/s
7.57.699.938 I slot print_timing: id 2 | task 1194 | n_decoded = 276, tg = 3.37 t/s, tg_3s = 3.35 t/s
8.00.979.527 I slot print_timing: id 2 | task 1194 | n_decoded = 287, tg = 3.37 t/s, tg_3s = 3.35 t/s
8.04.262.755 I slot print_timing: id 2 | task 1194 | n_decoded = 298, tg = 3.37 t/s, tg_3s = 3.35 t/s
8.07.541.710 I slot print_timing: id 2 | task 1194 | n_decoded = 309, tg = 3.37 t/s, tg_3s = 3.35 t/s
8.10.819.633 I slot print_timing: id 2 | task 1194 | n_decoded = 320, tg = 3.37 t/s, tg_3s = 3.36 t/s
8.14.097.663 I slot print_timing: id 2 | task 1194 | n_decoded = 331, tg = 3.37 t/s, tg_3s = 3.36 t/s
8.17.383.405 I slot print_timing: id 2 | task 1194 | n_decoded = 342, tg = 3.37 t/s, tg_3s = 3.35 t/s
8.20.662.156 I slot print_timing: id 2 | task 1194 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.35 t/s
8.23.942.058 I slot print_timing: id 2 | task 1194 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.35 t/s
8.27.223.567 I slot print_timing: id 2 | task 1194 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.35 t/s
8.30.501.288 I slot print_timing: id 2 | task 1194 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.36 t/s
8.33.777.674 I slot print_timing: id 2 | task 1194 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.36 t/s
8.37.065.907 I slot print_timing: id 2 | task 1194 | n_decoded = 408, tg = 3.36 t/s, tg_3s = 3.35 t/s
8.40.348.803 I slot print_timing: id 2 | task 1194 | n_decoded = 419, tg = 3.36 t/s, tg_3s = 3.35 t/s
8.43.636.504 I slot print_timing: id 2 | task 1194 | n_decoded = 430, tg = 3.36 t/s, tg_3s = 3.35 t/s
8.46.920.964 I slot print_timing: id 2 | task 1194 | n_decoded = 441, tg = 3.36 t/s, tg_3s = 3.35 t/s
8.50.203.314 I slot print_timing: id 2 | task 1194 | n_decoded = 452, tg = 3.36 t/s, tg_3s = 3.35 t/s
8.53.482.571 I slot print_timing: id 2 | task 1194 | n_decoded = 463, tg = 3.36 t/s, tg_3s = 3.35 t/s
8.56.761.997 I slot print_timing: id 2 | task 1194 | n_decoded = 474, tg = 3.36 t/s, tg_3s = 3.35 t/s
9.00.041.577 I slot print_timing: id 2 | task 1194 | n_decoded = 485, tg = 3.36 t/s, tg_3s = 3.35 t/s
9.03.323.336 I slot print_timing: id 2 | task 1194 | n_decoded = 496, tg = 3.36 t/s, tg_3s = 3.35 t/s
9.06.602.936 I slot print_timing: id 2 | task 1194 | n_decoded = 507, tg = 3.36 t/s, tg_3s = 3.35 t/s
9.09.885.634 I slot print_timing: id 2 | task 1194 | n_decoded = 518, tg = 3.36 t/s, tg_3s = 3.35 t/s
9.13.167.400 I slot print_timing: id 2 | task 1194 | n_decoded = 529, tg = 3.36 t/s, tg_3s = 3.35 t/s
9.16.448.061 I slot print_timing: id 2 | task 1194 | n_decoded = 540, tg = 3.36 t/s, tg_3s = 3.35 t/s
9.19.733.749 I slot print_timing: id 2 | task 1194 | n_decoded = 551, tg = 3.36 t/s, tg_3s = 3.35 t/s
9.23.012.339 I slot print_timing: id 2 | task 1194 | n_decoded = 562, tg = 3.36 t/s, tg_3s = 3.36 t/s
9.26.289.247 I slot print_timing: id 2 | task 1194 | n_decoded = 573, tg = 3.36 t/s, tg_3s = 3.36 t/s
9.29.564.349 I slot print_timing: id 2 | task 1194 | n_decoded = 584, tg = 3.36 t/s, tg_3s = 3.36 t/s
9.32.844.304 I slot print_timing: id 2 | task 1194 | n_decoded = 595, tg = 3.36 t/s, tg_3s = 3.35 t/s
9.33.441.035 I slot print_timing: id 2 | task 1194 | prompt eval time = 1776.11 ms / 56 tokens ( 31.72 ms per token, 31.53 tokens per second)
9.33.441.038 I slot print_timing: id 2 | task 1194 | eval time = 177633.41 ms / 597 tokens ( 297.54 ms per token, 3.36 tokens per second)
9.33.441.039 I slot print_timing: id 2 | task 1194 | total time = 179409.52 ms / 653 tokens
9.33.441.040 I slot print_timing: id 2 | task 1194 | graphs reused = 1758
9.33.441.080 I slot release: id 2 | task 1194 | stop processing: n_tokens = 711, truncated = 0
9.33.441.089 I srv update_slots: all slots are idle
9.33.443.986 I srv operator(): Chat format: peg-native
9.33.444.105 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.522 (> 0.100 thold), f_keep = 0.083
9.33.444.108 I srv get_availabl: updating prompt cache
9.33.444.180 W srv prompt_save: - saving prompt with length 711, total state size = 244.416 MiB (draft: 0.000 MiB)
9.33.741.904 I srv load: - looking for better prompt, base f_keep = 0.083, sim = 0.522
9.33.741.911 I srv update: - cache state: 8 prompts, 791.011 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
9.33.741.912 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
9.33.741.913 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
9.33.741.913 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
9.33.741.914 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
9.33.741.914 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
9.33.741.915 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
9.33.741.915 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
9.33.741.916 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
9.33.741.917 I srv get_availabl: prompt cache update took 297.81 ms
9.33.741.947 I slot launch_slot_: id 2 | task 1792 | processing task, is_child = 0
9.33.741.947 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
9.33.741.948 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
9.33.741.965 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
9.33.741.966 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
9.33.741.978 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
9.33.741.979 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
10.04.941.021 I slot print_timing: id 2 | task 1792 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s
10.08.205.686 I slot print_timing: id 2 | task 1792 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s
10.11.474.806 I slot print_timing: id 2 | task 1792 | n_decoded = 122, tg = 3.39 t/s, tg_3s = 3.36 t/s
10.14.740.519 I slot print_timing: id 2 | task 1792 | n_decoded = 133, tg = 3.39 t/s, tg_3s = 3.37 t/s
10.18.007.508 I slot print_timing: id 2 | task 1792 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s
10.21.299.366 I slot print_timing: id 2 | task 1792 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.34 t/s
10.24.577.489 I slot print_timing: id 2 | task 1792 | n_decoded = 166, tg = 3.38 t/s, tg_3s = 3.36 t/s
10.27.855.560 I slot print_timing: id 2 | task 1792 | n_decoded = 177, tg = 3.38 t/s, tg_3s = 3.36 t/s
10.31.134.904 I slot print_timing: id 2 | task 1792 | n_decoded = 188, tg = 3.38 t/s, tg_3s = 3.35 t/s
10.34.418.795 I slot print_timing: id 2 | task 1792 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.35 t/s
10.37.700.241 I slot print_timing: id 2 | task 1792 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.35 t/s
10.40.985.439 I slot print_timing: id 2 | task 1792 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.35 t/s
10.44.259.842 I slot print_timing: id 2 | task 1792 | n_decoded = 232, tg = 3.37 t/s, tg_3s = 3.36 t/s
10.47.535.639 I slot print_timing: id 2 | task 1792 | n_decoded = 243, tg = 3.37 t/s, tg_3s = 3.36 t/s
10.50.811.877 I slot print_timing: id 2 | task 1792 | n_decoded = 254, tg = 3.37 t/s, tg_3s = 3.36 t/s
10.54.090.467 I slot print_timing: id 2 | task 1792 | n_decoded = 265, tg = 3.37 t/s, tg_3s = 3.36 t/s
10.57.369.394 I slot print_timing: id 2 | task 1792 | n_decoded = 276, tg = 3.37 t/s, tg_3s = 3.35 t/s
11.00.647.616 I slot print_timing: id 2 | task 1792 | n_decoded = 287, tg = 3.37 t/s, tg_3s = 3.36 t/s
11.03.926.338 I slot print_timing: id 2 | task 1792 | n_decoded = 298, tg = 3.37 t/s, tg_3s = 3.35 t/s
11.07.204.308 I slot print_timing: id 2 | task 1792 | n_decoded = 309, tg = 3.37 t/s, tg_3s = 3.36 t/s
11.10.477.680 I slot print_timing: id 2 | task 1792 | n_decoded = 320, tg = 3.37 t/s, tg_3s = 3.36 t/s
11.13.755.502 I slot print_timing: id 2 | task 1792 | n_decoded = 331, tg = 3.37 t/s, tg_3s = 3.36 t/s
11.17.031.483 I slot print_timing: id 2 | task 1792 | n_decoded = 342, tg = 3.37 t/s, tg_3s = 3.36 t/s
11.20.306.894 I slot print_timing: id 2 | task 1792 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.36 t/s
11.23.585.137 I slot print_timing: id 2 | task 1792 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.36 t/s
11.26.862.295 I slot print_timing: id 2 | task 1792 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.36 t/s
11.30.140.206 I slot print_timing: id 2 | task 1792 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.36 t/s
11.33.420.018 I slot print_timing: id 2 | task 1792 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.35 t/s
11.36.710.032 I slot print_timing: id 2 | task 1792 | n_decoded = 408, tg = 3.37 t/s, tg_3s = 3.34 t/s
11.38.201.119 I slot print_timing: id 2 | task 1792 | prompt eval time = 1774.54 ms / 54 tokens ( 32.86 ms per token, 30.43 tokens per second)
11.38.201.124 I slot print_timing: id 2 | task 1792 | eval time = 122684.58 ms / 413 tokens ( 297.06 ms per token, 3.37 tokens per second)
11.38.201.124 I slot print_timing: id 2 | task 1792 | total time = 124459.11 ms / 467 tokens
11.38.201.125 I slot print_timing: id 2 | task 1792 | graphs reused = 2167
11.38.201.166 I slot release: id 2 | task 1792 | stop processing: n_tokens = 525, truncated = 0
11.38.201.175 I srv update_slots: all slots are idle
11.38.204.178 I srv operator(): Chat format: peg-native
11.38.204.293 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.480 (> 0.100 thold), f_keep = 0.112
11.38.204.295 I srv get_availabl: updating prompt cache
11.38.204.345 W srv prompt_save: - saving prompt with length 525, total state size = 180.477 MiB (draft: 0.000 MiB)
11.38.374.900 I srv load: - looking for better prompt, base f_keep = 0.112, sim = 0.480
11.38.374.907 I srv update: - cache state: 9 prompts, 971.488 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
11.38.374.908 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
11.38.374.909 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
11.38.374.910 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
11.38.374.910 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
11.38.374.911 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
11.38.374.911 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
11.38.374.912 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
11.38.374.912 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
11.38.374.913 I srv update: - prompt 0x5719f2442e40: 525 tokens, checkpoints: 0, 180.477 MiB
11.38.374.914 I srv get_availabl: prompt cache update took 170.62 ms
11.38.374.946 I slot launch_slot_: id 2 | task 2206 | processing task, is_child = 0
11.38.374.946 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
11.38.374.947 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
11.38.374.963 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
11.38.374.963 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
11.38.374.976 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
11.38.374.976 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
12.09.723.144 I slot print_timing: id 2 | task 2206 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s
12.12.989.589 I slot print_timing: id 2 | task 2206 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s
12.16.254.093 I slot print_timing: id 2 | task 2206 | n_decoded = 122, tg = 3.39 t/s, tg_3s = 3.37 t/s
12.19.519.161 I slot print_timing: id 2 | task 2206 | n_decoded = 133, tg = 3.39 t/s, tg_3s = 3.37 t/s
12.22.806.468 I slot print_timing: id 2 | task 2206 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.35 t/s
12.26.087.692 I slot print_timing: id 2 | task 2206 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.35 t/s
12.29.365.365 I slot print_timing: id 2 | task 2206 | n_decoded = 166, tg = 3.38 t/s, tg_3s = 3.36 t/s
12.32.646.315 I slot print_timing: id 2 | task 2206 | n_decoded = 177, tg = 3.38 t/s, tg_3s = 3.35 t/s
12.35.924.857 I slot print_timing: id 2 | task 2206 | n_decoded = 188, tg = 3.38 t/s, tg_3s = 3.36 t/s
12.39.201.344 I slot print_timing: id 2 | task 2206 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.36 t/s
12.42.478.404 I slot print_timing: id 2 | task 2206 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s
12.45.757.414 I slot print_timing: id 2 | task 2206 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.35 t/s
12.49.037.247 I slot print_timing: id 2 | task 2206 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.35 t/s
12.52.316.397 I slot print_timing: id 2 | task 2206 | n_decoded = 243, tg = 3.37 t/s, tg_3s = 3.35 t/s
12.55.597.317 I slot print_timing: id 2 | task 2206 | n_decoded = 254, tg = 3.37 t/s, tg_3s = 3.35 t/s
12.58.878.684 I slot print_timing: id 2 | task 2206 | n_decoded = 265, tg = 3.37 t/s, tg_3s = 3.35 t/s
13.02.155.940 I slot print_timing: id 2 | task 2206 | n_decoded = 276, tg = 3.37 t/s, tg_3s = 3.36 t/s
13.05.432.096 I slot print_timing: id 2 | task 2206 | n_decoded = 287, tg = 3.37 t/s, tg_3s = 3.36 t/s
13.08.706.874 I slot print_timing: id 2 | task 2206 | n_decoded = 298, tg = 3.37 t/s, tg_3s = 3.36 t/s
13.11.986.837 I slot print_timing: id 2 | task 2206 | n_decoded = 309, tg = 3.37 t/s, tg_3s = 3.35 t/s
13.15.262.624 I slot print_timing: id 2 | task 2206 | n_decoded = 320, tg = 3.37 t/s, tg_3s = 3.36 t/s
13.18.540.523 I slot print_timing: id 2 | task 2206 | n_decoded = 331, tg = 3.37 t/s, tg_3s = 3.36 t/s
13.21.820.310 I slot print_timing: id 2 | task 2206 | n_decoded = 342, tg = 3.37 t/s, tg_3s = 3.35 t/s
13.25.099.771 I slot print_timing: id 2 | task 2206 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.35 t/s
13.28.382.468 I slot print_timing: id 2 | task 2206 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.35 t/s
13.31.660.297 I slot print_timing: id 2 | task 2206 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.36 t/s
13.34.936.093 I slot print_timing: id 2 | task 2206 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.36 t/s
13.38.222.360 I slot print_timing: id 2 | task 2206 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.35 t/s
13.41.499.442 I slot print_timing: id 2 | task 2206 | n_decoded = 408, tg = 3.37 t/s, tg_3s = 3.36 t/s
13.44.776.655 I slot print_timing: id 2 | task 2206 | n_decoded = 419, tg = 3.37 t/s, tg_3s = 3.36 t/s
13.48.059.603 I slot print_timing: id 2 | task 2206 | n_decoded = 430, tg = 3.37 t/s, tg_3s = 3.35 t/s
13.51.336.695 I slot print_timing: id 2 | task 2206 | n_decoded = 441, tg = 3.37 t/s, tg_3s = 3.36 t/s
13.54.612.972 I slot print_timing: id 2 | task 2206 | n_decoded = 452, tg = 3.37 t/s, tg_3s = 3.36 t/s
13.57.893.325 I slot print_timing: id 2 | task 2206 | n_decoded = 463, tg = 3.37 t/s, tg_3s = 3.35 t/s
14.01.174.058 I slot print_timing: id 2 | task 2206 | n_decoded = 474, tg = 3.37 t/s, tg_3s = 3.35 t/s
14.04.453.482 I slot print_timing: id 2 | task 2206 | n_decoded = 485, tg = 3.36 t/s, tg_3s = 3.35 t/s
14.07.736.492 I slot print_timing: id 2 | task 2206 | n_decoded = 496, tg = 3.36 t/s, tg_3s = 3.35 t/s
14.11.018.938 I slot print_timing: id 2 | task 2206 | n_decoded = 507, tg = 3.36 t/s, tg_3s = 3.35 t/s
14.14.303.756 I slot print_timing: id 2 | task 2206 | n_decoded = 518, tg = 3.36 t/s, tg_3s = 3.35 t/s
14.17.583.268 I slot print_timing: id 2 | task 2206 | n_decoded = 529, tg = 3.36 t/s, tg_3s = 3.35 t/s
14.20.866.255 I slot print_timing: id 2 | task 2206 | n_decoded = 540, tg = 3.36 t/s, tg_3s = 3.35 t/s
14.24.148.578 I slot print_timing: id 2 | task 2206 | n_decoded = 551, tg = 3.36 t/s, tg_3s = 3.35 t/s
14.27.427.263 I slot print_timing: id 2 | task 2206 | n_decoded = 562, tg = 3.36 t/s, tg_3s = 3.36 t/s
14.30.704.568 I slot print_timing: id 2 | task 2206 | n_decoded = 573, tg = 3.36 t/s, tg_3s = 3.36 t/s
14.33.982.824 I slot print_timing: id 2 | task 2206 | n_decoded = 584, tg = 3.36 t/s, tg_3s = 3.36 t/s
14.37.261.272 I slot print_timing: id 2 | task 2206 | n_decoded = 595, tg = 3.36 t/s, tg_3s = 3.36 t/s
14.40.542.457 I slot print_timing: id 2 | task 2206 | n_decoded = 606, tg = 3.36 t/s, tg_3s = 3.35 t/s
14.43.823.514 I slot print_timing: id 2 | task 2206 | n_decoded = 617, tg = 3.36 t/s, tg_3s = 3.35 t/s
14.47.108.052 I slot print_timing: id 2 | task 2206 | n_decoded = 628, tg = 3.36 t/s, tg_3s = 3.35 t/s
14.50.390.866 I slot print_timing: id 2 | task 2206 | n_decoded = 639, tg = 3.36 t/s, tg_3s = 3.35 t/s
14.53.690.792 I slot print_timing: id 2 | task 2206 | n_decoded = 650, tg = 3.36 t/s, tg_3s = 3.33 t/s
14.56.989.422 I slot print_timing: id 2 | task 2206 | n_decoded = 661, tg = 3.36 t/s, tg_3s = 3.33 t/s
14.59.990.069 I slot print_timing: id 2 | task 2206 | n_decoded = 671, tg = 3.36 t/s, tg_3s = 3.33 t/s
15.03.290.077 I slot print_timing: id 2 | task 2206 | n_decoded = 682, tg = 3.36 t/s, tg_3s = 3.33 t/s
15.06.291.156 I slot print_timing: id 2 | task 2206 | n_decoded = 692, tg = 3.36 t/s, tg_3s = 3.33 t/s
15.09.591.083 I slot print_timing: id 2 | task 2206 | n_decoded = 703, tg = 3.36 t/s, tg_3s = 3.33 t/s
15.12.591.234 I slot print_timing: id 2 | task 2206 | n_decoded = 713, tg = 3.36 t/s, tg_3s = 3.33 t/s
15.15.889.550 I slot print_timing: id 2 | task 2206 | n_decoded = 724, tg = 3.36 t/s, tg_3s = 3.34 t/s
15.19.188.174 I slot print_timing: id 2 | task 2206 | n_decoded = 735, tg = 3.36 t/s, tg_3s = 3.33 t/s
15.22.485.350 I slot print_timing: id 2 | task 2206 | n_decoded = 746, tg = 3.36 t/s, tg_3s = 3.34 t/s
15.25.780.816 I slot print_timing: id 2 | task 2206 | n_decoded = 757, tg = 3.36 t/s, tg_3s = 3.34 t/s
15.29.079.357 I slot print_timing: id 2 | task 2206 | n_decoded = 768, tg = 3.36 t/s, tg_3s = 3.33 t/s
15.32.378.374 I slot print_timing: id 2 | task 2206 | n_decoded = 779, tg = 3.36 t/s, tg_3s = 3.33 t/s
15.35.380.514 I slot print_timing: id 2 | task 2206 | n_decoded = 789, tg = 3.36 t/s, tg_3s = 3.33 t/s
15.38.679.931 I slot print_timing: id 2 | task 2206 | n_decoded = 800, tg = 3.36 t/s, tg_3s = 3.33 t/s
15.41.977.534 I slot print_timing: id 2 | task 2206 | n_decoded = 811, tg = 3.36 t/s, tg_3s = 3.34 t/s
15.45.274.105 I slot print_timing: id 2 | task 2206 | n_decoded = 822, tg = 3.36 t/s, tg_3s = 3.34 t/s
15.48.568.102 I slot print_timing: id 2 | task 2206 | n_decoded = 833, tg = 3.36 t/s, tg_3s = 3.34 t/s
15.51.867.056 I slot print_timing: id 2 | task 2206 | n_decoded = 844, tg = 3.36 t/s, tg_3s = 3.33 t/s
15.55.161.994 I slot print_timing: id 2 | task 2206 | n_decoded = 855, tg = 3.35 t/s, tg_3s = 3.34 t/s
15.58.163.409 I slot print_timing: id 2 | task 2206 | n_decoded = 865, tg = 3.35 t/s, tg_3s = 3.33 t/s
16.01.461.533 I slot print_timing: id 2 | task 2206 | n_decoded = 876, tg = 3.35 t/s, tg_3s = 3.34 t/s
16.04.758.154 I slot print_timing: id 2 | task 2206 | prompt eval time = 1937.94 ms / 64 tokens ( 30.28 ms per token, 33.02 tokens per second)
16.04.758.158 I slot print_timing: id 2 | task 2206 | eval time = 264445.21 ms / 887 tokens ( 298.13 ms per token, 3.35 tokens per second)
16.04.758.158 I slot print_timing: id 2 | task 2206 | total time = 266383.15 ms / 951 tokens
16.04.758.159 I slot print_timing: id 2 | task 2206 | graphs reused = 3049
16.04.758.194 I slot release: id 2 | task 2206 | stop processing: n_tokens = 1009, truncated = 0
16.04.758.230 I srv update_slots: all slots are idle
16.04.761.256 I srv operator(): Chat format: peg-native
16.04.761.377 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.628 (> 0.100 thold), f_keep = 0.058
16.04.761.379 I srv get_availabl: updating prompt cache
16.04.761.433 W srv prompt_save: - saving prompt with length 1009, total state size = 346.857 MiB (draft: 0.000 MiB)
16.05.040.007 I srv load: - looking for better prompt, base f_keep = 0.058, sim = 0.628
16.05.040.018 I srv update: - cache state: 10 prompts, 1318.346 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
16.05.040.020 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
16.05.040.020 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
16.05.040.022 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
16.05.040.023 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
16.05.040.026 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
16.05.040.027 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
16.05.040.028 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
16.05.040.028 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
16.05.040.029 I srv update: - prompt 0x5719f2442e40: 525 tokens, checkpoints: 0, 180.477 MiB
16.05.040.030 I srv update: - prompt 0x5719f22d2570: 1009 tokens, checkpoints: 0, 346.857 MiB
16.05.040.031 I srv get_availabl: prompt cache update took 278.65 ms
16.05.040.069 I slot launch_slot_: id 2 | task 3094 | processing task, is_child = 0
16.05.040.070 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
16.05.040.070 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
16.05.040.091 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
16.05.040.092 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
16.05.040.107 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
16.05.040.108 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
16.35.863.017 I slot print_timing: id 2 | task 3094 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s
16.39.133.879 I slot print_timing: id 2 | task 3094 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.36 t/s
16.42.405.111 I slot print_timing: id 2 | task 3094 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.36 t/s
16.45.673.767 I slot print_timing: id 2 | task 3094 | n_decoded = 133, tg = 3.39 t/s, tg_3s = 3.37 t/s
16.48.941.765 I slot print_timing: id 2 | task 3094 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s
16.52.212.228 I slot print_timing: id 2 | task 3094 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.36 t/s
16.55.496.686 I slot print_timing: id 2 | task 3094 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.35 t/s
16.58.780.289 I slot print_timing: id 2 | task 3094 | n_decoded = 177, tg = 3.38 t/s, tg_3s = 3.35 t/s
17.02.061.009 I slot print_timing: id 2 | task 3094 | n_decoded = 188, tg = 3.38 t/s, tg_3s = 3.35 t/s
17.05.340.551 I slot print_timing: id 2 | task 3094 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.35 t/s
17.08.624.056 I slot print_timing: id 2 | task 3094 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.35 t/s
17.11.911.106 I slot print_timing: id 2 | task 3094 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.35 t/s
17.15.193.308 I slot print_timing: id 2 | task 3094 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.35 t/s
17.18.471.502 I slot print_timing: id 2 | task 3094 | n_decoded = 243, tg = 3.37 t/s, tg_3s = 3.36 t/s
17.21.750.428 I slot print_timing: id 2 | task 3094 | n_decoded = 254, tg = 3.37 t/s, tg_3s = 3.35 t/s
17.25.027.514 I slot print_timing: id 2 | task 3094 | n_decoded = 265, tg = 3.37 t/s, tg_3s = 3.36 t/s
17.28.306.302 I slot print_timing: id 2 | task 3094 | n_decoded = 276, tg = 3.37 t/s, tg_3s = 3.35 t/s
17.31.583.191 I slot print_timing: id 2 | task 3094 | n_decoded = 287, tg = 3.37 t/s, tg_3s = 3.36 t/s
17.34.857.518 I slot print_timing: id 2 | task 3094 | n_decoded = 298, tg = 3.37 t/s, tg_3s = 3.36 t/s
17.38.136.942 I slot print_timing: id 2 | task 3094 | n_decoded = 309, tg = 3.37 t/s, tg_3s = 3.35 t/s
17.41.413.699 I slot print_timing: id 2 | task 3094 | n_decoded = 320, tg = 3.37 t/s, tg_3s = 3.36 t/s
17.44.693.812 I slot print_timing: id 2 | task 3094 | n_decoded = 331, tg = 3.37 t/s, tg_3s = 3.35 t/s
17.47.973.835 I slot print_timing: id 2 | task 3094 | n_decoded = 342, tg = 3.37 t/s, tg_3s = 3.35 t/s
17.51.253.863 I slot print_timing: id 2 | task 3094 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.35 t/s
17.54.531.827 I slot print_timing: id 2 | task 3094 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.36 t/s
17.57.807.388 I slot print_timing: id 2 | task 3094 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.36 t/s
18.01.083.349 I slot print_timing: id 2 | task 3094 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.36 t/s
18.04.359.887 I slot print_timing: id 2 | task 3094 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.36 t/s
18.06.739.801 I slot print_timing: id 2 | task 3094 | prompt eval time = 1430.56 ms / 35 tokens ( 40.87 ms per token, 24.47 tokens per second)
18.06.739.805 I slot print_timing: id 2 | task 3094 | eval time = 120269.10 ms / 405 tokens ( 296.96 ms per token, 3.37 tokens per second)
18.06.739.806 I slot print_timing: id 2 | task 3094 | total time = 121699.66 ms / 440 tokens
18.06.739.806 I slot print_timing: id 2 | task 3094 | graphs reused = 3451
18.06.739.844 I slot release: id 2 | task 3094 | stop processing: n_tokens = 498, truncated = 0
18.06.739.854 I srv update_slots: all slots are idle
18.06.742.757 I srv operator(): Chat format: peg-native
18.06.742.878 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.541 (> 0.100 thold), f_keep = 0.118
18.06.742.880 I srv get_availabl: updating prompt cache
18.06.742.919 W srv prompt_save: - saving prompt with length 498, total state size = 171.195 MiB (draft: 0.000 MiB)
18.06.869.945 I srv load: - looking for better prompt, base f_keep = 0.118, sim = 0.541
18.06.869.958 I srv update: - cache state: 11 prompts, 1489.541 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
18.06.869.959 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
18.06.869.960 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
18.06.869.960 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
18.06.869.961 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
18.06.869.961 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
18.06.869.962 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
18.06.869.962 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
18.06.869.963 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
18.06.869.963 I srv update: - prompt 0x5719f2442e40: 525 tokens, checkpoints: 0, 180.477 MiB
18.06.869.964 I srv update: - prompt 0x5719f22d2570: 1009 tokens, checkpoints: 0, 346.857 MiB
18.06.869.964 I srv update: - prompt 0x5719f1da9fc0: 498 tokens, checkpoints: 0, 171.195 MiB
18.06.869.966 I srv get_availabl: prompt cache update took 127.08 ms
18.06.870.010 I slot launch_slot_: id 2 | task 3500 | processing task, is_child = 0
18.06.870.011 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
18.06.870.012 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
18.06.870.028 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
18.06.870.029 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
18.06.870.041 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
18.06.870.042 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
18.38.035.488 I slot print_timing: id 2 | task 3500 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s
18.41.303.552 I slot print_timing: id 2 | task 3500 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s
18.44.571.080 I slot print_timing: id 2 | task 3500 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s
18.47.838.451 I slot print_timing: id 2 | task 3500 | n_decoded = 133, tg = 3.39 t/s, tg_3s = 3.37 t/s
18.51.105.307 I slot print_timing: id 2 | task 3500 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.37 t/s
18.54.388.624 I slot print_timing: id 2 | task 3500 | n_decoded = 155, tg = 3.39 t/s, tg_3s = 3.35 t/s
18.57.664.204 I slot print_timing: id 2 | task 3500 | n_decoded = 166, tg = 3.39 t/s, tg_3s = 3.36 t/s
19.00.941.284 I slot print_timing: id 2 | task 3500 | n_decoded = 177, tg = 3.38 t/s, tg_3s = 3.36 t/s
19.04.218.433 I slot print_timing: id 2 | task 3500 | n_decoded = 188, tg = 3.38 t/s, tg_3s = 3.36 t/s
19.07.492.963 I slot print_timing: id 2 | task 3500 | n_decoded = 199, tg = 3.38 t/s, tg_3s = 3.36 t/s
19.10.768.823 I slot print_timing: id 2 | task 3500 | n_decoded = 210, tg = 3.38 t/s, tg_3s = 3.36 t/s
19.14.053.994 I slot print_timing: id 2 | task 3500 | n_decoded = 221, tg = 3.38 t/s, tg_3s = 3.35 t/s
19.17.337.411 I slot print_timing: id 2 | task 3500 | n_decoded = 232, tg = 3.38 t/s, tg_3s = 3.35 t/s
19.20.617.726 I slot print_timing: id 2 | task 3500 | n_decoded = 243, tg = 3.38 t/s, tg_3s = 3.35 t/s
19.23.901.880 I slot print_timing: id 2 | task 3500 | n_decoded = 254, tg = 3.37 t/s, tg_3s = 3.35 t/s
19.27.182.112 I slot print_timing: id 2 | task 3500 | n_decoded = 265, tg = 3.37 t/s, tg_3s = 3.35 t/s
19.30.459.130 I slot print_timing: id 2 | task 3500 | n_decoded = 276, tg = 3.37 t/s, tg_3s = 3.36 t/s
19.33.740.577 I slot print_timing: id 2 | task 3500 | n_decoded = 287, tg = 3.37 t/s, tg_3s = 3.35 t/s
19.37.021.130 I slot print_timing: id 2 | task 3500 | n_decoded = 298, tg = 3.37 t/s, tg_3s = 3.35 t/s
19.40.297.135 I slot print_timing: id 2 | task 3500 | n_decoded = 309, tg = 3.37 t/s, tg_3s = 3.36 t/s
19.43.573.233 I slot print_timing: id 2 | task 3500 | n_decoded = 320, tg = 3.37 t/s, tg_3s = 3.36 t/s
19.46.851.641 I slot print_timing: id 2 | task 3500 | n_decoded = 331, tg = 3.37 t/s, tg_3s = 3.36 t/s
19.50.129.978 I slot print_timing: id 2 | task 3500 | n_decoded = 342, tg = 3.37 t/s, tg_3s = 3.36 t/s
19.53.410.482 I slot print_timing: id 2 | task 3500 | n_decoded = 353, tg = 3.37 t/s, tg_3s = 3.35 t/s
19.56.692.221 I slot print_timing: id 2 | task 3500 | n_decoded = 364, tg = 3.37 t/s, tg_3s = 3.35 t/s
19.59.970.763 I slot print_timing: id 2 | task 3500 | n_decoded = 375, tg = 3.37 t/s, tg_3s = 3.36 t/s
20.03.247.602 I slot print_timing: id 2 | task 3500 | n_decoded = 386, tg = 3.37 t/s, tg_3s = 3.36 t/s
20.06.527.909 I slot print_timing: id 2 | task 3500 | n_decoded = 397, tg = 3.37 t/s, tg_3s = 3.35 t/s
20.09.816.395 I slot print_timing: id 2 | task 3500 | n_decoded = 408, tg = 3.37 t/s, tg_3s = 3.35 t/s
20.13.093.970 I slot print_timing: id 2 | task 3500 | n_decoded = 419, tg = 3.37 t/s, tg_3s = 3.36 t/s
20.16.372.493 I slot print_timing: id 2 | task 3500 | n_decoded = 430, tg = 3.37 t/s, tg_3s = 3.36 t/s
20.19.651.486 I slot print_timing: id 2 | task 3500 | n_decoded = 441, tg = 3.37 t/s, tg_3s = 3.35 t/s
20.22.930.745 I slot print_timing: id 2 | task 3500 | n_decoded = 452, tg = 3.37 t/s, tg_3s = 3.35 t/s
20.26.211.514 I slot print_timing: id 2 | task 3500 | n_decoded = 463, tg = 3.37 t/s, tg_3s = 3.35 t/s
20.29.492.704 I slot print_timing: id 2 | task 3500 | n_decoded = 474, tg = 3.37 t/s, tg_3s = 3.35 t/s
20.32.775.488 I slot print_timing: id 2 | task 3500 | n_decoded = 485, tg = 3.36 t/s, tg_3s = 3.35 t/s
20.36.054.804 I slot print_timing: id 2 | task 3500 | n_decoded = 496, tg = 3.36 t/s, tg_3s = 3.35 t/s
20.39.333.728 I slot print_timing: id 2 | task 3500 | n_decoded = 507, tg = 3.36 t/s, tg_3s = 3.35 t/s
20.42.616.692 I slot print_timing: id 2 | task 3500 | n_decoded = 518, tg = 3.36 t/s, tg_3s = 3.35 t/s
20.45.896.486 I slot print_timing: id 2 | task 3500 | n_decoded = 529, tg = 3.36 t/s, tg_3s = 3.35 t/s
20.49.172.572 I slot print_timing: id 2 | task 3500 | n_decoded = 540, tg = 3.36 t/s, tg_3s = 3.36 t/s
20.52.451.658 I slot print_timing: id 2 | task 3500 | n_decoded = 551, tg = 3.36 t/s, tg_3s = 3.35 t/s
20.55.729.169 I slot print_timing: id 2 | task 3500 | n_decoded = 562, tg = 3.36 t/s, tg_3s = 3.36 t/s
20.57.519.456 I slot print_timing: id 2 | task 3500 | prompt eval time = 1770.62 ms / 50 tokens ( 35.41 ms per token, 28.24 tokens per second)
20.57.519.460 I slot print_timing: id 2 | task 3500 | eval time = 168878.76 ms / 568 tokens ( 297.32 ms per token, 3.36 tokens per second)
20.57.519.461 I slot print_timing: id 2 | task 3500 | total time = 170649.39 ms / 618 tokens
20.57.519.461 I slot print_timing: id 2 | task 3500 | graphs reused = 4015
20.57.519.488 I slot release: id 2 | task 3500 | stop processing: n_tokens = 676, truncated = 0
20.57.519.498 I srv update_slots: all slots are idle
20.57.522.495 I srv operator(): Chat format: peg-native
20.57.522.602 I slot get_availabl: id 1 | task -1 | selected slot by LRU, t_last = -1
20.57.522.604 I srv get_availabl: updating prompt cache
20.57.522.606 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
20.57.522.609 I srv update: - cache state: 11 prompts, 1489.541 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
20.57.522.610 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
20.57.522.614 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
20.57.522.615 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
20.57.522.615 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
20.57.522.616 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
20.57.522.616 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
20.57.522.617 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
20.57.522.617 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
20.57.522.618 I srv update: - prompt 0x5719f2442e40: 525 tokens, checkpoints: 0, 180.477 MiB
20.57.522.621 I srv update: - prompt 0x5719f22d2570: 1009 tokens, checkpoints: 0, 346.857 MiB
20.57.522.622 I srv update: - prompt 0x5719f1da9fc0: 498 tokens, checkpoints: 0, 171.195 MiB
20.57.522.622 I srv get_availabl: prompt cache update took 0.02 ms
20.57.522.654 I slot launch_slot_: id 1 | task 4069 | processing task, is_child = 0
20.57.522.654 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
20.57.522.655 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
20.57.522.671 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
20.57.522.718 W srv prompt_save: - saving prompt with length 676, total state size = 232.385 MiB (draft: 0.000 MiB)
20.57.711.437 I srv update: - cache state: 12 prompts, 1721.926 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
20.57.711.441 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
20.57.711.441 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
20.57.711.442 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
20.57.711.442 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
20.57.711.443 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
20.57.711.443 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
20.57.711.444 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
20.57.711.444 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
20.57.711.445 I srv update: - prompt 0x5719f2442e40: 525 tokens, checkpoints: 0, 180.477 MiB
20.57.711.445 I srv update: - prompt 0x5719f22d2570: 1009 tokens, checkpoints: 0, 346.857 MiB
20.57.711.446 I srv update: - prompt 0x5719f1da9fc0: 498 tokens, checkpoints: 0, 171.195 MiB
20.57.711.448 I srv update: - prompt 0x5719f3373e40: 676 tokens, checkpoints: 0, 232.385 MiB
20.57.711.449 I slot prompt_clear: id 2 | task -1 | clearing prompt with 676 tokens
20.57.711.560 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
20.57.711.561 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
21.30.785.640 I slot print_timing: id 1 | task 4069 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s
21.31.676.188 I slot print_timing: id 1 | task 4069 | prompt eval time = 3673.38 ms / 125 tokens ( 29.39 ms per token, 34.03 tokens per second)
21.31.676.191 I slot print_timing: id 1 | task 4069 | eval time = 30291.15 ms / 103 tokens ( 294.09 ms per token, 3.40 tokens per second)
21.31.676.192 I slot print_timing: id 1 | task 4069 | total time = 33964.54 ms / 228 tokens
21.31.676.193 I slot print_timing: id 1 | task 4069 | graphs reused = 4116
21.31.676.243 I slot release: id 1 | task 4069 | stop processing: n_tokens = 227, truncated = 0
21.31.676.248 I srv update_slots: all slots are idle
21.31.738.680 I srv operator(): Chat format: peg-native
21.31.738.788 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.706 (> 0.100 thold), f_keep = 0.264
21.31.738.791 I srv get_availabl: updating prompt cache
21.31.738.847 W srv prompt_save: - saving prompt with length 227, total state size = 78.036 MiB (draft: 0.000 MiB)
21.31.799.007 I srv load: - looking for better prompt, base f_keep = 0.264, sim = 0.706
21.31.799.019 I srv update: - cache state: 13 prompts, 1799.961 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
21.31.799.020 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
21.31.799.021 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
21.31.799.022 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
21.31.799.022 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
21.31.799.023 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
21.31.799.024 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
21.31.799.024 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
21.31.799.025 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
21.31.799.026 I srv update: - prompt 0x5719f2442e40: 525 tokens, checkpoints: 0, 180.477 MiB
21.31.799.031 I srv update: - prompt 0x5719f22d2570: 1009 tokens, checkpoints: 0, 346.857 MiB
21.31.799.032 I srv update: - prompt 0x5719f1da9fc0: 498 tokens, checkpoints: 0, 171.195 MiB
21.31.799.033 I srv update: - prompt 0x5719f3373e40: 676 tokens, checkpoints: 0, 232.385 MiB
21.31.799.033 I srv update: - prompt 0x5719f2386a90: 227 tokens, checkpoints: 0, 78.036 MiB
21.31.799.035 I srv get_availabl: prompt cache update took 60.24 ms
21.31.799.092 I slot launch_slot_: id 1 | task 4173 | processing task, is_child = 0
21.31.799.093 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
21.31.799.094 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
21.31.799.114 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
21.31.799.114 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
21.31.799.129 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
21.31.799.129 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
21.46.475.611 I slot print_timing: id 1 | task 4173 | prompt eval time = 1308.11 ms / 25 tokens ( 52.32 ms per token, 19.11 tokens per second)
21.46.475.616 I slot print_timing: id 1 | task 4173 | eval time = 13368.34 ms / 46 tokens ( 290.62 ms per token, 3.44 tokens per second)
21.46.475.617 I slot print_timing: id 1 | task 4173 | total time = 14676.45 ms / 71 tokens
21.46.475.617 I slot print_timing: id 1 | task 4173 | graphs reused = 4160
21.46.475.651 I slot release: id 1 | task 4173 | stop processing: n_tokens = 130, truncated = 0
21.46.475.662 I srv update_slots: all slots are idle
21.46.524.752 I srv operator(): Chat format: peg-native
21.46.524.870 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.645 (> 0.100 thold), f_keep = 0.462
21.46.524.873 I srv get_availabl: updating prompt cache
21.46.524.932 W srv prompt_save: - saving prompt with length 130, total state size = 44.691 MiB (draft: 0.000 MiB)
21.46.563.169 I srv load: - looking for better prompt, base f_keep = 0.462, sim = 0.645
21.46.563.181 I srv update: - cache state: 14 prompts, 1844.652 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
21.46.563.182 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
21.46.563.183 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
21.46.563.183 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
21.46.563.184 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
21.46.563.184 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
21.46.563.185 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
21.46.563.185 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
21.46.563.186 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
21.46.563.186 I srv update: - prompt 0x5719f2442e40: 525 tokens, checkpoints: 0, 180.477 MiB
21.46.563.187 I srv update: - prompt 0x5719f22d2570: 1009 tokens, checkpoints: 0, 346.857 MiB
21.46.563.188 I srv update: - prompt 0x5719f1da9fc0: 498 tokens, checkpoints: 0, 171.195 MiB
21.46.563.189 I srv update: - prompt 0x5719f3373e40: 676 tokens, checkpoints: 0, 232.385 MiB
21.46.563.190 I srv update: - prompt 0x5719f2386a90: 227 tokens, checkpoints: 0, 78.036 MiB
21.46.563.191 I srv update: - prompt 0x5719f1d67110: 130 tokens, checkpoints: 0, 44.691 MiB
21.46.563.192 I srv get_availabl: prompt cache update took 38.32 ms
21.46.563.249 I slot launch_slot_: id 1 | task 4220 | processing task, is_child = 0
21.46.563.251 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
21.46.563.252 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
21.46.563.271 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
21.46.563.272 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
21.46.563.287 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
21.46.563.287 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
22.01.664.405 I slot print_timing: id 1 | task 4220 | prompt eval time = 1429.99 ms / 33 tokens ( 43.33 ms per token, 23.08 tokens per second)
22.01.664.409 I slot print_timing: id 1 | task 4220 | eval time = 13671.09 ms / 47 tokens ( 290.87 ms per token, 3.44 tokens per second)
22.01.664.410 I slot print_timing: id 1 | task 4220 | total time = 15101.09 ms / 80 tokens
22.01.664.411 I slot print_timing: id 1 | task 4220 | graphs reused = 4205
22.01.664.458 I slot release: id 1 | task 4220 | stop processing: n_tokens = 139, truncated = 0
22.01.664.468 I srv update_slots: all slots are idle
22.01.740.194 I srv operator(): Chat format: peg-native
22.01.740.322 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.484 (> 0.100 thold), f_keep = 0.432
22.01.740.323 I srv get_availabl: updating prompt cache
22.01.740.381 W srv prompt_save: - saving prompt with length 139, total state size = 47.785 MiB (draft: 0.000 MiB)
22.01.779.420 I srv load: - looking for better prompt, base f_keep = 0.432, sim = 0.484
22.01.779.431 I srv update: - cache state: 15 prompts, 1892.437 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
22.01.779.432 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
22.01.779.433 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
22.01.779.434 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
22.01.779.434 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
22.01.779.435 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
22.01.779.435 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
22.01.779.437 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
22.01.779.438 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
22.01.779.438 I srv update: - prompt 0x5719f2442e40: 525 tokens, checkpoints: 0, 180.477 MiB
22.01.779.439 I srv update: - prompt 0x5719f22d2570: 1009 tokens, checkpoints: 0, 346.857 MiB
22.01.779.439 I srv update: - prompt 0x5719f1da9fc0: 498 tokens, checkpoints: 0, 171.195 MiB
22.01.779.440 I srv update: - prompt 0x5719f3373e40: 676 tokens, checkpoints: 0, 232.385 MiB
22.01.779.440 I srv update: - prompt 0x5719f2386a90: 227 tokens, checkpoints: 0, 78.036 MiB
22.01.779.441 I srv update: - prompt 0x5719f1d67110: 130 tokens, checkpoints: 0, 44.691 MiB
22.01.779.441 I srv update: - prompt 0x5719f235eed0: 139 tokens, checkpoints: 0, 47.785 MiB
22.01.779.442 I srv get_availabl: prompt cache update took 39.12 ms
22.01.779.497 I slot launch_slot_: id 1 | task 4268 | processing task, is_child = 0
22.01.779.498 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
22.01.779.499 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
22.01.779.519 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
22.01.779.519 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
22.01.779.535 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
22.01.779.535 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
22.33.125.543 I slot print_timing: id 1 | task 4268 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s
22.36.393.168 I slot print_timing: id 1 | task 4268 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.37 t/s
22.39.665.297 I slot print_timing: id 1 | task 4268 | n_decoded = 122, tg = 3.39 t/s, tg_3s = 3.36 t/s
22.42.931.915 I slot print_timing: id 1 | task 4268 | n_decoded = 133, tg = 3.39 t/s, tg_3s = 3.37 t/s
22.46.215.577 I slot print_timing: id 1 | task 4268 | n_decoded = 144, tg = 3.39 t/s, tg_3s = 3.35 t/s
22.49.492.569 I slot print_timing: id 1 | task 4268 | prompt eval time = 1938.63 ms / 64 tokens ( 30.29 ms per token, 33.01 tokens per second)
22.49.492.572 I slot print_timing: id 1 | task 4268 | eval time = 45774.37 ms / 155 tokens ( 295.32 ms per token, 3.39 tokens per second)
22.49.492.573 I slot print_timing: id 1 | task 4268 | total time = 47713.00 ms / 219 tokens
22.49.492.573 I slot print_timing: id 1 | task 4268 | graphs reused = 4357
22.49.492.613 I slot release: id 1 | task 4268 | stop processing: n_tokens = 278, truncated = 0
22.49.492.621 I srv update_slots: all slots are idle
22.49.566.568 I srv operator(): Chat format: peg-native
22.49.566.693 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.612 (> 0.100 thold), f_keep = 0.216
22.49.566.695 I srv get_availabl: updating prompt cache
22.49.566.751 W srv prompt_save: - saving prompt with length 278, total state size = 95.568 MiB (draft: 0.000 MiB)
22.49.639.563 I srv load: - looking for better prompt, base f_keep = 0.216, sim = 0.612
22.49.639.577 I srv update: - cache state: 16 prompts, 1988.005 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
22.49.639.578 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
22.49.639.579 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
22.49.639.580 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
22.49.639.580 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
22.49.639.581 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
22.49.639.581 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
22.49.639.582 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
22.49.639.582 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
22.49.639.583 I srv update: - prompt 0x5719f2442e40: 525 tokens, checkpoints: 0, 180.477 MiB
22.49.639.583 I srv update: - prompt 0x5719f22d2570: 1009 tokens, checkpoints: 0, 346.857 MiB
22.49.639.584 I srv update: - prompt 0x5719f1da9fc0: 498 tokens, checkpoints: 0, 171.195 MiB
22.49.639.584 I srv update: - prompt 0x5719f3373e40: 676 tokens, checkpoints: 0, 232.385 MiB
22.49.639.585 I srv update: - prompt 0x5719f2386a90: 227 tokens, checkpoints: 0, 78.036 MiB
22.49.639.585 I srv update: - prompt 0x5719f1d67110: 130 tokens, checkpoints: 0, 44.691 MiB
22.49.639.586 I srv update: - prompt 0x5719f235eed0: 139 tokens, checkpoints: 0, 47.785 MiB
22.49.639.586 I srv update: - prompt 0x5719f3395180: 278 tokens, checkpoints: 0, 95.568 MiB
22.49.639.588 I srv get_availabl: prompt cache update took 72.89 ms
22.49.639.642 I slot launch_slot_: id 1 | task 4424 | processing task, is_child = 0
22.49.639.643 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
22.49.639.644 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
22.49.639.660 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
22.49.639.660 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
22.49.639.672 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
22.49.639.673 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
23.02.666.140 I slot print_timing: id 1 | task 4424 | prompt eval time = 1433.36 ms / 38 tokens ( 37.72 ms per token, 26.51 tokens per second)
23.02.666.144 I slot print_timing: id 1 | task 4424 | eval time = 11593.08 ms / 40 tokens ( 289.83 ms per token, 3.45 tokens per second)
23.02.666.144 I slot print_timing: id 1 | task 4424 | total time = 13026.44 ms / 78 tokens
23.02.666.145 I slot print_timing: id 1 | task 4424 | graphs reused = 4395
23.02.666.180 I slot release: id 1 | task 4424 | stop processing: n_tokens = 137, truncated = 0
23.02.666.188 I srv update_slots: all slots are idle
23.02.713.301 I srv operator(): Chat format: peg-native
23.02.713.420 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.524 (> 0.100 thold), f_keep = 0.401
23.02.713.422 I srv get_availabl: updating prompt cache
23.02.713.458 W srv prompt_save: - saving prompt with length 137, total state size = 47.097 MiB (draft: 0.000 MiB)
23.02.753.256 I srv load: - looking for better prompt, base f_keep = 0.401, sim = 0.524
23.02.753.267 I srv update: - cache state: 17 prompts, 2035.102 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
23.02.753.268 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
23.02.753.269 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
23.02.753.270 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
23.02.753.270 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
23.02.753.271 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
23.02.753.271 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
23.02.753.272 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
23.02.753.272 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
23.02.753.273 I srv update: - prompt 0x5719f2442e40: 525 tokens, checkpoints: 0, 180.477 MiB
23.02.753.273 I srv update: - prompt 0x5719f22d2570: 1009 tokens, checkpoints: 0, 346.857 MiB
23.02.753.274 I srv update: - prompt 0x5719f1da9fc0: 498 tokens, checkpoints: 0, 171.195 MiB
23.02.753.274 I srv update: - prompt 0x5719f3373e40: 676 tokens, checkpoints: 0, 232.385 MiB
23.02.753.275 I srv update: - prompt 0x5719f2386a90: 227 tokens, checkpoints: 0, 78.036 MiB
23.02.753.275 I srv update: - prompt 0x5719f1d67110: 130 tokens, checkpoints: 0, 44.691 MiB
23.02.753.275 I srv update: - prompt 0x5719f235eed0: 139 tokens, checkpoints: 0, 47.785 MiB
23.02.753.276 I srv update: - prompt 0x5719f3395180: 278 tokens, checkpoints: 0, 95.568 MiB
23.02.753.277 I srv update: - prompt 0x5719f2323520: 137 tokens, checkpoints: 0, 47.097 MiB
23.02.753.284 I srv get_availabl: prompt cache update took 39.86 ms
23.02.753.336 I slot launch_slot_: id 1 | task 4465 | processing task, is_child = 0
23.02.753.337 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
23.02.753.337 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
23.02.753.355 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
23.02.753.355 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
23.02.753.368 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
23.02.753.368 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
23.11.951.976 I slot print_timing: id 1 | task 4465 | prompt eval time = 1771.09 ms / 50 tokens ( 35.42 ms per token, 28.23 tokens per second)
23.11.951.980 I slot print_timing: id 1 | task 4465 | eval time = 7427.48 ms / 26 tokens ( 285.67 ms per token, 3.50 tokens per second)
23.11.951.981 I slot print_timing: id 1 | task 4465 | total time = 9198.58 ms / 76 tokens
23.11.951.981 I slot print_timing: id 1 | task 4465 | graphs reused = 4419
23.11.952.027 I slot release: id 1 | task 4465 | stop processing: n_tokens = 130, truncated = 0
23.11.952.039 I srv update_slots: all slots are idle
23.11.954.731 I srv operator(): Chat format: peg-native
23.11.954.840 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.591 (> 0.100 thold), f_keep = 0.423
23.11.954.841 I srv get_availabl: updating prompt cache
23.11.954.890 W srv prompt_save: - saving prompt with length 130, total state size = 44.691 MiB (draft: 0.000 MiB)
23.11.992.096 I srv load: - looking for better prompt, base f_keep = 0.423, sim = 0.591
23.11.992.106 I srv load: - found better prompt with f_keep = 0.462, sim = 0.645
23.12.013.372 I srv update: - cache state: 17 prompts, 2035.102 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
23.12.013.376 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
23.12.013.377 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
23.12.013.378 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
23.12.013.378 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
23.12.013.379 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
23.12.013.379 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
23.12.013.380 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
23.12.013.380 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
23.12.013.381 I srv update: - prompt 0x5719f2442e40: 525 tokens, checkpoints: 0, 180.477 MiB
23.12.013.381 I srv update: - prompt 0x5719f22d2570: 1009 tokens, checkpoints: 0, 346.857 MiB
23.12.013.382 I srv update: - prompt 0x5719f1da9fc0: 498 tokens, checkpoints: 0, 171.195 MiB
23.12.013.382 I srv update: - prompt 0x5719f3373e40: 676 tokens, checkpoints: 0, 232.385 MiB
23.12.013.383 I srv update: - prompt 0x5719f2386a90: 227 tokens, checkpoints: 0, 78.036 MiB
23.12.013.383 I srv update: - prompt 0x5719f235eed0: 139 tokens, checkpoints: 0, 47.785 MiB
23.12.013.384 I srv update: - prompt 0x5719f3395180: 278 tokens, checkpoints: 0, 95.568 MiB
23.12.013.384 I srv update: - prompt 0x5719f2323520: 137 tokens, checkpoints: 0, 47.097 MiB
23.12.013.385 I srv update: - prompt 0x5719f22e9320: 130 tokens, checkpoints: 0, 44.691 MiB
23.12.013.386 I srv get_availabl: prompt cache update took 58.54 ms
23.12.013.422 I slot launch_slot_: id 1 | task 4492 | processing task, is_child = 0
23.12.013.423 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
23.12.013.424 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
23.12.013.441 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
23.12.013.441 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
23.12.013.453 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
23.12.013.454 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
23.42.835.139 I slot print_timing: id 1 | task 4492 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s
23.46.104.165 I slot print_timing: id 1 | task 4492 | n_decoded = 111, tg = 3.40 t/s, tg_3s = 3.36 t/s
23.49.370.896 I slot print_timing: id 1 | task 4492 | n_decoded = 122, tg = 3.40 t/s, tg_3s = 3.37 t/s
23.49.964.185 I slot print_timing: id 1 | task 4492 | prompt eval time = 1430.34 ms / 33 tokens ( 43.34 ms per token, 23.07 tokens per second)
23.49.964.188 I slot print_timing: id 1 | task 4492 | eval time = 36520.36 ms / 124 tokens ( 294.52 ms per token, 3.40 tokens per second)
23.49.964.189 I slot print_timing: id 1 | task 4492 | total time = 37950.71 ms / 157 tokens
23.49.964.190 I slot print_timing: id 1 | task 4492 | graphs reused = 4541
23.49.964.232 I slot release: id 1 | task 4492 | stop processing: n_tokens = 216, truncated = 0
23.49.964.240 I srv update_slots: all slots are idle
23.50.027.520 I srv operator(): Chat format: peg-native
23.50.027.662 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.645 (> 0.100 thold), f_keep = 0.278
23.50.027.664 I srv get_availabl: updating prompt cache
23.50.027.717 W srv prompt_save: - saving prompt with length 216, total state size = 74.255 MiB (draft: 0.000 MiB)
23.50.085.356 I srv load: - looking for better prompt, base f_keep = 0.278, sim = 0.645
23.50.085.370 I srv update: - cache state: 18 prompts, 2109.357 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
23.50.085.371 I srv update: - prompt 0x5719f2358e50: 107 tokens, checkpoints: 0, 36.785 MiB
23.50.085.372 I srv update: - prompt 0x5719f22b68f0: 265 tokens, checkpoints: 0, 91.099 MiB
23.50.085.373 I srv update: - prompt 0x5719f1fdbdc0: 96 tokens, checkpoints: 0, 33.003 MiB
23.50.085.374 I srv update: - prompt 0x5719f244c2e0: 152 tokens, checkpoints: 0, 52.254 MiB
23.50.085.376 I srv update: - prompt 0x5719f1fb8430: 141 tokens, checkpoints: 0, 48.472 MiB
23.50.085.377 I srv update: - prompt 0x5719f2314ca0: 82 tokens, checkpoints: 0, 28.190 MiB
23.50.085.377 I srv update: - prompt 0x5719f22f7ba0: 747 tokens, checkpoints: 0, 256.792 MiB
23.50.085.378 I srv update: - prompt 0x5719f2077ce0: 711 tokens, checkpoints: 0, 244.416 MiB
23.50.085.378 I srv update: - prompt 0x5719f2442e40: 525 tokens, checkpoints: 0, 180.477 MiB
23.50.085.379 I srv update: - prompt 0x5719f22d2570: 1009 tokens, checkpoints: 0, 346.857 MiB
23.50.085.379 I srv update: - prompt 0x5719f1da9fc0: 498 tokens, checkpoints: 0, 171.195 MiB
23.50.085.380 I srv update: - prompt 0x5719f3373e40: 676 tokens, checkpoints: 0, 232.385 MiB
23.50.085.380 I srv update: - prompt 0x5719f2386a90: 227 tokens, checkpoints: 0, 78.036 MiB
23.50.085.381 I srv update: - prompt 0x5719f235eed0: 139 tokens, checkpoints: 0, 47.785 MiB
23.50.085.381 I srv update: - prompt 0x5719f3395180: 278 tokens, checkpoints: 0, 95.568 MiB
23.50.085.382 I srv update: - prompt 0x5719f2323520: 137 tokens, checkpoints: 0, 47.097 MiB
23.50.085.382 I srv update: - prompt 0x5719f22e9320: 130 tokens, checkpoints: 0, 44.691 MiB
23.50.085.383 I srv update: - prompt 0x5719f233b0c0: 216 tokens, checkpoints: 0, 74.255 MiB
23.50.085.384 I srv get_availabl: prompt cache update took 57.72 ms
23.50.085.435 I slot launch_slot_: id 1 | task 4617 | processing task, is_child = 0
23.50.085.436 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
23.50.085.437 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
23.50.085.455 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
23.50.085.455 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
23.50.085.468 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
23.50.085.468 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
24.06.976.072 I slot print_timing: id 1 | task 4617 | prompt eval time = 1430.13 ms / 33 tokens ( 43.34 ms per token, 23.07 tokens per second)
24.06.976.075 I slot print_timing: id 1 | task 4617 | eval time = 15460.45 ms / 53 tokens ( 291.71 ms per token, 3.43 tokens per second)
24.06.976.076 I slot print_timing: id 1 | task 4617 | total time = 16890.58 ms / 86 tokens
24.06.976.076 I slot print_timing: id 1 | task 4617 | graphs reused = 4592
24.06.976.111 I slot release: id 1 | task 4617 | stop processing: n_tokens = 145, truncated = 0
24.06.976.119 I srv update_slots: all slots are idle
24.07.036.835 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink