qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results_hard/aya-23-8b.server.log
G gael 5cb7da3855 readme report
2026-07-13 01:35:10 +00:00

809 lines
79 KiB
Plaintext
Raw Blame History

0.00.065.549 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.065.552 I device_info:
0.00.065.570 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.574 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.582 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.065.638 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.065.641 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.065.681 I srv init: running without SSL
0.00.065.740 I srv init: using 23 threads for HTTP server
0.00.066.062 I srv start: binding port with default address family
0.00.067.271 I srv llama_server: loading model
0.00.067.281 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/aya-23-8B-Q5_K_M.gguf'
0.00.067.336 I common_init_result: fitting params to device memory ...
0.00.067.339 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.01.419.056 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
0.04.791.795 W llama_context: n_ctx_seq (16384) > n_ctx_train (8192) -- possible training context overflow
0.04.836.401 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.04.964.870 I srv load_model: initializing slots, n_slots = 4
0.04.964.876 W srv load_model: the slot context (16384) exceeds the training context of the model (8192) - capping
0.05.016.093 W common_speculative_init: no implementations specified for speculative decoding
0.05.016.098 I slot load_model: id 0 | task -1 | new slot, n_ctx = 8192
0.05.016.103 I slot load_model: id 1 | task -1 | new slot, n_ctx = 8192
0.05.016.103 I slot load_model: id 2 | task -1 | new slot, n_ctx = 8192
0.05.016.104 I slot load_model: id 3 | task -1 | new slot, n_ctx = 8192
0.05.016.184 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.05.016.185 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.05.016.186 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.05.016.186 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.05.016.206 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.05.020.865 I init: chat template, example_format: '<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>You are a helpful assistant<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Hello<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>Hi there<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>How are you?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>'
0.05.022.443 I srv init: init: chat template, thinking = 0
0.05.022.479 I srv llama_server: model loaded
0.05.022.483 I srv llama_server: server is listening on http://127.0.0.1:8090
0.05.022.488 I srv update_slots: all slots are idle
0.09.965.590 I srv operator(): Chat format: peg-native
0.09.965.887 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.09.965.889 I srv get_availabl: updating prompt cache
0.09.965.895 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.09.965.900 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est)
0.09.965.900 I srv get_availabl: prompt cache update took 0.01 ms
0.09.965.952 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.09.965.953 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.09.965.954 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.09.965.972 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.09.965.972 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.09.965.985 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.09.965.986 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.13.236.235 I slot print_timing: id 3 | task 0 | n_decoded = 144, tg = 47.82 t/s, tg_3s = 47.82 t/s
0.16.246.297 I slot print_timing: id 3 | task 0 | n_decoded = 287, tg = 47.66 t/s, tg_3s = 47.51 t/s
0.19.252.202 I slot print_timing: id 3 | task 0 | n_decoded = 430, tg = 47.63 t/s, tg_3s = 47.57 t/s
0.19.926.198 I slot print_timing: id 3 | task 0 | prompt eval time = 258.92 ms / 92 tokens ( 2.81 ms per token, 355.32 tokens per second)
0.19.926.201 I slot print_timing: id 3 | task 0 | eval time = 9701.26 ms / 462 tokens ( 21.00 ms per token, 47.62 tokens per second)
0.19.926.202 I slot print_timing: id 3 | task 0 | total time = 9960.18 ms / 554 tokens
0.19.926.205 I slot print_timing: id 3 | task 0 | graphs reused = 459
0.19.926.247 I slot release: id 3 | task 0 | stop processing: n_tokens = 553, truncated = 0
0.19.926.254 I srv update_slots: all slots are idle
0.19.929.710 I srv operator(): Chat format: peg-native
0.19.929.970 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.816 (> 0.100 thold), f_keep = 0.128
0.19.929.971 I srv get_availabl: updating prompt cache
0.19.930.012 W srv prompt_save: - saving prompt with length 553, total state size = 69.132 MiB (draft: 0.000 MiB)
0.20.016.624 I srv load: - looking for better prompt, base f_keep = 0.128, sim = 0.816
0.20.016.631 I srv update: - cache state: 1 prompts, 69.132 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
0.20.016.632 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
0.20.016.632 I srv get_availabl: prompt cache update took 86.66 ms
0.20.016.679 I slot launch_slot_: id 3 | task 464 | processing task, is_child = 0
0.20.016.680 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.20.016.681 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.20.016.692 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.20.016.692 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.20.016.700 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.20.016.701 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.23.134.311 I slot print_timing: id 3 | task 464 | n_decoded = 144, tg = 47.74 t/s, tg_3s = 47.74 t/s
0.25.300.168 I slot print_timing: id 3 | task 464 | prompt eval time = 101.41 ms / 16 tokens ( 6.34 ms per token, 157.78 tokens per second)
0.25.300.172 I slot print_timing: id 3 | task 464 | eval time = 5182.04 ms / 247 tokens ( 20.98 ms per token, 47.66 tokens per second)
0.25.300.172 I slot print_timing: id 3 | task 464 | total time = 5283.45 ms / 263 tokens
0.25.300.173 I slot print_timing: id 3 | task 464 | graphs reused = 703
0.25.300.196 I slot release: id 3 | task 464 | stop processing: n_tokens = 333, truncated = 0
0.25.300.203 I srv update_slots: all slots are idle
0.25.303.385 I srv operator(): Chat format: peg-native
0.25.303.640 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.816 (> 0.100 thold), f_keep = 0.213
0.25.303.641 I srv get_availabl: updating prompt cache
0.25.303.670 W srv prompt_save: - saving prompt with length 333, total state size = 41.630 MiB (draft: 0.000 MiB)
0.25.336.511 I srv load: - looking for better prompt, base f_keep = 0.213, sim = 0.816
0.25.336.520 I srv update: - cache state: 2 prompts, 110.762 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
0.25.336.521 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
0.25.336.522 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
0.25.336.524 I srv get_availabl: prompt cache update took 32.88 ms
0.25.336.573 I slot launch_slot_: id 3 | task 712 | processing task, is_child = 0
0.25.336.574 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.25.336.575 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.25.336.588 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.25.336.588 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.25.336.597 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.25.336.597 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.28.447.217 I slot print_timing: id 3 | task 712 | n_decoded = 145, tg = 48.18 t/s, tg_3s = 48.18 t/s
0.28.551.600 I slot print_timing: id 3 | task 712 | prompt eval time = 101.34 ms / 16 tokens ( 6.33 ms per token, 157.89 tokens per second)
0.28.551.603 I slot print_timing: id 3 | task 712 | eval time = 3113.64 ms / 150 tokens ( 20.76 ms per token, 48.18 tokens per second)
0.28.551.604 I slot print_timing: id 3 | task 712 | total time = 3214.98 ms / 166 tokens
0.28.551.604 I slot print_timing: id 3 | task 712 | graphs reused = 851
0.28.551.629 I slot release: id 3 | task 712 | stop processing: n_tokens = 236, truncated = 0
0.28.551.636 I srv update_slots: all slots are idle
0.28.555.106 I srv operator(): Chat format: peg-native
0.28.555.365 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.835 (> 0.100 thold), f_keep = 0.301
0.28.555.367 I srv get_availabl: updating prompt cache
0.28.555.397 W srv prompt_save: - saving prompt with length 236, total state size = 29.503 MiB (draft: 0.000 MiB)
0.28.579.517 I srv load: - looking for better prompt, base f_keep = 0.301, sim = 0.835
0.28.579.527 I srv update: - cache state: 3 prompts, 140.265 MiB (limits: 8192.000 MiB, 16384 tokens, 65528 est)
0.28.579.528 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
0.28.579.529 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
0.28.579.530 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
0.28.579.531 I srv get_availabl: prompt cache update took 24.16 ms
0.28.579.581 I slot launch_slot_: id 3 | task 863 | processing task, is_child = 0
0.28.579.583 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.28.579.584 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.28.579.597 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.28.579.597 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.28.579.605 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.28.579.606 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.31.693.488 I slot print_timing: id 3 | task 863 | n_decoded = 145, tg = 48.12 t/s, tg_3s = 48.12 t/s
0.33.016.601 I slot print_timing: id 3 | task 863 | prompt eval time = 100.60 ms / 14 tokens ( 7.19 ms per token, 139.17 tokens per second)
0.33.016.604 I slot print_timing: id 3 | task 863 | eval time = 4336.37 ms / 208 tokens ( 20.85 ms per token, 47.97 tokens per second)
0.33.016.604 I slot print_timing: id 3 | task 863 | total time = 4436.97 ms / 222 tokens
0.33.016.605 I slot print_timing: id 3 | task 863 | graphs reused = 1056
0.33.016.628 I slot release: id 3 | task 863 | stop processing: n_tokens = 292, truncated = 0
0.33.016.635 I srv update_slots: all slots are idle
0.33.019.972 I srv operator(): Chat format: peg-native
0.33.020.228 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.717 (> 0.100 thold), f_keep = 0.243
0.33.020.229 I srv get_availabl: updating prompt cache
0.33.020.254 W srv prompt_save: - saving prompt with length 292, total state size = 36.504 MiB (draft: 0.000 MiB)
0.33.049.611 I srv load: - looking for better prompt, base f_keep = 0.243, sim = 0.717
0.33.049.619 I srv update: - cache state: 4 prompts, 176.769 MiB (limits: 8192.000 MiB, 16384 tokens, 65528 est)
0.33.049.619 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
0.33.049.620 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
0.33.049.621 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
0.33.049.621 I srv update: - prompt 0x5cf416e3c850: 292 tokens, checkpoints: 0, 36.504 MiB
0.33.049.622 I srv get_availabl: prompt cache update took 29.39 ms
0.33.049.673 I slot launch_slot_: id 3 | task 1072 | processing task, is_child = 0
0.33.049.673 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.33.049.675 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.33.049.687 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.33.049.687 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.33.049.695 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.33.049.696 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.36.189.152 I slot print_timing: id 3 | task 1072 | n_decoded = 145, tg = 48.26 t/s, tg_3s = 48.26 t/s
0.39.194.638 I slot print_timing: id 3 | task 1072 | n_decoded = 288, tg = 47.92 t/s, tg_3s = 47.58 t/s
0.42.201.715 I slot print_timing: id 3 | task 1072 | n_decoded = 431, tg = 47.80 t/s, tg_3s = 47.55 t/s
0.44.238.573 I slot print_timing: id 3 | task 1072 | prompt eval time = 134.65 ms / 28 tokens ( 4.81 ms per token, 207.94 tokens per second)
0.44.238.576 I slot print_timing: id 3 | task 1072 | eval time = 11054.20 ms / 528 tokens ( 20.94 ms per token, 47.76 tokens per second)
0.44.238.577 I slot print_timing: id 3 | task 1072 | total time = 11188.85 ms / 556 tokens
0.44.238.577 I slot print_timing: id 3 | task 1072 | graphs reused = 1580
0.44.238.602 I slot release: id 3 | task 1072 | stop processing: n_tokens = 626, truncated = 0
0.44.238.611 I srv update_slots: all slots are idle
0.44.241.925 I srv operator(): Chat format: peg-native
0.44.242.181 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.816 (> 0.100 thold), f_keep = 0.113
0.44.242.183 I srv get_availabl: updating prompt cache
0.44.242.216 W srv prompt_save: - saving prompt with length 626, total state size = 78.258 MiB (draft: 0.000 MiB)
0.44.333.695 I srv load: - looking for better prompt, base f_keep = 0.113, sim = 0.816
0.44.333.700 I srv load: - found better prompt with f_keep = 0.253, sim = 0.851
0.44.349.294 I srv update: - cache state: 4 prompts, 218.523 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
0.44.349.297 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
0.44.349.298 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
0.44.349.299 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
0.44.349.299 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
0.44.349.300 I srv get_availabl: prompt cache update took 107.12 ms
0.44.349.353 I slot launch_slot_: id 3 | task 1601 | processing task, is_child = 0
0.44.349.354 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.44.349.355 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.44.349.367 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.44.349.367 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.44.349.379 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.44.349.380 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.47.452.647 I slot print_timing: id 3 | task 1601 | n_decoded = 144, tg = 47.96 t/s, tg_3s = 47.96 t/s
0.50.458.696 I slot print_timing: id 3 | task 1601 | n_decoded = 287, tg = 47.77 t/s, tg_3s = 47.57 t/s
0.53.464.198 I slot print_timing: id 3 | task 1601 | n_decoded = 430, tg = 47.70 t/s, tg_3s = 47.58 t/s
0.54.828.825 I slot print_timing: id 3 | task 1601 | prompt eval time = 100.90 ms / 13 tokens ( 7.76 ms per token, 128.84 tokens per second)
0.54.828.829 I slot print_timing: id 3 | task 1601 | eval time = 10378.52 ms / 495 tokens ( 20.97 ms per token, 47.69 tokens per second)
0.54.828.829 I slot print_timing: id 3 | task 1601 | total time = 10479.42 ms / 508 tokens
0.54.828.830 I slot print_timing: id 3 | task 1601 | graphs reused = 2071
0.54.828.855 I slot release: id 3 | task 1601 | stop processing: n_tokens = 581, truncated = 0
0.54.828.864 I srv update_slots: all slots are idle
0.54.832.330 I srv operator(): Chat format: peg-native
0.54.832.599 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.857 (> 0.100 thold), f_keep = 0.124
0.54.832.602 I srv get_availabl: updating prompt cache
0.54.832.643 W srv prompt_save: - saving prompt with length 581, total state size = 72.632 MiB (draft: 0.000 MiB)
0.54.908.855 I srv load: - looking for better prompt, base f_keep = 0.124, sim = 0.857
0.54.908.863 I srv update: - cache state: 5 prompts, 291.155 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
0.54.908.864 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
0.54.908.865 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
0.54.908.865 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
0.54.908.866 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
0.54.908.866 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
0.54.908.868 I srv get_availabl: prompt cache update took 76.27 ms
0.54.908.908 I slot launch_slot_: id 3 | task 2097 | processing task, is_child = 0
0.54.908.908 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.54.908.909 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.54.908.919 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.54.908.919 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.54.908.940 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.54.908.940 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.58.015.707 I slot print_timing: id 3 | task 2097 | n_decoded = 144, tg = 47.90 t/s, tg_3s = 47.90 t/s
1.01.020.746 I slot print_timing: id 3 | task 2097 | n_decoded = 287, tg = 47.75 t/s, tg_3s = 47.59 t/s
1.02.217.835 I slot print_timing: id 3 | task 2097 | prompt eval time = 100.69 ms / 12 tokens ( 8.39 ms per token, 119.18 tokens per second)
1.02.217.839 I slot print_timing: id 3 | task 2097 | eval time = 7208.18 ms / 344 tokens ( 20.95 ms per token, 47.72 tokens per second)
1.02.217.839 I slot print_timing: id 3 | task 2097 | total time = 7308.86 ms / 356 tokens
1.02.217.840 I slot print_timing: id 3 | task 2097 | graphs reused = 2412
1.02.217.863 I slot release: id 3 | task 2097 | stop processing: n_tokens = 427, truncated = 0
1.02.217.870 I srv update_slots: all slots are idle
1.02.221.394 I srv operator(): Chat format: peg-native
1.02.221.652 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.837 (> 0.100 thold), f_keep = 0.169
1.02.221.654 I srv get_availabl: updating prompt cache
1.02.221.687 W srv prompt_save: - saving prompt with length 427, total state size = 53.381 MiB (draft: 0.000 MiB)
1.02.262.751 I srv load: - looking for better prompt, base f_keep = 0.169, sim = 0.837
1.02.262.760 I srv update: - cache state: 6 prompts, 344.536 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
1.02.262.761 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
1.02.262.762 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
1.02.262.762 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
1.02.262.763 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
1.02.262.763 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
1.02.262.764 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
1.02.262.766 I srv get_availabl: prompt cache update took 41.11 ms
1.02.262.817 I slot launch_slot_: id 3 | task 2442 | processing task, is_child = 0
1.02.262.819 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.02.262.819 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.02.262.831 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.02.262.832 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.02.262.840 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.02.262.841 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.05.371.766 I slot print_timing: id 3 | task 2442 | n_decoded = 144, tg = 47.87 t/s, tg_3s = 47.87 t/s
1.07.182.838 I slot print_timing: id 3 | task 2442 | prompt eval time = 100.83 ms / 14 tokens ( 7.20 ms per token, 138.84 tokens per second)
1.07.182.841 I slot print_timing: id 3 | task 2442 | eval time = 4819.14 ms / 230 tokens ( 20.95 ms per token, 47.73 tokens per second)
1.07.182.842 I slot print_timing: id 3 | task 2442 | total time = 4919.98 ms / 244 tokens
1.07.182.842 I slot print_timing: id 3 | task 2442 | graphs reused = 2639
1.07.182.866 I slot release: id 3 | task 2442 | stop processing: n_tokens = 315, truncated = 0
1.07.182.872 I srv update_slots: all slots are idle
1.07.186.310 I srv operator(): Chat format: peg-native
1.07.186.578 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.617 (> 0.100 thold), f_keep = 0.225
1.07.186.579 I srv get_availabl: updating prompt cache
1.07.186.608 W srv prompt_save: - saving prompt with length 315, total state size = 39.379 MiB (draft: 0.000 MiB)
1.07.217.987 I srv load: - looking for better prompt, base f_keep = 0.225, sim = 0.617
1.07.217.997 I srv update: - cache state: 7 prompts, 383.915 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
1.07.217.998 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
1.07.217.999 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
1.07.218.000 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
1.07.218.000 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
1.07.218.000 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
1.07.218.001 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
1.07.218.001 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
1.07.218.003 I srv get_availabl: prompt cache update took 31.42 ms
1.07.218.053 I slot launch_slot_: id 3 | task 2673 | processing task, is_child = 0
1.07.218.054 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.07.218.054 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.07.218.067 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.07.218.067 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.07.218.076 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.07.218.076 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.10.342.465 I slot print_timing: id 3 | task 2673 | n_decoded = 144, tg = 47.86 t/s, tg_3s = 47.86 t/s
1.12.676.993 I slot print_timing: id 3 | task 2673 | prompt eval time = 115.58 ms / 44 tokens ( 2.63 ms per token, 380.68 tokens per second)
1.12.676.997 I slot print_timing: id 3 | task 2673 | eval time = 5343.31 ms / 255 tokens ( 20.95 ms per token, 47.72 tokens per second)
1.12.676.997 I slot print_timing: id 3 | task 2673 | total time = 5458.90 ms / 299 tokens
1.12.676.998 I slot print_timing: id 3 | task 2673 | graphs reused = 2891
1.12.677.024 I slot release: id 3 | task 2673 | stop processing: n_tokens = 369, truncated = 0
1.12.677.032 I srv update_slots: all slots are idle
1.12.680.297 I srv operator(): Chat format: peg-native
1.12.680.568 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.529 (> 0.100 thold), f_keep = 0.195
1.12.680.570 I srv get_availabl: updating prompt cache
1.12.680.607 W srv prompt_save: - saving prompt with length 369, total state size = 46.130 MiB (draft: 0.000 MiB)
1.12.716.459 I srv load: - looking for better prompt, base f_keep = 0.195, sim = 0.529
1.12.716.471 I srv update: - cache state: 8 prompts, 430.045 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
1.12.716.472 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
1.12.716.473 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
1.12.716.473 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
1.12.716.474 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
1.12.716.474 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
1.12.716.475 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
1.12.716.475 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
1.12.716.476 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
1.12.716.477 I srv get_availabl: prompt cache update took 35.91 ms
1.12.716.529 I slot launch_slot_: id 3 | task 2929 | processing task, is_child = 0
1.12.716.530 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.12.716.530 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.12.716.542 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.12.716.543 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.12.716.551 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.12.716.551 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.15.887.012 I slot print_timing: id 3 | task 2929 | n_decoded = 145, tg = 48.16 t/s, tg_3s = 48.16 t/s
1.18.889.080 I slot print_timing: id 3 | task 2929 | n_decoded = 288, tg = 47.90 t/s, tg_3s = 47.63 t/s
1.19.433.912 I slot print_timing: id 3 | task 2929 | prompt eval time = 159.44 ms / 64 tokens ( 2.49 ms per token, 401.41 tokens per second)
1.19.433.915 I slot print_timing: id 3 | task 2929 | eval time = 6557.90 ms / 314 tokens ( 20.89 ms per token, 47.88 tokens per second)
1.19.433.916 I slot print_timing: id 3 | task 2929 | total time = 6717.33 ms / 378 tokens
1.19.433.916 I slot print_timing: id 3 | task 2929 | graphs reused = 3202
1.19.433.936 I slot release: id 3 | task 2929 | stop processing: n_tokens = 449, truncated = 0
1.19.433.943 I srv update_slots: all slots are idle
1.19.437.482 I srv operator(): Chat format: peg-native
1.19.437.738 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.657 (> 0.100 thold), f_keep = 0.158
1.19.437.739 I srv get_availabl: updating prompt cache
1.19.437.769 W srv prompt_save: - saving prompt with length 449, total state size = 56.131 MiB (draft: 0.000 MiB)
1.19.480.589 I srv load: - looking for better prompt, base f_keep = 0.158, sim = 0.657
1.19.480.600 I srv update: - cache state: 9 prompts, 486.176 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
1.19.480.601 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
1.19.480.602 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
1.19.480.602 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
1.19.480.603 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
1.19.480.603 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
1.19.480.604 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
1.19.480.604 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
1.19.480.605 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
1.19.480.605 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
1.19.480.606 I srv get_availabl: prompt cache update took 42.87 ms
1.19.480.663 I slot launch_slot_: id 3 | task 3244 | processing task, is_child = 0
1.19.480.663 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.19.480.665 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.19.480.677 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.19.480.677 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.19.480.685 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.19.480.686 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.22.596.241 I slot print_timing: id 3 | task 3244 | n_decoded = 146, tg = 48.53 t/s, tg_3s = 48.53 t/s
1.25.608.081 I slot print_timing: id 3 | task 3244 | n_decoded = 290, tg = 48.17 t/s, tg_3s = 47.81 t/s
1.27.487.992 I slot print_timing: id 3 | task 3244 | prompt eval time = 107.30 ms / 37 tokens ( 2.90 ms per token, 344.83 tokens per second)
1.27.487.995 I slot print_timing: id 3 | task 3244 | eval time = 7899.99 ms / 380 tokens ( 20.79 ms per token, 48.10 tokens per second)
1.27.487.996 I slot print_timing: id 3 | task 3244 | total time = 8007.28 ms / 417 tokens
1.27.487.997 I slot print_timing: id 3 | task 3244 | graphs reused = 3579
1.27.488.021 I slot release: id 3 | task 3244 | stop processing: n_tokens = 487, truncated = 0
1.27.488.029 I srv update_slots: all slots are idle
1.27.491.346 I srv operator(): Chat format: peg-native
1.27.491.617 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.530 (> 0.100 thold), f_keep = 0.146
1.27.491.619 I srv get_availabl: updating prompt cache
1.27.491.655 W srv prompt_save: - saving prompt with length 487, total state size = 60.881 MiB (draft: 0.000 MiB)
1.27.537.353 I srv load: - looking for better prompt, base f_keep = 0.146, sim = 0.530
1.27.537.366 I srv update: - cache state: 10 prompts, 547.058 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
1.27.537.367 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
1.27.537.367 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
1.27.537.368 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
1.27.537.368 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
1.27.537.369 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
1.27.537.369 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
1.27.537.370 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
1.27.537.370 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
1.27.537.371 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
1.27.537.371 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
1.27.537.372 I srv get_availabl: prompt cache update took 45.75 ms
1.27.537.434 I slot launch_slot_: id 3 | task 3625 | processing task, is_child = 0
1.27.537.435 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.27.537.436 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.27.537.448 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.27.537.449 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.27.537.457 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.27.537.458 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.30.698.119 I slot print_timing: id 3 | task 3625 | n_decoded = 145, tg = 48.29 t/s, tg_3s = 48.29 t/s
1.32.762.988 I slot print_timing: id 3 | task 3625 | prompt eval time = 157.75 ms / 63 tokens ( 2.50 ms per token, 399.37 tokens per second)
1.32.762.991 I slot print_timing: id 3 | task 3625 | eval time = 5067.76 ms / 243 tokens ( 20.85 ms per token, 47.95 tokens per second)
1.32.762.992 I slot print_timing: id 3 | task 3625 | total time = 5225.51 ms / 306 tokens
1.32.762.992 I slot print_timing: id 3 | task 3625 | graphs reused = 3819
1.32.763.018 I slot release: id 3 | task 3625 | stop processing: n_tokens = 376, truncated = 0
1.32.763.026 I srv update_slots: all slots are idle
1.32.766.330 I srv operator(): Chat format: peg-native
1.32.766.605 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.563 (> 0.100 thold), f_keep = 0.189
1.32.766.607 I srv get_availabl: updating prompt cache
1.32.766.635 W srv prompt_save: - saving prompt with length 376, total state size = 47.005 MiB (draft: 0.000 MiB)
1.32.802.502 I srv load: - looking for better prompt, base f_keep = 0.189, sim = 0.563
1.32.802.512 I srv update: - cache state: 11 prompts, 594.063 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
1.32.802.513 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
1.32.802.514 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
1.32.802.514 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
1.32.802.515 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
1.32.802.515 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
1.32.802.516 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
1.32.802.516 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
1.32.802.517 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
1.32.802.517 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
1.32.802.518 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
1.32.802.518 I srv update: - prompt 0x5cf416eb2160: 376 tokens, checkpoints: 0, 47.005 MiB
1.32.802.519 I srv get_availabl: prompt cache update took 35.91 ms
1.32.802.573 I slot launch_slot_: id 3 | task 3869 | processing task, is_child = 0
1.32.802.573 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.32.802.574 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.32.802.586 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.32.802.586 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.32.802.595 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.32.802.595 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.35.637.027 I slot print_timing: id 3 | task 3869 | prompt eval time = 143.31 ms / 55 tokens ( 2.61 ms per token, 383.79 tokens per second)
1.35.637.031 I slot print_timing: id 3 | task 3869 | eval time = 2691.10 ms / 130 tokens ( 20.70 ms per token, 48.31 tokens per second)
1.35.637.032 I slot print_timing: id 3 | task 3869 | total time = 2834.41 ms / 185 tokens
1.35.637.033 I slot print_timing: id 3 | task 3869 | graphs reused = 3947
1.35.637.059 I slot release: id 3 | task 3869 | stop processing: n_tokens = 255, truncated = 0
1.35.637.065 I srv update_slots: all slots are idle
1.35.640.317 I srv operator(): Chat format: peg-native
1.35.640.586 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.621 (> 0.100 thold), f_keep = 0.282
1.35.640.587 I srv get_availabl: updating prompt cache
1.35.640.612 W srv prompt_save: - saving prompt with length 255, total state size = 31.879 MiB (draft: 0.000 MiB)
1.35.666.336 I srv load: - looking for better prompt, base f_keep = 0.282, sim = 0.621
1.35.666.348 I srv update: - cache state: 12 prompts, 625.941 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
1.35.666.349 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
1.35.666.350 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
1.35.666.351 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
1.35.666.351 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
1.35.666.352 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
1.35.666.352 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
1.35.666.353 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
1.35.666.353 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
1.35.666.354 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
1.35.666.354 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
1.35.666.355 I srv update: - prompt 0x5cf416eb2160: 376 tokens, checkpoints: 0, 47.005 MiB
1.35.666.355 I srv update: - prompt 0x5cf416e55730: 255 tokens, checkpoints: 0, 31.879 MiB
1.35.666.356 I srv get_availabl: prompt cache update took 25.77 ms
1.35.666.420 I slot launch_slot_: id 3 | task 4000 | processing task, is_child = 0
1.35.666.421 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.35.666.421 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.35.666.433 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.35.666.433 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.35.666.442 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.35.666.442 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.38.782.080 I slot print_timing: id 3 | task 4000 | n_decoded = 144, tg = 48.00 t/s, tg_3s = 48.00 t/s
1.40.318.167 I slot print_timing: id 3 | task 4000 | prompt eval time = 115.36 ms / 44 tokens ( 2.62 ms per token, 381.41 tokens per second)
1.40.318.170 I slot print_timing: id 3 | task 4000 | eval time = 4536.34 ms / 217 tokens ( 20.90 ms per token, 47.84 tokens per second)
1.40.318.171 I slot print_timing: id 3 | task 4000 | total time = 4651.70 ms / 261 tokens
1.40.318.171 I slot print_timing: id 3 | task 4000 | graphs reused = 4161
1.40.318.195 I slot release: id 3 | task 4000 | stop processing: n_tokens = 332, truncated = 0
1.40.318.203 I srv update_slots: all slots are idle
1.40.321.639 I srv operator(): Chat format: peg-native
1.40.321.893 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.597 (> 0.100 thold), f_keep = 0.214
1.40.321.895 I srv get_availabl: updating prompt cache
1.40.321.930 W srv prompt_save: - saving prompt with length 332, total state size = 41.505 MiB (draft: 0.000 MiB)
1.40.354.557 I srv load: - looking for better prompt, base f_keep = 0.214, sim = 0.597
1.40.354.569 I srv update: - cache state: 13 prompts, 667.446 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
1.40.354.570 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
1.40.354.570 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
1.40.354.571 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
1.40.354.571 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
1.40.354.572 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
1.40.354.572 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
1.40.354.573 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
1.40.354.573 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
1.40.354.574 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
1.40.354.574 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
1.40.354.575 I srv update: - prompt 0x5cf416eb2160: 376 tokens, checkpoints: 0, 47.005 MiB
1.40.354.575 I srv update: - prompt 0x5cf416e55730: 255 tokens, checkpoints: 0, 31.879 MiB
1.40.354.576 I srv update: - prompt 0x5cf4174056d0: 332 tokens, checkpoints: 0, 41.505 MiB
1.40.354.577 I srv get_availabl: prompt cache update took 32.68 ms
1.40.354.630 I slot launch_slot_: id 3 | task 4218 | processing task, is_child = 0
1.40.354.631 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.40.354.632 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.40.354.644 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.40.354.644 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.40.354.653 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.40.354.653 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.43.479.184 I slot print_timing: id 3 | task 4218 | n_decoded = 144, tg = 47.86 t/s, tg_3s = 47.86 t/s
1.45.626.213 I slot print_timing: id 3 | task 4218 | prompt eval time = 115.70 ms / 48 tokens ( 2.41 ms per token, 414.87 tokens per second)
1.45.626.216 I slot print_timing: id 3 | task 4218 | eval time = 5155.84 ms / 246 tokens ( 20.96 ms per token, 47.71 tokens per second)
1.45.626.217 I slot print_timing: id 3 | task 4218 | total time = 5271.54 ms / 294 tokens
1.45.626.218 I slot print_timing: id 3 | task 4218 | graphs reused = 4404
1.45.626.244 I slot release: id 3 | task 4218 | stop processing: n_tokens = 364, truncated = 0
1.45.626.252 I srv update_slots: all slots are idle
1.45.629.637 I srv operator(): Chat format: peg-native
1.45.629.914 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.651 (> 0.100 thold), f_keep = 0.195
1.45.629.915 I srv get_availabl: updating prompt cache
1.45.629.947 W srv prompt_save: - saving prompt with length 364, total state size = 45.505 MiB (draft: 0.000 MiB)
1.45.664.721 I srv load: - looking for better prompt, base f_keep = 0.195, sim = 0.651
1.45.664.735 I srv update: - cache state: 14 prompts, 712.951 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
1.45.664.736 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
1.45.664.737 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
1.45.664.737 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
1.45.664.738 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
1.45.664.738 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
1.45.664.739 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
1.45.664.739 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
1.45.664.740 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
1.45.664.740 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
1.45.664.741 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
1.45.664.741 I srv update: - prompt 0x5cf416eb2160: 376 tokens, checkpoints: 0, 47.005 MiB
1.45.664.742 I srv update: - prompt 0x5cf416e55730: 255 tokens, checkpoints: 0, 31.879 MiB
1.45.664.743 I srv update: - prompt 0x5cf4174056d0: 332 tokens, checkpoints: 0, 41.505 MiB
1.45.664.747 I srv update: - prompt 0x5cf416e27280: 364 tokens, checkpoints: 0, 45.505 MiB
1.45.664.749 I srv get_availabl: prompt cache update took 34.83 ms
1.45.664.804 I slot launch_slot_: id 3 | task 4465 | processing task, is_child = 0
1.45.664.805 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.45.664.806 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.45.664.818 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.45.664.818 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.45.664.827 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.45.664.827 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.47.603.104 I slot print_timing: id 3 | task 4465 | prompt eval time = 107.39 ms / 38 tokens ( 2.83 ms per token, 353.87 tokens per second)
1.47.603.108 I slot print_timing: id 3 | task 4465 | eval time = 1830.87 ms / 88 tokens ( 20.81 ms per token, 48.06 tokens per second)
1.47.603.108 I slot print_timing: id 3 | task 4465 | total time = 1938.25 ms / 126 tokens
1.47.603.109 I slot print_timing: id 3 | task 4465 | graphs reused = 4490
1.47.603.136 I slot release: id 3 | task 4465 | stop processing: n_tokens = 196, truncated = 0
1.47.603.143 I srv update_slots: all slots are idle
1.47.606.331 I srv operator(): Chat format: peg-native
1.47.606.596 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.651 (> 0.100 thold), f_keep = 0.362
1.47.606.598 I srv get_availabl: updating prompt cache
1.47.606.625 W srv prompt_save: - saving prompt with length 196, total state size = 24.503 MiB (draft: 0.000 MiB)
1.47.626.661 I srv load: - looking for better prompt, base f_keep = 0.362, sim = 0.651
1.47.626.674 I srv update: - cache state: 15 prompts, 737.454 MiB (limits: 8192.000 MiB, 16384 tokens, 65528 est)
1.47.626.675 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
1.47.626.675 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
1.47.626.676 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
1.47.626.676 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
1.47.626.677 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
1.47.626.677 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
1.47.626.678 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
1.47.626.678 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
1.47.626.679 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
1.47.626.679 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
1.47.626.680 I srv update: - prompt 0x5cf416eb2160: 376 tokens, checkpoints: 0, 47.005 MiB
1.47.626.680 I srv update: - prompt 0x5cf416e55730: 255 tokens, checkpoints: 0, 31.879 MiB
1.47.626.681 I srv update: - prompt 0x5cf4174056d0: 332 tokens, checkpoints: 0, 41.505 MiB
1.47.626.681 I srv update: - prompt 0x5cf416e27280: 364 tokens, checkpoints: 0, 45.505 MiB
1.47.626.682 I srv update: - prompt 0x5cf416d081a0: 196 tokens, checkpoints: 0, 24.503 MiB
1.47.626.683 I srv get_availabl: prompt cache update took 20.09 ms
1.47.626.742 I slot launch_slot_: id 3 | task 4554 | processing task, is_child = 0
1.47.626.743 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.47.626.744 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.47.626.758 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.47.626.759 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.47.626.769 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.47.626.769 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.50.744.105 I slot print_timing: id 3 | task 4554 | n_decoded = 144, tg = 47.84 t/s, tg_3s = 47.84 t/s
1.53.750.249 I slot print_timing: id 3 | task 4554 | n_decoded = 287, tg = 47.71 t/s, tg_3s = 47.57 t/s
1.56.756.643 I slot print_timing: id 3 | task 4554 | n_decoded = 430, tg = 47.66 t/s, tg_3s = 47.57 t/s
1.58.520.689 I slot print_timing: id 3 | task 4554 | prompt eval time = 107.40 ms / 38 tokens ( 2.83 ms per token, 353.82 tokens per second)
1.58.520.693 I slot print_timing: id 3 | task 4554 | eval time = 10786.49 ms / 514 tokens ( 20.99 ms per token, 47.65 tokens per second)
1.58.520.694 I slot print_timing: id 3 | task 4554 | total time = 10893.89 ms / 552 tokens
1.58.520.695 I slot print_timing: id 3 | task 4554 | graphs reused = 5000
1.58.520.721 I slot release: id 3 | task 4554 | stop processing: n_tokens = 622, truncated = 0
1.58.520.730 I srv update_slots: all slots are idle
1.58.523.906 I srv operator(): Chat format: peg-native
1.58.524.167 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.114
1.58.524.168 I srv get_availabl: updating prompt cache
1.58.524.220 W srv prompt_save: - saving prompt with length 622, total state size = 77.758 MiB (draft: 0.000 MiB)
1.58.583.787 I srv load: - looking for better prompt, base f_keep = 0.114, sim = 0.710
1.58.583.800 I srv update: - cache state: 16 prompts, 815.212 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
1.58.583.801 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
1.58.583.802 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
1.58.583.803 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
1.58.583.803 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
1.58.583.804 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
1.58.583.805 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
1.58.583.806 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
1.58.583.806 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
1.58.583.807 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
1.58.583.807 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
1.58.583.808 I srv update: - prompt 0x5cf416eb2160: 376 tokens, checkpoints: 0, 47.005 MiB
1.58.583.808 I srv update: - prompt 0x5cf416e55730: 255 tokens, checkpoints: 0, 31.879 MiB
1.58.583.809 I srv update: - prompt 0x5cf4174056d0: 332 tokens, checkpoints: 0, 41.505 MiB
1.58.583.810 I srv update: - prompt 0x5cf416e27280: 364 tokens, checkpoints: 0, 45.505 MiB
1.58.583.810 I srv update: - prompt 0x5cf416d081a0: 196 tokens, checkpoints: 0, 24.503 MiB
1.58.583.810 I srv update: - prompt 0x5cf416e0ae60: 622 tokens, checkpoints: 0, 77.758 MiB
1.58.583.811 I srv get_availabl: prompt cache update took 59.64 ms
1.58.583.854 I slot launch_slot_: id 3 | task 5069 | processing task, is_child = 0
1.58.583.855 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.58.583.856 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.58.583.869 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.58.583.869 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.58.583.881 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.58.583.881 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.01.723.577 I slot print_timing: id 3 | task 5069 | n_decoded = 144, tg = 47.91 t/s, tg_3s = 47.91 t/s
2.04.731.827 I slot print_timing: id 3 | task 5069 | n_decoded = 287, tg = 47.72 t/s, tg_3s = 47.54 t/s
2.05.508.679 I slot print_timing: id 3 | task 5069 | prompt eval time = 134.12 ms / 29 tokens ( 4.62 ms per token, 216.22 tokens per second)
2.05.508.682 I slot print_timing: id 3 | task 5069 | eval time = 6790.65 ms / 324 tokens ( 20.96 ms per token, 47.71 tokens per second)
2.05.508.683 I slot print_timing: id 3 | task 5069 | total time = 6924.77 ms / 353 tokens
2.05.508.683 I slot print_timing: id 3 | task 5069 | graphs reused = 5321
2.05.508.708 I slot release: id 3 | task 5069 | stop processing: n_tokens = 423, truncated = 0
2.05.508.716 I srv update_slots: all slots are idle
2.05.512.122 I srv operator(): Chat format: peg-native
2.05.512.381 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.798 (> 0.100 thold), f_keep = 0.168
2.05.512.382 I srv get_availabl: updating prompt cache
2.05.512.416 W srv prompt_save: - saving prompt with length 423, total state size = 52.881 MiB (draft: 0.000 MiB)
2.05.552.994 I srv load: - looking for better prompt, base f_keep = 0.168, sim = 0.798
2.05.553.008 I srv update: - cache state: 17 prompts, 868.092 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
2.05.553.009 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
2.05.553.010 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
2.05.553.011 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
2.05.553.011 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
2.05.553.012 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
2.05.553.017 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
2.05.553.017 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
2.05.553.018 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
2.05.553.019 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
2.05.553.019 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
2.05.553.020 I srv update: - prompt 0x5cf416eb2160: 376 tokens, checkpoints: 0, 47.005 MiB
2.05.553.023 I srv update: - prompt 0x5cf416e55730: 255 tokens, checkpoints: 0, 31.879 MiB
2.05.553.024 I srv update: - prompt 0x5cf4174056d0: 332 tokens, checkpoints: 0, 41.505 MiB
2.05.553.025 I srv update: - prompt 0x5cf416e27280: 364 tokens, checkpoints: 0, 45.505 MiB
2.05.553.025 I srv update: - prompt 0x5cf416d081a0: 196 tokens, checkpoints: 0, 24.503 MiB
2.05.553.026 I srv update: - prompt 0x5cf416e0ae60: 622 tokens, checkpoints: 0, 77.758 MiB
2.05.553.027 I srv update: - prompt 0x5cf416e9ae90: 423 tokens, checkpoints: 0, 52.881 MiB
2.05.553.029 I srv get_availabl: prompt cache update took 40.65 ms
2.05.553.097 I slot launch_slot_: id 3 | task 5394 | processing task, is_child = 0
2.05.553.098 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.05.553.099 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.05.553.116 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.05.553.117 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.05.553.127 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.05.553.127 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.08.685.321 I slot print_timing: id 3 | task 5394 | n_decoded = 144, tg = 47.87 t/s, tg_3s = 47.87 t/s
2.11.690.498 I slot print_timing: id 3 | task 5394 | n_decoded = 287, tg = 47.73 t/s, tg_3s = 47.58 t/s
2.12.236.508 I slot print_timing: id 3 | task 5394 | prompt eval time = 124.25 ms / 18 tokens ( 6.90 ms per token, 144.87 tokens per second)
2.12.236.511 I slot print_timing: id 3 | task 5394 | eval time = 6559.10 ms / 313 tokens ( 20.96 ms per token, 47.72 tokens per second)
2.12.236.511 I slot print_timing: id 3 | task 5394 | total time = 6683.35 ms / 331 tokens
2.12.236.512 I slot print_timing: id 3 | task 5394 | graphs reused = 5631
2.12.236.537 I slot release: id 3 | task 5394 | stop processing: n_tokens = 401, truncated = 0
2.12.236.545 I srv update_slots: all slots are idle
2.12.240.020 I srv operator(): Chat format: peg-native
2.12.240.294 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.645 (> 0.100 thold), f_keep = 0.177
2.12.240.295 I srv get_availabl: updating prompt cache
2.12.240.324 W srv prompt_save: - saving prompt with length 401, total state size = 50.130 MiB (draft: 0.000 MiB)
2.12.278.717 I srv load: - looking for better prompt, base f_keep = 0.177, sim = 0.645
2.12.278.732 I srv update: - cache state: 18 prompts, 918.223 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
2.12.278.733 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
2.12.278.733 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
2.12.278.734 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
2.12.278.734 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
2.12.278.735 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
2.12.278.735 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
2.12.278.736 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
2.12.278.736 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
2.12.278.737 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
2.12.278.737 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
2.12.278.738 I srv update: - prompt 0x5cf416eb2160: 376 tokens, checkpoints: 0, 47.005 MiB
2.12.278.739 I srv update: - prompt 0x5cf416e55730: 255 tokens, checkpoints: 0, 31.879 MiB
2.12.278.750 I srv update: - prompt 0x5cf4174056d0: 332 tokens, checkpoints: 0, 41.505 MiB
2.12.278.751 I srv update: - prompt 0x5cf416e27280: 364 tokens, checkpoints: 0, 45.505 MiB
2.12.278.752 I srv update: - prompt 0x5cf416d081a0: 196 tokens, checkpoints: 0, 24.503 MiB
2.12.278.752 I srv update: - prompt 0x5cf416e0ae60: 622 tokens, checkpoints: 0, 77.758 MiB
2.12.278.753 I srv update: - prompt 0x5cf416e9ae90: 423 tokens, checkpoints: 0, 52.881 MiB
2.12.278.753 I srv update: - prompt 0x5cf416db5690: 401 tokens, checkpoints: 0, 50.130 MiB
2.12.278.755 I srv get_availabl: prompt cache update took 38.46 ms
2.12.278.811 I slot launch_slot_: id 3 | task 5708 | processing task, is_child = 0
2.12.278.811 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.12.278.812 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.12.278.825 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.12.278.825 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.12.278.834 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.12.278.834 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.15.395.403 I slot print_timing: id 3 | task 5708 | n_decoded = 144, tg = 47.86 t/s, tg_3s = 47.86 t/s
2.15.754.480 I slot print_timing: id 3 | task 5708 | prompt eval time = 107.55 ms / 39 tokens ( 2.76 ms per token, 362.63 tokens per second)
2.15.754.484 I slot print_timing: id 3 | task 5708 | eval time = 3368.08 ms / 161 tokens ( 20.92 ms per token, 47.80 tokens per second)
2.15.754.484 I slot print_timing: id 3 | task 5708 | total time = 3475.62 ms / 200 tokens
2.15.754.485 I slot print_timing: id 3 | task 5708 | graphs reused = 5789
2.15.754.513 I slot release: id 3 | task 5708 | stop processing: n_tokens = 270, truncated = 0
2.15.754.520 I srv update_slots: all slots are idle
2.15.757.682 I srv operator(): Chat format: peg-native
2.15.757.954 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.717 (> 0.100 thold), f_keep = 0.263
2.15.757.956 I srv get_availabl: updating prompt cache
2.15.757.988 W srv prompt_save: - saving prompt with length 270, total state size = 33.754 MiB (draft: 0.000 MiB)
2.15.784.516 I srv load: - looking for better prompt, base f_keep = 0.263, sim = 0.717
2.15.784.532 I srv update: - cache state: 19 prompts, 951.976 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
2.15.784.532 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
2.15.784.533 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
2.15.784.534 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
2.15.784.534 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
2.15.784.535 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
2.15.784.535 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
2.15.784.536 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
2.15.784.536 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
2.15.784.537 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
2.15.784.537 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
2.15.784.538 I srv update: - prompt 0x5cf416eb2160: 376 tokens, checkpoints: 0, 47.005 MiB
2.15.784.538 I srv update: - prompt 0x5cf416e55730: 255 tokens, checkpoints: 0, 31.879 MiB
2.15.784.539 I srv update: - prompt 0x5cf4174056d0: 332 tokens, checkpoints: 0, 41.505 MiB
2.15.784.542 I srv update: - prompt 0x5cf416e27280: 364 tokens, checkpoints: 0, 45.505 MiB
2.15.784.543 I srv update: - prompt 0x5cf416d081a0: 196 tokens, checkpoints: 0, 24.503 MiB
2.15.784.543 I srv update: - prompt 0x5cf416e0ae60: 622 tokens, checkpoints: 0, 77.758 MiB
2.15.784.543 I srv update: - prompt 0x5cf416e9ae90: 423 tokens, checkpoints: 0, 52.881 MiB
2.15.784.544 I srv update: - prompt 0x5cf416db5690: 401 tokens, checkpoints: 0, 50.130 MiB
2.15.784.544 I srv update: - prompt 0x5cf416e56960: 270 tokens, checkpoints: 0, 33.754 MiB
2.15.784.546 I srv get_availabl: prompt cache update took 26.59 ms
2.15.784.602 I slot launch_slot_: id 3 | task 5870 | processing task, is_child = 0
2.15.784.603 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.15.784.604 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.15.784.617 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.15.784.618 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.15.784.627 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.15.784.628 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.18.926.425 I slot print_timing: id 3 | task 5870 | n_decoded = 144, tg = 47.87 t/s, tg_3s = 47.87 t/s
2.21.471.531 I slot print_timing: id 3 | task 5870 | prompt eval time = 133.85 ms / 28 tokens ( 4.78 ms per token, 209.19 tokens per second)
2.21.471.534 I slot print_timing: id 3 | task 5870 | eval time = 5553.03 ms / 265 tokens ( 20.95 ms per token, 47.72 tokens per second)
2.21.471.535 I slot print_timing: id 3 | task 5870 | total time = 5686.88 ms / 293 tokens
2.21.471.536 I slot print_timing: id 3 | task 5870 | graphs reused = 6051
2.21.471.562 I slot release: id 3 | task 5870 | stop processing: n_tokens = 363, truncated = 0
2.21.471.568 I srv update_slots: all slots are idle
2.21.474.771 I srv operator(): Chat format: peg-native
2.21.475.044 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.670 (> 0.100 thold), f_keep = 0.196
2.21.475.046 I srv get_availabl: updating prompt cache
2.21.475.074 W srv prompt_save: - saving prompt with length 363, total state size = 45.380 MiB (draft: 0.000 MiB)
2.21.510.011 I srv load: - looking for better prompt, base f_keep = 0.196, sim = 0.670
2.21.510.023 I srv update: - cache state: 20 prompts, 997.356 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
2.21.510.024 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
2.21.510.025 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
2.21.510.025 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
2.21.510.026 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
2.21.510.030 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
2.21.510.031 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
2.21.510.031 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
2.21.510.031 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
2.21.510.032 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
2.21.510.032 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
2.21.510.033 I srv update: - prompt 0x5cf416eb2160: 376 tokens, checkpoints: 0, 47.005 MiB
2.21.510.036 I srv update: - prompt 0x5cf416e55730: 255 tokens, checkpoints: 0, 31.879 MiB
2.21.510.036 I srv update: - prompt 0x5cf4174056d0: 332 tokens, checkpoints: 0, 41.505 MiB
2.21.510.037 I srv update: - prompt 0x5cf416e27280: 364 tokens, checkpoints: 0, 45.505 MiB
2.21.510.037 I srv update: - prompt 0x5cf416d081a0: 196 tokens, checkpoints: 0, 24.503 MiB
2.21.510.038 I srv update: - prompt 0x5cf416e0ae60: 622 tokens, checkpoints: 0, 77.758 MiB
2.21.510.038 I srv update: - prompt 0x5cf416e9ae90: 423 tokens, checkpoints: 0, 52.881 MiB
2.21.510.050 I srv update: - prompt 0x5cf416db5690: 401 tokens, checkpoints: 0, 50.130 MiB
2.21.510.051 I srv update: - prompt 0x5cf416e56960: 270 tokens, checkpoints: 0, 33.754 MiB
2.21.510.051 I srv update: - prompt 0x5cf416d1d430: 363 tokens, checkpoints: 0, 45.380 MiB
2.21.510.052 I srv get_availabl: prompt cache update took 35.01 ms
2.21.510.107 I slot launch_slot_: id 3 | task 6136 | processing task, is_child = 0
2.21.510.108 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.21.510.109 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.21.510.121 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.21.510.121 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.21.510.129 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.21.510.130 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.24.206.477 I slot print_timing: id 3 | task 6136 | prompt eval time = 107.14 ms / 35 tokens ( 3.06 ms per token, 326.69 tokens per second)
2.24.206.481 I slot print_timing: id 3 | task 6136 | eval time = 2589.19 ms / 124 tokens ( 20.88 ms per token, 47.89 tokens per second)
2.24.206.481 I slot print_timing: id 3 | task 6136 | total time = 2696.32 ms / 159 tokens
2.24.206.482 I slot print_timing: id 3 | task 6136 | graphs reused = 6173
2.24.206.509 I slot release: id 3 | task 6136 | stop processing: n_tokens = 229, truncated = 0
2.24.206.518 I srv update_slots: all slots are idle
2.24.209.524 I srv operator(): Chat format: peg-native
2.24.209.787 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.628 (> 0.100 thold), f_keep = 0.310
2.24.209.789 I srv get_availabl: updating prompt cache
2.24.209.816 W srv prompt_save: - saving prompt with length 229, total state size = 28.628 MiB (draft: 0.000 MiB)
2.24.232.452 I srv load: - looking for better prompt, base f_keep = 0.310, sim = 0.628
2.24.232.467 I srv update: - cache state: 21 prompts, 1025.985 MiB (limits: 8192.000 MiB, 16384 tokens, 65528 est)
2.24.232.468 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
2.24.232.469 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
2.24.232.469 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
2.24.232.470 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
2.24.232.470 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
2.24.232.471 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
2.24.232.471 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
2.24.232.472 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
2.24.232.484 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
2.24.232.485 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
2.24.232.485 I srv update: - prompt 0x5cf416eb2160: 376 tokens, checkpoints: 0, 47.005 MiB
2.24.232.486 I srv update: - prompt 0x5cf416e55730: 255 tokens, checkpoints: 0, 31.879 MiB
2.24.232.486 I srv update: - prompt 0x5cf4174056d0: 332 tokens, checkpoints: 0, 41.505 MiB
2.24.232.487 I srv update: - prompt 0x5cf416e27280: 364 tokens, checkpoints: 0, 45.505 MiB
2.24.232.487 I srv update: - prompt 0x5cf416d081a0: 196 tokens, checkpoints: 0, 24.503 MiB
2.24.232.488 I srv update: - prompt 0x5cf416e0ae60: 622 tokens, checkpoints: 0, 77.758 MiB
2.24.232.488 I srv update: - prompt 0x5cf416e9ae90: 423 tokens, checkpoints: 0, 52.881 MiB
2.24.232.489 I srv update: - prompt 0x5cf416db5690: 401 tokens, checkpoints: 0, 50.130 MiB
2.24.232.489 I srv update: - prompt 0x5cf416e56960: 270 tokens, checkpoints: 0, 33.754 MiB
2.24.232.490 I srv update: - prompt 0x5cf416d1d430: 363 tokens, checkpoints: 0, 45.380 MiB
2.24.232.490 I srv update: - prompt 0x5cf414851260: 229 tokens, checkpoints: 0, 28.628 MiB
2.24.232.492 I srv get_availabl: prompt cache update took 22.70 ms
2.24.232.551 I slot launch_slot_: id 3 | task 6261 | processing task, is_child = 0
2.24.232.552 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.24.232.553 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.24.232.568 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.24.232.568 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.24.232.581 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.24.232.581 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.27.356.488 I slot print_timing: id 3 | task 6261 | n_decoded = 144, tg = 47.87 t/s, tg_3s = 47.87 t/s
2.30.362.613 I slot print_timing: id 3 | task 6261 | n_decoded = 287, tg = 47.72 t/s, tg_3s = 47.57 t/s
2.32.780.283 I slot print_timing: id 3 | task 6261 | prompt eval time = 115.45 ms / 42 tokens ( 2.75 ms per token, 363.80 tokens per second)
2.32.780.286 I slot print_timing: id 3 | task 6261 | eval time = 8432.23 ms / 402 tokens ( 20.98 ms per token, 47.67 tokens per second)
2.32.780.287 I slot print_timing: id 3 | task 6261 | total time = 8547.68 ms / 444 tokens
2.32.780.288 I slot print_timing: id 3 | task 6261 | graphs reused = 6571
2.32.780.314 I slot release: id 3 | task 6261 | stop processing: n_tokens = 514, truncated = 0
2.32.780.322 I srv update_slots: all slots are idle
2.32.783.708 I srv operator(): Chat format: peg-native
2.32.783.967 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.724 (> 0.100 thold), f_keep = 0.138
2.32.783.968 I srv get_availabl: updating prompt cache
2.32.784.007 W srv prompt_save: - saving prompt with length 514, total state size = 64.257 MiB (draft: 0.000 MiB)
2.32.839.487 I srv load: - looking for better prompt, base f_keep = 0.138, sim = 0.724
2.32.839.500 I srv update: - cache state: 22 prompts, 1090.241 MiB (limits: 8192.000 MiB, 16384 tokens, 65529 est)
2.32.839.501 I srv update: - prompt 0x5cf416e9c2c0: 553 tokens, checkpoints: 0, 69.132 MiB
2.32.839.501 I srv update: - prompt 0x5cf416defae0: 333 tokens, checkpoints: 0, 41.630 MiB
2.32.839.502 I srv update: - prompt 0x5cf416d0f570: 236 tokens, checkpoints: 0, 29.503 MiB
2.32.839.503 I srv update: - prompt 0x5cf416decc60: 626 tokens, checkpoints: 0, 78.258 MiB
2.32.839.503 I srv update: - prompt 0x5cf416cf5580: 581 tokens, checkpoints: 0, 72.632 MiB
2.32.839.504 I srv update: - prompt 0x5cf416d1cf40: 427 tokens, checkpoints: 0, 53.381 MiB
2.32.839.508 I srv update: - prompt 0x5cf416e15bb0: 315 tokens, checkpoints: 0, 39.379 MiB
2.32.839.508 I srv update: - prompt 0x5cf416e28a70: 369 tokens, checkpoints: 0, 46.130 MiB
2.32.839.509 I srv update: - prompt 0x5cf416d790c0: 449 tokens, checkpoints: 0, 56.131 MiB
2.32.839.509 I srv update: - prompt 0x5cf416e1bfd0: 487 tokens, checkpoints: 0, 60.881 MiB
2.32.839.510 I srv update: - prompt 0x5cf416eb2160: 376 tokens, checkpoints: 0, 47.005 MiB
2.32.839.511 I srv update: - prompt 0x5cf416e55730: 255 tokens, checkpoints: 0, 31.879 MiB
2.32.839.511 I srv update: - prompt 0x5cf4174056d0: 332 tokens, checkpoints: 0, 41.505 MiB
2.32.839.514 I srv update: - prompt 0x5cf416e27280: 364 tokens, checkpoints: 0, 45.505 MiB
2.32.839.514 I srv update: - prompt 0x5cf416d081a0: 196 tokens, checkpoints: 0, 24.503 MiB
2.32.839.515 I srv update: - prompt 0x5cf416e0ae60: 622 tokens, checkpoints: 0, 77.758 MiB
2.32.839.515 I srv update: - prompt 0x5cf416e9ae90: 423 tokens, checkpoints: 0, 52.881 MiB
2.32.839.516 I srv update: - prompt 0x5cf416db5690: 401 tokens, checkpoints: 0, 50.130 MiB
2.32.839.516 I srv update: - prompt 0x5cf416e56960: 270 tokens, checkpoints: 0, 33.754 MiB
2.32.839.518 I srv update: - prompt 0x5cf416d1d430: 363 tokens, checkpoints: 0, 45.380 MiB
2.32.839.519 I srv update: - prompt 0x5cf414851260: 229 tokens, checkpoints: 0, 28.628 MiB
2.32.839.520 I srv update: - prompt 0x5cf416d16880: 514 tokens, checkpoints: 0, 64.257 MiB
2.32.839.520 I srv get_availabl: prompt cache update took 55.55 ms
2.32.839.559 I slot launch_slot_: id 3 | task 6664 | processing task, is_child = 0
2.32.839.559 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.32.839.560 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.32.839.570 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.32.839.571 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.32.839.579 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.32.839.579 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.35.978.572 I slot print_timing: id 3 | task 6664 | n_decoded = 144, tg = 47.92 t/s, tg_3s = 47.92 t/s
2.38.984.742 I slot print_timing: id 3 | task 6664 | n_decoded = 287, tg = 47.74 t/s, tg_3s = 47.57 t/s
2.39.635.741 I slot print_timing: id 3 | task 6664 | prompt eval time = 134.00 ms / 27 tokens ( 4.96 ms per token, 201.49 tokens per second)
2.39.635.744 I slot print_timing: id 3 | task 6664 | eval time = 6662.14 ms / 318 tokens ( 20.95 ms per token, 47.73 tokens per second)
2.39.635.745 I slot print_timing: id 3 | task 6664 | total time = 6796.14 ms / 345 tokens
2.39.635.746 I slot print_timing: id 3 | task 6664 | graphs reused = 6886
2.39.635.770 I slot release: id 3 | task 6664 | stop processing: n_tokens = 415, truncated = 0
2.39.635.779 I srv update_slots: all slots are idle
2.39.636.529 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue View Git Blame Copy Permalink