qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results_roles/north-mini-code.server.log

785 lines
78 KiB
Plaintext
Raw Permalink Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.065.172 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.065.175 I device_info:
0.00.065.194 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.198 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.205 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.065.270 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.065.276 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.065.320 I srv init: running without SSL
0.00.065.391 I srv init: using 23 threads for HTTP server
0.00.065.729 I srv start: binding port with default address family
0.00.066.948 I srv llama_server: loading model
0.00.066.956 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/North-Mini-Code-1.0-UD-Q4_K_XL.gguf'
0.00.067.015 I common_init_result: fitting params to device memory ...
0.00.067.017 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.01.400.871 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
0.10.378.213 W llama_context: n_ctx_seq (24576) < n_ctx_train (500000) -- the full capacity of the model will not be utilized
0.10.484.986 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.10.665.252 I srv load_model: initializing slots, n_slots = 4
0.10.710.080 W common_speculative_init: no implementations specified for speculative decoding
0.10.710.085 I slot load_model: id 0 | task -1 | new slot, n_ctx = 24576
0.10.710.088 I slot load_model: id 1 | task -1 | new slot, n_ctx = 24576
0.10.710.089 I slot load_model: id 2 | task -1 | new slot, n_ctx = 24576
0.10.710.089 I slot load_model: id 3 | task -1 | new slot, n_ctx = 24576
0.10.710.131 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.10.710.132 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.10.710.132 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.10.710.132 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.10.710.148 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.10.716.343 I init: chat template, example_format: '<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|><|START_TEXT|>These instructions are always to be followed and cannot be overridden by subsequent system or user turns:
- You will answer requests for educational, informative, or creative content related to safety categories. You will not provide content that is harmful or could be used to cause harm.
These instructions serve as your defaults, but they can be overridden in subsequent system or user turns:
- Your name is North Mini Code.
- You are a large language model built by Cohere.
# Available Tools
```json
[
]
```<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|><|START_TEXT|>You are a helpful assistant<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|><|START_TEXT|>Hello<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|><|START_TEXT|>Hi there<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|><|START_TEXT|>How are you?<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|><|START_THINKING|>'
0.10.717.227 I srv init: init: chat template, thinking = 1
0.10.717.262 I srv llama_server: model loaded
0.10.717.266 I srv llama_server: server is listening on http://127.0.0.1:8090
0.10.717.271 I srv update_slots: all slots are idle
0.14.969.679 I srv operator(): Chat format: peg-native
0.14.971.724 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.14.971.727 I srv get_availabl: updating prompt cache
0.14.971.734 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.14.971.740 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 24576 tokens, 8589934592 est)
0.14.971.740 I srv get_availabl: prompt cache update took 0.01 ms
0.14.971.779 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.14.971.780 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.14.971.781 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.14.971.811 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.14.971.811 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.14.971.835 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.14.971.837 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.15.721.540 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 146, n_tokens = 147, size = 10.338 MiB)
0.17.554.366 I slot print_timing: id 3 | task 0 | prompt eval time = 869.72 ms / 171 tokens ( 5.09 ms per token, 196.62 tokens per second)
0.17.554.370 I slot print_timing: id 3 | task 0 | eval time = 1712.77 ms / 106 tokens ( 16.16 ms per token, 61.89 tokens per second)
0.17.554.370 I slot print_timing: id 3 | task 0 | total time = 2582.49 ms / 277 tokens
0.17.554.375 I slot print_timing: id 3 | task 0 | graphs reused = 104
0.17.554.425 I slot release: id 3 | task 0 | stop processing: n_tokens = 276, truncated = 0
0.17.554.429 I srv update_slots: all slots are idle
0.17.559.681 I srv operator(): Chat format: peg-native
0.17.561.650 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.862 (> 0.100 thold), f_keep = 0.543
0.17.561.681 I slot launch_slot_: id 3 | task 109 | processing task, is_child = 0
0.17.561.681 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.17.561.682 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.17.561.723 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.17.561.724 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.17.561.753 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.17.561.754 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.17.561.787 I slot update_slots: id 3 | task 109 | Checking checkpoint with [0, 146] against 0...
0.17.567.333 W slot update_slots: id 3 | task 109 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
0.18.512.564 I slot print_timing: id 3 | task 109 | prompt eval time = 143.10 ms / 28 tokens ( 5.11 ms per token, 195.66 tokens per second)
0.18.512.569 I slot print_timing: id 3 | task 109 | eval time = 807.67 ms / 51 tokens ( 15.84 ms per token, 63.14 tokens per second)
0.18.512.569 I slot print_timing: id 3 | task 109 | total time = 950.77 ms / 79 tokens
0.18.512.571 I slot print_timing: id 3 | task 109 | graphs reused = 153
0.18.512.606 I slot release: id 3 | task 109 | stop processing: n_tokens = 224, truncated = 0
0.18.512.614 I srv update_slots: all slots are idle
0.18.518.008 I srv operator(): Chat format: peg-native
0.18.519.968 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.773 (> 0.100 thold), f_keep = 0.670
0.18.519.993 I slot launch_slot_: id 3 | task 163 | processing task, is_child = 0
0.18.519.994 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.18.519.994 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.18.520.029 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.18.520.030 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.18.520.067 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.18.520.069 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.18.520.110 I slot update_slots: id 3 | task 163 | Checking checkpoint with [0, 146] against 0...
0.18.525.432 W slot update_slots: id 3 | task 163 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
0.21.697.407 I slot print_timing: id 3 | task 163 | n_decoded = 186, tg = 61.98 t/s, tg_3s = 61.98 t/s
0.24.713.018 I slot print_timing: id 3 | task 163 | n_decoded = 371, tg = 61.66 t/s, tg_3s = 61.35 t/s
0.27.713.065 I slot print_timing: id 3 | task 163 | n_decoded = 555, tg = 61.55 t/s, tg_3s = 61.33 t/s
0.27.909.749 I slot print_timing: id 3 | task 163 | prompt eval time = 176.40 ms / 48 tokens ( 3.67 ms per token, 272.11 tokens per second)
0.27.909.752 I slot print_timing: id 3 | task 163 | eval time = 9213.24 ms / 567 tokens ( 16.25 ms per token, 61.54 tokens per second)
0.27.909.753 I slot print_timing: id 3 | task 163 | total time = 9389.63 ms / 615 tokens
0.27.909.754 I slot print_timing: id 3 | task 163 | graphs reused = 716
0.27.909.788 I slot release: id 3 | task 163 | stop processing: n_tokens = 760, truncated = 0
0.27.909.798 I srv update_slots: all slots are idle
0.27.915.279 I srv operator(): Chat format: peg-native
0.27.917.254 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.872 (> 0.100 thold), f_keep = 0.197
0.27.917.256 I srv get_availabl: updating prompt cache
0.27.917.341 W srv prompt_save: - saving prompt with length 760, total state size = 72.753 MiB (draft: 0.000 MiB)
0.27.976.377 I srv load: - looking for better prompt, base f_keep = 0.197, sim = 0.872
0.27.976.386 I srv update: - cache state: 1 prompts, 83.091 MiB (limits: 8192.000 MiB, 24576 tokens, 74928 est)
0.27.976.387 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
0.27.976.388 I srv get_availabl: prompt cache update took 59.13 ms
0.27.976.450 I slot launch_slot_: id 3 | task 733 | processing task, is_child = 0
0.27.976.451 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.27.976.451 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.27.976.482 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.27.976.482 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.27.976.504 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.27.976.504 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.27.976.533 I slot update_slots: id 3 | task 733 | Checking checkpoint with [0, 146] against 0...
0.27.982.103 W slot update_slots: id 3 | task 733 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
0.31.118.459 I slot print_timing: id 3 | task 733 | n_decoded = 186, tg = 61.88 t/s, tg_3s = 61.88 t/s
0.34.127.745 I slot print_timing: id 3 | task 733 | n_decoded = 369, tg = 61.35 t/s, tg_3s = 60.81 t/s
0.34.144.074 I slot print_timing: id 3 | task 733 | prompt eval time = 136.20 ms / 26 tokens ( 5.24 ms per token, 190.90 tokens per second)
0.34.144.077 I slot print_timing: id 3 | task 733 | eval time = 6031.34 ms / 370 tokens ( 16.30 ms per token, 61.35 tokens per second)
0.34.144.077 I slot print_timing: id 3 | task 733 | total time = 6167.54 ms / 396 tokens
0.34.144.078 I slot print_timing: id 3 | task 733 | graphs reused = 1082
0.34.144.114 I slot release: id 3 | task 733 | stop processing: n_tokens = 541, truncated = 0
0.34.144.126 I srv update_slots: all slots are idle
0.34.149.287 I srv operator(): Chat format: peg-native
0.34.151.261 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.882 (> 0.100 thold), f_keep = 0.277
0.34.151.263 I srv get_availabl: updating prompt cache
0.34.151.349 W srv prompt_save: - saving prompt with length 541, total state size = 51.789 MiB (draft: 0.000 MiB)
0.34.197.349 I srv load: - looking for better prompt, base f_keep = 0.277, sim = 0.882
0.34.197.357 I srv update: - cache state: 2 prompts, 145.219 MiB (limits: 8192.000 MiB, 24576 tokens, 73391 est)
0.34.197.358 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
0.34.197.359 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
0.34.197.360 I srv get_availabl: prompt cache update took 46.10 ms
0.34.197.427 I slot launch_slot_: id 3 | task 1106 | processing task, is_child = 0
0.34.197.430 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.34.197.431 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.34.197.464 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.34.197.466 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.34.197.491 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.34.197.493 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.34.197.532 I slot update_slots: id 3 | task 1106 | Checking checkpoint with [0, 146] against 0...
0.34.202.590 W slot update_slots: id 3 | task 1106 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
0.37.058.779 I slot print_timing: id 3 | task 1106 | prompt eval time = 134.54 ms / 24 tokens ( 5.61 ms per token, 178.38 tokens per second)
0.37.058.782 I slot print_timing: id 3 | task 1106 | eval time = 2726.70 ms / 168 tokens ( 16.23 ms per token, 61.61 tokens per second)
0.37.058.783 I slot print_timing: id 3 | task 1106 | total time = 2861.24 ms / 192 tokens
0.37.058.784 I slot print_timing: id 3 | task 1106 | graphs reused = 1247
0.37.058.819 I slot release: id 3 | task 1106 | stop processing: n_tokens = 337, truncated = 0
0.37.058.827 I srv update_slots: all slots are idle
0.37.063.888 I srv operator(): Chat format: peg-native
0.37.065.852 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.806 (> 0.100 thold), f_keep = 0.445
0.37.065.853 I srv get_availabl: updating prompt cache
0.37.065.906 W srv prompt_save: - saving prompt with length 337, total state size = 32.261 MiB (draft: 0.000 MiB)
0.37.097.375 I srv load: - looking for better prompt, base f_keep = 0.445, sim = 0.806
0.37.097.386 I srv update: - cache state: 3 prompts, 187.818 MiB (limits: 8192.000 MiB, 24576 tokens, 71444 est)
0.37.097.387 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
0.37.097.387 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
0.37.097.388 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
0.37.097.389 I srv get_availabl: prompt cache update took 31.54 ms
0.37.097.448 I slot launch_slot_: id 3 | task 1277 | processing task, is_child = 0
0.37.097.449 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.37.097.449 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.37.097.479 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.37.097.479 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.37.097.501 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.37.097.501 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.37.097.532 I slot update_slots: id 3 | task 1277 | Checking checkpoint with [0, 146] against 0...
0.37.102.575 W slot update_slots: id 3 | task 1277 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
0.38.967.733 I slot print_timing: id 3 | task 1277 | prompt eval time = 167.04 ms / 40 tokens ( 4.18 ms per token, 239.47 tokens per second)
0.38.967.736 I slot print_timing: id 3 | task 1277 | eval time = 1703.16 ms / 105 tokens ( 16.22 ms per token, 61.65 tokens per second)
0.38.967.737 I slot print_timing: id 3 | task 1277 | total time = 1870.20 ms / 145 tokens
0.38.967.738 I slot print_timing: id 3 | task 1277 | graphs reused = 1349
0.38.967.774 I slot release: id 3 | task 1277 | stop processing: n_tokens = 290, truncated = 0
0.38.967.782 I srv update_slots: all slots are idle
0.38.973.033 I srv operator(): Chat format: peg-native
0.38.975.010 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.798 (> 0.100 thold), f_keep = 0.517
0.38.975.034 I slot launch_slot_: id 3 | task 1385 | processing task, is_child = 0
0.38.975.034 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.975.035 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.38.975.061 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.975.061 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.38.975.082 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.975.087 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.38.975.110 I slot update_slots: id 3 | task 1385 | Checking checkpoint with [0, 146] against 0...
0.38.980.456 W slot update_slots: id 3 | task 1385 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
0.42.152.778 I slot print_timing: id 3 | task 1385 | n_decoded = 184, tg = 61.18 t/s, tg_3s = 61.18 t/s
0.45.158.586 I slot print_timing: id 3 | task 1385 | n_decoded = 366, tg = 60.86 t/s, tg_3s = 60.55 t/s
0.48.161.626 I slot print_timing: id 3 | task 1385 | n_decoded = 548, tg = 60.78 t/s, tg_3s = 60.61 t/s
0.48.963.795 I slot print_timing: id 3 | task 1385 | prompt eval time = 170.01 ms / 42 tokens ( 4.05 ms per token, 247.05 tokens per second)
0.48.963.798 I slot print_timing: id 3 | task 1385 | eval time = 9818.67 ms / 596 tokens ( 16.47 ms per token, 60.70 tokens per second)
0.48.963.799 I slot print_timing: id 3 | task 1385 | total time = 9988.68 ms / 638 tokens
0.48.963.800 I slot print_timing: id 3 | task 1385 | graphs reused = 1940
0.48.963.835 I slot release: id 3 | task 1385 | stop processing: n_tokens = 783, truncated = 0
0.48.963.845 I srv update_slots: all slots are idle
0.48.969.062 I srv operator(): Chat format: peg-native
0.48.971.036 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.862 (> 0.100 thold), f_keep = 0.192
0.48.971.038 I srv get_availabl: updating prompt cache
0.48.971.130 W srv prompt_save: - saving prompt with length 783, total state size = 74.955 MiB (draft: 0.000 MiB)
0.49.033.358 I srv load: - looking for better prompt, base f_keep = 0.192, sim = 0.862
0.49.033.370 I srv update: - cache state: 4 prompts, 273.111 MiB (limits: 8192.000 MiB, 24576 tokens, 72618 est)
0.49.033.371 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
0.49.033.372 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
0.49.033.372 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
0.49.033.373 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
0.49.033.374 I srv get_availabl: prompt cache update took 62.34 ms
0.49.033.440 I slot launch_slot_: id 3 | task 1984 | processing task, is_child = 0
0.49.033.441 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.49.033.442 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.49.033.472 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.49.033.472 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.49.033.494 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.49.033.494 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.49.033.524 I slot update_slots: id 3 | task 1984 | Checking checkpoint with [0, 146] against 0...
0.49.038.876 W slot update_slots: id 3 | task 1984 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
0.52.189.871 I slot print_timing: id 3 | task 1984 | n_decoded = 184, tg = 61.04 t/s, tg_3s = 61.04 t/s
0.55.199.977 I slot print_timing: id 3 | task 1984 | n_decoded = 366, tg = 60.75 t/s, tg_3s = 60.46 t/s
0.58.203.001 I slot print_timing: id 3 | task 1984 | n_decoded = 548, tg = 60.70 t/s, tg_3s = 60.61 t/s
1.01.215.831 I slot print_timing: id 3 | task 1984 | n_decoded = 728, tg = 60.46 t/s, tg_3s = 59.74 t/s
1.04.220.770 I slot print_timing: id 3 | task 1984 | n_decoded = 906, tg = 60.22 t/s, tg_3s = 59.24 t/s
1.07.229.946 I slot print_timing: id 3 | task 1984 | n_decoded = 1083, tg = 59.99 t/s, tg_3s = 58.82 t/s
1.10.232.069 I slot print_timing: id 3 | task 1984 | n_decoded = 1259, tg = 59.79 t/s, tg_3s = 58.63 t/s
1.13.244.821 I slot print_timing: id 3 | task 1984 | n_decoded = 1437, tg = 59.70 t/s, tg_3s = 59.08 t/s
1.16.254.839 I slot print_timing: id 3 | task 1984 | n_decoded = 1614, tg = 59.60 t/s, tg_3s = 58.80 t/s
1.19.256.144 I slot print_timing: id 3 | task 1984 | n_decoded = 1790, tg = 59.51 t/s, tg_3s = 58.64 t/s
1.22.256.319 I slot print_timing: id 3 | task 1984 | n_decoded = 1966, tg = 59.43 t/s, tg_3s = 58.66 t/s
1.25.260.546 I slot print_timing: id 3 | task 1984 | n_decoded = 2142, tg = 59.36 t/s, tg_3s = 58.58 t/s
1.28.273.584 I slot print_timing: id 3 | task 1984 | n_decoded = 2319, tg = 59.31 t/s, tg_3s = 58.74 t/s
1.31.277.036 I slot print_timing: id 3 | task 1984 | n_decoded = 2494, tg = 59.24 t/s, tg_3s = 58.27 t/s
1.34.283.342 I slot print_timing: id 3 | task 1984 | n_decoded = 2667, tg = 59.13 t/s, tg_3s = 57.55 t/s
1.37.297.318 I slot print_timing: id 3 | task 1984 | n_decoded = 2838, tg = 58.98 t/s, tg_3s = 56.74 t/s
1.40.306.249 I slot print_timing: id 3 | task 1984 | n_decoded = 3008, tg = 58.83 t/s, tg_3s = 56.50 t/s
1.43.307.244 I slot print_timing: id 3 | task 1984 | n_decoded = 3177, tg = 58.69 t/s, tg_3s = 56.31 t/s
1.46.312.264 I slot print_timing: id 3 | task 1984 | n_decoded = 3346, tg = 58.56 t/s, tg_3s = 56.24 t/s
1.49.319.392 I slot print_timing: id 3 | task 1984 | n_decoded = 3514, tg = 58.43 t/s, tg_3s = 55.87 t/s
1.52.329.330 I slot print_timing: id 3 | task 1984 | n_decoded = 3682, tg = 58.30 t/s, tg_3s = 55.82 t/s
1.55.338.372 I slot print_timing: id 3 | task 1984 | n_decoded = 3849, tg = 58.17 t/s, tg_3s = 55.50 t/s
1.58.355.120 I slot print_timing: id 3 | task 1984 | n_decoded = 4018, tg = 58.08 t/s, tg_3s = 56.02 t/s
2.01.356.206 I slot print_timing: id 3 | task 1984 | n_decoded = 4188, tg = 58.02 t/s, tg_3s = 56.65 t/s
2.04.362.021 I slot print_timing: id 3 | task 1984 | n_decoded = 4358, tg = 57.96 t/s, tg_3s = 56.56 t/s
2.07.364.030 I slot print_timing: id 3 | task 1984 | n_decoded = 4527, tg = 57.90 t/s, tg_3s = 56.30 t/s
2.10.370.266 I slot print_timing: id 3 | task 1984 | n_decoded = 4695, tg = 57.82 t/s, tg_3s = 55.88 t/s
2.13.376.169 I slot print_timing: id 3 | task 1984 | n_decoded = 4862, tg = 57.74 t/s, tg_3s = 55.56 t/s
2.16.387.058 I slot print_timing: id 3 | task 1984 | n_decoded = 5029, tg = 57.66 t/s, tg_3s = 55.47 t/s
2.19.396.098 I slot print_timing: id 3 | task 1984 | n_decoded = 5196, tg = 57.59 t/s, tg_3s = 55.50 t/s
2.22.398.244 I slot print_timing: id 3 | task 1984 | n_decoded = 5362, tg = 57.52 t/s, tg_3s = 55.29 t/s
2.25.409.636 I slot print_timing: id 3 | task 1984 | n_decoded = 5528, tg = 57.44 t/s, tg_3s = 55.12 t/s
2.28.419.674 I slot print_timing: id 3 | task 1984 | n_decoded = 5695, tg = 57.38 t/s, tg_3s = 55.48 t/s
2.31.426.670 I slot print_timing: id 3 | task 1984 | n_decoded = 5861, tg = 57.32 t/s, tg_3s = 55.20 t/s
2.34.430.783 I slot print_timing: id 3 | task 1984 | n_decoded = 6027, tg = 57.26 t/s, tg_3s = 55.26 t/s
2.36.542.785 I slot print_timing: id 3 | task 1984 | prompt eval time = 142.04 ms / 28 tokens ( 5.07 ms per token, 197.12 tokens per second)
2.36.542.788 I slot print_timing: id 3 | task 1984 | eval time = 107367.21 ms / 6144 tokens ( 17.48 ms per token, 57.22 tokens per second)
2.36.542.789 I slot print_timing: id 3 | task 1984 | total time = 107509.26 ms / 6172 tokens
2.36.542.790 I slot print_timing: id 3 | task 1984 | graphs reused = 8058
2.36.542.828 I slot release: id 3 | task 1984 | stop processing: n_tokens = 6317, truncated = 0
2.36.542.857 I srv update_slots: all slots are idle
2.36.548.935 I srv operator(): Chat format: peg-native
2.36.550.950 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.559 (> 0.100 thold), f_keep = 0.018
2.36.550.951 I srv get_availabl: updating prompt cache
2.36.551.347 W srv prompt_save: - saving prompt with length 6317, total state size = 448.513 MiB (draft: 0.000 MiB)
2.36.993.165 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.559
2.36.993.175 I srv update: - cache state: 5 prompts, 731.963 MiB (limits: 8192.000 MiB, 24576 tokens, 97794 est)
2.36.993.176 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
2.36.993.177 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
2.36.993.177 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
2.36.993.178 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
2.36.993.178 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
2.36.993.180 I srv get_availabl: prompt cache update took 442.23 ms
2.36.993.224 I slot launch_slot_: id 3 | task 8131 | processing task, is_child = 0
2.36.993.225 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.36.993.225 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.36.993.279 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.36.993.280 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.36.993.327 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.36.993.327 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.36.993.383 I slot update_slots: id 3 | task 8131 | Checking checkpoint with [0, 146] against 0...
2.36.993.385 W slot update_slots: id 3 | task 8131 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
2.36.993.386 W slot update_slots: id 3 | task 8131 | erased invalidated context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_swa = 4096, pos_next = 0, size = 10.338 MiB)
2.37.310.436 I slot create_check: id 3 | task 8131 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 166, n_tokens = 167, size = 11.745 MiB)
2.40.458.679 I slot print_timing: id 3 | task 8131 | n_decoded = 183, tg = 60.96 t/s, tg_3s = 60.96 t/s
2.43.460.463 I slot print_timing: id 3 | task 8131 | n_decoded = 364, tg = 60.63 t/s, tg_3s = 60.30 t/s
2.46.468.652 I slot print_timing: id 3 | task 8131 | n_decoded = 546, tg = 60.59 t/s, tg_3s = 60.50 t/s
2.49.479.887 I slot print_timing: id 3 | task 8131 | n_decoded = 726, tg = 60.38 t/s, tg_3s = 59.78 t/s
2.52.486.159 I slot print_timing: id 3 | task 8131 | n_decoded = 907, tg = 60.35 t/s, tg_3s = 60.21 t/s
2.55.082.171 I slot print_timing: id 3 | task 8131 | prompt eval time = 463.37 ms / 204 tokens ( 2.27 ms per token, 440.25 tokens per second)
2.55.082.175 I slot print_timing: id 3 | task 8131 | eval time = 17625.41 ms / 1064 tokens ( 16.57 ms per token, 60.37 tokens per second)
2.55.082.176 I slot print_timing: id 3 | task 8131 | total time = 18088.78 ms / 1268 tokens
2.55.082.176 I slot print_timing: id 3 | task 8131 | graphs reused = 9116
2.55.082.215 I slot release: id 3 | task 8131 | stop processing: n_tokens = 1267, truncated = 0
2.55.082.227 I srv update_slots: all slots are idle
2.55.087.837 I srv operator(): Chat format: peg-native
2.55.089.872 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.746 (> 0.100 thold), f_keep = 0.134
2.55.089.873 I srv get_availabl: updating prompt cache
2.55.089.995 W srv prompt_save: - saving prompt with length 1267, total state size = 121.286 MiB (draft: 0.000 MiB)
2.55.192.554 I srv load: - looking for better prompt, base f_keep = 0.134, sim = 0.746
2.55.192.563 I srv update: - cache state: 6 prompts, 864.994 MiB (limits: 8192.000 MiB, 24576 tokens, 94753 est)
2.55.192.564 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
2.55.192.565 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
2.55.192.566 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
2.55.192.566 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
2.55.192.567 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
2.55.192.568 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
2.55.192.569 I srv get_availabl: prompt cache update took 102.69 ms
2.55.192.614 I slot launch_slot_: id 3 | task 9198 | processing task, is_child = 0
2.55.192.615 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.55.192.616 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.55.192.652 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.55.192.653 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.55.192.709 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.55.192.709 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.55.192.740 I slot update_slots: id 3 | task 9198 | Checking checkpoint with [0, 166] against 0...
2.55.198.353 W slot update_slots: id 3 | task 9198 | restored context checkpoint (pos_min = 0, pos_max = 166, n_tokens = 167, n_past = 166, size = 11.745 MiB)
2.58.461.941 I slot print_timing: id 3 | task 9198 | n_decoded = 184, tg = 61.24 t/s, tg_3s = 61.24 t/s
3.00.548.000 I slot print_timing: id 3 | task 9198 | prompt eval time = 264.86 ms / 62 tokens ( 4.27 ms per token, 234.08 tokens per second)
3.00.548.003 I slot print_timing: id 3 | task 9198 | eval time = 5090.39 ms / 311 tokens ( 16.37 ms per token, 61.10 tokens per second)
3.00.548.004 I slot print_timing: id 3 | task 9198 | total time = 5355.26 ms / 373 tokens
3.00.548.005 I slot print_timing: id 3 | task 9198 | graphs reused = 9423
3.00.548.047 I slot release: id 3 | task 9198 | stop processing: n_tokens = 538, truncated = 0
3.00.548.058 I srv update_slots: all slots are idle
3.00.553.423 I srv operator(): Chat format: peg-native
3.00.555.474 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.759 (> 0.100 thold), f_keep = 0.316
3.00.555.476 I srv get_availabl: updating prompt cache
3.00.555.547 W srv prompt_save: - saving prompt with length 538, total state size = 51.502 MiB (draft: 0.000 MiB)
3.00.601.233 I srv load: - looking for better prompt, base f_keep = 0.316, sim = 0.759
3.00.601.246 I srv update: - cache state: 7 prompts, 928.240 MiB (limits: 8192.000 MiB, 24576 tokens, 93045 est)
3.00.601.247 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
3.00.601.247 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
3.00.601.248 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
3.00.601.248 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
3.00.601.249 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
3.00.601.249 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
3.00.601.250 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
3.00.601.251 I srv get_availabl: prompt cache update took 45.77 ms
3.00.601.316 I slot launch_slot_: id 3 | task 9512 | processing task, is_child = 0
3.00.601.317 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.00.601.318 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.00.601.351 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.00.601.351 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.00.601.372 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.00.601.373 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.00.601.402 I slot update_slots: id 3 | task 9512 | Checking checkpoint with [0, 166] against 0...
3.00.606.955 W slot update_slots: id 3 | task 9512 | restored context checkpoint (pos_min = 0, pos_max = 166, n_tokens = 167, n_past = 166, size = 11.745 MiB)
3.03.840.849 I slot print_timing: id 3 | task 9512 | n_decoded = 185, tg = 61.37 t/s, tg_3s = 61.37 t/s
3.06.852.053 I slot print_timing: id 3 | task 9512 | n_decoded = 368, tg = 61.07 t/s, tg_3s = 60.77 t/s
3.09.860.126 I slot print_timing: id 3 | task 9512 | n_decoded = 550, tg = 60.88 t/s, tg_3s = 60.50 t/s
3.12.869.195 I slot print_timing: id 3 | task 9512 | n_decoded = 732, tg = 60.78 t/s, tg_3s = 60.48 t/s
3.15.882.937 I slot print_timing: id 3 | task 9512 | n_decoded = 912, tg = 60.57 t/s, tg_3s = 59.73 t/s
3.18.889.315 I slot print_timing: id 3 | task 9512 | n_decoded = 1091, tg = 60.40 t/s, tg_3s = 59.54 t/s
3.19.193.163 I slot print_timing: id 3 | task 9512 | prompt eval time = 224.96 ms / 58 tokens ( 3.88 ms per token, 257.82 tokens per second)
3.19.193.167 I slot print_timing: id 3 | task 9512 | eval time = 18366.79 ms / 1109 tokens ( 16.56 ms per token, 60.38 tokens per second)
3.19.193.167 I slot print_timing: id 3 | task 9512 | total time = 18591.76 ms / 1167 tokens
3.19.193.168 I slot print_timing: id 3 | task 9512 | graphs reused = 10525
3.19.193.243 I slot release: id 3 | task 9512 | stop processing: n_tokens = 1332, truncated = 0
3.19.193.257 I srv update_slots: all slots are idle
3.19.198.986 I srv operator(): Chat format: peg-native
3.19.201.120 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.726 (> 0.100 thold), f_keep = 0.128
3.19.201.122 I srv get_availabl: updating prompt cache
3.19.201.246 W srv prompt_save: - saving prompt with length 1332, total state size = 127.508 MiB (draft: 0.000 MiB)
3.19.343.081 I srv load: - looking for better prompt, base f_keep = 0.128, sim = 0.726
3.19.343.091 I srv update: - cache state: 8 prompts, 1067.493 MiB (limits: 8192.000 MiB, 24576 tokens, 91129 est)
3.19.343.092 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
3.19.343.092 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
3.19.343.093 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
3.19.343.093 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
3.19.343.094 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
3.19.343.094 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
3.19.343.095 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
3.19.343.095 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
3.19.343.097 I srv get_availabl: prompt cache update took 141.97 ms
3.19.343.147 I slot launch_slot_: id 3 | task 10624 | processing task, is_child = 0
3.19.343.148 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.19.343.149 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.19.343.181 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.19.343.182 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.19.343.203 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.19.343.204 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.19.343.234 I slot update_slots: id 3 | task 10624 | Checking checkpoint with [0, 166] against 0...
3.19.349.281 W slot update_slots: id 3 | task 10624 | restored context checkpoint (pos_min = 0, pos_max = 166, n_tokens = 167, n_past = 166, size = 11.745 MiB)
3.22.597.727 I slot print_timing: id 3 | task 10624 | n_decoded = 182, tg = 60.63 t/s, tg_3s = 60.63 t/s
3.25.601.122 I slot print_timing: id 3 | task 10624 | n_decoded = 363, tg = 60.45 t/s, tg_3s = 60.27 t/s
3.28.617.874 I slot print_timing: id 3 | task 10624 | n_decoded = 545, tg = 60.41 t/s, tg_3s = 60.33 t/s
3.31.620.881 I slot print_timing: id 3 | task 10624 | n_decoded = 725, tg = 60.29 t/s, tg_3s = 59.94 t/s
3.34.631.835 I slot print_timing: id 3 | task 10624 | n_decoded = 903, tg = 60.06 t/s, tg_3s = 59.12 t/s
3.37.643.467 I slot print_timing: id 3 | task 10624 | n_decoded = 1081, tg = 59.90 t/s, tg_3s = 59.10 t/s
3.40.646.476 I slot print_timing: id 3 | task 10624 | n_decoded = 1261, tg = 59.90 t/s, tg_3s = 59.94 t/s
3.43.650.101 I slot print_timing: id 3 | task 10624 | n_decoded = 1440, tg = 59.86 t/s, tg_3s = 59.59 t/s
3.46.659.681 I slot print_timing: id 3 | task 10624 | n_decoded = 1618, tg = 59.78 t/s, tg_3s = 59.14 t/s
3.49.672.715 I slot print_timing: id 3 | task 10624 | n_decoded = 1795, tg = 59.68 t/s, tg_3s = 58.74 t/s
3.52.673.112 I slot print_timing: id 3 | task 10624 | n_decoded = 1971, tg = 59.59 t/s, tg_3s = 58.66 t/s
3.55.685.278 I slot print_timing: id 3 | task 10624 | n_decoded = 2147, tg = 59.49 t/s, tg_3s = 58.43 t/s
3.58.695.280 I slot print_timing: id 3 | task 10624 | n_decoded = 2322, tg = 59.39 t/s, tg_3s = 58.14 t/s
4.01.569.988 I slot print_timing: id 3 | task 10624 | prompt eval time = 252.59 ms / 68 tokens ( 3.71 ms per token, 269.22 tokens per second)
4.01.569.992 I slot print_timing: id 3 | task 10624 | eval time = 41974.17 ms / 2488 tokens ( 16.87 ms per token, 59.27 tokens per second)
4.01.569.992 I slot print_timing: id 3 | task 10624 | total time = 42226.75 ms / 2556 tokens
4.01.569.993 I slot print_timing: id 3 | task 10624 | graphs reused = 13001
4.01.570.029 I slot release: id 3 | task 10624 | stop processing: n_tokens = 2721, truncated = 0
4.01.570.047 I srv update_slots: all slots are idle
4.01.575.654 I srv operator(): Chat format: peg-native
4.01.577.669 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.821 (> 0.100 thold), f_keep = 0.062
4.01.577.671 I srv get_availabl: updating prompt cache
4.01.577.886 W srv prompt_save: - saving prompt with length 2721, total state size = 260.472 MiB (draft: 0.000 MiB)
4.01.848.857 I srv load: - looking for better prompt, base f_keep = 0.062, sim = 0.821
4.01.848.869 I srv update: - cache state: 9 prompts, 1339.710 MiB (limits: 8192.000 MiB, 24576 tokens, 89250 est)
4.01.848.870 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
4.01.848.871 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
4.01.848.871 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
4.01.848.872 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
4.01.848.872 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
4.01.848.873 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
4.01.848.873 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
4.01.848.874 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
4.01.848.874 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
4.01.848.876 I srv get_availabl: prompt cache update took 271.20 ms
4.01.848.919 I slot launch_slot_: id 3 | task 13115 | processing task, is_child = 0
4.01.848.920 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.01.848.920 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.01.848.965 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.01.848.966 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.01.849.005 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.01.849.005 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.01.849.037 I slot update_slots: id 3 | task 13115 | Checking checkpoint with [0, 166] against 0...
4.01.855.125 W slot update_slots: id 3 | task 13115 | restored context checkpoint (pos_min = 0, pos_max = 166, n_tokens = 167, n_past = 166, size = 11.745 MiB)
4.05.029.683 I slot print_timing: id 3 | task 13115 | n_decoded = 184, tg = 61.03 t/s, tg_3s = 61.03 t/s
4.08.043.797 I slot print_timing: id 3 | task 13115 | n_decoded = 367, tg = 60.87 t/s, tg_3s = 60.71 t/s
4.11.054.842 I slot print_timing: id 3 | task 13115 | n_decoded = 549, tg = 60.73 t/s, tg_3s = 60.44 t/s
4.14.058.004 I slot print_timing: id 3 | task 13115 | n_decoded = 729, tg = 60.53 t/s, tg_3s = 59.94 t/s
4.14.925.558 I slot print_timing: id 3 | task 13115 | prompt eval time = 165.76 ms / 41 tokens ( 4.04 ms per token, 247.34 tokens per second)
4.14.925.562 I slot print_timing: id 3 | task 13115 | eval time = 12910.75 ms / 781 tokens ( 16.53 ms per token, 60.49 tokens per second)
4.14.925.563 I slot print_timing: id 3 | task 13115 | total time = 13076.51 ms / 822 tokens
4.14.925.564 I slot print_timing: id 3 | task 13115 | graphs reused = 13777
4.14.925.607 I slot release: id 3 | task 13115 | stop processing: n_tokens = 987, truncated = 0
4.14.925.619 I srv update_slots: all slots are idle
4.14.931.399 I srv operator(): Chat format: peg-native
4.14.933.427 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.762 (> 0.100 thold), f_keep = 0.172
4.14.933.428 I srv get_availabl: updating prompt cache
4.14.933.525 W srv prompt_save: - saving prompt with length 987, total state size = 94.483 MiB (draft: 0.000 MiB)
4.15.009.824 I srv load: - looking for better prompt, base f_keep = 0.172, sim = 0.762
4.15.009.839 I srv update: - cache state: 10 prompts, 1445.938 MiB (limits: 8192.000 MiB, 24576 tokens, 88285 est)
4.15.009.840 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
4.15.009.841 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
4.15.009.841 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
4.15.009.842 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
4.15.009.842 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
4.15.009.843 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
4.15.009.843 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
4.15.009.844 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
4.15.009.844 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
4.15.009.845 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
4.15.009.846 I srv get_availabl: prompt cache update took 76.42 ms
4.15.009.911 I slot launch_slot_: id 3 | task 13899 | processing task, is_child = 0
4.15.009.913 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.15.009.913 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.15.009.944 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.15.009.945 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.15.009.966 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.15.009.966 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.15.009.997 I slot update_slots: id 3 | task 13899 | Checking checkpoint with [0, 166] against 0...
4.15.016.277 W slot update_slots: id 3 | task 13899 | restored context checkpoint (pos_min = 0, pos_max = 166, n_tokens = 167, n_past = 166, size = 11.745 MiB)
4.18.240.379 I slot print_timing: id 3 | task 13899 | n_decoded = 182, tg = 60.59 t/s, tg_3s = 60.59 t/s
4.21.252.443 I slot print_timing: id 3 | task 13899 | n_decoded = 364, tg = 60.51 t/s, tg_3s = 60.42 t/s
4.24.256.959 I slot print_timing: id 3 | task 13899 | n_decoded = 546, tg = 60.53 t/s, tg_3s = 60.58 t/s
4.24.995.530 I slot print_timing: id 3 | task 13899 | prompt eval time = 226.71 ms / 57 tokens ( 3.98 ms per token, 251.43 tokens per second)
4.24.995.533 I slot print_timing: id 3 | task 13899 | eval time = 9758.82 ms / 589 tokens ( 16.57 ms per token, 60.36 tokens per second)
4.24.995.534 I slot print_timing: id 3 | task 13899 | total time = 9985.53 ms / 646 tokens
4.24.995.535 I slot print_timing: id 3 | task 13899 | graphs reused = 14361
4.24.995.580 I slot release: id 3 | task 13899 | stop processing: n_tokens = 811, truncated = 0
4.24.995.590 I srv update_slots: all slots are idle
4.25.001.184 I srv operator(): Chat format: peg-native
4.25.003.195 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.483 (> 0.100 thold), f_keep = 0.141
4.25.003.197 I srv get_availabl: updating prompt cache
4.25.003.287 W srv prompt_save: - saving prompt with length 811, total state size = 77.635 MiB (draft: 0.000 MiB)
4.25.067.968 I srv load: - looking for better prompt, base f_keep = 0.141, sim = 0.483
4.25.067.982 I srv update: - cache state: 11 prompts, 1535.318 MiB (limits: 8192.000 MiB, 24576 tokens, 87473 est)
4.25.067.983 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
4.25.067.984 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
4.25.067.985 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
4.25.067.985 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
4.25.067.986 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
4.25.067.986 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
4.25.067.987 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
4.25.067.987 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
4.25.067.988 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
4.25.067.988 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
4.25.067.989 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
4.25.067.990 I srv get_availabl: prompt cache update took 64.79 ms
4.25.068.055 I slot launch_slot_: id 3 | task 14491 | processing task, is_child = 0
4.25.068.056 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.25.068.056 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.25.068.085 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.25.068.086 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.25.068.107 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.25.068.107 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.25.068.138 I slot update_slots: id 3 | task 14491 | Checking checkpoint with [0, 166] against 0...
4.25.068.139 W slot update_slots: id 3 | task 14491 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
4.25.068.140 W slot update_slots: id 3 | task 14491 | erased invalidated context checkpoint (pos_min = 0, pos_max = 166, n_tokens = 167, n_swa = 4096, pos_next = 0, size = 11.745 MiB)
4.25.382.006 I slot create_check: id 3 | task 14491 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 161, n_tokens = 162, size = 11.393 MiB)
4.28.590.513 I slot print_timing: id 3 | task 14491 | n_decoded = 184, tg = 61.26 t/s, tg_3s = 61.26 t/s
4.29.915.933 I slot print_timing: id 3 | task 14491 | prompt eval time = 518.61 ms / 236 tokens ( 2.20 ms per token, 455.07 tokens per second)
4.29.915.938 I slot print_timing: id 3 | task 14491 | eval time = 4329.19 ms / 265 tokens ( 16.34 ms per token, 61.21 tokens per second)
4.29.915.938 I slot print_timing: id 3 | task 14491 | total time = 4847.79 ms / 501 tokens
4.29.915.939 I slot print_timing: id 3 | task 14491 | graphs reused = 14623
4.29.915.984 I slot release: id 3 | task 14491 | stop processing: n_tokens = 500, truncated = 0
4.29.915.993 I srv update_slots: all slots are idle
4.29.982.474 I srv operator(): Chat format: peg-native
4.29.984.520 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.863 (> 0.100 thold), f_keep = 0.340
4.29.984.522 I srv get_availabl: updating prompt cache
4.29.984.598 W srv prompt_save: - saving prompt with length 500, total state size = 47.864 MiB (draft: 0.000 MiB)
4.30.027.232 I srv load: - looking for better prompt, base f_keep = 0.340, sim = 0.863
4.30.027.247 I srv update: - cache state: 12 prompts, 1594.575 MiB (limits: 8192.000 MiB, 24576 tokens, 86791 est)
4.30.027.248 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
4.30.027.248 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
4.30.027.249 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
4.30.027.249 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
4.30.027.250 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
4.30.027.251 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
4.30.027.251 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
4.30.027.252 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
4.30.027.252 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
4.30.027.253 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
4.30.027.253 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
4.30.027.254 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
4.30.027.255 I srv get_availabl: prompt cache update took 42.73 ms
4.30.027.318 I slot launch_slot_: id 3 | task 14759 | processing task, is_child = 0
4.30.027.319 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.30.027.320 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.30.027.348 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.30.027.348 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.30.027.369 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.30.027.370 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.30.027.399 I slot update_slots: id 3 | task 14759 | Checking checkpoint with [0, 161] against 0...
4.30.032.812 W slot update_slots: id 3 | task 14759 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
4.32.925.419 I slot print_timing: id 3 | task 14759 | prompt eval time = 169.52 ms / 36 tokens ( 4.71 ms per token, 212.36 tokens per second)
4.32.925.424 I slot print_timing: id 3 | task 14759 | eval time = 2728.49 ms / 169 tokens ( 16.14 ms per token, 61.94 tokens per second)
4.32.925.425 I slot print_timing: id 3 | task 14759 | total time = 2898.01 ms / 205 tokens
4.32.925.425 I slot print_timing: id 3 | task 14759 | graphs reused = 14789
4.32.925.509 I slot release: id 3 | task 14759 | stop processing: n_tokens = 365, truncated = 0
4.32.925.522 I srv update_slots: all slots are idle
4.32.976.069 I srv operator(): Chat format: peg-native
4.32.978.229 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.466
4.32.978.232 I srv get_availabl: updating prompt cache
4.32.978.306 W srv prompt_save: - saving prompt with length 365, total state size = 34.941 MiB (draft: 0.000 MiB)
4.33.011.855 I srv load: - looking for better prompt, base f_keep = 0.466, sim = 0.833
4.33.011.873 I srv update: - cache state: 13 prompts, 1640.910 MiB (limits: 8192.000 MiB, 24576 tokens, 86163 est)
4.33.011.874 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
4.33.011.875 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
4.33.011.876 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
4.33.011.876 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
4.33.011.877 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
4.33.011.877 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
4.33.011.878 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
4.33.011.878 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
4.33.011.879 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
4.33.011.879 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
4.33.011.880 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
4.33.011.880 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
4.33.011.881 I srv update: - prompt 0x5a8713dd4a40: 365 tokens, checkpoints: 1, 46.334 MiB
4.33.011.882 I srv get_availabl: prompt cache update took 33.65 ms
4.33.011.944 I slot launch_slot_: id 3 | task 14931 | processing task, is_child = 0
4.33.011.945 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.33.011.946 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.33.011.976 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.33.011.976 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.33.011.997 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.33.011.997 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.33.012.026 I slot update_slots: id 3 | task 14931 | Checking checkpoint with [0, 161] against 0...
4.33.017.477 W slot update_slots: id 3 | task 14931 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
4.36.201.931 I slot print_timing: id 3 | task 14931 | n_decoded = 186, tg = 61.69 t/s, tg_3s = 61.69 t/s
4.39.210.578 I slot print_timing: id 3 | task 14931 | n_decoded = 371, tg = 61.59 t/s, tg_3s = 61.49 t/s
4.42.218.897 I slot print_timing: id 3 | task 14931 | n_decoded = 555, tg = 61.45 t/s, tg_3s = 61.16 t/s
4.43.414.821 I slot print_timing: id 3 | task 14931 | prompt eval time = 174.86 ms / 43 tokens ( 4.07 ms per token, 245.92 tokens per second)
4.43.414.825 I slot print_timing: id 3 | task 14931 | eval time = 10227.93 ms / 627 tokens ( 16.31 ms per token, 61.30 tokens per second)
4.43.414.825 I slot print_timing: id 3 | task 14931 | total time = 10402.79 ms / 670 tokens
4.43.414.826 I slot print_timing: id 3 | task 14931 | graphs reused = 15411
4.43.414.868 I slot release: id 3 | task 14931 | stop processing: n_tokens = 830, truncated = 0
4.43.414.879 I srv update_slots: all slots are idle
4.43.480.631 I srv operator(): Chat format: peg-native
4.43.482.884 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.733 (> 0.100 thold), f_keep = 0.205
4.43.482.887 I srv get_availabl: updating prompt cache
4.43.482.981 W srv prompt_save: - saving prompt with length 830, total state size = 79.454 MiB (draft: 0.000 MiB)
4.43.548.193 I srv load: - looking for better prompt, base f_keep = 0.205, sim = 0.733
4.43.548.209 I srv update: - cache state: 14 prompts, 1731.757 MiB (limits: 8192.000 MiB, 24576 tokens, 85569 est)
4.43.548.210 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
4.43.548.211 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
4.43.548.211 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
4.43.548.212 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
4.43.548.213 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
4.43.548.213 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
4.43.548.214 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
4.43.548.214 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
4.43.548.215 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
4.43.548.215 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
4.43.548.216 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
4.43.548.216 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
4.43.548.217 I srv update: - prompt 0x5a8713dd4a40: 365 tokens, checkpoints: 1, 46.334 MiB
4.43.548.217 I srv update: - prompt 0x5a8710079620: 830 tokens, checkpoints: 1, 90.847 MiB
4.43.548.218 I srv get_availabl: prompt cache update took 65.33 ms
4.43.548.291 I slot launch_slot_: id 3 | task 15561 | processing task, is_child = 0
4.43.548.291 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.43.548.292 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.43.548.325 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.43.548.325 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.43.548.346 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.43.548.347 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.43.548.376 I slot update_slots: id 3 | task 15561 | Checking checkpoint with [0, 161] against 0...
4.43.554.013 W slot update_slots: id 3 | task 15561 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
4.46.773.827 I slot print_timing: id 3 | task 15561 | n_decoded = 185, tg = 61.36 t/s, tg_3s = 61.36 t/s
4.49.785.627 I slot print_timing: id 3 | task 15561 | n_decoded = 368, tg = 61.06 t/s, tg_3s = 60.76 t/s
4.52.802.043 I slot print_timing: id 3 | task 15561 | n_decoded = 551, tg = 60.93 t/s, tg_3s = 60.67 t/s
4.55.817.547 I slot print_timing: id 3 | task 15561 | n_decoded = 732, tg = 60.70 t/s, tg_3s = 60.02 t/s
4.58.821.170 I slot print_timing: id 3 | task 15561 | n_decoded = 911, tg = 60.48 t/s, tg_3s = 59.59 t/s
5.01.835.766 I slot print_timing: id 3 | task 15561 | n_decoded = 1090, tg = 60.30 t/s, tg_3s = 59.38 t/s
5.04.850.739 I slot print_timing: id 3 | task 15561 | n_decoded = 1267, tg = 60.07 t/s, tg_3s = 58.71 t/s
5.07.857.858 I slot print_timing: id 3 | task 15561 | n_decoded = 1443, tg = 59.88 t/s, tg_3s = 58.53 t/s
5.10.862.994 I slot print_timing: id 3 | task 15561 | n_decoded = 1619, tg = 59.73 t/s, tg_3s = 58.57 t/s
5.13.872.105 I slot print_timing: id 3 | task 15561 | n_decoded = 1795, tg = 59.61 t/s, tg_3s = 58.49 t/s
5.14.532.111 I slot print_timing: id 3 | task 15561 | prompt eval time = 210.34 ms / 71 tokens ( 2.96 ms per token, 337.55 tokens per second)
5.14.532.115 I slot print_timing: id 3 | task 15561 | eval time = 30773.39 ms / 1833 tokens ( 16.79 ms per token, 59.56 tokens per second)
5.14.532.116 I slot print_timing: id 3 | task 15561 | total time = 30983.73 ms / 1904 tokens
5.14.532.117 I slot print_timing: id 3 | task 15561 | graphs reused = 17234
5.14.532.158 I slot release: id 3 | task 15561 | stop processing: n_tokens = 2064, truncated = 0
5.14.532.172 I srv update_slots: all slots are idle
5.14.609.746 I srv operator(): Chat format: peg-native
5.14.611.798 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.821 (> 0.100 thold), f_keep = 0.082
5.14.611.800 I srv get_availabl: updating prompt cache
5.14.611.957 W srv prompt_save: - saving prompt with length 2064, total state size = 197.580 MiB (draft: 0.000 MiB)
5.14.756.512 I srv load: - looking for better prompt, base f_keep = 0.082, sim = 0.821
5.14.756.527 I srv update: - cache state: 15 prompts, 1940.730 MiB (limits: 8192.000 MiB, 24576 tokens, 85067 est)
5.14.756.528 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
5.14.756.529 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
5.14.756.529 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
5.14.756.530 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
5.14.756.530 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
5.14.756.531 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
5.14.756.531 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
5.14.756.532 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
5.14.756.532 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
5.14.756.533 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
5.14.756.533 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
5.14.756.534 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
5.14.756.534 I srv update: - prompt 0x5a8713dd4a40: 365 tokens, checkpoints: 1, 46.334 MiB
5.14.756.535 I srv update: - prompt 0x5a8710079620: 830 tokens, checkpoints: 1, 90.847 MiB
5.14.756.535 I srv update: - prompt 0x5a870cfb0770: 2064 tokens, checkpoints: 1, 208.973 MiB
5.14.756.537 I srv get_availabl: prompt cache update took 144.74 ms
5.14.756.586 I slot launch_slot_: id 3 | task 17397 | processing task, is_child = 0
5.14.756.586 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.14.756.587 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.14.756.615 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.14.756.616 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.14.756.638 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.14.756.638 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.14.756.669 I slot update_slots: id 3 | task 17397 | Checking checkpoint with [0, 161] against 0...
5.14.762.487 W slot update_slots: id 3 | task 17397 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
5.17.948.439 I slot print_timing: id 3 | task 17397 | n_decoded = 185, tg = 61.48 t/s, tg_3s = 61.48 t/s
5.20.856.560 I slot print_timing: id 3 | task 17397 | prompt eval time = 182.83 ms / 46 tokens ( 3.97 ms per token, 251.60 tokens per second)
5.20.856.564 I slot print_timing: id 3 | task 17397 | eval time = 5917.06 ms / 363 tokens ( 16.30 ms per token, 61.35 tokens per second)
5.20.856.565 I slot print_timing: id 3 | task 17397 | total time = 6099.89 ms / 409 tokens
5.20.856.566 I slot print_timing: id 3 | task 17397 | graphs reused = 17593
5.20.856.610 I slot release: id 3 | task 17397 | stop processing: n_tokens = 569, truncated = 0
5.20.856.620 I srv update_slots: all slots are idle
5.20.906.755 I srv operator(): Chat format: peg-native
5.20.908.800 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.757 (> 0.100 thold), f_keep = 0.290
5.20.908.802 I srv get_availabl: updating prompt cache
5.20.908.869 W srv prompt_save: - saving prompt with length 569, total state size = 54.469 MiB (draft: 0.000 MiB)
5.20.955.876 I srv load: - looking for better prompt, base f_keep = 0.290, sim = 0.757
5.20.955.895 I srv update: - cache state: 16 prompts, 2006.592 MiB (limits: 8192.000 MiB, 24576 tokens, 84598 est)
5.20.955.896 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
5.20.955.896 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
5.20.955.897 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
5.20.955.897 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
5.20.955.898 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
5.20.955.898 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
5.20.955.899 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
5.20.955.899 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
5.20.955.900 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
5.20.955.901 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
5.20.955.902 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
5.20.955.903 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
5.20.955.903 I srv update: - prompt 0x5a8713dd4a40: 365 tokens, checkpoints: 1, 46.334 MiB
5.20.955.904 I srv update: - prompt 0x5a8710079620: 830 tokens, checkpoints: 1, 90.847 MiB
5.20.955.904 I srv update: - prompt 0x5a870cfb0770: 2064 tokens, checkpoints: 1, 208.973 MiB
5.20.955.905 I srv update: - prompt 0x5a871427acf0: 569 tokens, checkpoints: 1, 65.863 MiB
5.20.955.906 I srv get_availabl: prompt cache update took 47.10 ms
5.20.955.973 I slot launch_slot_: id 3 | task 17763 | processing task, is_child = 0
5.20.955.974 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.20.955.975 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.20.956.007 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.20.956.007 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.20.956.029 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.20.956.029 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.20.956.061 I slot update_slots: id 3 | task 17763 | Checking checkpoint with [0, 161] against 0...
5.20.961.893 W slot update_slots: id 3 | task 17763 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
5.23.507.468 I slot print_timing: id 3 | task 17763 | prompt eval time = 230.63 ms / 57 tokens ( 4.05 ms per token, 247.15 tokens per second)
5.23.507.472 I slot print_timing: id 3 | task 17763 | eval time = 2320.77 ms / 143 tokens ( 16.23 ms per token, 61.62 tokens per second)
5.23.507.473 I slot print_timing: id 3 | task 17763 | total time = 2551.40 ms / 200 tokens
5.23.507.473 I slot print_timing: id 3 | task 17763 | graphs reused = 17733
5.23.507.513 I slot release: id 3 | task 17763 | stop processing: n_tokens = 360, truncated = 0
5.23.507.524 I srv update_slots: all slots are idle
5.23.512.807 I srv operator(): Chat format: peg-native
5.23.514.861 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.805 (> 0.100 thold), f_keep = 0.458
5.23.514.863 I srv get_availabl: updating prompt cache
5.23.514.940 W srv prompt_save: - saving prompt with length 360, total state size = 34.463 MiB (draft: 0.000 MiB)
5.23.547.717 I srv load: - looking for better prompt, base f_keep = 0.458, sim = 0.805
5.23.547.729 I srv load: - found better prompt with f_keep = 0.466, sim = 0.829
5.23.563.466 I srv update: - cache state: 16 prompts, 2006.114 MiB (limits: 8192.000 MiB, 24576 tokens, 84598 est)
5.23.563.470 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
5.23.563.471 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
5.23.563.471 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
5.23.563.472 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
5.23.563.472 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
5.23.563.473 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
5.23.563.473 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
5.23.563.474 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
5.23.563.474 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
5.23.563.475 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
5.23.563.475 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
5.23.563.476 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
5.23.563.476 I srv update: - prompt 0x5a8710079620: 830 tokens, checkpoints: 1, 90.847 MiB
5.23.563.477 I srv update: - prompt 0x5a870cfb0770: 2064 tokens, checkpoints: 1, 208.973 MiB
5.23.563.477 I srv update: - prompt 0x5a871427acf0: 569 tokens, checkpoints: 1, 65.863 MiB
5.23.563.478 I srv update: - prompt 0x5a87143d9ae0: 360 tokens, checkpoints: 1, 45.856 MiB
5.23.563.479 I srv get_availabl: prompt cache update took 48.62 ms
5.23.563.532 I slot launch_slot_: id 3 | task 17909 | processing task, is_child = 0
5.23.563.533 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.23.563.534 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.23.563.566 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.23.563.566 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.23.563.587 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.23.563.587 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.23.563.616 I slot update_slots: id 3 | task 17909 | Checking checkpoint with [0, 161] against 0...
5.23.568.935 W slot update_slots: id 3 | task 17909 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
5.26.740.551 I slot print_timing: id 3 | task 17909 | n_decoded = 185, tg = 61.64 t/s, tg_3s = 61.64 t/s
5.29.754.579 I slot print_timing: id 3 | task 17909 | n_decoded = 369, tg = 61.35 t/s, tg_3s = 61.05 t/s
5.32.350.894 I slot print_timing: id 3 | task 17909 | prompt eval time = 175.82 ms / 44 tokens ( 4.00 ms per token, 250.25 tokens per second)
5.32.350.898 I slot print_timing: id 3 | task 17909 | eval time = 8611.44 ms / 527 tokens ( 16.34 ms per token, 61.20 tokens per second)
5.32.350.899 I slot print_timing: id 3 | task 17909 | total time = 8787.26 ms / 571 tokens
5.32.350.900 I slot print_timing: id 3 | task 17909 | graphs reused = 18256
5.32.350.930 I slot release: id 3 | task 17909 | stop processing: n_tokens = 731, truncated = 0
5.32.350.939 I srv update_slots: all slots are idle
5.32.417.735 I srv operator(): Chat format: peg-native
5.32.420.179 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.829 (> 0.100 thold), f_keep = 0.233
5.32.420.182 I srv get_availabl: updating prompt cache
5.32.420.274 W srv prompt_save: - saving prompt with length 731, total state size = 69.977 MiB (draft: 0.000 MiB)
5.32.474.716 I srv load: - looking for better prompt, base f_keep = 0.233, sim = 0.829
5.32.474.734 I srv update: - cache state: 17 prompts, 2087.484 MiB (limits: 8192.000 MiB, 24576 tokens, 84169 est)
5.32.474.736 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
5.32.474.737 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
5.32.474.738 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
5.32.474.738 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
5.32.474.739 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
5.32.474.739 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
5.32.474.740 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
5.32.474.740 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
5.32.474.741 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
5.32.474.741 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
5.32.474.742 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
5.32.474.742 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
5.32.474.743 I srv update: - prompt 0x5a8710079620: 830 tokens, checkpoints: 1, 90.847 MiB
5.32.474.744 I srv update: - prompt 0x5a870cfb0770: 2064 tokens, checkpoints: 1, 208.973 MiB
5.32.474.744 I srv update: - prompt 0x5a871427acf0: 569 tokens, checkpoints: 1, 65.863 MiB
5.32.474.744 I srv update: - prompt 0x5a87143d9ae0: 360 tokens, checkpoints: 1, 45.856 MiB
5.32.474.745 I srv update: - prompt 0x5a8713dd4a40: 731 tokens, checkpoints: 1, 81.370 MiB
5.32.474.746 I srv get_availabl: prompt cache update took 54.56 ms
5.32.474.798 I slot launch_slot_: id 3 | task 18439 | processing task, is_child = 0
5.32.474.799 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.32.474.800 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.32.474.828 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.32.474.828 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.32.474.849 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.32.474.850 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.32.474.880 I slot update_slots: id 3 | task 18439 | Checking checkpoint with [0, 161] against 0...
5.32.480.696 W slot update_slots: id 3 | task 18439 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
5.35.657.263 I slot print_timing: id 3 | task 18439 | n_decoded = 185, tg = 61.44 t/s, tg_3s = 61.44 t/s
5.38.667.693 I slot print_timing: id 3 | task 18439 | n_decoded = 368, tg = 61.12 t/s, tg_3s = 60.79 t/s
5.41.680.265 I slot print_timing: id 3 | task 18439 | n_decoded = 550, tg = 60.88 t/s, tg_3s = 60.41 t/s
5.44.693.942 I slot print_timing: id 3 | task 18439 | n_decoded = 731, tg = 60.68 t/s, tg_3s = 60.06 t/s
5.47.700.502 I slot print_timing: id 3 | task 18439 | n_decoded = 910, tg = 60.45 t/s, tg_3s = 59.54 t/s
5.48.608.528 I slot print_timing: id 3 | task 18439 | prompt eval time = 171.54 ms / 44 tokens ( 3.90 ms per token, 256.50 tokens per second)
5.48.608.532 I slot print_timing: id 3 | task 18439 | eval time = 15962.10 ms / 964 tokens ( 16.56 ms per token, 60.39 tokens per second)
5.48.608.533 I slot print_timing: id 3 | task 18439 | total time = 16133.64 ms / 1008 tokens
5.48.608.533 I slot print_timing: id 3 | task 18439 | graphs reused = 19214
5.48.608.576 I slot release: id 3 | task 18439 | stop processing: n_tokens = 1168, truncated = 0
5.48.608.588 I srv update_slots: all slots are idle
5.48.668.468 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink