qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results_roles/gpt-oss-120b.server.log

784 lines
77 KiB
Plaintext
Raw Permalink Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.064.887 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.064.891 I device_info:
0.00.064.913 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.064.918 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.064.925 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.064.983 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.064.986 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.065.025 I srv init: running without SSL
0.00.065.083 I srv init: using 23 threads for HTTP server
0.00.065.411 I srv start: binding port with default address family
0.00.066.598 I srv llama_server: loading model
0.00.066.607 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/openai_gpt-oss-120b-Q4_K_M/openai_gpt-oss-120b-Q4_K_M-00001-of-00002.gguf'
0.00.066.655 I common_init_result: fitting params to device memory ...
0.00.066.659 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.08.159.468 W load: setting token '<|message|>' (200008) attribute to USER_DEFINED (16), old attributes: 8
0.08.159.474 W load: setting token '<|start|>' (200006) attribute to USER_DEFINED (16), old attributes: 8
0.08.159.475 W load: setting token '<|constrain|>' (200003) attribute to USER_DEFINED (16), old attributes: 8
0.08.160.855 W load: setting token '<|channel|>' (200005) attribute to USER_DEFINED (16), old attributes: 8
0.08.165.867 W load: special_eog_ids contains both '<|return|>' and '<|call|>', or '<|calls|>' and '<|flush|>' tokens, removing '<|end|>' token from EOG list
0.47.441.244 W llama_context: n_ctx_seq (24576) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
0.47.668.471 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.47.867.283 I srv load_model: initializing slots, n_slots = 4
0.47.912.630 W common_speculative_init: no implementations specified for speculative decoding
0.47.912.636 I slot load_model: id 0 | task -1 | new slot, n_ctx = 24576
0.47.912.641 I slot load_model: id 1 | task -1 | new slot, n_ctx = 24576
0.47.912.641 I slot load_model: id 2 | task -1 | new slot, n_ctx = 24576
0.47.912.641 I slot load_model: id 3 | task -1 | new slot, n_ctx = 24576
0.47.912.682 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.47.912.682 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.47.912.683 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.47.912.683 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.47.913.409 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.47.918.608 I init: chat template, example_format: '<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.
Knowledge cutoff: 2024-06
Current date: 2026-07-12
Reasoning: medium
# Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>developer<|message|># Instructions
You are a helpful assistant<|end|><|start|>user<|message|>Hello<|end|><|start|>assistant<|channel|>final<|message|>Hi there<|end|><|start|>user<|message|>How are you?<|end|><|start|>assistant'
0.47.919.174 I srv init: init: chat template, thinking = 1
0.47.919.214 I srv llama_server: model loaded
0.47.919.217 I srv llama_server: server is listening on http://127.0.0.1:8090
0.47.919.226 I srv update_slots: all slots are idle
0.50.007.739 I srv operator(): Chat format: peg-native
0.50.011.650 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.50.011.653 I srv get_availabl: updating prompt cache
0.50.011.660 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.50.011.665 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 24576 tokens, 8589934592 est)
0.50.011.665 I srv get_availabl: prompt cache update took 0.01 ms
0.50.012.665 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.50.012.668 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.50.012.669 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.50.012.687 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.50.012.687 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.50.012.700 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.50.012.701 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.51.295.142 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 103, n_tokens = 104, size = 3.658 MiB)
0.54.540.175 I slot print_timing: id 3 | task 0 | n_decoded = 188, tg = 62.50 t/s, tg_3s = 62.50 t/s
0.55.212.173 I slot print_timing: id 3 | task 0 | prompt eval time = 1519.27 ms / 126 tokens ( 12.06 ms per token, 82.93 tokens per second)
0.55.212.176 I slot print_timing: id 3 | task 0 | eval time = 3680.18 ms / 230 tokens ( 16.00 ms per token, 62.50 tokens per second)
0.55.212.177 I slot print_timing: id 3 | task 0 | total time = 5199.44 ms / 356 tokens
0.55.212.181 I slot print_timing: id 3 | task 0 | graphs reused = 228
0.55.212.233 I slot release: id 3 | task 0 | stop processing: n_tokens = 355, truncated = 0
0.55.212.238 I srv update_slots: all slots are idle
0.55.221.161 I srv operator(): Chat format: peg-native
0.55.224.558 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.829 (> 0.100 thold), f_keep = 0.301
0.55.224.560 I srv get_availabl: updating prompt cache
0.55.224.618 W srv prompt_save: - saving prompt with length 355, total state size = 16.987 MiB (draft: 0.000 MiB)
0.55.233.799 I srv load: - looking for better prompt, base f_keep = 0.301, sim = 0.829
0.55.233.805 I srv update: - cache state: 1 prompts, 20.645 MiB (limits: 8192.000 MiB, 24576 tokens, 140866 est)
0.55.233.806 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
0.55.233.807 I srv get_availabl: prompt cache update took 9.25 ms
0.55.234.674 I slot launch_slot_: id 3 | task 233 | processing task, is_child = 0
0.55.234.676 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.55.234.677 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.55.234.694 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.55.234.694 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.55.234.706 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.55.234.706 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.55.234.724 I slot update_slots: id 3 | task 233 | Checking checkpoint with [0, 103] against 0...
0.55.236.880 W slot update_slots: id 3 | task 233 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 3.658 MiB)
0.56.224.067 I slot print_timing: id 3 | task 233 | prompt eval time = 264.15 ms / 26 tokens ( 10.16 ms per token, 98.43 tokens per second)
0.56.224.070 I slot print_timing: id 3 | task 233 | eval time = 725.18 ms / 46 tokens ( 15.76 ms per token, 63.43 tokens per second)
0.56.224.070 I slot print_timing: id 3 | task 233 | total time = 989.34 ms / 72 tokens
0.56.224.071 I slot print_timing: id 3 | task 233 | graphs reused = 272
0.56.224.100 I slot release: id 3 | task 233 | stop processing: n_tokens = 174, truncated = 0
0.56.224.108 I srv update_slots: all slots are idle
0.56.232.363 I srv operator(): Chat format: peg-native
0.56.236.059 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.718 (> 0.100 thold), f_keep = 0.615
0.56.236.898 I slot launch_slot_: id 3 | task 282 | processing task, is_child = 0
0.56.236.900 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.56.236.900 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.56.236.915 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.56.236.915 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.56.236.927 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.56.236.927 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.56.236.942 I slot update_slots: id 3 | task 282 | Checking checkpoint with [0, 103] against 0...
0.56.239.462 W slot update_slots: id 3 | task 282 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 3.658 MiB)
0.59.718.268 I slot print_timing: id 3 | task 282 | n_decoded = 188, tg = 62.64 t/s, tg_3s = 62.64 t/s
1.02.733.418 I slot print_timing: id 3 | task 282 | n_decoded = 376, tg = 62.49 t/s, tg_3s = 62.35 t/s
1.03.024.977 I slot print_timing: id 3 | task 282 | prompt eval time = 479.95 ms / 46 tokens ( 10.43 ms per token, 95.84 tokens per second)
1.03.024.981 I slot print_timing: id 3 | task 282 | eval time = 6308.08 ms / 394 tokens ( 16.01 ms per token, 62.46 tokens per second)
1.03.024.981 I slot print_timing: id 3 | task 282 | total time = 6788.03 ms / 440 tokens
1.03.024.982 I slot print_timing: id 3 | task 282 | graphs reused = 662
1.03.025.015 I slot release: id 3 | task 282 | stop processing: n_tokens = 542, truncated = 0
1.03.025.024 I srv update_slots: all slots are idle
1.03.035.111 I srv operator(): Chat format: peg-native
1.03.039.340 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.823 (> 0.100 thold), f_keep = 0.197
1.03.039.342 I srv get_availabl: updating prompt cache
1.03.039.399 W srv prompt_save: - saving prompt with length 542, total state size = 23.563 MiB (draft: 0.000 MiB)
1.03.061.504 I srv load: - looking for better prompt, base f_keep = 0.197, sim = 0.823
1.03.061.513 I srv update: - cache state: 2 prompts, 47.866 MiB (limits: 8192.000 MiB, 24576 tokens, 153517 est)
1.03.061.514 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
1.03.061.515 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
1.03.061.516 I srv get_availabl: prompt cache update took 22.17 ms
1.03.062.556 I slot launch_slot_: id 3 | task 679 | processing task, is_child = 0
1.03.062.559 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.03.062.559 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.03.062.591 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.03.062.591 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.03.062.610 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.03.062.610 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.03.062.633 I slot update_slots: id 3 | task 679 | Checking checkpoint with [0, 103] against 0...
1.03.064.943 W slot update_slots: id 3 | task 679 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 3.658 MiB)
1.06.343.124 I slot print_timing: id 3 | task 679 | n_decoded = 188, tg = 62.59 t/s, tg_3s = 62.59 t/s
1.09.351.340 I slot print_timing: id 3 | task 679 | n_decoded = 376, tg = 62.54 t/s, tg_3s = 62.50 t/s
1.12.355.196 I slot print_timing: id 3 | task 679 | n_decoded = 562, tg = 62.33 t/s, tg_3s = 61.92 t/s
1.13.139.051 I slot print_timing: id 3 | task 679 | prompt eval time = 276.62 ms / 27 tokens ( 10.25 ms per token, 97.61 tokens per second)
1.13.139.054 I slot print_timing: id 3 | task 679 | eval time = 9799.79 ms / 611 tokens ( 16.04 ms per token, 62.35 tokens per second)
1.13.139.054 I slot print_timing: id 3 | task 679 | total time = 10076.41 ms / 638 tokens
1.13.139.055 I slot print_timing: id 3 | task 679 | graphs reused = 1269
1.13.139.086 I slot release: id 3 | task 679 | stop processing: n_tokens = 740, truncated = 0
1.13.139.096 I srv update_slots: all slots are idle
1.13.147.523 I srv operator(): Chat format: peg-native
1.13.151.272 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.849 (> 0.100 thold), f_keep = 0.145
1.13.151.274 I srv get_availabl: updating prompt cache
1.13.151.342 W srv prompt_save: - saving prompt with length 740, total state size = 30.526 MiB (draft: 0.000 MiB)
1.13.176.322 I srv load: - looking for better prompt, base f_keep = 0.145, sim = 0.849
1.13.176.330 I srv update: - cache state: 3 prompts, 82.050 MiB (limits: 8192.000 MiB, 24576 tokens, 163440 est)
1.13.176.331 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
1.13.176.331 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
1.13.176.332 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
1.13.176.333 I srv get_availabl: prompt cache update took 25.06 ms
1.13.177.206 I slot launch_slot_: id 3 | task 1293 | processing task, is_child = 0
1.13.177.208 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.13.177.209 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.13.177.228 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.13.177.228 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.13.177.241 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.13.177.241 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.13.177.260 I slot update_slots: id 3 | task 1293 | Checking checkpoint with [0, 103] against 0...
1.13.179.638 W slot update_slots: id 3 | task 1293 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 3.658 MiB)
1.15.638.962 I slot print_timing: id 3 | task 1293 | prompt eval time = 253.22 ms / 23 tokens ( 11.01 ms per token, 90.83 tokens per second)
1.15.638.966 I slot print_timing: id 3 | task 1293 | eval time = 2208.48 ms / 141 tokens ( 15.66 ms per token, 63.84 tokens per second)
1.15.638.966 I slot print_timing: id 3 | task 1293 | total time = 2461.70 ms / 164 tokens
1.15.638.967 I slot print_timing: id 3 | task 1293 | graphs reused = 1407
1.15.638.997 I slot release: id 3 | task 1293 | stop processing: n_tokens = 266, truncated = 0
1.15.639.006 I srv update_slots: all slots are idle
1.15.647.701 I srv operator(): Chat format: peg-native
1.15.651.184 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.738 (> 0.100 thold), f_keep = 0.402
1.15.651.187 I srv get_availabl: updating prompt cache
1.15.651.230 W srv prompt_save: - saving prompt with length 266, total state size = 13.857 MiB (draft: 0.000 MiB)
1.15.664.711 I srv load: - looking for better prompt, base f_keep = 0.402, sim = 0.738
1.15.664.718 I srv update: - cache state: 4 prompts, 99.565 MiB (limits: 8192.000 MiB, 24576 tokens, 156574 est)
1.15.664.719 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
1.15.664.719 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
1.15.664.720 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
1.15.664.720 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
1.15.664.721 I srv get_availabl: prompt cache update took 13.53 ms
1.15.665.575 I slot launch_slot_: id 3 | task 1437 | processing task, is_child = 0
1.15.665.577 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.15.665.578 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.15.665.599 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.15.665.600 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.15.665.612 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.15.665.612 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.15.665.631 I slot update_slots: id 3 | task 1437 | Checking checkpoint with [0, 103] against 0...
1.15.667.954 W slot update_slots: id 3 | task 1437 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 3.658 MiB)
1.18.016.931 I slot print_timing: id 3 | task 1437 | prompt eval time = 412.29 ms / 42 tokens ( 9.82 ms per token, 101.87 tokens per second)
1.18.016.934 I slot print_timing: id 3 | task 1437 | eval time = 1939.00 ms / 123 tokens ( 15.76 ms per token, 63.43 tokens per second)
1.18.016.935 I slot print_timing: id 3 | task 1437 | total time = 2351.30 ms / 165 tokens
1.18.016.935 I slot print_timing: id 3 | task 1437 | graphs reused = 1527
1.18.016.967 I slot release: id 3 | task 1437 | stop processing: n_tokens = 267, truncated = 0
1.18.016.975 I srv update_slots: all slots are idle
1.18.025.535 I srv operator(): Chat format: peg-native
1.18.029.774 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.748 (> 0.100 thold), f_keep = 0.401
1.18.029.776 I srv get_availabl: updating prompt cache
1.18.029.820 W srv prompt_save: - saving prompt with length 267, total state size = 13.892 MiB (draft: 0.000 MiB)
1.18.043.577 I srv load: - looking for better prompt, base f_keep = 0.401, sim = 0.748
1.18.043.592 I srv update: - cache state: 5 prompts, 117.115 MiB (limits: 8192.000 MiB, 24576 tokens, 151788 est)
1.18.043.592 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
1.18.043.593 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
1.18.043.593 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
1.18.043.594 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
1.18.043.594 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
1.18.043.595 I srv get_availabl: prompt cache update took 13.82 ms
1.18.044.437 I slot launch_slot_: id 3 | task 1563 | processing task, is_child = 0
1.18.044.439 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.18.044.440 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.18.044.457 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.18.044.457 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.18.044.471 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.18.044.471 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.18.044.494 I slot update_slots: id 3 | task 1563 | Checking checkpoint with [0, 103] against 0...
1.18.046.744 W slot update_slots: id 3 | task 1563 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 3.658 MiB)
1.20.488.995 I slot print_timing: id 3 | task 1563 | prompt eval time = 424.33 ms / 40 tokens ( 10.61 ms per token, 94.27 tokens per second)
1.20.488.998 I slot print_timing: id 3 | task 1563 | eval time = 2020.17 ms / 128 tokens ( 15.78 ms per token, 63.36 tokens per second)
1.20.488.999 I slot print_timing: id 3 | task 1563 | total time = 2444.50 ms / 168 tokens
1.20.489.000 I slot print_timing: id 3 | task 1563 | graphs reused = 1652
1.20.489.032 I slot release: id 3 | task 1563 | stop processing: n_tokens = 270, truncated = 0
1.20.489.039 I srv update_slots: all slots are idle
1.20.497.154 I srv operator(): Chat format: peg-native
1.20.500.905 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.829 (> 0.100 thold), f_keep = 0.396
1.20.500.908 I srv get_availabl: updating prompt cache
1.20.500.955 W srv prompt_save: - saving prompt with length 270, total state size = 13.998 MiB (draft: 0.000 MiB)
1.20.514.718 I srv load: - looking for better prompt, base f_keep = 0.396, sim = 0.829
1.20.514.726 I srv update: - cache state: 6 prompts, 134.770 MiB (limits: 8192.000 MiB, 24576 tokens, 148315 est)
1.20.514.726 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
1.20.514.727 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
1.20.514.728 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
1.20.514.728 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
1.20.514.728 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
1.20.514.729 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
1.20.514.730 I srv get_availabl: prompt cache update took 13.82 ms
1.20.515.592 I slot launch_slot_: id 3 | task 1694 | processing task, is_child = 0
1.20.515.594 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.20.515.594 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.20.515.610 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.20.515.610 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.20.515.622 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.20.515.622 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.20.515.642 I slot update_slots: id 3 | task 1694 | Checking checkpoint with [0, 103] against 0...
1.20.518.038 W slot update_slots: id 3 | task 1694 | restored context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_past = 103, size = 3.658 MiB)
1.23.772.992 I slot print_timing: id 3 | task 1694 | n_decoded = 188, tg = 62.66 t/s, tg_3s = 62.66 t/s
1.23.964.962 I slot print_timing: id 3 | task 1694 | prompt eval time = 256.86 ms / 26 tokens ( 9.88 ms per token, 101.22 tokens per second)
1.23.964.965 I slot print_timing: id 3 | task 1694 | eval time = 3192.45 ms / 200 tokens ( 15.96 ms per token, 62.65 tokens per second)
1.23.964.966 I slot print_timing: id 3 | task 1694 | total time = 3449.32 ms / 226 tokens
1.23.964.966 I slot print_timing: id 3 | task 1694 | graphs reused = 1849
1.23.964.998 I slot release: id 3 | task 1694 | stop processing: n_tokens = 328, truncated = 0
1.23.965.005 I srv update_slots: all slots are idle
1.23.974.012 I srv operator(): Chat format: peg-native
1.23.977.865 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.446 (> 0.100 thold), f_keep = 0.213
1.23.977.867 I srv get_availabl: updating prompt cache
1.23.977.903 W srv prompt_save: - saving prompt with length 328, total state size = 16.037 MiB (draft: 0.000 MiB)
1.23.992.353 I srv load: - looking for better prompt, base f_keep = 0.213, sim = 0.446
1.23.992.362 I srv update: - cache state: 7 prompts, 154.466 MiB (limits: 8192.000 MiB, 24576 tokens, 146799 est)
1.23.992.363 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
1.23.992.364 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
1.23.992.364 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
1.23.992.365 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
1.23.992.365 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
1.23.992.366 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
1.23.992.366 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
1.23.992.367 I srv get_availabl: prompt cache update took 14.50 ms
1.23.993.236 I slot launch_slot_: id 3 | task 1897 | processing task, is_child = 0
1.23.993.238 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.23.993.239 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.23.993.259 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.23.993.259 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.23.993.271 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.23.993.271 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.23.993.290 I slot update_slots: id 3 | task 1897 | Checking checkpoint with [0, 103] against 0...
1.23.993.291 W slot update_slots: id 3 | task 1897 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.23.993.292 W slot update_slots: id 3 | task 1897 | erased invalidated context checkpoint (pos_min = 0, pos_max = 103, n_tokens = 104, n_swa = 128, pos_next = 0, size = 3.658 MiB)
1.24.831.128 I slot create_check: id 3 | task 1897 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 121, n_tokens = 122, size = 4.291 MiB)
1.28.181.078 I slot print_timing: id 3 | task 1897 | n_decoded = 188, tg = 62.63 t/s, tg_3s = 62.63 t/s
1.31.185.075 I slot print_timing: id 3 | task 1897 | n_decoded = 375, tg = 62.44 t/s, tg_3s = 62.25 t/s
1.34.200.241 I slot print_timing: id 3 | task 1897 | n_decoded = 563, tg = 62.41 t/s, tg_3s = 62.35 t/s
1.37.203.424 I slot print_timing: id 3 | task 1897 | n_decoded = 750, tg = 62.37 t/s, tg_3s = 62.27 t/s
1.40.207.379 I slot print_timing: id 3 | task 1897 | n_decoded = 936, tg = 62.28 t/s, tg_3s = 61.92 t/s
1.43.210.808 I slot print_timing: id 3 | task 1897 | n_decoded = 1120, tg = 62.11 t/s, tg_3s = 61.26 t/s
1.44.043.029 I slot print_timing: id 3 | task 1897 | prompt eval time = 1185.94 ms / 157 tokens ( 7.55 ms per token, 132.38 tokens per second)
1.44.043.032 I slot print_timing: id 3 | task 1897 | eval time = 18863.79 ms / 1170 tokens ( 16.12 ms per token, 62.02 tokens per second)
1.44.043.033 I slot print_timing: id 3 | task 1897 | total time = 20049.73 ms / 1327 tokens
1.44.043.034 I slot print_timing: id 3 | task 1897 | graphs reused = 3012
1.44.043.068 I slot release: id 3 | task 1897 | stop processing: n_tokens = 1326, truncated = 0
1.44.043.081 I srv update_slots: all slots are idle
1.44.051.812 I srv operator(): Chat format: peg-native
1.44.055.672 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.691 (> 0.100 thold), f_keep = 0.094
1.44.055.674 I srv get_availabl: updating prompt cache
1.44.055.756 W srv prompt_save: - saving prompt with length 1326, total state size = 51.135 MiB (draft: 0.000 MiB)
1.44.116.909 I srv load: - looking for better prompt, base f_keep = 0.094, sim = 0.691
1.44.116.918 I srv update: - cache state: 8 prompts, 209.891 MiB (limits: 8192.000 MiB, 24576 tokens, 159787 est)
1.44.116.919 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
1.44.116.919 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
1.44.116.920 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
1.44.116.921 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
1.44.116.921 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
1.44.116.922 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
1.44.116.922 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
1.44.116.923 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
1.44.116.923 I srv get_availabl: prompt cache update took 61.25 ms
1.44.117.867 I slot launch_slot_: id 3 | task 3070 | processing task, is_child = 0
1.44.117.869 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.44.117.870 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.44.117.887 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.44.117.887 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.44.117.901 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.44.117.901 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.44.117.921 I slot update_slots: id 3 | task 3070 | Checking checkpoint with [0, 121] against 0...
1.44.120.496 W slot update_slots: id 3 | task 3070 | restored context checkpoint (pos_min = 0, pos_max = 121, n_tokens = 122, n_past = 121, size = 4.291 MiB)
1.47.739.329 I slot print_timing: id 3 | task 3070 | n_decoded = 188, tg = 62.58 t/s, tg_3s = 62.58 t/s
1.50.748.509 I slot print_timing: id 3 | task 3070 | n_decoded = 375, tg = 62.36 t/s, tg_3s = 62.14 t/s
1.52.596.415 I slot print_timing: id 3 | task 3070 | prompt eval time = 617.35 ms / 60 tokens ( 10.29 ms per token, 97.19 tokens per second)
1.52.596.419 I slot print_timing: id 3 | task 3070 | eval time = 7861.14 ms / 490 tokens ( 16.04 ms per token, 62.33 tokens per second)
1.52.596.420 I slot print_timing: id 3 | task 3070 | total time = 8478.49 ms / 550 tokens
1.52.596.420 I slot print_timing: id 3 | task 3070 | graphs reused = 3498
1.52.596.457 I slot release: id 3 | task 3070 | stop processing: n_tokens = 670, truncated = 0
1.52.596.468 I srv update_slots: all slots are idle
1.52.604.840 I srv operator(): Chat format: peg-native
1.52.608.316 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.706 (> 0.100 thold), f_keep = 0.187
1.52.608.318 I srv get_availabl: updating prompt cache
1.52.608.367 W srv prompt_save: - saving prompt with length 670, total state size = 28.065 MiB (draft: 0.000 MiB)
1.52.631.733 I srv load: - looking for better prompt, base f_keep = 0.187, sim = 0.706
1.52.631.744 I srv update: - cache state: 9 prompts, 242.247 MiB (limits: 8192.000 MiB, 24576 tokens, 161103 est)
1.52.631.745 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
1.52.631.746 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
1.52.631.746 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
1.52.631.747 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
1.52.631.747 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
1.52.631.748 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
1.52.631.748 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
1.52.631.749 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
1.52.631.749 I srv update: - prompt 0x584e45f74060: 670 tokens, checkpoints: 1, 32.356 MiB
1.52.631.750 I srv get_availabl: prompt cache update took 23.43 ms
1.52.632.610 I slot launch_slot_: id 3 | task 3563 | processing task, is_child = 0
1.52.632.612 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.52.632.613 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.52.632.632 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.52.632.632 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.52.632.644 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.52.632.645 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.52.632.665 I slot update_slots: id 3 | task 3563 | Checking checkpoint with [0, 121] against 0...
1.52.635.174 W slot update_slots: id 3 | task 3563 | restored context checkpoint (pos_min = 0, pos_max = 121, n_tokens = 122, n_past = 121, size = 4.291 MiB)
1.56.234.290 I slot print_timing: id 3 | task 3563 | n_decoded = 188, tg = 62.51 t/s, tg_3s = 62.51 t/s
1.59.240.414 I slot print_timing: id 3 | task 3563 | n_decoded = 375, tg = 62.36 t/s, tg_3s = 62.21 t/s
2.02.247.848 I slot print_timing: id 3 | task 3563 | n_decoded = 562, tg = 62.30 t/s, tg_3s = 62.18 t/s
2.05.249.926 I slot print_timing: id 3 | task 3563 | n_decoded = 745, tg = 61.96 t/s, tg_3s = 60.96 t/s
2.08.257.430 I slot print_timing: id 3 | task 3563 | n_decoded = 930, tg = 61.87 t/s, tg_3s = 61.51 t/s
2.11.271.018 I slot print_timing: id 3 | task 3563 | n_decoded = 1116, tg = 61.85 t/s, tg_3s = 61.72 t/s
2.14.272.968 I slot print_timing: id 3 | task 3563 | n_decoded = 1301, tg = 61.82 t/s, tg_3s = 61.63 t/s
2.17.277.650 I slot print_timing: id 3 | task 3563 | n_decoded = 1485, tg = 61.74 t/s, tg_3s = 61.24 t/s
2.20.282.743 I slot print_timing: id 3 | task 3563 | n_decoded = 1668, tg = 61.65 t/s, tg_3s = 60.90 t/s
2.23.283.714 I slot print_timing: id 3 | task 3563 | n_decoded = 1849, tg = 61.52 t/s, tg_3s = 60.31 t/s
2.26.299.754 I slot print_timing: id 3 | task 3563 | n_decoded = 2031, tg = 61.41 t/s, tg_3s = 60.34 t/s
2.29.060.315 I slot print_timing: id 3 | task 3563 | prompt eval time = 594.00 ms / 56 tokens ( 10.61 ms per token, 94.28 tokens per second)
2.29.060.318 I slot print_timing: id 3 | task 3563 | eval time = 35833.65 ms / 2197 tokens ( 16.31 ms per token, 61.31 tokens per second)
2.29.060.319 I slot print_timing: id 3 | task 3563 | total time = 36427.65 ms / 2253 tokens
2.29.060.320 I slot print_timing: id 3 | task 3563 | graphs reused = 5684
2.29.060.358 I slot release: id 3 | task 3563 | stop processing: n_tokens = 2373, truncated = 0
2.29.060.373 I srv update_slots: all slots are idle
2.29.069.182 I srv operator(): Chat format: peg-native
2.29.073.103 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.668 (> 0.100 thold), f_keep = 0.053
2.29.073.105 I srv get_availabl: updating prompt cache
2.29.073.223 W srv prompt_save: - saving prompt with length 2373, total state size = 87.955 MiB (draft: 0.000 MiB)
2.29.192.153 I srv load: - looking for better prompt, base f_keep = 0.053, sim = 0.668
2.29.192.163 I srv update: - cache state: 10 prompts, 334.493 MiB (limits: 8192.000 MiB, 24576 tokens, 174790 est)
2.29.192.164 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
2.29.192.165 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
2.29.192.165 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
2.29.192.166 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
2.29.192.166 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
2.29.192.167 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
2.29.192.167 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
2.29.192.168 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
2.29.192.168 I srv update: - prompt 0x584e45f74060: 670 tokens, checkpoints: 1, 32.356 MiB
2.29.192.169 I srv update: - prompt 0x584e460738d0: 2373 tokens, checkpoints: 1, 92.246 MiB
2.29.192.170 I srv get_availabl: prompt cache update took 119.06 ms
2.29.193.032 I slot launch_slot_: id 3 | task 5763 | processing task, is_child = 0
2.29.193.034 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.29.193.035 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.29.193.056 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.29.193.056 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.29.193.069 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.29.193.069 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.29.193.090 I slot update_slots: id 3 | task 5763 | Checking checkpoint with [0, 121] against 0...
2.29.195.699 W slot update_slots: id 3 | task 5763 | restored context checkpoint (pos_min = 0, pos_max = 121, n_tokens = 122, n_past = 121, size = 4.291 MiB)
2.32.857.330 I slot print_timing: id 3 | task 5763 | n_decoded = 188, tg = 62.60 t/s, tg_3s = 62.60 t/s
2.35.859.944 I slot print_timing: id 3 | task 5763 | n_decoded = 375, tg = 62.44 t/s, tg_3s = 62.28 t/s
2.38.864.468 I slot print_timing: id 3 | task 5763 | n_decoded = 561, tg = 62.26 t/s, tg_3s = 61.91 t/s
2.41.864.510 I slot print_timing: id 3 | task 5763 | n_decoded = 744, tg = 61.95 t/s, tg_3s = 61.00 t/s
2.44.868.125 I slot print_timing: id 3 | task 5763 | n_decoded = 926, tg = 61.68 t/s, tg_3s = 60.59 t/s
2.45.048.560 I slot print_timing: id 3 | task 5763 | prompt eval time = 660.97 ms / 66 tokens ( 10.01 ms per token, 99.85 tokens per second)
2.45.048.562 I slot print_timing: id 3 | task 5763 | eval time = 15194.50 ms / 937 tokens ( 16.22 ms per token, 61.67 tokens per second)
2.45.048.563 I slot print_timing: id 3 | task 5763 | total time = 15855.47 ms / 1003 tokens
2.45.048.564 I slot print_timing: id 3 | task 5763 | graphs reused = 6615
2.45.048.607 I slot release: id 3 | task 5763 | stop processing: n_tokens = 1123, truncated = 0
2.45.048.618 I srv update_slots: all slots are idle
2.45.057.502 I srv operator(): Chat format: peg-native
2.45.061.419 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.111
2.45.061.421 I srv get_availabl: updating prompt cache
2.45.061.487 W srv prompt_save: - saving prompt with length 1123, total state size = 43.996 MiB (draft: 0.000 MiB)
2.45.123.053 I srv load: - looking for better prompt, base f_keep = 0.111, sim = 0.776
2.45.123.064 I srv update: - cache state: 11 prompts, 382.779 MiB (limits: 8192.000 MiB, 24576 tokens, 176775 est)
2.45.123.065 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
2.45.123.065 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
2.45.123.066 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
2.45.123.066 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
2.45.123.067 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
2.45.123.067 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
2.45.123.068 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
2.45.123.068 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
2.45.123.069 I srv update: - prompt 0x584e45f74060: 670 tokens, checkpoints: 1, 32.356 MiB
2.45.123.069 I srv update: - prompt 0x584e460738d0: 2373 tokens, checkpoints: 1, 92.246 MiB
2.45.123.070 I srv update: - prompt 0x584e462c17b0: 1123 tokens, checkpoints: 1, 48.287 MiB
2.45.123.071 I srv get_availabl: prompt cache update took 61.65 ms
2.45.123.913 I slot launch_slot_: id 3 | task 6703 | processing task, is_child = 0
2.45.123.915 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.45.123.915 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.45.123.931 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.45.123.931 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.45.123.943 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.45.123.944 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.45.123.962 I slot update_slots: id 3 | task 6703 | Checking checkpoint with [0, 121] against 0...
2.45.126.591 W slot update_slots: id 3 | task 6703 | restored context checkpoint (pos_min = 0, pos_max = 121, n_tokens = 122, n_past = 121, size = 4.291 MiB)
2.48.539.016 I slot print_timing: id 3 | task 6703 | n_decoded = 188, tg = 62.60 t/s, tg_3s = 62.60 t/s
2.51.541.052 I slot print_timing: id 3 | task 6703 | n_decoded = 375, tg = 62.45 t/s, tg_3s = 62.29 t/s
2.54.550.018 I slot print_timing: id 3 | task 6703 | n_decoded = 563, tg = 62.46 t/s, tg_3s = 62.48 t/s
2.57.550.317 I slot print_timing: id 3 | task 6703 | n_decoded = 750, tg = 62.42 t/s, tg_3s = 62.33 t/s
2.58.670.251 I slot print_timing: id 3 | task 6703 | prompt eval time = 411.91 ms / 40 tokens ( 10.30 ms per token, 97.11 tokens per second)
2.58.670.254 I slot print_timing: id 3 | task 6703 | eval time = 13134.37 ms / 820 tokens ( 16.02 ms per token, 62.43 tokens per second)
2.58.670.255 I slot print_timing: id 3 | task 6703 | total time = 13546.28 ms / 860 tokens
2.58.670.255 I slot print_timing: id 3 | task 6703 | graphs reused = 7430
2.58.670.295 I slot release: id 3 | task 6703 | stop processing: n_tokens = 980, truncated = 0
2.58.670.306 I srv update_slots: all slots are idle
2.58.678.425 I srv operator(): Chat format: peg-native
2.58.681.945 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.706 (> 0.100 thold), f_keep = 0.128
2.58.681.947 I srv get_availabl: updating prompt cache
2.58.682.018 W srv prompt_save: - saving prompt with length 980, total state size = 38.967 MiB (draft: 0.000 MiB)
2.58.715.494 I srv load: - looking for better prompt, base f_keep = 0.128, sim = 0.706
2.58.715.506 I srv update: - cache state: 12 prompts, 426.037 MiB (limits: 8192.000 MiB, 24576 tokens, 177670 est)
2.58.715.507 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
2.58.715.508 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
2.58.715.508 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
2.58.715.509 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
2.58.715.509 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
2.58.715.510 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
2.58.715.511 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
2.58.715.511 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
2.58.715.512 I srv update: - prompt 0x584e45f74060: 670 tokens, checkpoints: 1, 32.356 MiB
2.58.715.512 I srv update: - prompt 0x584e460738d0: 2373 tokens, checkpoints: 1, 92.246 MiB
2.58.715.513 I srv update: - prompt 0x584e462c17b0: 1123 tokens, checkpoints: 1, 48.287 MiB
2.58.715.513 I srv update: - prompt 0x584e46255e00: 980 tokens, checkpoints: 1, 43.258 MiB
2.58.715.514 I srv get_availabl: prompt cache update took 33.57 ms
2.58.716.562 I slot launch_slot_: id 3 | task 7526 | processing task, is_child = 0
2.58.716.564 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.58.716.565 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.58.716.595 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.58.716.596 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.58.716.611 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.58.716.612 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.58.716.636 I slot update_slots: id 3 | task 7526 | Checking checkpoint with [0, 121] against 0...
2.58.719.319 W slot update_slots: id 3 | task 7526 | restored context checkpoint (pos_min = 0, pos_max = 121, n_tokens = 122, n_past = 121, size = 4.291 MiB)
3.02.329.668 I slot print_timing: id 3 | task 7526 | n_decoded = 188, tg = 62.53 t/s, tg_3s = 62.53 t/s
3.05.344.207 I slot print_timing: id 3 | task 7526 | n_decoded = 376, tg = 62.45 t/s, tg_3s = 62.36 t/s
3.08.352.185 I slot print_timing: id 3 | task 7526 | n_decoded = 564, tg = 62.47 t/s, tg_3s = 62.50 t/s
3.11.362.036 I slot print_timing: id 3 | task 7526 | n_decoded = 750, tg = 62.30 t/s, tg_3s = 61.80 t/s
3.14.368.061 I slot print_timing: id 3 | task 7526 | n_decoded = 934, tg = 62.08 t/s, tg_3s = 61.21 t/s
3.15.334.452 I slot print_timing: id 3 | task 7526 | prompt eval time = 606.49 ms / 56 tokens ( 10.83 ms per token, 92.33 tokens per second)
3.15.334.456 I slot print_timing: id 3 | task 7526 | eval time = 16011.32 ms / 993 tokens ( 16.12 ms per token, 62.02 tokens per second)
3.15.334.456 I slot print_timing: id 3 | task 7526 | total time = 16617.81 ms / 1049 tokens
3.15.334.457 I slot print_timing: id 3 | task 7526 | graphs reused = 8417
3.15.334.495 I slot release: id 3 | task 7526 | stop processing: n_tokens = 1169, truncated = 0
3.15.334.506 I srv update_slots: all slots are idle
3.15.343.202 I srv operator(): Chat format: peg-native
3.15.347.050 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.374 (> 0.100 thold), f_keep = 0.060
3.15.347.052 I srv get_availabl: updating prompt cache
3.15.347.125 W srv prompt_save: - saving prompt with length 1169, total state size = 45.613 MiB (draft: 0.000 MiB)
3.15.387.227 I srv load: - looking for better prompt, base f_keep = 0.060, sim = 0.374
3.15.387.241 I srv update: - cache state: 13 prompts, 475.941 MiB (limits: 8192.000 MiB, 24576 tokens, 179161 est)
3.15.387.242 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
3.15.387.243 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
3.15.387.244 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
3.15.387.245 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
3.15.387.245 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
3.15.387.246 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
3.15.387.247 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
3.15.387.247 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
3.15.387.248 I srv update: - prompt 0x584e45f74060: 670 tokens, checkpoints: 1, 32.356 MiB
3.15.387.249 I srv update: - prompt 0x584e460738d0: 2373 tokens, checkpoints: 1, 92.246 MiB
3.15.387.249 I srv update: - prompt 0x584e462c17b0: 1123 tokens, checkpoints: 1, 48.287 MiB
3.15.387.250 I srv update: - prompt 0x584e46255e00: 980 tokens, checkpoints: 1, 43.258 MiB
3.15.387.251 I srv update: - prompt 0x584e4602efe0: 1169 tokens, checkpoints: 1, 49.904 MiB
3.15.387.253 I srv get_availabl: prompt cache update took 40.20 ms
3.15.388.339 I slot launch_slot_: id 3 | task 8522 | processing task, is_child = 0
3.15.388.342 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.15.388.342 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.15.388.361 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.15.388.361 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.15.388.377 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.15.388.377 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.15.388.400 I slot update_slots: id 3 | task 8522 | Checking checkpoint with [0, 121] against 0...
3.15.388.400 W slot update_slots: id 3 | task 8522 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
3.15.388.402 W slot update_slots: id 3 | task 8522 | erased invalidated context checkpoint (pos_min = 0, pos_max = 121, n_tokens = 122, n_swa = 128, pos_next = 0, size = 4.291 MiB)
3.16.222.561 I slot create_check: id 3 | task 8522 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 116, n_tokens = 117, size = 4.115 MiB)
3.19.771.147 I slot print_timing: id 3 | task 8522 | n_decoded = 188, tg = 62.54 t/s, tg_3s = 62.54 t/s
3.20.139.112 I slot print_timing: id 3 | task 8522 | prompt eval time = 1376.90 ms / 187 tokens ( 7.36 ms per token, 135.81 tokens per second)
3.20.139.115 I slot print_timing: id 3 | task 8522 | eval time = 3373.81 ms / 211 tokens ( 15.99 ms per token, 62.54 tokens per second)
3.20.139.116 I slot print_timing: id 3 | task 8522 | total time = 4750.71 ms / 398 tokens
3.20.139.117 I slot print_timing: id 3 | task 8522 | graphs reused = 8625
3.20.139.152 I slot release: id 3 | task 8522 | stop processing: n_tokens = 397, truncated = 0
3.20.139.160 I srv update_slots: all slots are idle
3.20.204.365 I srv operator(): Chat format: peg-native
3.20.208.196 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.315
3.20.208.199 I srv get_availabl: updating prompt cache
3.20.208.258 W srv prompt_save: - saving prompt with length 397, total state size = 18.464 MiB (draft: 0.000 MiB)
3.20.225.123 I srv load: - looking for better prompt, base f_keep = 0.315, sim = 0.833
3.20.225.136 I srv update: - cache state: 14 prompts, 498.520 MiB (limits: 8192.000 MiB, 24576 tokens, 177571 est)
3.20.225.137 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
3.20.225.138 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
3.20.225.138 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
3.20.225.138 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
3.20.225.139 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
3.20.225.139 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
3.20.225.140 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
3.20.225.140 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
3.20.225.141 I srv update: - prompt 0x584e45f74060: 670 tokens, checkpoints: 1, 32.356 MiB
3.20.225.141 I srv update: - prompt 0x584e460738d0: 2373 tokens, checkpoints: 1, 92.246 MiB
3.20.225.142 I srv update: - prompt 0x584e462c17b0: 1123 tokens, checkpoints: 1, 48.287 MiB
3.20.225.142 I srv update: - prompt 0x584e46255e00: 980 tokens, checkpoints: 1, 43.258 MiB
3.20.225.143 I srv update: - prompt 0x584e4602efe0: 1169 tokens, checkpoints: 1, 49.904 MiB
3.20.225.143 I srv update: - prompt 0x584e462be010: 397 tokens, checkpoints: 1, 22.579 MiB
3.20.225.145 I srv get_availabl: prompt cache update took 16.94 ms
3.20.225.998 I slot launch_slot_: id 3 | task 8736 | processing task, is_child = 0
3.20.226.000 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.20.226.001 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.20.226.018 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.20.226.018 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.20.226.031 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.20.226.031 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.20.226.051 I slot update_slots: id 3 | task 8736 | Checking checkpoint with [0, 116] against 0...
3.20.228.438 W slot update_slots: id 3 | task 8736 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 4.115 MiB)
3.22.995.096 I slot print_timing: id 3 | task 8736 | prompt eval time = 350.41 ms / 34 tokens ( 10.31 ms per token, 97.03 tokens per second)
3.22.995.099 I slot print_timing: id 3 | task 8736 | eval time = 2418.63 ms / 151 tokens ( 16.02 ms per token, 62.43 tokens per second)
3.22.995.100 I slot print_timing: id 3 | task 8736 | total time = 2769.04 ms / 185 tokens
3.22.995.100 I slot print_timing: id 3 | task 8736 | graphs reused = 8773
3.22.995.137 I slot release: id 3 | task 8736 | stop processing: n_tokens = 300, truncated = 0
3.22.995.146 I srv update_slots: all slots are idle
3.23.047.165 I srv operator(): Chat format: peg-native
3.23.051.051 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.796 (> 0.100 thold), f_keep = 0.417
3.23.051.053 I srv get_availabl: updating prompt cache
3.23.051.106 W srv prompt_save: - saving prompt with length 300, total state size = 15.053 MiB (draft: 0.000 MiB)
3.23.066.721 I srv load: - looking for better prompt, base f_keep = 0.417, sim = 0.796
3.23.066.734 I srv update: - cache state: 15 prompts, 517.688 MiB (limits: 8192.000 MiB, 24576 tokens, 175743 est)
3.23.066.736 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
3.23.066.737 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
3.23.066.737 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
3.23.066.738 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
3.23.066.739 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
3.23.066.739 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
3.23.066.742 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
3.23.066.743 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
3.23.066.743 I srv update: - prompt 0x584e45f74060: 670 tokens, checkpoints: 1, 32.356 MiB
3.23.066.744 I srv update: - prompt 0x584e460738d0: 2373 tokens, checkpoints: 1, 92.246 MiB
3.23.066.745 I srv update: - prompt 0x584e462c17b0: 1123 tokens, checkpoints: 1, 48.287 MiB
3.23.066.745 I srv update: - prompt 0x584e46255e00: 980 tokens, checkpoints: 1, 43.258 MiB
3.23.066.748 I srv update: - prompt 0x584e4602efe0: 1169 tokens, checkpoints: 1, 49.904 MiB
3.23.066.749 I srv update: - prompt 0x584e462be010: 397 tokens, checkpoints: 1, 22.579 MiB
3.23.066.749 I srv update: - prompt 0x584e4c7d7390: 300 tokens, checkpoints: 1, 19.168 MiB
3.23.066.750 I srv get_availabl: prompt cache update took 15.70 ms
3.23.067.814 I slot launch_slot_: id 3 | task 8890 | processing task, is_child = 0
3.23.067.816 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.23.067.817 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.23.067.835 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.23.067.836 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.23.067.851 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.23.067.851 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.23.067.872 I slot update_slots: id 3 | task 8890 | Checking checkpoint with [0, 116] against 0...
3.23.070.374 W slot update_slots: id 3 | task 8890 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 4.115 MiB)
3.26.334.149 I slot print_timing: id 3 | task 8890 | prompt eval time = 435.88 ms / 41 tokens ( 10.63 ms per token, 94.06 tokens per second)
3.26.334.153 I slot print_timing: id 3 | task 8890 | eval time = 2830.39 ms / 177 tokens ( 15.99 ms per token, 62.54 tokens per second)
3.26.334.153 I slot print_timing: id 3 | task 8890 | total time = 3266.27 ms / 218 tokens
3.26.334.154 I slot print_timing: id 3 | task 8890 | graphs reused = 8947
3.26.334.190 I slot release: id 3 | task 8890 | stop processing: n_tokens = 333, truncated = 0
3.26.334.198 I srv update_slots: all slots are idle
3.26.420.485 I srv operator(): Chat format: peg-native
3.26.424.341 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.672 (> 0.100 thold), f_keep = 0.375
3.26.424.343 I srv get_availabl: updating prompt cache
3.26.424.402 W srv prompt_save: - saving prompt with length 333, total state size = 16.213 MiB (draft: 0.000 MiB)
3.26.439.512 I srv load: - looking for better prompt, base f_keep = 0.375, sim = 0.672
3.26.439.525 I srv update: - cache state: 16 prompts, 538.016 MiB (limits: 8192.000 MiB, 24576 tokens, 174173 est)
3.26.439.526 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
3.26.439.527 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
3.26.439.527 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
3.26.439.532 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
3.26.439.532 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
3.26.439.533 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
3.26.439.533 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
3.26.439.534 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
3.26.439.534 I srv update: - prompt 0x584e45f74060: 670 tokens, checkpoints: 1, 32.356 MiB
3.26.439.535 I srv update: - prompt 0x584e460738d0: 2373 tokens, checkpoints: 1, 92.246 MiB
3.26.439.536 I srv update: - prompt 0x584e462c17b0: 1123 tokens, checkpoints: 1, 48.287 MiB
3.26.439.537 I srv update: - prompt 0x584e46255e00: 980 tokens, checkpoints: 1, 43.258 MiB
3.26.439.538 I srv update: - prompt 0x584e4602efe0: 1169 tokens, checkpoints: 1, 49.904 MiB
3.26.439.538 I srv update: - prompt 0x584e462be010: 397 tokens, checkpoints: 1, 22.579 MiB
3.26.439.538 I srv update: - prompt 0x584e4c7d7390: 300 tokens, checkpoints: 1, 19.168 MiB
3.26.439.539 I srv update: - prompt 0x584e45f6f690: 333 tokens, checkpoints: 1, 20.328 MiB
3.26.439.540 I srv get_availabl: prompt cache update took 15.20 ms
3.26.440.396 I slot launch_slot_: id 3 | task 9070 | processing task, is_child = 0
3.26.440.398 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.26.440.398 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.26.440.415 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.26.440.415 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.26.440.427 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.26.440.427 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.26.440.447 I slot update_slots: id 3 | task 9070 | Checking checkpoint with [0, 116] against 0...
3.26.442.754 W slot update_slots: id 3 | task 9070 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 4.115 MiB)
3.29.981.184 I slot print_timing: id 3 | task 9070 | n_decoded = 188, tg = 62.62 t/s, tg_3s = 62.62 t/s
3.32.982.755 I slot print_timing: id 3 | task 9070 | n_decoded = 374, tg = 62.29 t/s, tg_3s = 61.97 t/s
3.33.079.961 I slot print_timing: id 3 | task 9070 | prompt eval time = 538.47 ms / 70 tokens ( 7.69 ms per token, 130.00 tokens per second)
3.33.079.964 I slot print_timing: id 3 | task 9070 | eval time = 6101.04 ms / 380 tokens ( 16.06 ms per token, 62.28 tokens per second)
3.33.079.964 I slot print_timing: id 3 | task 9070 | total time = 6639.51 ms / 450 tokens
3.33.079.965 I slot print_timing: id 3 | task 9070 | graphs reused = 9323
3.33.080.004 I slot release: id 3 | task 9070 | stop processing: n_tokens = 565, truncated = 0
3.33.080.014 I srv update_slots: all slots are idle
3.33.159.979 I srv operator(): Chat format: peg-native
3.33.163.820 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.781 (> 0.100 thold), f_keep = 0.221
3.33.163.822 I srv get_availabl: updating prompt cache
3.33.163.892 W srv prompt_save: - saving prompt with length 565, total state size = 24.372 MiB (draft: 0.000 MiB)
3.33.185.654 I srv load: - looking for better prompt, base f_keep = 0.221, sim = 0.781
3.33.185.669 I srv update: - cache state: 17 prompts, 566.503 MiB (limits: 8192.000 MiB, 24576 tokens, 173585 est)
3.33.185.670 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
3.33.185.671 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
3.33.185.671 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
3.33.185.672 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
3.33.185.672 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
3.33.185.673 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
3.33.185.673 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
3.33.185.674 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
3.33.185.674 I srv update: - prompt 0x584e45f74060: 670 tokens, checkpoints: 1, 32.356 MiB
3.33.185.675 I srv update: - prompt 0x584e460738d0: 2373 tokens, checkpoints: 1, 92.246 MiB
3.33.185.675 I srv update: - prompt 0x584e462c17b0: 1123 tokens, checkpoints: 1, 48.287 MiB
3.33.185.676 I srv update: - prompt 0x584e46255e00: 980 tokens, checkpoints: 1, 43.258 MiB
3.33.185.676 I srv update: - prompt 0x584e4602efe0: 1169 tokens, checkpoints: 1, 49.904 MiB
3.33.185.677 I srv update: - prompt 0x584e462be010: 397 tokens, checkpoints: 1, 22.579 MiB
3.33.185.677 I srv update: - prompt 0x584e4c7d7390: 300 tokens, checkpoints: 1, 19.168 MiB
3.33.185.678 I srv update: - prompt 0x584e45f6f690: 333 tokens, checkpoints: 1, 20.328 MiB
3.33.185.678 I srv update: - prompt 0x584e4c8107b0: 565 tokens, checkpoints: 1, 28.487 MiB
3.33.185.679 I srv get_availabl: prompt cache update took 21.86 ms
3.33.186.615 I slot launch_slot_: id 3 | task 9453 | processing task, is_child = 0
3.33.186.617 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.33.186.618 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.33.186.638 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.33.186.638 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.33.186.651 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.33.186.652 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.33.186.673 I slot update_slots: id 3 | task 9453 | Checking checkpoint with [0, 116] against 0...
3.33.189.014 W slot update_slots: id 3 | task 9453 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 4.115 MiB)
3.36.666.248 I slot print_timing: id 3 | task 9453 | n_decoded = 188, tg = 62.41 t/s, tg_3s = 62.41 t/s
3.38.008.324 I slot print_timing: id 3 | task 9453 | prompt eval time = 467.22 ms / 44 tokens ( 10.62 ms per token, 94.17 tokens per second)
3.38.008.328 I slot print_timing: id 3 | task 9453 | eval time = 4354.43 ms / 271 tokens ( 16.07 ms per token, 62.24 tokens per second)
3.38.008.328 I slot print_timing: id 3 | task 9453 | total time = 4821.65 ms / 315 tokens
3.38.008.329 I slot print_timing: id 3 | task 9453 | graphs reused = 9591
3.38.008.366 I slot release: id 3 | task 9453 | stop processing: n_tokens = 430, truncated = 0
3.38.008.375 I srv update_slots: all slots are idle
3.38.062.293 I srv operator(): Chat format: peg-native
3.38.066.128 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.702 (> 0.100 thold), f_keep = 0.279
3.38.066.131 I srv get_availabl: updating prompt cache
3.38.066.181 W srv prompt_save: - saving prompt with length 430, total state size = 19.624 MiB (draft: 0.000 MiB)
3.38.084.791 I srv load: - looking for better prompt, base f_keep = 0.279, sim = 0.702
3.38.084.807 I srv update: - cache state: 18 prompts, 590.243 MiB (limits: 8192.000 MiB, 24576 tokens, 172571 est)
3.38.084.808 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
3.38.084.809 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
3.38.084.809 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
3.38.084.810 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
3.38.084.810 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
3.38.084.811 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
3.38.084.811 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
3.38.084.812 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
3.38.084.812 I srv update: - prompt 0x584e45f74060: 670 tokens, checkpoints: 1, 32.356 MiB
3.38.084.813 I srv update: - prompt 0x584e460738d0: 2373 tokens, checkpoints: 1, 92.246 MiB
3.38.084.813 I srv update: - prompt 0x584e462c17b0: 1123 tokens, checkpoints: 1, 48.287 MiB
3.38.084.814 I srv update: - prompt 0x584e46255e00: 980 tokens, checkpoints: 1, 43.258 MiB
3.38.084.814 I srv update: - prompt 0x584e4602efe0: 1169 tokens, checkpoints: 1, 49.904 MiB
3.38.084.815 I srv update: - prompt 0x584e462be010: 397 tokens, checkpoints: 1, 22.579 MiB
3.38.084.815 I srv update: - prompt 0x584e4c7d7390: 300 tokens, checkpoints: 1, 19.168 MiB
3.38.084.816 I srv update: - prompt 0x584e45f6f690: 333 tokens, checkpoints: 1, 20.328 MiB
3.38.084.816 I srv update: - prompt 0x584e4c8107b0: 565 tokens, checkpoints: 1, 28.487 MiB
3.38.084.817 I srv update: - prompt 0x584e461574e0: 430 tokens, checkpoints: 1, 23.739 MiB
3.38.084.818 I srv get_availabl: prompt cache update took 18.69 ms
3.38.085.735 I slot launch_slot_: id 3 | task 9727 | processing task, is_child = 0
3.38.085.737 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.38.085.738 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.38.085.757 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.38.085.758 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.38.085.771 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.38.085.771 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.38.085.792 I slot update_slots: id 3 | task 9727 | Checking checkpoint with [0, 116] against 0...
3.38.088.133 W slot update_slots: id 3 | task 9727 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 4.115 MiB)
3.40.310.755 I slot print_timing: id 3 | task 9727 | prompt eval time = 564.59 ms / 55 tokens ( 10.27 ms per token, 97.42 tokens per second)
3.40.310.759 I slot print_timing: id 3 | task 9727 | eval time = 1660.37 ms / 104 tokens ( 15.97 ms per token, 62.64 tokens per second)
3.40.310.760 I slot print_timing: id 3 | task 9727 | total time = 2224.96 ms / 159 tokens
3.40.310.760 I slot print_timing: id 3 | task 9727 | graphs reused = 9692
3.40.310.798 I slot release: id 3 | task 9727 | stop processing: n_tokens = 274, truncated = 0
3.40.310.806 I srv update_slots: all slots are idle
3.40.319.487 I srv operator(): Chat format: peg-native
3.40.323.531 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.759 (> 0.100 thold), f_keep = 0.438
3.40.323.534 I srv get_availabl: updating prompt cache
3.40.323.595 W srv prompt_save: - saving prompt with length 274, total state size = 14.138 MiB (draft: 0.000 MiB)
3.40.337.303 I srv load: - looking for better prompt, base f_keep = 0.438, sim = 0.759
3.40.337.316 I srv update: - cache state: 19 prompts, 608.496 MiB (limits: 8192.000 MiB, 24576 tokens, 171083 est)
3.40.337.317 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
3.40.337.318 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
3.40.337.318 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
3.40.337.319 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
3.40.337.319 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
3.40.337.319 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
3.40.337.320 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
3.40.337.320 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
3.40.337.321 I srv update: - prompt 0x584e45f74060: 670 tokens, checkpoints: 1, 32.356 MiB
3.40.337.321 I srv update: - prompt 0x584e460738d0: 2373 tokens, checkpoints: 1, 92.246 MiB
3.40.337.322 I srv update: - prompt 0x584e462c17b0: 1123 tokens, checkpoints: 1, 48.287 MiB
3.40.337.322 I srv update: - prompt 0x584e46255e00: 980 tokens, checkpoints: 1, 43.258 MiB
3.40.337.322 I srv update: - prompt 0x584e4602efe0: 1169 tokens, checkpoints: 1, 49.904 MiB
3.40.337.323 I srv update: - prompt 0x584e462be010: 397 tokens, checkpoints: 1, 22.579 MiB
3.40.337.323 I srv update: - prompt 0x584e4c7d7390: 300 tokens, checkpoints: 1, 19.168 MiB
3.40.337.324 I srv update: - prompt 0x584e45f6f690: 333 tokens, checkpoints: 1, 20.328 MiB
3.40.337.324 I srv update: - prompt 0x584e4c8107b0: 565 tokens, checkpoints: 1, 28.487 MiB
3.40.337.325 I srv update: - prompt 0x584e461574e0: 430 tokens, checkpoints: 1, 23.739 MiB
3.40.337.325 I srv update: - prompt 0x584e460a7360: 274 tokens, checkpoints: 1, 18.253 MiB
3.40.337.326 I srv get_availabl: prompt cache update took 13.79 ms
3.40.338.183 I slot launch_slot_: id 3 | task 9834 | processing task, is_child = 0
3.40.338.185 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.40.338.185 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.40.338.201 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.40.338.201 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.40.338.213 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.40.338.213 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.40.338.231 I slot update_slots: id 3 | task 9834 | Checking checkpoint with [0, 116] against 0...
3.40.340.660 W slot update_slots: id 3 | task 9834 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 4.115 MiB)
3.43.783.725 I slot print_timing: id 3 | task 9834 | n_decoded = 187, tg = 62.29 t/s, tg_3s = 62.29 t/s
3.44.103.847 I slot print_timing: id 3 | task 9834 | prompt eval time = 443.44 ms / 42 tokens ( 10.56 ms per token, 94.71 tokens per second)
3.44.103.851 I slot print_timing: id 3 | task 9834 | eval time = 3322.17 ms / 207 tokens ( 16.05 ms per token, 62.31 tokens per second)
3.44.103.852 I slot print_timing: id 3 | task 9834 | total time = 3765.61 ms / 249 tokens
3.44.103.852 I slot print_timing: id 3 | task 9834 | graphs reused = 9896
3.44.103.890 I slot release: id 3 | task 9834 | stop processing: n_tokens = 364, truncated = 0
3.44.103.900 I srv update_slots: all slots are idle
3.44.171.187 I srv operator(): Chat format: peg-native
3.44.175.021 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.786 (> 0.100 thold), f_keep = 0.343
3.44.175.023 I srv get_availabl: updating prompt cache
3.44.175.073 W srv prompt_save: - saving prompt with length 364, total state size = 17.303 MiB (draft: 0.000 MiB)
3.44.191.738 I srv load: - looking for better prompt, base f_keep = 0.343, sim = 0.786
3.44.191.753 I srv update: - cache state: 20 prompts, 629.915 MiB (limits: 8192.000 MiB, 24576 tokens, 170000 est)
3.44.191.753 I srv update: - prompt 0x584e46078f40: 355 tokens, checkpoints: 1, 20.645 MiB
3.44.191.754 I srv update: - prompt 0x584e4470d5c0: 542 tokens, checkpoints: 1, 27.221 MiB
3.44.191.754 I srv update: - prompt 0x584e460e2900: 740 tokens, checkpoints: 1, 34.184 MiB
3.44.191.755 I srv update: - prompt 0x584e46088cc0: 266 tokens, checkpoints: 1, 17.515 MiB
3.44.191.755 I srv update: - prompt 0x584e4627ec90: 267 tokens, checkpoints: 1, 17.550 MiB
3.44.191.756 I srv update: - prompt 0x584e460dea50: 270 tokens, checkpoints: 1, 17.655 MiB
3.44.191.756 I srv update: - prompt 0x584e462d3b50: 328 tokens, checkpoints: 1, 19.695 MiB
3.44.191.757 I srv update: - prompt 0x584e46385840: 1326 tokens, checkpoints: 1, 55.426 MiB
3.44.191.757 I srv update: - prompt 0x584e45f74060: 670 tokens, checkpoints: 1, 32.356 MiB
3.44.191.758 I srv update: - prompt 0x584e460738d0: 2373 tokens, checkpoints: 1, 92.246 MiB
3.44.191.758 I srv update: - prompt 0x584e462c17b0: 1123 tokens, checkpoints: 1, 48.287 MiB
3.44.191.759 I srv update: - prompt 0x584e46255e00: 980 tokens, checkpoints: 1, 43.258 MiB
3.44.191.759 I srv update: - prompt 0x584e4602efe0: 1169 tokens, checkpoints: 1, 49.904 MiB
3.44.191.760 I srv update: - prompt 0x584e462be010: 397 tokens, checkpoints: 1, 22.579 MiB
3.44.191.760 I srv update: - prompt 0x584e4c7d7390: 300 tokens, checkpoints: 1, 19.168 MiB
3.44.191.761 I srv update: - prompt 0x584e45f6f690: 333 tokens, checkpoints: 1, 20.328 MiB
3.44.191.761 I srv update: - prompt 0x584e4c8107b0: 565 tokens, checkpoints: 1, 28.487 MiB
3.44.191.762 I srv update: - prompt 0x584e461574e0: 430 tokens, checkpoints: 1, 23.739 MiB
3.44.191.762 I srv update: - prompt 0x584e460a7360: 274 tokens, checkpoints: 1, 18.253 MiB
3.44.191.763 I srv update: - prompt 0x584e46283560: 364 tokens, checkpoints: 1, 21.418 MiB
3.44.191.764 I srv get_availabl: prompt cache update took 16.74 ms
3.44.192.688 I slot launch_slot_: id 3 | task 10044 | processing task, is_child = 0
3.44.192.690 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.44.192.690 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.44.192.709 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.44.192.709 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.44.192.723 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.44.192.723 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.44.192.744 I slot update_slots: id 3 | task 10044 | Checking checkpoint with [0, 116] against 0...
3.44.195.125 W slot update_slots: id 3 | task 10044 | restored context checkpoint (pos_min = 0, pos_max = 116, n_tokens = 117, n_past = 116, size = 4.115 MiB)
3.47.645.999 I slot print_timing: id 3 | task 10044 | n_decoded = 188, tg = 62.57 t/s, tg_3s = 62.57 t/s
3.48.414.010 I slot print_timing: id 3 | task 10044 | prompt eval time = 448.42 ms / 43 tokens ( 10.43 ms per token, 95.89 tokens per second)
3.48.414.013 I slot print_timing: id 3 | task 10044 | eval time = 3772.84 ms / 236 tokens ( 15.99 ms per token, 62.55 tokens per second)
3.48.414.014 I slot print_timing: id 3 | task 10044 | total time = 4221.26 ms / 279 tokens
3.48.414.014 I slot print_timing: id 3 | task 10044 | graphs reused = 10129
3.48.414.046 I slot release: id 3 | task 10044 | stop processing: n_tokens = 394, truncated = 0
3.48.414.054 I srv update_slots: all slots are idle
3.48.473.234 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink