qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results_hard/devstral-small2-24b.server.log

868 lines
85 KiB
Plaintext
Raw Permalink Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.065.508 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.065.511 I device_info:
0.00.065.531 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.536 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.543 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.065.605 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.065.608 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.065.650 I srv init: running without SSL
0.00.065.712 I srv init: using 23 threads for HTTP server
0.00.066.056 I srv start: binding port with default address family
0.00.067.262 I srv llama_server: loading model
0.00.067.269 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/mistralai_Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf'
0.00.067.324 I common_init_result: fitting params to device memory ...
0.00.067.326 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.09.683.301 W llama_context: setting new yarn_attn_factor = 1.0000 (mscale == 1.0, mscale_all_dim = 1.0)
0.09.683.310 W llama_context: n_ctx_seq (16384) < n_ctx_train (393216) -- the full capacity of the model will not be utilized
0.09.755.129 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.09.917.967 I srv load_model: initializing slots, n_slots = 4
0.09.983.178 W common_speculative_init: no implementations specified for speculative decoding
0.09.983.181 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384
0.09.983.184 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384
0.09.983.184 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384
0.09.983.185 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384
0.09.983.397 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.09.983.397 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.09.983.398 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.09.983.398 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.09.983.415 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.09.986.376 I init: chat template, example_format: '[SYSTEM_PROMPT]You are a helpful assistant[/SYSTEM_PROMPT][INST]Hello[/INST]Hi there</s>[INST]How are you?[/INST]'
0.09.986.767 I srv init: init: chat template, thinking = 1
0.09.986.801 I srv llama_server: model loaded
0.09.986.804 I srv llama_server: server is listening on http://127.0.0.1:8090
0.09.986.809 I srv update_slots: all slots are idle
0.14.966.106 I srv operator(): Chat format: peg-native
0.14.966.341 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.14.966.344 I srv get_availabl: updating prompt cache
0.14.966.352 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.14.966.357 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est)
0.14.966.358 I srv get_availabl: prompt cache update took 0.01 ms
0.14.966.397 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.14.966.398 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.14.966.399 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.14.966.414 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.14.966.414 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.14.966.423 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.14.966.424 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.19.485.642 I slot print_timing: id 3 | task 0 | n_decoded = 100, tg = 24.45 t/s, tg_3s = 24.45 t/s
0.22.504.936 I slot print_timing: id 3 | task 0 | n_decoded = 173, tg = 24.33 t/s, tg_3s = 24.18 t/s
0.25.533.523 I slot print_timing: id 3 | task 0 | n_decoded = 246, tg = 24.27 t/s, tg_3s = 24.10 t/s
0.28.564.576 I slot print_timing: id 3 | task 0 | n_decoded = 319, tg = 24.22 t/s, tg_3s = 24.08 t/s
0.31.600.104 I slot print_timing: id 3 | task 0 | n_decoded = 392, tg = 24.19 t/s, tg_3s = 24.05 t/s
0.34.639.533 I slot print_timing: id 3 | task 0 | n_decoded = 465, tg = 24.16 t/s, tg_3s = 24.02 t/s
0.37.645.209 I slot print_timing: id 3 | task 0 | n_decoded = 537, tg = 24.14 t/s, tg_3s = 23.95 t/s
0.40.656.676 I slot print_timing: id 3 | task 0 | n_decoded = 609, tg = 24.11 t/s, tg_3s = 23.91 t/s
0.43.669.641 I slot print_timing: id 3 | task 0 | n_decoded = 681, tg = 24.09 t/s, tg_3s = 23.90 t/s
0.45.054.629 I slot print_timing: id 3 | task 0 | prompt eval time = 429.06 ms / 89 tokens ( 4.82 ms per token, 207.43 tokens per second)
0.45.054.632 I slot print_timing: id 3 | task 0 | eval time = 29659.11 ms / 714 tokens ( 41.54 ms per token, 24.07 tokens per second)
0.45.054.633 I slot print_timing: id 3 | task 0 | total time = 30088.17 ms / 803 tokens
0.45.054.638 I slot print_timing: id 3 | task 0 | graphs reused = 0
0.45.054.673 I slot release: id 3 | task 0 | stop processing: n_tokens = 802, truncated = 0
0.45.054.680 I srv update_slots: all slots are idle
0.45.057.616 I srv operator(): Chat format: peg-native
0.45.057.718 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.087
0.45.057.720 I srv get_availabl: updating prompt cache
0.45.057.763 W srv prompt_save: - saving prompt with length 802, total state size = 125.323 MiB (draft: 0.000 MiB)
0.45.177.962 I srv load: - looking for better prompt, base f_keep = 0.087, sim = 0.833
0.45.177.969 I srv update: - cache state: 1 prompts, 125.323 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
0.45.177.970 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
0.45.177.971 I srv get_availabl: prompt cache update took 120.25 ms
0.45.178.002 I slot launch_slot_: id 3 | task 715 | processing task, is_child = 0
0.45.178.003 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.45.178.003 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.45.178.013 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.45.178.014 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.45.178.022 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.45.178.023 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.49.412.483 I slot print_timing: id 3 | task 715 | n_decoded = 100, tg = 24.33 t/s, tg_3s = 24.33 t/s
0.50.359.816 I slot print_timing: id 3 | task 715 | prompt eval time = 123.61 ms / 14 tokens ( 8.83 ms per token, 113.26 tokens per second)
0.50.359.819 I slot print_timing: id 3 | task 715 | eval time = 5058.16 ms / 123 tokens ( 41.12 ms per token, 24.32 tokens per second)
0.50.359.820 I slot print_timing: id 3 | task 715 | total time = 5181.77 ms / 137 tokens
0.50.359.821 I slot print_timing: id 3 | task 715 | graphs reused = 0
0.50.359.840 I slot release: id 3 | task 715 | stop processing: n_tokens = 206, truncated = 0
0.50.359.849 I srv update_slots: all slots are idle
0.50.362.551 I srv operator(): Chat format: peg-native
0.50.362.666 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.340
0.50.362.667 I srv get_availabl: updating prompt cache
0.50.362.690 W srv prompt_save: - saving prompt with length 206, total state size = 32.191 MiB (draft: 0.000 MiB)
0.50.389.012 I srv load: - looking for better prompt, base f_keep = 0.340, sim = 0.833
0.50.389.020 I srv update: - cache state: 2 prompts, 157.513 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
0.50.389.021 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
0.50.389.022 I srv update: - prompt 0x5db836fca9e0: 206 tokens, checkpoints: 0, 32.191 MiB
0.50.389.024 I srv get_availabl: prompt cache update took 26.36 ms
0.50.389.060 I slot launch_slot_: id 3 | task 839 | processing task, is_child = 0
0.50.389.061 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.50.389.062 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.50.389.076 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.50.389.077 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.50.389.085 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.50.389.085 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.54.583.424 I slot print_timing: id 3 | task 839 | n_decoded = 100, tg = 24.56 t/s, tg_3s = 24.56 t/s
0.55.325.452 I slot print_timing: id 3 | task 839 | prompt eval time = 122.93 ms / 14 tokens ( 8.78 ms per token, 113.89 tokens per second)
0.55.325.456 I slot print_timing: id 3 | task 839 | eval time = 4813.42 ms / 118 tokens ( 40.79 ms per token, 24.51 tokens per second)
0.55.325.456 I slot print_timing: id 3 | task 839 | total time = 4936.34 ms / 132 tokens
0.55.325.457 I slot print_timing: id 3 | task 839 | graphs reused = 0
0.55.325.475 I slot release: id 3 | task 839 | stop processing: n_tokens = 201, truncated = 0
0.55.325.481 I srv update_slots: all slots are idle
0.55.327.926 I srv operator(): Chat format: peg-native
0.55.328.051 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.843 (> 0.100 thold), f_keep = 0.348
0.55.328.053 I srv get_availabl: updating prompt cache
0.55.328.076 W srv prompt_save: - saving prompt with length 201, total state size = 31.409 MiB (draft: 0.000 MiB)
0.55.353.830 I srv load: - looking for better prompt, base f_keep = 0.348, sim = 0.843
0.55.353.838 I srv update: - cache state: 3 prompts, 188.923 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
0.55.353.839 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
0.55.353.840 I srv update: - prompt 0x5db836fca9e0: 206 tokens, checkpoints: 0, 32.191 MiB
0.55.353.841 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
0.55.353.842 I srv get_availabl: prompt cache update took 25.79 ms
0.55.353.881 I slot launch_slot_: id 3 | task 958 | processing task, is_child = 0
0.55.353.881 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.55.353.882 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.55.353.895 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.55.353.895 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.55.353.904 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.55.353.904 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.59.557.365 I slot print_timing: id 3 | task 958 | n_decoded = 100, tg = 24.51 t/s, tg_3s = 24.51 t/s
1.02.573.492 I slot print_timing: id 3 | task 958 | n_decoded = 173, tg = 24.38 t/s, tg_3s = 24.20 t/s
1.05.603.259 I slot print_timing: id 3 | task 958 | n_decoded = 246, tg = 24.29 t/s, tg_3s = 24.09 t/s
1.08.632.520 I slot print_timing: id 3 | task 958 | n_decoded = 319, tg = 24.25 t/s, tg_3s = 24.10 t/s
1.11.662.654 I slot print_timing: id 3 | task 958 | n_decoded = 392, tg = 24.22 t/s, tg_3s = 24.09 t/s
1.14.693.759 I slot print_timing: id 3 | task 958 | n_decoded = 465, tg = 24.20 t/s, tg_3s = 24.08 t/s
1.17.731.527 I slot print_timing: id 3 | task 958 | n_decoded = 538, tg = 24.17 t/s, tg_3s = 24.03 t/s
1.19.230.381 I slot print_timing: id 3 | task 958 | prompt eval time = 122.97 ms / 13 tokens ( 9.46 ms per token, 105.72 tokens per second)
1.19.230.384 I slot print_timing: id 3 | task 958 | eval time = 23753.49 ms / 574 tokens ( 41.38 ms per token, 24.16 tokens per second)
1.19.230.385 I slot print_timing: id 3 | task 958 | total time = 23876.45 ms / 587 tokens
1.19.230.386 I slot print_timing: id 3 | task 958 | graphs reused = 0
1.19.230.405 I slot release: id 3 | task 958 | stop processing: n_tokens = 656, truncated = 0
1.19.230.414 I srv update_slots: all slots are idle
1.19.232.890 I srv operator(): Chat format: peg-native
1.19.233.001 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.729 (> 0.100 thold), f_keep = 0.107
1.19.233.003 I srv get_availabl: updating prompt cache
1.19.233.038 W srv prompt_save: - saving prompt with length 656, total state size = 102.508 MiB (draft: 0.000 MiB)
1.19.348.534 I srv load: - looking for better prompt, base f_keep = 0.107, sim = 0.729
1.19.348.538 I srv load: - found better prompt with f_keep = 0.345, sim = 0.740
1.19.362.980 I srv update: - cache state: 3 prompts, 259.241 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
1.19.362.983 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
1.19.362.984 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
1.19.362.984 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
1.19.362.986 I srv get_availabl: prompt cache update took 129.98 ms
1.19.363.023 I slot launch_slot_: id 3 | task 1533 | processing task, is_child = 0
1.19.363.023 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.19.363.024 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.19.363.037 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.19.363.037 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.19.363.045 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.19.363.046 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.23.722.637 I slot print_timing: id 3 | task 1533 | n_decoded = 100, tg = 24.34 t/s, tg_3s = 24.34 t/s
1.26.754.416 I slot print_timing: id 3 | task 1533 | n_decoded = 173, tg = 24.23 t/s, tg_3s = 24.08 t/s
1.29.795.514 I slot print_timing: id 3 | task 1533 | n_decoded = 246, tg = 24.16 t/s, tg_3s = 24.00 t/s
1.32.801.593 I slot print_timing: id 3 | task 1533 | n_decoded = 318, tg = 24.11 t/s, tg_3s = 23.95 t/s
1.35.804.759 I slot print_timing: id 3 | task 1533 | n_decoded = 390, tg = 24.09 t/s, tg_3s = 23.97 t/s
1.38.825.570 I slot print_timing: id 3 | task 1533 | n_decoded = 462, tg = 24.05 t/s, tg_3s = 23.83 t/s
1.41.847.621 I slot print_timing: id 3 | task 1533 | n_decoded = 534, tg = 24.02 t/s, tg_3s = 23.82 t/s
1.44.886.730 I slot print_timing: id 3 | task 1533 | n_decoded = 607, tg = 24.02 t/s, tg_3s = 24.02 t/s
1.47.917.601 I slot print_timing: id 3 | task 1533 | n_decoded = 680, tg = 24.03 t/s, tg_3s = 24.09 t/s
1.50.954.391 I slot print_timing: id 3 | task 1533 | n_decoded = 753, tg = 24.03 t/s, tg_3s = 24.04 t/s
1.53.994.434 I slot print_timing: id 3 | task 1533 | n_decoded = 826, tg = 24.03 t/s, tg_3s = 24.01 t/s
1.56.997.453 I slot print_timing: id 3 | task 1533 | n_decoded = 898, tg = 24.02 t/s, tg_3s = 23.98 t/s
2.00.008.579 I slot print_timing: id 3 | task 1533 | n_decoded = 970, tg = 24.01 t/s, tg_3s = 23.91 t/s
2.02.272.012 I slot print_timing: id 3 | task 1533 | prompt eval time = 251.56 ms / 25 tokens ( 10.06 ms per token, 99.38 tokens per second)
2.02.272.015 I slot print_timing: id 3 | task 1533 | eval time = 42657.39 ms / 1024 tokens ( 41.66 ms per token, 24.01 tokens per second)
2.02.272.015 I slot print_timing: id 3 | task 1533 | total time = 42908.94 ms / 1049 tokens
2.02.272.016 I slot print_timing: id 3 | task 1533 | graphs reused = 0
2.02.272.034 I slot release: id 3 | task 1533 | stop processing: n_tokens = 1119, truncated = 0
2.02.272.044 I srv update_slots: all slots are idle
2.02.274.572 I srv operator(): Chat format: peg-native
2.02.274.679 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.824 (> 0.100 thold), f_keep = 0.063
2.02.274.681 I srv get_availabl: updating prompt cache
2.02.274.733 W srv prompt_save: - saving prompt with length 1119, total state size = 174.857 MiB (draft: 0.000 MiB)
2.02.522.005 I srv load: - looking for better prompt, base f_keep = 0.063, sim = 0.824
2.02.522.011 I srv update: - cache state: 4 prompts, 434.098 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
2.02.522.012 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
2.02.522.013 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
2.02.522.014 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
2.02.522.014 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
2.02.522.016 I srv get_availabl: prompt cache update took 247.33 ms
2.02.522.043 I slot launch_slot_: id 3 | task 2558 | processing task, is_child = 0
2.02.522.044 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.02.522.045 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.02.522.057 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.02.522.058 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.02.522.067 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.02.522.067 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.06.756.710 I slot print_timing: id 3 | task 2558 | n_decoded = 100, tg = 24.32 t/s, tg_3s = 24.32 t/s
2.09.788.798 I slot print_timing: id 3 | task 2558 | n_decoded = 173, tg = 24.22 t/s, tg_3s = 24.08 t/s
2.11.454.880 I slot print_timing: id 3 | task 2558 | prompt eval time = 123.41 ms / 15 tokens ( 8.23 ms per token, 121.55 tokens per second)
2.11.454.884 I slot print_timing: id 3 | task 2558 | eval time = 8809.39 ms / 213 tokens ( 41.36 ms per token, 24.18 tokens per second)
2.11.454.884 I slot print_timing: id 3 | task 2558 | total time = 8932.79 ms / 228 tokens
2.11.454.885 I slot print_timing: id 3 | task 2558 | graphs reused = 0
2.11.454.904 I slot release: id 3 | task 2558 | stop processing: n_tokens = 297, truncated = 0
2.11.454.913 I srv update_slots: all slots are idle
2.11.457.510 I srv operator(): Chat format: peg-native
2.11.457.618 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.877 (> 0.100 thold), f_keep = 0.239
2.11.457.620 I srv get_availabl: updating prompt cache
2.11.457.646 W srv prompt_save: - saving prompt with length 297, total state size = 46.411 MiB (draft: 0.000 MiB)
2.11.493.854 I srv load: - looking for better prompt, base f_keep = 0.239, sim = 0.877
2.11.493.865 I srv update: - cache state: 5 prompts, 480.509 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
2.11.493.866 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
2.11.493.867 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
2.11.493.867 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
2.11.493.868 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
2.11.493.869 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
2.11.493.870 I srv get_availabl: prompt cache update took 36.25 ms
2.11.493.904 I slot launch_slot_: id 3 | task 2772 | processing task, is_child = 0
2.11.493.905 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.11.493.906 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.11.493.921 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.11.493.921 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.11.493.930 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.11.493.930 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.15.731.854 I slot print_timing: id 3 | task 2772 | n_decoded = 100, tg = 24.30 t/s, tg_3s = 24.30 t/s
2.18.768.762 I slot print_timing: id 3 | task 2772 | n_decoded = 173, tg = 24.19 t/s, tg_3s = 24.04 t/s
2.21.773.752 I slot print_timing: id 3 | task 2772 | n_decoded = 245, tg = 24.12 t/s, tg_3s = 23.96 t/s
2.24.779.536 I slot print_timing: id 3 | task 2772 | n_decoded = 317, tg = 24.08 t/s, tg_3s = 23.95 t/s
2.27.035.559 I slot print_timing: id 3 | task 2772 | prompt eval time = 122.46 ms / 10 tokens ( 12.25 ms per token, 81.66 tokens per second)
2.27.035.563 I slot print_timing: id 3 | task 2772 | eval time = 15419.15 ms / 371 tokens ( 41.56 ms per token, 24.06 tokens per second)
2.27.035.564 I slot print_timing: id 3 | task 2772 | total time = 15541.61 ms / 381 tokens
2.27.035.565 I slot print_timing: id 3 | task 2772 | graphs reused = 0
2.27.035.585 I slot release: id 3 | task 2772 | stop processing: n_tokens = 451, truncated = 0
2.27.035.594 I srv update_slots: all slots are idle
2.27.038.255 I srv operator(): Chat format: peg-native
2.27.038.364 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.855 (> 0.100 thold), f_keep = 0.157
2.27.038.366 I srv get_availabl: updating prompt cache
2.27.038.394 W srv prompt_save: - saving prompt with length 451, total state size = 70.475 MiB (draft: 0.000 MiB)
2.27.091.680 I srv load: - looking for better prompt, base f_keep = 0.157, sim = 0.855
2.27.091.690 I srv update: - cache state: 6 prompts, 550.983 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
2.27.091.691 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
2.27.091.692 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
2.27.091.692 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
2.27.091.693 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
2.27.091.693 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
2.27.091.694 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
2.27.091.695 I srv get_availabl: prompt cache update took 53.33 ms
2.27.091.734 I slot launch_slot_: id 3 | task 3144 | processing task, is_child = 0
2.27.091.736 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.27.091.736 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.27.091.751 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.27.091.751 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.27.091.760 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.27.091.760 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.31.311.437 I slot print_timing: id 3 | task 3144 | n_decoded = 100, tg = 24.41 t/s, tg_3s = 24.41 t/s
2.34.313.482 I slot print_timing: id 3 | task 3144 | n_decoded = 173, tg = 24.37 t/s, tg_3s = 24.32 t/s
2.37.332.627 I slot print_timing: id 3 | task 3144 | n_decoded = 246, tg = 24.31 t/s, tg_3s = 24.18 t/s
2.40.351.600 I slot print_timing: id 3 | task 3144 | n_decoded = 319, tg = 24.28 t/s, tg_3s = 24.18 t/s
2.43.377.630 I slot print_timing: id 3 | task 3144 | n_decoded = 392, tg = 24.25 t/s, tg_3s = 24.12 t/s
2.46.410.715 I slot print_timing: id 3 | task 3144 | n_decoded = 465, tg = 24.22 t/s, tg_3s = 24.07 t/s
2.49.441.395 I slot print_timing: id 3 | task 3144 | n_decoded = 538, tg = 24.21 t/s, tg_3s = 24.09 t/s
2.51.350.795 I slot print_timing: id 3 | task 3144 | prompt eval time = 122.82 ms / 12 tokens ( 10.23 ms per token, 97.71 tokens per second)
2.51.350.798 I slot print_timing: id 3 | task 3144 | eval time = 24136.19 ms / 584 tokens ( 41.33 ms per token, 24.20 tokens per second)
2.51.350.799 I slot print_timing: id 3 | task 3144 | total time = 24259.01 ms / 596 tokens
2.51.350.799 I slot print_timing: id 3 | task 3144 | graphs reused = 0
2.51.350.817 I slot release: id 3 | task 3144 | stop processing: n_tokens = 666, truncated = 0
2.51.350.827 I srv update_slots: all slots are idle
2.51.353.369 I srv operator(): Chat format: peg-native
2.51.353.483 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.631 (> 0.100 thold), f_keep = 0.105
2.51.353.485 I srv get_availabl: updating prompt cache
2.51.353.526 W srv prompt_save: - saving prompt with length 666, total state size = 104.071 MiB (draft: 0.000 MiB)
2.51.487.961 I srv load: - looking for better prompt, base f_keep = 0.105, sim = 0.631
2.51.487.970 I srv update: - cache state: 7 prompts, 655.055 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
2.51.487.971 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
2.51.487.973 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
2.51.487.974 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
2.51.487.975 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
2.51.487.976 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
2.51.487.976 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
2.51.487.977 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
2.51.487.978 I srv get_availabl: prompt cache update took 134.49 ms
2.51.488.004 I slot launch_slot_: id 3 | task 3729 | processing task, is_child = 0
2.51.488.005 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.51.488.005 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.51.488.016 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.51.488.016 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.51.488.025 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.51.488.025 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
2.55.832.313 I slot print_timing: id 3 | task 3729 | n_decoded = 100, tg = 24.40 t/s, tg_3s = 24.40 t/s
2.58.861.752 I slot print_timing: id 3 | task 3729 | n_decoded = 173, tg = 24.27 t/s, tg_3s = 24.10 t/s
3.01.899.223 I slot print_timing: id 3 | task 3729 | n_decoded = 246, tg = 24.20 t/s, tg_3s = 24.03 t/s
3.04.938.682 I slot print_timing: id 3 | task 3729 | n_decoded = 319, tg = 24.16 t/s, tg_3s = 24.02 t/s
3.07.939.494 I slot print_timing: id 3 | task 3729 | n_decoded = 391, tg = 24.13 t/s, tg_3s = 23.99 t/s
3.09.984.139 I slot print_timing: id 3 | task 3729 | prompt eval time = 245.40 ms / 41 tokens ( 5.99 ms per token, 167.07 tokens per second)
3.09.984.142 I slot print_timing: id 3 | task 3729 | eval time = 18250.69 ms / 440 tokens ( 41.48 ms per token, 24.11 tokens per second)
3.09.984.143 I slot print_timing: id 3 | task 3729 | total time = 18496.09 ms / 481 tokens
3.09.984.144 I slot print_timing: id 3 | task 3729 | graphs reused = 0
3.09.984.164 I slot release: id 3 | task 3729 | stop processing: n_tokens = 550, truncated = 0
3.09.984.174 I srv update_slots: all slots are idle
3.09.987.073 I srv operator(): Chat format: peg-native
3.09.987.191 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.542 (> 0.100 thold), f_keep = 0.129
3.09.987.192 I srv get_availabl: updating prompt cache
3.09.987.227 W srv prompt_save: - saving prompt with length 550, total state size = 85.945 MiB (draft: 0.000 MiB)
3.10.100.921 I srv load: - looking for better prompt, base f_keep = 0.129, sim = 0.542
3.10.100.928 I srv update: - cache state: 8 prompts, 740.999 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
3.10.100.929 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
3.10.100.930 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
3.10.100.931 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
3.10.100.934 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
3.10.100.935 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
3.10.100.935 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
3.10.100.936 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
3.10.100.937 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
3.10.100.938 I srv get_availabl: prompt cache update took 113.74 ms
3.10.100.967 I slot launch_slot_: id 3 | task 4170 | processing task, is_child = 0
3.10.100.967 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.10.100.968 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.10.100.978 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.10.100.979 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.10.100.987 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.10.100.988 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.14.535.247 I slot print_timing: id 3 | task 4170 | n_decoded = 100, tg = 24.25 t/s, tg_3s = 24.25 t/s
3.17.574.597 I slot print_timing: id 3 | task 4170 | n_decoded = 173, tg = 24.15 t/s, tg_3s = 24.02 t/s
3.20.577.768 I slot print_timing: id 3 | task 4170 | n_decoded = 245, tg = 24.10 t/s, tg_3s = 23.97 t/s
3.23.617.343 I slot print_timing: id 3 | task 4170 | n_decoded = 318, tg = 24.08 t/s, tg_3s = 24.02 t/s
3.24.820.629 I slot print_timing: id 3 | task 4170 | prompt eval time = 310.56 ms / 60 tokens ( 5.18 ms per token, 193.20 tokens per second)
3.24.820.632 I slot print_timing: id 3 | task 4170 | eval time = 14409.06 ms / 347 tokens ( 41.52 ms per token, 24.08 tokens per second)
3.24.820.633 I slot print_timing: id 3 | task 4170 | total time = 14719.62 ms / 407 tokens
3.24.820.633 I slot print_timing: id 3 | task 4170 | graphs reused = 0
3.24.820.656 I slot release: id 3 | task 4170 | stop processing: n_tokens = 477, truncated = 0
3.24.820.665 I srv update_slots: all slots are idle
3.24.823.189 I srv operator(): Chat format: peg-native
3.24.823.297 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.660 (> 0.100 thold), f_keep = 0.147
3.24.823.298 I srv get_availabl: updating prompt cache
3.24.823.326 W srv prompt_save: - saving prompt with length 477, total state size = 74.538 MiB (draft: 0.000 MiB)
3.24.879.700 I srv load: - looking for better prompt, base f_keep = 0.147, sim = 0.660
3.24.879.709 I srv update: - cache state: 9 prompts, 815.537 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
3.24.879.710 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
3.24.879.711 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
3.24.879.711 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
3.24.879.712 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
3.24.879.712 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
3.24.879.713 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
3.24.879.713 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
3.24.879.714 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
3.24.879.714 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
3.24.879.715 I srv get_availabl: prompt cache update took 56.42 ms
3.24.879.752 I slot launch_slot_: id 3 | task 4518 | processing task, is_child = 0
3.24.879.753 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.24.879.754 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.24.879.768 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.24.879.768 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.24.879.777 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.24.879.777 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.29.258.218 I slot print_timing: id 3 | task 4518 | n_decoded = 100, tg = 24.42 t/s, tg_3s = 24.42 t/s
3.32.281.697 I slot print_timing: id 3 | task 4518 | n_decoded = 173, tg = 24.30 t/s, tg_3s = 24.14 t/s
3.35.311.479 I slot print_timing: id 3 | task 4518 | n_decoded = 246, tg = 24.24 t/s, tg_3s = 24.09 t/s
3.38.341.496 I slot print_timing: id 3 | task 4518 | n_decoded = 319, tg = 24.21 t/s, tg_3s = 24.09 t/s
3.41.374.842 I slot print_timing: id 3 | task 4518 | n_decoded = 392, tg = 24.18 t/s, tg_3s = 24.07 t/s
3.43.993.824 I slot print_timing: id 3 | task 4518 | prompt eval time = 283.31 ms / 36 tokens ( 7.87 ms per token, 127.07 tokens per second)
3.43.993.827 I slot print_timing: id 3 | task 4518 | eval time = 18830.72 ms / 455 tokens ( 41.39 ms per token, 24.16 tokens per second)
3.43.993.827 I slot print_timing: id 3 | task 4518 | total time = 19114.02 ms / 491 tokens
3.43.993.828 I slot print_timing: id 3 | task 4518 | graphs reused = 0
3.43.993.849 I slot release: id 3 | task 4518 | stop processing: n_tokens = 560, truncated = 0
3.43.993.857 I srv update_slots: all slots are idle
3.43.996.711 I srv operator(): Chat format: peg-native
3.43.996.814 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.538 (> 0.100 thold), f_keep = 0.125
3.43.996.815 I srv get_availabl: updating prompt cache
3.43.996.847 W srv prompt_save: - saving prompt with length 560, total state size = 87.507 MiB (draft: 0.000 MiB)
3.44.102.223 I srv load: - looking for better prompt, base f_keep = 0.125, sim = 0.538
3.44.102.230 I srv update: - cache state: 10 prompts, 903.044 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
3.44.102.231 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
3.44.102.232 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
3.44.102.233 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
3.44.102.236 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
3.44.102.237 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
3.44.102.237 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
3.44.102.238 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
3.44.102.238 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
3.44.102.239 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
3.44.102.241 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
3.44.102.242 I srv get_availabl: prompt cache update took 105.43 ms
3.44.102.267 I slot launch_slot_: id 3 | task 4974 | processing task, is_child = 0
3.44.102.267 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.44.102.268 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.44.102.278 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.44.102.278 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.44.102.287 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.44.102.287 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.48.521.781 I slot print_timing: id 3 | task 4974 | n_decoded = 100, tg = 24.34 t/s, tg_3s = 24.34 t/s
3.51.554.622 I slot print_timing: id 3 | task 4974 | n_decoded = 173, tg = 24.22 t/s, tg_3s = 24.07 t/s
3.54.588.530 I slot print_timing: id 3 | task 4974 | n_decoded = 246, tg = 24.18 t/s, tg_3s = 24.06 t/s
3.57.627.662 I slot print_timing: id 3 | task 4974 | n_decoded = 319, tg = 24.14 t/s, tg_3s = 24.02 t/s
3.59.296.581 I slot print_timing: id 3 | task 4974 | prompt eval time = 310.72 ms / 60 tokens ( 5.18 ms per token, 193.10 tokens per second)
3.59.296.585 I slot print_timing: id 3 | task 4974 | eval time = 14883.55 ms / 359 tokens ( 41.46 ms per token, 24.12 tokens per second)
3.59.296.586 I slot print_timing: id 3 | task 4974 | total time = 15194.27 ms / 419 tokens
3.59.296.586 I slot print_timing: id 3 | task 4974 | graphs reused = 0
3.59.296.608 I slot release: id 3 | task 4974 | stop processing: n_tokens = 488, truncated = 0
3.59.296.616 I srv update_slots: all slots are idle
3.59.299.293 I srv operator(): Chat format: peg-native
3.59.299.399 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.569 (> 0.100 thold), f_keep = 0.143
3.59.299.401 I srv get_availabl: updating prompt cache
3.59.299.441 W srv prompt_save: - saving prompt with length 488, total state size = 76.257 MiB (draft: 0.000 MiB)
3.59.356.670 I srv load: - looking for better prompt, base f_keep = 0.143, sim = 0.569
3.59.356.682 I srv update: - cache state: 11 prompts, 979.301 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
3.59.356.683 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
3.59.356.684 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
3.59.356.684 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
3.59.356.685 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
3.59.356.686 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
3.59.356.686 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
3.59.356.687 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
3.59.356.687 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
3.59.356.688 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
3.59.356.689 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
3.59.356.689 I srv update: - prompt 0x5db83777bb60: 488 tokens, checkpoints: 0, 76.257 MiB
3.59.356.695 I srv get_availabl: prompt cache update took 57.29 ms
3.59.356.735 I slot launch_slot_: id 3 | task 5334 | processing task, is_child = 0
3.59.356.735 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.59.356.736 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.59.356.748 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
3.59.356.748 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
3.59.356.757 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.59.356.757 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.03.733.582 I slot print_timing: id 3 | task 5334 | n_decoded = 100, tg = 24.40 t/s, tg_3s = 24.40 t/s
4.06.759.572 I slot print_timing: id 3 | task 5334 | n_decoded = 173, tg = 24.29 t/s, tg_3s = 24.12 t/s
4.09.629.349 I slot print_timing: id 3 | task 5334 | prompt eval time = 279.10 ms / 53 tokens ( 5.27 ms per token, 189.89 tokens per second)
4.09.629.353 I slot print_timing: id 3 | task 5334 | eval time = 9993.47 ms / 242 tokens ( 41.30 ms per token, 24.22 tokens per second)
4.09.629.354 I slot print_timing: id 3 | task 5334 | total time = 10272.57 ms / 295 tokens
4.09.629.354 I slot print_timing: id 3 | task 5334 | graphs reused = 0
4.09.629.378 I slot release: id 3 | task 5334 | stop processing: n_tokens = 364, truncated = 0
4.09.629.384 I srv update_slots: all slots are idle
4.09.631.947 I srv operator(): Chat format: peg-native
4.09.632.111 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.628 (> 0.100 thold), f_keep = 0.195
4.09.632.112 I srv get_availabl: updating prompt cache
4.09.632.141 W srv prompt_save: - saving prompt with length 364, total state size = 56.880 MiB (draft: 0.000 MiB)
4.09.675.596 I srv load: - looking for better prompt, base f_keep = 0.195, sim = 0.628
4.09.675.607 I srv update: - cache state: 12 prompts, 1036.181 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
4.09.675.607 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
4.09.675.608 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
4.09.675.609 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
4.09.675.610 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
4.09.675.611 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
4.09.675.614 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
4.09.675.615 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
4.09.675.615 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
4.09.675.616 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
4.09.675.616 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
4.09.675.617 I srv update: - prompt 0x5db83777bb60: 488 tokens, checkpoints: 0, 76.257 MiB
4.09.675.618 I srv update: - prompt 0x5db83764fbd0: 364 tokens, checkpoints: 0, 56.880 MiB
4.09.675.619 I srv get_availabl: prompt cache update took 43.51 ms
4.09.675.658 I slot launch_slot_: id 3 | task 5577 | processing task, is_child = 0
4.09.675.659 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.09.675.659 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.09.675.671 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.09.675.672 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.09.675.680 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.09.675.680 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.14.000.580 I slot print_timing: id 3 | task 5577 | n_decoded = 100, tg = 24.52 t/s, tg_3s = 24.52 t/s
4.17.016.435 I slot print_timing: id 3 | task 5577 | n_decoded = 173, tg = 24.38 t/s, tg_3s = 24.21 t/s
4.20.043.078 I slot print_timing: id 3 | task 5577 | n_decoded = 246, tg = 24.31 t/s, tg_3s = 24.12 t/s
4.23.072.500 I slot print_timing: id 3 | task 5577 | n_decoded = 319, tg = 24.26 t/s, tg_3s = 24.10 t/s
4.26.102.137 I slot print_timing: id 3 | task 5577 | n_decoded = 392, tg = 24.23 t/s, tg_3s = 24.10 t/s
4.27.140.732 I slot print_timing: id 3 | task 5577 | prompt eval time = 246.01 ms / 42 tokens ( 5.86 ms per token, 170.73 tokens per second)
4.27.140.736 I slot print_timing: id 3 | task 5577 | eval time = 17219.02 ms / 417 tokens ( 41.29 ms per token, 24.22 tokens per second)
4.27.140.736 I slot print_timing: id 3 | task 5577 | total time = 17465.03 ms / 459 tokens
4.27.140.737 I slot print_timing: id 3 | task 5577 | graphs reused = 0
4.27.140.760 I slot release: id 3 | task 5577 | stop processing: n_tokens = 529, truncated = 0
4.27.140.769 I srv update_slots: all slots are idle
4.27.143.413 I srv operator(): Chat format: peg-native
4.27.143.519 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.588 (> 0.100 thold), f_keep = 0.132
4.27.143.520 I srv get_availabl: updating prompt cache
4.27.143.560 W srv prompt_save: - saving prompt with length 529, total state size = 82.663 MiB (draft: 0.000 MiB)
4.27.232.134 I srv load: - looking for better prompt, base f_keep = 0.132, sim = 0.588
4.27.232.143 I srv update: - cache state: 13 prompts, 1118.844 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
4.27.232.144 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
4.27.232.146 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
4.27.232.146 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
4.27.232.150 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
4.27.232.150 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
4.27.232.151 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
4.27.232.151 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
4.27.232.152 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
4.27.232.153 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
4.27.232.155 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
4.27.232.156 I srv update: - prompt 0x5db83777bb60: 488 tokens, checkpoints: 0, 76.257 MiB
4.27.232.156 I srv update: - prompt 0x5db83764fbd0: 364 tokens, checkpoints: 0, 56.880 MiB
4.27.232.157 I srv update: - prompt 0x5db83766b910: 529 tokens, checkpoints: 0, 82.663 MiB
4.27.232.158 I srv get_availabl: prompt cache update took 88.64 ms
4.27.232.185 I slot launch_slot_: id 3 | task 5995 | processing task, is_child = 0
4.27.232.186 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.27.232.187 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.27.232.198 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.27.232.198 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.27.232.207 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.27.232.207 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.31.618.108 I slot print_timing: id 3 | task 5995 | n_decoded = 100, tg = 24.34 t/s, tg_3s = 24.34 t/s
4.34.654.597 I slot print_timing: id 3 | task 5995 | n_decoded = 173, tg = 24.22 t/s, tg_3s = 24.04 t/s
4.35.692.576 I slot print_timing: id 3 | task 5995 | prompt eval time = 278.20 ms / 49 tokens ( 5.68 ms per token, 176.13 tokens per second)
4.35.692.579 I slot print_timing: id 3 | task 5995 | eval time = 8182.15 ms / 198 tokens ( 41.32 ms per token, 24.20 tokens per second)
4.35.692.580 I slot print_timing: id 3 | task 5995 | total time = 8460.35 ms / 247 tokens
4.35.692.581 I slot print_timing: id 3 | task 5995 | graphs reused = 0
4.35.692.602 I slot release: id 3 | task 5995 | stop processing: n_tokens = 316, truncated = 0
4.35.692.611 I srv update_slots: all slots are idle
4.35.695.259 I srv operator(): Chat format: peg-native
4.35.695.391 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.222
4.35.695.394 I srv get_availabl: updating prompt cache
4.35.695.426 W srv prompt_save: - saving prompt with length 316, total state size = 49.380 MiB (draft: 0.000 MiB)
4.35.733.314 I srv load: - looking for better prompt, base f_keep = 0.222, sim = 0.667
4.35.733.326 I srv update: - cache state: 14 prompts, 1168.224 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
4.35.733.327 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
4.35.733.328 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
4.35.733.328 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
4.35.733.329 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
4.35.733.329 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
4.35.733.330 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
4.35.733.330 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
4.35.733.331 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
4.35.733.331 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
4.35.733.332 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
4.35.733.332 I srv update: - prompt 0x5db83777bb60: 488 tokens, checkpoints: 0, 76.257 MiB
4.35.733.333 I srv update: - prompt 0x5db83764fbd0: 364 tokens, checkpoints: 0, 56.880 MiB
4.35.733.333 I srv update: - prompt 0x5db83766b910: 529 tokens, checkpoints: 0, 82.663 MiB
4.35.733.334 I srv update: - prompt 0x5db8376780b0: 316 tokens, checkpoints: 0, 49.380 MiB
4.35.733.335 I srv get_availabl: prompt cache update took 37.94 ms
4.35.733.370 I slot launch_slot_: id 3 | task 6194 | processing task, is_child = 0
4.35.733.371 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.35.733.371 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.35.733.382 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.35.733.383 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.35.733.391 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.35.733.392 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.40.134.554 I slot print_timing: id 3 | task 6194 | n_decoded = 100, tg = 24.28 t/s, tg_3s = 24.28 t/s
4.43.171.066 I slot print_timing: id 3 | task 6194 | n_decoded = 173, tg = 24.18 t/s, tg_3s = 24.04 t/s
4.46.181.284 I slot print_timing: id 3 | task 6194 | n_decoded = 245, tg = 24.10 t/s, tg_3s = 23.92 t/s
4.49.188.717 I slot print_timing: id 3 | task 6194 | n_decoded = 317, tg = 24.07 t/s, tg_3s = 23.94 t/s
4.51.866.352 I slot print_timing: id 3 | task 6194 | prompt eval time = 283.16 ms / 35 tokens ( 8.09 ms per token, 123.61 tokens per second)
4.51.866.356 I slot print_timing: id 3 | task 6194 | eval time = 15849.78 ms / 381 tokens ( 41.60 ms per token, 24.04 tokens per second)
4.51.866.356 I slot print_timing: id 3 | task 6194 | total time = 16132.94 ms / 416 tokens
4.51.866.357 I slot print_timing: id 3 | task 6194 | graphs reused = 0
4.51.866.377 I slot release: id 3 | task 6194 | stop processing: n_tokens = 485, truncated = 0
4.51.866.386 I srv update_slots: all slots are idle
4.51.868.870 I srv operator(): Chat format: peg-native
4.51.868.987 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.144
4.51.868.989 I srv get_availabl: updating prompt cache
4.51.869.024 W srv prompt_save: - saving prompt with length 485, total state size = 75.788 MiB (draft: 0.000 MiB)
4.51.926.099 I srv load: - looking for better prompt, base f_keep = 0.144, sim = 0.667
4.51.926.111 I srv update: - cache state: 15 prompts, 1244.011 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
4.51.926.112 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
4.51.926.113 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
4.51.926.113 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
4.51.926.114 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
4.51.926.114 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
4.51.926.115 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
4.51.926.119 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
4.51.926.119 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
4.51.926.120 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
4.51.926.120 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
4.51.926.121 I srv update: - prompt 0x5db83777bb60: 488 tokens, checkpoints: 0, 76.257 MiB
4.51.926.121 I srv update: - prompt 0x5db83764fbd0: 364 tokens, checkpoints: 0, 56.880 MiB
4.51.926.122 I srv update: - prompt 0x5db83766b910: 529 tokens, checkpoints: 0, 82.663 MiB
4.51.926.125 I srv update: - prompt 0x5db8376780b0: 316 tokens, checkpoints: 0, 49.380 MiB
4.51.926.125 I srv update: - prompt 0x5db83766bcf0: 485 tokens, checkpoints: 0, 75.788 MiB
4.51.926.126 I srv get_availabl: prompt cache update took 57.14 ms
4.51.926.167 I slot launch_slot_: id 3 | task 6576 | processing task, is_child = 0
4.51.926.168 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
4.51.926.168 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
4.51.926.182 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
4.51.926.182 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
4.51.926.191 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
4.51.926.191 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
4.56.332.671 I slot print_timing: id 3 | task 6576 | n_decoded = 100, tg = 24.25 t/s, tg_3s = 24.25 t/s
4.59.335.617 I slot print_timing: id 3 | task 6576 | n_decoded = 172, tg = 24.14 t/s, tg_3s = 23.98 t/s
5.01.092.137 I slot print_timing: id 3 | task 6576 | prompt eval time = 283.43 ms / 35 tokens ( 8.10 ms per token, 123.49 tokens per second)
5.01.092.141 I slot print_timing: id 3 | task 6576 | eval time = 8882.50 ms / 214 tokens ( 41.51 ms per token, 24.09 tokens per second)
5.01.092.142 I slot print_timing: id 3 | task 6576 | total time = 9165.92 ms / 249 tokens
5.01.092.142 I slot print_timing: id 3 | task 6576 | graphs reused = 0
5.01.092.165 I slot release: id 3 | task 6576 | stop processing: n_tokens = 318, truncated = 0
5.01.092.173 I srv update_slots: all slots are idle
5.01.094.631 I srv operator(): Chat format: peg-native
5.01.094.737 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.722 (> 0.100 thold), f_keep = 0.220
5.01.094.739 I srv get_availabl: updating prompt cache
5.01.094.764 W srv prompt_save: - saving prompt with length 318, total state size = 49.692 MiB (draft: 0.000 MiB)
5.01.132.861 I srv load: - looking for better prompt, base f_keep = 0.220, sim = 0.722
5.01.132.873 I srv update: - cache state: 16 prompts, 1293.704 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
5.01.132.874 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
5.01.132.875 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
5.01.132.876 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
5.01.132.876 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
5.01.132.877 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
5.01.132.877 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
5.01.132.878 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
5.01.132.878 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
5.01.132.879 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
5.01.132.879 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
5.01.132.880 I srv update: - prompt 0x5db83777bb60: 488 tokens, checkpoints: 0, 76.257 MiB
5.01.132.880 I srv update: - prompt 0x5db83764fbd0: 364 tokens, checkpoints: 0, 56.880 MiB
5.01.132.881 I srv update: - prompt 0x5db83766b910: 529 tokens, checkpoints: 0, 82.663 MiB
5.01.132.882 I srv update: - prompt 0x5db8376780b0: 316 tokens, checkpoints: 0, 49.380 MiB
5.01.132.883 I srv update: - prompt 0x5db83766bcf0: 485 tokens, checkpoints: 0, 75.788 MiB
5.01.132.884 I srv update: - prompt 0x5db837687e30: 318 tokens, checkpoints: 0, 49.692 MiB
5.01.132.886 I srv get_availabl: prompt cache update took 38.15 ms
5.01.132.922 I slot launch_slot_: id 3 | task 6791 | processing task, is_child = 0
5.01.132.923 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.01.132.924 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.01.132.936 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.01.132.937 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.01.132.945 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.01.132.945 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.05.470.721 I slot print_timing: id 3 | task 6791 | n_decoded = 100, tg = 24.47 t/s, tg_3s = 24.47 t/s
5.08.485.577 I slot print_timing: id 3 | task 6791 | n_decoded = 173, tg = 24.36 t/s, tg_3s = 24.21 t/s
5.09.521.511 I slot print_timing: id 3 | task 6791 | prompt eval time = 251.64 ms / 27 tokens ( 9.32 ms per token, 107.30 tokens per second)
5.09.521.514 I slot print_timing: id 3 | task 6791 | eval time = 8136.90 ms / 198 tokens ( 41.10 ms per token, 24.33 tokens per second)
5.09.521.515 I slot print_timing: id 3 | task 6791 | total time = 8388.54 ms / 225 tokens
5.09.521.515 I slot print_timing: id 3 | task 6791 | graphs reused = 0
5.09.521.541 I slot release: id 3 | task 6791 | stop processing: n_tokens = 294, truncated = 0
5.09.521.549 I srv update_slots: all slots are idle
5.09.524.078 I srv operator(): Chat format: peg-native
5.09.524.183 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.814 (> 0.100 thold), f_keep = 0.238
5.09.524.186 I srv get_availabl: updating prompt cache
5.09.524.221 W srv prompt_save: - saving prompt with length 294, total state size = 45.942 MiB (draft: 0.000 MiB)
5.09.560.301 I srv load: - looking for better prompt, base f_keep = 0.238, sim = 0.814
5.09.560.314 I srv update: - cache state: 17 prompts, 1339.645 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
5.09.560.314 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
5.09.560.315 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
5.09.560.316 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
5.09.560.316 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
5.09.560.317 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
5.09.560.317 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
5.09.560.318 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
5.09.560.318 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
5.09.560.319 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
5.09.560.319 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
5.09.560.320 I srv update: - prompt 0x5db83777bb60: 488 tokens, checkpoints: 0, 76.257 MiB
5.09.560.320 I srv update: - prompt 0x5db83764fbd0: 364 tokens, checkpoints: 0, 56.880 MiB
5.09.560.321 I srv update: - prompt 0x5db83766b910: 529 tokens, checkpoints: 0, 82.663 MiB
5.09.560.321 I srv update: - prompt 0x5db8376780b0: 316 tokens, checkpoints: 0, 49.380 MiB
5.09.560.322 I srv update: - prompt 0x5db83766bcf0: 485 tokens, checkpoints: 0, 75.788 MiB
5.09.560.323 I srv update: - prompt 0x5db837687e30: 318 tokens, checkpoints: 0, 49.692 MiB
5.09.560.324 I srv update: - prompt 0x5db837790720: 294 tokens, checkpoints: 0, 45.942 MiB
5.09.560.325 I srv get_availabl: prompt cache update took 36.14 ms
5.09.560.367 I slot launch_slot_: id 3 | task 6990 | processing task, is_child = 0
5.09.560.367 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.09.560.369 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.09.560.381 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.09.560.382 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.09.560.391 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.09.560.391 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.13.780.421 I slot print_timing: id 3 | task 6990 | n_decoded = 100, tg = 24.42 t/s, tg_3s = 24.42 t/s
5.16.798.620 I slot print_timing: id 3 | task 6990 | n_decoded = 173, tg = 24.32 t/s, tg_3s = 24.19 t/s
5.18.376.199 I slot print_timing: id 3 | task 6990 | prompt eval time = 124.74 ms / 16 tokens ( 7.80 ms per token, 128.27 tokens per second)
5.18.376.202 I slot print_timing: id 3 | task 6990 | eval time = 8691.05 ms / 211 tokens ( 41.19 ms per token, 24.28 tokens per second)
5.18.376.203 I slot print_timing: id 3 | task 6990 | total time = 8815.78 ms / 227 tokens
5.18.376.204 I slot print_timing: id 3 | task 6990 | graphs reused = 0
5.18.376.222 I slot release: id 3 | task 6990 | stop processing: n_tokens = 296, truncated = 0
5.18.376.229 I srv update_slots: all slots are idle
5.18.379.379 I srv operator(): Chat format: peg-native
5.18.379.487 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.648 (> 0.100 thold), f_keep = 0.236
5.18.379.489 I srv get_availabl: updating prompt cache
5.18.379.513 W srv prompt_save: - saving prompt with length 296, total state size = 46.254 MiB (draft: 0.000 MiB)
5.18.415.230 I srv load: - looking for better prompt, base f_keep = 0.236, sim = 0.648
5.18.415.242 I srv update: - cache state: 18 prompts, 1385.900 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
5.18.415.243 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
5.18.415.245 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
5.18.415.250 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
5.18.415.250 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
5.18.415.251 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
5.18.415.252 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
5.18.415.252 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
5.18.415.254 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
5.18.415.255 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
5.18.415.255 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
5.18.415.256 I srv update: - prompt 0x5db83777bb60: 488 tokens, checkpoints: 0, 76.257 MiB
5.18.415.256 I srv update: - prompt 0x5db83764fbd0: 364 tokens, checkpoints: 0, 56.880 MiB
5.18.415.257 I srv update: - prompt 0x5db83766b910: 529 tokens, checkpoints: 0, 82.663 MiB
5.18.415.257 I srv update: - prompt 0x5db8376780b0: 316 tokens, checkpoints: 0, 49.380 MiB
5.18.415.258 I srv update: - prompt 0x5db83766bcf0: 485 tokens, checkpoints: 0, 75.788 MiB
5.18.415.258 I srv update: - prompt 0x5db837687e30: 318 tokens, checkpoints: 0, 49.692 MiB
5.18.415.259 I srv update: - prompt 0x5db837790720: 294 tokens, checkpoints: 0, 45.942 MiB
5.18.415.259 I srv update: - prompt 0x5db8374fd050: 296 tokens, checkpoints: 0, 46.254 MiB
5.18.415.260 I srv get_availabl: prompt cache update took 35.77 ms
5.18.415.299 I slot launch_slot_: id 3 | task 7202 | processing task, is_child = 0
5.18.415.300 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.18.415.301 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.18.415.314 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.18.415.314 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.18.415.323 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.18.415.323 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.22.797.683 I slot print_timing: id 3 | task 7202 | n_decoded = 100, tg = 24.40 t/s, tg_3s = 24.40 t/s
5.25.410.282 I slot print_timing: id 3 | task 7202 | prompt eval time = 283.15 ms / 38 tokens ( 7.45 ms per token, 134.20 tokens per second)
5.25.410.285 I slot print_timing: id 3 | task 7202 | eval time = 6711.78 ms / 163 tokens ( 41.18 ms per token, 24.29 tokens per second)
5.25.410.285 I slot print_timing: id 3 | task 7202 | total time = 6994.93 ms / 201 tokens
5.25.410.286 I slot print_timing: id 3 | task 7202 | graphs reused = 0
5.25.410.307 I slot release: id 3 | task 7202 | stop processing: n_tokens = 270, truncated = 0
5.25.410.314 I srv update_slots: all slots are idle
5.25.412.716 I srv operator(): Chat format: peg-native
5.25.412.827 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.729 (> 0.100 thold), f_keep = 0.259
5.25.412.828 I srv get_availabl: updating prompt cache
5.25.412.857 W srv prompt_save: - saving prompt with length 270, total state size = 42.192 MiB (draft: 0.000 MiB)
5.25.445.580 I srv load: - looking for better prompt, base f_keep = 0.259, sim = 0.729
5.25.445.593 I srv update: - cache state: 19 prompts, 1428.091 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
5.25.445.593 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
5.25.445.594 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
5.25.445.595 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
5.25.445.596 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
5.25.445.600 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
5.25.445.600 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
5.25.445.601 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
5.25.445.601 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
5.25.445.602 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
5.25.445.602 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
5.25.445.604 I srv update: - prompt 0x5db83777bb60: 488 tokens, checkpoints: 0, 76.257 MiB
5.25.445.605 I srv update: - prompt 0x5db83764fbd0: 364 tokens, checkpoints: 0, 56.880 MiB
5.25.445.605 I srv update: - prompt 0x5db83766b910: 529 tokens, checkpoints: 0, 82.663 MiB
5.25.445.606 I srv update: - prompt 0x5db8376780b0: 316 tokens, checkpoints: 0, 49.380 MiB
5.25.445.606 I srv update: - prompt 0x5db83766bcf0: 485 tokens, checkpoints: 0, 75.788 MiB
5.25.445.607 I srv update: - prompt 0x5db837687e30: 318 tokens, checkpoints: 0, 49.692 MiB
5.25.445.607 I srv update: - prompt 0x5db837790720: 294 tokens, checkpoints: 0, 45.942 MiB
5.25.445.608 I srv update: - prompt 0x5db8374fd050: 296 tokens, checkpoints: 0, 46.254 MiB
5.25.445.608 I srv update: - prompt 0x5db8344de990: 270 tokens, checkpoints: 0, 42.192 MiB
5.25.445.609 I srv get_availabl: prompt cache update took 32.78 ms
5.25.445.649 I slot launch_slot_: id 3 | task 7366 | processing task, is_child = 0
5.25.445.651 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.25.445.652 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.25.445.666 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.25.445.666 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.25.445.674 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.25.445.675 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.29.799.692 I slot print_timing: id 3 | task 7366 | n_decoded = 100, tg = 24.38 t/s, tg_3s = 24.38 t/s
5.32.835.381 I slot print_timing: id 3 | task 7366 | n_decoded = 173, tg = 24.24 t/s, tg_3s = 24.05 t/s
5.35.835.607 I slot print_timing: id 3 | task 7366 | n_decoded = 245, tg = 24.17 t/s, tg_3s = 24.00 t/s
5.38.835.874 I slot print_timing: id 3 | task 7366 | n_decoded = 317, tg = 24.13 t/s, tg_3s = 24.00 t/s
5.41.840.731 I slot print_timing: id 3 | task 7366 | n_decoded = 389, tg = 24.10 t/s, tg_3s = 23.96 t/s
5.44.309.261 I slot print_timing: id 3 | task 7366 | prompt eval time = 251.53 ms / 26 tokens ( 9.67 ms per token, 103.37 tokens per second)
5.44.309.265 I slot print_timing: id 3 | task 7366 | eval time = 18612.04 ms / 448 tokens ( 41.54 ms per token, 24.07 tokens per second)
5.44.309.266 I slot print_timing: id 3 | task 7366 | total time = 18863.56 ms / 474 tokens
5.44.309.267 I slot print_timing: id 3 | task 7366 | graphs reused = 0
5.44.309.288 I slot release: id 3 | task 7366 | stop processing: n_tokens = 543, truncated = 0
5.44.309.297 I srv update_slots: all slots are idle
5.44.311.965 I srv operator(): Chat format: peg-native
5.44.312.083 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.673 (> 0.100 thold), f_keep = 0.129
5.44.312.085 I srv get_availabl: updating prompt cache
5.44.312.124 W srv prompt_save: - saving prompt with length 543, total state size = 84.851 MiB (draft: 0.000 MiB)
5.44.395.741 I srv load: - looking for better prompt, base f_keep = 0.129, sim = 0.673
5.44.395.751 I srv update: - cache state: 20 prompts, 1512.942 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
5.44.395.752 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
5.44.395.753 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
5.44.395.754 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
5.44.395.757 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
5.44.395.758 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
5.44.395.758 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
5.44.395.759 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
5.44.395.759 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
5.44.395.760 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
5.44.395.763 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
5.44.395.763 I srv update: - prompt 0x5db83777bb60: 488 tokens, checkpoints: 0, 76.257 MiB
5.44.395.764 I srv update: - prompt 0x5db83764fbd0: 364 tokens, checkpoints: 0, 56.880 MiB
5.44.395.764 I srv update: - prompt 0x5db83766b910: 529 tokens, checkpoints: 0, 82.663 MiB
5.44.395.765 I srv update: - prompt 0x5db8376780b0: 316 tokens, checkpoints: 0, 49.380 MiB
5.44.395.765 I srv update: - prompt 0x5db83766bcf0: 485 tokens, checkpoints: 0, 75.788 MiB
5.44.395.768 I srv update: - prompt 0x5db837687e30: 318 tokens, checkpoints: 0, 49.692 MiB
5.44.395.768 I srv update: - prompt 0x5db837790720: 294 tokens, checkpoints: 0, 45.942 MiB
5.44.395.769 I srv update: - prompt 0x5db8374fd050: 296 tokens, checkpoints: 0, 46.254 MiB
5.44.395.769 I srv update: - prompt 0x5db8344de990: 270 tokens, checkpoints: 0, 42.192 MiB
5.44.395.770 I srv update: - prompt 0x5db837699a10: 543 tokens, checkpoints: 0, 84.851 MiB
5.44.395.771 I srv get_availabl: prompt cache update took 83.69 ms
5.44.395.796 I slot launch_slot_: id 3 | task 7815 | processing task, is_child = 0
5.44.395.797 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.44.395.798 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.44.395.809 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.44.395.809 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.44.395.831 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.44.395.831 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.48.799.319 I slot print_timing: id 3 | task 7815 | n_decoded = 100, tg = 24.27 t/s, tg_3s = 24.27 t/s
5.51.822.998 I slot print_timing: id 3 | task 7815 | n_decoded = 173, tg = 24.22 t/s, tg_3s = 24.14 t/s
5.54.855.345 I slot print_timing: id 3 | task 7815 | n_decoded = 246, tg = 24.17 t/s, tg_3s = 24.07 t/s
5.55.270.323 I slot print_timing: id 3 | task 7815 | prompt eval time = 283.32 ms / 34 tokens ( 8.33 ms per token, 120.01 tokens per second)
5.55.270.327 I slot print_timing: id 3 | task 7815 | eval time = 10591.15 ms / 256 tokens ( 41.37 ms per token, 24.17 tokens per second)
5.55.270.328 I slot print_timing: id 3 | task 7815 | total time = 10874.47 ms / 290 tokens
5.55.270.329 I slot print_timing: id 3 | task 7815 | graphs reused = 0
5.55.270.355 I slot release: id 3 | task 7815 | stop processing: n_tokens = 359, truncated = 0
5.55.270.364 I srv update_slots: all slots are idle
5.55.272.992 I srv operator(): Chat format: peg-native
5.55.273.102 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.636 (> 0.100 thold), f_keep = 0.195
5.55.273.104 I srv get_availabl: updating prompt cache
5.55.273.148 W srv prompt_save: - saving prompt with length 359, total state size = 56.099 MiB (draft: 0.000 MiB)
5.55.315.724 I srv load: - looking for better prompt, base f_keep = 0.195, sim = 0.636
5.55.315.741 I srv update: - cache state: 21 prompts, 1569.041 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
5.55.315.742 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
5.55.315.743 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
5.55.315.744 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
5.55.315.745 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
5.55.315.746 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
5.55.315.746 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
5.55.315.747 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
5.55.315.748 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
5.55.315.748 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
5.55.315.749 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
5.55.315.749 I srv update: - prompt 0x5db83777bb60: 488 tokens, checkpoints: 0, 76.257 MiB
5.55.315.750 I srv update: - prompt 0x5db83764fbd0: 364 tokens, checkpoints: 0, 56.880 MiB
5.55.315.751 I srv update: - prompt 0x5db83766b910: 529 tokens, checkpoints: 0, 82.663 MiB
5.55.315.751 I srv update: - prompt 0x5db8376780b0: 316 tokens, checkpoints: 0, 49.380 MiB
5.55.315.752 I srv update: - prompt 0x5db83766bcf0: 485 tokens, checkpoints: 0, 75.788 MiB
5.55.315.752 I srv update: - prompt 0x5db837687e30: 318 tokens, checkpoints: 0, 49.692 MiB
5.55.315.753 I srv update: - prompt 0x5db837790720: 294 tokens, checkpoints: 0, 45.942 MiB
5.55.315.753 I srv update: - prompt 0x5db8374fd050: 296 tokens, checkpoints: 0, 46.254 MiB
5.55.315.754 I srv update: - prompt 0x5db8344de990: 270 tokens, checkpoints: 0, 42.192 MiB
5.55.315.755 I srv update: - prompt 0x5db837699a10: 543 tokens, checkpoints: 0, 84.851 MiB
5.55.315.755 I srv update: - prompt 0x5db837699510: 359 tokens, checkpoints: 0, 56.099 MiB
5.55.315.757 I srv get_availabl: prompt cache update took 42.65 ms
5.55.315.801 I slot launch_slot_: id 3 | task 8072 | processing task, is_child = 0
5.55.315.802 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
5.55.315.803 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
5.55.315.817 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
5.55.315.818 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
5.55.315.828 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
5.55.315.828 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
5.59.703.317 I slot print_timing: id 3 | task 8072 | n_decoded = 100, tg = 24.37 t/s, tg_3s = 24.37 t/s
6.02.729.027 I slot print_timing: id 3 | task 8072 | n_decoded = 173, tg = 24.26 t/s, tg_3s = 24.13 t/s
6.03.351.503 I slot print_timing: id 3 | task 8072 | prompt eval time = 283.50 ms / 40 tokens ( 7.09 ms per token, 141.09 tokens per second)
6.03.351.507 I slot print_timing: id 3 | task 8072 | eval time = 7752.15 ms / 188 tokens ( 41.23 ms per token, 24.25 tokens per second)
6.03.351.507 I slot print_timing: id 3 | task 8072 | total time = 8035.65 ms / 228 tokens
6.03.351.508 I slot print_timing: id 3 | task 8072 | graphs reused = 0
6.03.351.533 I slot release: id 3 | task 8072 | stop processing: n_tokens = 297, truncated = 0
6.03.351.542 I srv update_slots: all slots are idle
6.03.354.020 I srv operator(): Chat format: peg-native
6.03.354.123 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.737 (> 0.100 thold), f_keep = 0.236
6.03.354.125 I srv get_availabl: updating prompt cache
6.03.354.163 W srv prompt_save: - saving prompt with length 297, total state size = 46.411 MiB (draft: 0.000 MiB)
6.03.389.688 I srv load: - looking for better prompt, base f_keep = 0.236, sim = 0.737
6.03.389.702 I srv update: - cache state: 22 prompts, 1615.451 MiB (limits: 8192.000 MiB, 16384 tokens, 52424 est)
6.03.389.703 I srv update: - prompt 0x5db837666250: 802 tokens, checkpoints: 0, 125.323 MiB
6.03.389.704 I srv update: - prompt 0x5db83777e670: 201 tokens, checkpoints: 0, 31.409 MiB
6.03.389.705 I srv update: - prompt 0x5db83776d8b0: 656 tokens, checkpoints: 0, 102.508 MiB
6.03.389.706 I srv update: - prompt 0x5db83790e290: 1119 tokens, checkpoints: 0, 174.857 MiB
6.03.389.706 I srv update: - prompt 0x5db837124ea0: 297 tokens, checkpoints: 0, 46.411 MiB
6.03.389.707 I srv update: - prompt 0x5db8374fd570: 451 tokens, checkpoints: 0, 70.475 MiB
6.03.389.708 I srv update: - prompt 0x5db8377811c0: 666 tokens, checkpoints: 0, 104.071 MiB
6.03.389.709 I srv update: - prompt 0x5db837773c00: 550 tokens, checkpoints: 0, 85.945 MiB
6.03.389.711 I srv update: - prompt 0x5db83769a160: 477 tokens, checkpoints: 0, 74.538 MiB
6.03.389.712 I srv update: - prompt 0x5db83776a310: 560 tokens, checkpoints: 0, 87.507 MiB
6.03.389.712 I srv update: - prompt 0x5db83777bb60: 488 tokens, checkpoints: 0, 76.257 MiB
6.03.389.713 I srv update: - prompt 0x5db83764fbd0: 364 tokens, checkpoints: 0, 56.880 MiB
6.03.389.714 I srv update: - prompt 0x5db83766b910: 529 tokens, checkpoints: 0, 82.663 MiB
6.03.389.714 I srv update: - prompt 0x5db8376780b0: 316 tokens, checkpoints: 0, 49.380 MiB
6.03.389.715 I srv update: - prompt 0x5db83766bcf0: 485 tokens, checkpoints: 0, 75.788 MiB
6.03.389.715 I srv update: - prompt 0x5db837687e30: 318 tokens, checkpoints: 0, 49.692 MiB
6.03.389.716 I srv update: - prompt 0x5db837790720: 294 tokens, checkpoints: 0, 45.942 MiB
6.03.389.716 I srv update: - prompt 0x5db8374fd050: 296 tokens, checkpoints: 0, 46.254 MiB
6.03.389.717 I srv update: - prompt 0x5db8344de990: 270 tokens, checkpoints: 0, 42.192 MiB
6.03.389.718 I srv update: - prompt 0x5db837699a10: 543 tokens, checkpoints: 0, 84.851 MiB
6.03.389.718 I srv update: - prompt 0x5db837699510: 359 tokens, checkpoints: 0, 56.099 MiB
6.03.389.719 I srv update: - prompt 0x5db8371b2450: 297 tokens, checkpoints: 0, 46.411 MiB
6.03.389.720 I srv get_availabl: prompt cache update took 35.59 ms
6.03.389.762 I slot launch_slot_: id 3 | task 8261 | processing task, is_child = 0
6.03.389.763 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
6.03.389.764 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
6.03.389.779 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
6.03.389.779 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
6.03.389.789 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
6.03.389.790 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
6.07.749.432 I slot print_timing: id 3 | task 8261 | n_decoded = 100, tg = 24.34 t/s, tg_3s = 24.34 t/s
6.10.779.464 I slot print_timing: id 3 | task 8261 | n_decoded = 173, tg = 24.24 t/s, tg_3s = 24.09 t/s
6.13.815.171 I slot print_timing: id 3 | task 8261 | n_decoded = 246, tg = 24.18 t/s, tg_3s = 24.05 t/s
6.16.816.504 I slot print_timing: id 3 | task 8261 | n_decoded = 318, tg = 24.14 t/s, tg_3s = 23.99 t/s
6.19.817.118 I slot print_timing: id 3 | task 8261 | n_decoded = 390, tg = 24.11 t/s, tg_3s = 24.00 t/s
6.21.863.920 I slot print_timing: id 3 | task 8261 | prompt eval time = 251.50 ms / 25 tokens ( 10.06 ms per token, 99.40 tokens per second)
6.21.863.924 I slot print_timing: id 3 | task 8261 | eval time = 18222.60 ms / 439 tokens ( 41.51 ms per token, 24.09 tokens per second)
6.21.863.925 I slot print_timing: id 3 | task 8261 | total time = 18474.11 ms / 464 tokens
6.21.863.926 I slot print_timing: id 3 | task 8261 | graphs reused = 0
6.21.863.951 I slot release: id 3 | task 8261 | stop processing: n_tokens = 533, truncated = 0
6.21.863.969 I srv update_slots: all slots are idle
6.21.864.511 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink