qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results_agentic/qwen3.5-9b.server.log

415 lines
40 KiB
Plaintext
Raw Permalink Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.067.699 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.067.703 I device_info:
0.00.067.722 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.067.725 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.067.732 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.067.785 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.067.788 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.067.826 I srv init: running without SSL
0.00.067.886 I srv init: using 23 threads for HTTP server
0.00.068.216 I srv start: binding port with default address family
0.00.069.405 I srv llama_server: loading model
0.00.069.413 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/Qwen_Qwen3.5-9B-Q4_K_M.gguf'
0.00.069.464 I common_init_result: fitting params to device memory ...
0.00.069.466 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.02.972.121 W llama_context: n_ctx_seq (32768) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.03.018.560 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.03.221.400 I srv load_model: initializing slots, n_slots = 4
0.03.279.012 W srv load_model: speculative decoding will use checkpoints
0.03.279.033 W common_speculative_init: no implementations specified for speculative decoding
0.03.279.033 I slot load_model: id 0 | task -1 | new slot, n_ctx = 32768
0.03.279.040 I slot load_model: id 1 | task -1 | new slot, n_ctx = 32768
0.03.279.040 I slot load_model: id 2 | task -1 | new slot, n_ctx = 32768
0.03.279.041 I slot load_model: id 3 | task -1 | new slot, n_ctx = 32768
0.03.279.090 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.03.279.090 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.03.279.090 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.03.279.090 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.03.279.108 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.03.297.129 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
<think>
'
0.03.310.923 I srv init: init: chat template, thinking = 1
0.03.310.964 I srv llama_server: model loaded
0.03.310.968 I srv llama_server: server is listening on http://127.0.0.1:8090
0.03.310.974 I srv update_slots: all slots are idle
0.04.979.330 I srv operator(): Chat format: peg-native
0.04.979.984 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.04.979.988 I srv get_availabl: updating prompt cache
0.04.979.995 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.04.980.000 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 32768 tokens, 8589934592 est)
0.04.980.001 I srv get_availabl: prompt cache update took 0.01 ms
0.04.980.079 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.04.980.081 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.04.980.082 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.04.980.110 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.04.980.110 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.04.980.129 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.04.980.131 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.05.893.195 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 298, pos_max = 298, n_tokens = 299, size = 50.251 MiB)
0.09.125.599 I slot print_timing: id 3 | task 0 | n_decoded = 143, tg = 47.60 t/s, tg_3s = 47.60 t/s
0.11.731.374 I slot print_timing: id 3 | task 0 | prompt eval time = 1141.09 ms / 406 tokens ( 2.81 ms per token, 355.80 tokens per second)
0.11.731.378 I slot print_timing: id 3 | task 0 | eval time = 5610.12 ms / 267 tokens ( 21.01 ms per token, 47.59 tokens per second)
0.11.731.378 I slot print_timing: id 3 | task 0 | total time = 6751.20 ms / 673 tokens
0.11.731.383 I slot print_timing: id 3 | task 0 | graphs reused = 265
0.11.731.444 I slot release: id 3 | task 0 | stop processing: n_tokens = 672, truncated = 0
0.11.731.449 I srv update_slots: all slots are idle
0.11.749.297 I srv operator(): Chat format: peg-native
0.11.750.258 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.694 (> 0.100 thold), f_keep = 0.601
0.11.750.315 I slot launch_slot_: id 3 | task 270 | processing task, is_child = 0
0.11.750.317 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.11.750.317 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.11.750.336 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.11.750.337 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.11.750.353 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.11.750.354 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.11.750.373 I slot update_slots: id 3 | task 270 | Checking checkpoint with [298, 298] against 404...
0.11.763.430 W slot update_slots: id 3 | task 270 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
0.12.186.717 I slot create_check: id 3 | task 270 | created context checkpoint 2 of 32 (pos_min = 555, pos_max = 555, n_tokens = 556, size = 50.251 MiB)
0.13.145.457 I slot print_timing: id 3 | task 270 | prompt eval time = 549.04 ms / 283 tokens ( 1.94 ms per token, 515.44 tokens per second)
0.13.145.461 I slot print_timing: id 3 | task 270 | eval time = 846.03 ms / 41 tokens ( 20.63 ms per token, 48.46 tokens per second)
0.13.145.462 I slot print_timing: id 3 | task 270 | total time = 1395.08 ms / 324 tokens
0.13.145.463 I slot print_timing: id 3 | task 270 | graphs reused = 304
0.13.145.515 I slot release: id 3 | task 270 | stop processing: n_tokens = 622, truncated = 0
0.13.145.522 I srv update_slots: all slots are idle
0.13.178.440 I srv operator(): Chat format: peg-native
0.13.179.458 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.859 (> 0.100 thold), f_keep = 0.932
0.13.179.494 I slot launch_slot_: id 3 | task 314 | processing task, is_child = 0
0.13.179.495 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.13.179.495 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.13.179.515 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.13.179.516 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.13.179.533 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.13.179.533 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.13.179.554 I slot update_slots: id 3 | task 314 | Checking checkpoint with [555, 555] against 580...
0.13.192.544 W slot update_slots: id 3 | task 314 | restored context checkpoint (pos_min = 555, pos_max = 555, n_tokens = 556, n_past = 556, size = 50.251 MiB)
0.15.171.432 I slot print_timing: id 3 | task 314 | prompt eval time = 280.91 ms / 119 tokens ( 2.36 ms per token, 423.62 tokens per second)
0.15.171.436 I slot print_timing: id 3 | task 314 | eval time = 1710.96 ms / 82 tokens ( 20.87 ms per token, 47.93 tokens per second)
0.15.171.437 I slot print_timing: id 3 | task 314 | total time = 1991.87 ms / 201 tokens
0.15.171.437 I slot print_timing: id 3 | task 314 | graphs reused = 384
0.15.171.516 I slot release: id 3 | task 314 | stop processing: n_tokens = 756, truncated = 0
0.15.171.526 I srv update_slots: all slots are idle
0.15.204.681 I srv operator(): Chat format: peg-native
0.15.205.300 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.790 (> 0.100 thold), f_keep = 0.402
0.15.205.302 I srv get_availabl: updating prompt cache
0.15.205.365 W srv prompt_save: - saving prompt with length 756, total state size = 73.890 MiB (draft: 0.000 MiB)
0.15.316.032 I srv load: - looking for better prompt, base f_keep = 0.402, sim = 0.790
0.15.316.039 I srv update: - cache state: 1 prompts, 174.391 MiB (limits: 8192.000 MiB, 32768 tokens, 35512 est)
0.15.316.040 I srv update: - prompt 0x57a73ba161f0: 756 tokens, checkpoints: 2, 174.391 MiB
0.15.316.042 I srv get_availabl: prompt cache update took 110.74 ms
0.15.316.122 I slot launch_slot_: id 3 | task 399 | processing task, is_child = 0
0.15.316.125 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.15.316.126 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.15.316.159 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.15.316.161 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.15.316.204 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.15.316.206 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.15.316.257 I slot update_slots: id 3 | task 399 | Checking checkpoint with [555, 555] against 304...
0.15.316.259 I slot update_slots: id 3 | task 399 | Checking checkpoint with [298, 298] against 304...
0.15.329.129 W slot update_slots: id 3 | task 399 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
0.15.329.132 W slot update_slots: id 3 | task 399 | erased invalidated context checkpoint (pos_min = 555, pos_max = 555, n_tokens = 556, n_swa = 0, pos_next = 299, size = 50.251 MiB)
0.17.155.599 I slot print_timing: id 3 | task 399 | prompt eval time = 217.20 ms / 86 tokens ( 2.53 ms per token, 395.94 tokens per second)
0.17.155.602 I slot print_timing: id 3 | task 399 | eval time = 1622.13 ms / 78 tokens ( 20.80 ms per token, 48.08 tokens per second)
0.17.155.603 I slot print_timing: id 3 | task 399 | total time = 1839.34 ms / 164 tokens
0.17.155.604 I slot print_timing: id 3 | task 399 | graphs reused = 460
0.17.155.647 I slot release: id 3 | task 399 | stop processing: n_tokens = 462, truncated = 0
0.17.155.657 I srv update_slots: all slots are idle
0.17.173.305 I srv operator(): Chat format: peg-native
0.17.174.090 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.760 (> 0.100 thold), f_keep = 0.829
0.17.174.131 I slot launch_slot_: id 3 | task 479 | processing task, is_child = 0
0.17.174.136 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.17.174.136 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.17.174.160 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.17.174.161 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.17.174.202 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.17.174.203 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.17.174.223 I slot update_slots: id 3 | task 479 | Checking checkpoint with [298, 298] against 383...
0.17.187.221 W slot update_slots: id 3 | task 479 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
0.20.541.636 I slot print_timing: id 3 | task 479 | n_decoded = 144, tg = 47.79 t/s, tg_3s = 47.79 t/s
0.22.032.840 I slot print_timing: id 3 | task 479 | prompt eval time = 354.49 ms / 205 tokens ( 1.73 ms per token, 578.30 tokens per second)
0.22.032.844 I slot print_timing: id 3 | task 479 | eval time = 4504.12 ms / 215 tokens ( 20.95 ms per token, 47.73 tokens per second)
0.22.032.845 I slot print_timing: id 3 | task 479 | total time = 4858.61 ms / 420 tokens
0.22.032.845 I slot print_timing: id 3 | task 479 | graphs reused = 672
0.22.032.896 I slot release: id 3 | task 479 | stop processing: n_tokens = 718, truncated = 0
0.22.032.907 I srv update_slots: all slots are idle
0.22.050.768 I srv operator(): Chat format: peg-native
0.22.051.785 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.783 (> 0.100 thold), f_keep = 0.699
0.22.051.823 I slot launch_slot_: id 3 | task 697 | processing task, is_child = 0
0.22.051.824 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.22.051.825 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.22.051.846 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.22.051.847 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.22.051.864 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.22.051.864 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.22.051.884 I slot update_slots: id 3 | task 697 | Checking checkpoint with [298, 298] against 502...
0.22.064.997 W slot update_slots: id 3 | task 697 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
0.22.539.769 I slot create_check: id 3 | task 697 | created context checkpoint 2 of 32 (pos_min = 615, pos_max = 615, n_tokens = 616, size = 50.251 MiB)
0.23.477.547 I slot print_timing: id 3 | task 697 | prompt eval time = 600.62 ms / 342 tokens ( 1.76 ms per token, 569.41 tokens per second)
0.23.477.550 I slot print_timing: id 3 | task 697 | eval time = 825.03 ms / 40 tokens ( 20.63 ms per token, 48.48 tokens per second)
0.23.477.550 I slot print_timing: id 3 | task 697 | total time = 1425.66 ms / 382 tokens
0.23.477.551 I slot print_timing: id 3 | task 697 | graphs reused = 710
0.23.477.607 I slot release: id 3 | task 697 | stop processing: n_tokens = 680, truncated = 0
0.23.477.617 I srv update_slots: all slots are idle
0.23.510.833 I srv operator(): Chat format: peg-native
0.23.511.904 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.931 (> 0.100 thold), f_keep = 0.940
0.23.511.964 I slot launch_slot_: id 3 | task 740 | processing task, is_child = 0
0.23.511.966 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.23.511.966 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.23.511.987 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.23.511.987 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.23.512.004 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.23.512.006 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.23.512.026 I slot update_slots: id 3 | task 740 | Checking checkpoint with [615, 615] against 639...
0.23.525.000 W slot update_slots: id 3 | task 740 | restored context checkpoint (pos_min = 615, pos_max = 615, n_tokens = 616, n_past = 616, size = 50.251 MiB)
0.25.793.294 I slot print_timing: id 3 | task 740 | prompt eval time = 183.03 ms / 70 tokens ( 2.61 ms per token, 382.46 tokens per second)
0.25.793.298 I slot print_timing: id 3 | task 740 | eval time = 2098.24 ms / 100 tokens ( 20.98 ms per token, 47.66 tokens per second)
0.25.793.299 I slot print_timing: id 3 | task 740 | total time = 2281.26 ms / 170 tokens
0.25.793.300 I slot print_timing: id 3 | task 740 | graphs reused = 807
0.25.793.357 I slot release: id 3 | task 740 | stop processing: n_tokens = 785, truncated = 0
0.25.793.365 I srv update_slots: all slots are idle
0.25.825.855 I srv operator(): Chat format: peg-native
0.25.826.470 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.770 (> 0.100 thold), f_keep = 0.387
0.25.826.471 I srv get_availabl: updating prompt cache
0.25.826.530 W srv prompt_save: - saving prompt with length 785, total state size = 74.797 MiB (draft: 0.000 MiB)
0.25.943.997 I srv load: - looking for better prompt, base f_keep = 0.387, sim = 0.770
0.25.944.004 I srv update: - cache state: 2 prompts, 349.689 MiB (limits: 8192.000 MiB, 32768 tokens, 36100 est)
0.25.944.005 I srv update: - prompt 0x57a73ba161f0: 756 tokens, checkpoints: 2, 174.391 MiB
0.25.944.006 I srv update: - prompt 0x57a742652520: 785 tokens, checkpoints: 2, 175.298 MiB
0.25.944.007 I srv get_availabl: prompt cache update took 117.54 ms
0.25.944.083 I slot launch_slot_: id 3 | task 843 | processing task, is_child = 0
0.25.944.088 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.25.944.089 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.25.944.110 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.25.944.111 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.25.944.151 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.25.944.152 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.25.944.178 I slot update_slots: id 3 | task 843 | Checking checkpoint with [615, 615] against 304...
0.25.944.180 I slot update_slots: id 3 | task 843 | Checking checkpoint with [298, 298] against 304...
0.25.956.985 W slot update_slots: id 3 | task 843 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
0.25.956.988 W slot update_slots: id 3 | task 843 | erased invalidated context checkpoint (pos_min = 615, pos_max = 615, n_tokens = 616, n_swa = 0, pos_next = 299, size = 50.251 MiB)
0.29.167.066 I slot print_timing: id 3 | task 843 | n_decoded = 143, tg = 47.61 t/s, tg_3s = 47.61 t/s
0.29.379.294 I slot print_timing: id 3 | task 843 | prompt eval time = 219.31 ms / 96 tokens ( 2.28 ms per token, 437.74 tokens per second)
0.29.379.297 I slot print_timing: id 3 | task 843 | eval time = 3215.81 ms / 153 tokens ( 21.02 ms per token, 47.58 tokens per second)
0.29.379.298 I slot print_timing: id 3 | task 843 | total time = 3435.11 ms / 249 tokens
0.29.379.298 I slot print_timing: id 3 | task 843 | graphs reused = 957
0.29.379.347 I slot release: id 3 | task 843 | stop processing: n_tokens = 547, truncated = 0
0.29.379.355 I srv update_slots: all slots are idle
0.29.396.526 I srv operator(): Chat format: peg-native
0.29.397.294 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.938 (> 0.100 thold), f_keep = 0.718
0.29.397.337 I slot launch_slot_: id 3 | task 998 | processing task, is_child = 0
0.29.397.339 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.397.340 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.397.365 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.397.365 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.397.386 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.397.387 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.29.397.411 I slot update_slots: id 3 | task 998 | Checking checkpoint with [298, 298] against 393...
0.29.410.606 W slot update_slots: id 3 | task 998 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
0.30.481.351 I slot print_timing: id 3 | task 998 | prompt eval time = 254.19 ms / 120 tokens ( 2.12 ms per token, 472.10 tokens per second)
0.30.481.355 I slot print_timing: id 3 | task 998 | eval time = 829.75 ms / 40 tokens ( 20.74 ms per token, 48.21 tokens per second)
0.30.481.356 I slot print_timing: id 3 | task 998 | total time = 1083.93 ms / 160 tokens
0.30.481.356 I slot print_timing: id 3 | task 998 | graphs reused = 995
0.30.481.400 I slot release: id 3 | task 998 | stop processing: n_tokens = 458, truncated = 0
0.30.481.408 I srv update_slots: all slots are idle
0.30.514.243 I srv operator(): Chat format: peg-native
0.30.515.057 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.907 (> 0.100 thold), f_keep = 0.910
0.30.515.103 I slot launch_slot_: id 3 | task 1041 | processing task, is_child = 0
0.30.515.105 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.30.515.105 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.30.515.127 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.30.515.127 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.30.515.144 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.30.515.144 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.30.515.165 I slot update_slots: id 3 | task 1041 | Checking checkpoint with [298, 298] against 417...
0.30.528.460 W slot update_slots: id 3 | task 1041 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
0.32.628.372 I slot print_timing: id 3 | task 1041 | prompt eval time = 292.87 ms / 161 tokens ( 1.82 ms per token, 549.74 tokens per second)
0.32.628.376 I slot print_timing: id 3 | task 1041 | eval time = 1820.33 ms / 87 tokens ( 20.92 ms per token, 47.79 tokens per second)
0.32.628.377 I slot print_timing: id 3 | task 1041 | total time = 2113.20 ms / 248 tokens
0.32.628.378 I slot print_timing: id 3 | task 1041 | graphs reused = 1079
0.32.628.425 I slot release: id 3 | task 1041 | stop processing: n_tokens = 546, truncated = 0
0.32.628.432 I srv update_slots: all slots are idle
0.32.646.663 I srv operator(): Chat format: peg-native
0.32.647.571 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.851 (> 0.100 thold), f_keep = 0.839
0.32.647.626 I slot launch_slot_: id 3 | task 1131 | processing task, is_child = 0
0.32.647.628 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.32.647.628 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.32.647.655 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.32.647.656 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.32.647.677 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.32.647.677 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.32.647.702 I slot update_slots: id 3 | task 1131 | Checking checkpoint with [298, 298] against 458...
0.32.660.797 W slot update_slots: id 3 | task 1131 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
0.33.960.978 I slot print_timing: id 3 | task 1131 | prompt eval time = 420.61 ms / 239 tokens ( 1.76 ms per token, 568.23 tokens per second)
0.33.960.981 I slot print_timing: id 3 | task 1131 | eval time = 892.66 ms / 43 tokens ( 20.76 ms per token, 48.17 tokens per second)
0.33.960.982 I slot print_timing: id 3 | task 1131 | total time = 1313.27 ms / 282 tokens
0.33.960.983 I slot print_timing: id 3 | task 1131 | graphs reused = 1120
0.33.961.035 I slot release: id 3 | task 1131 | stop processing: n_tokens = 580, truncated = 0
0.33.961.043 I srv update_slots: all slots are idle
0.33.994.462 I srv operator(): Chat format: peg-native
0.33.995.397 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.919 (> 0.100 thold), f_keep = 0.924
0.33.995.452 I slot launch_slot_: id 3 | task 1177 | processing task, is_child = 0
0.33.995.454 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.33.995.455 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.33.995.475 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.33.995.475 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.33.995.516 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.33.995.516 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.33.995.537 I slot update_slots: id 3 | task 1177 | Checking checkpoint with [298, 298] against 536...
0.34.008.706 W slot update_slots: id 3 | task 1177 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
0.34.467.495 I slot create_check: id 3 | task 1177 | created context checkpoint 2 of 32 (pos_min = 578, pos_max = 578, n_tokens = 579, size = 50.251 MiB)
0.37.363.471 I slot print_timing: id 3 | task 1177 | prompt eval time = 518.74 ms / 284 tokens ( 1.83 ms per token, 547.48 tokens per second)
0.37.363.475 I slot print_timing: id 3 | task 1177 | eval time = 2849.18 ms / 136 tokens ( 20.95 ms per token, 47.73 tokens per second)
0.37.363.475 I slot print_timing: id 3 | task 1177 | total time = 3367.93 ms / 420 tokens
0.37.363.476 I slot print_timing: id 3 | task 1177 | graphs reused = 1254
0.37.363.529 I slot release: id 3 | task 1177 | stop processing: n_tokens = 718, truncated = 0
0.37.363.536 I srv update_slots: all slots are idle
0.37.395.057 I srv operator(): Chat format: peg-native
0.37.395.661 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.724 (> 0.100 thold), f_keep = 0.423
0.37.395.662 I srv get_availabl: updating prompt cache
0.37.395.719 W srv prompt_save: - saving prompt with length 718, total state size = 72.702 MiB (draft: 0.000 MiB)
0.37.510.284 I srv load: - looking for better prompt, base f_keep = 0.423, sim = 0.724
0.37.510.291 I srv update: - cache state: 3 prompts, 522.893 MiB (limits: 8192.000 MiB, 32768 tokens, 35391 est)
0.37.510.292 I srv update: - prompt 0x57a73ba161f0: 756 tokens, checkpoints: 2, 174.391 MiB
0.37.510.293 I srv update: - prompt 0x57a742652520: 785 tokens, checkpoints: 2, 175.298 MiB
0.37.510.293 I srv update: - prompt 0x57a731e2c5c0: 718 tokens, checkpoints: 2, 173.203 MiB
0.37.510.299 I srv get_availabl: prompt cache update took 114.64 ms
0.37.510.371 I slot launch_slot_: id 3 | task 1316 | processing task, is_child = 0
0.37.510.373 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.37.510.374 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.37.510.394 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.37.510.394 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.37.510.411 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.37.510.412 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.37.510.436 I slot update_slots: id 3 | task 1316 | Checking checkpoint with [578, 578] against 304...
0.37.510.437 I slot update_slots: id 3 | task 1316 | Checking checkpoint with [298, 298] against 304...
0.37.523.298 W slot update_slots: id 3 | task 1316 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
0.37.523.301 W slot update_slots: id 3 | task 1316 | erased invalidated context checkpoint (pos_min = 578, pos_max = 578, n_tokens = 579, n_swa = 0, pos_next = 299, size = 50.251 MiB)
0.40.800.035 I slot print_timing: id 3 | task 1316 | n_decoded = 144, tg = 47.67 t/s, tg_3s = 47.67 t/s
0.43.812.467 I slot print_timing: id 3 | task 1316 | n_decoded = 287, tg = 47.57 t/s, tg_3s = 47.47 t/s
0.46.820.104 I slot print_timing: id 3 | task 1316 | n_decoded = 429, tg = 47.45 t/s, tg_3s = 47.21 t/s
0.49.255.544 I slot print_timing: id 3 | task 1316 | prompt eval time = 269.05 ms / 121 tokens ( 2.22 ms per token, 449.74 tokens per second)
0.49.255.548 I slot print_timing: id 3 | task 1316 | eval time = 11476.06 ms / 544 tokens ( 21.10 ms per token, 47.40 tokens per second)
0.49.255.549 I slot print_timing: id 3 | task 1316 | total time = 11745.10 ms / 665 tokens
0.49.255.550 I slot print_timing: id 3 | task 1316 | graphs reused = 1794
0.49.255.595 I slot release: id 3 | task 1316 | stop processing: n_tokens = 963, truncated = 0
0.49.255.605 I srv update_slots: all slots are idle
0.49.272.964 I srv operator(): Chat format: peg-native
0.49.273.852 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.736 (> 0.100 thold), f_keep = 0.434
0.49.273.853 I srv get_availabl: updating prompt cache
0.49.273.915 W srv prompt_save: - saving prompt with length 963, total state size = 80.363 MiB (draft: 0.000 MiB)
0.49.363.611 I srv load: - looking for better prompt, base f_keep = 0.434, sim = 0.736
0.49.363.620 I srv update: - cache state: 4 prompts, 653.506 MiB (limits: 8192.000 MiB, 32768 tokens, 40389 est)
0.49.363.621 I srv update: - prompt 0x57a73ba161f0: 756 tokens, checkpoints: 2, 174.391 MiB
0.49.363.622 I srv update: - prompt 0x57a742652520: 785 tokens, checkpoints: 2, 175.298 MiB
0.49.363.623 I srv update: - prompt 0x57a731e2c5c0: 718 tokens, checkpoints: 2, 173.203 MiB
0.49.363.624 I srv update: - prompt 0x57a73b7cf130: 963 tokens, checkpoints: 1, 130.613 MiB
0.49.363.625 I srv get_availabl: prompt cache update took 89.77 ms
0.49.363.696 I slot launch_slot_: id 3 | task 1862 | processing task, is_child = 0
0.49.363.698 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.49.363.699 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.49.363.721 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.49.363.721 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.49.363.739 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.49.363.739 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.49.363.764 I slot update_slots: id 3 | task 1862 | Checking checkpoint with [298, 298] against 418...
0.49.376.723 W slot update_slots: id 3 | task 1862 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
0.49.821.146 I slot create_check: id 3 | task 1862 | created context checkpoint 2 of 32 (pos_min = 563, pos_max = 563, n_tokens = 564, size = 50.251 MiB)
0.50.905.570 I slot print_timing: id 3 | task 1862 | prompt eval time = 504.24 ms / 269 tokens ( 1.87 ms per token, 533.48 tokens per second)
0.50.905.574 I slot print_timing: id 3 | task 1862 | eval time = 1037.57 ms / 50 tokens ( 20.75 ms per token, 48.19 tokens per second)
0.50.905.574 I slot print_timing: id 3 | task 1862 | total time = 1541.80 ms / 319 tokens
0.50.905.575 I slot print_timing: id 3 | task 1862 | graphs reused = 1842
0.50.905.628 I slot release: id 3 | task 1862 | stop processing: n_tokens = 617, truncated = 0
0.50.905.636 I srv update_slots: all slots are idle
0.50.937.832 I srv operator(): Chat format: peg-native
0.50.938.776 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.913 (> 0.100 thold), f_keep = 0.917
0.50.938.834 I slot launch_slot_: id 3 | task 1915 | processing task, is_child = 0
0.50.938.836 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.50.938.837 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.50.938.858 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.50.938.859 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.50.938.876 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.50.938.876 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.50.938.896 I slot update_slots: id 3 | task 1915 | Checking checkpoint with [563, 563] against 566...
0.50.951.947 W slot update_slots: id 3 | task 1915 | restored context checkpoint (pos_min = 563, pos_max = 563, n_tokens = 564, n_past = 564, size = 50.251 MiB)
0.53.067.610 I slot print_timing: id 3 | task 1915 | prompt eval time = 191.43 ms / 56 tokens ( 3.42 ms per token, 292.53 tokens per second)
0.53.067.614 I slot print_timing: id 3 | task 1915 | eval time = 1937.28 ms / 93 tokens ( 20.83 ms per token, 48.01 tokens per second)
0.53.067.615 I slot print_timing: id 3 | task 1915 | total time = 2128.71 ms / 149 tokens
0.53.067.615 I slot print_timing: id 3 | task 1915 | graphs reused = 1933
0.53.067.673 I slot release: id 3 | task 1915 | stop processing: n_tokens = 712, truncated = 0
0.53.067.683 I srv update_slots: all slots are idle
0.53.099.225 I srv operator(): Chat format: peg-native
0.53.099.839 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.727 (> 0.100 thold), f_keep = 0.430
0.53.099.842 I srv get_availabl: updating prompt cache
0.53.099.901 W srv prompt_save: - saving prompt with length 712, total state size = 72.514 MiB (draft: 0.000 MiB)
0.53.207.815 I srv load: - looking for better prompt, base f_keep = 0.430, sim = 0.727
0.53.207.826 I srv update: - cache state: 5 prompts, 826.522 MiB (limits: 8192.000 MiB, 32768 tokens, 38991 est)
0.53.207.827 I srv update: - prompt 0x57a73ba161f0: 756 tokens, checkpoints: 2, 174.391 MiB
0.53.207.828 I srv update: - prompt 0x57a742652520: 785 tokens, checkpoints: 2, 175.298 MiB
0.53.207.828 I srv update: - prompt 0x57a731e2c5c0: 718 tokens, checkpoints: 2, 173.203 MiB
0.53.207.829 I srv update: - prompt 0x57a73b7cf130: 963 tokens, checkpoints: 1, 130.613 MiB
0.53.207.830 I srv update: - prompt 0x57a73b36a190: 712 tokens, checkpoints: 2, 173.016 MiB
0.53.207.831 I srv get_availabl: prompt cache update took 107.99 ms
0.53.207.910 I slot launch_slot_: id 3 | task 2011 | processing task, is_child = 0
0.53.207.912 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.53.207.913 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.53.207.940 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.53.207.940 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.53.207.969 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.53.207.971 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.53.207.997 I slot update_slots: id 3 | task 2011 | Checking checkpoint with [563, 563] against 306...
0.53.207.999 I slot update_slots: id 3 | task 2011 | Checking checkpoint with [298, 298] against 306...
0.53.220.934 W slot update_slots: id 3 | task 2011 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
0.53.220.937 W slot update_slots: id 3 | task 2011 | erased invalidated context checkpoint (pos_min = 563, pos_max = 563, n_tokens = 564, n_swa = 0, pos_next = 299, size = 50.251 MiB)
0.56.493.578 I slot print_timing: id 3 | task 2011 | n_decoded = 144, tg = 47.78 t/s, tg_3s = 47.78 t/s
0.59.496.803 I slot print_timing: id 3 | task 2011 | n_decoded = 287, tg = 47.70 t/s, tg_3s = 47.62 t/s
1.00.126.661 I slot print_timing: id 3 | task 2011 | prompt eval time = 271.75 ms / 122 tokens ( 2.23 ms per token, 448.95 tokens per second)
1.00.126.665 I slot print_timing: id 3 | task 2011 | eval time = 6646.91 ms / 317 tokens ( 20.97 ms per token, 47.69 tokens per second)
1.00.126.665 I slot print_timing: id 3 | task 2011 | total time = 6918.66 ms / 439 tokens
1.00.126.666 I slot print_timing: id 3 | task 2011 | graphs reused = 2247
1.00.126.718 I slot release: id 3 | task 2011 | stop processing: n_tokens = 737, truncated = 0
1.00.126.727 I srv update_slots: all slots are idle
1.00.144.408 I srv operator(): Chat format: peg-native
1.00.145.465 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.675 (> 0.100 thold), f_keep = 0.569
1.00.145.500 I slot launch_slot_: id 3 | task 2330 | processing task, is_child = 0
1.00.145.501 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.00.145.502 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.00.145.523 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.00.145.523 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.00.145.540 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.00.145.540 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.00.145.560 I slot update_slots: id 3 | task 2330 | Checking checkpoint with [298, 298] against 419...
1.00.158.680 W slot update_slots: id 3 | task 2330 | restored context checkpoint (pos_min = 298, pos_max = 298, n_tokens = 299, n_past = 299, size = 50.251 MiB)
1.00.625.837 I slot create_check: id 3 | task 2330 | created context checkpoint 2 of 32 (pos_min = 595, pos_max = 595, n_tokens = 596, size = 50.251 MiB)
1.01.541.592 I slot print_timing: id 3 | task 2330 | prompt eval time = 593.04 ms / 322 tokens ( 1.84 ms per token, 542.97 tokens per second)
1.01.541.595 I slot print_timing: id 3 | task 2330 | eval time = 802.99 ms / 39 tokens ( 20.59 ms per token, 48.57 tokens per second)
1.01.541.596 I slot print_timing: id 3 | task 2330 | total time = 1396.03 ms / 361 tokens
1.01.541.597 I slot print_timing: id 3 | task 2330 | graphs reused = 2284
1.01.541.656 I slot release: id 3 | task 2330 | stop processing: n_tokens = 659, truncated = 0
1.01.541.665 I srv update_slots: all slots are idle
1.01.574.671 I srv operator(): Chat format: peg-native
1.01.575.791 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.929 (> 0.100 thold), f_keep = 0.939
1.01.575.837 I slot launch_slot_: id 3 | task 2372 | processing task, is_child = 0
1.01.575.839 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.01.575.839 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.01.575.861 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.01.575.862 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.01.575.879 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.01.575.879 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.01.575.900 I slot update_slots: id 3 | task 2372 | Checking checkpoint with [595, 595] against 619...
1.01.589.228 W slot update_slots: id 3 | task 2372 | restored context checkpoint (pos_min = 595, pos_max = 595, n_tokens = 596, n_past = 596, size = 50.251 MiB)
1.03.549.637 I slot print_timing: id 3 | task 2372 | prompt eval time = 183.23 ms / 70 tokens ( 2.62 ms per token, 382.03 tokens per second)
1.03.549.641 I slot print_timing: id 3 | task 2372 | eval time = 1790.50 ms / 86 tokens ( 20.82 ms per token, 48.03 tokens per second)
1.03.549.642 I slot print_timing: id 3 | task 2372 | total time = 1973.73 ms / 156 tokens
1.03.549.643 I slot print_timing: id 3 | task 2372 | graphs reused = 2368
1.03.549.698 I slot release: id 3 | task 2372 | stop processing: n_tokens = 751, truncated = 0
1.03.549.705 I srv update_slots: all slots are idle
1.03.566.734 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink