1007 lines
100 KiB
Plaintext
1007 lines
100 KiB
Plaintext
0.00.064.572 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
|
|
0.00.064.576 I device_info:
|
|
0.00.064.596 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
|
0.00.064.599 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
|
0.00.064.606 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
|
|
0.00.064.666 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
|
0.00.064.669 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
|
|
0.00.064.707 I srv init: running without SSL
|
|
0.00.064.768 I srv init: using 23 threads for HTTP server
|
|
0.00.065.087 I srv start: binding port with default address family
|
|
0.00.066.309 I srv llama_server: loading model
|
|
0.00.066.319 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/google_gemma-4-26B-A4B-it-Q4_K_M.gguf'
|
|
0.00.066.377 I common_init_result: fitting params to device memory ...
|
|
0.00.066.378 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
|
|
0.01.829.838 W load: control-looking token: 212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
|
|
0.01.830.356 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
|
|
0.01.857.065 W load: special_eog_ids contains '<|tool_response>', removing '</s>' token from EOG list
|
|
0.12.655.572 W llama_context: n_ctx_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
|
|
0.12.743.247 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
|
0.12.921.194 I srv load_model: initializing slots, n_slots = 4
|
|
0.12.962.599 W common_speculative_init: no implementations specified for speculative decoding
|
|
0.12.962.604 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384
|
|
0.12.962.607 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384
|
|
0.12.962.608 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384
|
|
0.12.962.608 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384
|
|
0.12.962.660 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
|
|
0.12.962.660 I srv load_model: use `--cache-ram 0` to disable the prompt cache
|
|
0.12.962.661 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
|
0.12.962.661 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
|
|
0.12.962.681 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
|
|
0.12.970.244 I init: chat template, example_format: '<|turn>system
|
|
<|think|>
|
|
You are a helpful assistant<turn|>
|
|
<|turn>user
|
|
Hello<turn|>
|
|
<|turn>model
|
|
Hi there<turn|>
|
|
<|turn>user
|
|
How are you?<turn|>
|
|
<|turn>model
|
|
'
|
|
0.12.970.998 I srv init: init: chat template, thinking = 1
|
|
0.12.971.031 I srv llama_server: model loaded
|
|
0.12.971.035 I srv llama_server: server is listening on http://127.0.0.1:8090
|
|
0.12.971.041 I srv update_slots: all slots are idle
|
|
0.14.966.240 I srv operator(): Chat format: peg-gemma4
|
|
0.14.966.649 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
|
|
0.14.966.651 I srv get_availabl: updating prompt cache
|
|
0.14.966.657 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
|
0.14.966.661 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est)
|
|
0.14.966.662 I srv get_availabl: prompt cache update took 0.01 ms
|
|
0.14.966.698 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
|
|
0.14.966.699 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.14.966.700 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.14.966.718 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.14.966.718 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.14.966.732 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.14.966.732 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.15.636.404 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 72, n_tokens = 73, size = 14.259 MiB)
|
|
0.18.758.182 I slot print_timing: id 3 | task 0 | n_decoded = 167, tg = 55.61 t/s, tg_3s = 55.61 t/s
|
|
0.21.764.210 I slot print_timing: id 3 | task 0 | n_decoded = 334, tg = 55.58 t/s, tg_3s = 55.56 t/s
|
|
0.24.766.360 I slot print_timing: id 3 | task 0 | n_decoded = 500, tg = 55.49 t/s, tg_3s = 55.29 t/s
|
|
0.27.773.469 I slot print_timing: id 3 | task 0 | n_decoded = 667, tg = 55.50 t/s, tg_3s = 55.54 t/s
|
|
0.30.780.373 I slot print_timing: id 3 | task 0 | n_decoded = 831, tg = 55.31 t/s, tg_3s = 54.54 t/s
|
|
0.33.785.506 I slot print_timing: id 3 | task 0 | n_decoded = 995, tg = 55.18 t/s, tg_3s = 54.57 t/s
|
|
0.36.790.395 I slot print_timing: id 3 | task 0 | n_decoded = 1159, tg = 55.10 t/s, tg_3s = 54.58 t/s
|
|
0.39.802.230 I slot print_timing: id 3 | task 0 | n_decoded = 1321, tg = 54.93 t/s, tg_3s = 53.79 t/s
|
|
0.42.805.778 I slot print_timing: id 3 | task 0 | n_decoded = 1484, tg = 54.86 t/s, tg_3s = 54.27 t/s
|
|
0.45.821.272 I slot print_timing: id 3 | task 0 | n_decoded = 1647, tg = 54.78 t/s, tg_3s = 54.05 t/s
|
|
0.46.135.818 I slot print_timing: id 3 | task 0 | prompt eval time = 788.24 ms / 99 tokens ( 7.96 ms per token, 125.60 tokens per second)
|
|
0.46.135.822 I slot print_timing: id 3 | task 0 | eval time = 30380.81 ms / 1664 tokens ( 18.26 ms per token, 54.77 tokens per second)
|
|
0.46.135.823 I slot print_timing: id 3 | task 0 | total time = 31169.06 ms / 1763 tokens
|
|
0.46.135.828 I slot print_timing: id 3 | task 0 | graphs reused = 1657
|
|
0.46.135.879 I slot release: id 3 | task 0 | stop processing: n_tokens = 1762, truncated = 0
|
|
0.46.135.889 I srv update_slots: all slots are idle
|
|
0.46.138.433 I srv operator(): Chat format: peg-gemma4
|
|
0.46.138.916 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.817 (> 0.100 thold), f_keep = 0.043
|
|
0.46.138.918 I srv get_availabl: updating prompt cache
|
|
0.46.139.041 W srv prompt_save: - saving prompt with length 1762, total state size = 234.447 MiB (draft: 0.000 MiB)
|
|
0.46.373.211 I srv load: - looking for better prompt, base f_keep = 0.043, sim = 0.817
|
|
0.46.373.218 I srv update: - cache state: 1 prompts, 248.706 MiB (limits: 8192.000 MiB, 16384 tokens, 58037 est)
|
|
0.46.373.219 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
0.46.373.220 I srv get_availabl: prompt cache update took 234.30 ms
|
|
0.46.373.294 I slot launch_slot_: id 3 | task 1667 | processing task, is_child = 0
|
|
0.46.373.296 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.46.373.297 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.46.373.328 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.46.373.329 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.46.373.352 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.46.373.352 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.46.373.384 I slot update_slots: id 3 | task 1667 | Checking checkpoint with [0, 72] against 0...
|
|
0.46.379.683 W slot update_slots: id 3 | task 1667 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
0.49.485.886 I slot print_timing: id 3 | task 1667 | n_decoded = 168, tg = 55.88 t/s, tg_3s = 55.88 t/s
|
|
0.52.491.826 I slot print_timing: id 3 | task 1667 | n_decoded = 335, tg = 55.72 t/s, tg_3s = 55.56 t/s
|
|
0.55.492.963 I slot print_timing: id 3 | task 1667 | n_decoded = 501, tg = 55.58 t/s, tg_3s = 55.31 t/s
|
|
0.58.504.993 I slot print_timing: id 3 | task 1667 | n_decoded = 668, tg = 55.55 t/s, tg_3s = 55.44 t/s
|
|
1.01.510.237 I slot print_timing: id 3 | task 1667 | n_decoded = 833, tg = 55.42 t/s, tg_3s = 54.90 t/s
|
|
1.04.528.216 I slot print_timing: id 3 | task 1667 | n_decoded = 998, tg = 55.29 t/s, tg_3s = 54.67 t/s
|
|
1.07.535.268 I slot print_timing: id 3 | task 1667 | n_decoded = 1162, tg = 55.19 t/s, tg_3s = 54.54 t/s
|
|
1.07.608.714 I slot print_timing: id 3 | task 1667 | prompt eval time = 105.84 ms / 21 tokens ( 5.04 ms per token, 198.41 tokens per second)
|
|
1.07.608.717 I slot print_timing: id 3 | task 1667 | eval time = 21129.49 ms / 1166 tokens ( 18.12 ms per token, 55.18 tokens per second)
|
|
1.07.608.717 I slot print_timing: id 3 | task 1667 | total time = 21235.33 ms / 1187 tokens
|
|
1.07.608.718 I slot print_timing: id 3 | task 1667 | graphs reused = 2817
|
|
1.07.608.753 I slot release: id 3 | task 1667 | stop processing: n_tokens = 1258, truncated = 0
|
|
1.07.608.764 I srv update_slots: all slots are idle
|
|
1.07.611.231 I srv operator(): Chat format: peg-gemma4
|
|
1.07.611.559 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.809 (> 0.100 thold), f_keep = 0.060
|
|
1.07.611.561 I srv get_availabl: updating prompt cache
|
|
1.07.611.655 W srv prompt_save: - saving prompt with length 1258, total state size = 224.597 MiB (draft: 0.000 MiB)
|
|
1.07.909.302 I srv load: - looking for better prompt, base f_keep = 0.060, sim = 0.809
|
|
1.07.909.309 I srv update: - cache state: 2 prompts, 487.562 MiB (limits: 8192.000 MiB, 16384 tokens, 50741 est)
|
|
1.07.909.310 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
1.07.909.311 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
1.07.909.312 I srv get_availabl: prompt cache update took 297.75 ms
|
|
1.07.909.380 I slot launch_slot_: id 3 | task 2836 | processing task, is_child = 0
|
|
1.07.909.382 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.07.909.383 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.07.909.402 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.07.909.403 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.07.909.414 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
1.07.909.415 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
1.07.909.434 I slot update_slots: id 3 | task 2836 | Checking checkpoint with [0, 72] against 0...
|
|
1.07.915.591 W slot update_slots: id 3 | task 2836 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
1.11.047.841 I slot print_timing: id 3 | task 2836 | n_decoded = 167, tg = 55.54 t/s, tg_3s = 55.54 t/s
|
|
1.14.054.078 I slot print_timing: id 3 | task 2836 | n_decoded = 334, tg = 55.54 t/s, tg_3s = 55.55 t/s
|
|
1.17.071.187 I slot print_timing: id 3 | task 2836 | n_decoded = 501, tg = 55.48 t/s, tg_3s = 55.35 t/s
|
|
1.20.086.197 I slot print_timing: id 3 | task 2836 | n_decoded = 668, tg = 55.46 t/s, tg_3s = 55.39 t/s
|
|
1.20.404.643 I slot print_timing: id 3 | task 2836 | prompt eval time = 131.31 ms / 22 tokens ( 5.97 ms per token, 167.54 tokens per second)
|
|
1.20.404.646 I slot print_timing: id 3 | task 2836 | eval time = 12363.88 ms / 685 tokens ( 18.05 ms per token, 55.40 tokens per second)
|
|
1.20.404.646 I slot print_timing: id 3 | task 2836 | total time = 12495.19 ms / 707 tokens
|
|
1.20.404.647 I slot print_timing: id 3 | task 2836 | graphs reused = 3497
|
|
1.20.404.687 I slot release: id 3 | task 2836 | stop processing: n_tokens = 778, truncated = 0
|
|
1.20.404.696 I srv update_slots: all slots are idle
|
|
1.20.407.174 I srv operator(): Chat format: peg-gemma4
|
|
1.20.407.505 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.817 (> 0.100 thold), f_keep = 0.098
|
|
1.20.407.507 I srv get_availabl: updating prompt cache
|
|
1.20.407.587 W srv prompt_save: - saving prompt with length 778, total state size = 167.167 MiB (draft: 0.000 MiB)
|
|
1.20.581.736 I srv load: - looking for better prompt, base f_keep = 0.098, sim = 0.817
|
|
1.20.581.743 I srv update: - cache state: 3 prompts, 668.989 MiB (limits: 8192.000 MiB, 16384 tokens, 46507 est)
|
|
1.20.581.744 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
1.20.581.752 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
1.20.581.753 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
1.20.581.754 I srv get_availabl: prompt cache update took 174.25 ms
|
|
1.20.581.815 I slot launch_slot_: id 3 | task 3524 | processing task, is_child = 0
|
|
1.20.581.817 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.20.581.818 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.20.581.836 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.20.581.836 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.20.581.848 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
1.20.581.848 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
1.20.581.867 I slot update_slots: id 3 | task 3524 | Checking checkpoint with [0, 72] against 0...
|
|
1.20.587.826 W slot update_slots: id 3 | task 3524 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
1.23.692.107 I slot print_timing: id 3 | task 3524 | n_decoded = 167, tg = 55.58 t/s, tg_3s = 55.58 t/s
|
|
1.26.699.160 I slot print_timing: id 3 | task 3524 | n_decoded = 334, tg = 55.56 t/s, tg_3s = 55.54 t/s
|
|
1.29.706.589 I slot print_timing: id 3 | task 3524 | n_decoded = 500, tg = 55.44 t/s, tg_3s = 55.20 t/s
|
|
1.32.720.647 I slot print_timing: id 3 | task 3524 | n_decoded = 666, tg = 55.35 t/s, tg_3s = 55.08 t/s
|
|
1.35.726.141 I slot print_timing: id 3 | task 3524 | n_decoded = 831, tg = 55.26 t/s, tg_3s = 54.90 t/s
|
|
1.38.736.365 I slot print_timing: id 3 | task 3524 | n_decoded = 997, tg = 55.24 t/s, tg_3s = 55.15 t/s
|
|
1.41.740.356 I slot print_timing: id 3 | task 3524 | n_decoded = 1162, tg = 55.19 t/s, tg_3s = 54.93 t/s
|
|
1.44.755.889 I slot print_timing: id 3 | task 3524 | n_decoded = 1326, tg = 55.09 t/s, tg_3s = 54.39 t/s
|
|
1.47.767.075 I slot print_timing: id 3 | task 3524 | n_decoded = 1489, tg = 54.99 t/s, tg_3s = 54.13 t/s
|
|
1.49.469.461 I slot print_timing: id 3 | task 3524 | prompt eval time = 105.50 ms / 21 tokens ( 5.02 ms per token, 199.05 tokens per second)
|
|
1.49.469.464 I slot print_timing: id 3 | task 3524 | eval time = 28782.09 ms / 1581 tokens ( 18.20 ms per token, 54.93 tokens per second)
|
|
1.49.469.465 I slot print_timing: id 3 | task 3524 | total time = 28887.59 ms / 1602 tokens
|
|
1.49.469.465 I slot print_timing: id 3 | task 3524 | graphs reused = 5070
|
|
1.49.469.502 I slot release: id 3 | task 3524 | stop processing: n_tokens = 1673, truncated = 0
|
|
1.49.469.515 I srv update_slots: all slots are idle
|
|
1.49.472.225 I srv operator(): Chat format: peg-gemma4
|
|
1.49.472.559 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.717 (> 0.100 thold), f_keep = 0.045
|
|
1.49.472.562 I srv get_availabl: updating prompt cache
|
|
1.49.472.697 W srv prompt_save: - saving prompt with length 1673, total state size = 232.707 MiB (draft: 0.000 MiB)
|
|
1.49.714.319 I srv load: - looking for better prompt, base f_keep = 0.045, sim = 0.717
|
|
1.49.714.327 I srv update: - cache state: 4 prompts, 915.955 MiB (limits: 8192.000 MiB, 16384 tokens, 48930 est)
|
|
1.49.714.328 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
1.49.714.328 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
1.49.714.330 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
1.49.714.331 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
1.49.714.332 I srv get_availabl: prompt cache update took 241.77 ms
|
|
1.49.714.399 I slot launch_slot_: id 3 | task 5108 | processing task, is_child = 0
|
|
1.49.714.400 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.49.714.401 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.49.714.421 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.49.714.422 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.49.714.434 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
1.49.714.434 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
1.49.714.453 I slot update_slots: id 3 | task 5108 | Checking checkpoint with [0, 72] against 0...
|
|
1.49.720.682 W slot update_slots: id 3 | task 5108 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
1.52.873.037 I slot print_timing: id 3 | task 5108 | n_decoded = 167, tg = 55.59 t/s, tg_3s = 55.59 t/s
|
|
1.55.883.069 I slot print_timing: id 3 | task 5108 | n_decoded = 334, tg = 55.54 t/s, tg_3s = 55.48 t/s
|
|
1.58.892.255 I slot print_timing: id 3 | task 5108 | n_decoded = 500, tg = 55.41 t/s, tg_3s = 55.16 t/s
|
|
2.01.909.140 I slot print_timing: id 3 | task 5108 | n_decoded = 667, tg = 55.40 t/s, tg_3s = 55.36 t/s
|
|
2.04.922.260 I slot print_timing: id 3 | task 5108 | n_decoded = 832, tg = 55.27 t/s, tg_3s = 54.76 t/s
|
|
2.07.925.877 I slot print_timing: id 3 | task 5108 | n_decoded = 995, tg = 55.10 t/s, tg_3s = 54.27 t/s
|
|
2.10.941.325 I slot print_timing: id 3 | task 5108 | n_decoded = 1158, tg = 54.95 t/s, tg_3s = 54.05 t/s
|
|
2.13.946.450 I slot print_timing: id 3 | task 5108 | n_decoded = 1320, tg = 54.82 t/s, tg_3s = 53.91 t/s
|
|
2.16.958.985 I slot print_timing: id 3 | task 5108 | n_decoded = 1481, tg = 54.67 t/s, tg_3s = 53.44 t/s
|
|
2.19.974.658 I slot print_timing: id 3 | task 5108 | n_decoded = 1640, tg = 54.48 t/s, tg_3s = 52.72 t/s
|
|
2.22.983.208 I slot print_timing: id 3 | task 5108 | n_decoded = 1799, tg = 54.33 t/s, tg_3s = 52.85 t/s
|
|
2.23.822.178 I slot print_timing: id 3 | task 5108 | prompt eval time = 154.65 ms / 34 tokens ( 4.55 ms per token, 219.85 tokens per second)
|
|
2.23.822.181 I slot print_timing: id 3 | task 5108 | eval time = 33953.07 ms / 1843 tokens ( 18.42 ms per token, 54.28 tokens per second)
|
|
2.23.822.182 I slot print_timing: id 3 | task 5108 | total time = 34107.72 ms / 1877 tokens
|
|
2.23.822.183 I slot print_timing: id 3 | task 5108 | graphs reused = 6904
|
|
2.23.822.221 I slot release: id 3 | task 5108 | stop processing: n_tokens = 1948, truncated = 0
|
|
2.23.822.236 I srv update_slots: all slots are idle
|
|
2.23.824.849 I srv operator(): Chat format: peg-gemma4
|
|
2.23.825.175 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.800 (> 0.100 thold), f_keep = 0.039
|
|
2.23.825.177 I srv get_availabl: updating prompt cache
|
|
2.23.825.288 W srv prompt_save: - saving prompt with length 1948, total state size = 238.082 MiB (draft: 0.000 MiB)
|
|
2.24.046.129 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.800
|
|
2.24.046.138 I srv update: - cache state: 5 prompts, 1168.296 MiB (limits: 8192.000 MiB, 16384 tokens, 52021 est)
|
|
2.24.046.139 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
2.24.046.139 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
2.24.046.140 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
2.24.046.144 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
2.24.046.144 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
2.24.046.145 I srv get_availabl: prompt cache update took 220.97 ms
|
|
2.24.046.206 I slot launch_slot_: id 3 | task 6954 | processing task, is_child = 0
|
|
2.24.046.207 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.24.046.208 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.24.046.228 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.24.046.228 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.24.046.240 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.24.046.241 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.24.046.261 I slot update_slots: id 3 | task 6954 | Checking checkpoint with [0, 72] against 0...
|
|
2.24.052.455 W slot update_slots: id 3 | task 6954 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
2.27.180.519 I slot print_timing: id 3 | task 6954 | n_decoded = 168, tg = 55.97 t/s, tg_3s = 55.97 t/s
|
|
2.30.186.476 I slot print_timing: id 3 | task 6954 | n_decoded = 335, tg = 55.77 t/s, tg_3s = 55.56 t/s
|
|
2.33.203.892 I slot print_timing: id 3 | task 6954 | n_decoded = 502, tg = 55.62 t/s, tg_3s = 55.35 t/s
|
|
2.36.209.940 I slot print_timing: id 3 | task 6954 | n_decoded = 669, tg = 55.61 t/s, tg_3s = 55.55 t/s
|
|
2.39.218.084 I slot print_timing: id 3 | task 6954 | n_decoded = 834, tg = 55.46 t/s, tg_3s = 54.85 t/s
|
|
2.42.221.812 I slot print_timing: id 3 | task 6954 | n_decoded = 998, tg = 55.31 t/s, tg_3s = 54.60 t/s
|
|
2.45.228.613 I slot print_timing: id 3 | task 6954 | n_decoded = 1162, tg = 55.20 t/s, tg_3s = 54.54 t/s
|
|
2.46.935.189 I slot print_timing: id 3 | task 6954 | prompt eval time = 132.87 ms / 23 tokens ( 5.78 ms per token, 173.10 tokens per second)
|
|
2.46.935.193 I slot print_timing: id 3 | task 6954 | eval time = 22756.05 ms / 1254 tokens ( 18.15 ms per token, 55.11 tokens per second)
|
|
2.46.935.193 I slot print_timing: id 3 | task 6954 | total time = 22888.92 ms / 1277 tokens
|
|
2.46.935.194 I slot print_timing: id 3 | task 6954 | graphs reused = 8151
|
|
2.46.935.227 I slot release: id 3 | task 6954 | stop processing: n_tokens = 1348, truncated = 0
|
|
2.46.935.241 I srv update_slots: all slots are idle
|
|
2.46.937.915 I srv operator(): Chat format: peg-gemma4
|
|
2.46.938.361 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.846 (> 0.100 thold), f_keep = 0.057
|
|
2.46.938.363 I srv get_availabl: updating prompt cache
|
|
2.46.938.450 W srv prompt_save: - saving prompt with length 1348, total state size = 226.356 MiB (draft: 0.000 MiB)
|
|
2.47.127.907 I srv load: - looking for better prompt, base f_keep = 0.057, sim = 0.846
|
|
2.47.127.916 I srv update: - cache state: 6 prompts, 1408.911 MiB (limits: 8192.000 MiB, 16384 tokens, 50975 est)
|
|
2.47.127.917 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
2.47.127.920 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
2.47.127.921 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
2.47.127.921 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
2.47.127.922 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
2.47.127.922 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
2.47.127.924 I srv get_availabl: prompt cache update took 189.56 ms
|
|
2.47.127.989 I slot launch_slot_: id 3 | task 8211 | processing task, is_child = 0
|
|
2.47.127.990 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.47.127.991 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.47.128.012 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.47.128.012 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.47.128.035 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.47.128.035 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.47.128.055 I slot update_slots: id 3 | task 8211 | Checking checkpoint with [0, 72] against 0...
|
|
2.47.134.423 W slot update_slots: id 3 | task 8211 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
2.50.255.491 I slot print_timing: id 3 | task 8211 | n_decoded = 168, tg = 55.57 t/s, tg_3s = 55.57 t/s
|
|
2.53.264.630 I slot print_timing: id 3 | task 8211 | n_decoded = 335, tg = 55.53 t/s, tg_3s = 55.50 t/s
|
|
2.56.276.127 I slot print_timing: id 3 | task 8211 | n_decoded = 501, tg = 55.40 t/s, tg_3s = 55.12 t/s
|
|
2.59.288.777 I slot print_timing: id 3 | task 8211 | n_decoded = 667, tg = 55.32 t/s, tg_3s = 55.10 t/s
|
|
3.02.302.845 I slot print_timing: id 3 | task 8211 | n_decoded = 831, tg = 55.14 t/s, tg_3s = 54.41 t/s
|
|
3.05.311.956 I slot print_timing: id 3 | task 8211 | n_decoded = 994, tg = 54.98 t/s, tg_3s = 54.17 t/s
|
|
3.08.320.795 I slot print_timing: id 3 | task 8211 | n_decoded = 1157, tg = 54.86 t/s, tg_3s = 54.17 t/s
|
|
3.11.325.947 I slot print_timing: id 3 | task 8211 | n_decoded = 1318, tg = 54.70 t/s, tg_3s = 53.57 t/s
|
|
3.14.329.890 I slot print_timing: id 3 | task 8211 | n_decoded = 1479, tg = 54.58 t/s, tg_3s = 53.60 t/s
|
|
3.17.347.227 I slot print_timing: id 3 | task 8211 | n_decoded = 1642, tg = 54.52 t/s, tg_3s = 54.02 t/s
|
|
3.17.698.826 I slot print_timing: id 3 | task 8211 | prompt eval time = 104.22 ms / 19 tokens ( 5.49 ms per token, 182.30 tokens per second)
|
|
3.17.698.829 I slot print_timing: id 3 | task 8211 | eval time = 30466.55 ms / 1661 tokens ( 18.34 ms per token, 54.52 tokens per second)
|
|
3.17.698.830 I slot print_timing: id 3 | task 8211 | total time = 30570.77 ms / 1680 tokens
|
|
3.17.698.830 I slot print_timing: id 3 | task 8211 | graphs reused = 9804
|
|
3.17.698.864 I slot release: id 3 | task 8211 | stop processing: n_tokens = 1751, truncated = 0
|
|
3.17.698.877 I srv update_slots: all slots are idle
|
|
3.17.701.458 I srv operator(): Chat format: peg-gemma4
|
|
3.17.701.869 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.828 (> 0.100 thold), f_keep = 0.044
|
|
3.17.701.871 I srv get_availabl: updating prompt cache
|
|
3.17.701.990 W srv prompt_save: - saving prompt with length 1751, total state size = 234.232 MiB (draft: 0.000 MiB)
|
|
3.17.875.656 I srv load: - looking for better prompt, base f_keep = 0.044, sim = 0.828
|
|
3.17.875.665 I srv update: - cache state: 7 prompts, 1657.402 MiB (limits: 8192.000 MiB, 16384 tokens, 51987 est)
|
|
3.17.875.666 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
3.17.875.667 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
3.17.875.671 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
3.17.875.672 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
3.17.875.672 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
3.17.875.673 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
3.17.875.674 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
3.17.875.676 I srv get_availabl: prompt cache update took 173.81 ms
|
|
3.17.875.739 I slot launch_slot_: id 3 | task 9875 | processing task, is_child = 0
|
|
3.17.875.739 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.17.875.740 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.17.875.760 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.17.875.760 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.17.875.785 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.17.875.785 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.17.875.816 I slot update_slots: id 3 | task 9875 | Checking checkpoint with [0, 72] against 0...
|
|
3.17.882.012 W slot update_slots: id 3 | task 9875 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
3.20.993.863 I slot print_timing: id 3 | task 9875 | n_decoded = 169, tg = 56.10 t/s, tg_3s = 56.10 t/s
|
|
3.23.999.467 I slot print_timing: id 3 | task 9875 | n_decoded = 336, tg = 55.83 t/s, tg_3s = 55.56 t/s
|
|
3.27.016.030 I slot print_timing: id 3 | task 9875 | n_decoded = 503, tg = 55.68 t/s, tg_3s = 55.36 t/s
|
|
3.30.022.026 I slot print_timing: id 3 | task 9875 | n_decoded = 670, tg = 55.65 t/s, tg_3s = 55.56 t/s
|
|
3.33.024.188 I slot print_timing: id 3 | task 9875 | n_decoded = 834, tg = 55.44 t/s, tg_3s = 54.63 t/s
|
|
3.36.031.978 I slot print_timing: id 3 | task 9875 | n_decoded = 998, tg = 55.29 t/s, tg_3s = 54.53 t/s
|
|
3.38.822.017 I slot print_timing: id 3 | task 9875 | prompt eval time = 105.81 ms / 21 tokens ( 5.04 ms per token, 198.47 tokens per second)
|
|
3.38.822.021 I slot print_timing: id 3 | task 9875 | eval time = 20840.38 ms / 1150 tokens ( 18.12 ms per token, 55.18 tokens per second)
|
|
3.38.822.022 I slot print_timing: id 3 | task 9875 | total time = 20946.19 ms / 1171 tokens
|
|
3.38.822.022 I slot print_timing: id 3 | task 9875 | graphs reused = 10948
|
|
3.38.822.056 I slot release: id 3 | task 9875 | stop processing: n_tokens = 1242, truncated = 0
|
|
3.38.822.068 I srv update_slots: all slots are idle
|
|
3.38.824.872 I srv operator(): Chat format: peg-gemma4
|
|
3.38.825.198 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.613 (> 0.100 thold), f_keep = 0.061
|
|
3.38.825.199 I srv get_availabl: updating prompt cache
|
|
3.38.825.281 W srv prompt_save: - saving prompt with length 1242, total state size = 224.284 MiB (draft: 0.000 MiB)
|
|
3.38.993.542 I srv load: - looking for better prompt, base f_keep = 0.061, sim = 0.613
|
|
3.38.993.551 I srv update: - cache state: 8 prompts, 1895.946 MiB (limits: 8192.000 MiB, 16384 tokens, 50812 est)
|
|
3.38.993.552 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
3.38.993.555 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
3.38.993.556 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
3.38.993.556 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
3.38.993.557 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
3.38.993.558 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
3.38.993.559 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
3.38.993.560 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
3.38.993.561 I srv get_availabl: prompt cache update took 168.36 ms
|
|
3.38.993.622 I slot launch_slot_: id 3 | task 11028 | processing task, is_child = 0
|
|
3.38.993.623 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.38.993.624 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.38.993.644 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.38.993.644 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.38.993.656 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.38.993.657 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.38.993.675 I slot update_slots: id 3 | task 11028 | Checking checkpoint with [0, 72] against 0...
|
|
3.38.999.894 W slot update_slots: id 3 | task 11028 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
3.42.180.040 I slot print_timing: id 3 | task 11028 | n_decoded = 167, tg = 55.59 t/s, tg_3s = 55.59 t/s
|
|
3.45.188.097 I slot print_timing: id 3 | task 11028 | n_decoded = 334, tg = 55.55 t/s, tg_3s = 55.52 t/s
|
|
3.48.188.616 I slot print_timing: id 3 | task 11028 | n_decoded = 499, tg = 55.36 t/s, tg_3s = 54.99 t/s
|
|
3.51.202.265 I slot print_timing: id 3 | task 11028 | n_decoded = 664, tg = 55.21 t/s, tg_3s = 54.75 t/s
|
|
3.54.213.294 I slot print_timing: id 3 | task 11028 | n_decoded = 828, tg = 55.06 t/s, tg_3s = 54.47 t/s
|
|
3.55.207.901 I slot print_timing: id 3 | task 11028 | prompt eval time = 181.99 ms / 52 tokens ( 3.50 ms per token, 285.72 tokens per second)
|
|
3.55.207.904 I slot print_timing: id 3 | task 11028 | eval time = 16032.23 ms / 883 tokens ( 18.16 ms per token, 55.08 tokens per second)
|
|
3.55.207.905 I slot print_timing: id 3 | task 11028 | total time = 16214.22 ms / 935 tokens
|
|
3.55.207.905 I slot print_timing: id 3 | task 11028 | graphs reused = 11826
|
|
3.55.207.941 I slot release: id 3 | task 11028 | stop processing: n_tokens = 1006, truncated = 0
|
|
3.55.207.951 I srv update_slots: all slots are idle
|
|
3.55.210.816 I srv operator(): Chat format: peg-gemma4
|
|
3.55.211.158 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.538 (> 0.100 thold), f_keep = 0.077
|
|
3.55.211.160 I srv get_availabl: updating prompt cache
|
|
3.55.211.243 W srv prompt_save: - saving prompt with length 1006, total state size = 216.157 MiB (draft: 0.000 MiB)
|
|
3.55.368.340 I srv load: - looking for better prompt, base f_keep = 0.077, sim = 0.538
|
|
3.55.368.352 I srv update: - cache state: 9 prompts, 2126.362 MiB (limits: 8192.000 MiB, 16384 tokens, 49182 est)
|
|
3.55.368.353 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
3.55.368.354 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
3.55.368.355 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
3.55.368.359 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
3.55.368.360 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
3.55.368.361 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
3.55.368.362 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
3.55.368.362 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
3.55.368.365 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
3.55.368.367 I srv get_availabl: prompt cache update took 157.21 ms
|
|
3.55.368.433 I slot launch_slot_: id 3 | task 11914 | processing task, is_child = 0
|
|
3.55.368.434 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.55.368.435 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.55.368.452 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.55.368.452 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.55.368.466 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.55.368.466 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.55.368.504 I slot update_slots: id 3 | task 11914 | Checking checkpoint with [0, 72] against 0...
|
|
3.55.374.408 W slot update_slots: id 3 | task 11914 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
3.58.574.781 I slot print_timing: id 3 | task 11914 | n_decoded = 167, tg = 55.52 t/s, tg_3s = 55.52 t/s
|
|
4.01.585.287 I slot print_timing: id 3 | task 11914 | n_decoded = 334, tg = 55.50 t/s, tg_3s = 55.47 t/s
|
|
4.04.599.547 I slot print_timing: id 3 | task 11914 | n_decoded = 501, tg = 55.46 t/s, tg_3s = 55.40 t/s
|
|
4.07.614.698 I slot print_timing: id 3 | task 11914 | n_decoded = 668, tg = 55.45 t/s, tg_3s = 55.39 t/s
|
|
4.10.622.752 I slot print_timing: id 3 | task 11914 | n_decoded = 835, tg = 55.46 t/s, tg_3s = 55.52 t/s
|
|
4.13.630.749 I slot print_timing: id 3 | task 11914 | n_decoded = 1000, tg = 55.36 t/s, tg_3s = 54.85 t/s
|
|
4.16.640.546 I slot print_timing: id 3 | task 11914 | n_decoded = 1164, tg = 55.23 t/s, tg_3s = 54.49 t/s
|
|
4.19.655.928 I slot print_timing: id 3 | task 11914 | n_decoded = 1327, tg = 55.09 t/s, tg_3s = 54.06 t/s
|
|
4.22.667.802 I slot print_timing: id 3 | task 11914 | n_decoded = 1489, tg = 54.94 t/s, tg_3s = 53.79 t/s
|
|
4.25.437.852 I slot print_timing: id 3 | task 11914 | prompt eval time = 198.29 ms / 71 tokens ( 2.79 ms per token, 358.06 tokens per second)
|
|
4.25.437.855 I slot print_timing: id 3 | task 11914 | eval time = 29871.05 ms / 1638 tokens ( 18.24 ms per token, 54.84 tokens per second)
|
|
4.25.437.856 I slot print_timing: id 3 | task 11914 | total time = 30069.34 ms / 1709 tokens
|
|
4.25.437.857 I slot print_timing: id 3 | task 11914 | graphs reused = 13456
|
|
4.25.437.897 I slot release: id 3 | task 11914 | stop processing: n_tokens = 1780, truncated = 0
|
|
4.25.437.910 I srv update_slots: all slots are idle
|
|
4.25.440.652 I srv operator(): Chat format: peg-gemma4
|
|
4.25.440.978 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.043
|
|
4.25.440.979 I srv get_availabl: updating prompt cache
|
|
4.25.441.084 W srv prompt_save: - saving prompt with length 1780, total state size = 234.798 MiB (draft: 0.000 MiB)
|
|
4.25.607.844 I srv load: - looking for better prompt, base f_keep = 0.043, sim = 0.667
|
|
4.25.607.854 I srv update: - cache state: 10 prompts, 2375.419 MiB (limits: 8192.000 MiB, 16384 tokens, 50164 est)
|
|
4.25.607.855 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
4.25.607.856 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
4.25.607.860 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
4.25.607.860 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
4.25.607.861 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
4.25.607.861 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
4.25.607.862 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
4.25.607.862 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
4.25.607.863 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
4.25.607.864 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
4.25.607.866 I srv get_availabl: prompt cache update took 166.89 ms
|
|
4.25.607.926 I slot launch_slot_: id 3 | task 13555 | processing task, is_child = 0
|
|
4.25.607.927 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.25.607.927 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.25.607.942 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.25.607.943 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.25.607.955 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.25.607.955 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.25.607.974 I slot update_slots: id 3 | task 13555 | Checking checkpoint with [0, 72] against 0...
|
|
4.25.614.310 W slot update_slots: id 3 | task 13555 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
4.28.787.627 I slot print_timing: id 3 | task 13555 | n_decoded = 167, tg = 55.48 t/s, tg_3s = 55.48 t/s
|
|
4.31.800.626 I slot print_timing: id 3 | task 13555 | n_decoded = 334, tg = 55.45 t/s, tg_3s = 55.43 t/s
|
|
4.34.803.567 I slot print_timing: id 3 | task 13555 | n_decoded = 500, tg = 55.39 t/s, tg_3s = 55.28 t/s
|
|
4.37.819.553 I slot print_timing: id 3 | task 13555 | n_decoded = 666, tg = 55.31 t/s, tg_3s = 55.04 t/s
|
|
4.40.829.880 I slot print_timing: id 3 | task 13555 | n_decoded = 830, tg = 55.14 t/s, tg_3s = 54.48 t/s
|
|
4.43.840.556 I slot print_timing: id 3 | task 13555 | n_decoded = 993, tg = 54.97 t/s, tg_3s = 54.14 t/s
|
|
4.46.855.915 I slot print_timing: id 3 | task 13555 | n_decoded = 1156, tg = 54.84 t/s, tg_3s = 54.06 t/s
|
|
4.49.752.669 I slot print_timing: id 3 | task 13555 | prompt eval time = 169.32 ms / 42 tokens ( 4.03 ms per token, 248.05 tokens per second)
|
|
4.49.752.673 I slot print_timing: id 3 | task 13555 | eval time = 23975.37 ms / 1312 tokens ( 18.27 ms per token, 54.72 tokens per second)
|
|
4.49.752.674 I slot print_timing: id 3 | task 13555 | total time = 24144.69 ms / 1354 tokens
|
|
4.49.752.675 I slot print_timing: id 3 | task 13555 | graphs reused = 14761
|
|
4.49.752.723 I slot release: id 3 | task 13555 | stop processing: n_tokens = 1425, truncated = 0
|
|
4.49.752.738 I srv update_slots: all slots are idle
|
|
4.49.755.548 I srv operator(): Chat format: peg-gemma4
|
|
4.49.756.125 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.547 (> 0.100 thold), f_keep = 0.053
|
|
4.49.756.128 I srv get_availabl: updating prompt cache
|
|
4.49.756.262 W srv prompt_save: - saving prompt with length 1425, total state size = 227.861 MiB (draft: 0.000 MiB)
|
|
4.49.917.519 I srv load: - looking for better prompt, base f_keep = 0.053, sim = 0.547
|
|
4.49.917.528 I srv update: - cache state: 11 prompts, 2617.539 MiB (limits: 8192.000 MiB, 16384 tokens, 49983 est)
|
|
4.49.917.529 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
4.49.917.530 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
4.49.917.530 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
4.49.917.531 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
4.49.917.532 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
4.49.917.533 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
4.49.917.537 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
4.49.917.537 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
4.49.917.538 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
4.49.917.539 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
4.49.917.539 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
4.49.917.540 I srv get_availabl: prompt cache update took 161.41 ms
|
|
4.49.917.609 I slot launch_slot_: id 3 | task 14870 | processing task, is_child = 0
|
|
4.49.917.609 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.49.917.610 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.49.917.628 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.49.917.628 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.49.917.650 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.49.917.651 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.49.917.675 I slot update_slots: id 3 | task 14870 | Checking checkpoint with [0, 72] against 0...
|
|
4.49.923.834 W slot update_slots: id 3 | task 14870 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
4.53.167.648 I slot print_timing: id 3 | task 14870 | n_decoded = 167, tg = 55.53 t/s, tg_3s = 55.53 t/s
|
|
4.56.170.437 I slot print_timing: id 3 | task 14870 | n_decoded = 334, tg = 55.57 t/s, tg_3s = 55.61 t/s
|
|
4.59.187.570 I slot print_timing: id 3 | task 14870 | n_decoded = 501, tg = 55.50 t/s, tg_3s = 55.35 t/s
|
|
5.02.189.784 I slot print_timing: id 3 | task 14870 | n_decoded = 667, tg = 55.45 t/s, tg_3s = 55.29 t/s
|
|
5.04.767.537 I slot print_timing: id 3 | task 14870 | prompt eval time = 242.39 ms / 67 tokens ( 3.62 ms per token, 276.42 tokens per second)
|
|
5.04.767.541 I slot print_timing: id 3 | task 14870 | eval time = 14607.47 ms / 809 tokens ( 18.06 ms per token, 55.38 tokens per second)
|
|
5.04.767.541 I slot print_timing: id 3 | task 14870 | total time = 14849.86 ms / 876 tokens
|
|
5.04.767.542 I slot print_timing: id 3 | task 14870 | graphs reused = 15565
|
|
5.04.767.581 I slot release: id 3 | task 14870 | stop processing: n_tokens = 947, truncated = 0
|
|
5.04.767.596 I srv update_slots: all slots are idle
|
|
5.04.770.366 I srv operator(): Chat format: peg-gemma4
|
|
5.04.770.730 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.571 (> 0.100 thold), f_keep = 0.080
|
|
5.04.770.731 I srv get_availabl: updating prompt cache
|
|
5.04.770.812 W srv prompt_save: - saving prompt with length 947, total state size = 203.479 MiB (draft: 0.000 MiB)
|
|
5.04.918.461 I srv load: - looking for better prompt, base f_keep = 0.080, sim = 0.571
|
|
5.04.918.474 I srv update: - cache state: 12 prompts, 2835.278 MiB (limits: 8192.000 MiB, 16384 tokens, 48881 est)
|
|
5.04.918.476 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
5.04.918.479 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
5.04.918.480 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
5.04.918.481 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
5.04.918.482 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
5.04.918.483 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
5.04.918.486 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
5.04.918.487 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
5.04.918.488 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
5.04.918.488 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
5.04.918.489 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
5.04.918.492 I srv update: - prompt 0x57bec8c8c850: 947 tokens, checkpoints: 1, 217.739 MiB
|
|
5.04.918.494 I srv get_availabl: prompt cache update took 147.76 ms
|
|
5.04.918.556 I slot launch_slot_: id 3 | task 15682 | processing task, is_child = 0
|
|
5.04.918.558 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.04.918.558 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.04.918.579 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.04.918.579 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.04.918.609 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.04.918.610 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.04.918.655 I slot update_slots: id 3 | task 15682 | Checking checkpoint with [0, 72] against 0...
|
|
5.04.924.864 W slot update_slots: id 3 | task 15682 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
5.08.159.861 I slot print_timing: id 3 | task 15682 | n_decoded = 167, tg = 55.46 t/s, tg_3s = 55.46 t/s
|
|
5.11.166.516 I slot print_timing: id 3 | task 15682 | n_decoded = 334, tg = 55.50 t/s, tg_3s = 55.54 t/s
|
|
5.14.170.776 I slot print_timing: id 3 | task 15682 | n_decoded = 500, tg = 55.42 t/s, tg_3s = 55.25 t/s
|
|
5.17.176.571 I slot print_timing: id 3 | task 15682 | n_decoded = 666, tg = 55.37 t/s, tg_3s = 55.23 t/s
|
|
5.20.190.590 I slot print_timing: id 3 | task 15682 | n_decoded = 831, tg = 55.24 t/s, tg_3s = 54.74 t/s
|
|
5.23.205.664 I slot print_timing: id 3 | task 15682 | prompt eval time = 229.76 ms / 61 tokens ( 3.77 ms per token, 265.50 tokens per second)
|
|
5.23.205.669 I slot print_timing: id 3 | task 15682 | eval time = 18057.25 ms / 994 tokens ( 18.17 ms per token, 55.05 tokens per second)
|
|
5.23.205.670 I slot print_timing: id 3 | task 15682 | total time = 18287.01 ms / 1055 tokens
|
|
5.23.205.670 I slot print_timing: id 3 | task 15682 | graphs reused = 16553
|
|
5.23.205.722 I slot release: id 3 | task 15682 | stop processing: n_tokens = 1126, truncated = 0
|
|
5.23.205.734 I srv update_slots: all slots are idle
|
|
5.23.208.464 I srv operator(): Chat format: peg-gemma4
|
|
5.23.208.854 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.631 (> 0.100 thold), f_keep = 0.068
|
|
5.23.208.856 I srv get_availabl: updating prompt cache
|
|
5.23.208.950 W srv prompt_save: - saving prompt with length 1126, total state size = 222.018 MiB (draft: 0.000 MiB)
|
|
5.23.368.865 I srv load: - looking for better prompt, base f_keep = 0.068, sim = 0.631
|
|
5.23.368.877 I srv update: - cache state: 13 prompts, 3071.555 MiB (limits: 8192.000 MiB, 16384 tokens, 48124 est)
|
|
5.23.368.879 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
5.23.368.879 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
5.23.368.880 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
5.23.368.881 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
5.23.368.887 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
5.23.368.887 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
5.23.368.888 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
5.23.368.889 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
5.23.368.890 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
5.23.368.890 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
5.23.368.894 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
5.23.368.894 I srv update: - prompt 0x57bec8c8c850: 947 tokens, checkpoints: 1, 217.739 MiB
|
|
5.23.368.895 I srv update: - prompt 0x57bec8d451f0: 1126 tokens, checkpoints: 1, 236.277 MiB
|
|
5.23.368.896 I srv get_availabl: prompt cache update took 160.04 ms
|
|
5.23.368.969 I slot launch_slot_: id 3 | task 16679 | processing task, is_child = 0
|
|
5.23.368.970 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.23.368.971 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.23.368.988 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.23.368.988 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.23.369.005 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.23.369.005 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.23.369.028 I slot update_slots: id 3 | task 16679 | Checking checkpoint with [0, 72] against 0...
|
|
5.23.375.108 W slot update_slots: id 3 | task 16679 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
5.26.558.031 I slot print_timing: id 3 | task 16679 | n_decoded = 167, tg = 55.54 t/s, tg_3s = 55.54 t/s
|
|
5.29.570.728 I slot print_timing: id 3 | task 16679 | n_decoded = 334, tg = 55.49 t/s, tg_3s = 55.43 t/s
|
|
5.32.575.795 I slot print_timing: id 3 | task 16679 | n_decoded = 499, tg = 55.29 t/s, tg_3s = 54.91 t/s
|
|
5.35.587.582 I slot print_timing: id 3 | task 16679 | n_decoded = 663, tg = 55.08 t/s, tg_3s = 54.45 t/s
|
|
5.38.600.620 I slot print_timing: id 3 | task 16679 | n_decoded = 826, tg = 54.89 t/s, tg_3s = 54.10 t/s
|
|
5.41.612.736 I slot print_timing: id 3 | task 16679 | n_decoded = 991, tg = 54.87 t/s, tg_3s = 54.78 t/s
|
|
5.42.116.713 I slot print_timing: id 3 | task 16679 | prompt eval time = 182.34 ms / 50 tokens ( 3.65 ms per token, 274.22 tokens per second)
|
|
5.42.116.717 I slot print_timing: id 3 | task 16679 | eval time = 18565.35 ms / 1019 tokens ( 18.22 ms per token, 54.89 tokens per second)
|
|
5.42.116.718 I slot print_timing: id 3 | task 16679 | total time = 18747.68 ms / 1069 tokens
|
|
5.42.116.719 I slot print_timing: id 3 | task 16679 | graphs reused = 17566
|
|
5.42.116.766 I slot release: id 3 | task 16679 | stop processing: n_tokens = 1140, truncated = 0
|
|
5.42.116.779 I srv update_slots: all slots are idle
|
|
5.42.119.490 I srv operator(): Chat format: peg-gemma4
|
|
5.42.119.840 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.613 (> 0.100 thold), f_keep = 0.067
|
|
5.42.119.842 I srv get_availabl: updating prompt cache
|
|
5.42.119.933 W srv prompt_save: - saving prompt with length 1140, total state size = 222.291 MiB (draft: 0.000 MiB)
|
|
5.42.278.114 I srv load: - looking for better prompt, base f_keep = 0.067, sim = 0.613
|
|
5.42.278.130 I srv update: - cache state: 14 prompts, 3308.105 MiB (limits: 8192.000 MiB, 16384 tokens, 47506 est)
|
|
5.42.278.134 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
5.42.278.135 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
5.42.278.136 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
5.42.278.137 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
5.42.278.138 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
5.42.278.141 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
5.42.278.142 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
5.42.278.143 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
5.42.278.143 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
5.42.278.144 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
5.42.278.146 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
5.42.278.147 I srv update: - prompt 0x57bec8c8c850: 947 tokens, checkpoints: 1, 217.739 MiB
|
|
5.42.278.148 I srv update: - prompt 0x57bec8d451f0: 1126 tokens, checkpoints: 1, 236.277 MiB
|
|
5.42.278.148 I srv update: - prompt 0x57bec8caa9b0: 1140 tokens, checkpoints: 1, 236.550 MiB
|
|
5.42.278.149 I srv get_availabl: prompt cache update took 158.31 ms
|
|
5.42.278.221 I slot launch_slot_: id 3 | task 17701 | processing task, is_child = 0
|
|
5.42.278.222 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.42.278.223 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.42.278.240 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.42.278.241 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.42.278.255 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.42.278.255 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.42.278.279 I slot update_slots: id 3 | task 17701 | Checking checkpoint with [0, 72] against 0...
|
|
5.42.284.135 W slot update_slots: id 3 | task 17701 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
5.45.473.220 I slot print_timing: id 3 | task 17701 | n_decoded = 168, tg = 55.80 t/s, tg_3s = 55.80 t/s
|
|
5.48.479.281 I slot print_timing: id 3 | task 17701 | n_decoded = 335, tg = 55.68 t/s, tg_3s = 55.55 t/s
|
|
5.51.494.469 I slot print_timing: id 3 | task 17701 | n_decoded = 502, tg = 55.58 t/s, tg_3s = 55.39 t/s
|
|
5.54.509.750 I slot print_timing: id 3 | task 17701 | n_decoded = 669, tg = 55.53 t/s, tg_3s = 55.38 t/s
|
|
5.57.515.730 I slot print_timing: id 3 | task 17701 | n_decoded = 836, tg = 55.54 t/s, tg_3s = 55.56 t/s
|
|
6.00.523.898 I slot print_timing: id 3 | task 17701 | n_decoded = 1002, tg = 55.48 t/s, tg_3s = 55.18 t/s
|
|
6.03.534.886 I slot print_timing: id 3 | task 17701 | n_decoded = 1167, tg = 55.38 t/s, tg_3s = 54.80 t/s
|
|
6.03.993.008 I slot print_timing: id 3 | task 17701 | prompt eval time = 184.35 ms / 52 tokens ( 3.55 ms per token, 282.07 tokens per second)
|
|
6.03.993.012 I slot print_timing: id 3 | task 17701 | eval time = 21530.37 ms / 1192 tokens ( 18.06 ms per token, 55.36 tokens per second)
|
|
6.03.993.013 I slot print_timing: id 3 | task 17701 | total time = 21714.73 ms / 1244 tokens
|
|
6.03.993.014 I slot print_timing: id 3 | task 17701 | graphs reused = 18751
|
|
6.03.993.046 I slot release: id 3 | task 17701 | stop processing: n_tokens = 1315, truncated = 0
|
|
6.03.993.059 I srv update_slots: all slots are idle
|
|
6.03.995.598 I srv operator(): Chat format: peg-gemma4
|
|
6.03.996.245 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.655 (> 0.100 thold), f_keep = 0.058
|
|
6.03.996.247 I srv get_availabl: updating prompt cache
|
|
6.03.996.348 W srv prompt_save: - saving prompt with length 1315, total state size = 225.711 MiB (draft: 0.000 MiB)
|
|
6.04.157.570 I srv load: - looking for better prompt, base f_keep = 0.058, sim = 0.655
|
|
6.04.157.584 I srv update: - cache state: 15 prompts, 3548.076 MiB (limits: 8192.000 MiB, 16384 tokens, 47329 est)
|
|
6.04.157.585 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
6.04.157.586 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
6.04.157.586 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
6.04.157.587 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
6.04.157.587 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
6.04.157.588 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
6.04.157.589 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
6.04.157.589 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
6.04.157.590 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
6.04.157.591 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
6.04.157.591 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
6.04.157.592 I srv update: - prompt 0x57bec8c8c850: 947 tokens, checkpoints: 1, 217.739 MiB
|
|
6.04.157.592 I srv update: - prompt 0x57bec8d451f0: 1126 tokens, checkpoints: 1, 236.277 MiB
|
|
6.04.157.593 I srv update: - prompt 0x57bec8caa9b0: 1140 tokens, checkpoints: 1, 236.550 MiB
|
|
6.04.157.594 I srv update: - prompt 0x57bec0702290: 1315 tokens, checkpoints: 1, 239.970 MiB
|
|
6.04.157.595 I srv get_availabl: prompt cache update took 161.35 ms
|
|
6.04.157.667 I slot launch_slot_: id 3 | task 18896 | processing task, is_child = 0
|
|
6.04.157.668 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
6.04.157.669 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
6.04.157.687 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
6.04.157.688 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
6.04.157.702 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
6.04.157.702 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
6.04.157.724 I slot update_slots: id 3 | task 18896 | Checking checkpoint with [0, 72] against 0...
|
|
6.04.163.788 W slot update_slots: id 3 | task 18896 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
6.07.330.483 I slot print_timing: id 3 | task 18896 | n_decoded = 167, tg = 55.59 t/s, tg_3s = 55.59 t/s
|
|
6.10.336.496 I slot print_timing: id 3 | task 18896 | n_decoded = 334, tg = 55.57 t/s, tg_3s = 55.56 t/s
|
|
6.13.352.675 I slot print_timing: id 3 | task 18896 | n_decoded = 501, tg = 55.50 t/s, tg_3s = 55.37 t/s
|
|
6.16.354.449 I slot print_timing: id 3 | task 18896 | n_decoded = 667, tg = 55.45 t/s, tg_3s = 55.30 t/s
|
|
6.19.363.442 I slot print_timing: id 3 | task 18896 | n_decoded = 832, tg = 55.33 t/s, tg_3s = 54.84 t/s
|
|
6.22.376.014 I slot print_timing: id 3 | task 18896 | n_decoded = 996, tg = 55.18 t/s, tg_3s = 54.44 t/s
|
|
6.23.646.387 I slot print_timing: id 3 | task 18896 | prompt eval time = 168.52 ms / 44 tokens ( 3.83 ms per token, 261.10 tokens per second)
|
|
6.23.646.391 I slot print_timing: id 3 | task 18896 | eval time = 19320.14 ms / 1065 tokens ( 18.14 ms per token, 55.12 tokens per second)
|
|
6.23.646.392 I slot print_timing: id 3 | task 18896 | total time = 19488.66 ms / 1109 tokens
|
|
6.23.646.392 I slot print_timing: id 3 | task 18896 | graphs reused = 19810
|
|
6.23.646.430 I slot release: id 3 | task 18896 | stop processing: n_tokens = 1180, truncated = 0
|
|
6.23.646.442 I srv update_slots: all slots are idle
|
|
6.23.649.171 I srv operator(): Chat format: peg-gemma4
|
|
6.23.649.502 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.655 (> 0.100 thold), f_keep = 0.064
|
|
6.23.649.504 I srv get_availabl: updating prompt cache
|
|
6.23.649.604 W srv prompt_save: - saving prompt with length 1180, total state size = 223.073 MiB (draft: 0.000 MiB)
|
|
6.23.808.128 I srv load: - looking for better prompt, base f_keep = 0.064, sim = 0.655
|
|
6.23.808.141 I srv update: - cache state: 16 prompts, 3785.408 MiB (limits: 8192.000 MiB, 16384 tokens, 46915 est)
|
|
6.23.808.146 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
6.23.808.146 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
6.23.808.147 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
6.23.808.147 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
6.23.808.148 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
6.23.808.158 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
6.23.808.159 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
6.23.808.159 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
6.23.808.160 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
6.23.808.160 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
6.23.808.161 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
6.23.808.161 I srv update: - prompt 0x57bec8c8c850: 947 tokens, checkpoints: 1, 217.739 MiB
|
|
6.23.808.162 I srv update: - prompt 0x57bec8d451f0: 1126 tokens, checkpoints: 1, 236.277 MiB
|
|
6.23.808.162 I srv update: - prompt 0x57bec8caa9b0: 1140 tokens, checkpoints: 1, 236.550 MiB
|
|
6.23.808.163 I srv update: - prompt 0x57bec0702290: 1315 tokens, checkpoints: 1, 239.970 MiB
|
|
6.23.808.163 I srv update: - prompt 0x57bec03fa660: 1180 tokens, checkpoints: 1, 237.332 MiB
|
|
6.23.808.164 I srv get_availabl: prompt cache update took 158.66 ms
|
|
6.23.808.224 I slot launch_slot_: id 3 | task 19964 | processing task, is_child = 0
|
|
6.23.808.225 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
6.23.808.226 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
6.23.808.244 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
6.23.808.245 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
6.23.808.256 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
6.23.808.257 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
6.23.808.277 I slot update_slots: id 3 | task 19964 | Checking checkpoint with [0, 72] against 0...
|
|
6.23.814.387 W slot update_slots: id 3 | task 19964 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
6.26.995.718 I slot print_timing: id 3 | task 19964 | n_decoded = 167, tg = 55.46 t/s, tg_3s = 55.46 t/s
|
|
6.30.008.684 I slot print_timing: id 3 | task 19964 | n_decoded = 334, tg = 55.44 t/s, tg_3s = 55.43 t/s
|
|
6.33.024.792 I slot print_timing: id 3 | task 19964 | n_decoded = 501, tg = 55.42 t/s, tg_3s = 55.37 t/s
|
|
6.36.039.702 I slot print_timing: id 3 | task 19964 | n_decoded = 668, tg = 55.41 t/s, tg_3s = 55.39 t/s
|
|
6.39.045.730 I slot print_timing: id 3 | task 19964 | n_decoded = 835, tg = 55.44 t/s, tg_3s = 55.56 t/s
|
|
6.42.047.735 I slot print_timing: id 3 | task 19964 | n_decoded = 1000, tg = 55.36 t/s, tg_3s = 54.96 t/s
|
|
6.43.322.424 I slot print_timing: id 3 | task 19964 | prompt eval time = 176.00 ms / 44 tokens ( 4.00 ms per token, 250.00 tokens per second)
|
|
6.43.322.428 I slot print_timing: id 3 | task 19964 | eval time = 19338.14 ms / 1070 tokens ( 18.07 ms per token, 55.33 tokens per second)
|
|
6.43.322.429 I slot print_timing: id 3 | task 19964 | total time = 19514.14 ms / 1114 tokens
|
|
6.43.322.429 I slot print_timing: id 3 | task 19964 | graphs reused = 20874
|
|
6.43.322.467 I slot release: id 3 | task 19964 | stop processing: n_tokens = 1185, truncated = 0
|
|
6.43.322.478 I srv update_slots: all slots are idle
|
|
6.43.325.234 I srv operator(): Chat format: peg-gemma4
|
|
6.43.325.570 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.064
|
|
6.43.325.572 I srv get_availabl: updating prompt cache
|
|
6.43.325.664 W srv prompt_save: - saving prompt with length 1185, total state size = 223.171 MiB (draft: 0.000 MiB)
|
|
6.43.482.672 I srv load: - looking for better prompt, base f_keep = 0.064, sim = 0.710
|
|
6.43.482.685 I srv update: - cache state: 17 prompts, 4022.837 MiB (limits: 8192.000 MiB, 16384 tokens, 46559 est)
|
|
6.43.482.687 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
6.43.482.691 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
6.43.482.691 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
6.43.482.692 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
6.43.482.692 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
6.43.482.693 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
6.43.482.693 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
6.43.482.695 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
6.43.482.696 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
6.43.482.696 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
6.43.482.697 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
6.43.482.697 I srv update: - prompt 0x57bec8c8c850: 947 tokens, checkpoints: 1, 217.739 MiB
|
|
6.43.482.698 I srv update: - prompt 0x57bec8d451f0: 1126 tokens, checkpoints: 1, 236.277 MiB
|
|
6.43.482.698 I srv update: - prompt 0x57bec8caa9b0: 1140 tokens, checkpoints: 1, 236.550 MiB
|
|
6.43.482.699 I srv update: - prompt 0x57bec0702290: 1315 tokens, checkpoints: 1, 239.970 MiB
|
|
6.43.482.699 I srv update: - prompt 0x57bec03fa660: 1180 tokens, checkpoints: 1, 237.332 MiB
|
|
6.43.482.700 I srv update: - prompt 0x57beb68c5010: 1185 tokens, checkpoints: 1, 237.430 MiB
|
|
6.43.482.701 I srv get_availabl: prompt cache update took 157.13 ms
|
|
6.43.482.761 I slot launch_slot_: id 3 | task 21037 | processing task, is_child = 0
|
|
6.43.482.762 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
6.43.482.762 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
6.43.482.780 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
6.43.482.780 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
6.43.482.792 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
6.43.482.792 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
6.43.482.813 I slot update_slots: id 3 | task 21037 | Checking checkpoint with [0, 72] against 0...
|
|
6.43.488.802 W slot update_slots: id 3 | task 21037 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
6.46.653.462 I slot print_timing: id 3 | task 21037 | n_decoded = 167, tg = 55.50 t/s, tg_3s = 55.50 t/s
|
|
6.49.659.488 I slot print_timing: id 3 | task 21037 | n_decoded = 334, tg = 55.53 t/s, tg_3s = 55.56 t/s
|
|
6.52.674.673 I slot print_timing: id 3 | task 21037 | n_decoded = 501, tg = 55.48 t/s, tg_3s = 55.39 t/s
|
|
6.55.689.897 I slot print_timing: id 3 | task 21037 | n_decoded = 668, tg = 55.46 t/s, tg_3s = 55.39 t/s
|
|
6.58.703.932 I slot print_timing: id 3 | task 21037 | n_decoded = 835, tg = 55.45 t/s, tg_3s = 55.41 t/s
|
|
7.01.199.985 I slot print_timing: id 3 | task 21037 | prompt eval time = 161.38 ms / 35 tokens ( 4.61 ms per token, 216.88 tokens per second)
|
|
7.01.199.988 I slot print_timing: id 3 | task 21037 | eval time = 17555.78 ms / 972 tokens ( 18.06 ms per token, 55.37 tokens per second)
|
|
7.01.199.989 I slot print_timing: id 3 | task 21037 | total time = 17717.17 ms / 1007 tokens
|
|
7.01.199.989 I slot print_timing: id 3 | task 21037 | graphs reused = 21840
|
|
7.01.200.028 I slot release: id 3 | task 21037 | stop processing: n_tokens = 1078, truncated = 0
|
|
7.01.200.038 I srv update_slots: all slots are idle
|
|
7.01.202.935 I srv operator(): Chat format: peg-gemma4
|
|
7.01.203.468 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.792 (> 0.100 thold), f_keep = 0.071
|
|
7.01.203.470 I srv get_availabl: updating prompt cache
|
|
7.01.203.549 W srv prompt_save: - saving prompt with length 1078, total state size = 221.079 MiB (draft: 0.000 MiB)
|
|
7.01.359.363 I srv load: - looking for better prompt, base f_keep = 0.071, sim = 0.792
|
|
7.01.359.376 I srv update: - cache state: 18 prompts, 4258.176 MiB (limits: 8192.000 MiB, 16384 tokens, 46060 est)
|
|
7.01.359.377 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
7.01.359.378 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
7.01.359.378 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
7.01.359.379 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
7.01.359.379 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
7.01.359.380 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
7.01.359.380 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
7.01.359.381 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
7.01.359.381 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
7.01.359.382 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
7.01.359.382 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
7.01.359.383 I srv update: - prompt 0x57bec8c8c850: 947 tokens, checkpoints: 1, 217.739 MiB
|
|
7.01.359.383 I srv update: - prompt 0x57bec8d451f0: 1126 tokens, checkpoints: 1, 236.277 MiB
|
|
7.01.359.384 I srv update: - prompt 0x57bec8caa9b0: 1140 tokens, checkpoints: 1, 236.550 MiB
|
|
7.01.359.384 I srv update: - prompt 0x57bec0702290: 1315 tokens, checkpoints: 1, 239.970 MiB
|
|
7.01.359.385 I srv update: - prompt 0x57bec03fa660: 1180 tokens, checkpoints: 1, 237.332 MiB
|
|
7.01.359.385 I srv update: - prompt 0x57beb68c5010: 1185 tokens, checkpoints: 1, 237.430 MiB
|
|
7.01.359.386 I srv update: - prompt 0x57bec905f960: 1078 tokens, checkpoints: 1, 235.339 MiB
|
|
7.01.359.387 I srv get_availabl: prompt cache update took 155.92 ms
|
|
7.01.359.458 I slot launch_slot_: id 3 | task 22012 | processing task, is_child = 0
|
|
7.01.359.459 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.01.359.459 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.01.359.479 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.01.359.479 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.01.359.495 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.01.359.496 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.01.359.522 I slot update_slots: id 3 | task 22012 | Checking checkpoint with [0, 72] against 0...
|
|
7.01.365.445 W slot update_slots: id 3 | task 22012 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
7.04.497.751 I slot print_timing: id 3 | task 22012 | n_decoded = 167, tg = 55.57 t/s, tg_3s = 55.57 t/s
|
|
7.07.504.845 I slot print_timing: id 3 | task 22012 | n_decoded = 334, tg = 55.55 t/s, tg_3s = 55.54 t/s
|
|
7.10.508.915 I slot print_timing: id 3 | task 22012 | n_decoded = 500, tg = 55.45 t/s, tg_3s = 55.26 t/s
|
|
7.13.514.891 I slot print_timing: id 3 | task 22012 | n_decoded = 667, tg = 55.48 t/s, tg_3s = 55.56 t/s
|
|
7.16.523.705 I slot print_timing: id 3 | task 22012 | n_decoded = 833, tg = 55.42 t/s, tg_3s = 55.17 t/s
|
|
7.17.805.912 I slot print_timing: id 3 | task 22012 | prompt eval time = 132.98 ms / 24 tokens ( 5.54 ms per token, 180.48 tokens per second)
|
|
7.17.805.916 I slot print_timing: id 3 | task 22012 | eval time = 16313.40 ms / 904 tokens ( 18.05 ms per token, 55.41 tokens per second)
|
|
7.17.805.917 I slot print_timing: id 3 | task 22012 | total time = 16446.38 ms / 928 tokens
|
|
7.17.805.918 I slot print_timing: id 3 | task 22012 | graphs reused = 22739
|
|
7.17.805.955 I slot release: id 3 | task 22012 | stop processing: n_tokens = 999, truncated = 0
|
|
7.17.805.966 I srv update_slots: all slots are idle
|
|
7.17.808.517 I srv operator(): Chat format: peg-gemma4
|
|
7.17.808.857 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.650 (> 0.100 thold), f_keep = 0.076
|
|
7.17.808.859 I srv get_availabl: updating prompt cache
|
|
7.17.808.941 W srv prompt_save: - saving prompt with length 999, total state size = 214.652 MiB (draft: 0.000 MiB)
|
|
7.17.964.279 I srv load: - looking for better prompt, base f_keep = 0.076, sim = 0.650
|
|
7.17.964.295 I srv update: - cache state: 19 prompts, 4487.088 MiB (limits: 8192.000 MiB, 16384 tokens, 45534 est)
|
|
7.17.964.296 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
7.17.964.297 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
7.17.964.298 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
7.17.964.299 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
7.17.964.299 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
7.17.964.300 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
7.17.964.301 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
7.17.964.301 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
7.17.964.302 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
7.17.964.303 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
7.17.964.303 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
7.17.964.304 I srv update: - prompt 0x57bec8c8c850: 947 tokens, checkpoints: 1, 217.739 MiB
|
|
7.17.964.305 I srv update: - prompt 0x57bec8d451f0: 1126 tokens, checkpoints: 1, 236.277 MiB
|
|
7.17.964.305 I srv update: - prompt 0x57bec8caa9b0: 1140 tokens, checkpoints: 1, 236.550 MiB
|
|
7.17.964.306 I srv update: - prompt 0x57bec0702290: 1315 tokens, checkpoints: 1, 239.970 MiB
|
|
7.17.964.307 I srv update: - prompt 0x57bec03fa660: 1180 tokens, checkpoints: 1, 237.332 MiB
|
|
7.17.964.307 I srv update: - prompt 0x57beb68c5010: 1185 tokens, checkpoints: 1, 237.430 MiB
|
|
7.17.964.308 I srv update: - prompt 0x57bec905f960: 1078 tokens, checkpoints: 1, 235.339 MiB
|
|
7.17.964.308 I srv update: - prompt 0x57bec8d43c80: 999 tokens, checkpoints: 1, 228.912 MiB
|
|
7.17.964.310 I srv get_availabl: prompt cache update took 155.45 ms
|
|
7.17.964.387 I slot launch_slot_: id 3 | task 22919 | processing task, is_child = 0
|
|
7.17.964.388 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.17.964.389 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.17.964.414 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.17.964.414 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.17.964.435 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.17.964.435 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.17.964.464 I slot update_slots: id 3 | task 22919 | Checking checkpoint with [0, 72] against 0...
|
|
7.17.970.622 W slot update_slots: id 3 | task 22919 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
7.21.150.129 I slot print_timing: id 3 | task 22919 | n_decoded = 167, tg = 55.51 t/s, tg_3s = 55.51 t/s
|
|
7.24.156.255 I slot print_timing: id 3 | task 22919 | n_decoded = 334, tg = 55.53 t/s, tg_3s = 55.55 t/s
|
|
7.27.170.416 I slot print_timing: id 3 | task 22919 | n_decoded = 501, tg = 55.49 t/s, tg_3s = 55.41 t/s
|
|
7.30.184.663 I slot print_timing: id 3 | task 22919 | n_decoded = 668, tg = 55.47 t/s, tg_3s = 55.40 t/s
|
|
7.31.786.744 I slot print_timing: id 3 | task 22919 | prompt eval time = 176.97 ms / 45 tokens ( 3.93 ms per token, 254.27 tokens per second)
|
|
7.31.786.748 I slot print_timing: id 3 | task 22919 | eval time = 13645.30 ms / 757 tokens ( 18.03 ms per token, 55.48 tokens per second)
|
|
7.31.786.748 I slot print_timing: id 3 | task 22919 | total time = 13822.27 ms / 802 tokens
|
|
7.31.786.749 I slot print_timing: id 3 | task 22919 | graphs reused = 23491
|
|
7.31.786.791 I slot release: id 3 | task 22919 | stop processing: n_tokens = 873, truncated = 0
|
|
7.31.786.803 I srv update_slots: all slots are idle
|
|
7.31.789.411 I srv operator(): Chat format: peg-gemma4
|
|
7.31.789.747 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.717 (> 0.100 thold), f_keep = 0.087
|
|
7.31.789.748 I srv get_availabl: updating prompt cache
|
|
7.31.789.822 W srv prompt_save: - saving prompt with length 873, total state size = 187.579 MiB (draft: 0.000 MiB)
|
|
7.31.927.513 I srv load: - looking for better prompt, base f_keep = 0.087, sim = 0.717
|
|
7.31.927.526 I srv update: - cache state: 20 prompts, 4688.926 MiB (limits: 8192.000 MiB, 16384 tokens, 45099 est)
|
|
7.31.927.527 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
7.31.927.528 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
7.31.927.528 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
7.31.927.529 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
7.31.927.530 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
7.31.927.530 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
7.31.927.531 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
7.31.927.531 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
7.31.927.532 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
7.31.927.532 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
7.31.927.533 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
7.31.927.534 I srv update: - prompt 0x57bec8c8c850: 947 tokens, checkpoints: 1, 217.739 MiB
|
|
7.31.927.534 I srv update: - prompt 0x57bec8d451f0: 1126 tokens, checkpoints: 1, 236.277 MiB
|
|
7.31.927.535 I srv update: - prompt 0x57bec8caa9b0: 1140 tokens, checkpoints: 1, 236.550 MiB
|
|
7.31.927.535 I srv update: - prompt 0x57bec0702290: 1315 tokens, checkpoints: 1, 239.970 MiB
|
|
7.31.927.536 I srv update: - prompt 0x57bec03fa660: 1180 tokens, checkpoints: 1, 237.332 MiB
|
|
7.31.927.536 I srv update: - prompt 0x57beb68c5010: 1185 tokens, checkpoints: 1, 237.430 MiB
|
|
7.31.927.537 I srv update: - prompt 0x57bec905f960: 1078 tokens, checkpoints: 1, 235.339 MiB
|
|
7.31.927.537 I srv update: - prompt 0x57bec8d43c80: 999 tokens, checkpoints: 1, 228.912 MiB
|
|
7.31.927.538 I srv update: - prompt 0x57bec8d3dce0: 873 tokens, checkpoints: 1, 201.839 MiB
|
|
7.31.927.539 I srv get_availabl: prompt cache update took 137.79 ms
|
|
7.31.927.596 I slot launch_slot_: id 3 | task 23679 | processing task, is_child = 0
|
|
7.31.927.596 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.31.927.597 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.31.927.611 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.31.927.611 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.31.927.622 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.31.927.623 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.31.927.649 I slot update_slots: id 3 | task 23679 | Checking checkpoint with [0, 72] against 0...
|
|
7.31.933.692 W slot update_slots: id 3 | task 23679 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
7.35.095.398 I slot print_timing: id 3 | task 23679 | n_decoded = 167, tg = 55.47 t/s, tg_3s = 55.47 t/s
|
|
7.38.110.392 I slot print_timing: id 3 | task 23679 | n_decoded = 334, tg = 55.43 t/s, tg_3s = 55.39 t/s
|
|
7.41.125.596 I slot print_timing: id 3 | task 23679 | n_decoded = 501, tg = 55.41 t/s, tg_3s = 55.39 t/s
|
|
7.44.125.695 I slot print_timing: id 3 | task 23679 | n_decoded = 667, tg = 55.39 t/s, tg_3s = 55.33 t/s
|
|
7.47.136.822 I slot print_timing: id 3 | task 23679 | n_decoded = 834, tg = 55.41 t/s, tg_3s = 55.46 t/s
|
|
7.50.146.291 I slot print_timing: id 3 | task 23679 | n_decoded = 999, tg = 55.31 t/s, tg_3s = 54.83 t/s
|
|
7.53.157.927 I slot print_timing: id 3 | task 23679 | n_decoded = 1163, tg = 55.19 t/s, tg_3s = 54.46 t/s
|
|
7.53.741.919 I slot print_timing: id 3 | task 23679 | prompt eval time = 156.90 ms / 34 tokens ( 4.61 ms per token, 216.70 tokens per second)
|
|
7.53.741.923 I slot print_timing: id 3 | task 23679 | eval time = 21657.36 ms / 1194 tokens ( 18.14 ms per token, 55.13 tokens per second)
|
|
7.53.741.924 I slot print_timing: id 3 | task 23679 | total time = 21814.26 ms / 1228 tokens
|
|
7.53.741.925 I slot print_timing: id 3 | task 23679 | graphs reused = 24678
|
|
7.53.741.963 I slot release: id 3 | task 23679 | stop processing: n_tokens = 1299, truncated = 0
|
|
7.53.741.976 I srv update_slots: all slots are idle
|
|
7.53.744.654 I srv operator(): Chat format: peg-gemma4
|
|
7.53.744.988 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.673 (> 0.100 thold), f_keep = 0.059
|
|
7.53.744.990 I srv get_availabl: updating prompt cache
|
|
7.53.745.082 W srv prompt_save: - saving prompt with length 1299, total state size = 225.398 MiB (draft: 0.000 MiB)
|
|
7.53.907.747 I srv load: - looking for better prompt, base f_keep = 0.059, sim = 0.673
|
|
7.53.907.761 I srv update: - cache state: 21 prompts, 4928.584 MiB (limits: 8192.000 MiB, 16384 tokens, 45065 est)
|
|
7.53.907.762 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
7.53.907.763 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
7.53.907.763 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
7.53.907.764 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
7.53.907.764 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
7.53.907.765 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
7.53.907.765 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
7.53.907.766 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
7.53.907.766 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
7.53.907.767 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
7.53.907.767 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
7.53.907.768 I srv update: - prompt 0x57bec8c8c850: 947 tokens, checkpoints: 1, 217.739 MiB
|
|
7.53.907.768 I srv update: - prompt 0x57bec8d451f0: 1126 tokens, checkpoints: 1, 236.277 MiB
|
|
7.53.907.769 I srv update: - prompt 0x57bec8caa9b0: 1140 tokens, checkpoints: 1, 236.550 MiB
|
|
7.53.907.769 I srv update: - prompt 0x57bec0702290: 1315 tokens, checkpoints: 1, 239.970 MiB
|
|
7.53.907.770 I srv update: - prompt 0x57bec03fa660: 1180 tokens, checkpoints: 1, 237.332 MiB
|
|
7.53.907.770 I srv update: - prompt 0x57beb68c5010: 1185 tokens, checkpoints: 1, 237.430 MiB
|
|
7.53.907.771 I srv update: - prompt 0x57bec905f960: 1078 tokens, checkpoints: 1, 235.339 MiB
|
|
7.53.907.771 I srv update: - prompt 0x57bec8d43c80: 999 tokens, checkpoints: 1, 228.912 MiB
|
|
7.53.907.772 I srv update: - prompt 0x57bec8d3dce0: 873 tokens, checkpoints: 1, 201.839 MiB
|
|
7.53.907.773 I srv update: - prompt 0x57bec067b170: 1299 tokens, checkpoints: 1, 239.658 MiB
|
|
7.53.907.774 I srv get_availabl: prompt cache update took 162.78 ms
|
|
7.53.907.829 I slot launch_slot_: id 3 | task 24876 | processing task, is_child = 0
|
|
7.53.907.830 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.53.907.831 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.53.907.846 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.53.907.846 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.53.907.858 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.53.907.858 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.53.907.879 I slot update_slots: id 3 | task 24876 | Checking checkpoint with [0, 72] against 0...
|
|
7.53.914.219 W slot update_slots: id 3 | task 24876 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
7.57.091.555 I slot print_timing: id 3 | task 24876 | n_decoded = 167, tg = 55.53 t/s, tg_3s = 55.53 t/s
|
|
8.00.102.560 I slot print_timing: id 3 | task 24876 | n_decoded = 334, tg = 55.50 t/s, tg_3s = 55.46 t/s
|
|
8.03.117.723 I slot print_timing: id 3 | task 24876 | n_decoded = 501, tg = 55.46 t/s, tg_3s = 55.39 t/s
|
|
8.06.117.751 I slot print_timing: id 3 | task 24876 | n_decoded = 667, tg = 55.43 t/s, tg_3s = 55.33 t/s
|
|
8.09.133.897 I slot print_timing: id 3 | task 24876 | n_decoded = 833, tg = 55.35 t/s, tg_3s = 55.04 t/s
|
|
8.12.144.925 I slot print_timing: id 3 | task 24876 | n_decoded = 999, tg = 55.31 t/s, tg_3s = 55.13 t/s
|
|
8.14.196.892 I slot print_timing: id 3 | task 24876 | prompt eval time = 176.18 ms / 41 tokens ( 4.30 ms per token, 232.71 tokens per second)
|
|
8.14.196.896 I slot print_timing: id 3 | task 24876 | eval time = 20112.82 ms / 1113 tokens ( 18.07 ms per token, 55.34 tokens per second)
|
|
8.14.196.897 I slot print_timing: id 3 | task 24876 | total time = 20289.01 ms / 1154 tokens
|
|
8.14.196.897 I slot print_timing: id 3 | task 24876 | graphs reused = 25785
|
|
8.14.196.936 I slot release: id 3 | task 24876 | stop processing: n_tokens = 1225, truncated = 0
|
|
8.14.196.949 I srv update_slots: all slots are idle
|
|
8.14.199.396 I srv operator(): Chat format: peg-gemma4
|
|
8.14.199.734 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.633 (> 0.100 thold), f_keep = 0.062
|
|
8.14.199.736 I srv get_availabl: updating prompt cache
|
|
8.14.199.839 W srv prompt_save: - saving prompt with length 1225, total state size = 223.952 MiB (draft: 0.000 MiB)
|
|
8.14.360.389 I srv load: - looking for better prompt, base f_keep = 0.062, sim = 0.633
|
|
8.14.360.404 I srv update: - cache state: 22 prompts, 5166.795 MiB (limits: 8192.000 MiB, 16384 tokens, 44930 est)
|
|
8.14.360.405 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
8.14.360.405 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
8.14.360.406 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
8.14.360.406 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
8.14.360.407 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
8.14.360.408 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
8.14.360.408 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
8.14.360.409 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
8.14.360.409 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
8.14.360.410 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
8.14.360.410 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
8.14.360.411 I srv update: - prompt 0x57bec8c8c850: 947 tokens, checkpoints: 1, 217.739 MiB
|
|
8.14.360.411 I srv update: - prompt 0x57bec8d451f0: 1126 tokens, checkpoints: 1, 236.277 MiB
|
|
8.14.360.412 I srv update: - prompt 0x57bec8caa9b0: 1140 tokens, checkpoints: 1, 236.550 MiB
|
|
8.14.360.412 I srv update: - prompt 0x57bec0702290: 1315 tokens, checkpoints: 1, 239.970 MiB
|
|
8.14.360.413 I srv update: - prompt 0x57bec03fa660: 1180 tokens, checkpoints: 1, 237.332 MiB
|
|
8.14.360.413 I srv update: - prompt 0x57beb68c5010: 1185 tokens, checkpoints: 1, 237.430 MiB
|
|
8.14.360.414 I srv update: - prompt 0x57bec905f960: 1078 tokens, checkpoints: 1, 235.339 MiB
|
|
8.14.360.414 I srv update: - prompt 0x57bec8d43c80: 999 tokens, checkpoints: 1, 228.912 MiB
|
|
8.14.360.415 I srv update: - prompt 0x57bec8d3dce0: 873 tokens, checkpoints: 1, 201.839 MiB
|
|
8.14.360.415 I srv update: - prompt 0x57bec067b170: 1299 tokens, checkpoints: 1, 239.658 MiB
|
|
8.14.360.416 I srv update: - prompt 0x57bec7e0e5f0: 1225 tokens, checkpoints: 1, 238.211 MiB
|
|
8.14.360.417 I srv get_availabl: prompt cache update took 160.68 ms
|
|
8.14.360.480 I slot launch_slot_: id 3 | task 25992 | processing task, is_child = 0
|
|
8.14.360.480 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
8.14.360.481 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
8.14.360.499 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
8.14.360.499 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
8.14.360.525 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
8.14.360.525 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
8.14.360.558 I slot update_slots: id 3 | task 25992 | Checking checkpoint with [0, 72] against 0...
|
|
8.14.366.396 W slot update_slots: id 3 | task 25992 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
8.17.554.283 I slot print_timing: id 3 | task 25992 | n_decoded = 167, tg = 55.50 t/s, tg_3s = 55.50 t/s
|
|
8.20.563.395 I slot print_timing: id 3 | task 25992 | n_decoded = 334, tg = 55.50 t/s, tg_3s = 55.50 t/s
|
|
8.23.567.691 I slot print_timing: id 3 | task 25992 | n_decoded = 500, tg = 55.42 t/s, tg_3s = 55.25 t/s
|
|
8.26.572.476 I slot print_timing: id 3 | task 25992 | n_decoded = 666, tg = 55.37 t/s, tg_3s = 55.25 t/s
|
|
8.28.750.845 I slot print_timing: id 3 | task 25992 | prompt eval time = 184.48 ms / 48 tokens ( 3.84 ms per token, 260.19 tokens per second)
|
|
8.28.750.849 I slot print_timing: id 3 | task 25992 | eval time = 14205.80 ms / 787 tokens ( 18.05 ms per token, 55.40 tokens per second)
|
|
8.28.750.850 I slot print_timing: id 3 | task 25992 | total time = 14390.28 ms / 835 tokens
|
|
8.28.750.850 I slot print_timing: id 3 | task 25992 | graphs reused = 26567
|
|
8.28.750.891 I slot release: id 3 | task 25992 | stop processing: n_tokens = 906, truncated = 0
|
|
8.28.750.901 I srv update_slots: all slots are idle
|
|
8.28.753.509 I srv operator(): Chat format: peg-gemma4
|
|
8.28.753.861 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.724 (> 0.100 thold), f_keep = 0.084
|
|
8.28.753.863 I srv get_availabl: updating prompt cache
|
|
8.28.753.950 W srv prompt_save: - saving prompt with length 906, total state size = 194.670 MiB (draft: 0.000 MiB)
|
|
8.28.896.568 I srv load: - looking for better prompt, base f_keep = 0.084, sim = 0.724
|
|
8.28.896.584 I srv update: - cache state: 23 prompts, 5375.725 MiB (limits: 8192.000 MiB, 16384 tokens, 44564 est)
|
|
8.28.896.585 I srv update: - prompt 0x57bec0494090: 1762 tokens, checkpoints: 1, 248.706 MiB
|
|
8.28.896.586 I srv update: - prompt 0x57bec7e0bb50: 1258 tokens, checkpoints: 1, 238.856 MiB
|
|
8.28.896.587 I srv update: - prompt 0x57bec04b67e0: 778 tokens, checkpoints: 1, 181.426 MiB
|
|
8.28.896.587 I srv update: - prompt 0x57bec0681e70: 1673 tokens, checkpoints: 1, 246.967 MiB
|
|
8.28.896.588 I srv update: - prompt 0x57bebba181b0: 1948 tokens, checkpoints: 1, 252.341 MiB
|
|
8.28.896.588 I srv update: - prompt 0x57bec03ff970: 1348 tokens, checkpoints: 1, 240.615 MiB
|
|
8.28.896.589 I srv update: - prompt 0x57bec8c8c6c0: 1751 tokens, checkpoints: 1, 248.491 MiB
|
|
8.28.896.589 I srv update: - prompt 0x57bec0700d90: 1242 tokens, checkpoints: 1, 238.544 MiB
|
|
8.28.896.590 I srv update: - prompt 0x57bec7e0a9b0: 1006 tokens, checkpoints: 1, 230.416 MiB
|
|
8.28.896.590 I srv update: - prompt 0x57bec0710aa0: 1780 tokens, checkpoints: 1, 249.058 MiB
|
|
8.28.896.591 I srv update: - prompt 0x57bec0592d90: 1425 tokens, checkpoints: 1, 242.120 MiB
|
|
8.28.896.592 I srv update: - prompt 0x57bec8c8c850: 947 tokens, checkpoints: 1, 217.739 MiB
|
|
8.28.896.593 I srv update: - prompt 0x57bec8d451f0: 1126 tokens, checkpoints: 1, 236.277 MiB
|
|
8.28.896.593 I srv update: - prompt 0x57bec8caa9b0: 1140 tokens, checkpoints: 1, 236.550 MiB
|
|
8.28.896.594 I srv update: - prompt 0x57bec0702290: 1315 tokens, checkpoints: 1, 239.970 MiB
|
|
8.28.896.594 I srv update: - prompt 0x57bec03fa660: 1180 tokens, checkpoints: 1, 237.332 MiB
|
|
8.28.896.595 I srv update: - prompt 0x57beb68c5010: 1185 tokens, checkpoints: 1, 237.430 MiB
|
|
8.28.896.595 I srv update: - prompt 0x57bec905f960: 1078 tokens, checkpoints: 1, 235.339 MiB
|
|
8.28.896.596 I srv update: - prompt 0x57bec8d43c80: 999 tokens, checkpoints: 1, 228.912 MiB
|
|
8.28.896.596 I srv update: - prompt 0x57bec8d3dce0: 873 tokens, checkpoints: 1, 201.839 MiB
|
|
8.28.896.597 I srv update: - prompt 0x57bec067b170: 1299 tokens, checkpoints: 1, 239.658 MiB
|
|
8.28.896.597 I srv update: - prompt 0x57bec7e0e5f0: 1225 tokens, checkpoints: 1, 238.211 MiB
|
|
8.28.896.598 I srv update: - prompt 0x57bec7aabb80: 906 tokens, checkpoints: 1, 208.929 MiB
|
|
8.28.896.600 I srv get_availabl: prompt cache update took 142.74 ms
|
|
8.28.896.669 I slot launch_slot_: id 3 | task 26782 | processing task, is_child = 0
|
|
8.28.896.670 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
8.28.896.670 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
8.28.896.687 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
8.28.896.687 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
8.28.896.710 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
8.28.896.710 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
8.28.896.729 I slot update_slots: id 3 | task 26782 | Checking checkpoint with [0, 72] against 0...
|
|
8.28.902.525 W slot update_slots: id 3 | task 26782 | restored context checkpoint (pos_min = 0, pos_max = 72, n_tokens = 73, n_past = 72, size = 14.259 MiB)
|
|
8.32.056.519 I slot print_timing: id 3 | task 26782 | n_decoded = 167, tg = 55.60 t/s, tg_3s = 55.60 t/s
|
|
8.35.072.586 I slot print_timing: id 3 | task 26782 | n_decoded = 334, tg = 55.49 t/s, tg_3s = 55.37 t/s
|
|
8.38.077.738 I slot print_timing: id 3 | task 26782 | n_decoded = 500, tg = 55.40 t/s, tg_3s = 55.24 t/s
|
|
8.41.094.516 I slot print_timing: id 3 | task 26782 | n_decoded = 667, tg = 55.39 t/s, tg_3s = 55.36 t/s
|
|
8.44.104.970 I slot print_timing: id 3 | task 26782 | n_decoded = 834, tg = 55.41 t/s, tg_3s = 55.47 t/s
|
|
8.47.119.863 I slot print_timing: id 3 | task 26782 | n_decoded = 999, tg = 55.29 t/s, tg_3s = 54.73 t/s
|
|
8.50.126.962 I slot print_timing: id 3 | task 26782 | n_decoded = 1162, tg = 55.14 t/s, tg_3s = 54.21 t/s
|
|
8.53.132.782 I slot print_timing: id 3 | task 26782 | n_decoded = 1324, tg = 54.98 t/s, tg_3s = 53.90 t/s
|
|
8.53.595.129 I slot print_timing: id 3 | task 26782 | prompt eval time = 156.26 ms / 33 tokens ( 4.74 ms per token, 211.18 tokens per second)
|
|
8.53.595.132 I slot print_timing: id 3 | task 26782 | eval time = 24542.13 ms / 1349 tokens ( 18.19 ms per token, 54.97 tokens per second)
|
|
8.53.595.132 I slot print_timing: id 3 | task 26782 | total time = 24698.40 ms / 1382 tokens
|
|
8.53.595.133 I slot print_timing: id 3 | task 26782 | graphs reused = 27909
|
|
8.53.595.173 I slot release: id 3 | task 26782 | stop processing: n_tokens = 1453, truncated = 0
|
|
8.53.595.185 I srv update_slots: all slots are idle
|
|
8.53.595.850 I srv operator(): operator(): cleaning up before exit...
|