1107 lines
107 KiB
Plaintext
1107 lines
107 KiB
Plaintext
|
G
|
0.00.204.896 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
|
||
|
|
0.00.204.899 I device_info:
|
||
|
|
0.00.204.919 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
||
|
|
0.00.204.923 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
||
|
|
0.00.204.929 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
|
||
|
|
0.00.204.986 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
||
|
|
0.00.204.989 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
|
||
|
|
0.00.205.210 I srv init: running without SSL
|
||
|
|
0.00.205.273 I srv init: using 23 threads for HTTP server
|
||
|
|
0.00.205.601 I srv start: binding port with default address family
|
||
|
|
0.00.206.802 I srv llama_server: loading model
|
||
|
|
0.00.206.810 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/Qwen3-Next-80B-A3B-Instruct-Q4_K_S.gguf'
|
||
|
|
0.00.206.854 I common_init_result: fitting params to device memory ...
|
||
|
|
0.00.206.856 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
|
||
|
|
0.01.151.588 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
|
||
|
|
0.28.070.945 W llama_context: n_ctx_seq (8192) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
|
||
|
|
0.28.260.927 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
||
|
|
0.28.485.199 I srv load_model: initializing slots, n_slots = 4
|
||
|
|
0.28.563.604 W srv load_model: speculative decoding will use checkpoints
|
||
|
|
0.28.563.619 W common_speculative_init: no implementations specified for speculative decoding
|
||
|
|
0.28.563.621 I slot load_model: id 0 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.28.563.624 I slot load_model: id 1 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.28.563.625 I slot load_model: id 2 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.28.563.651 I slot load_model: id 3 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.28.563.691 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
|
||
|
|
0.28.563.691 I srv load_model: use `--cache-ram 0` to disable the prompt cache
|
||
|
|
0.28.563.691 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
||
|
|
0.28.563.692 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
|
||
|
|
0.28.563.751 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
|
||
|
|
0.28.569.403 I init: chat template, example_format: '<|im_start|>system
|
||
|
|
You are a helpful assistant<|im_end|>
|
||
|
|
<|im_start|>user
|
||
|
|
Hello<|im_end|>
|
||
|
|
<|im_start|>assistant
|
||
|
|
Hi there<|im_end|>
|
||
|
|
<|im_start|>user
|
||
|
|
How are you?<|im_end|>
|
||
|
|
<|im_start|>assistant
|
||
|
|
'
|
||
|
|
0.28.572.676 I srv init: init: chat template, thinking = 0
|
||
|
|
0.28.572.720 I srv llama_server: model loaded
|
||
|
|
0.28.572.724 I srv llama_server: server is listening on http://127.0.0.1:8090
|
||
|
|
0.28.572.729 I srv update_slots: all slots are idle
|
||
|
|
0.29.937.685 I srv operator(): Chat format: peg-native
|
||
|
|
0.29.937.893 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
|
||
|
|
0.29.937.896 I srv get_availabl: updating prompt cache
|
||
|
|
0.29.937.901 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
||
|
|
0.29.937.920 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
|
||
|
|
0.29.937.922 I srv get_availabl: prompt cache update took 0.03 ms
|
||
|
|
0.29.937.967 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
|
||
|
|
0.29.937.967 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.29.937.968 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.29.937.981 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.29.937.981 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.29.937.986 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.29.937.987 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.30.681.402 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 45, pos_max = 45, n_tokens = 46, size = 75.376 MiB)
|
||
|
|
0.32.194.760 I slot print_timing: id 3 | task 0 | prompt eval time = 925.86 ms / 72 tokens ( 12.86 ms per token, 77.77 tokens per second)
|
||
|
|
0.32.194.763 I slot print_timing: id 3 | task 0 | eval time = 1330.89 ms / 54 tokens ( 24.65 ms per token, 40.57 tokens per second)
|
||
|
|
0.32.194.764 I slot print_timing: id 3 | task 0 | total time = 2256.75 ms / 126 tokens
|
||
|
|
0.32.194.813 I slot print_timing: id 3 | task 0 | graphs reused = 53
|
||
|
|
0.32.194.856 I slot release: id 3 | task 0 | stop processing: n_tokens = 125, truncated = 0
|
||
|
|
0.32.194.862 I srv update_slots: all slots are idle
|
||
|
|
0.32.199.714 I srv operator(): Chat format: peg-native
|
||
|
|
0.32.199.858 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.766 (> 0.100 thold), f_keep = 0.392
|
||
|
|
0.32.199.859 I srv get_availabl: updating prompt cache
|
||
|
|
0.32.199.881 W srv prompt_save: - saving prompt with length 125, total state size = 78.307 MiB (draft: 0.000 MiB)
|
||
|
|
0.32.298.561 I srv load: - looking for better prompt, base f_keep = 0.392, sim = 0.766
|
||
|
|
0.32.298.567 I srv update: - cache state: 1 prompts, 153.683 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.32.298.568 I srv update: - prompt 0x64c11b82b730: 125 tokens, checkpoints: 1, 153.683 MiB
|
||
|
|
0.32.298.569 I srv get_availabl: prompt cache update took 98.71 ms
|
||
|
|
0.32.298.639 I slot launch_slot_: id 3 | task 57 | processing task, is_child = 0
|
||
|
|
0.32.298.640 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.32.298.641 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.32.298.649 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.32.298.649 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.32.298.659 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.32.298.660 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.32.298.702 I slot update_slots: id 3 | task 57 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.32.317.682 W slot update_slots: id 3 | task 57 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.32.955.346 I slot print_timing: id 3 | task 57 | prompt eval time = 148.81 ms / 18 tokens ( 8.27 ms per token, 120.96 tokens per second)
|
||
|
|
0.32.955.350 I slot print_timing: id 3 | task 57 | eval time = 507.85 ms / 21 tokens ( 24.18 ms per token, 41.35 tokens per second)
|
||
|
|
0.32.955.351 I slot print_timing: id 3 | task 57 | total time = 656.66 ms / 39 tokens
|
||
|
|
0.32.955.351 I slot print_timing: id 3 | task 57 | graphs reused = 72
|
||
|
|
0.32.955.373 I slot release: id 3 | task 57 | stop processing: n_tokens = 84, truncated = 0
|
||
|
|
0.32.955.380 I srv update_slots: all slots are idle
|
||
|
|
0.32.960.328 I srv operator(): Chat format: peg-native
|
||
|
|
0.32.960.469 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.583
|
||
|
|
0.32.960.504 I slot launch_slot_: id 3 | task 80 | processing task, is_child = 0
|
||
|
|
0.32.960.506 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.32.960.506 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.32.960.514 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.32.960.515 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.32.960.547 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.32.960.548 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.32.960.564 I slot update_slots: id 3 | task 80 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.32.979.853 W slot update_slots: id 3 | task 80 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.34.188.707 I slot print_timing: id 3 | task 80 | prompt eval time = 154.11 ms / 20 tokens ( 7.71 ms per token, 129.77 tokens per second)
|
||
|
|
0.34.188.711 I slot print_timing: id 3 | task 80 | eval time = 1074.02 ms / 44 tokens ( 24.41 ms per token, 40.97 tokens per second)
|
||
|
|
0.34.188.711 I slot print_timing: id 3 | task 80 | total time = 1228.14 ms / 64 tokens
|
||
|
|
0.34.188.712 I slot print_timing: id 3 | task 80 | graphs reused = 114
|
||
|
|
0.34.188.731 I slot release: id 3 | task 80 | stop processing: n_tokens = 109, truncated = 0
|
||
|
|
0.34.188.738 I srv update_slots: all slots are idle
|
||
|
|
0.34.193.944 I srv operator(): Chat format: peg-native
|
||
|
|
0.34.194.087 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.769 (> 0.100 thold), f_keep = 0.459
|
||
|
|
0.34.194.090 I srv get_availabl: updating prompt cache
|
||
|
|
0.34.194.110 W srv prompt_save: - saving prompt with length 109, total state size = 77.932 MiB (draft: 0.000 MiB)
|
||
|
|
0.34.304.655 I srv load: - looking for better prompt, base f_keep = 0.459, sim = 0.769
|
||
|
|
0.34.304.691 I srv update: - cache state: 2 prompts, 306.991 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.34.304.693 I srv update: - prompt 0x64c11b82b730: 125 tokens, checkpoints: 1, 153.683 MiB
|
||
|
|
0.34.304.694 I srv update: - prompt 0x64c122d4b6d0: 109 tokens, checkpoints: 1, 153.308 MiB
|
||
|
|
0.34.304.695 I srv get_availabl: prompt cache update took 110.61 ms
|
||
|
|
0.34.304.784 I slot launch_slot_: id 3 | task 126 | processing task, is_child = 0
|
||
|
|
0.34.304.785 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.34.304.786 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.34.304.797 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.34.304.798 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.34.304.805 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.34.304.806 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.34.304.823 I slot update_slots: id 3 | task 126 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.34.323.593 W slot update_slots: id 3 | task 126 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.35.531.603 I slot print_timing: id 3 | task 126 | prompt eval time = 152.16 ms / 19 tokens ( 8.01 ms per token, 124.87 tokens per second)
|
||
|
|
0.35.531.606 I slot print_timing: id 3 | task 126 | eval time = 1074.62 ms / 44 tokens ( 24.42 ms per token, 40.94 tokens per second)
|
||
|
|
0.35.531.638 I slot print_timing: id 3 | task 126 | total time = 1226.77 ms / 63 tokens
|
||
|
|
0.35.531.640 I slot print_timing: id 3 | task 126 | graphs reused = 156
|
||
|
|
0.35.531.662 I slot release: id 3 | task 126 | stop processing: n_tokens = 108, truncated = 0
|
||
|
|
0.35.531.668 I srv update_slots: all slots are idle
|
||
|
|
0.35.536.485 I srv operator(): Chat format: peg-native
|
||
|
|
0.35.536.635 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.746 (> 0.100 thold), f_keep = 0.463
|
||
|
|
0.35.536.636 I srv get_availabl: updating prompt cache
|
||
|
|
0.35.536.655 W srv prompt_save: - saving prompt with length 108, total state size = 77.909 MiB (draft: 0.000 MiB)
|
||
|
|
0.35.638.632 I srv load: - looking for better prompt, base f_keep = 0.463, sim = 0.746
|
||
|
|
0.35.638.637 I srv load: - found better prompt with f_keep = 0.477, sim = 0.776
|
||
|
|
0.35.675.578 I srv update: - cache state: 2 prompts, 306.968 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.35.675.583 I srv update: - prompt 0x64c11b82b730: 125 tokens, checkpoints: 1, 153.683 MiB
|
||
|
|
0.35.675.583 I srv update: - prompt 0x64c11b655f80: 108 tokens, checkpoints: 1, 153.284 MiB
|
||
|
|
0.35.675.585 I srv get_availabl: prompt cache update took 138.95 ms
|
||
|
|
0.35.675.655 I slot launch_slot_: id 3 | task 172 | processing task, is_child = 0
|
||
|
|
0.35.675.688 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.35.675.688 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.35.675.697 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.35.675.697 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.35.675.702 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.35.675.702 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.35.675.716 I slot update_slots: id 3 | task 172 | Checking checkpoint with [45, 45] against 52...
|
||
|
|
0.35.711.561 W slot update_slots: id 3 | task 172 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.37.670.795 I slot print_timing: id 3 | task 172 | prompt eval time = 203.37 ms / 21 tokens ( 9.68 ms per token, 103.26 tokens per second)
|
||
|
|
0.37.670.798 I slot print_timing: id 3 | task 172 | eval time = 1791.71 ms / 73 tokens ( 24.54 ms per token, 40.74 tokens per second)
|
||
|
|
0.37.670.799 I slot print_timing: id 3 | task 172 | total time = 1995.08 ms / 94 tokens
|
||
|
|
0.37.670.799 I slot print_timing: id 3 | task 172 | graphs reused = 227
|
||
|
|
0.37.670.825 I slot release: id 3 | task 172 | stop processing: n_tokens = 139, truncated = 0
|
||
|
|
0.37.670.830 I srv update_slots: all slots are idle
|
||
|
|
0.37.675.608 I srv operator(): Chat format: peg-native
|
||
|
|
0.37.675.779 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.353
|
||
|
|
0.37.675.781 I srv get_availabl: updating prompt cache
|
||
|
|
0.37.675.804 W srv prompt_save: - saving prompt with length 139, total state size = 78.636 MiB (draft: 0.000 MiB)
|
||
|
|
0.37.771.154 I srv load: - looking for better prompt, base f_keep = 0.353, sim = 0.731
|
||
|
|
0.37.771.161 I srv update: - cache state: 3 prompts, 460.979 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.37.771.162 I srv update: - prompt 0x64c11b82b730: 125 tokens, checkpoints: 1, 153.683 MiB
|
||
|
|
0.37.771.162 I srv update: - prompt 0x64c11b655f80: 108 tokens, checkpoints: 1, 153.284 MiB
|
||
|
|
0.37.771.163 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.37.771.164 I srv get_availabl: prompt cache update took 95.38 ms
|
||
|
|
0.37.771.233 I slot launch_slot_: id 3 | task 247 | processing task, is_child = 0
|
||
|
|
0.37.771.234 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.771.235 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.771.242 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.771.243 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.771.247 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.771.278 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.771.292 I slot update_slots: id 3 | task 247 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.37.790.086 W slot update_slots: id 3 | task 247 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.38.613.979 I slot print_timing: id 3 | task 247 | prompt eval time = 188.39 ms / 21 tokens ( 8.97 ms per token, 111.47 tokens per second)
|
||
|
|
0.38.613.982 I slot print_timing: id 3 | task 247 | eval time = 654.30 ms / 27 tokens ( 24.23 ms per token, 41.27 tokens per second)
|
||
|
|
0.38.613.983 I slot print_timing: id 3 | task 247 | total time = 842.68 ms / 48 tokens
|
||
|
|
0.38.613.984 I slot print_timing: id 3 | task 247 | graphs reused = 252
|
||
|
|
0.38.614.008 I slot release: id 3 | task 247 | stop processing: n_tokens = 93, truncated = 0
|
||
|
|
0.38.614.014 I srv update_slots: all slots are idle
|
||
|
|
0.38.618.921 I srv operator(): Chat format: peg-native
|
||
|
|
0.38.619.056 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.690 (> 0.100 thold), f_keep = 0.527
|
||
|
|
0.38.619.106 I slot launch_slot_: id 3 | task 276 | processing task, is_child = 0
|
||
|
|
0.38.619.108 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.38.619.109 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.38.619.116 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.38.619.147 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.38.619.153 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.38.619.153 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.38.619.161 I slot update_slots: id 3 | task 276 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.38.638.369 W slot update_slots: id 3 | task 276 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.39.595.966 I slot print_timing: id 3 | task 276 | prompt eval time = 196.74 ms / 25 tokens ( 7.87 ms per token, 127.07 tokens per second)
|
||
|
|
0.39.595.970 I slot print_timing: id 3 | task 276 | eval time = 780.06 ms / 32 tokens ( 24.38 ms per token, 41.02 tokens per second)
|
||
|
|
0.39.595.970 I slot print_timing: id 3 | task 276 | total time = 976.80 ms / 57 tokens
|
||
|
|
0.39.595.971 I slot print_timing: id 3 | task 276 | graphs reused = 282
|
||
|
|
0.39.595.996 I slot release: id 3 | task 276 | stop processing: n_tokens = 102, truncated = 0
|
||
|
|
0.39.596.002 I srv update_slots: all slots are idle
|
||
|
|
0.39.601.031 I srv operator(): Chat format: peg-native
|
||
|
|
0.39.601.171 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.820 (> 0.100 thold), f_keep = 0.490
|
||
|
|
0.39.601.173 I srv get_availabl: updating prompt cache
|
||
|
|
0.39.601.189 W srv prompt_save: - saving prompt with length 102, total state size = 77.768 MiB (draft: 0.000 MiB)
|
||
|
|
0.39.697.159 I srv load: - looking for better prompt, base f_keep = 0.490, sim = 0.820
|
||
|
|
0.39.697.166 I srv update: - cache state: 4 prompts, 614.123 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.39.697.167 I srv update: - prompt 0x64c11b82b730: 125 tokens, checkpoints: 1, 153.683 MiB
|
||
|
|
0.39.697.168 I srv update: - prompt 0x64c11b655f80: 108 tokens, checkpoints: 1, 153.284 MiB
|
||
|
|
0.39.697.169 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.39.697.169 I srv update: - prompt 0x64c11b77ce30: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
0.39.697.170 I srv get_availabl: prompt cache update took 96.00 ms
|
||
|
|
0.39.697.244 I slot launch_slot_: id 3 | task 310 | processing task, is_child = 0
|
||
|
|
0.39.697.245 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.39.697.246 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.39.697.253 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.39.697.253 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.39.697.257 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.39.697.257 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.39.697.270 I slot update_slots: id 3 | task 310 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.39.716.158 W slot update_slots: id 3 | task 310 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.41.459.426 I slot print_timing: id 3 | task 310 | prompt eval time = 144.71 ms / 15 tokens ( 9.65 ms per token, 103.65 tokens per second)
|
||
|
|
0.41.459.429 I slot print_timing: id 3 | task 310 | eval time = 1617.44 ms / 66 tokens ( 24.51 ms per token, 40.81 tokens per second)
|
||
|
|
0.41.459.430 I slot print_timing: id 3 | task 310 | total time = 1762.15 ms / 81 tokens
|
||
|
|
0.41.459.430 I slot print_timing: id 3 | task 310 | graphs reused = 346
|
||
|
|
0.41.459.457 I slot release: id 3 | task 310 | stop processing: n_tokens = 126, truncated = 0
|
||
|
|
0.41.459.463 I srv update_slots: all slots are idle
|
||
|
|
0.41.464.325 I srv operator(): Chat format: peg-native
|
||
|
|
0.41.464.465 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.671 (> 0.100 thold), f_keep = 0.389
|
||
|
|
0.41.464.467 I srv get_availabl: updating prompt cache
|
||
|
|
0.41.464.488 W srv prompt_save: - saving prompt with length 126, total state size = 78.331 MiB (draft: 0.000 MiB)
|
||
|
|
0.41.565.536 I srv load: - looking for better prompt, base f_keep = 0.389, sim = 0.671
|
||
|
|
0.41.565.544 I srv update: - cache state: 5 prompts, 767.829 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.41.565.545 I srv update: - prompt 0x64c11b82b730: 125 tokens, checkpoints: 1, 153.683 MiB
|
||
|
|
0.41.565.546 I srv update: - prompt 0x64c11b655f80: 108 tokens, checkpoints: 1, 153.284 MiB
|
||
|
|
0.41.565.582 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.41.565.584 I srv update: - prompt 0x64c11b77ce30: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
0.41.565.604 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
0.41.565.606 I srv get_availabl: prompt cache update took 101.14 ms
|
||
|
|
0.41.565.686 I slot launch_slot_: id 3 | task 378 | processing task, is_child = 0
|
||
|
|
0.41.565.688 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.41.565.688 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.41.565.696 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.41.565.696 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.41.565.707 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.41.565.707 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.41.565.720 I slot update_slots: id 3 | task 378 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.41.584.572 W slot update_slots: id 3 | task 378 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.42.468.601 I slot print_timing: id 3 | task 378 | prompt eval time = 196.98 ms / 27 tokens ( 7.30 ms per token, 137.07 tokens per second)
|
||
|
|
0.42.468.604 I slot print_timing: id 3 | task 378 | eval time = 705.90 ms / 29 tokens ( 24.34 ms per token, 41.08 tokens per second)
|
||
|
|
0.42.468.636 I slot print_timing: id 3 | task 378 | total time = 902.88 ms / 56 tokens
|
||
|
|
0.42.468.637 I slot print_timing: id 3 | task 378 | graphs reused = 373
|
||
|
|
0.42.468.661 I slot release: id 3 | task 378 | stop processing: n_tokens = 101, truncated = 0
|
||
|
|
0.42.468.667 I srv update_slots: all slots are idle
|
||
|
|
0.42.473.314 I srv operator(): Chat format: peg-native
|
||
|
|
0.42.473.465 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.485
|
||
|
|
0.42.473.466 I srv get_availabl: updating prompt cache
|
||
|
|
0.42.473.485 W srv prompt_save: - saving prompt with length 101, total state size = 77.744 MiB (draft: 0.000 MiB)
|
||
|
|
0.42.566.455 I srv load: - looking for better prompt, base f_keep = 0.485, sim = 0.742
|
||
|
|
0.42.566.460 I srv load: - found better prompt with f_keep = 0.490, sim = 0.758
|
||
|
|
0.42.604.442 I srv update: - cache state: 5 prompts, 767.806 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.42.604.447 I srv update: - prompt 0x64c11b82b730: 125 tokens, checkpoints: 1, 153.683 MiB
|
||
|
|
0.42.604.447 I srv update: - prompt 0x64c11b655f80: 108 tokens, checkpoints: 1, 153.284 MiB
|
||
|
|
0.42.604.448 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.42.604.449 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
0.42.604.480 I srv update: - prompt 0x64c11b83cc20: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
0.42.604.483 I srv get_availabl: prompt cache update took 131.01 ms
|
||
|
|
0.42.604.552 I slot launch_slot_: id 3 | task 409 | processing task, is_child = 0
|
||
|
|
0.42.604.563 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.42.604.564 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.42.604.572 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.42.604.573 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.42.604.578 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.42.604.579 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.42.604.593 I slot update_slots: id 3 | task 409 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.42.638.539 W slot update_slots: id 3 | task 409 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.44.540.922 I slot print_timing: id 3 | task 409 | prompt eval time = 170.05 ms / 20 tokens ( 8.50 ms per token, 117.61 tokens per second)
|
||
|
|
0.44.540.925 I slot print_timing: id 3 | task 409 | eval time = 1766.27 ms / 72 tokens ( 24.53 ms per token, 40.76 tokens per second)
|
||
|
|
0.44.540.926 I slot print_timing: id 3 | task 409 | total time = 1936.32 ms / 92 tokens
|
||
|
|
0.44.540.927 I slot print_timing: id 3 | task 409 | graphs reused = 443
|
||
|
|
0.44.541.009 I slot release: id 3 | task 409 | stop processing: n_tokens = 137, truncated = 0
|
||
|
|
0.44.541.018 I srv update_slots: all slots are idle
|
||
|
|
0.44.545.669 I srv operator(): Chat format: peg-native
|
||
|
|
0.44.545.808 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.681 (> 0.100 thold), f_keep = 0.358
|
||
|
|
0.44.545.810 I srv get_availabl: updating prompt cache
|
||
|
|
0.44.545.829 W srv prompt_save: - saving prompt with length 137, total state size = 78.589 MiB (draft: 0.000 MiB)
|
||
|
|
0.44.650.152 I srv load: - looking for better prompt, base f_keep = 0.358, sim = 0.681
|
||
|
|
0.44.650.161 I srv update: - cache state: 6 prompts, 921.770 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.44.650.162 I srv update: - prompt 0x64c11b82b730: 125 tokens, checkpoints: 1, 153.683 MiB
|
||
|
|
0.44.650.162 I srv update: - prompt 0x64c11b655f80: 108 tokens, checkpoints: 1, 153.284 MiB
|
||
|
|
0.44.650.163 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.44.650.164 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
0.44.650.164 I srv update: - prompt 0x64c11b83cc20: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
0.44.650.165 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
0.44.650.166 I srv get_availabl: prompt cache update took 104.35 ms
|
||
|
|
0.44.650.265 I slot launch_slot_: id 3 | task 483 | processing task, is_child = 0
|
||
|
|
0.44.650.267 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.44.650.268 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.44.650.275 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.44.650.276 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.44.650.280 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.44.650.281 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.44.650.296 I slot update_slots: id 3 | task 483 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.44.669.035 W slot update_slots: id 3 | task 483 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.46.121.532 I slot print_timing: id 3 | task 483 | prompt eval time = 198.00 ms / 26 tokens ( 7.62 ms per token, 131.32 tokens per second)
|
||
|
|
0.46.121.536 I slot print_timing: id 3 | task 483 | eval time = 1273.23 ms / 52 tokens ( 24.49 ms per token, 40.84 tokens per second)
|
||
|
|
0.46.121.536 I slot print_timing: id 3 | task 483 | total time = 1471.23 ms / 78 tokens
|
||
|
|
0.46.121.538 I slot print_timing: id 3 | task 483 | graphs reused = 493
|
||
|
|
0.46.121.572 I slot release: id 3 | task 483 | stop processing: n_tokens = 123, truncated = 0
|
||
|
|
0.46.121.580 I srv update_slots: all slots are idle
|
||
|
|
0.46.126.365 I srv operator(): Chat format: peg-native
|
||
|
|
0.46.126.535 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.398
|
||
|
|
0.46.126.537 I srv get_availabl: updating prompt cache
|
||
|
|
0.46.126.553 W srv prompt_save: - saving prompt with length 123, total state size = 78.260 MiB (draft: 0.000 MiB)
|
||
|
|
0.46.224.281 I srv load: - looking for better prompt, base f_keep = 0.398, sim = 0.731
|
||
|
|
0.46.224.287 I srv load: - found better prompt with f_keep = 0.400, sim = 0.746
|
||
|
|
0.46.262.405 I srv update: - cache state: 6 prompts, 921.723 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.46.262.408 I srv update: - prompt 0x64c11b655f80: 108 tokens, checkpoints: 1, 153.284 MiB
|
||
|
|
0.46.262.409 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.46.262.409 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
0.46.262.410 I srv update: - prompt 0x64c11b83cc20: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
0.46.262.410 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
0.46.262.411 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
0.46.262.453 I srv get_availabl: prompt cache update took 135.88 ms
|
||
|
|
0.46.262.530 I slot launch_slot_: id 3 | task 537 | processing task, is_child = 0
|
||
|
|
0.46.262.532 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.262.540 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.262.547 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.262.548 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.262.552 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.262.552 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.262.575 I slot update_slots: id 3 | task 537 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.46.298.019 W slot update_slots: id 3 | task 537 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.47.618.184 I slot print_timing: id 3 | task 537 | prompt eval time = 204.33 ms / 21 tokens ( 9.73 ms per token, 102.78 tokens per second)
|
||
|
|
0.47.618.187 I slot print_timing: id 3 | task 537 | eval time = 1151.27 ms / 47 tokens ( 24.50 ms per token, 40.82 tokens per second)
|
||
|
|
0.47.618.188 I slot print_timing: id 3 | task 537 | total time = 1355.60 ms / 68 tokens
|
||
|
|
0.47.618.188 I slot print_timing: id 3 | task 537 | graphs reused = 538
|
||
|
|
0.47.618.214 I slot release: id 3 | task 537 | stop processing: n_tokens = 113, truncated = 0
|
||
|
|
0.47.618.219 I srv update_slots: all slots are idle
|
||
|
|
0.47.622.897 I srv operator(): Chat format: peg-native
|
||
|
|
0.47.623.065 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.434
|
||
|
|
0.47.623.067 I srv get_availabl: updating prompt cache
|
||
|
|
0.47.623.087 W srv prompt_save: - saving prompt with length 113, total state size = 78.026 MiB (draft: 0.000 MiB)
|
||
|
|
0.47.736.065 I srv load: - looking for better prompt, base f_keep = 0.434, sim = 0.721
|
||
|
|
0.47.736.074 I srv update: - cache state: 7 prompts, 1075.125 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.47.736.075 I srv update: - prompt 0x64c11b655f80: 108 tokens, checkpoints: 1, 153.284 MiB
|
||
|
|
0.47.736.075 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.47.736.076 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
0.47.736.077 I srv update: - prompt 0x64c11b83cc20: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
0.47.736.078 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
0.47.736.082 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
0.47.736.083 I srv update: - prompt 0x64c122e3d390: 113 tokens, checkpoints: 1, 153.402 MiB
|
||
|
|
0.47.736.084 I srv get_availabl: prompt cache update took 113.02 ms
|
||
|
|
0.47.736.157 I slot launch_slot_: id 3 | task 586 | processing task, is_child = 0
|
||
|
|
0.47.736.158 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.47.736.158 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.47.736.194 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.47.736.196 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.47.736.200 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.47.736.200 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.47.736.214 I slot update_slots: id 3 | task 586 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.47.755.074 W slot update_slots: id 3 | task 586 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.48.360.792 I slot print_timing: id 3 | task 586 | prompt eval time = 190.24 ms / 22 tokens ( 8.65 ms per token, 115.64 tokens per second)
|
||
|
|
0.48.360.795 I slot print_timing: id 3 | task 586 | eval time = 434.33 ms / 18 tokens ( 24.13 ms per token, 41.44 tokens per second)
|
||
|
|
0.48.360.796 I slot print_timing: id 3 | task 586 | total time = 624.57 ms / 40 tokens
|
||
|
|
0.48.360.796 I slot print_timing: id 3 | task 586 | graphs reused = 554
|
||
|
|
0.48.360.823 I slot release: id 3 | task 586 | stop processing: n_tokens = 85, truncated = 0
|
||
|
|
0.48.360.829 I srv update_slots: all slots are idle
|
||
|
|
0.48.365.590 I srv operator(): Chat format: peg-native
|
||
|
|
0.48.365.738 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.576
|
||
|
|
0.48.365.777 I slot launch_slot_: id 3 | task 606 | processing task, is_child = 0
|
||
|
|
0.48.365.808 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.48.365.810 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.48.365.816 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.48.365.817 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.48.365.822 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.48.365.823 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.48.365.832 I slot update_slots: id 3 | task 606 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.48.385.041 W slot update_slots: id 3 | task 606 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.49.752.492 I slot print_timing: id 3 | task 606 | prompt eval time = 185.28 ms / 21 tokens ( 8.82 ms per token, 113.34 tokens per second)
|
||
|
|
0.49.752.495 I slot print_timing: id 3 | task 606 | eval time = 1201.38 ms / 49 tokens ( 24.52 ms per token, 40.79 tokens per second)
|
||
|
|
0.49.752.496 I slot print_timing: id 3 | task 606 | total time = 1386.65 ms / 70 tokens
|
||
|
|
0.49.752.496 I slot print_timing: id 3 | task 606 | graphs reused = 601
|
||
|
|
0.49.752.523 I slot release: id 3 | task 606 | stop processing: n_tokens = 115, truncated = 0
|
||
|
|
0.49.752.531 I srv update_slots: all slots are idle
|
||
|
|
0.49.757.440 I srv operator(): Chat format: peg-native
|
||
|
|
0.49.757.616 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.426
|
||
|
|
0.49.757.618 I srv get_availabl: updating prompt cache
|
||
|
|
0.49.757.637 W srv prompt_save: - saving prompt with length 115, total state size = 78.073 MiB (draft: 0.000 MiB)
|
||
|
|
0.49.851.652 I srv load: - looking for better prompt, base f_keep = 0.426, sim = 0.710
|
||
|
|
0.49.851.657 I srv load: - found better prompt with f_keep = 0.463, sim = 0.725
|
||
|
|
0.49.888.604 I srv update: - cache state: 7 prompts, 1075.289 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.49.888.608 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.49.888.609 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
0.49.888.610 I srv update: - prompt 0x64c11b83cc20: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
0.49.888.611 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
0.49.888.611 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
0.49.888.612 I srv update: - prompt 0x64c122e3d390: 113 tokens, checkpoints: 1, 153.402 MiB
|
||
|
|
0.49.888.613 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
0.49.888.614 I srv get_availabl: prompt cache update took 131.00 ms
|
||
|
|
0.49.888.693 I slot launch_slot_: id 3 | task 657 | processing task, is_child = 0
|
||
|
|
0.49.888.727 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.49.888.728 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.49.888.736 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.49.888.737 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.49.888.742 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.49.888.742 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.49.888.758 I slot update_slots: id 3 | task 657 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.49.914.365 W slot update_slots: id 3 | task 657 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.51.091.940 I slot print_timing: id 3 | task 657 | prompt eval time = 199.13 ms / 23 tokens ( 8.66 ms per token, 115.50 tokens per second)
|
||
|
|
0.51.091.943 I slot print_timing: id 3 | task 657 | eval time = 1004.05 ms / 41 tokens ( 24.49 ms per token, 40.83 tokens per second)
|
||
|
|
0.51.091.944 I slot print_timing: id 3 | task 657 | total time = 1203.18 ms / 64 tokens
|
||
|
|
0.51.091.945 I slot print_timing: id 3 | task 657 | graphs reused = 640
|
||
|
|
0.51.091.971 I slot release: id 3 | task 657 | stop processing: n_tokens = 109, truncated = 0
|
||
|
|
0.51.091.976 I srv update_slots: all slots are idle
|
||
|
|
0.51.096.841 I srv operator(): Chat format: peg-native
|
||
|
|
0.51.097.010 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.450
|
||
|
|
0.51.097.012 I srv get_availabl: updating prompt cache
|
||
|
|
0.51.097.034 W srv prompt_save: - saving prompt with length 109, total state size = 77.932 MiB (draft: 0.000 MiB)
|
||
|
|
0.51.196.810 I srv load: - looking for better prompt, base f_keep = 0.450, sim = 0.742
|
||
|
|
0.51.196.815 I srv load: - found better prompt with f_keep = 0.495, sim = 0.758
|
||
|
|
0.51.235.093 I srv update: - cache state: 7 prompts, 1075.477 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.51.235.097 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.51.235.098 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
0.51.235.098 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
0.51.235.099 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
0.51.235.099 I srv update: - prompt 0x64c122e3d390: 113 tokens, checkpoints: 1, 153.402 MiB
|
||
|
|
0.51.235.100 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
0.51.235.100 I srv update: - prompt 0x64c11b476fa0: 109 tokens, checkpoints: 1, 153.308 MiB
|
||
|
|
0.51.235.102 I srv get_availabl: prompt cache update took 138.09 ms
|
||
|
|
0.51.235.176 I slot launch_slot_: id 3 | task 700 | processing task, is_child = 0
|
||
|
|
0.51.235.205 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.51.235.207 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.51.235.213 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.51.235.213 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.51.235.218 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.51.235.218 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.51.235.231 I slot update_slots: id 3 | task 700 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.51.259.629 W slot update_slots: id 3 | task 700 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.52.495.993 I slot print_timing: id 3 | task 700 | prompt eval time = 160.02 ms / 20 tokens ( 8.00 ms per token, 124.98 tokens per second)
|
||
|
|
0.52.495.997 I slot print_timing: id 3 | task 700 | eval time = 1100.74 ms / 45 tokens ( 24.46 ms per token, 40.88 tokens per second)
|
||
|
|
0.52.495.998 I slot print_timing: id 3 | task 700 | total time = 1260.76 ms / 65 tokens
|
||
|
|
0.52.495.998 I slot print_timing: id 3 | task 700 | graphs reused = 683
|
||
|
|
0.52.496.022 I slot release: id 3 | task 700 | stop processing: n_tokens = 110, truncated = 0
|
||
|
|
0.52.496.029 I srv update_slots: all slots are idle
|
||
|
|
0.52.500.800 I srv operator(): Chat format: peg-native
|
||
|
|
0.52.500.967 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.445
|
||
|
|
0.52.500.969 I srv get_availabl: updating prompt cache
|
||
|
|
0.52.500.988 W srv prompt_save: - saving prompt with length 110, total state size = 77.956 MiB (draft: 0.000 MiB)
|
||
|
|
0.52.595.157 I srv load: - looking for better prompt, base f_keep = 0.445, sim = 0.710
|
||
|
|
0.52.595.163 I srv load: - found better prompt with f_keep = 0.459, sim = 0.725
|
||
|
|
0.52.633.078 I srv update: - cache state: 7 prompts, 1075.500 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.52.633.081 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.52.633.082 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
0.52.633.082 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
0.52.633.083 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
0.52.633.083 I srv update: - prompt 0x64c122e3d390: 113 tokens, checkpoints: 1, 153.402 MiB
|
||
|
|
0.52.633.084 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
0.52.633.084 I srv update: - prompt 0x64c122d4e2e0: 110 tokens, checkpoints: 1, 153.331 MiB
|
||
|
|
0.52.633.085 I srv get_availabl: prompt cache update took 132.12 ms
|
||
|
|
0.52.633.157 I slot launch_slot_: id 3 | task 747 | processing task, is_child = 0
|
||
|
|
0.52.633.188 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.52.633.190 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.52.633.198 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.52.633.198 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.52.633.203 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.52.633.203 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.52.633.217 I slot update_slots: id 3 | task 747 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.52.659.877 W slot update_slots: id 3 | task 747 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.53.808.393 I slot print_timing: id 3 | task 747 | prompt eval time = 197.20 ms / 23 tokens ( 8.57 ms per token, 116.63 tokens per second)
|
||
|
|
0.53.808.396 I slot print_timing: id 3 | task 747 | eval time = 977.97 ms / 40 tokens ( 24.45 ms per token, 40.90 tokens per second)
|
||
|
|
0.53.808.397 I slot print_timing: id 3 | task 747 | total time = 1175.17 ms / 63 tokens
|
||
|
|
0.53.808.397 I slot print_timing: id 3 | task 747 | graphs reused = 721
|
||
|
|
0.53.808.425 I slot release: id 3 | task 747 | stop processing: n_tokens = 108, truncated = 0
|
||
|
|
0.53.808.431 I srv update_slots: all slots are idle
|
||
|
|
0.53.813.018 I srv operator(): Chat format: peg-native
|
||
|
|
0.53.813.191 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.766 (> 0.100 thold), f_keep = 0.454
|
||
|
|
0.53.813.193 I srv get_availabl: updating prompt cache
|
||
|
|
0.53.813.212 W srv prompt_save: - saving prompt with length 108, total state size = 77.909 MiB (draft: 0.000 MiB)
|
||
|
|
0.53.904.354 I srv load: - looking for better prompt, base f_keep = 0.454, sim = 0.766
|
||
|
|
0.53.904.360 I srv load: - found better prompt with f_keep = 0.455, sim = 0.781
|
||
|
|
0.53.941.611 I srv update: - cache state: 7 prompts, 1075.453 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.53.941.615 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.53.941.616 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
0.53.941.617 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
0.53.941.618 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
0.53.941.619 I srv update: - prompt 0x64c122e3d390: 113 tokens, checkpoints: 1, 153.402 MiB
|
||
|
|
0.53.941.619 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
0.53.941.620 I srv update: - prompt 0x64c11b5761c0: 108 tokens, checkpoints: 1, 153.284 MiB
|
||
|
|
0.53.941.622 I srv get_availabl: prompt cache update took 128.43 ms
|
||
|
|
0.53.941.724 I slot launch_slot_: id 3 | task 789 | processing task, is_child = 0
|
||
|
|
0.53.941.761 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.53.941.763 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.53.941.795 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.53.941.796 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.53.941.802 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.53.941.802 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.53.941.817 I slot update_slots: id 3 | task 789 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.53.970.495 W slot update_slots: id 3 | task 789 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.55.403.858 I slot print_timing: id 3 | task 789 | prompt eval time = 158.17 ms / 18 tokens ( 8.79 ms per token, 113.80 tokens per second)
|
||
|
|
0.55.403.861 I slot print_timing: id 3 | task 789 | eval time = 1303.87 ms / 53 tokens ( 24.60 ms per token, 40.65 tokens per second)
|
||
|
|
0.55.403.862 I slot print_timing: id 3 | task 789 | total time = 1462.04 ms / 71 tokens
|
||
|
|
0.55.403.863 I slot print_timing: id 3 | task 789 | graphs reused = 772
|
||
|
|
0.55.403.889 I slot release: id 3 | task 789 | stop processing: n_tokens = 116, truncated = 0
|
||
|
|
0.55.403.895 I srv update_slots: all slots are idle
|
||
|
|
0.55.408.542 I srv operator(): Chat format: peg-native
|
||
|
|
0.55.408.724 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.778 (> 0.100 thold), f_keep = 0.422
|
||
|
|
0.55.408.726 I srv get_availabl: updating prompt cache
|
||
|
|
0.55.408.743 W srv prompt_save: - saving prompt with length 116, total state size = 78.096 MiB (draft: 0.000 MiB)
|
||
|
|
0.55.501.822 I srv load: - looking for better prompt, base f_keep = 0.422, sim = 0.778
|
||
|
|
0.55.501.827 I srv load: - found better prompt with f_keep = 0.442, sim = 0.794
|
||
|
|
0.55.540.180 I srv update: - cache state: 7 prompts, 1075.524 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.55.540.184 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.55.540.184 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
0.55.540.185 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
0.55.540.185 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
0.55.540.186 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
0.55.540.186 I srv update: - prompt 0x64c11b5761c0: 108 tokens, checkpoints: 1, 153.284 MiB
|
||
|
|
0.55.540.187 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
0.55.540.189 I srv get_availabl: prompt cache update took 131.46 ms
|
||
|
|
0.55.540.264 I slot launch_slot_: id 3 | task 844 | processing task, is_child = 0
|
||
|
|
0.55.540.265 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.55.540.266 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.55.540.273 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.55.540.274 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.55.540.278 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.55.540.278 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.55.540.293 I slot update_slots: id 3 | task 844 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.55.567.777 W slot update_slots: id 3 | task 844 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.56.799.607 I slot print_timing: id 3 | task 844 | prompt eval time = 155.23 ms / 17 tokens ( 9.13 ms per token, 109.52 tokens per second)
|
||
|
|
0.56.799.611 I slot print_timing: id 3 | task 844 | eval time = 1104.08 ms / 45 tokens ( 24.54 ms per token, 40.76 tokens per second)
|
||
|
|
0.56.799.612 I slot print_timing: id 3 | task 844 | total time = 1259.31 ms / 62 tokens
|
||
|
|
0.56.799.613 I slot print_timing: id 3 | task 844 | graphs reused = 815
|
||
|
|
0.56.799.663 I slot release: id 3 | task 844 | stop processing: n_tokens = 107, truncated = 0
|
||
|
|
0.56.799.669 I srv update_slots: all slots are idle
|
||
|
|
0.56.804.286 I srv operator(): Chat format: peg-native
|
||
|
|
0.56.804.436 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.781 (> 0.100 thold), f_keep = 0.467
|
||
|
|
0.56.804.438 I srv get_availabl: updating prompt cache
|
||
|
|
0.56.804.457 W srv prompt_save: - saving prompt with length 107, total state size = 77.885 MiB (draft: 0.000 MiB)
|
||
|
|
0.56.899.487 I srv load: - looking for better prompt, base f_keep = 0.467, sim = 0.781
|
||
|
|
0.56.899.494 I srv load: - found better prompt with f_keep = 0.481, sim = 0.812
|
||
|
|
0.56.936.977 I srv update: - cache state: 7 prompts, 1075.500 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.56.936.980 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.56.936.980 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
0.56.936.981 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
0.56.936.981 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
0.56.936.982 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
0.56.936.983 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
0.56.936.983 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
0.56.936.985 I srv get_availabl: prompt cache update took 132.54 ms
|
||
|
|
0.56.937.055 I slot launch_slot_: id 3 | task 891 | processing task, is_child = 0
|
||
|
|
0.56.937.056 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.56.937.057 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.56.937.065 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.56.937.066 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.56.937.070 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.56.937.070 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.56.937.085 I slot update_slots: id 3 | task 891 | Checking checkpoint with [45, 45] against 52...
|
||
|
|
0.56.961.615 W slot update_slots: id 3 | task 891 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.57.449.142 I slot print_timing: id 3 | task 891 | prompt eval time = 152.68 ms / 18 tokens ( 8.48 ms per token, 117.89 tokens per second)
|
||
|
|
0.57.449.145 I slot print_timing: id 3 | task 891 | eval time = 359.37 ms / 15 tokens ( 23.96 ms per token, 41.74 tokens per second)
|
||
|
|
0.57.449.145 I slot print_timing: id 3 | task 891 | total time = 512.05 ms / 33 tokens
|
||
|
|
0.57.449.146 I slot print_timing: id 3 | task 891 | graphs reused = 828
|
||
|
|
0.57.449.172 I slot release: id 3 | task 891 | stop processing: n_tokens = 78, truncated = 0
|
||
|
|
0.57.449.178 I srv update_slots: all slots are idle
|
||
|
|
0.57.453.819 I srv operator(): Chat format: peg-native
|
||
|
|
0.57.453.962 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.778 (> 0.100 thold), f_keep = 0.628
|
||
|
|
0.57.453.993 I slot launch_slot_: id 3 | task 908 | processing task, is_child = 0
|
||
|
|
0.57.453.995 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.57.453.996 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.57.454.002 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.57.454.002 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.57.454.007 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.57.454.007 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.57.454.014 I slot update_slots: id 3 | task 908 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.57.473.178 W slot update_slots: id 3 | task 908 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
0.59.021.430 I slot print_timing: id 3 | task 908 | prompt eval time = 146.56 ms / 17 tokens ( 8.62 ms per token, 116.00 tokens per second)
|
||
|
|
0.59.021.433 I slot print_timing: id 3 | task 908 | eval time = 1420.85 ms / 58 tokens ( 24.50 ms per token, 40.82 tokens per second)
|
||
|
|
0.59.021.434 I slot print_timing: id 3 | task 908 | total time = 1567.41 ms / 75 tokens
|
||
|
|
0.59.021.435 I slot print_timing: id 3 | task 908 | graphs reused = 884
|
||
|
|
0.59.021.460 I slot release: id 3 | task 908 | stop processing: n_tokens = 120, truncated = 0
|
||
|
|
0.59.021.466 I srv update_slots: all slots are idle
|
||
|
|
0.59.026.517 I srv operator(): Chat format: peg-native
|
||
|
|
0.59.026.664 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.408
|
||
|
|
0.59.026.666 I srv get_availabl: updating prompt cache
|
||
|
|
0.59.026.686 W srv prompt_save: - saving prompt with length 120, total state size = 78.190 MiB (draft: 0.000 MiB)
|
||
|
|
0.59.119.878 I srv load: - looking for better prompt, base f_keep = 0.408, sim = 0.742
|
||
|
|
0.59.119.888 I srv update: - cache state: 8 prompts, 1229.066 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.59.119.889 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
0.59.119.890 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
0.59.119.890 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
0.59.119.891 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
0.59.119.891 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
0.59.119.892 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
0.59.119.893 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
0.59.119.893 I srv update: - prompt 0x64c122e628c0: 120 tokens, checkpoints: 1, 153.566 MiB
|
||
|
|
0.59.119.898 I srv get_availabl: prompt cache update took 93.23 ms
|
||
|
|
0.59.119.967 I slot launch_slot_: id 3 | task 968 | processing task, is_child = 0
|
||
|
|
0.59.119.968 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.59.119.969 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.59.119.977 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.59.119.977 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.59.119.982 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.59.119.982 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.59.119.994 I slot update_slots: id 3 | task 968 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.59.138.664 W slot update_slots: id 3 | task 968 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.00.177.109 I slot print_timing: id 3 | task 968 | prompt eval time = 153.46 ms / 20 tokens ( 7.67 ms per token, 130.33 tokens per second)
|
||
|
|
1.00.177.115 I slot print_timing: id 3 | task 968 | eval time = 903.65 ms / 37 tokens ( 24.42 ms per token, 40.95 tokens per second)
|
||
|
|
1.00.177.115 I slot print_timing: id 3 | task 968 | total time = 1057.11 ms / 57 tokens
|
||
|
|
1.00.177.116 I slot print_timing: id 3 | task 968 | graphs reused = 919
|
||
|
|
1.00.177.139 I slot release: id 3 | task 968 | stop processing: n_tokens = 102, truncated = 0
|
||
|
|
1.00.177.145 I srv update_slots: all slots are idle
|
||
|
|
1.00.181.779 I srv operator(): Chat format: peg-native
|
||
|
|
1.00.181.918 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.754 (> 0.100 thold), f_keep = 0.480
|
||
|
|
1.00.181.920 I srv get_availabl: updating prompt cache
|
||
|
|
1.00.181.938 W srv prompt_save: - saving prompt with length 102, total state size = 77.768 MiB (draft: 0.000 MiB)
|
||
|
|
1.00.275.228 I srv load: - looking for better prompt, base f_keep = 0.480, sim = 0.754
|
||
|
|
1.00.275.237 I srv update: - cache state: 9 prompts, 1382.210 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
1.00.275.242 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
1.00.275.242 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
1.00.275.243 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
1.00.275.244 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
1.00.275.246 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
1.00.275.247 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
1.00.275.248 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
1.00.275.249 I srv update: - prompt 0x64c122e628c0: 120 tokens, checkpoints: 1, 153.566 MiB
|
||
|
|
1.00.275.249 I srv update: - prompt 0x64c11b7bffb0: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.00.275.251 I srv get_availabl: prompt cache update took 93.33 ms
|
||
|
|
1.00.275.332 I slot launch_slot_: id 3 | task 1007 | processing task, is_child = 0
|
||
|
|
1.00.275.334 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.00.275.335 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.00.275.345 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.00.275.345 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.00.275.353 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.00.275.354 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.00.275.371 I slot update_slots: id 3 | task 1007 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.00.294.129 W slot update_slots: id 3 | task 1007 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.00.829.688 I slot print_timing: id 3 | task 1007 | prompt eval time = 149.07 ms / 19 tokens ( 7.85 ms per token, 127.46 tokens per second)
|
||
|
|
1.00.829.692 I slot print_timing: id 3 | task 1007 | eval time = 405.24 ms / 17 tokens ( 23.84 ms per token, 41.95 tokens per second)
|
||
|
|
1.00.829.692 I slot print_timing: id 3 | task 1007 | total time = 554.31 ms / 36 tokens
|
||
|
|
1.00.829.693 I slot print_timing: id 3 | task 1007 | graphs reused = 934
|
||
|
|
1.00.829.719 I slot release: id 3 | task 1007 | stop processing: n_tokens = 81, truncated = 0
|
||
|
|
1.00.829.725 I srv update_slots: all slots are idle
|
||
|
|
1.00.834.903 I srv operator(): Chat format: peg-native
|
||
|
|
1.00.835.042 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.690 (> 0.100 thold), f_keep = 0.605
|
||
|
|
1.00.835.074 I slot launch_slot_: id 3 | task 1026 | processing task, is_child = 0
|
||
|
|
1.00.835.077 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.00.835.078 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.00.835.084 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.00.835.084 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.00.835.089 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.00.835.089 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.00.835.096 I slot update_slots: id 3 | task 1026 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.00.854.199 W slot update_slots: id 3 | task 1026 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.01.680.652 I slot print_timing: id 3 | task 1026 | prompt eval time = 194.95 ms / 25 tokens ( 7.80 ms per token, 128.24 tokens per second)
|
||
|
|
1.01.680.655 I slot print_timing: id 3 | task 1026 | eval time = 650.60 ms / 27 tokens ( 24.10 ms per token, 41.50 tokens per second)
|
||
|
|
1.01.680.656 I slot print_timing: id 3 | task 1026 | total time = 845.55 ms / 52 tokens
|
||
|
|
1.01.680.657 I slot print_timing: id 3 | task 1026 | graphs reused = 959
|
||
|
|
1.01.680.684 I slot release: id 3 | task 1026 | stop processing: n_tokens = 97, truncated = 0
|
||
|
|
1.01.680.690 I srv update_slots: all slots are idle
|
||
|
|
1.01.685.318 I srv operator(): Chat format: peg-native
|
||
|
|
1.01.685.459 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.700 (> 0.100 thold), f_keep = 0.505
|
||
|
|
1.01.685.514 I slot launch_slot_: id 3 | task 1055 | processing task, is_child = 0
|
||
|
|
1.01.685.515 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.01.685.515 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.01.685.523 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.01.685.523 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.01.685.528 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.01.685.529 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.01.685.538 I slot update_slots: id 3 | task 1055 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.01.704.535 W slot update_slots: id 3 | task 1055 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.02.358.156 I slot print_timing: id 3 | task 1055 | prompt eval time = 192.68 ms / 24 tokens ( 8.03 ms per token, 124.56 tokens per second)
|
||
|
|
1.02.358.159 I slot print_timing: id 3 | task 1055 | eval time = 479.93 ms / 20 tokens ( 24.00 ms per token, 41.67 tokens per second)
|
||
|
|
1.02.358.160 I slot print_timing: id 3 | task 1055 | total time = 672.61 ms / 44 tokens
|
||
|
|
1.02.358.161 I slot print_timing: id 3 | task 1055 | graphs reused = 977
|
||
|
|
1.02.358.187 I slot release: id 3 | task 1055 | stop processing: n_tokens = 89, truncated = 0
|
||
|
|
1.02.358.194 I srv update_slots: all slots are idle
|
||
|
|
1.02.362.693 I srv operator(): Chat format: peg-native
|
||
|
|
1.02.362.841 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.790 (> 0.100 thold), f_keep = 0.551
|
||
|
|
1.02.362.873 I slot launch_slot_: id 3 | task 1077 | processing task, is_child = 0
|
||
|
|
1.02.362.875 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.02.362.875 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.02.362.882 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.02.362.882 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.02.362.887 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.02.362.888 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.02.362.895 I slot update_slots: id 3 | task 1077 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.02.381.743 W slot update_slots: id 3 | task 1077 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.02.791.224 I slot print_timing: id 3 | task 1077 | prompt eval time = 144.98 ms / 16 tokens ( 9.06 ms per token, 110.36 tokens per second)
|
||
|
|
1.02.791.227 I slot print_timing: id 3 | task 1077 | eval time = 283.34 ms / 12 tokens ( 23.61 ms per token, 42.35 tokens per second)
|
||
|
|
1.02.791.228 I slot print_timing: id 3 | task 1077 | total time = 428.32 ms / 28 tokens
|
||
|
|
1.02.791.229 I slot print_timing: id 3 | task 1077 | graphs reused = 987
|
||
|
|
1.02.791.254 I slot release: id 3 | task 1077 | stop processing: n_tokens = 73, truncated = 0
|
||
|
|
1.02.791.261 I srv update_slots: all slots are idle
|
||
|
|
1.02.795.749 I srv operator(): Chat format: peg-native
|
||
|
|
1.02.795.896 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.754 (> 0.100 thold), f_keep = 0.671
|
||
|
|
1.02.795.929 I slot launch_slot_: id 3 | task 1091 | processing task, is_child = 0
|
||
|
|
1.02.795.930 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.02.795.931 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.02.795.938 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.02.795.938 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.02.795.943 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.02.795.943 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.02.795.950 I slot update_slots: id 3 | task 1091 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.02.814.780 W slot update_slots: id 3 | task 1091 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.03.840.700 I slot print_timing: id 3 | task 1091 | prompt eval time = 149.39 ms / 19 tokens ( 7.86 ms per token, 127.18 tokens per second)
|
||
|
|
1.03.840.703 I slot print_timing: id 3 | task 1091 | eval time = 895.35 ms / 37 tokens ( 24.20 ms per token, 41.32 tokens per second)
|
||
|
|
1.03.840.704 I slot print_timing: id 3 | task 1091 | total time = 1044.74 ms / 56 tokens
|
||
|
|
1.03.840.705 I slot print_timing: id 3 | task 1091 | graphs reused = 1022
|
||
|
|
1.03.840.733 I slot release: id 3 | task 1091 | stop processing: n_tokens = 101, truncated = 0
|
||
|
|
1.03.840.740 I srv update_slots: all slots are idle
|
||
|
|
1.03.845.423 I srv operator(): Chat format: peg-native
|
||
|
|
1.03.845.577 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.485
|
||
|
|
1.03.845.578 I srv get_availabl: updating prompt cache
|
||
|
|
1.03.845.597 W srv prompt_save: - saving prompt with length 101, total state size = 77.744 MiB (draft: 0.000 MiB)
|
||
|
|
1.03.939.941 I srv load: - looking for better prompt, base f_keep = 0.485, sim = 0.731
|
||
|
|
1.03.939.950 I srv update: - cache state: 10 prompts, 1535.330 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
1.03.939.951 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
1.03.939.955 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
1.03.939.956 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
1.03.939.956 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
1.03.939.957 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
1.03.939.958 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
1.03.939.960 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
1.03.939.961 I srv update: - prompt 0x64c122e628c0: 120 tokens, checkpoints: 1, 153.566 MiB
|
||
|
|
1.03.939.961 I srv update: - prompt 0x64c11b7bffb0: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.03.939.962 I srv update: - prompt 0x64c122dbf1d0: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
1.03.939.963 I srv get_availabl: prompt cache update took 94.38 ms
|
||
|
|
1.03.940.035 I slot launch_slot_: id 3 | task 1130 | processing task, is_child = 0
|
||
|
|
1.03.940.036 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.03.940.037 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.03.940.043 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.03.940.044 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.03.940.048 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.03.940.049 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.03.940.061 I slot update_slots: id 3 | task 1130 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.03.959.035 W slot update_slots: id 3 | task 1130 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.04.511.970 I slot print_timing: id 3 | task 1130 | prompt eval time = 190.67 ms / 21 tokens ( 9.08 ms per token, 110.14 tokens per second)
|
||
|
|
1.04.511.973 I slot print_timing: id 3 | task 1130 | eval time = 381.24 ms / 16 tokens ( 23.83 ms per token, 41.97 tokens per second)
|
||
|
|
1.04.511.974 I slot print_timing: id 3 | task 1130 | total time = 571.90 ms / 37 tokens
|
||
|
|
1.04.511.975 I slot print_timing: id 3 | task 1130 | graphs reused = 1036
|
||
|
|
1.04.512.001 I slot release: id 3 | task 1130 | stop processing: n_tokens = 82, truncated = 0
|
||
|
|
1.04.512.007 I srv update_slots: all slots are idle
|
||
|
|
1.04.516.504 I srv operator(): Chat format: peg-native
|
||
|
|
1.04.516.653 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.636 (> 0.100 thold), f_keep = 0.598
|
||
|
|
1.04.516.684 I slot launch_slot_: id 3 | task 1148 | processing task, is_child = 0
|
||
|
|
1.04.516.684 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.04.516.685 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.04.516.692 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.04.516.693 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.04.516.697 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.04.516.697 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.04.516.705 I slot update_slots: id 3 | task 1148 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.04.535.563 W slot update_slots: id 3 | task 1148 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.06.959.728 I slot print_timing: id 3 | task 1148 | prompt eval time = 226.87 ms / 31 tokens ( 7.32 ms per token, 136.64 tokens per second)
|
||
|
|
1.06.959.731 I slot print_timing: id 3 | task 1148 | eval time = 2216.15 ms / 91 tokens ( 24.35 ms per token, 41.06 tokens per second)
|
||
|
|
1.06.959.732 I slot print_timing: id 3 | task 1148 | total time = 2443.02 ms / 122 tokens
|
||
|
|
1.06.959.733 I slot print_timing: id 3 | task 1148 | graphs reused = 1125
|
||
|
|
1.06.959.758 I slot release: id 3 | task 1148 | stop processing: n_tokens = 167, truncated = 0
|
||
|
|
1.06.959.763 I srv update_slots: all slots are idle
|
||
|
|
1.06.964.408 I srv operator(): Chat format: peg-native
|
||
|
|
1.06.964.553 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.725 (> 0.100 thold), f_keep = 0.299
|
||
|
|
1.06.964.564 I srv get_availabl: updating prompt cache
|
||
|
|
1.06.964.587 W srv prompt_save: - saving prompt with length 167, total state size = 79.292 MiB (draft: 0.000 MiB)
|
||
|
|
1.07.059.252 I srv load: - looking for better prompt, base f_keep = 0.299, sim = 0.725
|
||
|
|
1.07.059.263 I srv update: - cache state: 11 prompts, 1689.998 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
1.07.059.264 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
1.07.059.265 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
1.07.059.266 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
1.07.059.267 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
1.07.059.268 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
1.07.059.268 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
1.07.059.269 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
1.07.059.269 I srv update: - prompt 0x64c122e628c0: 120 tokens, checkpoints: 1, 153.566 MiB
|
||
|
|
1.07.059.270 I srv update: - prompt 0x64c11b7bffb0: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.07.059.271 I srv update: - prompt 0x64c122dbf1d0: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
1.07.059.271 I srv update: - prompt 0x64c11b1eeec0: 167 tokens, checkpoints: 1, 154.668 MiB
|
||
|
|
1.07.059.273 I srv get_availabl: prompt cache update took 94.71 ms
|
||
|
|
1.07.059.348 I slot launch_slot_: id 3 | task 1241 | processing task, is_child = 0
|
||
|
|
1.07.059.350 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.07.059.351 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.07.059.358 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.07.059.358 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.07.059.368 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.07.059.369 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.07.059.381 I slot update_slots: id 3 | task 1241 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
1.07.078.103 W slot update_slots: id 3 | task 1241 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.08.274.043 I slot print_timing: id 3 | task 1241 | prompt eval time = 194.25 ms / 23 tokens ( 8.45 ms per token, 118.40 tokens per second)
|
||
|
|
1.08.274.046 I slot print_timing: id 3 | task 1241 | eval time = 1020.40 ms / 42 tokens ( 24.30 ms per token, 41.16 tokens per second)
|
||
|
|
1.08.274.047 I slot print_timing: id 3 | task 1241 | total time = 1214.66 ms / 65 tokens
|
||
|
|
1.08.274.047 I slot print_timing: id 3 | task 1241 | graphs reused = 1165
|
||
|
|
1.08.274.075 I slot release: id 3 | task 1241 | stop processing: n_tokens = 110, truncated = 0
|
||
|
|
1.08.274.082 I srv update_slots: all slots are idle
|
||
|
|
1.08.278.857 I srv operator(): Chat format: peg-native
|
||
|
|
1.08.279.002 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.766 (> 0.100 thold), f_keep = 0.445
|
||
|
|
1.08.279.004 I srv get_availabl: updating prompt cache
|
||
|
|
1.08.279.023 W srv prompt_save: - saving prompt with length 110, total state size = 77.956 MiB (draft: 0.000 MiB)
|
||
|
|
1.08.370.987 I srv load: - looking for better prompt, base f_keep = 0.445, sim = 0.766
|
||
|
|
1.08.370.996 I srv update: - cache state: 12 prompts, 1843.330 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
1.08.370.997 I srv update: - prompt 0x64c11b430f80: 139 tokens, checkpoints: 1, 154.011 MiB
|
||
|
|
1.08.370.998 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
1.08.370.999 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
1.08.370.999 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
1.08.371.000 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
1.08.371.001 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
1.08.371.006 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
1.08.371.007 I srv update: - prompt 0x64c122e628c0: 120 tokens, checkpoints: 1, 153.566 MiB
|
||
|
|
1.08.371.007 I srv update: - prompt 0x64c11b7bffb0: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.08.371.008 I srv update: - prompt 0x64c122dbf1d0: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
1.08.371.009 I srv update: - prompt 0x64c11b1eeec0: 167 tokens, checkpoints: 1, 154.668 MiB
|
||
|
|
1.08.371.009 I srv update: - prompt 0x64c122e50be0: 110 tokens, checkpoints: 1, 153.331 MiB
|
||
|
|
1.08.371.013 I srv get_availabl: prompt cache update took 92.01 ms
|
||
|
|
1.08.371.085 I slot launch_slot_: id 3 | task 1285 | processing task, is_child = 0
|
||
|
|
1.08.371.085 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.08.371.086 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.08.371.093 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.08.371.094 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.08.371.098 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.08.371.098 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.08.371.110 I slot update_slots: id 3 | task 1285 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.08.389.845 W slot update_slots: id 3 | task 1285 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.10.574.354 I slot print_timing: id 3 | task 1285 | prompt eval time = 149.56 ms / 18 tokens ( 8.31 ms per token, 120.36 tokens per second)
|
||
|
|
1.10.574.358 I slot print_timing: id 3 | task 1285 | eval time = 2053.68 ms / 84 tokens ( 24.45 ms per token, 40.90 tokens per second)
|
||
|
|
1.10.574.358 I slot print_timing: id 3 | task 1285 | total time = 2203.24 ms / 102 tokens
|
||
|
|
1.10.574.359 I slot print_timing: id 3 | task 1285 | graphs reused = 1247
|
||
|
|
1.10.574.384 I slot release: id 3 | task 1285 | stop processing: n_tokens = 147, truncated = 0
|
||
|
|
1.10.574.390 I srv update_slots: all slots are idle
|
||
|
|
1.10.579.224 I srv operator(): Chat format: peg-native
|
||
|
|
1.10.579.368 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.333
|
||
|
|
1.10.579.369 I srv get_availabl: updating prompt cache
|
||
|
|
1.10.579.389 W srv prompt_save: - saving prompt with length 147, total state size = 78.823 MiB (draft: 0.000 MiB)
|
||
|
|
1.10.674.363 I srv load: - looking for better prompt, base f_keep = 0.333, sim = 0.710
|
||
|
|
1.10.674.369 I srv load: - found better prompt with f_keep = 0.360, sim = 0.725
|
||
|
|
1.10.712.594 I srv update: - cache state: 12 prompts, 1843.517 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
1.10.712.598 I srv update: - prompt 0x64c11b7ae120: 126 tokens, checkpoints: 1, 153.707 MiB
|
||
|
|
1.10.712.599 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
1.10.712.600 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
1.10.712.600 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
1.10.712.601 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
1.10.712.602 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
1.10.712.603 I srv update: - prompt 0x64c122e628c0: 120 tokens, checkpoints: 1, 153.566 MiB
|
||
|
|
1.10.712.604 I srv update: - prompt 0x64c11b7bffb0: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.10.712.604 I srv update: - prompt 0x64c122dbf1d0: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
1.10.712.605 I srv update: - prompt 0x64c11b1eeec0: 167 tokens, checkpoints: 1, 154.668 MiB
|
||
|
|
1.10.712.608 I srv update: - prompt 0x64c122e50be0: 110 tokens, checkpoints: 1, 153.331 MiB
|
||
|
|
1.10.712.609 I srv update: - prompt 0x64c122da28b0: 147 tokens, checkpoints: 1, 154.199 MiB
|
||
|
|
1.10.712.610 I srv get_availabl: prompt cache update took 133.24 ms
|
||
|
|
1.10.712.693 I slot launch_slot_: id 3 | task 1371 | processing task, is_child = 0
|
||
|
|
1.10.712.694 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.10.712.695 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.10.712.703 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.10.712.704 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.10.712.715 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.10.712.715 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.10.712.730 I slot update_slots: id 3 | task 1371 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
1.10.755.944 W slot update_slots: id 3 | task 1371 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.12.290.256 I slot print_timing: id 3 | task 1371 | prompt eval time = 215.78 ms / 23 tokens ( 9.38 ms per token, 106.59 tokens per second)
|
||
|
|
1.12.290.259 I slot print_timing: id 3 | task 1371 | eval time = 1361.73 ms / 56 tokens ( 24.32 ms per token, 41.12 tokens per second)
|
||
|
|
1.12.290.260 I slot print_timing: id 3 | task 1371 | total time = 1577.52 ms / 79 tokens
|
||
|
|
1.12.290.261 I slot print_timing: id 3 | task 1371 | graphs reused = 1301
|
||
|
|
1.12.290.292 I slot release: id 3 | task 1371 | stop processing: n_tokens = 124, truncated = 0
|
||
|
|
1.12.290.301 I srv update_slots: all slots are idle
|
||
|
|
1.12.295.049 I srv operator(): Chat format: peg-native
|
||
|
|
1.12.295.191 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.694 (> 0.100 thold), f_keep = 0.403
|
||
|
|
1.12.295.193 I srv get_availabl: updating prompt cache
|
||
|
|
1.12.295.213 W srv prompt_save: - saving prompt with length 124, total state size = 78.284 MiB (draft: 0.000 MiB)
|
||
|
|
1.12.404.625 I srv load: - looking for better prompt, base f_keep = 0.403, sim = 0.694
|
||
|
|
1.12.404.631 I srv load: - found better prompt with f_keep = 0.405, sim = 0.708
|
||
|
|
1.12.442.174 I srv update: - cache state: 12 prompts, 1843.470 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
1.12.442.177 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
1.12.442.178 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
1.12.442.178 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
1.12.442.179 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
1.12.442.179 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
1.12.442.180 I srv update: - prompt 0x64c122e628c0: 120 tokens, checkpoints: 1, 153.566 MiB
|
||
|
|
1.12.442.180 I srv update: - prompt 0x64c11b7bffb0: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.12.442.181 I srv update: - prompt 0x64c122dbf1d0: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
1.12.442.181 I srv update: - prompt 0x64c11b1eeec0: 167 tokens, checkpoints: 1, 154.668 MiB
|
||
|
|
1.12.442.182 I srv update: - prompt 0x64c122e50be0: 110 tokens, checkpoints: 1, 153.331 MiB
|
||
|
|
1.12.442.183 I srv update: - prompt 0x64c122da28b0: 147 tokens, checkpoints: 1, 154.199 MiB
|
||
|
|
1.12.442.184 I srv update: - prompt 0x64c11b3b51e0: 124 tokens, checkpoints: 1, 153.660 MiB
|
||
|
|
1.12.442.186 I srv get_availabl: prompt cache update took 146.99 ms
|
||
|
|
1.12.442.261 I slot launch_slot_: id 3 | task 1429 | processing task, is_child = 0
|
||
|
|
1.12.442.262 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.12.442.263 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.12.442.270 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.12.442.270 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.12.442.275 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.12.442.275 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.12.442.287 I slot update_slots: id 3 | task 1429 | Checking checkpoint with [45, 45] against 51...
|
||
|
|
1.12.476.661 W slot update_slots: id 3 | task 1429 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.13.409.696 I slot print_timing: id 3 | task 1429 | prompt eval time = 215.65 ms / 26 tokens ( 8.29 ms per token, 120.57 tokens per second)
|
||
|
|
1.13.409.699 I slot print_timing: id 3 | task 1429 | eval time = 751.75 ms / 31 tokens ( 24.25 ms per token, 41.24 tokens per second)
|
||
|
|
1.13.409.700 I slot print_timing: id 3 | task 1429 | total time = 967.40 ms / 57 tokens
|
||
|
|
1.13.409.701 I slot print_timing: id 3 | task 1429 | graphs reused = 1330
|
||
|
|
1.13.409.727 I slot release: id 3 | task 1429 | stop processing: n_tokens = 102, truncated = 0
|
||
|
|
1.13.409.732 I srv update_slots: all slots are idle
|
||
|
|
1.13.414.408 I srv operator(): Chat format: peg-native
|
||
|
|
1.13.414.545 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.766 (> 0.100 thold), f_keep = 0.480
|
||
|
|
1.13.414.546 I srv get_availabl: updating prompt cache
|
||
|
|
1.13.414.573 W srv prompt_save: - saving prompt with length 102, total state size = 77.768 MiB (draft: 0.000 MiB)
|
||
|
|
1.13.507.153 I srv load: - looking for better prompt, base f_keep = 0.480, sim = 0.766
|
||
|
|
1.13.507.164 I srv update: - cache state: 13 prompts, 1996.614 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
1.13.507.165 I srv update: - prompt 0x64c11b69a840: 137 tokens, checkpoints: 1, 153.964 MiB
|
||
|
|
1.13.507.165 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
1.13.507.166 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
1.13.507.166 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
1.13.507.167 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
1.13.507.167 I srv update: - prompt 0x64c122e628c0: 120 tokens, checkpoints: 1, 153.566 MiB
|
||
|
|
1.13.507.168 I srv update: - prompt 0x64c11b7bffb0: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.13.507.168 I srv update: - prompt 0x64c122dbf1d0: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
1.13.507.170 I srv update: - prompt 0x64c11b1eeec0: 167 tokens, checkpoints: 1, 154.668 MiB
|
||
|
|
1.13.507.171 I srv update: - prompt 0x64c122e50be0: 110 tokens, checkpoints: 1, 153.331 MiB
|
||
|
|
1.13.507.171 I srv update: - prompt 0x64c122da28b0: 147 tokens, checkpoints: 1, 154.199 MiB
|
||
|
|
1.13.507.172 I srv update: - prompt 0x64c11b3b51e0: 124 tokens, checkpoints: 1, 153.660 MiB
|
||
|
|
1.13.507.172 I srv update: - prompt 0x64c11b35bd80: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.13.507.173 I srv get_availabl: prompt cache update took 92.63 ms
|
||
|
|
1.13.507.246 I slot launch_slot_: id 3 | task 1462 | processing task, is_child = 0
|
||
|
|
1.13.507.247 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.13.507.248 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.13.507.254 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.13.507.255 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.13.507.259 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.13.507.259 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.13.507.272 I slot update_slots: id 3 | task 1462 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.13.526.129 W slot update_slots: id 3 | task 1462 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.14.456.140 I slot print_timing: id 3 | task 1462 | prompt eval time = 148.66 ms / 18 tokens ( 8.26 ms per token, 121.08 tokens per second)
|
||
|
|
1.14.456.143 I slot print_timing: id 3 | task 1462 | eval time = 800.20 ms / 33 tokens ( 24.25 ms per token, 41.24 tokens per second)
|
||
|
|
1.14.456.144 I slot print_timing: id 3 | task 1462 | total time = 948.86 ms / 51 tokens
|
||
|
|
1.14.456.145 I slot print_timing: id 3 | task 1462 | graphs reused = 1361
|
||
|
|
1.14.456.171 I slot release: id 3 | task 1462 | stop processing: n_tokens = 96, truncated = 0
|
||
|
|
1.14.456.177 I srv update_slots: all slots are idle
|
||
|
|
1.14.460.766 I srv operator(): Chat format: peg-native
|
||
|
|
1.14.460.910 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.817 (> 0.100 thold), f_keep = 0.510
|
||
|
|
1.14.460.943 I slot launch_slot_: id 3 | task 1497 | processing task, is_child = 0
|
||
|
|
1.14.460.944 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.14.460.945 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.14.460.951 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.14.460.951 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.14.460.956 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.14.460.956 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.14.460.963 I slot update_slots: id 3 | task 1497 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.14.479.839 W slot update_slots: id 3 | task 1497 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.15.058.564 I slot print_timing: id 3 | task 1497 | prompt eval time = 140.88 ms / 14 tokens ( 10.06 ms per token, 99.38 tokens per second)
|
||
|
|
1.15.058.567 I slot print_timing: id 3 | task 1497 | eval time = 456.71 ms / 19 tokens ( 24.04 ms per token, 41.60 tokens per second)
|
||
|
|
1.15.058.568 I slot print_timing: id 3 | task 1497 | total time = 597.59 ms / 33 tokens
|
||
|
|
1.15.058.568 I slot print_timing: id 3 | task 1497 | graphs reused = 1378
|
||
|
|
1.15.058.595 I slot release: id 3 | task 1497 | stop processing: n_tokens = 78, truncated = 0
|
||
|
|
1.15.058.600 I srv update_slots: all slots are idle
|
||
|
|
1.15.063.487 I srv operator(): Chat format: peg-native
|
||
|
|
1.15.063.639 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.628
|
||
|
|
1.15.063.674 I slot launch_slot_: id 3 | task 1518 | processing task, is_child = 0
|
||
|
|
1.15.063.676 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.15.063.677 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.15.063.684 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.15.063.684 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.15.063.690 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.15.063.690 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.15.063.699 I slot update_slots: id 3 | task 1518 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.15.082.933 W slot update_slots: id 3 | task 1518 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.16.940.691 I slot print_timing: id 3 | task 1518 | prompt eval time = 192.04 ms / 22 tokens ( 8.73 ms per token, 114.56 tokens per second)
|
||
|
|
1.16.940.694 I slot print_timing: id 3 | task 1518 | eval time = 1684.94 ms / 69 tokens ( 24.42 ms per token, 40.95 tokens per second)
|
||
|
|
1.16.940.695 I slot print_timing: id 3 | task 1518 | total time = 1876.99 ms / 91 tokens
|
||
|
|
1.16.940.696 I slot print_timing: id 3 | task 1518 | graphs reused = 1445
|
||
|
|
1.16.940.723 I slot release: id 3 | task 1518 | stop processing: n_tokens = 136, truncated = 0
|
||
|
|
1.16.940.730 I srv update_slots: all slots are idle
|
||
|
|
1.16.945.274 I srv operator(): Chat format: peg-native
|
||
|
|
1.16.945.419 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.766 (> 0.100 thold), f_keep = 0.360
|
||
|
|
1.16.945.420 I srv get_availabl: updating prompt cache
|
||
|
|
1.16.945.439 W srv prompt_save: - saving prompt with length 136, total state size = 78.565 MiB (draft: 0.000 MiB)
|
||
|
|
1.17.037.473 I srv load: - looking for better prompt, base f_keep = 0.360, sim = 0.766
|
||
|
|
1.17.037.483 I srv load: - found better prompt with f_keep = 0.380, sim = 0.812
|
||
|
|
1.17.075.606 I srv update: - cache state: 13 prompts, 1996.591 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
1.17.075.611 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
1.17.075.612 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
1.17.075.613 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
1.17.075.613 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
1.17.075.614 I srv update: - prompt 0x64c122e628c0: 120 tokens, checkpoints: 1, 153.566 MiB
|
||
|
|
1.17.075.615 I srv update: - prompt 0x64c11b7bffb0: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.17.075.616 I srv update: - prompt 0x64c122dbf1d0: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
1.17.075.616 I srv update: - prompt 0x64c11b1eeec0: 167 tokens, checkpoints: 1, 154.668 MiB
|
||
|
|
1.17.075.617 I srv update: - prompt 0x64c122e50be0: 110 tokens, checkpoints: 1, 153.331 MiB
|
||
|
|
1.17.075.618 I srv update: - prompt 0x64c122da28b0: 147 tokens, checkpoints: 1, 154.199 MiB
|
||
|
|
1.17.075.620 I srv update: - prompt 0x64c11b3b51e0: 124 tokens, checkpoints: 1, 153.660 MiB
|
||
|
|
1.17.075.620 I srv update: - prompt 0x64c11b35bd80: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.17.075.621 I srv update: - prompt 0x64c11b3bb250: 136 tokens, checkpoints: 1, 153.941 MiB
|
||
|
|
1.17.075.625 I srv get_availabl: prompt cache update took 130.20 ms
|
||
|
|
1.17.075.709 I slot launch_slot_: id 3 | task 1589 | processing task, is_child = 0
|
||
|
|
1.17.075.711 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.17.075.712 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.17.075.721 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.17.075.721 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.17.075.727 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.17.075.728 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.17.075.743 I slot update_slots: id 3 | task 1589 | Checking checkpoint with [45, 45] against 52...
|
||
|
|
1.17.106.549 W slot update_slots: id 3 | task 1589 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.19.069.129 I slot print_timing: id 3 | task 1589 | prompt eval time = 160.17 ms / 18 tokens ( 8.90 ms per token, 112.38 tokens per second)
|
||
|
|
1.19.069.132 I slot print_timing: id 3 | task 1589 | eval time = 1833.21 ms / 75 tokens ( 24.44 ms per token, 40.91 tokens per second)
|
||
|
|
1.19.069.132 I slot print_timing: id 3 | task 1589 | total time = 1993.38 ms / 93 tokens
|
||
|
|
1.19.069.133 I slot print_timing: id 3 | task 1589 | graphs reused = 1518
|
||
|
|
1.19.069.157 I slot release: id 3 | task 1589 | stop processing: n_tokens = 138, truncated = 0
|
||
|
|
1.19.069.163 I srv update_slots: all slots are idle
|
||
|
|
1.19.073.928 I srv operator(): Chat format: peg-native
|
||
|
|
1.19.074.079 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.794 (> 0.100 thold), f_keep = 0.362
|
||
|
|
1.19.074.080 I srv get_availabl: updating prompt cache
|
||
|
|
1.19.074.098 W srv prompt_save: - saving prompt with length 138, total state size = 78.612 MiB (draft: 0.000 MiB)
|
||
|
|
1.19.172.421 I srv load: - looking for better prompt, base f_keep = 0.362, sim = 0.794
|
||
|
|
1.19.172.432 I srv update: - cache state: 14 prompts, 2150.579 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
1.19.172.434 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
1.19.172.435 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
1.19.172.435 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
1.19.172.436 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
1.19.172.436 I srv update: - prompt 0x64c122e628c0: 120 tokens, checkpoints: 1, 153.566 MiB
|
||
|
|
1.19.172.437 I srv update: - prompt 0x64c11b7bffb0: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.19.172.440 I srv update: - prompt 0x64c122dbf1d0: 101 tokens, checkpoints: 1, 153.120 MiB
|
||
|
|
1.19.172.440 I srv update: - prompt 0x64c11b1eeec0: 167 tokens, checkpoints: 1, 154.668 MiB
|
||
|
|
1.19.172.441 I srv update: - prompt 0x64c122e50be0: 110 tokens, checkpoints: 1, 153.331 MiB
|
||
|
|
1.19.172.441 I srv update: - prompt 0x64c122da28b0: 147 tokens, checkpoints: 1, 154.199 MiB
|
||
|
|
1.19.172.442 I srv update: - prompt 0x64c11b3b51e0: 124 tokens, checkpoints: 1, 153.660 MiB
|
||
|
|
1.19.172.442 I srv update: - prompt 0x64c11b35bd80: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.19.172.446 I srv update: - prompt 0x64c11b3bb250: 136 tokens, checkpoints: 1, 153.941 MiB
|
||
|
|
1.19.172.446 I srv update: - prompt 0x64c11b617960: 138 tokens, checkpoints: 1, 153.988 MiB
|
||
|
|
1.19.172.448 I srv get_availabl: prompt cache update took 98.37 ms
|
||
|
|
1.19.172.523 I slot launch_slot_: id 3 | task 1666 | processing task, is_child = 0
|
||
|
|
1.19.172.524 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.19.172.527 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.19.172.534 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.19.172.534 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.19.172.539 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.19.172.539 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.19.172.553 I slot update_slots: id 3 | task 1666 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
1.19.191.358 W slot update_slots: id 3 | task 1666 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.20.451.544 I slot print_timing: id 3 | task 1666 | prompt eval time = 146.78 ms / 17 tokens ( 8.63 ms per token, 115.82 tokens per second)
|
||
|
|
1.20.451.549 I slot print_timing: id 3 | task 1666 | eval time = 1132.20 ms / 46 tokens ( 24.61 ms per token, 40.63 tokens per second)
|
||
|
|
1.20.451.550 I slot print_timing: id 3 | task 1666 | total time = 1278.99 ms / 63 tokens
|
||
|
|
1.20.451.551 I slot print_timing: id 3 | task 1666 | graphs reused = 1562
|
||
|
|
1.20.451.592 I slot release: id 3 | task 1666 | stop processing: n_tokens = 108, truncated = 0
|
||
|
|
1.20.451.600 I srv update_slots: all slots are idle
|
||
|
|
1.20.456.373 I srv operator(): Chat format: peg-native
|
||
|
|
1.20.456.516 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.454
|
||
|
|
1.20.456.519 I srv get_availabl: updating prompt cache
|
||
|
|
1.20.456.548 W srv prompt_save: - saving prompt with length 108, total state size = 77.909 MiB (draft: 0.000 MiB)
|
||
|
|
1.20.549.439 I srv load: - looking for better prompt, base f_keep = 0.454, sim = 0.742
|
||
|
|
1.20.549.447 I srv load: - found better prompt with f_keep = 0.495, sim = 0.758
|
||
|
|
1.20.587.357 I srv update: - cache state: 14 prompts, 2150.743 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
1.20.587.361 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
1.20.587.361 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
1.20.587.362 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
1.20.587.363 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
1.20.587.364 I srv update: - prompt 0x64c122e628c0: 120 tokens, checkpoints: 1, 153.566 MiB
|
||
|
|
1.20.587.364 I srv update: - prompt 0x64c11b7bffb0: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.20.587.365 I srv update: - prompt 0x64c11b1eeec0: 167 tokens, checkpoints: 1, 154.668 MiB
|
||
|
|
1.20.587.366 I srv update: - prompt 0x64c122e50be0: 110 tokens, checkpoints: 1, 153.331 MiB
|
||
|
|
1.20.587.367 I srv update: - prompt 0x64c122da28b0: 147 tokens, checkpoints: 1, 154.199 MiB
|
||
|
|
1.20.587.367 I srv update: - prompt 0x64c11b3b51e0: 124 tokens, checkpoints: 1, 153.660 MiB
|
||
|
|
1.20.587.368 I srv update: - prompt 0x64c11b35bd80: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.20.587.369 I srv update: - prompt 0x64c11b3bb250: 136 tokens, checkpoints: 1, 153.941 MiB
|
||
|
|
1.20.587.369 I srv update: - prompt 0x64c11b617960: 138 tokens, checkpoints: 1, 153.988 MiB
|
||
|
|
1.20.587.370 I srv update: - prompt 0x64c11b52b280: 108 tokens, checkpoints: 1, 153.284 MiB
|
||
|
|
1.20.587.372 I srv get_availabl: prompt cache update took 130.85 ms
|
||
|
|
1.20.587.457 I slot launch_slot_: id 3 | task 1714 | processing task, is_child = 0
|
||
|
|
1.20.587.458 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.20.587.459 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.20.587.467 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.20.587.468 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.20.587.473 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.20.587.474 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.20.587.489 I slot update_slots: id 3 | task 1714 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
1.20.620.390 W slot update_slots: id 3 | task 1714 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.22.203.604 I slot print_timing: id 3 | task 1714 | prompt eval time = 170.51 ms / 20 tokens ( 8.53 ms per token, 117.30 tokens per second)
|
||
|
|
1.22.203.607 I slot print_timing: id 3 | task 1714 | eval time = 1445.60 ms / 59 tokens ( 24.50 ms per token, 40.81 tokens per second)
|
||
|
|
1.22.203.608 I slot print_timing: id 3 | task 1714 | total time = 1616.11 ms / 79 tokens
|
||
|
|
1.22.203.609 I slot print_timing: id 3 | task 1714 | graphs reused = 1619
|
||
|
|
1.22.203.636 I slot release: id 3 | task 1714 | stop processing: n_tokens = 124, truncated = 0
|
||
|
|
1.22.203.642 I srv update_slots: all slots are idle
|
||
|
|
1.22.208.348 I srv operator(): Chat format: peg-native
|
||
|
|
1.22.208.487 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.790 (> 0.100 thold), f_keep = 0.395
|
||
|
|
1.22.208.489 I srv get_availabl: updating prompt cache
|
||
|
|
1.22.208.507 W srv prompt_save: - saving prompt with length 124, total state size = 78.284 MiB (draft: 0.000 MiB)
|
||
|
|
1.22.301.964 I srv load: - looking for better prompt, base f_keep = 0.395, sim = 0.790
|
||
|
|
1.22.301.972 I srv load: - found better prompt with f_keep = 0.500, sim = 0.823
|
||
|
|
1.22.340.930 I srv update: - cache state: 14 prompts, 2151.259 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
1.22.340.933 I srv update: - prompt 0x64c11b3f3280: 123 tokens, checkpoints: 1, 153.636 MiB
|
||
|
|
1.22.340.934 I srv update: - prompt 0x64c11b305890: 115 tokens, checkpoints: 1, 153.449 MiB
|
||
|
|
1.22.340.934 I srv update: - prompt 0x64c122d9b910: 116 tokens, checkpoints: 1, 153.472 MiB
|
||
|
|
1.22.340.935 I srv update: - prompt 0x64c11b60a380: 107 tokens, checkpoints: 1, 153.261 MiB
|
||
|
|
1.22.340.936 I srv update: - prompt 0x64c122e628c0: 120 tokens, checkpoints: 1, 153.566 MiB
|
||
|
|
1.22.340.936 I srv update: - prompt 0x64c11b7bffb0: 102 tokens, checkpoints: 1, 153.144 MiB
|
||
|
|
1.22.340.937 I srv update: - prompt 0x64c11b1eeec0: 167 tokens, checkpoints: 1, 154.668 MiB
|
||
|
|
1.22.340.937 I srv update: - prompt 0x64c122e50be0: 110 tokens, checkpoints: 1, 153.331 MiB
|
||
|
|
1.22.340.938 I srv update: - prompt 0x64c122da28b0: 147 tokens, checkpoints: 1, 154.199 MiB
|
||
|
|
1.22.340.938 I srv update: - prompt 0x64c11b3b51e0: 124 tokens, checkpoints: 1, 153.660 MiB
|
||
|
|
1.22.340.939 I srv update: - prompt 0x64c11b3bb250: 136 tokens, checkpoints: 1, 153.941 MiB
|
||
|
|
1.22.340.939 I srv update: - prompt 0x64c11b617960: 138 tokens, checkpoints: 1, 153.988 MiB
|
||
|
|
1.22.340.940 I srv update: - prompt 0x64c11b52b280: 108 tokens, checkpoints: 1, 153.284 MiB
|
||
|
|
1.22.340.940 I srv update: - prompt 0x64c122dce9b0: 124 tokens, checkpoints: 1, 153.660 MiB
|
||
|
|
1.22.340.941 I srv get_availabl: prompt cache update took 132.45 ms
|
||
|
|
1.22.341.019 I slot launch_slot_: id 3 | task 1775 | processing task, is_child = 0
|
||
|
|
1.22.341.020 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.22.341.021 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.22.341.030 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.22.341.030 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.22.341.034 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.22.341.035 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.22.341.048 I slot update_slots: id 3 | task 1775 | Checking checkpoint with [45, 45] against 51...
|
||
|
|
1.22.365.756 W slot update_slots: id 3 | task 1775 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 75.376 MiB)
|
||
|
|
1.25.140.763 I slot print_timing: id 3 | task 1775 | prompt eval time = 151.22 ms / 16 tokens ( 9.45 ms per token, 105.80 tokens per second)
|
||
|
|
1.25.140.768 I slot print_timing: id 3 | task 1775 | eval time = 2648.49 ms / 108 tokens ( 24.52 ms per token, 40.78 tokens per second)
|
||
|
|
1.25.140.768 I slot print_timing: id 3 | task 1775 | total time = 2799.71 ms / 124 tokens
|
||
|
|
1.25.140.769 I slot print_timing: id 3 | task 1775 | graphs reused = 1725
|
||
|
|
1.25.140.799 I slot release: id 3 | task 1775 | stop processing: n_tokens = 169, truncated = 0
|
||
|
|
1.25.140.807 I srv update_slots: all slots are idle
|
||
|
|
1.25.141.277 I srv operator(): operator(): cleaning up before exit...
|