1276 lines
124 KiB
Plaintext
1276 lines
124 KiB
Plaintext
|
G
|
0.00.065.629 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
|
||
|
|
0.00.065.633 I device_info:
|
||
|
|
0.00.065.652 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
||
|
|
0.00.065.657 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
||
|
|
0.00.065.663 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
|
||
|
|
0.00.065.720 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
||
|
|
0.00.065.729 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
|
||
|
|
0.00.065.770 I srv init: running without SSL
|
||
|
|
0.00.065.837 I srv init: using 23 threads for HTTP server
|
||
|
|
0.00.066.163 I srv start: binding port with default address family
|
||
|
|
0.00.067.362 I srv llama_server: loading model
|
||
|
|
0.00.067.368 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf'
|
||
|
|
0.00.067.418 I common_init_result: fitting params to device memory ...
|
||
|
|
0.00.067.420 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
|
||
|
|
0.15.114.730 W llama_context: n_ctx_seq (8192) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
|
||
|
|
0.15.209.304 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
||
|
|
0.15.417.510 I srv load_model: initializing slots, n_slots = 4
|
||
|
|
0.15.475.709 W srv load_model: speculative decoding will use checkpoints
|
||
|
|
0.15.475.735 W common_speculative_init: no implementations specified for speculative decoding
|
||
|
|
0.15.475.737 I slot load_model: id 0 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.15.475.741 I slot load_model: id 1 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.15.475.742 I slot load_model: id 2 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.15.475.742 I slot load_model: id 3 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.15.475.794 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
|
||
|
|
0.15.475.796 I srv load_model: use `--cache-ram 0` to disable the prompt cache
|
||
|
|
0.15.475.797 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
||
|
|
0.15.475.797 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
|
||
|
|
0.15.475.819 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
|
||
|
|
0.15.494.557 I init: chat template, example_format: '<|im_start|>system
|
||
|
|
You are a helpful assistant<|im_end|>
|
||
|
|
<|im_start|>user
|
||
|
|
Hello<|im_end|>
|
||
|
|
<|im_start|>assistant
|
||
|
|
Hi there<|im_end|>
|
||
|
|
<|im_start|>user
|
||
|
|
How are you?<|im_end|>
|
||
|
|
<|im_start|>assistant
|
||
|
|
<think>
|
||
|
|
'
|
||
|
|
0.15.508.277 I srv init: init: chat template, thinking = 1
|
||
|
|
0.15.508.313 I srv llama_server: model loaded
|
||
|
|
0.15.508.317 I srv llama_server: server is listening on http://127.0.0.1:8090
|
||
|
|
0.15.508.321 I srv update_slots: all slots are idle
|
||
|
|
0.19.980.744 I srv operator(): Chat format: peg-native
|
||
|
|
0.19.981.014 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
|
||
|
|
0.19.981.016 I srv get_availabl: updating prompt cache
|
||
|
|
0.19.981.021 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
||
|
|
0.19.981.024 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
|
||
|
|
0.19.981.025 I srv get_availabl: prompt cache update took 0.01 ms
|
||
|
|
0.19.981.073 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
|
||
|
|
0.19.981.073 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.19.981.074 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.19.981.083 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.19.981.083 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.19.981.088 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.19.981.088 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.20.626.540 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 45, pos_max = 45, n_tokens = 46, size = 62.813 MiB)
|
||
|
|
0.21.988.991 I slot print_timing: id 3 | task 0 | prompt eval time = 796.88 ms / 76 tokens ( 10.49 ms per token, 95.37 tokens per second)
|
||
|
|
0.21.988.994 I slot print_timing: id 3 | task 0 | eval time = 1211.00 ms / 62 tokens ( 19.53 ms per token, 51.20 tokens per second)
|
||
|
|
0.21.988.995 I slot print_timing: id 3 | task 0 | total time = 2007.88 ms / 138 tokens
|
||
|
|
0.21.988.999 I slot print_timing: id 3 | task 0 | graphs reused = 61
|
||
|
|
0.21.989.046 I slot release: id 3 | task 0 | stop processing: n_tokens = 137, truncated = 0
|
||
|
|
0.21.989.050 I srv update_slots: all slots are idle
|
||
|
|
0.22.004.886 I srv operator(): Chat format: peg-native
|
||
|
|
0.22.005.121 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.358
|
||
|
|
0.22.005.124 I srv get_availabl: updating prompt cache
|
||
|
|
0.22.005.148 W srv prompt_save: - saving prompt with length 137, total state size = 65.492 MiB (draft: 0.000 MiB)
|
||
|
|
0.22.103.750 I srv load: - looking for better prompt, base f_keep = 0.358, sim = 0.721
|
||
|
|
0.22.103.756 I srv update: - cache state: 1 prompts, 128.305 MiB (limits: 8192.000 MiB, 8192 tokens, 8747 est)
|
||
|
|
0.22.103.757 I srv update: - prompt 0x5a5b91425890: 137 tokens, checkpoints: 1, 128.305 MiB
|
||
|
|
0.22.103.758 I srv get_availabl: prompt cache update took 98.63 ms
|
||
|
|
0.22.103.846 I slot launch_slot_: id 3 | task 65 | processing task, is_child = 0
|
||
|
|
0.22.103.847 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.22.103.848 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.22.103.854 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.22.103.855 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.22.103.859 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.22.103.859 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.22.103.871 I slot update_slots: id 3 | task 65 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.22.119.823 W slot update_slots: id 3 | task 65 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.23.033.016 I slot print_timing: id 3 | task 65 | prompt eval time = 135.10 ms / 22 tokens ( 6.14 ms per token, 162.84 tokens per second)
|
||
|
|
0.23.033.019 I slot print_timing: id 3 | task 65 | eval time = 794.04 ms / 41 tokens ( 19.37 ms per token, 51.63 tokens per second)
|
||
|
|
0.23.033.020 I slot print_timing: id 3 | task 65 | total time = 929.14 ms / 63 tokens
|
||
|
|
0.23.033.021 I slot print_timing: id 3 | task 65 | graphs reused = 100
|
||
|
|
0.23.033.050 I slot release: id 3 | task 65 | stop processing: n_tokens = 108, truncated = 0
|
||
|
|
0.23.033.058 I srv update_slots: all slots are idle
|
||
|
|
0.23.048.721 I srv operator(): Chat format: peg-native
|
||
|
|
0.23.048.953 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.700 (> 0.100 thold), f_keep = 0.454
|
||
|
|
0.23.048.954 I srv get_availabl: updating prompt cache
|
||
|
|
0.23.048.973 W srv prompt_save: - saving prompt with length 108, total state size = 64.925 MiB (draft: 0.000 MiB)
|
||
|
|
0.23.141.229 I srv load: - looking for better prompt, base f_keep = 0.454, sim = 0.700
|
||
|
|
0.23.141.235 I srv update: - cache state: 2 prompts, 256.043 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.23.141.236 I srv update: - prompt 0x5a5b91425890: 137 tokens, checkpoints: 1, 128.305 MiB
|
||
|
|
0.23.141.237 I srv update: - prompt 0x5a5b9125a520: 108 tokens, checkpoints: 1, 127.738 MiB
|
||
|
|
0.23.141.238 I srv get_availabl: prompt cache update took 92.28 ms
|
||
|
|
0.23.141.312 I slot launch_slot_: id 3 | task 108 | processing task, is_child = 0
|
||
|
|
0.23.141.313 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.23.141.314 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.23.141.320 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.23.141.320 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.23.141.331 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.23.141.331 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.23.141.344 I slot update_slots: id 3 | task 108 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.23.157.239 W slot update_slots: id 3 | task 108 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.24.384.944 I slot print_timing: id 3 | task 108 | prompt eval time = 139.04 ms / 24 tokens ( 5.79 ms per token, 172.61 tokens per second)
|
||
|
|
0.24.384.948 I slot print_timing: id 3 | task 108 | eval time = 1104.55 ms / 57 tokens ( 19.38 ms per token, 51.60 tokens per second)
|
||
|
|
0.24.384.948 I slot print_timing: id 3 | task 108 | total time = 1243.59 ms / 81 tokens
|
||
|
|
0.24.384.949 I slot print_timing: id 3 | task 108 | graphs reused = 155
|
||
|
|
0.24.384.980 I slot release: id 3 | task 108 | stop processing: n_tokens = 126, truncated = 0
|
||
|
|
0.24.384.987 I srv update_slots: all slots are idle
|
||
|
|
0.24.400.167 I srv operator(): Chat format: peg-native
|
||
|
|
0.24.400.380 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.725 (> 0.100 thold), f_keep = 0.397
|
||
|
|
0.24.400.381 I srv get_availabl: updating prompt cache
|
||
|
|
0.24.400.402 W srv prompt_save: - saving prompt with length 126, total state size = 65.277 MiB (draft: 0.000 MiB)
|
||
|
|
0.24.489.043 I srv load: - looking for better prompt, base f_keep = 0.397, sim = 0.725
|
||
|
|
0.24.489.052 I srv update: - cache state: 3 prompts, 384.133 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.24.489.053 I srv update: - prompt 0x5a5b91425890: 137 tokens, checkpoints: 1, 128.305 MiB
|
||
|
|
0.24.489.054 I srv update: - prompt 0x5a5b9125a520: 108 tokens, checkpoints: 1, 127.738 MiB
|
||
|
|
0.24.489.054 I srv update: - prompt 0x5a5b9881cbe0: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
0.24.489.056 I srv get_availabl: prompt cache update took 88.67 ms
|
||
|
|
0.24.489.136 I slot launch_slot_: id 3 | task 167 | processing task, is_child = 0
|
||
|
|
0.24.489.138 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.24.489.139 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.24.489.146 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.24.489.147 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.24.489.157 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.24.489.157 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.24.489.176 I slot update_slots: id 3 | task 167 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.24.504.792 W slot update_slots: id 3 | task 167 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.25.733.909 I slot print_timing: id 3 | task 167 | prompt eval time = 137.88 ms / 23 tokens ( 5.99 ms per token, 166.81 tokens per second)
|
||
|
|
0.25.733.912 I slot print_timing: id 3 | task 167 | eval time = 1106.84 ms / 57 tokens ( 19.42 ms per token, 51.50 tokens per second)
|
||
|
|
0.25.733.913 I slot print_timing: id 3 | task 167 | total time = 1244.73 ms / 80 tokens
|
||
|
|
0.25.733.914 I slot print_timing: id 3 | task 167 | graphs reused = 210
|
||
|
|
0.25.733.945 I slot release: id 3 | task 167 | stop processing: n_tokens = 125, truncated = 0
|
||
|
|
0.25.733.952 I srv update_slots: all slots are idle
|
||
|
|
0.25.749.389 I srv operator(): Chat format: peg-native
|
||
|
|
0.25.749.602 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.704 (> 0.100 thold), f_keep = 0.400
|
||
|
|
0.25.749.604 I srv get_availabl: updating prompt cache
|
||
|
|
0.25.749.624 W srv prompt_save: - saving prompt with length 125, total state size = 65.257 MiB (draft: 0.000 MiB)
|
||
|
|
0.25.838.030 I srv load: - looking for better prompt, base f_keep = 0.400, sim = 0.704
|
||
|
|
0.25.838.034 I srv load: - found better prompt with f_keep = 0.413, sim = 0.732
|
||
|
|
0.25.869.276 I srv update: - cache state: 3 prompts, 384.114 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.25.869.280 I srv update: - prompt 0x5a5b91425890: 137 tokens, checkpoints: 1, 128.305 MiB
|
||
|
|
0.25.869.281 I srv update: - prompt 0x5a5b9125a520: 108 tokens, checkpoints: 1, 127.738 MiB
|
||
|
|
0.25.869.282 I srv update: - prompt 0x5a5b9144ce90: 125 tokens, checkpoints: 1, 128.070 MiB
|
||
|
|
0.25.869.283 I srv get_availabl: prompt cache update took 119.68 ms
|
||
|
|
0.25.869.367 I slot launch_slot_: id 3 | task 226 | processing task, is_child = 0
|
||
|
|
0.25.869.370 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.25.869.370 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.25.869.378 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.25.869.379 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.25.869.389 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.25.869.390 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.25.869.409 I slot update_slots: id 3 | task 226 | Checking checkpoint with [45, 45] against 52...
|
||
|
|
0.25.894.550 W slot update_slots: id 3 | task 226 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.27.693.737 I slot print_timing: id 3 | task 226 | prompt eval time = 148.15 ms / 25 tokens ( 5.93 ms per token, 168.75 tokens per second)
|
||
|
|
0.27.693.740 I slot print_timing: id 3 | task 226 | eval time = 1676.17 ms / 86 tokens ( 19.49 ms per token, 51.31 tokens per second)
|
||
|
|
0.27.693.741 I slot print_timing: id 3 | task 226 | total time = 1824.32 ms / 111 tokens
|
||
|
|
0.27.693.741 I slot print_timing: id 3 | task 226 | graphs reused = 294
|
||
|
|
0.27.693.773 I slot release: id 3 | task 226 | stop processing: n_tokens = 156, truncated = 0
|
||
|
|
0.27.693.782 I srv update_slots: all slots are idle
|
||
|
|
0.27.709.091 I srv operator(): Chat format: peg-native
|
||
|
|
0.27.709.306 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.690 (> 0.100 thold), f_keep = 0.314
|
||
|
|
0.27.709.308 I srv get_availabl: updating prompt cache
|
||
|
|
0.27.709.330 W srv prompt_save: - saving prompt with length 156, total state size = 65.863 MiB (draft: 0.000 MiB)
|
||
|
|
0.27.797.603 I srv load: - looking for better prompt, base f_keep = 0.314, sim = 0.690
|
||
|
|
0.27.797.608 I srv load: - found better prompt with f_keep = 0.463, sim = 0.704
|
||
|
|
0.27.828.851 I srv update: - cache state: 3 prompts, 385.052 MiB (limits: 8192.000 MiB, 8192 tokens, 8892 est)
|
||
|
|
0.27.828.855 I srv update: - prompt 0x5a5b91425890: 137 tokens, checkpoints: 1, 128.305 MiB
|
||
|
|
0.27.828.855 I srv update: - prompt 0x5a5b9144ce90: 125 tokens, checkpoints: 1, 128.070 MiB
|
||
|
|
0.27.828.857 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.27.828.858 I srv get_availabl: prompt cache update took 119.55 ms
|
||
|
|
0.27.828.946 I slot launch_slot_: id 3 | task 314 | processing task, is_child = 0
|
||
|
|
0.27.828.948 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.27.828.949 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.27.828.957 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.27.828.958 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.27.828.971 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.27.828.971 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.27.828.986 I slot update_slots: id 3 | task 314 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.27.852.791 W slot update_slots: id 3 | task 314 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.29.298.006 I slot print_timing: id 3 | task 314 | prompt eval time = 146.65 ms / 25 tokens ( 5.87 ms per token, 170.48 tokens per second)
|
||
|
|
0.29.298.010 I slot print_timing: id 3 | task 314 | eval time = 1322.37 ms / 68 tokens ( 19.45 ms per token, 51.42 tokens per second)
|
||
|
|
0.29.298.010 I slot print_timing: id 3 | task 314 | total time = 1469.02 ms / 93 tokens
|
||
|
|
0.29.298.011 I slot print_timing: id 3 | task 314 | graphs reused = 360
|
||
|
|
0.29.298.040 I slot release: id 3 | task 314 | stop processing: n_tokens = 138, truncated = 0
|
||
|
|
0.29.298.047 I srv update_slots: all slots are idle
|
||
|
|
0.29.313.437 I srv operator(): Chat format: peg-native
|
||
|
|
0.29.313.667 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.653 (> 0.100 thold), f_keep = 0.355
|
||
|
|
0.29.313.670 I srv get_availabl: updating prompt cache
|
||
|
|
0.29.313.693 W srv prompt_save: - saving prompt with length 138, total state size = 65.511 MiB (draft: 0.000 MiB)
|
||
|
|
0.29.392.966 I srv load: - looking for better prompt, base f_keep = 0.355, sim = 0.653
|
||
|
|
0.29.392.970 I srv load: - found better prompt with f_keep = 0.365, sim = 0.667
|
||
|
|
0.29.424.888 I srv update: - cache state: 3 prompts, 385.072 MiB (limits: 8192.000 MiB, 8192 tokens, 8913 est)
|
||
|
|
0.29.424.892 I srv update: - prompt 0x5a5b9144ce90: 125 tokens, checkpoints: 1, 128.070 MiB
|
||
|
|
0.29.424.893 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.29.424.893 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.29.424.895 I srv get_availabl: prompt cache update took 111.22 ms
|
||
|
|
0.29.424.981 I slot launch_slot_: id 3 | task 384 | processing task, is_child = 0
|
||
|
|
0.29.424.983 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.29.424.984 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.29.424.992 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.29.424.992 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.29.425.003 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.29.425.003 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.29.425.017 I slot update_slots: id 3 | task 384 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.29.447.641 W slot update_slots: id 3 | task 384 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.30.521.653 I slot print_timing: id 3 | task 384 | prompt eval time = 166.47 ms / 29 tokens ( 5.74 ms per token, 174.21 tokens per second)
|
||
|
|
0.30.521.657 I slot print_timing: id 3 | task 384 | eval time = 930.16 ms / 48 tokens ( 19.38 ms per token, 51.60 tokens per second)
|
||
|
|
0.30.521.658 I slot print_timing: id 3 | task 384 | total time = 1096.63 ms / 77 tokens
|
||
|
|
0.30.521.658 I slot print_timing: id 3 | task 384 | graphs reused = 406
|
||
|
|
0.30.521.691 I slot release: id 3 | task 384 | stop processing: n_tokens = 122, truncated = 0
|
||
|
|
0.30.521.699 I srv update_slots: all slots are idle
|
||
|
|
0.30.537.031 I srv operator(): Chat format: peg-native
|
||
|
|
0.30.537.261 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.769 (> 0.100 thold), f_keep = 0.410
|
||
|
|
0.30.537.263 I srv get_availabl: updating prompt cache
|
||
|
|
0.30.537.291 W srv prompt_save: - saving prompt with length 122, total state size = 65.199 MiB (draft: 0.000 MiB)
|
||
|
|
0.30.619.325 I srv load: - looking for better prompt, base f_keep = 0.410, sim = 0.769
|
||
|
|
0.30.619.333 I srv update: - cache state: 4 prompts, 513.083 MiB (limits: 8192.000 MiB, 8192 tokens, 8637 est)
|
||
|
|
0.30.619.334 I srv update: - prompt 0x5a5b9144ce90: 125 tokens, checkpoints: 1, 128.070 MiB
|
||
|
|
0.30.619.335 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.30.619.335 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.30.619.336 I srv update: - prompt 0x5a5b914cd770: 122 tokens, checkpoints: 1, 128.012 MiB
|
||
|
|
0.30.619.338 I srv get_availabl: prompt cache update took 82.07 ms
|
||
|
|
0.30.619.427 I slot launch_slot_: id 3 | task 434 | processing task, is_child = 0
|
||
|
|
0.30.619.431 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.30.619.432 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.30.619.439 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.30.619.440 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.30.619.445 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.30.619.446 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.30.619.458 I slot update_slots: id 3 | task 434 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.30.635.237 W slot update_slots: id 3 | task 434 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.31.888.268 I slot print_timing: id 3 | task 434 | prompt eval time = 103.04 ms / 19 tokens ( 5.42 ms per token, 184.40 tokens per second)
|
||
|
|
0.31.888.272 I slot print_timing: id 3 | task 434 | eval time = 1165.77 ms / 60 tokens ( 19.43 ms per token, 51.47 tokens per second)
|
||
|
|
0.31.888.273 I slot print_timing: id 3 | task 434 | total time = 1268.80 ms / 79 tokens
|
||
|
|
0.31.888.274 I slot print_timing: id 3 | task 434 | graphs reused = 464
|
||
|
|
0.31.888.311 I slot release: id 3 | task 434 | stop processing: n_tokens = 124, truncated = 0
|
||
|
|
0.31.888.321 I srv update_slots: all slots are idle
|
||
|
|
0.31.903.699 I srv operator(): Chat format: peg-native
|
||
|
|
0.31.903.928 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.636 (> 0.100 thold), f_keep = 0.395
|
||
|
|
0.31.903.931 I srv get_availabl: updating prompt cache
|
||
|
|
0.31.903.959 W srv prompt_save: - saving prompt with length 124, total state size = 65.238 MiB (draft: 0.000 MiB)
|
||
|
|
0.31.985.082 I srv load: - looking for better prompt, base f_keep = 0.395, sim = 0.636
|
||
|
|
0.31.985.091 I srv update: - cache state: 5 prompts, 641.134 MiB (limits: 8192.000 MiB, 8192 tokens, 8496 est)
|
||
|
|
0.31.985.093 I srv update: - prompt 0x5a5b9144ce90: 125 tokens, checkpoints: 1, 128.070 MiB
|
||
|
|
0.31.985.093 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.31.985.094 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.31.985.095 I srv update: - prompt 0x5a5b914cd770: 122 tokens, checkpoints: 1, 128.012 MiB
|
||
|
|
0.31.985.095 I srv update: - prompt 0x5a5b912f8f10: 124 tokens, checkpoints: 1, 128.051 MiB
|
||
|
|
0.31.985.097 I srv get_availabl: prompt cache update took 81.16 ms
|
||
|
|
0.31.985.190 I slot launch_slot_: id 3 | task 496 | processing task, is_child = 0
|
||
|
|
0.31.985.192 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.31.985.193 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.31.985.201 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.31.985.201 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.31.985.206 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.31.985.207 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.31.985.223 I slot update_slots: id 3 | task 496 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.32.001.009 W slot update_slots: id 3 | task 496 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.32.727.317 I slot print_timing: id 3 | task 496 | prompt eval time = 163.19 ms / 31 tokens ( 5.26 ms per token, 189.97 tokens per second)
|
||
|
|
0.32.727.320 I slot print_timing: id 3 | task 496 | eval time = 578.90 ms / 30 tokens ( 19.30 ms per token, 51.82 tokens per second)
|
||
|
|
0.32.727.322 I slot print_timing: id 3 | task 496 | total time = 742.09 ms / 61 tokens
|
||
|
|
0.32.727.322 I slot print_timing: id 3 | task 496 | graphs reused = 492
|
||
|
|
0.32.727.362 I slot release: id 3 | task 496 | stop processing: n_tokens = 106, truncated = 0
|
||
|
|
0.32.727.371 I srv update_slots: all slots are idle
|
||
|
|
0.32.742.620 I srv operator(): Chat format: peg-native
|
||
|
|
0.32.742.858 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.700 (> 0.100 thold), f_keep = 0.462
|
||
|
|
0.32.742.859 I srv get_availabl: updating prompt cache
|
||
|
|
0.32.742.878 W srv prompt_save: - saving prompt with length 106, total state size = 64.886 MiB (draft: 0.000 MiB)
|
||
|
|
0.32.821.433 I srv load: - looking for better prompt, base f_keep = 0.462, sim = 0.700
|
||
|
|
0.32.821.441 I srv update: - cache state: 6 prompts, 768.833 MiB (limits: 8192.000 MiB, 8192 tokens, 8215 est)
|
||
|
|
0.32.821.442 I srv update: - prompt 0x5a5b9144ce90: 125 tokens, checkpoints: 1, 128.070 MiB
|
||
|
|
0.32.821.443 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.32.821.443 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.32.821.444 I srv update: - prompt 0x5a5b914cd770: 122 tokens, checkpoints: 1, 128.012 MiB
|
||
|
|
0.32.821.444 I srv update: - prompt 0x5a5b912f8f10: 124 tokens, checkpoints: 1, 128.051 MiB
|
||
|
|
0.32.821.445 I srv update: - prompt 0x5a5b91425890: 106 tokens, checkpoints: 1, 127.699 MiB
|
||
|
|
0.32.821.447 I srv get_availabl: prompt cache update took 78.59 ms
|
||
|
|
0.32.821.534 I slot launch_slot_: id 3 | task 528 | processing task, is_child = 0
|
||
|
|
0.32.821.536 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.32.821.536 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.32.821.546 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.32.821.546 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.32.821.557 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.32.821.557 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.32.821.576 I slot update_slots: id 3 | task 528 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.32.837.554 W slot update_slots: id 3 | task 528 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.34.588.453 I slot print_timing: id 3 | task 528 | prompt eval time = 140.39 ms / 24 tokens ( 5.85 ms per token, 170.95 tokens per second)
|
||
|
|
0.34.588.456 I slot print_timing: id 3 | task 528 | eval time = 1626.48 ms / 83 tokens ( 19.60 ms per token, 51.03 tokens per second)
|
||
|
|
0.34.588.457 I slot print_timing: id 3 | task 528 | total time = 1766.87 ms / 107 tokens
|
||
|
|
0.34.588.457 I slot print_timing: id 3 | task 528 | graphs reused = 573
|
||
|
|
0.34.588.491 I slot release: id 3 | task 528 | stop processing: n_tokens = 152, truncated = 0
|
||
|
|
0.34.588.499 I srv update_slots: all slots are idle
|
||
|
|
0.34.603.946 I srv operator(): Chat format: peg-native
|
||
|
|
0.34.604.163 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.653 (> 0.100 thold), f_keep = 0.322
|
||
|
|
0.34.604.165 I srv get_availabl: updating prompt cache
|
||
|
|
0.34.604.187 W srv prompt_save: - saving prompt with length 152, total state size = 65.785 MiB (draft: 0.000 MiB)
|
||
|
|
0.34.683.040 I srv load: - looking for better prompt, base f_keep = 0.322, sim = 0.653
|
||
|
|
0.34.683.046 I srv update: - cache state: 7 prompts, 897.432 MiB (limits: 8192.000 MiB, 8192 tokens, 8425 est)
|
||
|
|
0.34.683.047 I srv update: - prompt 0x5a5b9144ce90: 125 tokens, checkpoints: 1, 128.070 MiB
|
||
|
|
0.34.683.048 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.34.683.048 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.34.683.049 I srv update: - prompt 0x5a5b914cd770: 122 tokens, checkpoints: 1, 128.012 MiB
|
||
|
|
0.34.683.050 I srv update: - prompt 0x5a5b912f8f10: 124 tokens, checkpoints: 1, 128.051 MiB
|
||
|
|
0.34.683.050 I srv update: - prompt 0x5a5b91425890: 106 tokens, checkpoints: 1, 127.699 MiB
|
||
|
|
0.34.683.051 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.34.683.052 I srv get_availabl: prompt cache update took 78.89 ms
|
||
|
|
0.34.683.107 I slot launch_slot_: id 3 | task 613 | processing task, is_child = 0
|
||
|
|
0.34.683.108 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.34.683.109 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.34.683.115 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.34.683.115 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.34.683.120 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.34.683.120 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.34.683.128 I slot update_slots: id 3 | task 613 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.34.698.935 W slot update_slots: id 3 | task 613 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.35.876.465 I slot print_timing: id 3 | task 613 | prompt eval time = 161.34 ms / 29 tokens ( 5.56 ms per token, 179.74 tokens per second)
|
||
|
|
0.35.876.468 I slot print_timing: id 3 | task 613 | eval time = 1031.99 ms / 53 tokens ( 19.47 ms per token, 51.36 tokens per second)
|
||
|
|
0.35.876.469 I slot print_timing: id 3 | task 613 | total time = 1193.33 ms / 82 tokens
|
||
|
|
0.35.876.469 I slot print_timing: id 3 | task 613 | graphs reused = 624
|
||
|
|
0.35.876.504 I slot release: id 3 | task 613 | stop processing: n_tokens = 127, truncated = 0
|
||
|
|
0.35.876.512 I srv update_slots: all slots are idle
|
||
|
|
0.35.892.307 I srv operator(): Chat format: peg-native
|
||
|
|
0.35.892.539 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.690 (> 0.100 thold), f_keep = 0.386
|
||
|
|
0.35.892.539 I srv get_availabl: updating prompt cache
|
||
|
|
0.35.892.559 W srv prompt_save: - saving prompt with length 127, total state size = 65.296 MiB (draft: 0.000 MiB)
|
||
|
|
0.35.973.570 I srv load: - looking for better prompt, base f_keep = 0.386, sim = 0.690
|
||
|
|
0.35.973.576 I srv load: - found better prompt with f_keep = 0.400, sim = 0.704
|
||
|
|
0.36.004.854 I srv update: - cache state: 7 prompts, 897.471 MiB (limits: 8192.000 MiB, 8192 tokens, 8443 est)
|
||
|
|
0.36.004.858 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.36.004.858 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.36.004.859 I srv update: - prompt 0x5a5b914cd770: 122 tokens, checkpoints: 1, 128.012 MiB
|
||
|
|
0.36.004.860 I srv update: - prompt 0x5a5b912f8f10: 124 tokens, checkpoints: 1, 128.051 MiB
|
||
|
|
0.36.004.861 I srv update: - prompt 0x5a5b91425890: 106 tokens, checkpoints: 1, 127.699 MiB
|
||
|
|
0.36.004.862 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.36.004.863 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.36.004.864 I srv get_availabl: prompt cache update took 112.32 ms
|
||
|
|
0.36.004.945 I slot launch_slot_: id 3 | task 668 | processing task, is_child = 0
|
||
|
|
0.36.004.947 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.36.004.948 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.36.004.957 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.36.004.957 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.36.004.963 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.36.004.963 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.36.004.980 I slot update_slots: id 3 | task 668 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.36.026.951 W slot update_slots: id 3 | task 668 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.37.099.773 I slot print_timing: id 3 | task 668 | prompt eval time = 144.60 ms / 25 tokens ( 5.78 ms per token, 172.89 tokens per second)
|
||
|
|
0.37.099.776 I slot print_timing: id 3 | task 668 | eval time = 950.18 ms / 49 tokens ( 19.39 ms per token, 51.57 tokens per second)
|
||
|
|
0.37.099.777 I slot print_timing: id 3 | task 668 | total time = 1094.79 ms / 74 tokens
|
||
|
|
0.37.099.777 I slot print_timing: id 3 | task 668 | graphs reused = 671
|
||
|
|
0.37.099.810 I slot release: id 3 | task 668 | stop processing: n_tokens = 119, truncated = 0
|
||
|
|
0.37.099.825 I srv update_slots: all slots are idle
|
||
|
|
0.37.115.705 I srv operator(): Chat format: peg-native
|
||
|
|
0.37.115.926 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.681 (> 0.100 thold), f_keep = 0.412
|
||
|
|
0.37.115.927 I srv get_availabl: updating prompt cache
|
||
|
|
0.37.115.951 W srv prompt_save: - saving prompt with length 119, total state size = 65.140 MiB (draft: 0.000 MiB)
|
||
|
|
0.37.199.844 I srv load: - looking for better prompt, base f_keep = 0.412, sim = 0.681
|
||
|
|
0.37.199.853 I srv update: - cache state: 8 prompts, 1025.424 MiB (limits: 8192.000 MiB, 8192 tokens, 8340 est)
|
||
|
|
0.37.199.854 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.37.199.855 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.37.199.856 I srv update: - prompt 0x5a5b914cd770: 122 tokens, checkpoints: 1, 128.012 MiB
|
||
|
|
0.37.199.856 I srv update: - prompt 0x5a5b912f8f10: 124 tokens, checkpoints: 1, 128.051 MiB
|
||
|
|
0.37.199.857 I srv update: - prompt 0x5a5b91425890: 106 tokens, checkpoints: 1, 127.699 MiB
|
||
|
|
0.37.199.858 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.37.199.858 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.37.199.859 I srv update: - prompt 0x5a5b91273c00: 119 tokens, checkpoints: 1, 127.953 MiB
|
||
|
|
0.37.199.860 I srv get_availabl: prompt cache update took 83.93 ms
|
||
|
|
0.37.199.943 I slot launch_slot_: id 3 | task 719 | processing task, is_child = 0
|
||
|
|
0.37.199.945 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.199.946 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.199.953 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.199.953 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.199.964 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.199.964 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.199.977 I slot update_slots: id 3 | task 719 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.37.215.629 W slot update_slots: id 3 | task 719 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.37.683.175 I slot print_timing: id 3 | task 719 | prompt eval time = 139.83 ms / 26 tokens ( 5.38 ms per token, 185.94 tokens per second)
|
||
|
|
0.37.683.179 I slot print_timing: id 3 | task 719 | eval time = 343.36 ms / 18 tokens ( 19.08 ms per token, 52.42 tokens per second)
|
||
|
|
0.37.683.179 I slot print_timing: id 3 | task 719 | total time = 483.19 ms / 44 tokens
|
||
|
|
0.37.683.180 I slot print_timing: id 3 | task 719 | graphs reused = 687
|
||
|
|
0.37.683.212 I slot release: id 3 | task 719 | stop processing: n_tokens = 89, truncated = 0
|
||
|
|
0.37.683.219 I srv update_slots: all slots are idle
|
||
|
|
0.37.698.672 I srv operator(): Chat format: peg-native
|
||
|
|
0.37.698.910 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.690 (> 0.100 thold), f_keep = 0.551
|
||
|
|
0.37.698.957 I slot launch_slot_: id 3 | task 739 | processing task, is_child = 0
|
||
|
|
0.37.698.959 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.698.960 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.698.967 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.698.967 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.698.973 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.698.973 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.698.982 I slot update_slots: id 3 | task 739 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.37.715.099 W slot update_slots: id 3 | task 739 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.38.874.828 I slot print_timing: id 3 | task 739 | prompt eval time = 138.25 ms / 25 tokens ( 5.53 ms per token, 180.84 tokens per second)
|
||
|
|
0.38.874.832 I slot print_timing: id 3 | task 739 | eval time = 1037.60 ms / 53 tokens ( 19.58 ms per token, 51.08 tokens per second)
|
||
|
|
0.38.874.833 I slot print_timing: id 3 | task 739 | total time = 1175.84 ms / 78 tokens
|
||
|
|
0.38.874.833 I slot print_timing: id 3 | task 739 | graphs reused = 738
|
||
|
|
0.38.874.866 I slot release: id 3 | task 739 | stop processing: n_tokens = 123, truncated = 0
|
||
|
|
0.38.874.874 I srv update_slots: all slots are idle
|
||
|
|
0.38.890.106 I srv operator(): Chat format: peg-native
|
||
|
|
0.38.890.318 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.671 (> 0.100 thold), f_keep = 0.398
|
||
|
|
0.38.890.320 I srv get_availabl: updating prompt cache
|
||
|
|
0.38.890.338 W srv prompt_save: - saving prompt with length 123, total state size = 65.218 MiB (draft: 0.000 MiB)
|
||
|
|
0.38.969.293 I srv load: - looking for better prompt, base f_keep = 0.398, sim = 0.671
|
||
|
|
0.38.969.299 I srv load: - found better prompt with f_keep = 0.410, sim = 0.685
|
||
|
|
0.39.001.745 I srv update: - cache state: 8 prompts, 1025.443 MiB (limits: 8192.000 MiB, 8192 tokens, 8348 est)
|
||
|
|
0.39.001.748 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.39.001.749 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.39.001.750 I srv update: - prompt 0x5a5b912f8f10: 124 tokens, checkpoints: 1, 128.051 MiB
|
||
|
|
0.39.001.751 I srv update: - prompt 0x5a5b91425890: 106 tokens, checkpoints: 1, 127.699 MiB
|
||
|
|
0.39.001.752 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.39.001.753 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.39.001.753 I srv update: - prompt 0x5a5b91273c00: 119 tokens, checkpoints: 1, 127.953 MiB
|
||
|
|
0.39.001.754 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.39.001.755 I srv get_availabl: prompt cache update took 111.44 ms
|
||
|
|
0.39.001.846 I slot launch_slot_: id 3 | task 794 | processing task, is_child = 0
|
||
|
|
0.39.001.849 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.39.001.849 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.39.001.875 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.39.001.877 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.39.001.883 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.39.001.884 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.39.001.898 I slot update_slots: id 3 | task 794 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.39.027.310 W slot update_slots: id 3 | task 794 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.39.972.187 I slot print_timing: id 3 | task 794 | prompt eval time = 151.33 ms / 27 tokens ( 5.60 ms per token, 178.42 tokens per second)
|
||
|
|
0.39.972.190 I slot print_timing: id 3 | task 794 | eval time = 818.96 ms / 42 tokens ( 19.50 ms per token, 51.28 tokens per second)
|
||
|
|
0.39.972.191 I slot print_timing: id 3 | task 794 | total time = 970.28 ms / 69 tokens
|
||
|
|
0.39.972.192 I slot print_timing: id 3 | task 794 | graphs reused = 778
|
||
|
|
0.39.972.231 I slot release: id 3 | task 794 | stop processing: n_tokens = 114, truncated = 0
|
||
|
|
0.39.972.239 I srv update_slots: all slots are idle
|
||
|
|
0.39.987.766 I srv operator(): Chat format: peg-native
|
||
|
|
0.39.988.006 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.430
|
||
|
|
0.39.988.008 I srv get_availabl: updating prompt cache
|
||
|
|
0.39.988.037 W srv prompt_save: - saving prompt with length 114, total state size = 65.042 MiB (draft: 0.000 MiB)
|
||
|
|
0.40.068.135 I srv load: - looking for better prompt, base f_keep = 0.430, sim = 0.710
|
||
|
|
0.40.068.141 I srv load: - found better prompt with f_keep = 0.472, sim = 0.725
|
||
|
|
0.40.099.439 I srv update: - cache state: 8 prompts, 1025.600 MiB (limits: 8192.000 MiB, 8192 tokens, 8410 est)
|
||
|
|
0.40.099.443 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.40.099.443 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.40.099.444 I srv update: - prompt 0x5a5b912f8f10: 124 tokens, checkpoints: 1, 128.051 MiB
|
||
|
|
0.40.099.445 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.40.099.446 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.40.099.447 I srv update: - prompt 0x5a5b91273c00: 119 tokens, checkpoints: 1, 127.953 MiB
|
||
|
|
0.40.099.447 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.40.099.448 I srv update: - prompt 0x5a5b90c798c0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.40.099.449 I srv get_availabl: prompt cache update took 111.44 ms
|
||
|
|
0.40.099.533 I slot launch_slot_: id 3 | task 838 | processing task, is_child = 0
|
||
|
|
0.40.099.536 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.40.099.537 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.40.099.544 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.40.099.544 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.40.099.550 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.40.099.550 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.40.099.564 I slot update_slots: id 3 | task 838 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.40.125.105 W slot update_slots: id 3 | task 838 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.41.575.230 I slot print_timing: id 3 | task 838 | prompt eval time = 145.96 ms / 23 tokens ( 6.35 ms per token, 157.57 tokens per second)
|
||
|
|
0.41.575.233 I slot print_timing: id 3 | task 838 | eval time = 1329.70 ms / 68 tokens ( 19.55 ms per token, 51.14 tokens per second)
|
||
|
|
0.41.575.234 I slot print_timing: id 3 | task 838 | total time = 1475.66 ms / 91 tokens
|
||
|
|
0.41.575.234 I slot print_timing: id 3 | task 838 | graphs reused = 844
|
||
|
|
0.41.575.267 I slot release: id 3 | task 838 | stop processing: n_tokens = 136, truncated = 0
|
||
|
|
0.41.575.275 I srv update_slots: all slots are idle
|
||
|
|
0.41.590.663 I srv operator(): Chat format: peg-native
|
||
|
|
0.41.590.902 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.671 (> 0.100 thold), f_keep = 0.360
|
||
|
|
0.41.590.904 I srv get_availabl: updating prompt cache
|
||
|
|
0.41.590.926 W srv prompt_save: - saving prompt with length 136, total state size = 65.472 MiB (draft: 0.000 MiB)
|
||
|
|
0.41.670.436 I srv load: - looking for better prompt, base f_keep = 0.360, sim = 0.671
|
||
|
|
0.41.670.441 I srv load: - found better prompt with f_keep = 0.411, sim = 0.699
|
||
|
|
0.41.701.874 I srv update: - cache state: 8 prompts, 1025.834 MiB (limits: 8192.000 MiB, 8192 tokens, 8504 est)
|
||
|
|
0.41.701.878 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.41.701.878 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.41.701.879 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.41.701.880 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.41.701.881 I srv update: - prompt 0x5a5b91273c00: 119 tokens, checkpoints: 1, 127.953 MiB
|
||
|
|
0.41.701.881 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.41.701.882 I srv update: - prompt 0x5a5b90c798c0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.41.701.883 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.41.701.887 I srv get_availabl: prompt cache update took 110.98 ms
|
||
|
|
0.41.701.984 I slot launch_slot_: id 3 | task 908 | processing task, is_child = 0
|
||
|
|
0.41.701.987 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.41.701.987 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.41.701.994 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.41.701.994 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.41.701.999 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.41.701.999 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.41.702.014 I slot update_slots: id 3 | task 908 | Checking checkpoint with [45, 45] against 51...
|
||
|
|
0.41.722.182 W slot update_slots: id 3 | task 908 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.42.644.884 I slot print_timing: id 3 | task 908 | prompt eval time = 145.38 ms / 27 tokens ( 5.38 ms per token, 185.72 tokens per second)
|
||
|
|
0.42.644.887 I slot print_timing: id 3 | task 908 | eval time = 797.49 ms / 41 tokens ( 19.45 ms per token, 51.41 tokens per second)
|
||
|
|
0.42.644.888 I slot print_timing: id 3 | task 908 | total time = 942.87 ms / 68 tokens
|
||
|
|
0.42.644.889 I slot print_timing: id 3 | task 908 | graphs reused = 883
|
||
|
|
0.42.644.921 I slot release: id 3 | task 908 | stop processing: n_tokens = 113, truncated = 0
|
||
|
|
0.42.644.930 I srv update_slots: all slots are idle
|
||
|
|
0.42.660.435 I srv operator(): Chat format: peg-native
|
||
|
|
0.42.660.658 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.434
|
||
|
|
0.42.660.660 I srv get_availabl: updating prompt cache
|
||
|
|
0.42.660.681 W srv prompt_save: - saving prompt with length 113, total state size = 65.023 MiB (draft: 0.000 MiB)
|
||
|
|
0.42.742.652 I srv load: - looking for better prompt, base f_keep = 0.434, sim = 0.721
|
||
|
|
0.42.742.661 I srv update: - cache state: 9 prompts, 1153.670 MiB (limits: 8192.000 MiB, 8192 tokens, 8364 est)
|
||
|
|
0.42.742.662 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.42.742.663 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.42.742.663 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.42.742.664 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.42.742.665 I srv update: - prompt 0x5a5b91273c00: 119 tokens, checkpoints: 1, 127.953 MiB
|
||
|
|
0.42.742.665 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.42.742.666 I srv update: - prompt 0x5a5b90c798c0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.42.742.666 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.42.742.667 I srv update: - prompt 0x5a5b91062170: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
0.42.742.672 I srv get_availabl: prompt cache update took 82.01 ms
|
||
|
|
0.42.742.748 I slot launch_slot_: id 3 | task 951 | processing task, is_child = 0
|
||
|
|
0.42.742.749 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.42.742.749 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.42.742.756 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.42.742.757 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.42.742.761 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.42.742.761 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.42.742.774 I slot update_slots: id 3 | task 951 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.42.758.597 W slot update_slots: id 3 | task 951 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.43.870.095 I slot print_timing: id 3 | task 951 | prompt eval time = 133.79 ms / 22 tokens ( 6.08 ms per token, 164.44 tokens per second)
|
||
|
|
0.43.870.098 I slot print_timing: id 3 | task 951 | eval time = 993.53 ms / 51 tokens ( 19.48 ms per token, 51.33 tokens per second)
|
||
|
|
0.43.870.098 I slot print_timing: id 3 | task 951 | total time = 1127.32 ms / 73 tokens
|
||
|
|
0.43.870.099 I slot print_timing: id 3 | task 951 | graphs reused = 932
|
||
|
|
0.43.870.130 I slot release: id 3 | task 951 | stop processing: n_tokens = 118, truncated = 0
|
||
|
|
0.43.870.139 I srv update_slots: all slots are idle
|
||
|
|
0.43.885.803 I srv operator(): Chat format: peg-native
|
||
|
|
0.43.886.039 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.415
|
||
|
|
0.43.886.042 I srv get_availabl: updating prompt cache
|
||
|
|
0.43.886.061 W srv prompt_save: - saving prompt with length 118, total state size = 65.120 MiB (draft: 0.000 MiB)
|
||
|
|
0.43.964.528 I srv load: - looking for better prompt, base f_keep = 0.415, sim = 0.731
|
||
|
|
0.43.964.533 I srv load: - found better prompt with f_keep = 0.420, sim = 0.746
|
||
|
|
0.43.996.867 I srv update: - cache state: 9 prompts, 1153.651 MiB (limits: 8192.000 MiB, 8192 tokens, 8357 est)
|
||
|
|
0.43.996.871 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.43.996.872 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.43.996.872 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.43.996.873 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.43.996.874 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.43.996.874 I srv update: - prompt 0x5a5b90c798c0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.43.996.875 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.43.996.875 I srv update: - prompt 0x5a5b91062170: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
0.43.996.876 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
0.43.996.878 I srv get_availabl: prompt cache update took 110.83 ms
|
||
|
|
0.43.996.931 I slot launch_slot_: id 3 | task 1004 | processing task, is_child = 0
|
||
|
|
0.43.996.933 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.43.996.933 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.43.996.940 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.43.996.941 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.43.996.946 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.43.996.946 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.43.996.956 I slot update_slots: id 3 | task 1004 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.44.019.553 W slot update_slots: id 3 | task 1004 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.45.074.351 I slot print_timing: id 3 | task 1004 | prompt eval time = 140.76 ms / 21 tokens ( 6.70 ms per token, 149.18 tokens per second)
|
||
|
|
0.45.074.354 I slot print_timing: id 3 | task 1004 | eval time = 936.62 ms / 48 tokens ( 19.51 ms per token, 51.25 tokens per second)
|
||
|
|
0.45.074.354 I slot print_timing: id 3 | task 1004 | total time = 1077.39 ms / 69 tokens
|
||
|
|
0.45.074.355 I slot print_timing: id 3 | task 1004 | graphs reused = 978
|
||
|
|
0.45.074.386 I slot release: id 3 | task 1004 | stop processing: n_tokens = 114, truncated = 0
|
||
|
|
0.45.074.393 I srv update_slots: all slots are idle
|
||
|
|
0.45.089.683 I srv operator(): Chat format: peg-native
|
||
|
|
0.45.089.917 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.735 (> 0.100 thold), f_keep = 0.439
|
||
|
|
0.45.089.919 I srv get_availabl: updating prompt cache
|
||
|
|
0.45.089.937 W srv prompt_save: - saving prompt with length 114, total state size = 65.042 MiB (draft: 0.000 MiB)
|
||
|
|
0.45.168.189 I srv load: - looking for better prompt, base f_keep = 0.439, sim = 0.735
|
||
|
|
0.45.168.194 I srv load: - found better prompt with f_keep = 0.460, sim = 0.765
|
||
|
|
0.45.198.858 I srv update: - cache state: 9 prompts, 1153.670 MiB (limits: 8192.000 MiB, 8192 tokens, 8364 est)
|
||
|
|
0.45.198.862 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.45.198.863 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.45.198.864 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.45.198.865 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.45.198.865 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.45.198.866 I srv update: - prompt 0x5a5b90c798c0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.45.198.867 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.45.198.868 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
0.45.198.869 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.45.198.871 I srv get_availabl: prompt cache update took 108.95 ms
|
||
|
|
0.45.198.954 I slot launch_slot_: id 3 | task 1054 | processing task, is_child = 0
|
||
|
|
0.45.198.956 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.45.198.957 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.45.198.965 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.45.198.966 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.45.198.971 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.45.198.972 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.45.198.986 I slot update_slots: id 3 | task 1054 | Checking checkpoint with [45, 45] against 52...
|
||
|
|
0.45.218.956 W slot update_slots: id 3 | task 1054 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.45.621.499 I slot print_timing: id 3 | task 1054 | prompt eval time = 137.54 ms / 22 tokens ( 6.25 ms per token, 159.95 tokens per second)
|
||
|
|
0.45.621.503 I slot print_timing: id 3 | task 1054 | eval time = 284.96 ms / 15 tokens ( 19.00 ms per token, 52.64 tokens per second)
|
||
|
|
0.45.621.503 I slot print_timing: id 3 | task 1054 | total time = 422.51 ms / 37 tokens
|
||
|
|
0.45.621.505 I slot print_timing: id 3 | task 1054 | graphs reused = 991
|
||
|
|
0.45.621.537 I slot release: id 3 | task 1054 | stop processing: n_tokens = 82, truncated = 0
|
||
|
|
0.45.621.545 I srv update_slots: all slots are idle
|
||
|
|
0.45.636.922 I srv operator(): Chat format: peg-native
|
||
|
|
0.45.637.155 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.598
|
||
|
|
0.45.637.193 I slot launch_slot_: id 3 | task 1071 | processing task, is_child = 0
|
||
|
|
0.45.637.195 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.45.637.195 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.45.637.201 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.45.637.201 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.45.637.206 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.45.637.206 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.45.637.214 I slot update_slots: id 3 | task 1071 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.45.653.139 W slot update_slots: id 3 | task 1071 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.46.676.964 I slot print_timing: id 3 | task 1071 | prompt eval time = 103.22 ms / 20 tokens ( 5.16 ms per token, 193.75 tokens per second)
|
||
|
|
0.46.676.967 I slot print_timing: id 3 | task 1071 | eval time = 936.52 ms / 48 tokens ( 19.51 ms per token, 51.25 tokens per second)
|
||
|
|
0.46.676.968 I slot print_timing: id 3 | task 1071 | total time = 1039.74 ms / 68 tokens
|
||
|
|
0.46.676.969 I slot print_timing: id 3 | task 1071 | graphs reused = 1037
|
||
|
|
0.46.677.001 I slot release: id 3 | task 1071 | stop processing: n_tokens = 113, truncated = 0
|
||
|
|
0.46.677.008 I srv update_slots: all slots are idle
|
||
|
|
0.46.692.539 I srv operator(): Chat format: peg-native
|
||
|
|
0.46.692.756 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.700 (> 0.100 thold), f_keep = 0.434
|
||
|
|
0.46.692.758 I srv get_availabl: updating prompt cache
|
||
|
|
0.46.692.776 W srv prompt_save: - saving prompt with length 113, total state size = 65.023 MiB (draft: 0.000 MiB)
|
||
|
|
0.46.771.601 I srv load: - looking for better prompt, base f_keep = 0.434, sim = 0.700
|
||
|
|
0.46.771.611 I srv update: - cache state: 10 prompts, 1281.506 MiB (limits: 8192.000 MiB, 8192 tokens, 8252 est)
|
||
|
|
0.46.771.612 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.46.771.613 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.46.771.613 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.46.771.614 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.46.771.619 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.46.771.619 I srv update: - prompt 0x5a5b90c798c0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.46.771.620 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.46.771.621 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
0.46.771.621 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.46.771.622 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
0.46.771.626 I srv get_availabl: prompt cache update took 78.87 ms
|
||
|
|
0.46.771.714 I slot launch_slot_: id 3 | task 1121 | processing task, is_child = 0
|
||
|
|
0.46.771.714 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.771.715 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.771.722 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.771.722 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.771.727 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.771.727 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.771.741 I slot update_slots: id 3 | task 1121 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.46.787.613 W slot update_slots: id 3 | task 1121 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.47.783.326 I slot print_timing: id 3 | task 1121 | prompt eval time = 135.28 ms / 24 tokens ( 5.64 ms per token, 177.40 tokens per second)
|
||
|
|
0.47.783.329 I slot print_timing: id 3 | task 1121 | eval time = 876.30 ms / 45 tokens ( 19.47 ms per token, 51.35 tokens per second)
|
||
|
|
0.47.783.329 I slot print_timing: id 3 | task 1121 | total time = 1011.58 ms / 69 tokens
|
||
|
|
0.47.783.330 I slot print_timing: id 3 | task 1121 | graphs reused = 1080
|
||
|
|
0.47.783.361 I slot release: id 3 | task 1121 | stop processing: n_tokens = 114, truncated = 0
|
||
|
|
0.47.783.368 I srv update_slots: all slots are idle
|
||
|
|
0.47.799.087 I srv operator(): Chat format: peg-native
|
||
|
|
0.47.799.312 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.430
|
||
|
|
0.47.799.313 I srv get_availabl: updating prompt cache
|
||
|
|
0.47.799.333 W srv prompt_save: - saving prompt with length 114, total state size = 65.042 MiB (draft: 0.000 MiB)
|
||
|
|
0.47.878.453 I srv load: - looking for better prompt, base f_keep = 0.430, sim = 0.710
|
||
|
|
0.47.878.460 I srv load: - found better prompt with f_keep = 0.439, sim = 0.725
|
||
|
|
0.47.910.446 I srv update: - cache state: 10 prompts, 1281.506 MiB (limits: 8192.000 MiB, 8192 tokens, 8252 est)
|
||
|
|
0.47.910.449 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.47.910.449 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.47.910.450 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.47.910.451 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.47.910.451 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.47.910.452 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.47.910.453 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
0.47.910.453 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.47.910.454 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
0.47.910.454 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.47.910.456 I srv get_availabl: prompt cache update took 111.14 ms
|
||
|
|
0.47.910.533 I slot launch_slot_: id 3 | task 1168 | processing task, is_child = 0
|
||
|
|
0.47.910.534 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.47.910.535 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.47.910.543 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.47.910.543 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.47.910.548 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.47.910.548 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.47.910.561 I slot update_slots: id 3 | task 1168 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.47.934.422 W slot update_slots: id 3 | task 1168 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.48.735.543 I slot print_timing: id 3 | task 1168 | prompt eval time = 145.52 ms / 23 tokens ( 6.33 ms per token, 158.05 tokens per second)
|
||
|
|
0.48.735.546 I slot print_timing: id 3 | task 1168 | eval time = 679.46 ms / 35 tokens ( 19.41 ms per token, 51.51 tokens per second)
|
||
|
|
0.48.735.547 I slot print_timing: id 3 | task 1168 | total time = 824.98 ms / 58 tokens
|
||
|
|
0.48.735.547 I slot print_timing: id 3 | task 1168 | graphs reused = 1113
|
||
|
|
0.48.735.580 I slot release: id 3 | task 1168 | stop processing: n_tokens = 103, truncated = 0
|
||
|
|
0.48.735.587 I srv update_slots: all slots are idle
|
||
|
|
0.48.750.908 I srv operator(): Chat format: peg-native
|
||
|
|
0.48.751.136 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.653 (> 0.100 thold), f_keep = 0.476
|
||
|
|
0.48.751.138 I srv get_availabl: updating prompt cache
|
||
|
|
0.48.751.162 W srv prompt_save: - saving prompt with length 103, total state size = 64.827 MiB (draft: 0.000 MiB)
|
||
|
|
0.48.830.123 I srv load: - looking for better prompt, base f_keep = 0.476, sim = 0.653
|
||
|
|
0.48.830.133 I srv update: - cache state: 11 prompts, 1409.147 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.48.830.134 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.48.830.135 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.48.830.135 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.48.830.137 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.48.830.140 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.48.830.141 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.48.830.141 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
0.48.830.142 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.48.830.142 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
0.48.830.143 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.48.830.146 I srv update: - prompt 0x5a5b91034900: 103 tokens, checkpoints: 1, 127.640 MiB
|
||
|
|
0.48.830.147 I srv get_availabl: prompt cache update took 79.01 ms
|
||
|
|
0.48.830.230 I slot launch_slot_: id 3 | task 1205 | processing task, is_child = 0
|
||
|
|
0.48.830.231 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.48.830.232 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.48.830.238 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.48.830.238 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.48.830.243 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.48.830.243 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.48.830.254 I slot update_slots: id 3 | task 1205 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.48.846.136 W slot update_slots: id 3 | task 1205 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.49.489.229 I slot print_timing: id 3 | task 1205 | prompt eval time = 158.08 ms / 29 tokens ( 5.45 ms per token, 183.45 tokens per second)
|
||
|
|
0.49.489.233 I slot print_timing: id 3 | task 1205 | eval time = 500.89 ms / 26 tokens ( 19.26 ms per token, 51.91 tokens per second)
|
||
|
|
0.49.489.234 I slot print_timing: id 3 | task 1205 | total time = 658.97 ms / 55 tokens
|
||
|
|
0.49.489.234 I slot print_timing: id 3 | task 1205 | graphs reused = 1137
|
||
|
|
0.49.489.268 I slot release: id 3 | task 1205 | stop processing: n_tokens = 100, truncated = 0
|
||
|
|
0.49.489.275 I srv update_slots: all slots are idle
|
||
|
|
0.49.504.824 I srv operator(): Chat format: peg-native
|
||
|
|
0.49.505.037 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.662 (> 0.100 thold), f_keep = 0.490
|
||
|
|
0.49.505.039 I srv get_availabl: updating prompt cache
|
||
|
|
0.49.505.059 W srv prompt_save: - saving prompt with length 100, total state size = 64.768 MiB (draft: 0.000 MiB)
|
||
|
|
0.49.582.685 I srv load: - looking for better prompt, base f_keep = 0.490, sim = 0.662
|
||
|
|
0.49.582.696 I srv update: - cache state: 12 prompts, 1536.728 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.49.582.697 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.49.582.697 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.49.582.698 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.49.582.699 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.49.582.704 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.49.582.705 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.49.582.705 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
0.49.582.706 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.49.582.707 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
0.49.582.707 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.49.582.709 I srv update: - prompt 0x5a5b91034900: 103 tokens, checkpoints: 1, 127.640 MiB
|
||
|
|
0.49.582.710 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
0.49.582.711 I srv get_availabl: prompt cache update took 77.67 ms
|
||
|
|
0.49.582.793 I slot launch_slot_: id 3 | task 1233 | processing task, is_child = 0
|
||
|
|
0.49.582.795 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.49.582.796 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.49.582.803 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.49.582.803 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.49.582.820 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.49.582.821 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.49.582.833 I slot update_slots: id 3 | task 1233 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.49.598.693 W slot update_slots: id 3 | task 1233 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.50.169.140 I slot print_timing: id 3 | task 1233 | prompt eval time = 143.52 ms / 28 tokens ( 5.13 ms per token, 195.10 tokens per second)
|
||
|
|
0.50.169.143 I slot print_timing: id 3 | task 1233 | eval time = 442.78 ms / 23 tokens ( 19.25 ms per token, 51.94 tokens per second)
|
||
|
|
0.50.169.143 I slot print_timing: id 3 | task 1233 | total time = 586.30 ms / 51 tokens
|
||
|
|
0.50.169.144 I slot print_timing: id 3 | task 1233 | graphs reused = 1158
|
||
|
|
0.50.169.176 I slot release: id 3 | task 1233 | stop processing: n_tokens = 96, truncated = 0
|
||
|
|
0.50.169.184 I srv update_slots: all slots are idle
|
||
|
|
0.50.184.416 I srv operator(): Chat format: peg-native
|
||
|
|
0.50.184.633 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.510
|
||
|
|
0.50.184.696 I slot launch_slot_: id 3 | task 1258 | processing task, is_child = 0
|
||
|
|
0.50.184.697 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.50.184.698 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.50.184.704 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.50.184.705 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.50.184.710 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.50.184.710 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.50.184.718 I slot update_slots: id 3 | task 1258 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.50.200.782 W slot update_slots: id 3 | task 1258 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.50.514.799 I slot print_timing: id 3 | task 1258 | prompt eval time = 104.14 ms / 20 tokens ( 5.21 ms per token, 192.05 tokens per second)
|
||
|
|
0.50.514.804 I slot print_timing: id 3 | task 1258 | eval time = 225.94 ms / 12 tokens ( 18.83 ms per token, 53.11 tokens per second)
|
||
|
|
0.50.514.805 I slot print_timing: id 3 | task 1258 | total time = 330.08 ms / 32 tokens
|
||
|
|
0.50.514.805 I slot print_timing: id 3 | task 1258 | graphs reused = 1168
|
||
|
|
0.50.514.838 I slot release: id 3 | task 1258 | stop processing: n_tokens = 77, truncated = 0
|
||
|
|
0.50.514.847 I srv update_slots: all slots are idle
|
||
|
|
0.50.530.148 I srv operator(): Chat format: peg-native
|
||
|
|
0.50.530.375 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.636
|
||
|
|
0.50.530.436 I slot launch_slot_: id 3 | task 1272 | processing task, is_child = 0
|
||
|
|
0.50.530.438 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.50.530.439 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.50.530.445 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.50.530.446 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.50.530.451 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.50.530.451 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.50.530.459 I slot update_slots: id 3 | task 1272 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.50.546.642 W slot update_slots: id 3 | task 1272 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.51.800.617 I slot print_timing: id 3 | task 1272 | prompt eval time = 135.65 ms / 23 tokens ( 5.90 ms per token, 169.56 tokens per second)
|
||
|
|
0.51.800.621 I slot print_timing: id 3 | task 1272 | eval time = 1134.50 ms / 58 tokens ( 19.56 ms per token, 51.12 tokens per second)
|
||
|
|
0.51.800.621 I slot print_timing: id 3 | task 1272 | total time = 1270.15 ms / 81 tokens
|
||
|
|
0.51.800.622 I slot print_timing: id 3 | task 1272 | graphs reused = 1224
|
||
|
|
0.51.800.650 I slot release: id 3 | task 1272 | stop processing: n_tokens = 126, truncated = 0
|
||
|
|
0.51.800.658 I srv update_slots: all slots are idle
|
||
|
|
0.51.816.179 I srv operator(): Chat format: peg-native
|
||
|
|
0.51.816.407 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.690 (> 0.100 thold), f_keep = 0.389
|
||
|
|
0.51.816.409 I srv get_availabl: updating prompt cache
|
||
|
|
0.51.816.432 W srv prompt_save: - saving prompt with length 126, total state size = 65.277 MiB (draft: 0.000 MiB)
|
||
|
|
0.51.895.132 I srv load: - looking for better prompt, base f_keep = 0.389, sim = 0.690
|
||
|
|
0.51.895.141 I srv update: - cache state: 13 prompts, 1664.818 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.51.895.142 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.51.895.143 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.51.895.143 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.51.895.144 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.51.895.145 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.51.895.148 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.51.895.148 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
0.51.895.149 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.51.895.149 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
0.51.895.150 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.51.895.151 I srv update: - prompt 0x5a5b91034900: 103 tokens, checkpoints: 1, 127.640 MiB
|
||
|
|
0.51.895.151 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
0.51.895.152 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
0.51.895.153 I srv get_availabl: prompt cache update took 78.74 ms
|
||
|
|
0.51.895.238 I slot launch_slot_: id 3 | task 1332 | processing task, is_child = 0
|
||
|
|
0.51.895.239 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.51.895.239 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.51.895.247 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.51.895.247 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.51.895.251 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.51.895.252 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.51.895.263 I slot update_slots: id 3 | task 1332 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.51.911.163 W slot update_slots: id 3 | task 1332 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.52.953.612 I slot print_timing: id 3 | task 1332 | prompt eval time = 140.19 ms / 25 tokens ( 5.61 ms per token, 178.33 tokens per second)
|
||
|
|
0.52.953.615 I slot print_timing: id 3 | task 1332 | eval time = 918.16 ms / 47 tokens ( 19.54 ms per token, 51.19 tokens per second)
|
||
|
|
0.52.953.616 I slot print_timing: id 3 | task 1332 | total time = 1058.34 ms / 72 tokens
|
||
|
|
0.52.953.617 I slot print_timing: id 3 | task 1332 | graphs reused = 1269
|
||
|
|
0.52.953.651 I slot release: id 3 | task 1332 | stop processing: n_tokens = 117, truncated = 0
|
||
|
|
0.52.953.661 I srv update_slots: all slots are idle
|
||
|
|
0.52.968.981 I srv operator(): Chat format: peg-native
|
||
|
|
0.52.969.195 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.605 (> 0.100 thold), f_keep = 0.419
|
||
|
|
0.52.969.196 I srv get_availabl: updating prompt cache
|
||
|
|
0.52.969.215 W srv prompt_save: - saving prompt with length 117, total state size = 65.101 MiB (draft: 0.000 MiB)
|
||
|
|
0.53.046.691 I srv load: - looking for better prompt, base f_keep = 0.419, sim = 0.605
|
||
|
|
0.53.046.698 I srv load: - found better prompt with f_keep = 0.505, sim = 0.642
|
||
|
|
0.53.078.679 I srv update: - cache state: 13 prompts, 1665.092 MiB (limits: 8192.000 MiB, 8192 tokens, 8192 est)
|
||
|
|
0.53.078.682 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.53.078.682 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.53.078.683 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.53.078.684 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.53.078.685 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.53.078.686 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.53.078.686 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
0.53.078.687 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.53.078.688 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
0.53.078.689 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.53.078.690 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
0.53.078.691 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
0.53.078.691 I srv update: - prompt 0x5a5b912f8f40: 117 tokens, checkpoints: 1, 127.914 MiB
|
||
|
|
0.53.078.693 I srv get_availabl: prompt cache update took 109.50 ms
|
||
|
|
0.53.078.779 I slot launch_slot_: id 3 | task 1381 | processing task, is_child = 0
|
||
|
|
0.53.078.781 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.53.078.782 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.53.078.790 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.53.078.790 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.53.078.796 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.53.078.796 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.53.078.811 I slot update_slots: id 3 | task 1381 | Checking checkpoint with [45, 45] against 52...
|
||
|
|
0.53.100.568 W slot update_slots: id 3 | task 1381 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.55.354.474 I slot print_timing: id 3 | task 1381 | prompt eval time = 173.03 ms / 35 tokens ( 4.94 ms per token, 202.28 tokens per second)
|
||
|
|
0.55.354.478 I slot print_timing: id 3 | task 1381 | eval time = 2102.63 ms / 107 tokens ( 19.65 ms per token, 50.89 tokens per second)
|
||
|
|
0.55.354.478 I slot print_timing: id 3 | task 1381 | total time = 2275.66 ms / 142 tokens
|
||
|
|
0.55.354.479 I slot print_timing: id 3 | task 1381 | graphs reused = 1374
|
||
|
|
0.55.354.513 I slot release: id 3 | task 1381 | stop processing: n_tokens = 187, truncated = 0
|
||
|
|
0.55.354.520 I srv update_slots: all slots are idle
|
||
|
|
0.55.369.909 I srv operator(): Chat format: peg-native
|
||
|
|
0.55.370.138 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.685 (> 0.100 thold), f_keep = 0.267
|
||
|
|
0.55.370.140 I srv get_availabl: updating prompt cache
|
||
|
|
0.55.370.165 W srv prompt_save: - saving prompt with length 187, total state size = 66.469 MiB (draft: 0.000 MiB)
|
||
|
|
0.55.450.576 I srv load: - looking for better prompt, base f_keep = 0.267, sim = 0.685
|
||
|
|
0.55.450.588 I srv update: - cache state: 14 prompts, 1794.375 MiB (limits: 8192.000 MiB, 8192 tokens, 8313 est)
|
||
|
|
0.55.450.593 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.55.450.593 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.55.450.594 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.55.450.594 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.55.450.595 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.55.450.598 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.55.450.598 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
0.55.450.599 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.55.450.599 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
0.55.450.600 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.55.450.601 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
0.55.450.604 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
0.55.450.605 I srv update: - prompt 0x5a5b912f8f40: 117 tokens, checkpoints: 1, 127.914 MiB
|
||
|
|
0.55.450.605 I srv update: - prompt 0x5a5b9117dd70: 187 tokens, checkpoints: 1, 129.283 MiB
|
||
|
|
0.55.450.607 I srv get_availabl: prompt cache update took 80.47 ms
|
||
|
|
0.55.450.687 I slot launch_slot_: id 3 | task 1490 | processing task, is_child = 0
|
||
|
|
0.55.450.688 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.55.450.689 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.55.450.696 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.55.450.696 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.55.450.706 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.55.450.707 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.55.450.725 I slot update_slots: id 3 | task 1490 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.55.466.712 W slot update_slots: id 3 | task 1490 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.56.888.659 I slot print_timing: id 3 | task 1490 | prompt eval time = 144.67 ms / 27 tokens ( 5.36 ms per token, 186.64 tokens per second)
|
||
|
|
0.56.888.662 I slot print_timing: id 3 | task 1490 | eval time = 1293.26 ms / 66 tokens ( 19.59 ms per token, 51.03 tokens per second)
|
||
|
|
0.56.888.662 I slot print_timing: id 3 | task 1490 | total time = 1437.93 ms / 93 tokens
|
||
|
|
0.56.888.663 I slot print_timing: id 3 | task 1490 | graphs reused = 1438
|
||
|
|
0.56.888.696 I slot release: id 3 | task 1490 | stop processing: n_tokens = 138, truncated = 0
|
||
|
|
0.56.888.703 I srv update_slots: all slots are idle
|
||
|
|
0.56.904.189 I srv operator(): Chat format: peg-native
|
||
|
|
0.56.904.419 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.355
|
||
|
|
0.56.904.421 I srv get_availabl: updating prompt cache
|
||
|
|
0.56.904.445 W srv prompt_save: - saving prompt with length 138, total state size = 65.511 MiB (draft: 0.000 MiB)
|
||
|
|
0.56.983.130 I srv load: - looking for better prompt, base f_keep = 0.355, sim = 0.721
|
||
|
|
0.56.983.141 I srv update: - cache state: 15 prompts, 1922.699 MiB (limits: 8192.000 MiB, 8192 tokens, 8346 est)
|
||
|
|
0.56.983.142 I srv update: - prompt 0x5a5b90c73400: 156 tokens, checkpoints: 1, 128.677 MiB
|
||
|
|
0.56.983.143 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.56.983.143 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.56.983.144 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.56.983.144 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.56.983.145 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.56.983.145 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
0.56.983.146 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.56.983.146 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
0.56.983.147 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.56.983.147 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
0.56.983.148 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
0.56.983.148 I srv update: - prompt 0x5a5b912f8f40: 117 tokens, checkpoints: 1, 127.914 MiB
|
||
|
|
0.56.983.149 I srv update: - prompt 0x5a5b9117dd70: 187 tokens, checkpoints: 1, 129.283 MiB
|
||
|
|
0.56.983.149 I srv update: - prompt 0x5a5b91329460: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.56.983.150 I srv get_availabl: prompt cache update took 78.73 ms
|
||
|
|
0.56.983.226 I slot launch_slot_: id 3 | task 1558 | processing task, is_child = 0
|
||
|
|
0.56.983.227 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.56.983.228 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.56.983.236 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.56.983.236 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.56.983.246 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.56.983.247 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.56.983.258 I slot update_slots: id 3 | task 1558 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
0.56.999.074 W slot update_slots: id 3 | task 1558 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
0.59.471.081 I slot print_timing: id 3 | task 1558 | prompt eval time = 136.62 ms / 22 tokens ( 6.21 ms per token, 161.03 tokens per second)
|
||
|
|
0.59.471.084 I slot print_timing: id 3 | task 1558 | eval time = 2351.20 ms / 119 tokens ( 19.76 ms per token, 50.61 tokens per second)
|
||
|
|
0.59.471.085 I slot print_timing: id 3 | task 1558 | total time = 2487.82 ms / 141 tokens
|
||
|
|
0.59.471.086 I slot print_timing: id 3 | task 1558 | graphs reused = 1555
|
||
|
|
0.59.471.116 I slot release: id 3 | task 1558 | stop processing: n_tokens = 186, truncated = 0
|
||
|
|
0.59.471.125 I srv update_slots: all slots are idle
|
||
|
|
0.59.486.362 I srv operator(): Chat format: peg-native
|
||
|
|
0.59.486.588 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.671 (> 0.100 thold), f_keep = 0.263
|
||
|
|
0.59.486.590 I srv get_availabl: updating prompt cache
|
||
|
|
0.59.486.612 W srv prompt_save: - saving prompt with length 186, total state size = 66.450 MiB (draft: 0.000 MiB)
|
||
|
|
0.59.564.413 I srv load: - looking for better prompt, base f_keep = 0.263, sim = 0.671
|
||
|
|
0.59.564.422 I srv load: - found better prompt with f_keep = 0.321, sim = 0.685
|
||
|
|
0.59.595.862 I srv update: - cache state: 15 prompts, 1923.286 MiB (limits: 8192.000 MiB, 8192 tokens, 8471 est)
|
||
|
|
0.59.595.866 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.59.595.867 I srv update: - prompt 0x5a5b912f8ca0: 152 tokens, checkpoints: 1, 128.598 MiB
|
||
|
|
0.59.595.868 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
0.59.595.869 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
0.59.595.869 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
0.59.595.870 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
0.59.595.871 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.59.595.871 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
0.59.595.872 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
0.59.595.873 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
0.59.595.874 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
0.59.595.875 I srv update: - prompt 0x5a5b912f8f40: 117 tokens, checkpoints: 1, 127.914 MiB
|
||
|
|
0.59.595.876 I srv update: - prompt 0x5a5b9117dd70: 187 tokens, checkpoints: 1, 129.283 MiB
|
||
|
|
0.59.595.877 I srv update: - prompt 0x5a5b91329460: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
0.59.595.878 I srv update: - prompt 0x5a5b98862670: 186 tokens, checkpoints: 1, 129.263 MiB
|
||
|
|
0.59.595.879 I srv get_availabl: prompt cache update took 109.29 ms
|
||
|
|
0.59.595.963 I slot launch_slot_: id 3 | task 1679 | processing task, is_child = 0
|
||
|
|
0.59.595.965 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.59.595.966 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.59.595.975 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.59.595.976 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.59.595.981 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.59.595.982 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.59.595.997 I slot update_slots: id 3 | task 1679 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
0.59.617.253 W slot update_slots: id 3 | task 1679 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
1.01.215.369 I slot print_timing: id 3 | task 1679 | prompt eval time = 145.96 ms / 27 tokens ( 5.41 ms per token, 184.98 tokens per second)
|
||
|
|
1.01.215.372 I slot print_timing: id 3 | task 1679 | eval time = 1473.41 ms / 75 tokens ( 19.65 ms per token, 50.90 tokens per second)
|
||
|
|
1.01.215.373 I slot print_timing: id 3 | task 1679 | total time = 1619.37 ms / 102 tokens
|
||
|
|
1.01.215.373 I slot print_timing: id 3 | task 1679 | graphs reused = 1628
|
||
|
|
1.01.215.406 I slot release: id 3 | task 1679 | stop processing: n_tokens = 147, truncated = 0
|
||
|
|
1.01.215.414 I srv update_slots: all slots are idle
|
||
|
|
1.01.230.811 I srv operator(): Chat format: peg-native
|
||
|
|
1.01.231.048 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.658 (> 0.100 thold), f_keep = 0.340
|
||
|
|
1.01.231.050 I srv get_availabl: updating prompt cache
|
||
|
|
1.01.231.069 W srv prompt_save: - saving prompt with length 147, total state size = 65.687 MiB (draft: 0.000 MiB)
|
||
|
|
1.01.309.165 I srv load: - looking for better prompt, base f_keep = 0.340, sim = 0.658
|
||
|
|
1.01.309.172 I srv load: - found better prompt with f_keep = 0.342, sim = 0.684
|
||
|
|
1.01.339.842 I srv update: - cache state: 15 prompts, 1923.188 MiB (limits: 8192.000 MiB, 8192 tokens, 8451 est)
|
||
|
|
1.01.339.846 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.01.339.847 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
1.01.339.847 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
1.01.339.848 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
1.01.339.849 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
1.01.339.850 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
1.01.339.850 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
1.01.339.851 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
1.01.339.852 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.01.339.853 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
1.01.339.853 I srv update: - prompt 0x5a5b912f8f40: 117 tokens, checkpoints: 1, 127.914 MiB
|
||
|
|
1.01.339.854 I srv update: - prompt 0x5a5b9117dd70: 187 tokens, checkpoints: 1, 129.283 MiB
|
||
|
|
1.01.339.857 I srv update: - prompt 0x5a5b91329460: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.01.339.858 I srv update: - prompt 0x5a5b98862670: 186 tokens, checkpoints: 1, 129.263 MiB
|
||
|
|
1.01.339.859 I srv update: - prompt 0x5a5b9117f4e0: 147 tokens, checkpoints: 1, 128.501 MiB
|
||
|
|
1.01.339.860 I srv get_availabl: prompt cache update took 108.81 ms
|
||
|
|
1.01.339.947 I slot launch_slot_: id 3 | task 1756 | processing task, is_child = 0
|
||
|
|
1.01.339.963 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.01.339.967 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.01.339.975 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.01.339.975 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.01.339.981 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.01.339.981 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.01.339.996 I slot update_slots: id 3 | task 1756 | Checking checkpoint with [45, 45] against 52...
|
||
|
|
1.01.365.584 W slot update_slots: id 3 | task 1756 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
1.03.480.486 I slot print_timing: id 3 | task 1756 | prompt eval time = 174.07 ms / 30 tokens ( 5.80 ms per token, 172.35 tokens per second)
|
||
|
|
1.03.480.489 I slot print_timing: id 3 | task 1756 | eval time = 1966.42 ms / 100 tokens ( 19.66 ms per token, 50.85 tokens per second)
|
||
|
|
1.03.480.489 I slot print_timing: id 3 | task 1756 | total time = 2140.49 ms / 130 tokens
|
||
|
|
1.03.480.490 I slot print_timing: id 3 | task 1756 | graphs reused = 1726
|
||
|
|
1.03.480.523 I slot release: id 3 | task 1756 | stop processing: n_tokens = 175, truncated = 0
|
||
|
|
1.03.480.531 I srv update_slots: all slots are idle
|
||
|
|
1.03.495.773 I srv operator(): Chat format: peg-native
|
||
|
|
1.03.496.014 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.280
|
||
|
|
1.03.496.016 I srv get_availabl: updating prompt cache
|
||
|
|
1.03.496.037 W srv prompt_save: - saving prompt with length 175, total state size = 66.235 MiB (draft: 0.000 MiB)
|
||
|
|
1.03.578.226 I srv load: - looking for better prompt, base f_keep = 0.280, sim = 0.721
|
||
|
|
1.03.578.238 I srv update: - cache state: 16 prompts, 2052.236 MiB (limits: 8192.000 MiB, 8192 tokens, 8618 est)
|
||
|
|
1.03.578.239 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.03.578.239 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
1.03.578.240 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
1.03.578.241 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
1.03.578.241 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
1.03.578.243 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
1.03.578.244 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
1.03.578.244 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
1.03.578.245 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.03.578.245 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
1.03.578.246 I srv update: - prompt 0x5a5b912f8f40: 117 tokens, checkpoints: 1, 127.914 MiB
|
||
|
|
1.03.578.246 I srv update: - prompt 0x5a5b9117dd70: 187 tokens, checkpoints: 1, 129.283 MiB
|
||
|
|
1.03.578.247 I srv update: - prompt 0x5a5b91329460: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.03.578.247 I srv update: - prompt 0x5a5b98862670: 186 tokens, checkpoints: 1, 129.263 MiB
|
||
|
|
1.03.578.248 I srv update: - prompt 0x5a5b9117f4e0: 147 tokens, checkpoints: 1, 128.501 MiB
|
||
|
|
1.03.578.248 I srv update: - prompt 0x5a5b9115a5a0: 175 tokens, checkpoints: 1, 129.048 MiB
|
||
|
|
1.03.578.249 I srv get_availabl: prompt cache update took 82.23 ms
|
||
|
|
1.03.578.335 I slot launch_slot_: id 3 | task 1858 | processing task, is_child = 0
|
||
|
|
1.03.578.335 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.03.578.336 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.03.578.343 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.03.578.343 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.03.578.347 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.03.578.347 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.03.578.360 I slot update_slots: id 3 | task 1858 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.03.594.389 W slot update_slots: id 3 | task 1858 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
1.04.359.041 I slot print_timing: id 3 | task 1858 | prompt eval time = 137.30 ms / 22 tokens ( 6.24 ms per token, 160.24 tokens per second)
|
||
|
|
1.04.359.045 I slot print_timing: id 3 | task 1858 | eval time = 643.38 ms / 33 tokens ( 19.50 ms per token, 51.29 tokens per second)
|
||
|
|
1.04.359.045 I slot print_timing: id 3 | task 1858 | total time = 780.68 ms / 55 tokens
|
||
|
|
1.04.359.046 I slot print_timing: id 3 | task 1858 | graphs reused = 1757
|
||
|
|
1.04.359.075 I slot release: id 3 | task 1858 | stop processing: n_tokens = 100, truncated = 0
|
||
|
|
1.04.359.084 I srv update_slots: all slots are idle
|
||
|
|
1.04.374.274 I srv operator(): Chat format: peg-native
|
||
|
|
1.04.374.495 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.766 (> 0.100 thold), f_keep = 0.490
|
||
|
|
1.04.374.498 I srv get_availabl: updating prompt cache
|
||
|
|
1.04.374.517 W srv prompt_save: - saving prompt with length 100, total state size = 64.768 MiB (draft: 0.000 MiB)
|
||
|
|
1.04.451.842 I srv load: - looking for better prompt, base f_keep = 0.490, sim = 0.766
|
||
|
|
1.04.451.856 I srv update: - cache state: 17 prompts, 2179.818 MiB (limits: 8192.000 MiB, 8192 tokens, 8489 est)
|
||
|
|
1.04.451.858 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.04.451.862 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
1.04.451.863 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
1.04.451.864 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
1.04.451.864 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
1.04.451.865 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
1.04.451.865 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
1.04.451.866 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
1.04.451.866 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.04.451.870 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
1.04.451.870 I srv update: - prompt 0x5a5b912f8f40: 117 tokens, checkpoints: 1, 127.914 MiB
|
||
|
|
1.04.451.871 I srv update: - prompt 0x5a5b9117dd70: 187 tokens, checkpoints: 1, 129.283 MiB
|
||
|
|
1.04.451.871 I srv update: - prompt 0x5a5b91329460: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.04.451.872 I srv update: - prompt 0x5a5b98862670: 186 tokens, checkpoints: 1, 129.263 MiB
|
||
|
|
1.04.451.872 I srv update: - prompt 0x5a5b9117f4e0: 147 tokens, checkpoints: 1, 128.501 MiB
|
||
|
|
1.04.451.873 I srv update: - prompt 0x5a5b9115a5a0: 175 tokens, checkpoints: 1, 129.048 MiB
|
||
|
|
1.04.451.874 I srv update: - prompt 0x5a5b9106a380: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.04.451.876 I srv get_availabl: prompt cache update took 77.38 ms
|
||
|
|
1.04.451.957 I slot launch_slot_: id 3 | task 1893 | processing task, is_child = 0
|
||
|
|
1.04.451.958 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.04.451.959 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.04.451.965 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.04.451.965 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.04.451.976 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.04.451.976 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.04.451.989 I slot update_slots: id 3 | task 1893 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.04.467.879 W slot update_slots: id 3 | task 1893 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
1.06.127.088 I slot print_timing: id 3 | task 1893 | prompt eval time = 101.64 ms / 18 tokens ( 5.65 ms per token, 177.09 tokens per second)
|
||
|
|
1.06.127.092 I slot print_timing: id 3 | task 1893 | eval time = 1573.45 ms / 80 tokens ( 19.67 ms per token, 50.84 tokens per second)
|
||
|
|
1.06.127.093 I slot print_timing: id 3 | task 1893 | total time = 1675.09 ms / 98 tokens
|
||
|
|
1.06.127.094 I slot print_timing: id 3 | task 1893 | graphs reused = 1835
|
||
|
|
1.06.127.133 I slot release: id 3 | task 1893 | stop processing: n_tokens = 143, truncated = 0
|
||
|
|
1.06.127.143 I srv update_slots: all slots are idle
|
||
|
|
1.06.142.637 I srv operator(): Chat format: peg-native
|
||
|
|
1.06.142.874 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.681 (> 0.100 thold), f_keep = 0.343
|
||
|
|
1.06.142.877 I srv get_availabl: updating prompt cache
|
||
|
|
1.06.142.906 W srv prompt_save: - saving prompt with length 143, total state size = 65.609 MiB (draft: 0.000 MiB)
|
||
|
|
1.06.220.349 I srv load: - looking for better prompt, base f_keep = 0.343, sim = 0.681
|
||
|
|
1.06.220.363 I srv update: - cache state: 18 prompts, 2308.240 MiB (limits: 8192.000 MiB, 8192 tokens, 8524 est)
|
||
|
|
1.06.220.364 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.06.220.366 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
1.06.220.371 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
1.06.220.371 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
1.06.220.372 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
1.06.220.373 I srv update: - prompt 0x5a5b98949de0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
1.06.220.374 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
1.06.220.374 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
1.06.220.377 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.06.220.378 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
1.06.220.379 I srv update: - prompt 0x5a5b912f8f40: 117 tokens, checkpoints: 1, 127.914 MiB
|
||
|
|
1.06.220.380 I srv update: - prompt 0x5a5b9117dd70: 187 tokens, checkpoints: 1, 129.283 MiB
|
||
|
|
1.06.220.380 I srv update: - prompt 0x5a5b91329460: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.06.220.384 I srv update: - prompt 0x5a5b98862670: 186 tokens, checkpoints: 1, 129.263 MiB
|
||
|
|
1.06.220.384 I srv update: - prompt 0x5a5b9117f4e0: 147 tokens, checkpoints: 1, 128.501 MiB
|
||
|
|
1.06.220.385 I srv update: - prompt 0x5a5b9115a5a0: 175 tokens, checkpoints: 1, 129.048 MiB
|
||
|
|
1.06.220.386 I srv update: - prompt 0x5a5b9106a380: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.06.220.387 I srv update: - prompt 0x5a5b982ef8f0: 143 tokens, checkpoints: 1, 128.422 MiB
|
||
|
|
1.06.220.390 I srv get_availabl: prompt cache update took 77.51 ms
|
||
|
|
1.06.220.478 I slot launch_slot_: id 3 | task 1975 | processing task, is_child = 0
|
||
|
|
1.06.220.479 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.06.220.480 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.06.220.489 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.06.220.489 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.06.220.495 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.06.220.495 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.06.220.508 I slot update_slots: id 3 | task 1975 | Checking checkpoint with [45, 45] against 49...
|
||
|
|
1.06.236.299 W slot update_slots: id 3 | task 1975 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
1.08.129.163 I slot print_timing: id 3 | task 1975 | prompt eval time = 141.92 ms / 26 tokens ( 5.46 ms per token, 183.20 tokens per second)
|
||
|
|
1.08.129.166 I slot print_timing: id 3 | task 1975 | eval time = 1766.73 ms / 91 tokens ( 19.41 ms per token, 51.51 tokens per second)
|
||
|
|
1.08.129.167 I slot print_timing: id 3 | task 1975 | total time = 1908.65 ms / 117 tokens
|
||
|
|
1.08.129.167 I slot print_timing: id 3 | task 1975 | graphs reused = 1924
|
||
|
|
1.08.129.197 I slot release: id 3 | task 1975 | stop processing: n_tokens = 162, truncated = 0
|
||
|
|
1.08.129.206 I srv update_slots: all slots are idle
|
||
|
|
1.08.144.575 I srv operator(): Chat format: peg-native
|
||
|
|
1.08.144.805 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.302
|
||
|
|
1.08.144.807 I srv get_availabl: updating prompt cache
|
||
|
|
1.08.144.835 W srv prompt_save: - saving prompt with length 162, total state size = 65.981 MiB (draft: 0.000 MiB)
|
||
|
|
1.08.224.225 I srv load: - looking for better prompt, base f_keep = 0.302, sim = 0.721
|
||
|
|
1.08.224.235 I srv load: - found better prompt with f_keep = 0.439, sim = 0.735
|
||
|
|
1.08.256.201 I srv update: - cache state: 18 prompts, 2309.178 MiB (limits: 8192.000 MiB, 8192 tokens, 8691 est)
|
||
|
|
1.08.256.204 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.08.256.205 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
1.08.256.206 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
1.08.256.207 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
1.08.256.208 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
1.08.256.208 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
1.08.256.209 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
1.08.256.210 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.08.256.210 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
1.08.256.211 I srv update: - prompt 0x5a5b912f8f40: 117 tokens, checkpoints: 1, 127.914 MiB
|
||
|
|
1.08.256.212 I srv update: - prompt 0x5a5b9117dd70: 187 tokens, checkpoints: 1, 129.283 MiB
|
||
|
|
1.08.256.213 I srv update: - prompt 0x5a5b91329460: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.08.256.214 I srv update: - prompt 0x5a5b98862670: 186 tokens, checkpoints: 1, 129.263 MiB
|
||
|
|
1.08.256.215 I srv update: - prompt 0x5a5b9117f4e0: 147 tokens, checkpoints: 1, 128.501 MiB
|
||
|
|
1.08.256.216 I srv update: - prompt 0x5a5b9115a5a0: 175 tokens, checkpoints: 1, 129.048 MiB
|
||
|
|
1.08.256.216 I srv update: - prompt 0x5a5b9106a380: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.08.256.217 I srv update: - prompt 0x5a5b982ef8f0: 143 tokens, checkpoints: 1, 128.422 MiB
|
||
|
|
1.08.256.218 I srv update: - prompt 0x5a5b91237610: 162 tokens, checkpoints: 1, 128.794 MiB
|
||
|
|
1.08.256.219 I srv get_availabl: prompt cache update took 111.41 ms
|
||
|
|
1.08.256.307 I slot launch_slot_: id 3 | task 2068 | processing task, is_child = 0
|
||
|
|
1.08.256.308 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.08.256.309 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.08.256.316 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.08.256.317 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.08.256.322 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.08.256.323 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.08.256.336 I slot update_slots: id 3 | task 2068 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
1.08.278.893 W slot update_slots: id 3 | task 2068 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
1.09.660.736 I slot print_timing: id 3 | task 2068 | prompt eval time = 142.95 ms / 22 tokens ( 6.50 ms per token, 153.90 tokens per second)
|
||
|
|
1.09.660.739 I slot print_timing: id 3 | task 2068 | eval time = 1261.44 ms / 65 tokens ( 19.41 ms per token, 51.53 tokens per second)
|
||
|
|
1.09.660.739 I slot print_timing: id 3 | task 2068 | total time = 1404.39 ms / 87 tokens
|
||
|
|
1.09.660.740 I slot print_timing: id 3 | task 2068 | graphs reused = 1987
|
||
|
|
1.09.660.769 I slot release: id 3 | task 2068 | stop processing: n_tokens = 132, truncated = 0
|
||
|
|
1.09.660.778 I srv update_slots: all slots are idle
|
||
|
|
1.09.676.380 I srv operator(): Chat format: peg-native
|
||
|
|
1.09.676.611 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.746 (> 0.100 thold), f_keep = 0.379
|
||
|
|
1.09.676.613 I srv get_availabl: updating prompt cache
|
||
|
|
1.09.676.635 W srv prompt_save: - saving prompt with length 132, total state size = 65.394 MiB (draft: 0.000 MiB)
|
||
|
|
1.09.754.847 I srv load: - looking for better prompt, base f_keep = 0.379, sim = 0.746
|
||
|
|
1.09.754.861 I srv update: - cache state: 19 prompts, 2437.386 MiB (limits: 8192.000 MiB, 8192 tokens, 8678 est)
|
||
|
|
1.09.754.862 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.09.754.862 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
1.09.754.863 I srv update: - prompt 0x5a5b910b35f0: 123 tokens, checkpoints: 1, 128.031 MiB
|
||
|
|
1.09.754.864 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
1.09.754.864 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
1.09.754.865 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
1.09.754.865 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
1.09.754.866 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.09.754.866 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
1.09.754.867 I srv update: - prompt 0x5a5b912f8f40: 117 tokens, checkpoints: 1, 127.914 MiB
|
||
|
|
1.09.754.867 I srv update: - prompt 0x5a5b9117dd70: 187 tokens, checkpoints: 1, 129.283 MiB
|
||
|
|
1.09.754.868 I srv update: - prompt 0x5a5b91329460: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.09.754.868 I srv update: - prompt 0x5a5b98862670: 186 tokens, checkpoints: 1, 129.263 MiB
|
||
|
|
1.09.754.869 I srv update: - prompt 0x5a5b9117f4e0: 147 tokens, checkpoints: 1, 128.501 MiB
|
||
|
|
1.09.754.869 I srv update: - prompt 0x5a5b9115a5a0: 175 tokens, checkpoints: 1, 129.048 MiB
|
||
|
|
1.09.754.870 I srv update: - prompt 0x5a5b9106a380: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.09.754.870 I srv update: - prompt 0x5a5b982ef8f0: 143 tokens, checkpoints: 1, 128.422 MiB
|
||
|
|
1.09.754.871 I srv update: - prompt 0x5a5b91237610: 162 tokens, checkpoints: 1, 128.794 MiB
|
||
|
|
1.09.754.871 I srv update: - prompt 0x5a5b910ca290: 132 tokens, checkpoints: 1, 128.207 MiB
|
||
|
|
1.09.754.872 I srv get_availabl: prompt cache update took 78.26 ms
|
||
|
|
1.09.754.948 I slot launch_slot_: id 3 | task 2135 | processing task, is_child = 0
|
||
|
|
1.09.754.949 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.09.754.950 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.09.754.957 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.09.754.957 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.09.754.968 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.09.754.968 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.09.754.981 I slot update_slots: id 3 | task 2135 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
1.09.770.644 W slot update_slots: id 3 | task 2135 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
1.11.091.752 I slot print_timing: id 3 | task 2135 | prompt eval time = 133.57 ms / 21 tokens ( 6.36 ms per token, 157.22 tokens per second)
|
||
|
|
1.11.091.755 I slot print_timing: id 3 | task 2135 | eval time = 1203.19 ms / 62 tokens ( 19.41 ms per token, 51.53 tokens per second)
|
||
|
|
1.11.091.755 I slot print_timing: id 3 | task 2135 | total time = 1336.76 ms / 83 tokens
|
||
|
|
1.11.091.756 I slot print_timing: id 3 | task 2135 | graphs reused = 2047
|
||
|
|
1.11.091.785 I slot release: id 3 | task 2135 | stop processing: n_tokens = 128, truncated = 0
|
||
|
|
1.11.091.791 I srv update_slots: all slots are idle
|
||
|
|
1.11.107.521 I srv operator(): Chat format: peg-native
|
||
|
|
1.11.107.747 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.700 (> 0.100 thold), f_keep = 0.383
|
||
|
|
1.11.107.748 I srv get_availabl: updating prompt cache
|
||
|
|
1.11.107.771 W srv prompt_save: - saving prompt with length 128, total state size = 65.316 MiB (draft: 0.000 MiB)
|
||
|
|
1.11.184.944 I srv load: - looking for better prompt, base f_keep = 0.383, sim = 0.700
|
||
|
|
1.11.184.955 I srv load: - found better prompt with f_keep = 0.407, sim = 0.714
|
||
|
|
1.11.216.445 I srv update: - cache state: 19 prompts, 2437.483 MiB (limits: 8192.000 MiB, 8192 tokens, 8694 est)
|
||
|
|
1.11.216.448 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.11.216.449 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
1.11.216.449 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
1.11.216.450 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
1.11.216.450 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
1.11.216.451 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
1.11.216.451 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.11.216.452 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
1.11.216.452 I srv update: - prompt 0x5a5b912f8f40: 117 tokens, checkpoints: 1, 127.914 MiB
|
||
|
|
1.11.216.453 I srv update: - prompt 0x5a5b9117dd70: 187 tokens, checkpoints: 1, 129.283 MiB
|
||
|
|
1.11.216.453 I srv update: - prompt 0x5a5b91329460: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.11.216.454 I srv update: - prompt 0x5a5b98862670: 186 tokens, checkpoints: 1, 129.263 MiB
|
||
|
|
1.11.216.454 I srv update: - prompt 0x5a5b9117f4e0: 147 tokens, checkpoints: 1, 128.501 MiB
|
||
|
|
1.11.216.456 I srv update: - prompt 0x5a5b9115a5a0: 175 tokens, checkpoints: 1, 129.048 MiB
|
||
|
|
1.11.216.457 I srv update: - prompt 0x5a5b9106a380: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.11.216.457 I srv update: - prompt 0x5a5b982ef8f0: 143 tokens, checkpoints: 1, 128.422 MiB
|
||
|
|
1.11.216.458 I srv update: - prompt 0x5a5b91237610: 162 tokens, checkpoints: 1, 128.794 MiB
|
||
|
|
1.11.216.458 I srv update: - prompt 0x5a5b910ca290: 132 tokens, checkpoints: 1, 128.207 MiB
|
||
|
|
1.11.216.459 I srv update: - prompt 0x5a5b912c1550: 128 tokens, checkpoints: 1, 128.129 MiB
|
||
|
|
1.11.216.460 I srv get_availabl: prompt cache update took 108.71 ms
|
||
|
|
1.11.216.535 I slot launch_slot_: id 3 | task 2199 | processing task, is_child = 0
|
||
|
|
1.11.216.535 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.11.216.539 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.11.216.547 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.11.216.547 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.11.216.557 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.11.216.557 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.11.216.576 I slot update_slots: id 3 | task 2199 | Checking checkpoint with [45, 45] against 50...
|
||
|
|
1.11.237.824 W slot update_slots: id 3 | task 2199 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
1.12.542.478 I slot print_timing: id 3 | task 2199 | prompt eval time = 142.68 ms / 24 tokens ( 5.94 ms per token, 168.21 tokens per second)
|
||
|
|
1.12.542.481 I slot print_timing: id 3 | task 2199 | eval time = 1183.22 ms / 61 tokens ( 19.40 ms per token, 51.55 tokens per second)
|
||
|
|
1.12.542.482 I slot print_timing: id 3 | task 2199 | total time = 1325.89 ms / 85 tokens
|
||
|
|
1.12.542.483 I slot print_timing: id 3 | task 2199 | graphs reused = 2106
|
||
|
|
1.12.542.514 I slot release: id 3 | task 2199 | stop processing: n_tokens = 130, truncated = 0
|
||
|
|
1.12.542.523 I srv update_slots: all slots are idle
|
||
|
|
1.12.557.858 I srv operator(): Chat format: peg-native
|
||
|
|
1.12.558.083 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.377
|
||
|
|
1.12.558.085 I srv get_availabl: updating prompt cache
|
||
|
|
1.12.558.106 W srv prompt_save: - saving prompt with length 130, total state size = 65.355 MiB (draft: 0.000 MiB)
|
||
|
|
1.12.636.298 I srv load: - looking for better prompt, base f_keep = 0.377, sim = 0.742
|
||
|
|
1.12.636.308 I srv load: - found better prompt with f_keep = 0.386, sim = 0.773
|
||
|
|
1.12.667.887 I srv update: - cache state: 19 prompts, 2437.444 MiB (limits: 8192.000 MiB, 8192 tokens, 8687 est)
|
||
|
|
1.12.667.890 I srv update: - prompt 0x5a5b910a68e0: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.12.667.891 I srv update: - prompt 0x5a5b91269ab0: 127 tokens, checkpoints: 1, 128.110 MiB
|
||
|
|
1.12.667.892 I srv update: - prompt 0x5a5b987c5870: 136 tokens, checkpoints: 1, 128.286 MiB
|
||
|
|
1.12.667.893 I srv update: - prompt 0x5a5b987b5110: 118 tokens, checkpoints: 1, 127.934 MiB
|
||
|
|
1.12.667.893 I srv update: - prompt 0x5a5b912c5e90: 113 tokens, checkpoints: 1, 127.836 MiB
|
||
|
|
1.12.667.894 I srv update: - prompt 0x5a5b914313d0: 114 tokens, checkpoints: 1, 127.855 MiB
|
||
|
|
1.12.667.895 I srv update: - prompt 0x5a5b90ed0400: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.12.667.896 I srv update: - prompt 0x5a5b914fe950: 126 tokens, checkpoints: 1, 128.090 MiB
|
||
|
|
1.12.667.897 I srv update: - prompt 0x5a5b912f8f40: 117 tokens, checkpoints: 1, 127.914 MiB
|
||
|
|
1.12.667.899 I srv update: - prompt 0x5a5b9117dd70: 187 tokens, checkpoints: 1, 129.283 MiB
|
||
|
|
1.12.667.899 I srv update: - prompt 0x5a5b91329460: 138 tokens, checkpoints: 1, 128.325 MiB
|
||
|
|
1.12.667.900 I srv update: - prompt 0x5a5b98862670: 186 tokens, checkpoints: 1, 129.263 MiB
|
||
|
|
1.12.667.901 I srv update: - prompt 0x5a5b9117f4e0: 147 tokens, checkpoints: 1, 128.501 MiB
|
||
|
|
1.12.667.901 I srv update: - prompt 0x5a5b9115a5a0: 175 tokens, checkpoints: 1, 129.048 MiB
|
||
|
|
1.12.667.902 I srv update: - prompt 0x5a5b9106a380: 100 tokens, checkpoints: 1, 127.582 MiB
|
||
|
|
1.12.667.903 I srv update: - prompt 0x5a5b982ef8f0: 143 tokens, checkpoints: 1, 128.422 MiB
|
||
|
|
1.12.667.904 I srv update: - prompt 0x5a5b91237610: 162 tokens, checkpoints: 1, 128.794 MiB
|
||
|
|
1.12.667.904 I srv update: - prompt 0x5a5b912c1550: 128 tokens, checkpoints: 1, 128.129 MiB
|
||
|
|
1.12.667.905 I srv update: - prompt 0x5a5b912bdc60: 130 tokens, checkpoints: 1, 128.168 MiB
|
||
|
|
1.12.667.908 I srv get_availabl: prompt cache update took 109.82 ms
|
||
|
|
1.12.667.990 I slot launch_slot_: id 3 | task 2262 | processing task, is_child = 0
|
||
|
|
1.12.667.991 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.12.667.992 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.12.668.002 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.12.668.002 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.12.668.008 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.12.668.008 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.12.668.022 I slot update_slots: id 3 | task 2262 | Checking checkpoint with [45, 45] against 51...
|
||
|
|
1.12.687.315 W slot update_slots: id 3 | task 2262 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 62.813 MiB)
|
||
|
|
1.14.510.487 I slot print_timing: id 3 | task 2262 | prompt eval time = 108.32 ms / 20 tokens ( 5.42 ms per token, 184.63 tokens per second)
|
||
|
|
1.14.510.490 I slot print_timing: id 3 | task 2262 | eval time = 1734.14 ms / 89 tokens ( 19.48 ms per token, 51.32 tokens per second)
|
||
|
|
1.14.510.491 I slot print_timing: id 3 | task 2262 | total time = 1842.46 ms / 109 tokens
|
||
|
|
1.14.510.491 I slot print_timing: id 3 | task 2262 | graphs reused = 2193
|
||
|
|
1.14.510.518 I slot release: id 3 | task 2262 | stop processing: n_tokens = 154, truncated = 0
|
||
|
|
1.14.510.526 I srv update_slots: all slots are idle
|
||
|
|
1.14.511.257 I srv operator(): operator(): cleaning up before exit...
|