1803 lines
181 KiB
Plaintext
1803 lines
181 KiB
Plaintext
0.00.066.928 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
|
|
0.00.066.931 I device_info:
|
|
0.00.066.950 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
|
0.00.066.955 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
|
0.00.066.961 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
|
|
0.00.067.018 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
|
0.00.067.022 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
|
|
0.00.067.064 I srv init: running without SSL
|
|
0.00.067.134 I srv init: using 23 threads for HTTP server
|
|
0.00.067.475 I srv start: binding port with default address family
|
|
0.00.068.687 I srv llama_server: loading model
|
|
0.00.068.694 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/MiniMaxAI_MiniMax-M2-IQ2_S/MiniMaxAI_MiniMax-M2-IQ2_S-00001-of-00002.gguf'
|
|
0.00.068.754 I common_init_result: fitting params to device memory ...
|
|
0.00.068.755 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
|
|
0.06.328.831 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
|
|
0.06.407.854 W llama_model_loader: tensor overrides to CPU are used with mmap enabled - consider using --no-mmap for better performance
|
|
0.35.944.134 W llama_context: n_ctx_seq (8192) < n_ctx_train (196608) -- the full capacity of the model will not be utilized
|
|
0.36.180.614 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
|
0.36.403.711 I srv load_model: initializing slots, n_slots = 4
|
|
0.36.502.095 W common_speculative_init: no implementations specified for speculative decoding
|
|
0.36.502.100 I slot load_model: id 0 | task -1 | new slot, n_ctx = 8192
|
|
0.36.502.105 I slot load_model: id 1 | task -1 | new slot, n_ctx = 8192
|
|
0.36.502.105 I slot load_model: id 2 | task -1 | new slot, n_ctx = 8192
|
|
0.36.502.106 I slot load_model: id 3 | task -1 | new slot, n_ctx = 8192
|
|
0.36.502.154 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
|
|
0.36.502.154 I srv load_model: use `--cache-ram 0` to disable the prompt cache
|
|
0.36.502.155 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
|
0.36.502.155 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
|
|
0.36.502.176 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
|
|
0.36.507.699 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
0.36.521.810 I init: chat template, example_format: ']~!b[]~b]system
|
|
You are a helpful assistant[e~[
|
|
]~b]user
|
|
Hello[e~[
|
|
]~b]ai
|
|
Hi there[e~[
|
|
]~b]user
|
|
How are you?[e~[
|
|
]~b]ai
|
|
<think>
|
|
'
|
|
0.36.521.816 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
0.36.534.759 I srv init: init: chat template, thinking = 1
|
|
0.36.534.939 I srv llama_server: model loaded
|
|
0.36.534.942 I srv llama_server: server is listening on http://127.0.0.1:8090
|
|
0.36.534.948 I srv update_slots: all slots are idle
|
|
0.39.961.710 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
0.39.980.334 I srv operator(): Chat format: peg-native
|
|
0.39.986.266 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
|
|
0.39.986.268 I srv get_availabl: updating prompt cache
|
|
0.39.986.274 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
|
0.39.986.279 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
|
|
0.39.986.280 I srv get_availabl: prompt cache update took 0.01 ms
|
|
0.39.988.016 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
|
|
0.39.988.018 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.39.988.019 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.39.988.027 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.39.988.028 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.39.988.032 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.39.988.032 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.46.731.015 I slot print_timing: id 3 | task 0 | n_decoded = 100, tg = 23.23 t/s, tg_3s = 23.23 t/s
|
|
0.49.763.787 I slot print_timing: id 3 | task 0 | n_decoded = 170, tg = 23.17 t/s, tg_3s = 23.08 t/s
|
|
0.52.796.752 I slot print_timing: id 3 | task 0 | n_decoded = 239, tg = 23.05 t/s, tg_3s = 22.75 t/s
|
|
0.55.802.569 I slot print_timing: id 3 | task 0 | n_decoded = 308, tg = 23.03 t/s, tg_3s = 22.96 t/s
|
|
0.58.845.862 I slot print_timing: id 3 | task 0 | n_decoded = 378, tg = 23.02 t/s, tg_3s = 23.00 t/s
|
|
1.01.856.319 I slot print_timing: id 3 | task 0 | n_decoded = 447, tg = 23.01 t/s, tg_3s = 22.92 t/s
|
|
1.04.840.129 I slot print_timing: id 3 | task 0 | prompt eval time = 2438.86 ms / 74 tokens ( 32.96 ms per token, 30.34 tokens per second)
|
|
1.04.840.133 I slot print_timing: id 3 | task 0 | eval time = 22413.21 ms / 515 tokens ( 43.52 ms per token, 22.98 tokens per second)
|
|
1.04.840.133 I slot print_timing: id 3 | task 0 | total time = 24852.08 ms / 589 tokens
|
|
1.04.840.139 I slot print_timing: id 3 | task 0 | graphs reused = 512
|
|
1.04.840.178 I slot release: id 3 | task 0 | stop processing: n_tokens = 588, truncated = 0
|
|
1.04.840.185 I srv update_slots: all slots are idle
|
|
1.04.841.876 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
1.04.861.135 I srv operator(): Chat format: peg-native
|
|
1.04.867.039 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.766 (> 0.100 thold), f_keep = 0.083
|
|
1.04.867.041 I srv get_availabl: updating prompt cache
|
|
1.04.867.088 W srv prompt_save: - saving prompt with length 588, total state size = 142.414 MiB (draft: 0.000 MiB)
|
|
1.05.071.725 I srv load: - looking for better prompt, base f_keep = 0.083, sim = 0.766
|
|
1.05.071.730 I srv update: - cache state: 1 prompts, 142.414 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
1.05.071.731 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
1.05.071.732 I srv get_availabl: prompt cache update took 204.69 ms
|
|
1.05.073.333 I slot launch_slot_: id 3 | task 517 | processing task, is_child = 0
|
|
1.05.073.335 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.05.073.336 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.05.073.349 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.05.073.350 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.05.073.354 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
1.05.073.355 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
1.09.777.777 I slot print_timing: id 3 | task 517 | n_decoded = 100, tg = 23.08 t/s, tg_3s = 23.08 t/s
|
|
1.12.797.356 I slot print_timing: id 3 | task 517 | n_decoded = 169, tg = 22.99 t/s, tg_3s = 22.85 t/s
|
|
1.15.808.056 I slot print_timing: id 3 | task 517 | n_decoded = 238, tg = 22.97 t/s, tg_3s = 22.92 t/s
|
|
1.18.849.395 I slot print_timing: id 3 | task 517 | n_decoded = 308, tg = 22.98 t/s, tg_3s = 23.02 t/s
|
|
1.21.887.232 I slot print_timing: id 3 | task 517 | n_decoded = 378, tg = 22.99 t/s, tg_3s = 23.04 t/s
|
|
1.24.929.110 I slot print_timing: id 3 | task 517 | n_decoded = 448, tg = 22.99 t/s, tg_3s = 23.01 t/s
|
|
1.27.951.586 I slot print_timing: id 3 | task 517 | n_decoded = 517, tg = 22.97 t/s, tg_3s = 22.83 t/s
|
|
1.30.962.834 I slot print_timing: id 3 | task 517 | n_decoded = 586, tg = 22.96 t/s, tg_3s = 22.91 t/s
|
|
1.33.999.853 I slot print_timing: id 3 | task 517 | n_decoded = 655, tg = 22.94 t/s, tg_3s = 22.72 t/s
|
|
1.37.030.230 I slot print_timing: id 3 | task 517 | n_decoded = 724, tg = 22.92 t/s, tg_3s = 22.77 t/s
|
|
1.40.073.858 I slot print_timing: id 3 | task 517 | n_decoded = 792, tg = 22.87 t/s, tg_3s = 22.34 t/s
|
|
1.43.080.171 I slot print_timing: id 3 | task 517 | n_decoded = 860, tg = 22.85 t/s, tg_3s = 22.62 t/s
|
|
1.46.087.924 I slot print_timing: id 3 | task 517 | n_decoded = 928, tg = 22.83 t/s, tg_3s = 22.61 t/s
|
|
1.49.124.385 I slot print_timing: id 3 | task 517 | n_decoded = 996, tg = 22.80 t/s, tg_3s = 22.39 t/s
|
|
1.52.154.741 I slot print_timing: id 3 | task 517 | n_decoded = 1064, tg = 22.78 t/s, tg_3s = 22.44 t/s
|
|
1.55.195.251 I slot print_timing: id 3 | task 517 | n_decoded = 1132, tg = 22.75 t/s, tg_3s = 22.36 t/s
|
|
1.58.207.773 I slot print_timing: id 3 | task 517 | n_decoded = 1199, tg = 22.72 t/s, tg_3s = 22.24 t/s
|
|
1.58.252.261 I slot print_timing: id 3 | task 517 | prompt eval time = 371.55 ms / 15 tokens ( 24.77 ms per token, 40.37 tokens per second)
|
|
1.58.252.264 I slot print_timing: id 3 | task 517 | eval time = 52807.35 ms / 1200 tokens ( 44.01 ms per token, 22.72 tokens per second)
|
|
1.58.252.265 I slot print_timing: id 3 | task 517 | total time = 53178.89 ms / 1215 tokens
|
|
1.58.252.265 I slot print_timing: id 3 | task 517 | graphs reused = 1706
|
|
1.58.252.287 I slot release: id 3 | task 517 | stop processing: n_tokens = 1263, truncated = 0
|
|
1.58.252.299 I srv update_slots: all slots are idle
|
|
1.58.253.259 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
1.58.272.630 I srv operator(): Chat format: peg-native
|
|
1.58.278.598 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.039
|
|
1.58.278.600 I srv get_availabl: updating prompt cache
|
|
1.58.278.653 W srv prompt_save: - saving prompt with length 1263, total state size = 305.899 MiB (draft: 0.000 MiB)
|
|
1.58.662.952 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.731
|
|
1.58.662.957 I srv update: - cache state: 2 prompts, 448.313 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
1.58.662.959 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
1.58.662.959 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
1.58.662.961 I srv get_availabl: prompt cache update took 384.36 ms
|
|
1.58.664.608 I slot launch_slot_: id 3 | task 1718 | processing task, is_child = 0
|
|
1.58.664.610 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.58.664.610 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.58.664.619 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.58.664.619 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.58.664.624 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
1.58.664.624 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.03.581.893 I slot print_timing: id 3 | task 1718 | n_decoded = 100, tg = 23.02 t/s, tg_3s = 23.02 t/s
|
|
2.06.605.867 I slot print_timing: id 3 | task 1718 | n_decoded = 170, tg = 23.07 t/s, tg_3s = 23.15 t/s
|
|
2.07.955.422 I slot print_timing: id 3 | task 1718 | prompt eval time = 572.50 ms / 18 tokens ( 31.81 ms per token, 31.44 tokens per second)
|
|
2.07.955.425 I slot print_timing: id 3 | task 1718 | eval time = 8718.28 ms / 201 tokens ( 43.37 ms per token, 23.05 tokens per second)
|
|
2.07.955.425 I slot print_timing: id 3 | task 1718 | total time = 9290.78 ms / 219 tokens
|
|
2.07.955.426 I slot print_timing: id 3 | task 1718 | graphs reused = 1904
|
|
2.07.955.446 I slot release: id 3 | task 1718 | stop processing: n_tokens = 267, truncated = 0
|
|
2.07.955.454 I srv update_slots: all slots are idle
|
|
2.07.956.298 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
2.07.974.338 I srv operator(): Chat format: peg-native
|
|
2.07.979.818 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.746 (> 0.100 thold), f_keep = 0.187
|
|
2.07.979.820 I srv get_availabl: updating prompt cache
|
|
2.07.979.850 W srv prompt_save: - saving prompt with length 267, total state size = 64.669 MiB (draft: 0.000 MiB)
|
|
2.08.027.546 I srv load: - looking for better prompt, base f_keep = 0.187, sim = 0.746
|
|
2.08.027.553 I srv update: - cache state: 3 prompts, 512.982 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
2.08.027.554 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
2.08.027.555 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
2.08.027.555 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
2.08.027.557 I srv get_availabl: prompt cache update took 47.73 ms
|
|
2.08.029.220 I slot launch_slot_: id 3 | task 1920 | processing task, is_child = 0
|
|
2.08.029.222 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.08.029.223 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.08.029.231 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.08.029.231 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.08.029.236 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.08.029.236 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.12.872.194 I slot print_timing: id 3 | task 1920 | n_decoded = 100, tg = 23.32 t/s, tg_3s = 23.32 t/s
|
|
2.13.304.184 I slot print_timing: id 3 | task 1920 | prompt eval time = 554.07 ms / 17 tokens ( 32.59 ms per token, 30.68 tokens per second)
|
|
2.13.304.187 I slot print_timing: id 3 | task 1920 | eval time = 4720.86 ms / 110 tokens ( 42.92 ms per token, 23.30 tokens per second)
|
|
2.13.304.187 I slot print_timing: id 3 | task 1920 | total time = 5274.93 ms / 127 tokens
|
|
2.13.304.188 I slot print_timing: id 3 | task 1920 | graphs reused = 2012
|
|
2.13.304.207 I slot release: id 3 | task 1920 | stop processing: n_tokens = 176, truncated = 0
|
|
2.13.304.215 I srv update_slots: all slots are idle
|
|
2.13.305.058 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
2.13.323.089 I srv operator(): Chat format: peg-native
|
|
2.13.328.590 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.725 (> 0.100 thold), f_keep = 0.284
|
|
2.13.328.592 I srv get_availabl: updating prompt cache
|
|
2.13.328.619 W srv prompt_save: - saving prompt with length 176, total state size = 42.628 MiB (draft: 0.000 MiB)
|
|
2.13.361.286 I srv load: - looking for better prompt, base f_keep = 0.284, sim = 0.725
|
|
2.13.361.295 I srv update: - cache state: 4 prompts, 555.610 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
2.13.361.296 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
2.13.361.296 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
2.13.361.297 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
2.13.361.297 I srv update: - prompt 0x60380702f610: 176 tokens, checkpoints: 0, 42.628 MiB
|
|
2.13.361.299 I srv get_availabl: prompt cache update took 32.71 ms
|
|
2.13.362.945 I slot launch_slot_: id 3 | task 2031 | processing task, is_child = 0
|
|
2.13.362.947 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.13.362.948 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.13.362.955 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.13.362.955 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.13.362.959 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.13.362.960 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.18.226.732 I slot print_timing: id 3 | task 2031 | n_decoded = 100, tg = 23.34 t/s, tg_3s = 23.34 t/s
|
|
2.19.486.253 I slot print_timing: id 3 | task 2031 | prompt eval time = 580.11 ms / 19 tokens ( 30.53 ms per token, 32.75 tokens per second)
|
|
2.19.486.256 I slot print_timing: id 3 | task 2031 | eval time = 5543.16 ms / 129 tokens ( 42.97 ms per token, 23.27 tokens per second)
|
|
2.19.486.257 I slot print_timing: id 3 | task 2031 | total time = 6123.27 ms / 148 tokens
|
|
2.19.486.258 I slot print_timing: id 3 | task 2031 | graphs reused = 2139
|
|
2.19.486.279 I slot release: id 3 | task 2031 | stop processing: n_tokens = 197, truncated = 0
|
|
2.19.486.290 I srv update_slots: all slots are idle
|
|
2.19.487.086 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
2.19.507.154 I srv operator(): Chat format: peg-native
|
|
2.19.513.068 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.249
|
|
2.19.513.070 I srv get_availabl: updating prompt cache
|
|
2.19.513.103 W srv prompt_save: - saving prompt with length 197, total state size = 47.715 MiB (draft: 0.000 MiB)
|
|
2.19.548.937 I srv load: - looking for better prompt, base f_keep = 0.249, sim = 0.731
|
|
2.19.548.946 I srv update: - cache state: 5 prompts, 603.325 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
2.19.548.948 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
2.19.548.948 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
2.19.548.949 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
2.19.548.950 I srv update: - prompt 0x60380702f610: 176 tokens, checkpoints: 0, 42.628 MiB
|
|
2.19.548.955 I srv update: - prompt 0x6037ffc00990: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
2.19.548.956 I srv get_availabl: prompt cache update took 35.88 ms
|
|
2.19.550.588 I slot launch_slot_: id 3 | task 2161 | processing task, is_child = 0
|
|
2.19.550.590 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.19.550.591 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.19.550.597 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.19.550.598 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.19.550.602 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.19.550.602 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.24.399.123 I slot print_timing: id 3 | task 2161 | n_decoded = 100, tg = 23.25 t/s, tg_3s = 23.25 t/s
|
|
2.27.430.331 I slot print_timing: id 3 | task 2161 | n_decoded = 170, tg = 23.19 t/s, tg_3s = 23.09 t/s
|
|
2.30.440.223 I slot print_timing: id 3 | task 2161 | n_decoded = 239, tg = 23.11 t/s, tg_3s = 22.92 t/s
|
|
2.33.454.288 I slot print_timing: id 3 | task 2161 | n_decoded = 307, tg = 22.99 t/s, tg_3s = 22.56 t/s
|
|
2.36.455.743 I slot print_timing: id 3 | task 2161 | n_decoded = 376, tg = 22.99 t/s, tg_3s = 22.99 t/s
|
|
2.39.473.792 I slot print_timing: id 3 | task 2161 | n_decoded = 445, tg = 22.97 t/s, tg_3s = 22.86 t/s
|
|
2.42.501.790 I slot print_timing: id 3 | task 2161 | n_decoded = 514, tg = 22.94 t/s, tg_3s = 22.79 t/s
|
|
2.45.539.453 I slot print_timing: id 3 | task 2161 | n_decoded = 583, tg = 22.92 t/s, tg_3s = 22.71 t/s
|
|
2.48.566.171 I slot print_timing: id 3 | task 2161 | n_decoded = 652, tg = 22.90 t/s, tg_3s = 22.80 t/s
|
|
2.51.601.808 I slot print_timing: id 3 | task 2161 | n_decoded = 720, tg = 22.85 t/s, tg_3s = 22.40 t/s
|
|
2.54.615.666 I slot print_timing: id 3 | task 2161 | n_decoded = 788, tg = 22.83 t/s, tg_3s = 22.56 t/s
|
|
2.57.626.737 I slot print_timing: id 3 | task 2161 | n_decoded = 856, tg = 22.81 t/s, tg_3s = 22.58 t/s
|
|
3.00.671.411 I slot print_timing: id 3 | task 2161 | n_decoded = 925, tg = 22.80 t/s, tg_3s = 22.66 t/s
|
|
3.03.676.924 I slot print_timing: id 3 | task 2161 | n_decoded = 992, tg = 22.76 t/s, tg_3s = 22.29 t/s
|
|
3.06.689.558 I slot print_timing: id 3 | task 2161 | n_decoded = 1060, tg = 22.75 t/s, tg_3s = 22.57 t/s
|
|
3.09.725.351 I slot print_timing: id 3 | task 2161 | n_decoded = 1128, tg = 22.73 t/s, tg_3s = 22.40 t/s
|
|
3.12.727.837 I slot print_timing: id 3 | task 2161 | n_decoded = 1195, tg = 22.71 t/s, tg_3s = 22.31 t/s
|
|
3.12.949.483 I slot print_timing: id 3 | task 2161 | prompt eval time = 548.23 ms / 18 tokens ( 30.46 ms per token, 32.83 tokens per second)
|
|
3.12.949.487 I slot print_timing: id 3 | task 2161 | eval time = 52850.64 ms / 1200 tokens ( 44.04 ms per token, 22.71 tokens per second)
|
|
3.12.949.487 I slot print_timing: id 3 | task 2161 | total time = 53398.86 ms / 1218 tokens
|
|
3.12.949.488 I slot print_timing: id 3 | task 2161 | graphs reused = 3333
|
|
3.12.949.511 I slot release: id 3 | task 2161 | stop processing: n_tokens = 1266, truncated = 0
|
|
3.12.949.527 I srv update_slots: all slots are idle
|
|
3.12.950.426 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
3.12.969.644 I srv operator(): Chat format: peg-native
|
|
3.12.975.555 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.671 (> 0.100 thold), f_keep = 0.039
|
|
3.12.975.557 I srv get_availabl: updating prompt cache
|
|
3.12.975.620 W srv prompt_save: - saving prompt with length 1266, total state size = 306.625 MiB (draft: 0.000 MiB)
|
|
3.13.216.307 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.671
|
|
3.13.216.314 I srv load: - found better prompt with f_keep = 0.284, sim = 0.685
|
|
3.13.236.118 I srv update: - cache state: 5 prompts, 867.322 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
3.13.236.124 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
3.13.236.125 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
3.13.236.126 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
3.13.236.126 I srv update: - prompt 0x6037ffc00990: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
3.13.236.127 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
3.13.236.128 I srv get_availabl: prompt cache update took 260.57 ms
|
|
3.13.237.899 I slot launch_slot_: id 3 | task 3362 | processing task, is_child = 0
|
|
3.13.237.901 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.13.237.902 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.13.237.910 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.13.237.910 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.13.237.915 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.13.237.915 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.18.178.968 I slot print_timing: id 3 | task 3362 | n_decoded = 100, tg = 23.15 t/s, tg_3s = 23.15 t/s
|
|
3.21.183.882 I slot print_timing: id 3 | task 3362 | n_decoded = 169, tg = 23.08 t/s, tg_3s = 22.96 t/s
|
|
3.24.187.416 I slot print_timing: id 3 | task 3362 | n_decoded = 238, tg = 23.05 t/s, tg_3s = 22.97 t/s
|
|
3.27.230.432 I slot print_timing: id 3 | task 3362 | n_decoded = 308, tg = 23.04 t/s, tg_3s = 23.00 t/s
|
|
3.29.279.314 I slot print_timing: id 3 | task 3362 | prompt eval time = 622.25 ms / 23 tokens ( 27.05 ms per token, 36.96 tokens per second)
|
|
3.29.279.318 I slot print_timing: id 3 | task 3362 | eval time = 15419.13 ms / 355 tokens ( 43.43 ms per token, 23.02 tokens per second)
|
|
3.29.279.319 I slot print_timing: id 3 | task 3362 | total time = 16041.38 ms / 378 tokens
|
|
3.29.279.320 I slot print_timing: id 3 | task 3362 | graphs reused = 3685
|
|
3.29.279.354 I slot release: id 3 | task 3362 | stop processing: n_tokens = 427, truncated = 0
|
|
3.29.279.366 I srv update_slots: all slots are idle
|
|
3.29.280.321 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
3.29.299.170 I srv operator(): Chat format: peg-native
|
|
3.29.305.045 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.794 (> 0.100 thold), f_keep = 0.117
|
|
3.29.305.048 I srv get_availabl: updating prompt cache
|
|
3.29.305.089 W srv prompt_save: - saving prompt with length 427, total state size = 103.420 MiB (draft: 0.000 MiB)
|
|
3.29.383.358 I srv load: - looking for better prompt, base f_keep = 0.117, sim = 0.794
|
|
3.29.383.365 I srv load: - found better prompt with f_keep = 0.259, sim = 0.810
|
|
3.29.404.066 I srv update: - cache state: 5 prompts, 923.027 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
3.29.404.071 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
3.29.404.072 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
3.29.404.073 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
3.29.404.073 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
3.29.404.074 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
3.29.404.075 I srv get_availabl: prompt cache update took 99.03 ms
|
|
3.29.405.796 I slot launch_slot_: id 3 | task 3718 | processing task, is_child = 0
|
|
3.29.405.798 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.29.405.799 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.29.405.808 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.29.405.808 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.29.405.813 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.29.405.813 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.34.074.331 I slot print_timing: id 3 | task 3718 | n_decoded = 100, tg = 23.08 t/s, tg_3s = 23.08 t/s
|
|
3.37.104.642 I slot print_timing: id 3 | task 3718 | n_decoded = 170, tg = 23.09 t/s, tg_3s = 23.10 t/s
|
|
3.40.109.281 I slot print_timing: id 3 | task 3718 | n_decoded = 239, tg = 23.05 t/s, tg_3s = 22.96 t/s
|
|
3.42.024.313 I slot print_timing: id 3 | task 3718 | prompt eval time = 335.44 ms / 12 tokens ( 27.95 ms per token, 35.77 tokens per second)
|
|
3.42.024.316 I slot print_timing: id 3 | task 3718 | eval time = 12283.04 ms / 283 tokens ( 43.40 ms per token, 23.04 tokens per second)
|
|
3.42.024.317 I slot print_timing: id 3 | task 3718 | total time = 12618.48 ms / 295 tokens
|
|
3.42.024.318 I slot print_timing: id 3 | task 3718 | graphs reused = 3965
|
|
3.42.024.337 I slot release: id 3 | task 3718 | stop processing: n_tokens = 345, truncated = 0
|
|
3.42.024.350 I srv update_slots: all slots are idle
|
|
3.42.025.219 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
3.42.044.385 I srv operator(): Chat format: peg-native
|
|
3.42.050.409 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.671 (> 0.100 thold), f_keep = 0.142
|
|
3.42.050.412 I srv get_availabl: updating prompt cache
|
|
3.42.050.445 W srv prompt_save: - saving prompt with length 345, total state size = 83.560 MiB (draft: 0.000 MiB)
|
|
3.42.110.780 I srv load: - looking for better prompt, base f_keep = 0.142, sim = 0.671
|
|
3.42.110.790 I srv update: - cache state: 6 prompts, 1006.587 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
3.42.110.791 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
3.42.110.791 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
3.42.110.792 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
3.42.110.796 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
3.42.110.797 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
3.42.110.797 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
3.42.110.798 I srv get_availabl: prompt cache update took 60.39 ms
|
|
3.42.112.447 I slot launch_slot_: id 3 | task 4002 | processing task, is_child = 0
|
|
3.42.112.449 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.42.112.450 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.42.112.457 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.42.112.458 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.42.112.462 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.42.112.462 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.47.054.307 I slot print_timing: id 3 | task 4002 | n_decoded = 100, tg = 23.16 t/s, tg_3s = 23.16 t/s
|
|
3.50.086.067 I slot print_timing: id 3 | task 4002 | n_decoded = 170, tg = 23.13 t/s, tg_3s = 23.09 t/s
|
|
3.53.101.689 I slot print_timing: id 3 | task 4002 | n_decoded = 239, tg = 23.06 t/s, tg_3s = 22.88 t/s
|
|
3.56.123.464 I slot print_timing: id 3 | task 4002 | n_decoded = 307, tg = 22.93 t/s, tg_3s = 22.50 t/s
|
|
3.59.129.743 I slot print_timing: id 3 | task 4002 | n_decoded = 376, tg = 22.94 t/s, tg_3s = 22.95 t/s
|
|
4.02.131.859 I slot print_timing: id 3 | task 4002 | n_decoded = 445, tg = 22.94 t/s, tg_3s = 22.98 t/s
|
|
4.05.165.019 I slot print_timing: id 3 | task 4002 | n_decoded = 514, tg = 22.92 t/s, tg_3s = 22.75 t/s
|
|
4.08.199.889 I slot print_timing: id 3 | task 4002 | n_decoded = 583, tg = 22.90 t/s, tg_3s = 22.74 t/s
|
|
4.11.210.802 I slot print_timing: id 3 | task 4002 | n_decoded = 652, tg = 22.90 t/s, tg_3s = 22.92 t/s
|
|
4.14.246.093 I slot print_timing: id 3 | task 4002 | n_decoded = 721, tg = 22.88 t/s, tg_3s = 22.73 t/s
|
|
4.17.267.787 I slot print_timing: id 3 | task 4002 | n_decoded = 789, tg = 22.85 t/s, tg_3s = 22.50 t/s
|
|
4.20.299.804 I slot print_timing: id 3 | task 4002 | n_decoded = 857, tg = 22.82 t/s, tg_3s = 22.43 t/s
|
|
4.23.311.313 I slot print_timing: id 3 | task 4002 | n_decoded = 925, tg = 22.80 t/s, tg_3s = 22.58 t/s
|
|
4.26.336.076 I slot print_timing: id 3 | task 4002 | n_decoded = 993, tg = 22.78 t/s, tg_3s = 22.48 t/s
|
|
4.29.356.613 I slot print_timing: id 3 | task 4002 | n_decoded = 1061, tg = 22.76 t/s, tg_3s = 22.51 t/s
|
|
4.32.376.481 I slot print_timing: id 3 | task 4002 | n_decoded = 1129, tg = 22.74 t/s, tg_3s = 22.52 t/s
|
|
4.35.403.687 I slot print_timing: id 3 | task 4002 | n_decoded = 1197, tg = 22.73 t/s, tg_3s = 22.46 t/s
|
|
4.35.538.954 I slot print_timing: id 3 | task 4002 | prompt eval time = 624.76 ms / 24 tokens ( 26.03 ms per token, 38.41 tokens per second)
|
|
4.35.538.958 I slot print_timing: id 3 | task 4002 | eval time = 52801.71 ms / 1200 tokens ( 44.00 ms per token, 22.73 tokens per second)
|
|
4.35.538.958 I slot print_timing: id 3 | task 4002 | total time = 53426.47 ms / 1224 tokens
|
|
4.35.538.959 I slot print_timing: id 3 | task 4002 | graphs reused = 5159
|
|
4.35.538.989 I slot release: id 3 | task 4002 | stop processing: n_tokens = 1272, truncated = 0
|
|
4.35.539.008 I srv update_slots: all slots are idle
|
|
4.35.539.989 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
4.35.557.861 I srv operator(): Chat format: peg-native
|
|
4.35.563.384 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.039
|
|
4.35.563.386 I srv get_availabl: updating prompt cache
|
|
4.35.563.450 W srv prompt_save: - saving prompt with length 1272, total state size = 308.078 MiB (draft: 0.000 MiB)
|
|
4.35.796.056 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.721
|
|
4.35.796.063 I srv update: - cache state: 7 prompts, 1314.666 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
4.35.796.064 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
4.35.796.064 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
4.35.796.065 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
4.35.796.065 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
4.35.796.066 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
4.35.796.066 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
4.35.796.067 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
4.35.796.068 I srv get_availabl: prompt cache update took 232.68 ms
|
|
4.35.797.695 I slot launch_slot_: id 3 | task 5203 | processing task, is_child = 0
|
|
4.35.797.697 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.35.797.698 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.35.797.706 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.35.797.706 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.35.797.711 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.35.797.711 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.40.696.198 I slot print_timing: id 3 | task 5203 | n_decoded = 100, tg = 23.24 t/s, tg_3s = 23.24 t/s
|
|
4.43.714.221 I slot print_timing: id 3 | task 5203 | n_decoded = 170, tg = 23.22 t/s, tg_3s = 23.19 t/s
|
|
4.46.751.697 I slot print_timing: id 3 | task 5203 | n_decoded = 240, tg = 23.17 t/s, tg_3s = 23.05 t/s
|
|
4.49.788.574 I slot print_timing: id 3 | task 5203 | n_decoded = 310, tg = 23.14 t/s, tg_3s = 23.05 t/s
|
|
4.52.789.890 I slot print_timing: id 3 | task 5203 | n_decoded = 378, tg = 23.05 t/s, tg_3s = 22.66 t/s
|
|
4.52.922.088 I slot print_timing: id 3 | task 5203 | prompt eval time = 595.89 ms / 19 tokens ( 31.36 ms per token, 31.88 tokens per second)
|
|
4.52.922.092 I slot print_timing: id 3 | task 5203 | eval time = 16528.46 ms / 381 tokens ( 43.38 ms per token, 23.05 tokens per second)
|
|
4.52.922.093 I slot print_timing: id 3 | task 5203 | total time = 17124.36 ms / 400 tokens
|
|
4.52.922.094 I slot print_timing: id 3 | task 5203 | graphs reused = 5537
|
|
4.52.922.125 I slot release: id 3 | task 5203 | stop processing: n_tokens = 448, truncated = 0
|
|
4.52.922.136 I srv update_slots: all slots are idle
|
|
4.52.923.090 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
4.52.942.334 I srv operator(): Chat format: peg-native
|
|
4.52.948.321 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.662 (> 0.100 thold), f_keep = 0.109
|
|
4.52.948.323 I srv get_availabl: updating prompt cache
|
|
4.52.948.369 W srv prompt_save: - saving prompt with length 448, total state size = 108.507 MiB (draft: 0.000 MiB)
|
|
4.53.028.359 I srv load: - looking for better prompt, base f_keep = 0.109, sim = 0.662
|
|
4.53.028.370 I srv update: - cache state: 8 prompts, 1423.173 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
4.53.028.370 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
4.53.028.371 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
4.53.028.372 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
4.53.028.373 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
4.53.028.374 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
4.53.028.374 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
4.53.028.375 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
4.53.028.376 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
4.53.028.377 I srv get_availabl: prompt cache update took 80.05 ms
|
|
4.53.030.114 I slot launch_slot_: id 3 | task 5585 | processing task, is_child = 0
|
|
4.53.030.117 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.53.030.117 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.53.030.126 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.53.030.126 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.53.030.131 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.53.030.131 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.58.104.327 I slot print_timing: id 3 | task 5585 | n_decoded = 100, tg = 23.15 t/s, tg_3s = 23.15 t/s
|
|
5.01.123.669 I slot print_timing: id 3 | task 5585 | n_decoded = 169, tg = 23.03 t/s, tg_3s = 22.85 t/s
|
|
5.04.151.495 I slot print_timing: id 3 | task 5585 | n_decoded = 238, tg = 22.96 t/s, tg_3s = 22.79 t/s
|
|
5.07.152.236 I slot print_timing: id 3 | task 5585 | n_decoded = 307, tg = 22.97 t/s, tg_3s = 22.99 t/s
|
|
5.10.153.645 I slot print_timing: id 3 | task 5585 | n_decoded = 376, tg = 22.97 t/s, tg_3s = 22.99 t/s
|
|
5.10.197.208 I slot print_timing: id 3 | task 5585 | prompt eval time = 755.05 ms / 25 tokens ( 30.20 ms per token, 33.11 tokens per second)
|
|
5.10.197.211 I slot print_timing: id 3 | task 5585 | eval time = 16412.01 ms / 377 tokens ( 43.53 ms per token, 22.97 tokens per second)
|
|
5.10.197.212 I slot print_timing: id 3 | task 5585 | total time = 17167.06 ms / 402 tokens
|
|
5.10.197.213 I slot print_timing: id 3 | task 5585 | graphs reused = 5911
|
|
5.10.197.238 I slot release: id 3 | task 5585 | stop processing: n_tokens = 450, truncated = 0
|
|
5.10.197.248 I srv update_slots: all slots are idle
|
|
5.10.198.185 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
5.10.216.291 I srv operator(): Chat format: peg-native
|
|
5.10.221.767 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.109
|
|
5.10.221.769 I srv get_availabl: updating prompt cache
|
|
5.10.221.800 W srv prompt_save: - saving prompt with length 450, total state size = 108.991 MiB (draft: 0.000 MiB)
|
|
5.10.298.920 I srv load: - looking for better prompt, base f_keep = 0.109, sim = 0.721
|
|
5.10.298.930 I srv update: - cache state: 9 prompts, 1532.163 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
5.10.298.931 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
5.10.298.932 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
5.10.298.932 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
5.10.298.933 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
5.10.298.934 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
5.10.298.937 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
5.10.298.937 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
5.10.298.938 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
5.10.298.938 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
5.10.298.939 I srv get_availabl: prompt cache update took 77.17 ms
|
|
5.10.300.620 I slot launch_slot_: id 3 | task 5963 | processing task, is_child = 0
|
|
5.10.300.622 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.10.300.623 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.10.300.631 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.10.300.631 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.10.300.636 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.10.300.636 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.15.168.315 I slot print_timing: id 3 | task 5963 | n_decoded = 100, tg = 23.24 t/s, tg_3s = 23.24 t/s
|
|
5.18.202.041 I slot print_timing: id 3 | task 5963 | n_decoded = 170, tg = 23.17 t/s, tg_3s = 23.07 t/s
|
|
5.21.211.810 I slot print_timing: id 3 | task 5963 | n_decoded = 239, tg = 23.10 t/s, tg_3s = 22.93 t/s
|
|
5.24.254.281 I slot print_timing: id 3 | task 5963 | n_decoded = 309, tg = 23.08 t/s, tg_3s = 23.01 t/s
|
|
5.26.204.226 I slot print_timing: id 3 | task 5963 | prompt eval time = 565.61 ms / 19 tokens ( 29.77 ms per token, 33.59 tokens per second)
|
|
5.26.204.230 I slot print_timing: id 3 | task 5963 | eval time = 15337.96 ms / 354 tokens ( 43.33 ms per token, 23.08 tokens per second)
|
|
5.26.204.231 I slot print_timing: id 3 | task 5963 | total time = 15903.57 ms / 373 tokens
|
|
5.26.204.232 I slot print_timing: id 3 | task 5963 | graphs reused = 6262
|
|
5.26.204.256 I slot release: id 3 | task 5963 | stop processing: n_tokens = 421, truncated = 0
|
|
5.26.204.266 I srv update_slots: all slots are idle
|
|
5.26.205.065 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
5.26.222.151 I srv operator(): Chat format: peg-native
|
|
5.26.227.678 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.116
|
|
5.26.227.680 I srv get_availabl: updating prompt cache
|
|
5.26.227.710 W srv prompt_save: - saving prompt with length 421, total state size = 101.967 MiB (draft: 0.000 MiB)
|
|
5.26.299.699 I srv load: - looking for better prompt, base f_keep = 0.116, sim = 0.710
|
|
5.26.299.711 I srv update: - cache state: 10 prompts, 1634.131 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
5.26.299.712 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
5.26.299.712 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
5.26.299.713 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
5.26.299.714 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
5.26.299.714 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
5.26.299.715 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
5.26.299.716 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
5.26.299.717 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
5.26.299.717 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
5.26.299.718 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
5.26.299.719 I srv get_availabl: prompt cache update took 72.04 ms
|
|
5.26.301.366 I slot launch_slot_: id 3 | task 6318 | processing task, is_child = 0
|
|
5.26.301.369 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.26.301.370 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.26.301.377 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.26.301.377 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.26.301.381 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.26.301.382 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.31.191.573 I slot print_timing: id 3 | task 6318 | n_decoded = 100, tg = 23.30 t/s, tg_3s = 23.30 t/s
|
|
5.31.840.539 I slot print_timing: id 3 | task 6318 | prompt eval time = 598.41 ms / 20 tokens ( 29.92 ms per token, 33.42 tokens per second)
|
|
5.31.840.542 I slot print_timing: id 3 | task 6318 | eval time = 4940.73 ms / 115 tokens ( 42.96 ms per token, 23.28 tokens per second)
|
|
5.31.840.543 I slot print_timing: id 3 | task 6318 | total time = 5539.14 ms / 135 tokens
|
|
5.31.840.544 I slot print_timing: id 3 | task 6318 | graphs reused = 6375
|
|
5.31.840.563 I slot release: id 3 | task 6318 | stop processing: n_tokens = 183, truncated = 0
|
|
5.31.840.574 I srv update_slots: all slots are idle
|
|
5.31.841.412 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
5.31.860.305 I srv operator(): Chat format: peg-native
|
|
5.31.866.449 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.268
|
|
5.31.866.451 I srv get_availabl: updating prompt cache
|
|
5.31.866.476 W srv prompt_save: - saving prompt with length 183, total state size = 44.324 MiB (draft: 0.000 MiB)
|
|
5.31.899.833 I srv load: - looking for better prompt, base f_keep = 0.268, sim = 0.710
|
|
5.31.899.845 I srv update: - cache state: 11 prompts, 1678.455 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
5.31.899.849 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
5.31.899.849 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
5.31.899.850 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
5.31.899.851 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
5.31.899.851 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
5.31.899.852 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
5.31.899.852 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
5.31.899.853 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
5.31.899.853 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
5.31.899.854 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
5.31.899.855 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
5.31.899.856 I srv get_availabl: prompt cache update took 33.40 ms
|
|
5.31.901.484 I slot launch_slot_: id 3 | task 6434 | processing task, is_child = 0
|
|
5.31.901.485 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.31.901.486 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.31.901.493 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.31.901.493 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.31.901.497 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.31.901.498 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.36.757.049 I slot print_timing: id 3 | task 6434 | n_decoded = 100, tg = 23.34 t/s, tg_3s = 23.34 t/s
|
|
5.39.790.492 I slot print_timing: id 3 | task 6434 | n_decoded = 170, tg = 23.23 t/s, tg_3s = 23.08 t/s
|
|
5.42.792.847 I slot print_timing: id 3 | task 6434 | n_decoded = 239, tg = 23.16 t/s, tg_3s = 22.98 t/s
|
|
5.43.140.539 I slot print_timing: id 3 | task 6434 | prompt eval time = 571.38 ms / 20 tokens ( 28.57 ms per token, 35.00 tokens per second)
|
|
5.43.140.542 I slot print_timing: id 3 | task 6434 | eval time = 10667.65 ms / 247 tokens ( 43.19 ms per token, 23.15 tokens per second)
|
|
5.43.140.543 I slot print_timing: id 3 | task 6434 | total time = 11239.02 ms / 267 tokens
|
|
5.43.140.544 I slot print_timing: id 3 | task 6434 | graphs reused = 6619
|
|
5.43.140.561 I slot release: id 3 | task 6434 | stop processing: n_tokens = 315, truncated = 0
|
|
5.43.140.569 I srv update_slots: all slots are idle
|
|
5.43.141.430 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
5.43.160.629 I srv operator(): Chat format: peg-native
|
|
5.43.166.775 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.690 (> 0.100 thold), f_keep = 0.156
|
|
5.43.166.777 I srv get_availabl: updating prompt cache
|
|
5.43.166.807 W srv prompt_save: - saving prompt with length 315, total state size = 76.294 MiB (draft: 0.000 MiB)
|
|
5.43.225.706 I srv load: - looking for better prompt, base f_keep = 0.156, sim = 0.690
|
|
5.43.225.718 I srv update: - cache state: 12 prompts, 1754.749 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
5.43.225.722 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
5.43.225.723 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
5.43.225.724 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
5.43.225.724 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
5.43.225.725 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
5.43.225.726 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
5.43.225.726 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
5.43.225.727 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
5.43.225.731 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
5.43.225.731 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
5.43.225.732 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
5.43.225.733 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
5.43.225.734 I srv get_availabl: prompt cache update took 58.96 ms
|
|
5.43.227.716 I slot launch_slot_: id 3 | task 6682 | processing task, is_child = 0
|
|
5.43.227.718 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.43.227.719 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.43.227.727 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.43.227.727 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.43.227.732 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.43.227.733 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.48.195.763 I slot print_timing: id 3 | task 6682 | n_decoded = 100, tg = 22.99 t/s, tg_3s = 22.99 t/s
|
|
5.51.201.908 I slot print_timing: id 3 | task 6682 | n_decoded = 169, tg = 22.97 t/s, tg_3s = 22.95 t/s
|
|
5.52.424.318 I slot print_timing: id 3 | task 6682 | prompt eval time = 617.87 ms / 22 tokens ( 28.08 ms per token, 35.61 tokens per second)
|
|
5.52.424.321 I slot print_timing: id 3 | task 6682 | eval time = 8578.69 ms / 197 tokens ( 43.55 ms per token, 22.96 tokens per second)
|
|
5.52.424.322 I slot print_timing: id 3 | task 6682 | total time = 9196.56 ms / 219 tokens
|
|
5.52.424.323 I slot print_timing: id 3 | task 6682 | graphs reused = 6813
|
|
5.52.424.350 I slot release: id 3 | task 6682 | stop processing: n_tokens = 267, truncated = 0
|
|
5.52.424.361 I srv update_slots: all slots are idle
|
|
5.52.425.158 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
5.52.444.132 I srv operator(): Chat format: peg-native
|
|
5.52.450.172 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.184
|
|
5.52.450.175 I srv get_availabl: updating prompt cache
|
|
5.52.450.215 W srv prompt_save: - saving prompt with length 267, total state size = 64.669 MiB (draft: 0.000 MiB)
|
|
5.52.497.648 I srv load: - looking for better prompt, base f_keep = 0.184, sim = 0.731
|
|
5.52.497.659 I srv update: - cache state: 13 prompts, 1819.417 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
5.52.497.661 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
5.52.497.662 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
5.52.497.663 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
5.52.497.663 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
5.52.497.664 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
5.52.497.664 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
5.52.497.665 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
5.52.497.665 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
5.52.497.666 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
5.52.497.666 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
5.52.497.667 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
5.52.497.667 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
5.52.497.668 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
5.52.497.669 I srv get_availabl: prompt cache update took 47.49 ms
|
|
5.52.499.354 I slot launch_slot_: id 3 | task 6880 | processing task, is_child = 0
|
|
5.52.499.356 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.52.499.357 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.52.499.366 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.52.499.366 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.52.499.370 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.52.499.371 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.57.399.797 I slot print_timing: id 3 | task 6880 | n_decoded = 100, tg = 23.01 t/s, tg_3s = 23.01 t/s
|
|
6.00.402.290 I slot print_timing: id 3 | task 6880 | n_decoded = 169, tg = 23.00 t/s, tg_3s = 22.98 t/s
|
|
6.03.410.726 I slot print_timing: id 3 | task 6880 | n_decoded = 238, tg = 22.98 t/s, tg_3s = 22.94 t/s
|
|
6.06.327.765 I slot print_timing: id 3 | task 6880 | prompt eval time = 555.18 ms / 18 tokens ( 30.84 ms per token, 32.42 tokens per second)
|
|
6.06.327.768 I slot print_timing: id 3 | task 6880 | eval time = 13273.19 ms / 305 tokens ( 43.52 ms per token, 22.98 tokens per second)
|
|
6.06.327.769 I slot print_timing: id 3 | task 6880 | total time = 13828.38 ms / 323 tokens
|
|
6.06.327.769 I slot print_timing: id 3 | task 6880 | graphs reused = 7115
|
|
6.06.327.790 I slot release: id 3 | task 6880 | stop processing: n_tokens = 371, truncated = 0
|
|
6.06.327.797 I srv update_slots: all slots are idle
|
|
6.06.328.699 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
6.06.348.497 I srv operator(): Chat format: peg-native
|
|
6.06.354.484 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.690 (> 0.100 thold), f_keep = 0.132
|
|
6.06.354.486 I srv get_availabl: updating prompt cache
|
|
6.06.354.522 W srv prompt_save: - saving prompt with length 371, total state size = 89.857 MiB (draft: 0.000 MiB)
|
|
6.06.418.911 I srv load: - looking for better prompt, base f_keep = 0.132, sim = 0.690
|
|
6.06.418.922 I srv update: - cache state: 14 prompts, 1909.274 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
6.06.418.923 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
6.06.418.923 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
6.06.418.924 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
6.06.418.925 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
6.06.418.926 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
6.06.418.927 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
6.06.418.927 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
6.06.418.928 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
6.06.418.928 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
6.06.418.929 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
6.06.418.929 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
6.06.418.930 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
6.06.418.930 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
6.06.418.931 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
6.06.418.932 I srv get_availabl: prompt cache update took 64.44 ms
|
|
6.06.420.556 I slot launch_slot_: id 3 | task 7186 | processing task, is_child = 0
|
|
6.06.420.559 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
6.06.420.560 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
6.06.420.568 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
6.06.420.568 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
6.06.420.572 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
6.06.420.572 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
6.11.334.813 I slot print_timing: id 3 | task 7186 | n_decoded = 100, tg = 23.20 t/s, tg_3s = 23.20 t/s
|
|
6.14.342.164 I slot print_timing: id 3 | task 7186 | n_decoded = 169, tg = 23.09 t/s, tg_3s = 22.94 t/s
|
|
6.17.354.259 I slot print_timing: id 3 | task 7186 | n_decoded = 238, tg = 23.04 t/s, tg_3s = 22.91 t/s
|
|
6.18.356.545 I slot print_timing: id 3 | task 7186 | prompt eval time = 603.20 ms / 22 tokens ( 27.42 ms per token, 36.47 tokens per second)
|
|
6.18.356.549 I slot print_timing: id 3 | task 7186 | eval time = 11332.76 ms / 261 tokens ( 43.42 ms per token, 23.03 tokens per second)
|
|
6.18.356.549 I slot print_timing: id 3 | task 7186 | total time = 11935.95 ms / 283 tokens
|
|
6.18.356.550 I slot print_timing: id 3 | task 7186 | graphs reused = 7373
|
|
6.18.356.572 I slot release: id 3 | task 7186 | stop processing: n_tokens = 331, truncated = 0
|
|
6.18.356.582 I srv update_slots: all slots are idle
|
|
6.18.357.475 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
6.18.377.103 I srv operator(): Chat format: peg-native
|
|
6.18.383.115 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.148
|
|
6.18.383.118 I srv get_availabl: updating prompt cache
|
|
6.18.383.153 W srv prompt_save: - saving prompt with length 331, total state size = 80.169 MiB (draft: 0.000 MiB)
|
|
6.18.441.075 I srv load: - looking for better prompt, base f_keep = 0.148, sim = 0.742
|
|
6.18.441.088 I srv update: - cache state: 15 prompts, 1989.444 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
6.18.441.089 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
6.18.441.090 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
6.18.441.091 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
6.18.441.096 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
6.18.441.096 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
6.18.441.097 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
6.18.441.098 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
6.18.441.098 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
6.18.441.099 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
6.18.441.099 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
6.18.441.099 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
6.18.441.100 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
6.18.441.104 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
6.18.441.105 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
6.18.441.105 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
6.18.441.106 I srv get_availabl: prompt cache update took 57.99 ms
|
|
6.18.442.765 I slot launch_slot_: id 3 | task 7448 | processing task, is_child = 0
|
|
6.18.442.767 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
6.18.442.768 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
6.18.442.776 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
6.18.442.776 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
6.18.442.781 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
6.18.442.781 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
6.23.308.293 I slot print_timing: id 3 | task 7448 | n_decoded = 100, tg = 23.14 t/s, tg_3s = 23.14 t/s
|
|
6.26.313.844 I slot print_timing: id 3 | task 7448 | n_decoded = 169, tg = 23.07 t/s, tg_3s = 22.96 t/s
|
|
6.29.347.803 I slot print_timing: id 3 | task 7448 | n_decoded = 237, tg = 22.88 t/s, tg_3s = 22.41 t/s
|
|
6.32.388.261 I slot print_timing: id 3 | task 7448 | n_decoded = 307, tg = 22.91 t/s, tg_3s = 23.02 t/s
|
|
6.35.208.713 I slot print_timing: id 3 | task 7448 | prompt eval time = 544.39 ms / 17 tokens ( 32.02 ms per token, 31.23 tokens per second)
|
|
6.35.208.716 I slot print_timing: id 3 | task 7448 | eval time = 16221.52 ms / 372 tokens ( 43.61 ms per token, 22.93 tokens per second)
|
|
6.35.208.717 I slot print_timing: id 3 | task 7448 | total time = 16765.91 ms / 389 tokens
|
|
6.35.208.718 I slot print_timing: id 3 | task 7448 | graphs reused = 7742
|
|
6.35.208.739 I slot release: id 3 | task 7448 | stop processing: n_tokens = 437, truncated = 0
|
|
6.35.208.749 I srv update_slots: all slots are idle
|
|
6.35.209.699 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
6.35.229.637 I srv operator(): Chat format: peg-native
|
|
6.35.235.679 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.754 (> 0.100 thold), f_keep = 0.112
|
|
6.35.235.682 I srv get_availabl: updating prompt cache
|
|
6.35.235.722 W srv prompt_save: - saving prompt with length 437, total state size = 105.842 MiB (draft: 0.000 MiB)
|
|
6.35.310.634 I srv load: - looking for better prompt, base f_keep = 0.112, sim = 0.754
|
|
6.35.310.650 I srv update: - cache state: 16 prompts, 2095.286 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
6.35.310.655 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
6.35.310.656 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
6.35.310.656 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
6.35.310.657 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
6.35.310.657 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
6.35.310.658 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
6.35.310.658 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
6.35.310.659 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
6.35.310.660 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
6.35.310.661 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
6.35.310.661 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
6.35.310.662 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
6.35.310.662 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
6.35.310.663 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
6.35.310.663 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
6.35.310.664 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
6.35.310.665 I srv get_availabl: prompt cache update took 74.98 ms
|
|
6.35.312.283 I slot launch_slot_: id 3 | task 7821 | processing task, is_child = 0
|
|
6.35.312.286 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
6.35.312.286 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
6.35.312.295 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
6.35.312.295 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
6.35.312.300 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
6.35.312.300 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
6.39.999.643 I slot print_timing: id 3 | task 7821 | n_decoded = 100, tg = 23.25 t/s, tg_3s = 23.25 t/s
|
|
6.41.277.865 I slot print_timing: id 3 | task 7821 | prompt eval time = 386.78 ms / 16 tokens ( 24.17 ms per token, 41.37 tokens per second)
|
|
6.41.277.868 I slot print_timing: id 3 | task 7821 | eval time = 5578.77 ms / 129 tokens ( 43.25 ms per token, 23.12 tokens per second)
|
|
6.41.277.869 I slot print_timing: id 3 | task 7821 | total time = 5965.55 ms / 145 tokens
|
|
6.41.277.869 I slot print_timing: id 3 | task 7821 | graphs reused = 7869
|
|
6.41.277.892 I slot release: id 3 | task 7821 | stop processing: n_tokens = 193, truncated = 0
|
|
6.41.277.903 I srv update_slots: all slots are idle
|
|
6.41.278.766 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
6.41.298.060 I srv operator(): Chat format: peg-native
|
|
6.41.303.991 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.781 (> 0.100 thold), f_keep = 0.259
|
|
6.41.303.993 I srv get_availabl: updating prompt cache
|
|
6.41.304.023 W srv prompt_save: - saving prompt with length 193, total state size = 46.746 MiB (draft: 0.000 MiB)
|
|
6.41.341.006 I srv load: - looking for better prompt, base f_keep = 0.259, sim = 0.781
|
|
6.41.341.020 I srv update: - cache state: 17 prompts, 2142.032 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
6.41.341.021 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
6.41.341.021 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
6.41.341.022 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
6.41.341.023 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
6.41.341.023 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
6.41.341.024 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
6.41.341.024 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
6.41.341.025 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
6.41.341.025 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
6.41.341.026 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
6.41.341.026 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
6.41.341.027 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
6.41.341.027 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
6.41.341.032 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
6.41.341.032 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
6.41.341.033 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
6.41.341.034 I srv update: - prompt 0x603800178db0: 193 tokens, checkpoints: 0, 46.746 MiB
|
|
6.41.341.035 I srv get_availabl: prompt cache update took 37.04 ms
|
|
6.41.342.796 I slot launch_slot_: id 3 | task 7951 | processing task, is_child = 0
|
|
6.41.342.798 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
6.41.342.799 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
6.41.342.807 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
6.41.342.807 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
6.41.342.811 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
6.41.342.812 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
6.45.687.568 I slot print_timing: id 3 | task 7951 | prompt eval time = 358.32 ms / 14 tokens ( 25.59 ms per token, 39.07 tokens per second)
|
|
6.45.687.572 I slot print_timing: id 3 | task 7951 | eval time = 3986.42 ms / 93 tokens ( 42.86 ms per token, 23.33 tokens per second)
|
|
6.45.687.572 I slot print_timing: id 3 | task 7951 | total time = 4344.74 ms / 107 tokens
|
|
6.45.687.573 I slot print_timing: id 3 | task 7951 | graphs reused = 7960
|
|
6.45.687.593 I slot release: id 3 | task 7951 | stop processing: n_tokens = 156, truncated = 0
|
|
6.45.687.600 I srv update_slots: all slots are idle
|
|
6.45.688.389 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
6.45.707.354 I srv operator(): Chat format: peg-native
|
|
6.45.713.506 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.778 (> 0.100 thold), f_keep = 0.314
|
|
6.45.713.509 I srv get_availabl: updating prompt cache
|
|
6.45.713.534 W srv prompt_save: - saving prompt with length 156, total state size = 37.784 MiB (draft: 0.000 MiB)
|
|
6.45.743.238 I srv load: - looking for better prompt, base f_keep = 0.314, sim = 0.778
|
|
6.45.743.251 I srv update: - cache state: 18 prompts, 2179.816 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
6.45.743.255 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
6.45.743.256 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
6.45.743.257 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
6.45.743.257 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
6.45.743.258 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
6.45.743.258 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
6.45.743.259 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
6.45.743.259 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
6.45.743.260 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
6.45.743.260 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
6.45.743.261 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
6.45.743.265 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
6.45.743.265 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
6.45.743.265 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
6.45.743.266 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
6.45.743.266 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
6.45.743.267 I srv update: - prompt 0x603800178db0: 193 tokens, checkpoints: 0, 46.746 MiB
|
|
6.45.743.267 I srv update: - prompt 0x6037fffbbd90: 156 tokens, checkpoints: 0, 37.784 MiB
|
|
6.45.743.269 I srv get_availabl: prompt cache update took 29.76 ms
|
|
6.45.744.914 I slot launch_slot_: id 3 | task 8045 | processing task, is_child = 0
|
|
6.45.744.916 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
6.45.744.917 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
6.45.744.924 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
6.45.744.924 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
6.45.744.928 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
6.45.744.928 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
6.50.066.537 I slot print_timing: id 3 | task 8045 | prompt eval time = 362.25 ms / 14 tokens ( 25.88 ms per token, 38.65 tokens per second)
|
|
6.50.066.540 I slot print_timing: id 3 | task 8045 | eval time = 3959.34 ms / 92 tokens ( 43.04 ms per token, 23.24 tokens per second)
|
|
6.50.066.540 I slot print_timing: id 3 | task 8045 | total time = 4321.59 ms / 106 tokens
|
|
6.50.066.541 I slot print_timing: id 3 | task 8045 | graphs reused = 8050
|
|
6.50.066.573 I slot release: id 3 | task 8045 | stop processing: n_tokens = 154, truncated = 0
|
|
6.50.066.586 I srv update_slots: all slots are idle
|
|
6.50.067.431 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
6.50.086.061 I srv operator(): Chat format: peg-native
|
|
6.50.091.955 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.318
|
|
6.50.091.957 I srv get_availabl: updating prompt cache
|
|
6.50.091.980 W srv prompt_save: - saving prompt with length 154, total state size = 37.300 MiB (draft: 0.000 MiB)
|
|
6.50.121.085 I srv load: - looking for better prompt, base f_keep = 0.318, sim = 0.721
|
|
6.50.121.098 I srv update: - cache state: 19 prompts, 2217.117 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
6.50.121.100 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
6.50.121.100 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
6.50.121.101 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
6.50.121.102 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
6.50.121.102 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
6.50.121.103 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
6.50.121.103 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
6.50.121.104 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
6.50.121.104 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
6.50.121.106 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
6.50.121.106 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
6.50.121.107 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
6.50.121.107 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
6.50.121.107 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
6.50.121.108 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
6.50.121.108 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
6.50.121.109 I srv update: - prompt 0x603800178db0: 193 tokens, checkpoints: 0, 46.746 MiB
|
|
6.50.121.109 I srv update: - prompt 0x6037fffbbd90: 156 tokens, checkpoints: 0, 37.784 MiB
|
|
6.50.121.110 I srv update: - prompt 0x603800122720: 154 tokens, checkpoints: 0, 37.300 MiB
|
|
6.50.121.111 I srv get_availabl: prompt cache update took 29.15 ms
|
|
6.50.122.738 I slot launch_slot_: id 3 | task 8138 | processing task, is_child = 0
|
|
6.50.122.740 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
6.50.122.741 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
6.50.122.749 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
6.50.122.750 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
6.50.122.754 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
6.50.122.754 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
6.54.994.014 I slot print_timing: id 3 | task 8138 | n_decoded = 100, tg = 23.20 t/s, tg_3s = 23.19 t/s
|
|
6.57.999.853 I slot print_timing: id 3 | task 8138 | n_decoded = 169, tg = 23.10 t/s, tg_3s = 22.96 t/s
|
|
7.01.038.320 I slot print_timing: id 3 | task 8138 | n_decoded = 238, tg = 22.98 t/s, tg_3s = 22.71 t/s
|
|
7.04.069.140 I slot print_timing: id 3 | task 8138 | n_decoded = 306, tg = 22.86 t/s, tg_3s = 22.44 t/s
|
|
7.05.222.179 I slot print_timing: id 3 | task 8138 | prompt eval time = 559.97 ms / 19 tokens ( 29.47 ms per token, 33.93 tokens per second)
|
|
7.05.222.185 I slot print_timing: id 3 | task 8138 | eval time = 14539.43 ms / 332 tokens ( 43.79 ms per token, 22.83 tokens per second)
|
|
7.05.222.185 I slot print_timing: id 3 | task 8138 | total time = 15099.41 ms / 351 tokens
|
|
7.05.222.186 I slot print_timing: id 3 | task 8138 | graphs reused = 8379
|
|
7.05.222.215 I slot release: id 3 | task 8138 | stop processing: n_tokens = 399, truncated = 0
|
|
7.05.222.227 I srv update_slots: all slots are idle
|
|
7.05.223.086 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
7.05.242.004 I srv operator(): Chat format: peg-native
|
|
7.05.248.044 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.123
|
|
7.05.248.046 I srv get_availabl: updating prompt cache
|
|
7.05.248.081 W srv prompt_save: - saving prompt with length 399, total state size = 96.639 MiB (draft: 0.000 MiB)
|
|
7.05.319.647 I srv load: - looking for better prompt, base f_keep = 0.123, sim = 0.731
|
|
7.05.319.659 I srv load: - found better prompt with f_keep = 0.333, sim = 0.776
|
|
7.05.337.027 I srv update: - cache state: 19 prompts, 2275.971 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
7.05.337.031 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
7.05.337.031 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
7.05.337.032 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
7.05.337.033 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
7.05.337.033 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
7.05.337.034 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
7.05.337.034 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
7.05.337.035 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
7.05.337.035 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
7.05.337.036 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
7.05.337.036 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
7.05.337.038 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
7.05.337.041 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
7.05.337.041 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
7.05.337.042 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
7.05.337.042 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
7.05.337.043 I srv update: - prompt 0x603800178db0: 193 tokens, checkpoints: 0, 46.746 MiB
|
|
7.05.337.043 I srv update: - prompt 0x603800122720: 154 tokens, checkpoints: 0, 37.300 MiB
|
|
7.05.337.044 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
7.05.337.046 I srv get_availabl: prompt cache update took 89.00 ms
|
|
7.05.338.848 I slot launch_slot_: id 3 | task 8471 | processing task, is_child = 0
|
|
7.05.338.850 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.05.338.850 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.05.338.858 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.05.338.858 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.05.338.863 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.05.338.863 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.10.023.671 I slot print_timing: id 3 | task 8471 | n_decoded = 100, tg = 23.21 t/s, tg_3s = 23.21 t/s
|
|
7.13.049.625 I slot print_timing: id 3 | task 8471 | n_decoded = 170, tg = 23.18 t/s, tg_3s = 23.13 t/s
|
|
7.15.894.958 I slot print_timing: id 3 | task 8471 | prompt eval time = 376.30 ms / 15 tokens ( 25.09 ms per token, 39.86 tokens per second)
|
|
7.15.894.963 I slot print_timing: id 3 | task 8471 | eval time = 10179.78 ms / 235 tokens ( 43.32 ms per token, 23.08 tokens per second)
|
|
7.15.894.964 I slot print_timing: id 3 | task 8471 | total time = 10556.08 ms / 250 tokens
|
|
7.15.894.964 I slot print_timing: id 3 | task 8471 | graphs reused = 8611
|
|
7.15.894.987 I slot release: id 3 | task 8471 | stop processing: n_tokens = 301, truncated = 0
|
|
7.15.895.000 I srv update_slots: all slots are idle
|
|
7.15.895.743 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
7.15.914.312 I srv operator(): Chat format: peg-native
|
|
7.15.920.307 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.671 (> 0.100 thold), f_keep = 0.163
|
|
7.15.920.309 I srv get_availabl: updating prompt cache
|
|
7.15.920.337 W srv prompt_save: - saving prompt with length 301, total state size = 72.903 MiB (draft: 0.000 MiB)
|
|
7.15.975.935 I srv load: - looking for better prompt, base f_keep = 0.163, sim = 0.671
|
|
7.15.975.951 I srv load: - found better prompt with f_keep = 0.331, sim = 0.699
|
|
7.15.994.387 I srv update: - cache state: 19 prompts, 2311.574 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
7.15.994.393 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
7.15.994.394 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
7.15.994.394 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
7.15.994.395 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
7.15.994.396 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
7.15.994.396 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
7.15.994.397 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
7.15.994.397 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
7.15.994.398 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
7.15.994.398 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
7.15.994.399 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
7.15.994.399 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
7.15.994.400 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
7.15.994.400 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
7.15.994.401 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
7.15.994.401 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
7.15.994.402 I srv update: - prompt 0x603800178db0: 193 tokens, checkpoints: 0, 46.746 MiB
|
|
7.15.994.402 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
7.15.994.403 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
7.15.994.407 I srv get_availabl: prompt cache update took 74.10 ms
|
|
7.15.996.164 I slot launch_slot_: id 3 | task 8707 | processing task, is_child = 0
|
|
7.15.996.166 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.15.996.167 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.15.996.174 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.15.996.174 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.15.996.179 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.15.996.179 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.20.926.169 I slot print_timing: id 3 | task 8707 | n_decoded = 100, tg = 23.20 t/s, tg_3s = 23.20 t/s
|
|
7.22.011.274 I slot print_timing: id 3 | task 8707 | prompt eval time = 618.91 ms / 22 tokens ( 28.13 ms per token, 35.55 tokens per second)
|
|
7.22.011.278 I slot print_timing: id 3 | task 8707 | eval time = 5396.17 ms / 125 tokens ( 43.17 ms per token, 23.16 tokens per second)
|
|
7.22.011.278 I slot print_timing: id 3 | task 8707 | total time = 6015.08 ms / 147 tokens
|
|
7.22.011.279 I slot print_timing: id 3 | task 8707 | graphs reused = 8734
|
|
7.22.011.301 I slot release: id 3 | task 8707 | stop processing: n_tokens = 197, truncated = 0
|
|
7.22.011.311 I srv update_slots: all slots are idle
|
|
7.22.012.156 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
7.22.030.287 I srv operator(): Chat format: peg-native
|
|
7.22.035.731 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.681 (> 0.100 thold), f_keep = 0.249
|
|
7.22.035.733 I srv get_availabl: updating prompt cache
|
|
7.22.035.760 W srv prompt_save: - saving prompt with length 197, total state size = 47.715 MiB (draft: 0.000 MiB)
|
|
7.22.075.782 I srv load: - looking for better prompt, base f_keep = 0.249, sim = 0.681
|
|
7.22.075.792 I srv load: - found better prompt with f_keep = 0.259, sim = 0.694
|
|
7.22.096.423 I srv update: - cache state: 19 prompts, 2312.543 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
7.22.096.428 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
7.22.096.429 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
7.22.096.429 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
7.22.096.430 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
7.22.096.431 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
7.22.096.431 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
7.22.096.432 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
7.22.096.433 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
7.22.096.433 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
7.22.096.434 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
7.22.096.447 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
7.22.096.448 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
7.22.096.449 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
7.22.096.449 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
7.22.096.450 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
7.22.096.450 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
7.22.096.451 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
7.22.096.452 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
7.22.096.452 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
7.22.096.453 I srv get_availabl: prompt cache update took 60.72 ms
|
|
7.22.098.465 I slot launch_slot_: id 3 | task 8833 | processing task, is_child = 0
|
|
7.22.098.467 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.22.098.468 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.22.098.475 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.22.098.475 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.22.098.480 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.22.098.480 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.27.010.291 I slot print_timing: id 3 | task 8833 | n_decoded = 100, tg = 23.28 t/s, tg_3s = 23.28 t/s
|
|
7.30.019.634 I slot print_timing: id 3 | task 8833 | n_decoded = 169, tg = 23.13 t/s, tg_3s = 22.93 t/s
|
|
7.30.062.872 I slot print_timing: id 3 | task 8833 | prompt eval time = 615.57 ms / 22 tokens ( 27.98 ms per token, 35.74 tokens per second)
|
|
7.30.062.876 I slot print_timing: id 3 | task 8833 | eval time = 7348.80 ms / 170 tokens ( 43.23 ms per token, 23.13 tokens per second)
|
|
7.30.062.876 I slot print_timing: id 3 | task 8833 | total time = 7964.38 ms / 192 tokens
|
|
7.30.062.877 I slot print_timing: id 3 | task 8833 | graphs reused = 8902
|
|
7.30.062.904 I slot release: id 3 | task 8833 | stop processing: n_tokens = 241, truncated = 0
|
|
7.30.062.913 I srv update_slots: all slots are idle
|
|
7.30.063.735 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
7.30.080.703 I srv operator(): Chat format: peg-native
|
|
7.30.086.165 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.778 (> 0.100 thold), f_keep = 0.203
|
|
7.30.086.167 I srv get_availabl: updating prompt cache
|
|
7.30.086.190 W srv prompt_save: - saving prompt with length 241, total state size = 58.371 MiB (draft: 0.000 MiB)
|
|
7.30.129.593 I srv load: - looking for better prompt, base f_keep = 0.203, sim = 0.778
|
|
7.30.129.608 I srv update: - cache state: 20 prompts, 2370.914 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
7.30.129.609 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
7.30.129.611 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
7.30.129.611 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
7.30.129.612 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
7.30.129.612 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
7.30.129.613 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
7.30.129.613 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
7.30.129.615 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
7.30.129.616 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
7.30.129.616 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
7.30.129.617 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
7.30.129.617 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
7.30.129.617 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
7.30.129.618 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
7.30.129.618 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
7.30.129.619 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
7.30.129.619 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
7.30.129.620 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
7.30.129.620 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
7.30.129.620 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
7.30.129.622 I srv get_availabl: prompt cache update took 43.45 ms
|
|
7.30.131.279 I slot launch_slot_: id 3 | task 9004 | processing task, is_child = 0
|
|
7.30.131.281 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.30.131.282 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.30.131.289 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.30.131.289 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.30.131.293 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.30.131.294 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.34.792.843 I slot print_timing: id 3 | task 9004 | n_decoded = 100, tg = 23.23 t/s, tg_3s = 23.23 t/s
|
|
7.35.919.625 I slot print_timing: id 3 | task 9004 | prompt eval time = 357.65 ms / 14 tokens ( 25.55 ms per token, 39.14 tokens per second)
|
|
7.35.919.629 I slot print_timing: id 3 | task 9004 | eval time = 5430.66 ms / 126 tokens ( 43.10 ms per token, 23.20 tokens per second)
|
|
7.35.919.630 I slot print_timing: id 3 | task 9004 | total time = 5788.32 ms / 140 tokens
|
|
7.35.919.630 I slot print_timing: id 3 | task 9004 | graphs reused = 9026
|
|
7.35.919.656 I slot release: id 3 | task 9004 | stop processing: n_tokens = 188, truncated = 0
|
|
7.35.919.667 I srv update_slots: all slots are idle
|
|
7.35.920.479 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
7.35.937.480 I srv operator(): Chat format: peg-native
|
|
7.35.942.981 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.261
|
|
7.35.942.983 I srv get_availabl: updating prompt cache
|
|
7.35.943.005 W srv prompt_save: - saving prompt with length 188, total state size = 45.535 MiB (draft: 0.000 MiB)
|
|
7.35.977.653 I srv load: - looking for better prompt, base f_keep = 0.261, sim = 0.731
|
|
7.35.977.668 I srv update: - cache state: 21 prompts, 2416.449 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
7.35.977.669 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
7.35.977.670 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
7.35.977.670 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
7.35.977.671 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
7.35.977.672 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
7.35.977.672 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
7.35.977.673 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
7.35.977.673 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
7.35.977.674 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
7.35.977.674 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
7.35.977.674 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
7.35.977.675 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
7.35.977.675 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
7.35.977.676 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
7.35.977.676 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
7.35.977.677 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
7.35.977.677 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
7.35.977.678 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
7.35.977.678 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
7.35.977.678 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
7.35.977.679 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
7.35.977.680 I srv get_availabl: prompt cache update took 34.70 ms
|
|
7.35.979.351 I slot launch_slot_: id 3 | task 9131 | processing task, is_child = 0
|
|
7.35.979.353 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.35.979.354 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.35.979.361 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.35.979.361 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.35.979.365 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.35.979.365 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.40.847.642 I slot print_timing: id 3 | task 9131 | n_decoded = 100, tg = 23.29 t/s, tg_3s = 23.29 t/s
|
|
7.43.059.890 I slot print_timing: id 3 | task 9131 | prompt eval time = 574.41 ms / 18 tokens ( 31.91 ms per token, 31.34 tokens per second)
|
|
7.43.059.894 I slot print_timing: id 3 | task 9131 | eval time = 6506.09 ms / 151 tokens ( 43.09 ms per token, 23.21 tokens per second)
|
|
7.43.059.894 I slot print_timing: id 3 | task 9131 | total time = 7080.51 ms / 169 tokens
|
|
7.43.059.895 I slot print_timing: id 3 | task 9131 | graphs reused = 9175
|
|
7.43.059.917 I slot release: id 3 | task 9131 | stop processing: n_tokens = 217, truncated = 0
|
|
7.43.059.925 I srv update_slots: all slots are idle
|
|
7.43.060.682 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
7.43.080.190 I srv operator(): Chat format: peg-native
|
|
7.43.086.312 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.226
|
|
7.43.086.313 I srv get_availabl: updating prompt cache
|
|
7.43.086.353 W srv prompt_save: - saving prompt with length 217, total state size = 52.559 MiB (draft: 0.000 MiB)
|
|
7.43.125.360 I srv load: - looking for better prompt, base f_keep = 0.226, sim = 0.710
|
|
7.43.125.378 I srv update: - cache state: 22 prompts, 2469.008 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
7.43.125.379 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
7.43.125.380 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
7.43.125.380 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
7.43.125.381 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
7.43.125.381 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
7.43.125.382 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
7.43.125.382 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
7.43.125.383 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
7.43.125.383 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
7.43.125.384 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
7.43.125.384 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
7.43.125.384 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
7.43.125.385 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
7.43.125.385 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
7.43.125.386 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
7.43.125.386 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
7.43.125.387 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
7.43.125.387 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
7.43.125.388 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
7.43.125.388 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
7.43.125.388 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
7.43.125.389 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
7.43.125.390 I srv get_availabl: prompt cache update took 39.08 ms
|
|
7.43.127.289 I slot launch_slot_: id 3 | task 9283 | processing task, is_child = 0
|
|
7.43.127.293 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.43.127.294 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.43.127.304 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.43.127.304 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.43.127.309 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.43.127.309 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.48.047.972 I slot print_timing: id 3 | task 9283 | n_decoded = 100, tg = 23.07 t/s, tg_3s = 23.07 t/s
|
|
7.51.077.199 I slot print_timing: id 3 | task 9283 | n_decoded = 169, tg = 22.95 t/s, tg_3s = 22.78 t/s
|
|
7.54.105.047 I slot print_timing: id 3 | task 9283 | n_decoded = 238, tg = 22.90 t/s, tg_3s = 22.79 t/s
|
|
7.57.110.961 I slot print_timing: id 3 | task 9283 | n_decoded = 307, tg = 22.91 t/s, tg_3s = 22.95 t/s
|
|
8.00.147.242 I slot print_timing: id 3 | task 9283 | n_decoded = 377, tg = 22.94 t/s, tg_3s = 23.05 t/s
|
|
8.03.165.587 I slot print_timing: id 3 | task 9283 | n_decoded = 446, tg = 22.93 t/s, tg_3s = 22.86 t/s
|
|
8.06.172.673 I slot print_timing: id 3 | task 9283 | n_decoded = 515, tg = 22.93 t/s, tg_3s = 22.95 t/s
|
|
8.09.180.693 I slot print_timing: id 3 | task 9283 | n_decoded = 584, tg = 22.93 t/s, tg_3s = 22.94 t/s
|
|
8.12.188.980 I slot print_timing: id 3 | task 9283 | n_decoded = 653, tg = 22.93 t/s, tg_3s = 22.94 t/s
|
|
8.15.219.511 I slot print_timing: id 3 | task 9283 | n_decoded = 722, tg = 22.92 t/s, tg_3s = 22.77 t/s
|
|
8.18.220.784 I slot print_timing: id 3 | task 9283 | n_decoded = 790, tg = 22.89 t/s, tg_3s = 22.66 t/s
|
|
8.21.222.783 I slot print_timing: id 3 | task 9283 | n_decoded = 858, tg = 22.87 t/s, tg_3s = 22.65 t/s
|
|
8.24.223.755 I slot print_timing: id 3 | task 9283 | n_decoded = 926, tg = 22.86 t/s, tg_3s = 22.66 t/s
|
|
8.27.250.072 I slot print_timing: id 3 | task 9283 | n_decoded = 994, tg = 22.83 t/s, tg_3s = 22.47 t/s
|
|
8.30.266.836 I slot print_timing: id 3 | task 9283 | n_decoded = 1062, tg = 22.81 t/s, tg_3s = 22.54 t/s
|
|
8.33.282.227 I slot print_timing: id 3 | task 9283 | n_decoded = 1130, tg = 22.80 t/s, tg_3s = 22.55 t/s
|
|
8.36.300.027 I slot print_timing: id 3 | task 9283 | n_decoded = 1198, tg = 22.78 t/s, tg_3s = 22.53 t/s
|
|
8.36.388.603 I slot print_timing: id 3 | task 9283 | prompt eval time = 585.62 ms / 20 tokens ( 29.28 ms per token, 34.15 tokens per second)
|
|
8.36.388.606 I slot print_timing: id 3 | task 9283 | eval time = 52675.66 ms / 1200 tokens ( 43.90 ms per token, 22.78 tokens per second)
|
|
8.36.388.607 I slot print_timing: id 3 | task 9283 | total time = 53261.28 ms / 1220 tokens
|
|
8.36.388.607 I slot print_timing: id 3 | task 9283 | graphs reused = 10369
|
|
8.36.388.631 I slot release: id 3 | task 9283 | stop processing: n_tokens = 1268, truncated = 0
|
|
8.36.388.645 I srv update_slots: all slots are idle
|
|
8.36.389.481 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
8.36.407.579 I srv operator(): Chat format: peg-native
|
|
8.36.413.101 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.613 (> 0.100 thold), f_keep = 0.039
|
|
8.36.413.103 I srv get_availabl: updating prompt cache
|
|
8.36.413.154 W srv prompt_save: - saving prompt with length 1268, total state size = 307.110 MiB (draft: 0.000 MiB)
|
|
8.36.647.874 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.613
|
|
8.36.647.892 I srv update: - cache state: 23 prompts, 2776.117 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
8.36.647.893 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
8.36.647.894 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
8.36.647.895 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
8.36.647.895 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
8.36.647.896 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
8.36.647.903 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
8.36.647.903 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
8.36.647.904 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
8.36.647.905 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
8.36.647.906 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
8.36.647.910 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
8.36.647.911 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
8.36.647.912 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
8.36.647.912 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
8.36.647.913 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
8.36.647.913 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
8.36.647.916 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
8.36.647.917 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
8.36.647.918 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
8.36.647.918 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
8.36.647.919 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
8.36.647.920 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
8.36.647.921 I srv update: - prompt 0x6037ffb8e900: 1268 tokens, checkpoints: 0, 307.110 MiB
|
|
8.36.647.922 I srv get_availabl: prompt cache update took 234.82 ms
|
|
8.36.649.571 I slot launch_slot_: id 3 | task 10484 | processing task, is_child = 0
|
|
8.36.649.574 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
8.36.649.574 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
8.36.649.583 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
8.36.649.584 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
8.36.649.589 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
8.36.649.589 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
8.41.822.165 I slot print_timing: id 3 | task 10484 | n_decoded = 100, tg = 23.12 t/s, tg_3s = 23.12 t/s
|
|
8.44.861.947 I slot print_timing: id 3 | task 10484 | n_decoded = 170, tg = 23.08 t/s, tg_3s = 23.03 t/s
|
|
8.47.903.638 I slot print_timing: id 3 | task 10484 | n_decoded = 240, tg = 23.06 t/s, tg_3s = 23.01 t/s
|
|
8.50.921.794 I slot print_timing: id 3 | task 10484 | n_decoded = 309, tg = 23.02 t/s, tg_3s = 22.86 t/s
|
|
8.53.953.595 I slot print_timing: id 3 | task 10484 | n_decoded = 378, tg = 22.97 t/s, tg_3s = 22.76 t/s
|
|
8.56.956.756 I slot print_timing: id 3 | task 10484 | n_decoded = 447, tg = 22.97 t/s, tg_3s = 22.98 t/s
|
|
8.59.970.127 I slot print_timing: id 3 | task 10484 | n_decoded = 516, tg = 22.96 t/s, tg_3s = 22.90 t/s
|
|
9.02.996.900 I slot print_timing: id 3 | task 10484 | n_decoded = 585, tg = 22.94 t/s, tg_3s = 22.80 t/s
|
|
9.06.010.136 I slot print_timing: id 3 | task 10484 | n_decoded = 654, tg = 22.94 t/s, tg_3s = 22.90 t/s
|
|
9.09.016.037 I slot print_timing: id 3 | task 10484 | n_decoded = 722, tg = 22.91 t/s, tg_3s = 22.62 t/s
|
|
9.12.056.442 I slot print_timing: id 3 | task 10484 | n_decoded = 790, tg = 22.86 t/s, tg_3s = 22.37 t/s
|
|
9.15.090.024 I slot print_timing: id 3 | task 10484 | n_decoded = 858, tg = 22.82 t/s, tg_3s = 22.42 t/s
|
|
9.18.093.467 I slot print_timing: id 3 | task 10484 | n_decoded = 926, tg = 22.81 t/s, tg_3s = 22.64 t/s
|
|
9.21.115.468 I slot print_timing: id 3 | task 10484 | n_decoded = 994, tg = 22.79 t/s, tg_3s = 22.50 t/s
|
|
9.24.134.126 I slot print_timing: id 3 | task 10484 | n_decoded = 1062, tg = 22.77 t/s, tg_3s = 22.53 t/s
|
|
9.27.146.604 I slot print_timing: id 3 | task 10484 | n_decoded = 1130, tg = 22.76 t/s, tg_3s = 22.57 t/s
|
|
9.30.179.614 I slot print_timing: id 3 | task 10484 | n_decoded = 1198, tg = 22.74 t/s, tg_3s = 22.42 t/s
|
|
9.30.268.559 I slot print_timing: id 3 | task 10484 | prompt eval time = 848.09 ms / 31 tokens ( 27.36 ms per token, 36.55 tokens per second)
|
|
9.30.268.562 I slot print_timing: id 3 | task 10484 | eval time = 52770.86 ms / 1200 tokens ( 43.98 ms per token, 22.74 tokens per second)
|
|
9.30.268.563 I slot print_timing: id 3 | task 10484 | total time = 53618.95 ms / 1231 tokens
|
|
9.30.268.564 I slot print_timing: id 3 | task 10484 | graphs reused = 11563
|
|
9.30.268.587 I slot release: id 3 | task 10484 | stop processing: n_tokens = 1279, truncated = 0
|
|
9.30.268.602 I srv update_slots: all slots are idle
|
|
9.30.269.374 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
9.30.286.326 I srv operator(): Chat format: peg-native
|
|
9.30.291.816 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.704 (> 0.100 thold), f_keep = 0.039
|
|
9.30.291.818 I srv get_availabl: updating prompt cache
|
|
9.30.291.869 W srv prompt_save: - saving prompt with length 1279, total state size = 309.774 MiB (draft: 0.000 MiB)
|
|
9.30.510.208 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.704
|
|
9.30.510.224 I srv update: - cache state: 24 prompts, 3085.891 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
9.30.510.225 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
9.30.510.226 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
9.30.510.227 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
9.30.510.227 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
9.30.510.228 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
9.30.510.229 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
9.30.510.230 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
9.30.510.231 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
9.30.510.231 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
9.30.510.232 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
9.30.510.233 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
9.30.510.234 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
9.30.510.234 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
9.30.510.235 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
9.30.510.236 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
9.30.510.236 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
9.30.510.237 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
9.30.510.238 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
9.30.510.238 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
9.30.510.239 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
9.30.510.240 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
9.30.510.240 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
9.30.510.241 I srv update: - prompt 0x6037ffb8e900: 1268 tokens, checkpoints: 0, 307.110 MiB
|
|
9.30.510.242 I srv update: - prompt 0x6037ffff8ec0: 1279 tokens, checkpoints: 0, 309.774 MiB
|
|
9.30.510.243 I srv get_availabl: prompt cache update took 218.42 ms
|
|
9.30.512.213 I slot launch_slot_: id 3 | task 11685 | processing task, is_child = 0
|
|
9.30.512.215 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
9.30.512.216 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
9.30.512.226 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
9.30.512.227 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
9.30.512.232 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
9.30.512.232 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
9.35.443.946 I slot print_timing: id 3 | task 11685 | n_decoded = 100, tg = 23.12 t/s, tg_3s = 23.12 t/s
|
|
9.38.478.821 I slot print_timing: id 3 | task 11685 | n_decoded = 170, tg = 23.10 t/s, tg_3s = 23.07 t/s
|
|
9.41.519.809 I slot print_timing: id 3 | task 11685 | n_decoded = 240, tg = 23.07 t/s, tg_3s = 23.02 t/s
|
|
9.44.560.668 I slot print_timing: id 3 | task 11685 | n_decoded = 310, tg = 23.06 t/s, tg_3s = 23.02 t/s
|
|
9.47.597.153 I slot print_timing: id 3 | task 11685 | n_decoded = 380, tg = 23.06 t/s, tg_3s = 23.05 t/s
|
|
9.50.607.406 I slot print_timing: id 3 | task 11685 | n_decoded = 449, tg = 23.04 t/s, tg_3s = 22.92 t/s
|
|
9.53.618.984 I slot print_timing: id 3 | task 11685 | n_decoded = 518, tg = 23.02 t/s, tg_3s = 22.91 t/s
|
|
9.56.630.699 I slot print_timing: id 3 | task 11685 | n_decoded = 587, tg = 23.01 t/s, tg_3s = 22.91 t/s
|
|
9.59.648.376 I slot print_timing: id 3 | task 11685 | n_decoded = 656, tg = 22.99 t/s, tg_3s = 22.87 t/s
|
|
10.02.673.630 I slot print_timing: id 3 | task 11685 | n_decoded = 723, tg = 22.91 t/s, tg_3s = 22.15 t/s
|
|
10.05.716.085 I slot print_timing: id 3 | task 11685 | n_decoded = 791, tg = 22.86 t/s, tg_3s = 22.35 t/s
|
|
10.08.727.741 I slot print_timing: id 3 | task 11685 | n_decoded = 859, tg = 22.84 t/s, tg_3s = 22.58 t/s
|
|
10.11.739.549 I slot print_timing: id 3 | task 11685 | n_decoded = 927, tg = 22.82 t/s, tg_3s = 22.58 t/s
|
|
10.14.767.138 I slot print_timing: id 3 | task 11685 | n_decoded = 995, tg = 22.80 t/s, tg_3s = 22.46 t/s
|
|
10.17.786.202 I slot print_timing: id 3 | task 11685 | n_decoded = 1063, tg = 22.78 t/s, tg_3s = 22.52 t/s
|
|
10.20.811.587 I slot print_timing: id 3 | task 11685 | n_decoded = 1131, tg = 22.76 t/s, tg_3s = 22.48 t/s
|
|
10.23.832.526 I slot print_timing: id 3 | task 11685 | n_decoded = 1199, tg = 22.75 t/s, tg_3s = 22.51 t/s
|
|
10.23.877.884 I slot print_timing: id 3 | task 11685 | prompt eval time = 606.35 ms / 21 tokens ( 28.87 ms per token, 34.63 tokens per second)
|
|
10.23.877.888 I slot print_timing: id 3 | task 11685 | eval time = 52759.28 ms / 1200 tokens ( 43.97 ms per token, 22.74 tokens per second)
|
|
10.23.877.889 I slot print_timing: id 3 | task 11685 | total time = 53365.63 ms / 1221 tokens
|
|
10.23.877.889 I slot print_timing: id 3 | task 11685 | graphs reused = 12757
|
|
10.23.877.914 I slot release: id 3 | task 11685 | stop processing: n_tokens = 1270, truncated = 0
|
|
10.23.877.933 I srv update_slots: all slots are idle
|
|
10.23.878.852 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
10.23.897.977 I srv operator(): Chat format: peg-native
|
|
10.23.904.302 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.039
|
|
10.23.904.304 I srv get_availabl: updating prompt cache
|
|
10.23.904.369 W srv prompt_save: - saving prompt with length 1270, total state size = 307.594 MiB (draft: 0.000 MiB)
|
|
10.24.110.110 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.742
|
|
10.24.110.125 I srv update: - cache state: 25 prompts, 3393.485 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
10.24.110.130 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
10.24.110.130 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
10.24.110.131 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
10.24.110.132 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
10.24.110.135 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
10.24.110.136 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
10.24.110.137 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
10.24.110.137 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
10.24.110.138 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
10.24.110.141 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
10.24.110.142 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
10.24.110.143 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
10.24.110.143 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
10.24.110.144 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
10.24.110.145 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
10.24.110.146 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
10.24.110.146 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
10.24.110.147 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
10.24.110.148 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
10.24.110.148 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
10.24.110.149 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
10.24.110.149 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
10.24.110.150 I srv update: - prompt 0x6037ffb8e900: 1268 tokens, checkpoints: 0, 307.110 MiB
|
|
10.24.110.150 I srv update: - prompt 0x6037ffff8ec0: 1279 tokens, checkpoints: 0, 309.774 MiB
|
|
10.24.110.151 I srv update: - prompt 0x6037ffb5be30: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
10.24.110.153 I srv get_availabl: prompt cache update took 205.85 ms
|
|
10.24.111.798 I slot launch_slot_: id 3 | task 12886 | processing task, is_child = 0
|
|
10.24.111.800 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
10.24.111.800 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
10.24.111.809 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
10.24.111.809 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
10.24.111.813 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
10.24.111.814 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
10.28.953.449 I slot print_timing: id 3 | task 12886 | n_decoded = 100, tg = 23.29 t/s, tg_3s = 23.29 t/s
|
|
10.31.978.633 I slot print_timing: id 3 | task 12886 | n_decoded = 170, tg = 23.23 t/s, tg_3s = 23.14 t/s
|
|
10.35.021.111 I slot print_timing: id 3 | task 12886 | n_decoded = 240, tg = 23.16 t/s, tg_3s = 23.01 t/s
|
|
10.38.057.692 I slot print_timing: id 3 | task 12886 | n_decoded = 310, tg = 23.14 t/s, tg_3s = 23.05 t/s
|
|
10.41.069.113 I slot print_timing: id 3 | task 12886 | n_decoded = 379, tg = 23.10 t/s, tg_3s = 22.91 t/s
|
|
10.44.077.261 I slot print_timing: id 3 | task 12886 | n_decoded = 448, tg = 23.07 t/s, tg_3s = 22.94 t/s
|
|
10.45.218.897 I slot print_timing: id 3 | task 12886 | prompt eval time = 547.35 ms / 17 tokens ( 32.20 ms per token, 31.06 tokens per second)
|
|
10.45.218.901 I slot print_timing: id 3 | task 12886 | eval time = 20559.72 ms / 474 tokens ( 43.37 ms per token, 23.05 tokens per second)
|
|
10.45.218.902 I slot print_timing: id 3 | task 12886 | total time = 21107.07 ms / 491 tokens
|
|
10.45.218.903 I slot print_timing: id 3 | task 12886 | graphs reused = 13227
|
|
10.45.218.925 I slot release: id 3 | task 12886 | stop processing: n_tokens = 539, truncated = 0
|
|
10.45.218.933 I srv update_slots: all slots are idle
|
|
10.45.220.421 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
10.45.239.020 I srv operator(): Chat format: peg-native
|
|
10.45.244.488 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.690 (> 0.100 thold), f_keep = 0.091
|
|
10.45.244.490 I srv get_availabl: updating prompt cache
|
|
10.45.244.528 W srv prompt_save: - saving prompt with length 539, total state size = 130.547 MiB (draft: 0.000 MiB)
|
|
10.45.339.720 I srv load: - looking for better prompt, base f_keep = 0.091, sim = 0.690
|
|
10.45.339.732 I srv update: - cache state: 26 prompts, 3524.032 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
10.45.339.733 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
10.45.339.734 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
10.45.339.734 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
10.45.339.735 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
10.45.339.735 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
10.45.339.736 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
10.45.339.737 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
10.45.339.737 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
10.45.339.737 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
10.45.339.738 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
10.45.339.738 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
10.45.339.739 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
10.45.339.739 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
10.45.339.740 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
10.45.339.740 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
10.45.339.741 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
10.45.339.741 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
10.45.339.741 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
10.45.339.742 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
10.45.339.742 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
10.45.339.743 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
10.45.339.745 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
10.45.339.745 I srv update: - prompt 0x6037ffb8e900: 1268 tokens, checkpoints: 0, 307.110 MiB
|
|
10.45.339.746 I srv update: - prompt 0x6037ffff8ec0: 1279 tokens, checkpoints: 0, 309.774 MiB
|
|
10.45.339.746 I srv update: - prompt 0x6037ffb5be30: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
10.45.339.747 I srv update: - prompt 0x60380004bc50: 539 tokens, checkpoints: 0, 130.547 MiB
|
|
10.45.339.748 I srv get_availabl: prompt cache update took 95.26 ms
|
|
10.45.341.348 I slot launch_slot_: id 3 | task 13361 | processing task, is_child = 0
|
|
10.45.341.350 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
10.45.341.350 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
10.45.341.356 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
10.45.341.357 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
10.45.341.361 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
10.45.341.361 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
10.50.274.797 I slot print_timing: id 3 | task 13361 | n_decoded = 100, tg = 23.23 t/s, tg_3s = 23.23 t/s
|
|
10.53.310.806 I slot print_timing: id 3 | task 13361 | n_decoded = 170, tg = 23.16 t/s, tg_3s = 23.06 t/s
|
|
10.56.317.501 I slot print_timing: id 3 | task 13361 | n_decoded = 239, tg = 23.10 t/s, tg_3s = 22.95 t/s
|
|
10.59.320.977 I slot print_timing: id 3 | task 13361 | n_decoded = 308, tg = 23.07 t/s, tg_3s = 22.97 t/s
|
|
11.02.331.575 I slot print_timing: id 3 | task 13361 | n_decoded = 377, tg = 23.04 t/s, tg_3s = 22.92 t/s
|
|
11.05.341.780 I slot print_timing: id 3 | task 13361 | n_decoded = 446, tg = 23.02 t/s, tg_3s = 22.92 t/s
|
|
11.08.357.940 I slot print_timing: id 3 | task 13361 | n_decoded = 515, tg = 23.00 t/s, tg_3s = 22.88 t/s
|
|
11.11.370.598 I slot print_timing: id 3 | task 13361 | n_decoded = 584, tg = 22.99 t/s, tg_3s = 22.90 t/s
|
|
11.14.382.663 I slot print_timing: id 3 | task 13361 | n_decoded = 653, tg = 22.98 t/s, tg_3s = 22.91 t/s
|
|
11.17.392.592 I slot print_timing: id 3 | task 13361 | n_decoded = 721, tg = 22.95 t/s, tg_3s = 22.59 t/s
|
|
11.20.430.640 I slot print_timing: id 3 | task 13361 | n_decoded = 790, tg = 22.92 t/s, tg_3s = 22.71 t/s
|
|
11.23.470.954 I slot print_timing: id 3 | task 13361 | n_decoded = 859, tg = 22.91 t/s, tg_3s = 22.70 t/s
|
|
11.26.514.946 I slot print_timing: id 3 | task 13361 | n_decoded = 928, tg = 22.89 t/s, tg_3s = 22.67 t/s
|
|
11.29.529.320 I slot print_timing: id 3 | task 13361 | n_decoded = 996, tg = 22.87 t/s, tg_3s = 22.56 t/s
|
|
11.32.540.269 I slot print_timing: id 3 | task 13361 | n_decoded = 1064, tg = 22.85 t/s, tg_3s = 22.58 t/s
|
|
11.35.551.304 I slot print_timing: id 3 | task 13361 | n_decoded = 1132, tg = 22.83 t/s, tg_3s = 22.58 t/s
|
|
11.38.586.465 I slot print_timing: id 3 | task 13361 | prompt eval time = 628.62 ms / 22 tokens ( 28.57 ms per token, 35.00 tokens per second)
|
|
11.38.586.469 I slot print_timing: id 3 | task 13361 | eval time = 52616.47 ms / 1200 tokens ( 43.85 ms per token, 22.81 tokens per second)
|
|
11.38.586.469 I slot print_timing: id 3 | task 13361 | total time = 53245.09 ms / 1222 tokens
|
|
11.38.586.470 I slot print_timing: id 3 | task 13361 | graphs reused = 14421
|
|
11.38.586.497 I slot release: id 3 | task 13361 | stop processing: n_tokens = 1270, truncated = 0
|
|
11.38.586.510 I srv update_slots: all slots are idle
|
|
11.38.587.399 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
11.38.605.991 I srv operator(): Chat format: peg-native
|
|
11.38.611.942 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.704 (> 0.100 thold), f_keep = 0.039
|
|
11.38.611.945 I srv get_availabl: updating prompt cache
|
|
11.38.612.010 W srv prompt_save: - saving prompt with length 1270, total state size = 307.594 MiB (draft: 0.000 MiB)
|
|
11.38.827.300 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.704
|
|
11.38.827.314 I srv update: - cache state: 27 prompts, 3831.626 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
11.38.827.315 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
11.38.827.316 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
11.38.827.316 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
11.38.827.317 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
11.38.827.318 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
11.38.827.318 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
11.38.827.319 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
11.38.827.319 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
11.38.827.320 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
11.38.827.320 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
11.38.827.321 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
11.38.827.321 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
11.38.827.322 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
11.38.827.322 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
11.38.827.323 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
11.38.827.323 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
11.38.827.324 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
11.38.827.324 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
11.38.827.325 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
11.38.827.325 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
11.38.827.326 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
11.38.827.326 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
11.38.827.327 I srv update: - prompt 0x6037ffb8e900: 1268 tokens, checkpoints: 0, 307.110 MiB
|
|
11.38.827.327 I srv update: - prompt 0x6037ffff8ec0: 1279 tokens, checkpoints: 0, 309.774 MiB
|
|
11.38.827.328 I srv update: - prompt 0x6037ffb5be30: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
11.38.827.328 I srv update: - prompt 0x60380004bc50: 539 tokens, checkpoints: 0, 130.547 MiB
|
|
11.38.827.329 I srv update: - prompt 0x6037fffc7250: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
11.38.827.330 I srv get_availabl: prompt cache update took 215.38 ms
|
|
11.38.829.119 I slot launch_slot_: id 3 | task 14562 | processing task, is_child = 0
|
|
11.38.829.121 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
11.38.829.122 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
11.38.829.129 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
11.38.829.130 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
11.38.829.135 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
11.38.829.135 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
11.43.771.067 I slot print_timing: id 3 | task 14562 | n_decoded = 100, tg = 23.13 t/s, tg_3s = 23.13 t/s
|
|
11.46.809.171 I slot print_timing: id 3 | task 14562 | n_decoded = 170, tg = 23.09 t/s, tg_3s = 23.04 t/s
|
|
11.47.285.237 I slot print_timing: id 3 | task 14562 | prompt eval time = 617.75 ms / 21 tokens ( 29.42 ms per token, 33.99 tokens per second)
|
|
11.47.285.240 I slot print_timing: id 3 | task 14562 | eval time = 7838.34 ms / 181 tokens ( 43.31 ms per token, 23.09 tokens per second)
|
|
11.47.285.241 I slot print_timing: id 3 | task 14562 | total time = 8456.09 ms / 202 tokens
|
|
11.47.285.241 I slot print_timing: id 3 | task 14562 | graphs reused = 14600
|
|
11.47.285.259 I slot release: id 3 | task 14562 | stop processing: n_tokens = 251, truncated = 0
|
|
11.47.285.267 I srv update_slots: all slots are idle
|
|
11.47.286.052 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
11.47.305.089 I srv operator(): Chat format: peg-native
|
|
11.47.311.177 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.754 (> 0.100 thold), f_keep = 0.195
|
|
11.47.311.180 I srv get_availabl: updating prompt cache
|
|
11.47.311.213 W srv prompt_save: - saving prompt with length 251, total state size = 60.793 MiB (draft: 0.000 MiB)
|
|
11.47.356.030 I srv load: - looking for better prompt, base f_keep = 0.195, sim = 0.754
|
|
11.47.356.047 I srv update: - cache state: 28 prompts, 3892.420 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
11.47.356.048 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
11.47.356.048 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
11.47.356.049 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
11.47.356.050 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
11.47.356.050 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
11.47.356.051 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
11.47.356.051 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
11.47.356.052 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
11.47.356.052 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
11.47.356.053 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
11.47.356.053 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
11.47.356.054 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
11.47.356.055 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
11.47.356.056 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
11.47.356.056 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
11.47.356.057 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
11.47.356.057 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
11.47.356.058 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
11.47.356.058 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
11.47.356.059 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
11.47.356.059 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
11.47.356.059 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
11.47.356.060 I srv update: - prompt 0x6037ffb8e900: 1268 tokens, checkpoints: 0, 307.110 MiB
|
|
11.47.356.060 I srv update: - prompt 0x6037ffff8ec0: 1279 tokens, checkpoints: 0, 309.774 MiB
|
|
11.47.356.061 I srv update: - prompt 0x6037ffb5be30: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
11.47.356.061 I srv update: - prompt 0x60380004bc50: 539 tokens, checkpoints: 0, 130.547 MiB
|
|
11.47.356.062 I srv update: - prompt 0x6037fffc7250: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
11.47.356.062 I srv update: - prompt 0x6037fff13e70: 251 tokens, checkpoints: 0, 60.793 MiB
|
|
11.47.356.064 I srv get_availabl: prompt cache update took 44.88 ms
|
|
11.47.357.716 I slot launch_slot_: id 3 | task 14744 | processing task, is_child = 0
|
|
11.47.357.718 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
11.47.357.719 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
11.47.357.726 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
11.47.357.727 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
11.47.357.731 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
11.47.357.731 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
11.52.045.616 I slot print_timing: id 3 | task 14744 | n_decoded = 100, tg = 23.29 t/s, tg_3s = 23.29 t/s
|
|
11.55.079.328 I slot print_timing: id 3 | task 14744 | n_decoded = 170, tg = 23.20 t/s, tg_3s = 23.07 t/s
|
|
11.58.092.159 I slot print_timing: id 3 | task 14744 | n_decoded = 239, tg = 23.12 t/s, tg_3s = 22.90 t/s
|
|
12.01.094.149 I slot print_timing: id 3 | task 14744 | n_decoded = 308, tg = 23.09 t/s, tg_3s = 22.98 t/s
|
|
12.01.224.777 I slot print_timing: id 3 | task 14744 | prompt eval time = 394.89 ms / 16 tokens ( 24.68 ms per token, 40.52 tokens per second)
|
|
12.01.224.780 I slot print_timing: id 3 | task 14744 | eval time = 13472.14 ms / 311 tokens ( 43.32 ms per token, 23.08 tokens per second)
|
|
12.01.224.781 I slot print_timing: id 3 | task 14744 | total time = 13867.03 ms / 327 tokens
|
|
12.01.224.782 I slot print_timing: id 3 | task 14744 | graphs reused = 14908
|
|
12.01.224.801 I slot release: id 3 | task 14744 | stop processing: n_tokens = 375, truncated = 0
|
|
12.01.224.810 I srv update_slots: all slots are idle
|
|
12.01.225.658 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
12.01.243.335 I srv operator(): Chat format: peg-native
|
|
12.01.248.846 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.790 (> 0.100 thold), f_keep = 0.131
|
|
12.01.248.848 I srv get_availabl: updating prompt cache
|
|
12.01.248.879 W srv prompt_save: - saving prompt with length 375, total state size = 90.826 MiB (draft: 0.000 MiB)
|
|
12.01.317.674 I srv load: - looking for better prompt, base f_keep = 0.131, sim = 0.790
|
|
12.01.317.693 I srv update: - cache state: 29 prompts, 3983.246 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
12.01.317.694 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
12.01.317.694 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
12.01.317.695 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
12.01.317.696 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
12.01.317.696 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
12.01.317.697 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
12.01.317.698 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
12.01.317.699 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
12.01.317.700 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
12.01.317.700 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
12.01.317.701 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
12.01.317.701 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
12.01.317.702 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
12.01.317.702 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
12.01.317.703 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
12.01.317.704 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
12.01.317.704 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
12.01.317.705 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
12.01.317.705 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
12.01.317.706 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
12.01.317.706 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
12.01.317.707 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
12.01.317.707 I srv update: - prompt 0x6037ffb8e900: 1268 tokens, checkpoints: 0, 307.110 MiB
|
|
12.01.317.708 I srv update: - prompt 0x6037ffff8ec0: 1279 tokens, checkpoints: 0, 309.774 MiB
|
|
12.01.317.709 I srv update: - prompt 0x6037ffb5be30: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
12.01.317.709 I srv update: - prompt 0x60380004bc50: 539 tokens, checkpoints: 0, 130.547 MiB
|
|
12.01.317.710 I srv update: - prompt 0x6037fffc7250: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
12.01.317.710 I srv update: - prompt 0x6037fff13e70: 251 tokens, checkpoints: 0, 60.793 MiB
|
|
12.01.317.711 I srv update: - prompt 0x603807489630: 375 tokens, checkpoints: 0, 90.826 MiB
|
|
12.01.317.713 I srv get_availabl: prompt cache update took 68.86 ms
|
|
12.01.319.744 I slot launch_slot_: id 3 | task 15056 | processing task, is_child = 0
|
|
12.01.319.747 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
12.01.319.748 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
12.01.319.757 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
12.01.319.758 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
12.01.319.765 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
12.01.319.765 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
12.06.016.863 I slot print_timing: id 3 | task 15056 | n_decoded = 100, tg = 23.00 t/s, tg_3s = 23.00 t/s
|
|
12.06.798.365 I slot print_timing: id 3 | task 15056 | prompt eval time = 348.67 ms / 13 tokens ( 26.82 ms per token, 37.28 tokens per second)
|
|
12.06.798.369 I slot print_timing: id 3 | task 15056 | eval time = 5129.91 ms / 118 tokens ( 43.47 ms per token, 23.00 tokens per second)
|
|
12.06.798.369 I slot print_timing: id 3 | task 15056 | total time = 5478.58 ms / 131 tokens
|
|
12.06.798.370 I slot print_timing: id 3 | task 15056 | graphs reused = 15024
|
|
12.06.798.390 I slot release: id 3 | task 15056 | stop processing: n_tokens = 179, truncated = 0
|
|
12.06.798.399 I srv update_slots: all slots are idle
|
|
12.06.799.191 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
12.06.818.499 I srv operator(): Chat format: peg-native
|
|
12.06.824.522 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.274
|
|
12.06.824.524 I srv get_availabl: updating prompt cache
|
|
12.06.824.550 W srv prompt_save: - saving prompt with length 179, total state size = 43.355 MiB (draft: 0.000 MiB)
|
|
12.06.857.563 I srv load: - looking for better prompt, base f_keep = 0.274, sim = 0.721
|
|
12.06.857.579 I srv update: - cache state: 30 prompts, 4026.601 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
12.06.857.580 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
12.06.857.581 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
12.06.857.581 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
12.06.857.582 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
12.06.857.583 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
12.06.857.583 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
12.06.857.584 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
12.06.857.584 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
12.06.857.586 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
12.06.857.587 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
12.06.857.587 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
12.06.857.588 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
12.06.857.588 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
12.06.857.589 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
12.06.857.589 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
12.06.857.590 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
12.06.857.590 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
12.06.857.591 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
12.06.857.591 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
12.06.857.591 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
12.06.857.592 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
12.06.857.592 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
12.06.857.593 I srv update: - prompt 0x6037ffb8e900: 1268 tokens, checkpoints: 0, 307.110 MiB
|
|
12.06.857.593 I srv update: - prompt 0x6037ffff8ec0: 1279 tokens, checkpoints: 0, 309.774 MiB
|
|
12.06.857.594 I srv update: - prompt 0x6037ffb5be30: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
12.06.857.594 I srv update: - prompt 0x60380004bc50: 539 tokens, checkpoints: 0, 130.547 MiB
|
|
12.06.857.595 I srv update: - prompt 0x6037fffc7250: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
12.06.857.596 I srv update: - prompt 0x6037fff13e70: 251 tokens, checkpoints: 0, 60.793 MiB
|
|
12.06.857.596 I srv update: - prompt 0x603807489630: 375 tokens, checkpoints: 0, 90.826 MiB
|
|
12.06.857.597 I srv update: - prompt 0x6037fff27c00: 179 tokens, checkpoints: 0, 43.355 MiB
|
|
12.06.857.598 I srv get_availabl: prompt cache update took 33.07 ms
|
|
12.06.859.211 I slot launch_slot_: id 3 | task 15175 | processing task, is_child = 0
|
|
12.06.859.214 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
12.06.859.215 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
12.06.859.221 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
12.06.859.222 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
12.06.859.226 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
12.06.859.226 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
12.11.736.295 I slot print_timing: id 3 | task 15175 | n_decoded = 100, tg = 23.22 t/s, tg_3s = 23.22 t/s
|
|
12.14.773.087 I slot print_timing: id 3 | task 15175 | n_decoded = 170, tg = 23.15 t/s, tg_3s = 23.05 t/s
|
|
12.17.796.844 I slot print_timing: id 3 | task 15175 | n_decoded = 238, tg = 22.96 t/s, tg_3s = 22.49 t/s
|
|
12.20.834.589 I slot print_timing: id 3 | task 15175 | n_decoded = 308, tg = 22.98 t/s, tg_3s = 23.04 t/s
|
|
12.23.871.564 I slot print_timing: id 3 | task 15175 | n_decoded = 378, tg = 22.99 t/s, tg_3s = 23.05 t/s
|
|
12.26.915.521 I slot print_timing: id 3 | task 15175 | n_decoded = 447, tg = 22.94 t/s, tg_3s = 22.67 t/s
|
|
12.29.956.913 I slot print_timing: id 3 | task 15175 | n_decoded = 516, tg = 22.91 t/s, tg_3s = 22.69 t/s
|
|
12.32.975.596 I slot print_timing: id 3 | task 15175 | n_decoded = 585, tg = 22.90 t/s, tg_3s = 22.86 t/s
|
|
12.35.991.182 I slot print_timing: id 3 | task 15175 | n_decoded = 654, tg = 22.90 t/s, tg_3s = 22.88 t/s
|
|
12.39.029.434 I slot print_timing: id 3 | task 15175 | n_decoded = 723, tg = 22.88 t/s, tg_3s = 22.71 t/s
|
|
12.42.038.330 I slot print_timing: id 3 | task 15175 | n_decoded = 791, tg = 22.86 t/s, tg_3s = 22.60 t/s
|
|
12.45.052.281 I slot print_timing: id 3 | task 15175 | n_decoded = 859, tg = 22.83 t/s, tg_3s = 22.56 t/s
|
|
12.48.064.340 I slot print_timing: id 3 | task 15175 | n_decoded = 927, tg = 22.81 t/s, tg_3s = 22.58 t/s
|
|
12.51.087.405 I slot print_timing: id 3 | task 15175 | n_decoded = 994, tg = 22.77 t/s, tg_3s = 22.16 t/s
|
|
12.54.114.618 I slot print_timing: id 3 | task 15175 | n_decoded = 1062, tg = 22.75 t/s, tg_3s = 22.46 t/s
|
|
12.57.132.496 I slot print_timing: id 3 | task 15175 | n_decoded = 1130, tg = 22.74 t/s, tg_3s = 22.53 t/s
|
|
13.00.137.340 I slot print_timing: id 3 | task 15175 | n_decoded = 1198, tg = 22.73 t/s, tg_3s = 22.63 t/s
|
|
13.00.225.621 I slot print_timing: id 3 | task 15175 | prompt eval time = 570.60 ms / 19 tokens ( 30.03 ms per token, 33.30 tokens per second)
|
|
13.00.225.623 I slot print_timing: id 3 | task 15175 | eval time = 52795.78 ms / 1200 tokens ( 44.00 ms per token, 22.73 tokens per second)
|
|
13.00.225.624 I slot print_timing: id 3 | task 15175 | total time = 53366.38 ms / 1219 tokens
|
|
13.00.225.625 I slot print_timing: id 3 | task 15175 | graphs reused = 16218
|
|
13.00.225.649 I slot release: id 3 | task 15175 | stop processing: n_tokens = 1267, truncated = 0
|
|
13.00.225.665 I srv update_slots: all slots are idle
|
|
13.00.226.465 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
13.00.243.424 I srv operator(): Chat format: peg-native
|
|
13.00.248.957 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.039
|
|
13.00.248.959 I srv get_availabl: updating prompt cache
|
|
13.00.249.019 W srv prompt_save: - saving prompt with length 1267, total state size = 306.868 MiB (draft: 0.000 MiB)
|
|
13.00.462.331 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.742
|
|
13.00.462.351 I srv update: - cache state: 31 prompts, 4333.468 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
13.00.462.352 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
13.00.462.352 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
13.00.462.353 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
13.00.462.354 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
13.00.462.354 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
13.00.462.355 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
13.00.462.356 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
13.00.462.362 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
13.00.462.363 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
13.00.462.363 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
13.00.462.364 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
13.00.462.364 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
13.00.462.365 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
13.00.462.365 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
13.00.462.366 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
13.00.462.367 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
13.00.462.370 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
13.00.462.371 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
13.00.462.371 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
13.00.462.372 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
13.00.462.372 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
13.00.462.373 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
13.00.462.374 I srv update: - prompt 0x6037ffb8e900: 1268 tokens, checkpoints: 0, 307.110 MiB
|
|
13.00.462.374 I srv update: - prompt 0x6037ffff8ec0: 1279 tokens, checkpoints: 0, 309.774 MiB
|
|
13.00.462.375 I srv update: - prompt 0x6037ffb5be30: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
13.00.462.375 I srv update: - prompt 0x60380004bc50: 539 tokens, checkpoints: 0, 130.547 MiB
|
|
13.00.462.376 I srv update: - prompt 0x6037fffc7250: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
13.00.462.376 I srv update: - prompt 0x6037fff13e70: 251 tokens, checkpoints: 0, 60.793 MiB
|
|
13.00.462.377 I srv update: - prompt 0x603807489630: 375 tokens, checkpoints: 0, 90.826 MiB
|
|
13.00.462.378 I srv update: - prompt 0x6037fff27c00: 179 tokens, checkpoints: 0, 43.355 MiB
|
|
13.00.462.379 I srv update: - prompt 0x603807265a40: 1267 tokens, checkpoints: 0, 306.868 MiB
|
|
13.00.462.381 I srv get_availabl: prompt cache update took 213.42 ms
|
|
13.00.464.375 I slot launch_slot_: id 3 | task 16376 | processing task, is_child = 0
|
|
13.00.464.378 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
13.00.464.379 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
13.00.464.388 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
13.00.464.389 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
13.00.464.394 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
13.00.464.395 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
13.05.328.138 I slot print_timing: id 3 | task 16376 | n_decoded = 100, tg = 23.25 t/s, tg_3s = 23.25 t/s
|
|
13.08.349.191 I slot print_timing: id 3 | task 16376 | n_decoded = 170, tg = 23.22 t/s, tg_3s = 23.17 t/s
|
|
13.10.911.224 I slot print_timing: id 3 | task 16376 | prompt eval time = 563.32 ms / 17 tokens ( 33.14 ms per token, 30.18 tokens per second)
|
|
13.10.911.227 I slot print_timing: id 3 | task 16376 | eval time = 9883.49 ms / 229 tokens ( 43.16 ms per token, 23.17 tokens per second)
|
|
13.10.911.228 I slot print_timing: id 3 | task 16376 | total time = 10446.81 ms / 246 tokens
|
|
13.10.911.229 I slot print_timing: id 3 | task 16376 | graphs reused = 16444
|
|
13.10.911.247 I slot release: id 3 | task 16376 | stop processing: n_tokens = 294, truncated = 0
|
|
13.10.911.256 I srv update_slots: all slots are idle
|
|
13.10.912.010 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
13.10.931.293 I srv operator(): Chat format: peg-native
|
|
13.10.937.277 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.781 (> 0.100 thold), f_keep = 0.170
|
|
13.10.937.279 I srv get_availabl: updating prompt cache
|
|
13.10.937.310 W srv prompt_save: - saving prompt with length 294, total state size = 71.208 MiB (draft: 0.000 MiB)
|
|
13.10.988.917 I srv load: - looking for better prompt, base f_keep = 0.170, sim = 0.781
|
|
13.10.988.936 I srv update: - cache state: 32 prompts, 4404.676 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
13.10.988.937 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
13.10.988.938 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
13.10.988.938 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
13.10.988.939 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
13.10.988.939 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
13.10.988.940 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
13.10.988.940 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
13.10.988.941 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
13.10.988.941 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
13.10.988.942 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
13.10.988.943 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
13.10.988.946 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
13.10.988.947 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
13.10.988.947 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
13.10.988.947 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
13.10.988.948 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
13.10.988.948 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
13.10.988.949 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
13.10.988.949 I srv update: - prompt 0x6037fff4b910: 197 tokens, checkpoints: 0, 47.715 MiB
|
|
13.10.988.950 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
13.10.988.950 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
13.10.988.950 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
13.10.988.951 I srv update: - prompt 0x6037ffb8e900: 1268 tokens, checkpoints: 0, 307.110 MiB
|
|
13.10.988.952 I srv update: - prompt 0x6037ffff8ec0: 1279 tokens, checkpoints: 0, 309.774 MiB
|
|
13.10.988.953 I srv update: - prompt 0x6037ffb5be30: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
13.10.988.954 I srv update: - prompt 0x60380004bc50: 539 tokens, checkpoints: 0, 130.547 MiB
|
|
13.10.988.954 I srv update: - prompt 0x6037fffc7250: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
13.10.988.954 I srv update: - prompt 0x6037fff13e70: 251 tokens, checkpoints: 0, 60.793 MiB
|
|
13.10.988.955 I srv update: - prompt 0x603807489630: 375 tokens, checkpoints: 0, 90.826 MiB
|
|
13.10.988.955 I srv update: - prompt 0x6037fff27c00: 179 tokens, checkpoints: 0, 43.355 MiB
|
|
13.10.988.956 I srv update: - prompt 0x603807265a40: 1267 tokens, checkpoints: 0, 306.868 MiB
|
|
13.10.988.956 I srv update: - prompt 0x6037ffbdce70: 294 tokens, checkpoints: 0, 71.208 MiB
|
|
13.10.988.957 I srv get_availabl: prompt cache update took 51.68 ms
|
|
13.10.990.612 I slot launch_slot_: id 3 | task 16606 | processing task, is_child = 0
|
|
13.10.990.614 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
13.10.990.615 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
13.10.990.625 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
13.10.990.625 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
13.10.990.634 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
13.10.990.635 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
13.15.649.159 I slot print_timing: id 3 | task 16606 | n_decoded = 100, tg = 23.28 t/s, tg_3s = 23.28 t/s
|
|
13.18.675.411 I slot print_timing: id 3 | task 16606 | n_decoded = 170, tg = 23.22 t/s, tg_3s = 23.13 t/s
|
|
13.21.679.573 I slot print_timing: id 3 | task 16606 | n_decoded = 239, tg = 23.15 t/s, tg_3s = 22.97 t/s
|
|
13.24.721.568 I slot print_timing: id 3 | task 16606 | n_decoded = 309, tg = 23.11 t/s, tg_3s = 23.01 t/s
|
|
13.27.734.141 I slot print_timing: id 3 | task 16606 | n_decoded = 378, tg = 23.08 t/s, tg_3s = 22.90 t/s
|
|
13.30.736.095 I slot print_timing: id 3 | task 16606 | n_decoded = 447, tg = 23.06 t/s, tg_3s = 22.99 t/s
|
|
13.33.738.183 I slot print_timing: id 3 | task 16606 | n_decoded = 515, tg = 23.01 t/s, tg_3s = 22.65 t/s
|
|
13.36.755.192 I slot print_timing: id 3 | task 16606 | n_decoded = 584, tg = 22.99 t/s, tg_3s = 22.87 t/s
|
|
13.39.769.894 I slot print_timing: id 3 | task 16606 | n_decoded = 653, tg = 22.98 t/s, tg_3s = 22.89 t/s
|
|
13.42.808.396 I slot print_timing: id 3 | task 16606 | n_decoded = 722, tg = 22.95 t/s, tg_3s = 22.71 t/s
|
|
13.45.852.119 I slot print_timing: id 3 | task 16606 | n_decoded = 791, tg = 22.93 t/s, tg_3s = 22.67 t/s
|
|
13.48.893.164 I slot print_timing: id 3 | task 16606 | n_decoded = 860, tg = 22.91 t/s, tg_3s = 22.69 t/s
|
|
13.51.899.757 I slot print_timing: id 3 | task 16606 | n_decoded = 928, tg = 22.89 t/s, tg_3s = 22.62 t/s
|
|
13.54.923.922 I slot print_timing: id 3 | task 16606 | n_decoded = 996, tg = 22.86 t/s, tg_3s = 22.49 t/s
|
|
13.57.933.131 I slot print_timing: id 3 | task 16606 | n_decoded = 1064, tg = 22.84 t/s, tg_3s = 22.60 t/s
|
|
14.00.942.860 I slot print_timing: id 3 | task 16606 | n_decoded = 1132, tg = 22.83 t/s, tg_3s = 22.59 t/s
|
|
14.03.954.577 I slot print_timing: id 3 | task 16606 | prompt eval time = 362.94 ms / 14 tokens ( 25.92 ms per token, 38.57 tokens per second)
|
|
14.03.954.580 I slot print_timing: id 3 | task 16606 | eval time = 52600.98 ms / 1200 tokens ( 43.83 ms per token, 22.81 tokens per second)
|
|
14.03.954.581 I slot print_timing: id 3 | task 16606 | total time = 52963.92 ms / 1214 tokens
|
|
14.03.954.582 I slot print_timing: id 3 | task 16606 | graphs reused = 17638
|
|
14.03.954.600 I slot release: id 3 | task 16606 | stop processing: n_tokens = 1263, truncated = 0
|
|
14.03.954.613 I srv update_slots: all slots are idle
|
|
14.03.955.621 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
14.03.974.724 I srv operator(): Chat format: peg-native
|
|
14.03.980.792 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.039
|
|
14.03.980.794 I srv get_availabl: updating prompt cache
|
|
14.03.980.848 W srv prompt_save: - saving prompt with length 1263, total state size = 305.899 MiB (draft: 0.000 MiB)
|
|
14.04.177.632 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.721
|
|
14.04.177.645 I srv load: - found better prompt with f_keep = 0.254, sim = 0.735
|
|
14.04.198.374 I srv update: - cache state: 32 prompts, 4662.860 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
14.04.198.378 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
14.04.198.379 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
14.04.198.380 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
14.04.198.381 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
14.04.198.381 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
14.04.198.382 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
14.04.198.383 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
14.04.198.383 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
14.04.198.384 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
14.04.198.384 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
14.04.198.385 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
14.04.198.385 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
14.04.198.387 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
14.04.198.387 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
14.04.198.388 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
14.04.198.388 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
14.04.198.389 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
14.04.198.389 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
14.04.198.390 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
14.04.198.390 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
14.04.198.390 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
14.04.198.391 I srv update: - prompt 0x6037ffb8e900: 1268 tokens, checkpoints: 0, 307.110 MiB
|
|
14.04.198.391 I srv update: - prompt 0x6037ffff8ec0: 1279 tokens, checkpoints: 0, 309.774 MiB
|
|
14.04.198.392 I srv update: - prompt 0x6037ffb5be30: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
14.04.198.392 I srv update: - prompt 0x60380004bc50: 539 tokens, checkpoints: 0, 130.547 MiB
|
|
14.04.198.393 I srv update: - prompt 0x6037fffc7250: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
14.04.198.393 I srv update: - prompt 0x6037fff13e70: 251 tokens, checkpoints: 0, 60.793 MiB
|
|
14.04.198.394 I srv update: - prompt 0x603807489630: 375 tokens, checkpoints: 0, 90.826 MiB
|
|
14.04.198.394 I srv update: - prompt 0x6037fff27c00: 179 tokens, checkpoints: 0, 43.355 MiB
|
|
14.04.198.395 I srv update: - prompt 0x603807265a40: 1267 tokens, checkpoints: 0, 306.868 MiB
|
|
14.04.198.395 I srv update: - prompt 0x6037ffbdce70: 294 tokens, checkpoints: 0, 71.208 MiB
|
|
14.04.198.395 I srv update: - prompt 0x6037ffc14eb0: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
14.04.198.397 I srv get_availabl: prompt cache update took 217.60 ms
|
|
14.04.200.008 I slot launch_slot_: id 3 | task 17807 | processing task, is_child = 0
|
|
14.04.200.010 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
14.04.200.011 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
14.04.200.018 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
14.04.200.018 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
14.04.200.023 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
14.04.200.023 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
14.09.051.141 I slot print_timing: id 3 | task 17807 | n_decoded = 100, tg = 23.26 t/s, tg_3s = 23.26 t/s
|
|
14.12.083.504 I slot print_timing: id 3 | task 17807 | n_decoded = 170, tg = 23.18 t/s, tg_3s = 23.08 t/s
|
|
14.15.094.934 I slot print_timing: id 3 | task 17807 | n_decoded = 239, tg = 23.11 t/s, tg_3s = 22.91 t/s
|
|
14.18.097.620 I slot print_timing: id 3 | task 17807 | n_decoded = 308, tg = 23.08 t/s, tg_3s = 22.98 t/s
|
|
14.21.099.625 I slot print_timing: id 3 | task 17807 | n_decoded = 377, tg = 23.06 t/s, tg_3s = 22.98 t/s
|
|
14.24.116.786 I slot print_timing: id 3 | task 17807 | n_decoded = 446, tg = 23.03 t/s, tg_3s = 22.87 t/s
|
|
14.27.143.582 I slot print_timing: id 3 | task 17807 | n_decoded = 515, tg = 23.00 t/s, tg_3s = 22.80 t/s
|
|
14.30.174.828 I slot print_timing: id 3 | task 17807 | n_decoded = 584, tg = 22.97 t/s, tg_3s = 22.76 t/s
|
|
14.33.194.883 I slot print_timing: id 3 | task 17807 | n_decoded = 653, tg = 22.96 t/s, tg_3s = 22.85 t/s
|
|
14.36.232.013 I slot print_timing: id 3 | task 17807 | n_decoded = 722, tg = 22.93 t/s, tg_3s = 22.72 t/s
|
|
14.39.233.374 I slot print_timing: id 3 | task 17807 | n_decoded = 790, tg = 22.91 t/s, tg_3s = 22.66 t/s
|
|
14.42.235.875 I slot print_timing: id 3 | task 17807 | n_decoded = 858, tg = 22.89 t/s, tg_3s = 22.65 t/s
|
|
14.45.239.651 I slot print_timing: id 3 | task 17807 | n_decoded = 926, tg = 22.87 t/s, tg_3s = 22.64 t/s
|
|
14.48.260.331 I slot print_timing: id 3 | task 17807 | n_decoded = 994, tg = 22.85 t/s, tg_3s = 22.51 t/s
|
|
14.51.279.980 I slot print_timing: id 3 | task 17807 | n_decoded = 1062, tg = 22.82 t/s, tg_3s = 22.52 t/s
|
|
14.54.291.632 I slot print_timing: id 3 | task 17807 | n_decoded = 1130, tg = 22.81 t/s, tg_3s = 22.58 t/s
|
|
14.57.304.640 I slot print_timing: id 3 | task 17807 | n_decoded = 1198, tg = 22.80 t/s, tg_3s = 22.57 t/s
|
|
14.57.393.297 I slot print_timing: id 3 | task 17807 | prompt eval time = 551.13 ms / 18 tokens ( 30.62 ms per token, 32.66 tokens per second)
|
|
14.57.393.299 I slot print_timing: id 3 | task 17807 | eval time = 52642.13 ms / 1200 tokens ( 43.87 ms per token, 22.80 tokens per second)
|
|
14.57.393.300 I slot print_timing: id 3 | task 17807 | total time = 53193.26 ms / 1218 tokens
|
|
14.57.393.301 I slot print_timing: id 3 | task 17807 | graphs reused = 18832
|
|
14.57.393.320 I slot release: id 3 | task 17807 | stop processing: n_tokens = 1267, truncated = 0
|
|
14.57.393.332 I srv update_slots: all slots are idle
|
|
14.57.394.254 W render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
|
|
14.57.411.825 I srv operator(): Chat format: peg-native
|
|
14.57.417.369 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.766 (> 0.100 thold), f_keep = 0.039
|
|
14.57.417.371 I srv get_availabl: updating prompt cache
|
|
14.57.417.422 W srv prompt_save: - saving prompt with length 1267, total state size = 306.868 MiB (draft: 0.000 MiB)
|
|
14.57.613.692 I srv load: - looking for better prompt, base f_keep = 0.039, sim = 0.766
|
|
14.57.613.712 I srv update: - cache state: 33 prompts, 4969.728 MiB (limits: 8192.000 MiB, 8192 tokens, 33823 est)
|
|
14.57.613.713 I srv update: - prompt 0x6038070bf530: 588 tokens, checkpoints: 0, 142.414 MiB
|
|
14.57.613.714 I srv update: - prompt 0x6038000e2b60: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
14.57.613.715 I srv update: - prompt 0x6037fff16af0: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
14.57.613.715 I srv update: - prompt 0x6038001611b0: 1266 tokens, checkpoints: 0, 306.625 MiB
|
|
14.57.613.716 I srv update: - prompt 0x60380704c5b0: 427 tokens, checkpoints: 0, 103.420 MiB
|
|
14.57.613.717 I srv update: - prompt 0x6038000e2a70: 345 tokens, checkpoints: 0, 83.560 MiB
|
|
14.57.613.718 I srv update: - prompt 0x60380702f610: 1272 tokens, checkpoints: 0, 308.078 MiB
|
|
14.57.613.719 I srv update: - prompt 0x6037fff18450: 448 tokens, checkpoints: 0, 108.507 MiB
|
|
14.57.613.719 I srv update: - prompt 0x6037fef11ec0: 450 tokens, checkpoints: 0, 108.991 MiB
|
|
14.57.613.720 I srv update: - prompt 0x6037ff09b7a0: 421 tokens, checkpoints: 0, 101.967 MiB
|
|
14.57.613.721 I srv update: - prompt 0x6037fff0ed40: 183 tokens, checkpoints: 0, 44.324 MiB
|
|
14.57.613.722 I srv update: - prompt 0x603800177040: 315 tokens, checkpoints: 0, 76.294 MiB
|
|
14.57.613.722 I srv update: - prompt 0x603800178540: 267 tokens, checkpoints: 0, 64.669 MiB
|
|
14.57.613.723 I srv update: - prompt 0x603800068ce0: 371 tokens, checkpoints: 0, 89.857 MiB
|
|
14.57.613.724 I srv update: - prompt 0x6037fff04c70: 331 tokens, checkpoints: 0, 80.169 MiB
|
|
14.57.613.724 I srv update: - prompt 0x6037fff01ef0: 437 tokens, checkpoints: 0, 105.842 MiB
|
|
14.57.613.725 I srv update: - prompt 0x6037ffdf6f70: 399 tokens, checkpoints: 0, 96.639 MiB
|
|
14.57.613.726 I srv update: - prompt 0x60380705ce50: 301 tokens, checkpoints: 0, 72.903 MiB
|
|
14.57.613.726 I srv update: - prompt 0x6037ffb84980: 241 tokens, checkpoints: 0, 58.371 MiB
|
|
14.57.613.727 I srv update: - prompt 0x6037fff29560: 188 tokens, checkpoints: 0, 45.535 MiB
|
|
14.57.613.728 I srv update: - prompt 0x603807489740: 217 tokens, checkpoints: 0, 52.559 MiB
|
|
14.57.613.729 I srv update: - prompt 0x6037ffb8e900: 1268 tokens, checkpoints: 0, 307.110 MiB
|
|
14.57.613.729 I srv update: - prompt 0x6037ffff8ec0: 1279 tokens, checkpoints: 0, 309.774 MiB
|
|
14.57.613.730 I srv update: - prompt 0x6037ffb5be30: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
14.57.613.731 I srv update: - prompt 0x60380004bc50: 539 tokens, checkpoints: 0, 130.547 MiB
|
|
14.57.613.732 I srv update: - prompt 0x6037fffc7250: 1270 tokens, checkpoints: 0, 307.594 MiB
|
|
14.57.613.732 I srv update: - prompt 0x6037fff13e70: 251 tokens, checkpoints: 0, 60.793 MiB
|
|
14.57.613.733 I srv update: - prompt 0x603807489630: 375 tokens, checkpoints: 0, 90.826 MiB
|
|
14.57.613.734 I srv update: - prompt 0x6037fff27c00: 179 tokens, checkpoints: 0, 43.355 MiB
|
|
14.57.613.735 I srv update: - prompt 0x603807265a40: 1267 tokens, checkpoints: 0, 306.868 MiB
|
|
14.57.613.736 I srv update: - prompt 0x6037ffbdce70: 294 tokens, checkpoints: 0, 71.208 MiB
|
|
14.57.613.736 I srv update: - prompt 0x6037ffc14eb0: 1263 tokens, checkpoints: 0, 305.899 MiB
|
|
14.57.613.737 I srv update: - prompt 0x6037ffbc0f80: 1267 tokens, checkpoints: 0, 306.868 MiB
|
|
14.57.613.738 I srv get_availabl: prompt cache update took 196.37 ms
|
|
14.57.615.839 I slot launch_slot_: id 3 | task 19008 | processing task, is_child = 0
|
|
14.57.615.842 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
14.57.615.843 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
14.57.615.853 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
14.57.615.854 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
14.57.615.861 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
14.57.615.862 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
15.02.317.494 I slot print_timing: id 3 | task 19008 | n_decoded = 100, tg = 23.16 t/s, tg_3s = 23.16 t/s
|
|
15.05.360.262 I slot print_timing: id 3 | task 19008 | n_decoded = 169, tg = 22.96 t/s, tg_3s = 22.68 t/s
|
|
15.08.365.621 I slot print_timing: id 3 | task 19008 | n_decoded = 238, tg = 22.96 t/s, tg_3s = 22.96 t/s
|
|
15.11.408.576 I slot print_timing: id 3 | task 19008 | n_decoded = 308, tg = 22.97 t/s, tg_3s = 23.00 t/s
|
|
15.14.410.553 I slot print_timing: id 3 | task 19008 | n_decoded = 377, tg = 22.97 t/s, tg_3s = 22.98 t/s
|
|
15.17.421.541 I slot print_timing: id 3 | task 19008 | n_decoded = 446, tg = 22.96 t/s, tg_3s = 22.92 t/s
|
|
15.20.460.676 I slot print_timing: id 3 | task 19008 | n_decoded = 515, tg = 22.93 t/s, tg_3s = 22.70 t/s
|
|
15.23.491.947 I slot print_timing: id 3 | task 19008 | n_decoded = 584, tg = 22.91 t/s, tg_3s = 22.76 t/s
|
|
15.24.668.154 I slot print_timing: id 3 | task 19008 | prompt eval time = 383.22 ms / 15 tokens ( 25.55 ms per token, 39.14 tokens per second)
|
|
15.24.668.159 I slot print_timing: id 3 | task 19008 | eval time = 26669.05 ms / 611 tokens ( 43.65 ms per token, 22.91 tokens per second)
|
|
15.24.668.160 I slot print_timing: id 3 | task 19008 | total time = 27052.27 ms / 626 tokens
|
|
15.24.668.161 I slot print_timing: id 3 | task 19008 | graphs reused = 19439
|
|
15.24.668.188 I slot release: id 3 | task 19008 | stop processing: n_tokens = 674, truncated = 0
|
|
15.24.668.200 I srv update_slots: all slots are idle
|
|
15.24.668.854 I srv operator(): operator(): cleaning up before exit...
|