1186 lines
114 KiB
Plaintext
1186 lines
114 KiB
Plaintext
0.00.071.608 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
|
|
0.00.071.611 I device_info:
|
|
0.00.071.630 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
|
0.00.071.635 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
|
0.00.071.643 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
|
|
0.00.071.701 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
|
0.00.071.705 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
|
|
0.00.071.858 I srv init: running without SSL
|
|
0.00.071.920 I srv init: using 23 threads for HTTP server
|
|
0.00.072.410 I srv start: binding port with default address family
|
|
0.00.073.615 I srv llama_server: loading model
|
|
0.00.073.622 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/Devstral-2-123B-Instruct-2512-UD-IQ2_M.gguf'
|
|
0.00.073.672 I common_init_result: fitting params to device memory ...
|
|
0.00.073.673 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
|
|
0.21.107.248 W llama_context: n_ctx_seq (8192) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
|
|
0.21.287.550 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
|
0.21.723.403 I srv load_model: initializing slots, n_slots = 4
|
|
0.22.079.024 W common_speculative_init: no implementations specified for speculative decoding
|
|
0.22.079.028 I slot load_model: id 0 | task -1 | new slot, n_ctx = 8192
|
|
0.22.079.032 I slot load_model: id 1 | task -1 | new slot, n_ctx = 8192
|
|
0.22.079.033 I slot load_model: id 2 | task -1 | new slot, n_ctx = 8192
|
|
0.22.079.033 I slot load_model: id 3 | task -1 | new slot, n_ctx = 8192
|
|
0.22.079.075 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
|
|
0.22.079.076 I srv load_model: use `--cache-ram 0` to disable the prompt cache
|
|
0.22.079.076 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
|
0.22.079.076 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
|
|
0.22.079.095 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
|
|
0.22.085.388 I init: chat template, example_format: '[SYSTEM_PROMPT]You are a helpful assistant[/SYSTEM_PROMPT][INST]Hello[/INST]Hi there</s>[INST]How are you?[/INST]'
|
|
0.22.086.068 I srv init: init: chat template, thinking = 1
|
|
0.22.086.193 I srv llama_server: model loaded
|
|
0.22.086.197 I srv llama_server: server is listening on http://127.0.0.1:8090
|
|
0.22.086.202 I srv update_slots: all slots are idle
|
|
0.24.972.871 I srv operator(): Chat format: peg-native
|
|
0.24.973.058 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
|
|
0.24.973.062 I srv get_availabl: updating prompt cache
|
|
0.24.973.069 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
|
0.24.973.074 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
|
|
0.24.973.074 I srv get_availabl: prompt cache update took 0.01 ms
|
|
0.24.973.104 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
|
|
0.24.973.104 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.24.973.106 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.24.973.114 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.24.973.115 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.24.973.119 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.24.973.120 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.55.257.389 I slot print_timing: id 3 | task 0 | prompt eval time = 1950.56 ms / 64 tokens ( 30.48 ms per token, 32.81 tokens per second)
|
|
0.55.257.393 I slot print_timing: id 3 | task 0 | eval time = 28333.68 ms / 97 tokens ( 292.10 ms per token, 3.42 tokens per second)
|
|
0.55.257.394 I slot print_timing: id 3 | task 0 | total time = 30284.24 ms / 161 tokens
|
|
0.55.257.397 I slot print_timing: id 3 | task 0 | graphs reused = 96
|
|
0.55.257.449 I slot release: id 3 | task 0 | stop processing: n_tokens = 160, truncated = 0
|
|
0.55.257.455 I srv update_slots: all slots are idle
|
|
0.55.261.387 I srv operator(): Chat format: peg-native
|
|
0.55.261.494 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.818 (> 0.100 thold), f_keep = 0.281
|
|
0.55.261.496 I srv get_availabl: updating prompt cache
|
|
0.55.261.526 W srv prompt_save: - saving prompt with length 160, total state size = 55.004 MiB (draft: 0.000 MiB)
|
|
0.55.306.541 I srv load: - looking for better prompt, base f_keep = 0.281, sim = 0.818
|
|
0.55.306.549 I srv update: - cache state: 1 prompts, 55.004 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
0.55.306.550 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
0.55.306.551 I srv get_availabl: prompt cache update took 45.05 ms
|
|
0.55.306.595 I slot launch_slot_: id 3 | task 98 | processing task, is_child = 0
|
|
0.55.306.596 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.55.306.597 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.55.306.606 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.55.306.607 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.55.306.611 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.55.306.611 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
1.20.771.125 I slot print_timing: id 3 | task 98 | prompt eval time = 931.65 ms / 10 tokens ( 93.17 ms per token, 10.73 tokens per second)
|
|
1.20.771.129 I slot print_timing: id 3 | task 98 | eval time = 24532.84 ms / 84 tokens ( 292.06 ms per token, 3.42 tokens per second)
|
|
1.20.771.130 I slot print_timing: id 3 | task 98 | total time = 25464.50 ms / 94 tokens
|
|
1.20.771.131 I slot print_timing: id 3 | task 98 | graphs reused = 178
|
|
1.20.771.156 I slot release: id 3 | task 98 | stop processing: n_tokens = 138, truncated = 0
|
|
1.20.771.164 I srv update_slots: all slots are idle
|
|
1.20.774.020 I srv operator(): Chat format: peg-native
|
|
1.20.774.128 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.326
|
|
1.20.774.129 I srv get_availabl: updating prompt cache
|
|
1.20.774.155 W srv prompt_save: - saving prompt with length 138, total state size = 47.441 MiB (draft: 0.000 MiB)
|
|
1.20.813.994 I srv load: - looking for better prompt, base f_keep = 0.326, sim = 0.776
|
|
1.20.814.002 I srv update: - cache state: 2 prompts, 102.445 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
1.20.814.003 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
1.20.814.004 I srv update: - prompt 0x5b0d9a042890: 138 tokens, checkpoints: 0, 47.441 MiB
|
|
1.20.814.005 I srv get_availabl: prompt cache update took 39.88 ms
|
|
1.20.814.051 I slot launch_slot_: id 3 | task 183 | processing task, is_child = 0
|
|
1.20.814.052 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.20.814.053 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.20.814.061 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.20.814.062 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.20.814.066 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
1.20.814.067 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
1.38.334.765 I slot print_timing: id 3 | task 183 | prompt eval time = 934.33 ms / 13 tokens ( 71.87 ms per token, 13.91 tokens per second)
|
|
1.38.334.769 I slot print_timing: id 3 | task 183 | eval time = 16586.35 ms / 57 tokens ( 290.99 ms per token, 3.44 tokens per second)
|
|
1.38.334.769 I slot print_timing: id 3 | task 183 | total time = 17520.68 ms / 70 tokens
|
|
1.38.334.770 I slot print_timing: id 3 | task 183 | graphs reused = 233
|
|
1.38.334.791 I slot release: id 3 | task 183 | stop processing: n_tokens = 114, truncated = 0
|
|
1.38.334.798 I srv update_slots: all slots are idle
|
|
1.38.337.574 I srv operator(): Chat format: peg-native
|
|
1.38.337.678 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.807 (> 0.100 thold), f_keep = 0.404
|
|
1.38.337.680 I srv get_availabl: updating prompt cache
|
|
1.38.337.707 W srv prompt_save: - saving prompt with length 114, total state size = 39.191 MiB (draft: 0.000 MiB)
|
|
1.38.371.232 I srv load: - looking for better prompt, base f_keep = 0.404, sim = 0.807
|
|
1.38.371.241 I srv update: - cache state: 3 prompts, 141.636 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
1.38.371.242 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
1.38.371.243 I srv update: - prompt 0x5b0d9a042890: 138 tokens, checkpoints: 0, 47.441 MiB
|
|
1.38.371.243 I srv update: - prompt 0x5b0d9a61aec0: 114 tokens, checkpoints: 0, 39.191 MiB
|
|
1.38.371.244 I srv get_availabl: prompt cache update took 33.56 ms
|
|
1.38.371.288 I slot launch_slot_: id 3 | task 241 | processing task, is_child = 0
|
|
1.38.371.289 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.38.371.290 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.38.371.298 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.38.371.298 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.38.371.303 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
1.38.371.303 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.01.821.309 I slot print_timing: id 3 | task 241 | prompt eval time = 933.34 ms / 11 tokens ( 84.85 ms per token, 11.79 tokens per second)
|
|
2.01.821.313 I slot print_timing: id 3 | task 241 | eval time = 22516.65 ms / 77 tokens ( 292.42 ms per token, 3.42 tokens per second)
|
|
2.01.821.314 I slot print_timing: id 3 | task 241 | total time = 23449.99 ms / 88 tokens
|
|
2.01.821.314 I slot print_timing: id 3 | task 241 | graphs reused = 308
|
|
2.01.821.346 I slot release: id 3 | task 241 | stop processing: n_tokens = 133, truncated = 0
|
|
2.01.821.352 I srv update_slots: all slots are idle
|
|
2.01.824.255 I srv operator(): Chat format: peg-native
|
|
2.01.824.361 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.780 (> 0.100 thold), f_keep = 0.346
|
|
2.01.824.362 I srv get_availabl: updating prompt cache
|
|
2.01.824.386 W srv prompt_save: - saving prompt with length 133, total state size = 45.722 MiB (draft: 0.000 MiB)
|
|
2.01.862.778 I srv load: - looking for better prompt, base f_keep = 0.346, sim = 0.780
|
|
2.01.862.783 I srv load: - found better prompt with f_keep = 0.421, sim = 0.814
|
|
2.01.882.682 I srv update: - cache state: 3 prompts, 148.167 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
2.01.882.686 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
2.01.882.686 I srv update: - prompt 0x5b0d9a042890: 138 tokens, checkpoints: 0, 47.441 MiB
|
|
2.01.882.687 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
2.01.882.688 I srv get_availabl: prompt cache update took 58.33 ms
|
|
2.01.882.722 I slot launch_slot_: id 3 | task 319 | processing task, is_child = 0
|
|
2.01.882.723 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.01.882.724 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.01.882.731 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.01.882.732 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.01.882.736 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.01.882.736 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.27.744.256 I slot print_timing: id 3 | task 319 | prompt eval time = 935.16 ms / 11 tokens ( 85.01 ms per token, 11.76 tokens per second)
|
|
2.27.744.259 I slot print_timing: id 3 | task 319 | eval time = 24926.34 ms / 85 tokens ( 293.25 ms per token, 3.41 tokens per second)
|
|
2.27.744.260 I slot print_timing: id 3 | task 319 | total time = 25861.50 ms / 96 tokens
|
|
2.27.744.261 I slot print_timing: id 3 | task 319 | graphs reused = 391
|
|
2.27.744.286 I slot release: id 3 | task 319 | stop processing: n_tokens = 143, truncated = 0
|
|
2.27.744.293 I srv update_slots: all slots are idle
|
|
2.27.747.167 I srv operator(): Chat format: peg-native
|
|
2.27.747.275 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.315
|
|
2.27.747.276 I srv get_availabl: updating prompt cache
|
|
2.27.747.301 W srv prompt_save: - saving prompt with length 143, total state size = 49.160 MiB (draft: 0.000 MiB)
|
|
2.27.787.892 I srv load: - looking for better prompt, base f_keep = 0.315, sim = 0.789
|
|
2.27.787.897 I srv load: - found better prompt with f_keep = 0.333, sim = 0.807
|
|
2.27.810.457 I srv update: - cache state: 3 prompts, 149.886 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
2.27.810.461 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
2.27.810.462 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
2.27.810.462 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
2.27.810.463 I srv get_availabl: prompt cache update took 63.19 ms
|
|
2.27.810.500 I slot launch_slot_: id 3 | task 405 | processing task, is_child = 0
|
|
2.27.810.501 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.27.810.502 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.27.810.509 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.27.810.510 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.27.810.514 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.27.810.514 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.40.010.015 I slot print_timing: id 3 | task 405 | prompt eval time = 934.77 ms / 11 tokens ( 84.98 ms per token, 11.77 tokens per second)
|
|
2.40.010.019 I slot print_timing: id 3 | task 405 | eval time = 11264.70 ms / 39 tokens ( 288.84 ms per token, 3.46 tokens per second)
|
|
2.40.010.020 I slot print_timing: id 3 | task 405 | total time = 12199.47 ms / 50 tokens
|
|
2.40.010.021 I slot print_timing: id 3 | task 405 | graphs reused = 428
|
|
2.40.010.051 I slot release: id 3 | task 405 | stop processing: n_tokens = 95, truncated = 0
|
|
2.40.010.066 I srv update_slots: all slots are idle
|
|
2.40.013.134 I srv operator(): Chat format: peg-native
|
|
2.40.013.264 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.474
|
|
2.40.013.265 I srv get_availabl: updating prompt cache
|
|
2.40.013.295 W srv prompt_save: - saving prompt with length 95, total state size = 32.659 MiB (draft: 0.000 MiB)
|
|
2.40.041.979 I srv load: - looking for better prompt, base f_keep = 0.474, sim = 0.714
|
|
2.40.041.986 I srv update: - cache state: 4 prompts, 182.545 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
2.40.041.987 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
2.40.041.988 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
2.40.041.988 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
2.40.041.989 I srv update: - prompt 0x5b0d9a17b8a0: 95 tokens, checkpoints: 0, 32.659 MiB
|
|
2.40.041.990 I srv get_availabl: prompt cache update took 28.72 ms
|
|
2.40.042.036 I slot launch_slot_: id 3 | task 445 | processing task, is_child = 0
|
|
2.40.042.037 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.40.042.038 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.40.042.044 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.40.042.045 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.40.042.049 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.40.042.049 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.00.192.132 I slot print_timing: id 3 | task 445 | prompt eval time = 1136.33 ms / 18 tokens ( 63.13 ms per token, 15.84 tokens per second)
|
|
3.00.192.135 I slot print_timing: id 3 | task 445 | eval time = 19013.73 ms / 65 tokens ( 292.52 ms per token, 3.42 tokens per second)
|
|
3.00.192.136 I slot print_timing: id 3 | task 445 | total time = 20150.07 ms / 83 tokens
|
|
3.00.192.137 I slot print_timing: id 3 | task 445 | graphs reused = 491
|
|
3.00.192.169 I slot release: id 3 | task 445 | stop processing: n_tokens = 127, truncated = 0
|
|
3.00.192.176 I srv update_slots: all slots are idle
|
|
3.00.195.292 I srv operator(): Chat format: peg-native
|
|
3.00.195.443 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.868 (> 0.100 thold), f_keep = 0.362
|
|
3.00.195.444 I srv get_availabl: updating prompt cache
|
|
3.00.195.467 W srv prompt_save: - saving prompt with length 127, total state size = 43.660 MiB (draft: 0.000 MiB)
|
|
3.00.232.197 I srv load: - looking for better prompt, base f_keep = 0.362, sim = 0.868
|
|
3.00.232.205 I srv update: - cache state: 5 prompts, 226.205 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
3.00.232.206 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
3.00.232.207 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
3.00.232.208 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
3.00.232.208 I srv update: - prompt 0x5b0d9a17b8a0: 95 tokens, checkpoints: 0, 32.659 MiB
|
|
3.00.232.208 I srv update: - prompt 0x5b0d9a490370: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
3.00.232.210 I srv get_availabl: prompt cache update took 36.77 ms
|
|
3.00.232.256 I slot launch_slot_: id 3 | task 511 | processing task, is_child = 0
|
|
3.00.232.257 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.00.232.258 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.00.232.265 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.00.232.266 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.00.232.270 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.00.232.270 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.20.971.436 I slot print_timing: id 3 | task 511 | prompt eval time = 2055.25 ms / 7 tokens ( 293.61 ms per token, 3.41 tokens per second)
|
|
3.20.971.439 I slot print_timing: id 3 | task 511 | eval time = 18683.90 ms / 64 tokens ( 291.94 ms per token, 3.43 tokens per second)
|
|
3.20.971.440 I slot print_timing: id 3 | task 511 | total time = 20739.15 ms / 71 tokens
|
|
3.20.971.441 I slot print_timing: id 3 | task 511 | graphs reused = 553
|
|
3.20.971.475 I slot release: id 3 | task 511 | stop processing: n_tokens = 116, truncated = 0
|
|
3.20.971.482 I srv update_slots: all slots are idle
|
|
3.20.974.387 I srv operator(): Chat format: peg-native
|
|
3.20.974.507 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.388
|
|
3.20.974.510 I srv get_availabl: updating prompt cache
|
|
3.20.974.536 W srv prompt_save: - saving prompt with length 116, total state size = 39.878 MiB (draft: 0.000 MiB)
|
|
3.21.009.834 I srv load: - looking for better prompt, base f_keep = 0.388, sim = 0.714
|
|
3.21.009.842 I srv load: - found better prompt with f_keep = 0.484, sim = 0.730
|
|
3.21.027.185 I srv update: - cache state: 5 prompts, 233.424 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
3.21.027.189 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
3.21.027.190 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
3.21.027.191 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
3.21.027.191 I srv update: - prompt 0x5b0d9a490370: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
3.21.027.192 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
3.21.027.193 I srv get_availabl: prompt cache update took 52.68 ms
|
|
3.21.027.230 I slot launch_slot_: id 3 | task 576 | processing task, is_child = 0
|
|
3.21.027.231 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.21.027.232 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.21.027.239 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.21.027.239 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.21.027.243 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.21.027.244 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.40.865.632 I slot print_timing: id 3 | task 576 | prompt eval time = 1136.73 ms / 17 tokens ( 66.87 ms per token, 14.96 tokens per second)
|
|
3.40.865.635 I slot print_timing: id 3 | task 576 | eval time = 18701.64 ms / 64 tokens ( 292.21 ms per token, 3.42 tokens per second)
|
|
3.40.865.636 I slot print_timing: id 3 | task 576 | total time = 19838.37 ms / 81 tokens
|
|
3.40.865.636 I slot print_timing: id 3 | task 576 | graphs reused = 615
|
|
3.40.865.666 I slot release: id 3 | task 576 | stop processing: n_tokens = 126, truncated = 0
|
|
3.40.865.675 I srv update_slots: all slots are idle
|
|
3.40.868.495 I srv operator(): Chat format: peg-native
|
|
3.40.868.603 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.357
|
|
3.40.868.605 I srv get_availabl: updating prompt cache
|
|
3.40.868.628 W srv prompt_save: - saving prompt with length 126, total state size = 43.316 MiB (draft: 0.000 MiB)
|
|
3.40.905.168 I srv load: - looking for better prompt, base f_keep = 0.357, sim = 0.776
|
|
3.40.905.175 I srv load: - found better prompt with f_keep = 0.405, sim = 0.810
|
|
3.40.924.965 I srv update: - cache state: 5 prompts, 236.862 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
3.40.924.969 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
3.40.924.969 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
3.40.924.970 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
3.40.924.971 I srv update: - prompt 0x5b0d9a490370: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
3.40.924.971 I srv update: - prompt 0x5b0d9b2799c0: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
3.40.924.972 I srv get_availabl: prompt cache update took 56.37 ms
|
|
3.40.925.006 I slot launch_slot_: id 3 | task 641 | processing task, is_child = 0
|
|
3.40.925.007 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.40.925.008 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.40.925.016 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.40.925.016 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.40.925.020 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.40.925.021 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.05.620.068 I slot print_timing: id 3 | task 641 | prompt eval time = 935.92 ms / 11 tokens ( 85.08 ms per token, 11.75 tokens per second)
|
|
4.05.620.071 I slot print_timing: id 3 | task 641 | eval time = 23759.10 ms / 81 tokens ( 293.32 ms per token, 3.41 tokens per second)
|
|
4.05.620.072 I slot print_timing: id 3 | task 641 | total time = 24695.03 ms / 92 tokens
|
|
4.05.620.073 I slot print_timing: id 3 | task 641 | graphs reused = 694
|
|
4.05.620.106 I slot release: id 3 | task 641 | stop processing: n_tokens = 138, truncated = 0
|
|
4.05.620.113 I srv update_slots: all slots are idle
|
|
4.05.622.905 I srv operator(): Chat format: peg-native
|
|
4.05.623.007 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.326
|
|
4.05.623.008 I srv get_availabl: updating prompt cache
|
|
4.05.623.036 W srv prompt_save: - saving prompt with length 138, total state size = 47.441 MiB (draft: 0.000 MiB)
|
|
4.05.663.124 I srv load: - looking for better prompt, base f_keep = 0.326, sim = 0.714
|
|
4.05.663.133 I srv update: - cache state: 6 prompts, 284.303 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
4.05.663.133 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
4.05.663.134 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
4.05.663.135 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
4.05.663.135 I srv update: - prompt 0x5b0d9a490370: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
4.05.663.136 I srv update: - prompt 0x5b0d9b2799c0: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
4.05.663.136 I srv update: - prompt 0x5b0d9b3bafd0: 138 tokens, checkpoints: 0, 47.441 MiB
|
|
4.05.663.137 I srv get_availabl: prompt cache update took 40.13 ms
|
|
4.05.663.188 I slot launch_slot_: id 3 | task 723 | processing task, is_child = 0
|
|
4.05.663.188 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.05.663.189 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.05.663.197 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.05.663.197 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.05.663.202 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.05.663.202 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.18.693.538 I slot print_timing: id 3 | task 723 | prompt eval time = 1137.47 ms / 18 tokens ( 63.19 ms per token, 15.82 tokens per second)
|
|
4.18.693.541 I slot print_timing: id 3 | task 723 | eval time = 11892.85 ms / 41 tokens ( 290.07 ms per token, 3.45 tokens per second)
|
|
4.18.693.542 I slot print_timing: id 3 | task 723 | total time = 13030.32 ms / 59 tokens
|
|
4.18.693.543 I slot print_timing: id 3 | task 723 | graphs reused = 733
|
|
4.18.693.575 I slot release: id 3 | task 723 | stop processing: n_tokens = 103, truncated = 0
|
|
4.18.693.584 I srv update_slots: all slots are idle
|
|
4.18.696.677 I srv operator(): Chat format: peg-native
|
|
4.18.696.790 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.437
|
|
4.18.696.792 I srv get_availabl: updating prompt cache
|
|
4.18.696.830 W srv prompt_save: - saving prompt with length 103, total state size = 35.409 MiB (draft: 0.000 MiB)
|
|
4.18.727.878 I srv load: - looking for better prompt, base f_keep = 0.437, sim = 0.776
|
|
4.18.727.887 I srv update: - cache state: 7 prompts, 319.712 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
4.18.727.887 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
4.18.727.888 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
4.18.727.889 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
4.18.727.889 I srv update: - prompt 0x5b0d9a490370: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
4.18.727.890 I srv update: - prompt 0x5b0d9b2799c0: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
4.18.727.890 I srv update: - prompt 0x5b0d9b3bafd0: 138 tokens, checkpoints: 0, 47.441 MiB
|
|
4.18.727.891 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
4.18.727.892 I srv get_availabl: prompt cache update took 31.10 ms
|
|
4.18.727.938 I slot launch_slot_: id 3 | task 765 | processing task, is_child = 0
|
|
4.18.727.939 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.18.727.940 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.18.727.947 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.18.727.947 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.18.727.952 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.18.727.952 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.32.130.667 I slot print_timing: id 3 | task 765 | prompt eval time = 936.54 ms / 13 tokens ( 72.04 ms per token, 13.88 tokens per second)
|
|
4.32.130.670 I slot print_timing: id 3 | task 765 | eval time = 12466.16 ms / 43 tokens ( 289.91 ms per token, 3.45 tokens per second)
|
|
4.32.130.671 I slot print_timing: id 3 | task 765 | total time = 13402.70 ms / 56 tokens
|
|
4.32.130.672 I slot print_timing: id 3 | task 765 | graphs reused = 774
|
|
4.32.130.703 I slot release: id 3 | task 765 | stop processing: n_tokens = 100, truncated = 0
|
|
4.32.130.710 I srv update_slots: all slots are idle
|
|
4.32.133.562 I srv operator(): Chat format: peg-native
|
|
4.32.133.671 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.750 (> 0.100 thold), f_keep = 0.450
|
|
4.32.133.673 I srv get_availabl: updating prompt cache
|
|
4.32.133.699 W srv prompt_save: - saving prompt with length 100, total state size = 34.378 MiB (draft: 0.000 MiB)
|
|
4.32.164.015 I srv load: - looking for better prompt, base f_keep = 0.450, sim = 0.750
|
|
4.32.164.025 I srv update: - cache state: 8 prompts, 354.091 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
4.32.164.025 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
4.32.164.026 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
4.32.164.027 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
4.32.164.028 I srv update: - prompt 0x5b0d9a490370: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
4.32.164.028 I srv update: - prompt 0x5b0d9b2799c0: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
4.32.164.032 I srv update: - prompt 0x5b0d9b3bafd0: 138 tokens, checkpoints: 0, 47.441 MiB
|
|
4.32.164.033 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
4.32.164.033 I srv update: - prompt 0x5b0d9a473550: 100 tokens, checkpoints: 0, 34.378 MiB
|
|
4.32.164.034 I srv get_availabl: prompt cache update took 30.36 ms
|
|
4.32.164.082 I slot launch_slot_: id 3 | task 809 | processing task, is_child = 0
|
|
4.32.164.083 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.32.164.084 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.32.164.091 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.32.164.091 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.32.164.096 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.32.164.096 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.38.450.065 I slot print_timing: id 3 | task 809 | prompt eval time = 937.54 ms / 15 tokens ( 62.50 ms per token, 16.00 tokens per second)
|
|
4.38.450.068 I slot print_timing: id 3 | task 809 | eval time = 5348.41 ms / 19 tokens ( 281.50 ms per token, 3.55 tokens per second)
|
|
4.38.450.069 I slot print_timing: id 3 | task 809 | total time = 6285.95 ms / 34 tokens
|
|
4.38.450.070 I slot print_timing: id 3 | task 809 | graphs reused = 791
|
|
4.38.450.103 I slot release: id 3 | task 809 | stop processing: n_tokens = 78, truncated = 0
|
|
4.38.450.110 I srv update_slots: all slots are idle
|
|
4.38.452.936 I srv operator(): Chat format: peg-native
|
|
4.38.453.088 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.763 (> 0.100 thold), f_keep = 0.577
|
|
4.38.453.109 I slot launch_slot_: id 3 | task 829 | processing task, is_child = 0
|
|
4.38.453.110 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.38.453.111 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.38.453.118 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.38.453.118 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.38.453.122 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.38.453.123 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.59.900.499 I slot print_timing: id 3 | task 829 | prompt eval time = 937.05 ms / 14 tokens ( 66.93 ms per token, 14.94 tokens per second)
|
|
4.59.900.504 I slot print_timing: id 3 | task 829 | eval time = 20510.32 ms / 70 tokens ( 293.00 ms per token, 3.41 tokens per second)
|
|
4.59.900.505 I slot print_timing: id 3 | task 829 | total time = 21447.36 ms / 84 tokens
|
|
4.59.900.505 I slot print_timing: id 3 | task 829 | graphs reused = 859
|
|
4.59.900.539 I slot release: id 3 | task 829 | stop processing: n_tokens = 128, truncated = 0
|
|
4.59.900.549 I srv update_slots: all slots are idle
|
|
4.59.903.379 I srv operator(): Chat format: peg-native
|
|
4.59.903.497 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.738 (> 0.100 thold), f_keep = 0.352
|
|
4.59.903.498 I srv get_availabl: updating prompt cache
|
|
4.59.903.525 W srv prompt_save: - saving prompt with length 128, total state size = 44.004 MiB (draft: 0.000 MiB)
|
|
4.59.940.747 I srv load: - looking for better prompt, base f_keep = 0.352, sim = 0.738
|
|
4.59.940.753 I srv load: - found better prompt with f_keep = 0.362, sim = 0.754
|
|
4.59.962.255 I srv update: - cache state: 8 prompts, 354.434 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
4.59.962.259 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
4.59.962.260 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
4.59.962.260 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
4.59.962.261 I srv update: - prompt 0x5b0d9b2799c0: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
4.59.962.261 I srv update: - prompt 0x5b0d9b3bafd0: 138 tokens, checkpoints: 0, 47.441 MiB
|
|
4.59.962.262 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
4.59.962.262 I srv update: - prompt 0x5b0d9a473550: 100 tokens, checkpoints: 0, 34.378 MiB
|
|
4.59.962.263 I srv update: - prompt 0x5b0d98ec5f30: 128 tokens, checkpoints: 0, 44.004 MiB
|
|
4.59.962.264 I srv get_availabl: prompt cache update took 58.77 ms
|
|
4.59.962.296 I slot launch_slot_: id 3 | task 900 | processing task, is_child = 0
|
|
4.59.962.297 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.59.962.298 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.59.962.306 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.59.962.306 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.59.962.310 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.59.962.311 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.21.117.202 I slot print_timing: id 3 | task 900 | prompt eval time = 937.36 ms / 15 tokens ( 62.49 ms per token, 16.00 tokens per second)
|
|
5.21.117.206 I slot print_timing: id 3 | task 900 | eval time = 20217.51 ms / 69 tokens ( 293.01 ms per token, 3.41 tokens per second)
|
|
5.21.117.207 I slot print_timing: id 3 | task 900 | total time = 21154.87 ms / 84 tokens
|
|
5.21.117.207 I slot print_timing: id 3 | task 900 | graphs reused = 926
|
|
5.21.117.237 I slot release: id 3 | task 900 | stop processing: n_tokens = 129, truncated = 0
|
|
5.21.117.244 I srv update_slots: all slots are idle
|
|
5.21.120.093 I srv operator(): Chat format: peg-native
|
|
5.21.120.221 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.349
|
|
5.21.120.223 I srv get_availabl: updating prompt cache
|
|
5.21.120.249 W srv prompt_save: - saving prompt with length 129, total state size = 44.347 MiB (draft: 0.000 MiB)
|
|
5.21.156.914 I srv load: - looking for better prompt, base f_keep = 0.349, sim = 0.789
|
|
5.21.156.922 I srv load: - found better prompt with f_keep = 0.365, sim = 0.807
|
|
5.21.177.224 I srv update: - cache state: 8 prompts, 355.466 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
5.21.177.229 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
5.21.177.229 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
5.21.177.230 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
5.21.177.231 I srv update: - prompt 0x5b0d9b3bafd0: 138 tokens, checkpoints: 0, 47.441 MiB
|
|
5.21.177.231 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
5.21.177.232 I srv update: - prompt 0x5b0d9a473550: 100 tokens, checkpoints: 0, 34.378 MiB
|
|
5.21.177.232 I srv update: - prompt 0x5b0d98ec5f30: 128 tokens, checkpoints: 0, 44.004 MiB
|
|
5.21.177.233 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
5.21.177.234 I srv get_availabl: prompt cache update took 57.01 ms
|
|
5.21.177.270 I slot launch_slot_: id 3 | task 970 | processing task, is_child = 0
|
|
5.21.177.271 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.21.177.271 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.21.177.280 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.21.177.280 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.21.177.284 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.21.177.285 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.44.093.019 I slot print_timing: id 3 | task 970 | prompt eval time = 935.71 ms / 11 tokens ( 85.06 ms per token, 11.76 tokens per second)
|
|
5.44.093.023 I slot print_timing: id 3 | task 970 | eval time = 21980.01 ms / 75 tokens ( 293.07 ms per token, 3.41 tokens per second)
|
|
5.44.093.024 I slot print_timing: id 3 | task 970 | total time = 22915.71 ms / 86 tokens
|
|
5.44.093.025 I slot print_timing: id 3 | task 970 | graphs reused = 999
|
|
5.44.093.057 I slot release: id 3 | task 970 | stop processing: n_tokens = 131, truncated = 0
|
|
5.44.093.063 I srv update_slots: all slots are idle
|
|
5.44.095.732 I srv operator(): Chat format: peg-native
|
|
5.44.095.845 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.738 (> 0.100 thold), f_keep = 0.344
|
|
5.44.095.846 I srv get_availabl: updating prompt cache
|
|
5.44.095.871 W srv prompt_save: - saving prompt with length 131, total state size = 45.035 MiB (draft: 0.000 MiB)
|
|
5.44.133.624 I srv load: - looking for better prompt, base f_keep = 0.344, sim = 0.738
|
|
5.44.133.631 I srv load: - found better prompt with f_keep = 0.348, sim = 0.787
|
|
5.44.155.583 I srv update: - cache state: 8 prompts, 353.059 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
5.44.155.587 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
5.44.155.588 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
5.44.155.588 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
5.44.155.589 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
5.44.155.590 I srv update: - prompt 0x5b0d9a473550: 100 tokens, checkpoints: 0, 34.378 MiB
|
|
5.44.155.590 I srv update: - prompt 0x5b0d98ec5f30: 128 tokens, checkpoints: 0, 44.004 MiB
|
|
5.44.155.591 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
5.44.155.591 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
5.44.155.592 I srv get_availabl: prompt cache update took 59.75 ms
|
|
5.44.155.627 I slot launch_slot_: id 3 | task 1046 | processing task, is_child = 0
|
|
5.44.155.627 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.44.155.629 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.44.155.636 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.44.155.636 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.44.155.640 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.44.155.641 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.59.071.871 I slot print_timing: id 3 | task 1046 | prompt eval time = 935.59 ms / 13 tokens ( 71.97 ms per token, 13.89 tokens per second)
|
|
5.59.071.877 I slot print_timing: id 3 | task 1046 | eval time = 13980.62 ms / 48 tokens ( 291.26 ms per token, 3.43 tokens per second)
|
|
5.59.071.878 I slot print_timing: id 3 | task 1046 | total time = 14916.21 ms / 61 tokens
|
|
5.59.071.878 I slot print_timing: id 3 | task 1046 | graphs reused = 1045
|
|
5.59.071.916 I slot release: id 3 | task 1046 | stop processing: n_tokens = 108, truncated = 0
|
|
5.59.071.924 I srv update_slots: all slots are idle
|
|
5.59.074.567 I srv operator(): Chat format: peg-native
|
|
5.59.074.710 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.804 (> 0.100 thold), f_keep = 0.417
|
|
5.59.074.712 I srv get_availabl: updating prompt cache
|
|
5.59.074.742 W srv prompt_save: - saving prompt with length 108, total state size = 37.128 MiB (draft: 0.000 MiB)
|
|
5.59.107.026 I srv load: - looking for better prompt, base f_keep = 0.417, sim = 0.804
|
|
5.59.107.037 I srv update: - cache state: 9 prompts, 390.188 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
5.59.107.038 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
5.59.107.039 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
5.59.107.043 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
5.59.107.043 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
5.59.107.044 I srv update: - prompt 0x5b0d9a473550: 100 tokens, checkpoints: 0, 34.378 MiB
|
|
5.59.107.044 I srv update: - prompt 0x5b0d98ec5f30: 128 tokens, checkpoints: 0, 44.004 MiB
|
|
5.59.107.045 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
5.59.107.045 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
5.59.107.046 I srv update: - prompt 0x5b0d9a490370: 108 tokens, checkpoints: 0, 37.128 MiB
|
|
5.59.107.047 I srv get_availabl: prompt cache update took 32.33 ms
|
|
5.59.107.099 I slot launch_slot_: id 3 | task 1095 | processing task, is_child = 0
|
|
5.59.107.099 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.59.107.101 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.59.107.109 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.59.107.109 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.59.107.113 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.59.107.114 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
6.21.146.671 I slot print_timing: id 3 | task 1095 | prompt eval time = 936.23 ms / 11 tokens ( 85.11 ms per token, 11.75 tokens per second)
|
|
6.21.146.675 I slot print_timing: id 3 | task 1095 | eval time = 21103.31 ms / 72 tokens ( 293.10 ms per token, 3.41 tokens per second)
|
|
6.21.146.676 I slot print_timing: id 3 | task 1095 | total time = 22039.54 ms / 83 tokens
|
|
6.21.146.676 I slot print_timing: id 3 | task 1095 | graphs reused = 1115
|
|
6.21.146.708 I slot release: id 3 | task 1095 | stop processing: n_tokens = 127, truncated = 0
|
|
6.21.146.716 I srv update_slots: all slots are idle
|
|
6.21.149.528 I srv operator(): Chat format: peg-native
|
|
6.21.149.641 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.818 (> 0.100 thold), f_keep = 0.354
|
|
6.21.149.643 I srv get_availabl: updating prompt cache
|
|
6.21.149.667 W srv prompt_save: - saving prompt with length 127, total state size = 43.660 MiB (draft: 0.000 MiB)
|
|
6.21.186.136 I srv load: - looking for better prompt, base f_keep = 0.354, sim = 0.818
|
|
6.21.186.145 I srv load: - found better prompt with f_keep = 0.460, sim = 0.836
|
|
6.21.203.718 I srv update: - cache state: 9 prompts, 399.469 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
6.21.203.722 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
6.21.203.723 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
6.21.203.723 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
6.21.203.724 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
6.21.203.724 I srv update: - prompt 0x5b0d98ec5f30: 128 tokens, checkpoints: 0, 44.004 MiB
|
|
6.21.203.725 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
6.21.203.726 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
6.21.203.726 I srv update: - prompt 0x5b0d9a490370: 108 tokens, checkpoints: 0, 37.128 MiB
|
|
6.21.203.726 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
6.21.203.728 I srv get_availabl: prompt cache update took 54.08 ms
|
|
6.21.203.762 I slot launch_slot_: id 3 | task 1168 | processing task, is_child = 0
|
|
6.21.203.763 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
6.21.203.764 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
6.21.203.772 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
6.21.203.773 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
6.21.203.777 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
6.21.203.777 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
6.36.698.569 I slot print_timing: id 3 | task 1168 | prompt eval time = 934.69 ms / 9 tokens ( 103.85 ms per token, 9.63 tokens per second)
|
|
6.36.698.573 I slot print_timing: id 3 | task 1168 | eval time = 14560.08 ms / 50 tokens ( 291.20 ms per token, 3.43 tokens per second)
|
|
6.36.698.574 I slot print_timing: id 3 | task 1168 | total time = 15494.77 ms / 59 tokens
|
|
6.36.698.574 I slot print_timing: id 3 | task 1168 | graphs reused = 1163
|
|
6.36.698.607 I slot release: id 3 | task 1168 | stop processing: n_tokens = 104, truncated = 0
|
|
6.36.698.615 I srv update_slots: all slots are idle
|
|
6.36.701.486 I srv operator(): Chat format: peg-native
|
|
6.36.701.594 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.852 (> 0.100 thold), f_keep = 0.442
|
|
6.36.701.596 I srv get_availabl: updating prompt cache
|
|
6.36.701.626 W srv prompt_save: - saving prompt with length 104, total state size = 35.753 MiB (draft: 0.000 MiB)
|
|
6.36.732.671 I srv load: - looking for better prompt, base f_keep = 0.442, sim = 0.852
|
|
6.36.732.679 I srv load: - found better prompt with f_keep = 0.444, sim = 0.889
|
|
6.36.751.249 I srv update: - cache state: 9 prompts, 398.094 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
6.36.751.254 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
6.36.751.255 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
6.36.751.256 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
6.36.751.256 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
6.36.751.257 I srv update: - prompt 0x5b0d98ec5f30: 128 tokens, checkpoints: 0, 44.004 MiB
|
|
6.36.751.257 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
6.36.751.258 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
6.36.751.258 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
6.36.751.259 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
6.36.751.260 I srv get_availabl: prompt cache update took 49.66 ms
|
|
6.36.751.292 I slot launch_slot_: id 3 | task 1219 | processing task, is_child = 0
|
|
6.36.751.293 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
6.36.751.294 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
6.36.751.303 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
6.36.751.303 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
6.36.751.308 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
6.36.751.308 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
6.51.008.149 I slot print_timing: id 3 | task 1219 | prompt eval time = 1760.95 ms / 6 tokens ( 293.49 ms per token, 3.41 tokens per second)
|
|
6.51.008.153 I slot print_timing: id 3 | task 1219 | eval time = 12495.87 ms / 43 tokens ( 290.60 ms per token, 3.44 tokens per second)
|
|
6.51.008.153 I slot print_timing: id 3 | task 1219 | total time = 14256.82 ms / 49 tokens
|
|
6.51.008.154 I slot print_timing: id 3 | task 1219 | graphs reused = 1204
|
|
6.51.008.187 I slot release: id 3 | task 1219 | stop processing: n_tokens = 96, truncated = 0
|
|
6.51.008.194 I srv update_slots: all slots are idle
|
|
6.51.010.996 I srv operator(): Chat format: peg-native
|
|
6.51.011.105 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.849 (> 0.100 thold), f_keep = 0.469
|
|
6.51.011.107 I srv get_availabl: updating prompt cache
|
|
6.51.011.135 W srv prompt_save: - saving prompt with length 96, total state size = 33.003 MiB (draft: 0.000 MiB)
|
|
6.51.039.843 I srv load: - looking for better prompt, base f_keep = 0.469, sim = 0.849
|
|
6.51.039.854 I srv update: - cache state: 10 prompts, 431.097 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
6.51.039.854 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
6.51.039.855 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
6.51.039.856 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
6.51.039.856 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
6.51.039.857 I srv update: - prompt 0x5b0d98ec5f30: 128 tokens, checkpoints: 0, 44.004 MiB
|
|
6.51.039.857 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
6.51.039.858 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
6.51.039.858 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
6.51.039.858 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
6.51.039.859 I srv update: - prompt 0x5b0d9a61aec0: 96 tokens, checkpoints: 0, 33.003 MiB
|
|
6.51.039.860 I srv get_availabl: prompt cache update took 28.75 ms
|
|
6.51.039.908 I slot launch_slot_: id 3 | task 1263 | processing task, is_child = 0
|
|
6.51.039.909 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
6.51.039.910 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
6.51.039.918 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
6.51.039.918 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
6.51.039.922 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
6.51.039.923 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.12.123.672 I slot print_timing: id 3 | task 1263 | prompt eval time = 2374.28 ms / 8 tokens ( 296.78 ms per token, 3.37 tokens per second)
|
|
7.12.123.675 I slot print_timing: id 3 | task 1263 | eval time = 18709.45 ms / 64 tokens ( 292.34 ms per token, 3.42 tokens per second)
|
|
7.12.123.676 I slot print_timing: id 3 | task 1263 | total time = 21083.73 ms / 72 tokens
|
|
7.12.123.677 I slot print_timing: id 3 | task 1263 | graphs reused = 1266
|
|
7.12.123.708 I slot release: id 3 | task 1263 | stop processing: n_tokens = 116, truncated = 0
|
|
7.12.123.716 I srv update_slots: all slots are idle
|
|
7.12.126.647 I srv operator(): Chat format: peg-native
|
|
7.12.126.804 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.388
|
|
7.12.126.812 I srv get_availabl: updating prompt cache
|
|
7.12.126.841 W srv prompt_save: - saving prompt with length 116, total state size = 39.878 MiB (draft: 0.000 MiB)
|
|
7.12.160.568 I srv load: - looking for better prompt, base f_keep = 0.388, sim = 0.776
|
|
7.12.160.578 I srv update: - cache state: 11 prompts, 470.976 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
7.12.160.579 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
7.12.160.579 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
7.12.160.580 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
7.12.160.581 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
7.12.160.581 I srv update: - prompt 0x5b0d98ec5f30: 128 tokens, checkpoints: 0, 44.004 MiB
|
|
7.12.160.582 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
7.12.160.582 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
7.12.160.583 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
7.12.160.583 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
7.12.160.583 I srv update: - prompt 0x5b0d9a61aec0: 96 tokens, checkpoints: 0, 33.003 MiB
|
|
7.12.160.584 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
7.12.160.587 I srv get_availabl: prompt cache update took 33.77 ms
|
|
7.12.160.636 I slot launch_slot_: id 3 | task 1328 | processing task, is_child = 0
|
|
7.12.160.637 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.12.160.638 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.12.160.646 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.12.160.646 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.12.160.651 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.12.160.651 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.33.011.072 I slot print_timing: id 3 | task 1328 | prompt eval time = 937.25 ms / 13 tokens ( 72.10 ms per token, 13.87 tokens per second)
|
|
7.33.011.076 I slot print_timing: id 3 | task 1328 | eval time = 19913.16 ms / 68 tokens ( 292.84 ms per token, 3.41 tokens per second)
|
|
7.33.011.077 I slot print_timing: id 3 | task 1328 | total time = 20850.40 ms / 81 tokens
|
|
7.33.011.078 I slot print_timing: id 3 | task 1328 | graphs reused = 1332
|
|
7.33.011.110 I slot release: id 3 | task 1328 | stop processing: n_tokens = 125, truncated = 0
|
|
7.33.011.118 I srv update_slots: all slots are idle
|
|
7.33.013.656 I srv operator(): Chat format: peg-native
|
|
7.33.013.765 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.360
|
|
7.33.013.767 I srv get_availabl: updating prompt cache
|
|
7.33.013.791 W srv prompt_save: - saving prompt with length 125, total state size = 42.972 MiB (draft: 0.000 MiB)
|
|
7.33.049.529 I srv load: - looking for better prompt, base f_keep = 0.360, sim = 0.789
|
|
7.33.049.539 I srv load: - found better prompt with f_keep = 0.500, sim = 0.842
|
|
7.33.066.658 I srv update: - cache state: 11 prompts, 480.945 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
7.33.066.663 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
7.33.066.663 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
7.33.066.664 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
7.33.066.664 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
7.33.066.665 I srv update: - prompt 0x5b0d98ec5f30: 128 tokens, checkpoints: 0, 44.004 MiB
|
|
7.33.066.665 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
7.33.066.666 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
7.33.066.666 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
7.33.066.667 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
7.33.066.667 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
7.33.066.668 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
7.33.066.669 I srv get_availabl: prompt cache update took 52.90 ms
|
|
7.33.066.704 I slot launch_slot_: id 3 | task 1397 | processing task, is_child = 0
|
|
7.33.066.705 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.33.066.706 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.33.066.715 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.33.066.715 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.33.066.720 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.33.066.720 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.44.107.606 I slot print_timing: id 3 | task 1397 | prompt eval time = 935.40 ms / 9 tokens ( 103.93 ms per token, 9.62 tokens per second)
|
|
7.44.107.609 I slot print_timing: id 3 | task 1397 | eval time = 10105.47 ms / 35 tokens ( 288.73 ms per token, 3.46 tokens per second)
|
|
7.44.107.610 I slot print_timing: id 3 | task 1397 | total time = 11040.87 ms / 44 tokens
|
|
7.44.107.611 I slot print_timing: id 3 | task 1397 | graphs reused = 1365
|
|
7.44.107.643 I slot release: id 3 | task 1397 | stop processing: n_tokens = 91, truncated = 0
|
|
7.44.107.650 I srv update_slots: all slots are idle
|
|
7.44.110.441 I srv operator(): Chat format: peg-native
|
|
7.44.110.549 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.495
|
|
7.44.110.551 I srv get_availabl: updating prompt cache
|
|
7.44.110.574 W srv prompt_save: - saving prompt with length 91, total state size = 31.284 MiB (draft: 0.000 MiB)
|
|
7.44.137.661 I srv load: - looking for better prompt, base f_keep = 0.495, sim = 0.714
|
|
7.44.137.673 I srv update: - cache state: 12 prompts, 512.229 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
7.44.137.674 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
7.44.137.674 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
7.44.137.675 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
7.44.137.677 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
7.44.137.678 I srv update: - prompt 0x5b0d98ec5f30: 128 tokens, checkpoints: 0, 44.004 MiB
|
|
7.44.137.678 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
7.44.137.679 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
7.44.137.679 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
7.44.137.680 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
7.44.137.680 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
7.44.137.681 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
7.44.137.681 I srv update: - prompt 0x5b0d9a43ddb0: 91 tokens, checkpoints: 0, 31.284 MiB
|
|
7.44.137.683 I srv get_availabl: prompt cache update took 27.13 ms
|
|
7.44.137.732 I slot launch_slot_: id 3 | task 1433 | processing task, is_child = 0
|
|
7.44.137.732 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.44.137.733 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.44.137.741 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.44.137.742 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.44.137.746 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.44.137.746 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.52.114.345 I slot print_timing: id 3 | task 1433 | prompt eval time = 1138.39 ms / 18 tokens ( 63.24 ms per token, 15.81 tokens per second)
|
|
7.52.114.348 I slot print_timing: id 3 | task 1433 | eval time = 6838.19 ms / 24 tokens ( 284.92 ms per token, 3.51 tokens per second)
|
|
7.52.114.349 I slot print_timing: id 3 | task 1433 | total time = 7976.58 ms / 42 tokens
|
|
7.52.114.349 I slot print_timing: id 3 | task 1433 | graphs reused = 1387
|
|
7.52.114.381 I slot release: id 3 | task 1433 | stop processing: n_tokens = 86, truncated = 0
|
|
7.52.114.389 I srv update_slots: all slots are idle
|
|
7.52.117.318 I srv operator(): Chat format: peg-native
|
|
7.52.117.426 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.726 (> 0.100 thold), f_keep = 0.523
|
|
7.52.117.446 I slot launch_slot_: id 3 | task 1458 | processing task, is_child = 0
|
|
7.52.117.447 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.52.117.447 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.52.117.454 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.52.117.454 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.52.117.458 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.52.117.459 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
7.58.919.131 I slot print_timing: id 3 | task 1458 | prompt eval time = 1138.46 ms / 17 tokens ( 66.97 ms per token, 14.93 tokens per second)
|
|
7.58.919.134 I slot print_timing: id 3 | task 1458 | eval time = 5663.20 ms / 20 tokens ( 283.16 ms per token, 3.53 tokens per second)
|
|
7.58.919.135 I slot print_timing: id 3 | task 1458 | total time = 6801.66 ms / 37 tokens
|
|
7.58.919.136 I slot print_timing: id 3 | task 1458 | graphs reused = 1405
|
|
7.58.919.192 I slot release: id 3 | task 1458 | stop processing: n_tokens = 81, truncated = 0
|
|
7.58.919.199 I srv update_slots: all slots are idle
|
|
7.58.921.889 I srv operator(): Chat format: peg-native
|
|
7.58.922.002 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.849 (> 0.100 thold), f_keep = 0.556
|
|
7.58.922.022 I slot launch_slot_: id 3 | task 1479 | processing task, is_child = 0
|
|
7.58.922.024 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
7.58.922.024 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
7.58.922.032 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
7.58.922.033 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
7.58.922.039 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
7.58.922.040 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
8.23.008.328 I slot print_timing: id 3 | task 1479 | prompt eval time = 2368.88 ms / 8 tokens ( 296.11 ms per token, 3.38 tokens per second)
|
|
8.23.008.331 I slot print_timing: id 3 | task 1479 | eval time = 21717.40 ms / 74 tokens ( 293.48 ms per token, 3.41 tokens per second)
|
|
8.23.008.331 I slot print_timing: id 3 | task 1479 | total time = 24086.27 ms / 82 tokens
|
|
8.23.008.332 I slot print_timing: id 3 | task 1479 | graphs reused = 1477
|
|
8.23.008.362 I slot release: id 3 | task 1479 | stop processing: n_tokens = 126, truncated = 0
|
|
8.23.008.369 I srv update_slots: all slots are idle
|
|
8.23.011.081 I srv operator(): Chat format: peg-native
|
|
8.23.011.190 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.357
|
|
8.23.011.192 I srv get_availabl: updating prompt cache
|
|
8.23.011.219 W srv prompt_save: - saving prompt with length 126, total state size = 43.316 MiB (draft: 0.000 MiB)
|
|
8.23.047.111 I srv load: - looking for better prompt, base f_keep = 0.357, sim = 0.789
|
|
8.23.047.120 I srv load: - found better prompt with f_keep = 0.359, sim = 0.807
|
|
8.23.067.747 I srv update: - cache state: 12 prompts, 511.541 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
8.23.067.752 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
8.23.067.752 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
8.23.067.753 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
8.23.067.753 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
8.23.067.754 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
8.23.067.754 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
8.23.067.755 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
8.23.067.755 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
8.23.067.756 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
8.23.067.756 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
8.23.067.757 I srv update: - prompt 0x5b0d9a43ddb0: 91 tokens, checkpoints: 0, 31.284 MiB
|
|
8.23.067.757 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
8.23.067.759 I srv get_availabl: prompt cache update took 56.56 ms
|
|
8.23.067.792 I slot launch_slot_: id 3 | task 1554 | processing task, is_child = 0
|
|
8.23.067.793 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
8.23.067.794 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
8.23.067.801 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
8.23.067.801 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
8.23.067.813 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
8.23.067.813 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
8.39.465.108 I slot print_timing: id 3 | task 1554 | prompt eval time = 935.11 ms / 11 tokens ( 85.01 ms per token, 11.76 tokens per second)
|
|
8.39.465.112 I slot print_timing: id 3 | task 1554 | eval time = 15462.16 ms / 53 tokens ( 291.74 ms per token, 3.43 tokens per second)
|
|
8.39.465.113 I slot print_timing: id 3 | task 1554 | total time = 16397.27 ms / 64 tokens
|
|
8.39.465.113 I slot print_timing: id 3 | task 1554 | graphs reused = 1528
|
|
8.39.465.147 I slot release: id 3 | task 1554 | stop processing: n_tokens = 109, truncated = 0
|
|
8.39.465.156 I srv update_slots: all slots are idle
|
|
8.39.467.912 I srv operator(): Chat format: peg-native
|
|
8.39.468.023 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.763 (> 0.100 thold), f_keep = 0.413
|
|
8.39.468.024 I srv get_availabl: updating prompt cache
|
|
8.39.468.049 W srv prompt_save: - saving prompt with length 109, total state size = 37.472 MiB (draft: 0.000 MiB)
|
|
8.39.500.491 I srv load: - looking for better prompt, base f_keep = 0.413, sim = 0.763
|
|
8.39.500.503 I srv update: - cache state: 13 prompts, 549.014 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
8.39.500.504 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
8.39.500.505 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
8.39.500.505 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
8.39.500.506 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
8.39.500.507 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
8.39.500.511 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
8.39.500.511 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
8.39.500.512 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
8.39.500.512 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
8.39.500.513 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
8.39.500.513 I srv update: - prompt 0x5b0d9a43ddb0: 91 tokens, checkpoints: 0, 31.284 MiB
|
|
8.39.500.514 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
8.39.500.514 I srv update: - prompt 0x5b0d9a4043d0: 109 tokens, checkpoints: 0, 37.472 MiB
|
|
8.39.500.516 I srv get_availabl: prompt cache update took 32.49 ms
|
|
8.39.500.564 I slot launch_slot_: id 3 | task 1608 | processing task, is_child = 0
|
|
8.39.500.564 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
8.39.500.565 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
8.39.500.572 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
8.39.500.572 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
8.39.500.577 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
8.39.500.577 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
9.01.817.370 I slot print_timing: id 3 | task 1608 | prompt eval time = 938.02 ms / 14 tokens ( 67.00 ms per token, 14.93 tokens per second)
|
|
9.01.817.374 I slot print_timing: id 3 | task 1608 | eval time = 21378.76 ms / 73 tokens ( 292.86 ms per token, 3.41 tokens per second)
|
|
9.01.817.375 I slot print_timing: id 3 | task 1608 | total time = 22316.77 ms / 87 tokens
|
|
9.01.817.376 I slot print_timing: id 3 | task 1608 | graphs reused = 1599
|
|
9.01.817.413 I slot release: id 3 | task 1608 | stop processing: n_tokens = 131, truncated = 0
|
|
9.01.817.422 I srv update_slots: all slots are idle
|
|
9.01.820.267 I srv operator(): Chat format: peg-native
|
|
9.01.820.377 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.643 (> 0.100 thold), f_keep = 0.344
|
|
9.01.820.379 I srv get_availabl: updating prompt cache
|
|
9.01.820.410 W srv prompt_save: - saving prompt with length 131, total state size = 45.035 MiB (draft: 0.000 MiB)
|
|
9.01.858.230 I srv load: - looking for better prompt, base f_keep = 0.344, sim = 0.643
|
|
9.01.858.241 I srv load: - found better prompt with f_keep = 0.527, sim = 0.686
|
|
9.01.875.227 I srv update: - cache state: 13 prompts, 562.764 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
9.01.875.233 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
9.01.875.234 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
9.01.875.235 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
9.01.875.236 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
9.01.875.237 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
9.01.875.237 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
9.01.875.238 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
9.01.875.238 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
9.01.875.239 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
9.01.875.239 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
9.01.875.240 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
9.01.875.241 I srv update: - prompt 0x5b0d9a4043d0: 109 tokens, checkpoints: 0, 37.472 MiB
|
|
9.01.875.241 I srv update: - prompt 0x5b0d9a47eb40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
9.01.875.243 I srv get_availabl: prompt cache update took 54.86 ms
|
|
9.01.875.283 I slot launch_slot_: id 3 | task 1682 | processing task, is_child = 0
|
|
9.01.875.284 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
9.01.875.285 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
9.01.875.294 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
9.01.875.294 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
9.01.875.299 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
9.01.875.300 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
9.32.457.002 I slot print_timing: id 3 | task 1682 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s
|
|
9.33.646.536 I slot print_timing: id 3 | task 1682 | prompt eval time = 1140.80 ms / 22 tokens ( 51.85 ms per token, 19.28 tokens per second)
|
|
9.33.646.540 I slot print_timing: id 3 | task 1682 | eval time = 30630.42 ms / 104 tokens ( 294.52 ms per token, 3.40 tokens per second)
|
|
9.33.646.541 I slot print_timing: id 3 | task 1682 | total time = 31771.22 ms / 126 tokens
|
|
9.33.646.541 I slot print_timing: id 3 | task 1682 | graphs reused = 1701
|
|
9.33.646.584 I slot release: id 3 | task 1682 | stop processing: n_tokens = 173, truncated = 0
|
|
9.33.646.592 I srv update_slots: all slots are idle
|
|
9.33.649.537 I srv operator(): Chat format: peg-native
|
|
9.33.649.645 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.754 (> 0.100 thold), f_keep = 0.266
|
|
9.33.649.647 I srv get_availabl: updating prompt cache
|
|
9.33.649.683 W srv prompt_save: - saving prompt with length 173, total state size = 59.473 MiB (draft: 0.000 MiB)
|
|
9.33.698.296 I srv load: - looking for better prompt, base f_keep = 0.266, sim = 0.754
|
|
9.33.698.312 I srv update: - cache state: 14 prompts, 622.237 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
9.33.698.313 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
9.33.698.314 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
9.33.698.314 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
9.33.698.315 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
9.33.698.316 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
9.33.698.316 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
9.33.698.317 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
9.33.698.317 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
9.33.698.318 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
9.33.698.319 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
9.33.698.319 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
9.33.698.320 I srv update: - prompt 0x5b0d9a4043d0: 109 tokens, checkpoints: 0, 37.472 MiB
|
|
9.33.698.320 I srv update: - prompt 0x5b0d9a47eb40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
9.33.698.321 I srv update: - prompt 0x5b0d98ec5f30: 173 tokens, checkpoints: 0, 59.473 MiB
|
|
9.33.698.323 I srv get_availabl: prompt cache update took 48.67 ms
|
|
9.33.698.380 I slot launch_slot_: id 3 | task 1787 | processing task, is_child = 0
|
|
9.33.698.380 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
9.33.698.381 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
9.33.698.390 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
9.33.698.390 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
9.33.698.396 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
9.33.698.396 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
9.51.576.727 I slot print_timing: id 3 | task 1787 | prompt eval time = 937.89 ms / 15 tokens ( 62.53 ms per token, 15.99 tokens per second)
|
|
9.51.576.732 I slot print_timing: id 3 | task 1787 | eval time = 16940.42 ms / 58 tokens ( 292.08 ms per token, 3.42 tokens per second)
|
|
9.51.576.732 I slot print_timing: id 3 | task 1787 | total time = 17878.31 ms / 73 tokens
|
|
9.51.576.733 I slot print_timing: id 3 | task 1787 | graphs reused = 1757
|
|
9.51.576.766 I slot release: id 3 | task 1787 | stop processing: n_tokens = 118, truncated = 0
|
|
9.51.576.773 I srv update_slots: all slots are idle
|
|
9.51.579.429 I srv operator(): Chat format: peg-native
|
|
9.51.579.538 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.804 (> 0.100 thold), f_keep = 0.381
|
|
9.51.579.539 I srv get_availabl: updating prompt cache
|
|
9.51.579.564 W srv prompt_save: - saving prompt with length 118, total state size = 40.566 MiB (draft: 0.000 MiB)
|
|
9.51.614.528 I srv load: - looking for better prompt, base f_keep = 0.381, sim = 0.804
|
|
9.51.614.542 I srv update: - cache state: 15 prompts, 662.803 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
9.51.614.542 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
9.51.614.543 I srv update: - prompt 0x5b0d9a4211f0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
9.51.614.544 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
9.51.614.544 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
9.51.614.545 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
9.51.614.545 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
9.51.614.546 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
9.51.614.546 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
9.51.614.546 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
9.51.614.547 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
9.51.614.547 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
9.51.614.548 I srv update: - prompt 0x5b0d9a4043d0: 109 tokens, checkpoints: 0, 37.472 MiB
|
|
9.51.614.549 I srv update: - prompt 0x5b0d9a47eb40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
9.51.614.549 I srv update: - prompt 0x5b0d98ec5f30: 173 tokens, checkpoints: 0, 59.473 MiB
|
|
9.51.614.554 I srv update: - prompt 0x5b0d9a3fc940: 118 tokens, checkpoints: 0, 40.566 MiB
|
|
9.51.614.555 I srv get_availabl: prompt cache update took 35.01 ms
|
|
9.51.614.608 I slot launch_slot_: id 3 | task 1846 | processing task, is_child = 0
|
|
9.51.614.609 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
9.51.614.610 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
9.51.614.617 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
9.51.614.617 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
9.51.614.622 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
9.51.614.622 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
10.14.842.443 I slot print_timing: id 3 | task 1846 | prompt eval time = 936.83 ms / 11 tokens ( 85.17 ms per token, 11.74 tokens per second)
|
|
10.14.842.447 I slot print_timing: id 3 | task 1846 | eval time = 22290.97 ms / 76 tokens ( 293.30 ms per token, 3.41 tokens per second)
|
|
10.14.842.449 I slot print_timing: id 3 | task 1846 | total time = 23227.80 ms / 87 tokens
|
|
10.14.842.449 I slot print_timing: id 3 | task 1846 | graphs reused = 1831
|
|
10.14.842.483 I slot release: id 3 | task 1846 | stop processing: n_tokens = 131, truncated = 0
|
|
10.14.842.490 I srv update_slots: all slots are idle
|
|
10.14.845.058 I srv operator(): Chat format: peg-native
|
|
10.14.845.168 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.738 (> 0.100 thold), f_keep = 0.344
|
|
10.14.845.170 I srv get_availabl: updating prompt cache
|
|
10.14.845.199 W srv prompt_save: - saving prompt with length 131, total state size = 45.035 MiB (draft: 0.000 MiB)
|
|
10.14.881.987 I srv load: - looking for better prompt, base f_keep = 0.344, sim = 0.738
|
|
10.14.881.997 I srv load: - found better prompt with f_keep = 0.346, sim = 0.754
|
|
10.14.903.114 I srv update: - cache state: 15 prompts, 662.115 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
10.14.903.118 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
10.14.903.119 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
10.14.903.119 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
10.14.903.120 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
10.14.903.120 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
10.14.903.121 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
10.14.903.121 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
10.14.903.122 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
10.14.903.122 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
10.14.903.123 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
10.14.903.123 I srv update: - prompt 0x5b0d9a4043d0: 109 tokens, checkpoints: 0, 37.472 MiB
|
|
10.14.903.124 I srv update: - prompt 0x5b0d9a47eb40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
10.14.903.124 I srv update: - prompt 0x5b0d98ec5f30: 173 tokens, checkpoints: 0, 59.473 MiB
|
|
10.14.903.125 I srv update: - prompt 0x5b0d9a3fc940: 118 tokens, checkpoints: 0, 40.566 MiB
|
|
10.14.903.125 I srv update: - prompt 0x5b0d9a420f40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
10.14.903.126 I srv get_availabl: prompt cache update took 57.95 ms
|
|
10.14.903.164 I slot launch_slot_: id 3 | task 1923 | processing task, is_child = 0
|
|
10.14.903.164 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
10.14.903.165 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
10.14.903.173 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
10.14.903.173 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
10.14.903.177 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
10.14.903.177 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
10.37.233.811 I slot print_timing: id 3 | task 1923 | prompt eval time = 938.54 ms / 15 tokens ( 62.57 ms per token, 15.98 tokens per second)
|
|
10.37.233.814 I slot print_timing: id 3 | task 1923 | eval time = 21392.07 ms / 73 tokens ( 293.04 ms per token, 3.41 tokens per second)
|
|
10.37.233.815 I slot print_timing: id 3 | task 1923 | total time = 22330.61 ms / 88 tokens
|
|
10.37.233.815 I slot print_timing: id 3 | task 1923 | graphs reused = 1902
|
|
10.37.233.846 I slot release: id 3 | task 1923 | stop processing: n_tokens = 133, truncated = 0
|
|
10.37.233.854 I srv update_slots: all slots are idle
|
|
10.37.236.520 I srv operator(): Chat format: peg-native
|
|
10.37.236.639 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.754 (> 0.100 thold), f_keep = 0.346
|
|
10.37.236.641 I srv get_availabl: updating prompt cache
|
|
10.37.236.673 W srv prompt_save: - saving prompt with length 133, total state size = 45.722 MiB (draft: 0.000 MiB)
|
|
10.37.274.784 I srv load: - looking for better prompt, base f_keep = 0.346, sim = 0.754
|
|
10.37.274.797 I srv update: - cache state: 16 prompts, 707.837 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
10.37.274.798 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
10.37.274.799 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
10.37.274.801 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
10.37.274.802 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
10.37.274.802 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
10.37.274.803 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
10.37.274.803 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
10.37.274.804 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
10.37.274.804 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
10.37.274.804 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
10.37.274.812 I srv update: - prompt 0x5b0d9a4043d0: 109 tokens, checkpoints: 0, 37.472 MiB
|
|
10.37.274.813 I srv update: - prompt 0x5b0d9a47eb40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
10.37.274.813 I srv update: - prompt 0x5b0d98ec5f30: 173 tokens, checkpoints: 0, 59.473 MiB
|
|
10.37.274.814 I srv update: - prompt 0x5b0d9a3fc940: 118 tokens, checkpoints: 0, 40.566 MiB
|
|
10.37.274.814 I srv update: - prompt 0x5b0d9a420f40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
10.37.274.815 I srv update: - prompt 0x5b0d9a3c20e0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
10.37.274.816 I srv get_availabl: prompt cache update took 38.17 ms
|
|
10.37.274.863 I slot launch_slot_: id 3 | task 1997 | processing task, is_child = 0
|
|
10.37.274.863 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
10.37.274.864 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
10.37.274.871 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
10.37.274.871 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
10.37.274.876 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
10.37.274.876 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
11.07.643.479 I slot print_timing: id 3 | task 1997 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s
|
|
11.10.914.615 I slot print_timing: id 3 | task 1997 | n_decoded = 111, tg = 3.39 t/s, tg_3s = 3.36 t/s
|
|
11.14.189.632 I slot print_timing: id 3 | task 1997 | n_decoded = 122, tg = 3.39 t/s, tg_3s = 3.36 t/s
|
|
11.16.868.798 I slot print_timing: id 3 | task 1997 | prompt eval time = 938.67 ms / 15 tokens ( 62.58 ms per token, 15.98 tokens per second)
|
|
11.16.868.801 I slot print_timing: id 3 | task 1997 | eval time = 38655.24 ms / 131 tokens ( 295.08 ms per token, 3.39 tokens per second)
|
|
11.16.868.802 I slot print_timing: id 3 | task 1997 | total time = 39593.90 ms / 146 tokens
|
|
11.16.868.802 I slot print_timing: id 3 | task 1997 | graphs reused = 2031
|
|
11.16.868.841 I slot release: id 3 | task 1997 | stop processing: n_tokens = 191, truncated = 0
|
|
11.16.868.849 I srv update_slots: all slots are idle
|
|
11.16.871.384 I srv operator(): Chat format: peg-native
|
|
11.16.871.489 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.818 (> 0.100 thold), f_keep = 0.236
|
|
11.16.871.491 I srv get_availabl: updating prompt cache
|
|
11.16.871.522 W srv prompt_save: - saving prompt with length 191, total state size = 65.660 MiB (draft: 0.000 MiB)
|
|
11.16.923.021 I srv load: - looking for better prompt, base f_keep = 0.236, sim = 0.818
|
|
11.16.923.031 I srv load: - found better prompt with f_keep = 0.359, sim = 0.855
|
|
11.16.944.116 I srv update: - cache state: 16 prompts, 728.463 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
11.16.944.120 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
11.16.944.121 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
11.16.944.121 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
11.16.944.122 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
11.16.944.122 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
11.16.944.123 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
11.16.944.123 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
11.16.944.124 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
11.16.944.124 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
11.16.944.124 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
11.16.944.125 I srv update: - prompt 0x5b0d9a4043d0: 109 tokens, checkpoints: 0, 37.472 MiB
|
|
11.16.944.125 I srv update: - prompt 0x5b0d9a47eb40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
11.16.944.126 I srv update: - prompt 0x5b0d98ec5f30: 173 tokens, checkpoints: 0, 59.473 MiB
|
|
11.16.944.126 I srv update: - prompt 0x5b0d9a3fc940: 118 tokens, checkpoints: 0, 40.566 MiB
|
|
11.16.944.127 I srv update: - prompt 0x5b0d9a3c20e0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
11.16.944.127 I srv update: - prompt 0x5b0d9a58fee0: 191 tokens, checkpoints: 0, 65.660 MiB
|
|
11.16.944.129 I srv get_availabl: prompt cache update took 72.64 ms
|
|
11.16.944.164 I slot launch_slot_: id 3 | task 2129 | processing task, is_child = 0
|
|
11.16.944.164 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
11.16.944.165 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
11.16.944.173 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
11.16.944.173 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
11.16.944.178 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
11.16.944.178 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
11.32.993.709 I slot print_timing: id 3 | task 2129 | prompt eval time = 2377.82 ms / 8 tokens ( 297.23 ms per token, 3.36 tokens per second)
|
|
11.32.993.712 I slot print_timing: id 3 | task 2129 | eval time = 13671.69 ms / 47 tokens ( 290.89 ms per token, 3.44 tokens per second)
|
|
11.32.993.713 I slot print_timing: id 3 | task 2129 | total time = 16049.51 ms / 55 tokens
|
|
11.32.993.714 I slot print_timing: id 3 | task 2129 | graphs reused = 2076
|
|
11.32.993.744 I slot release: id 3 | task 2129 | stop processing: n_tokens = 101, truncated = 0
|
|
11.32.993.751 I srv update_slots: all slots are idle
|
|
11.32.996.464 I srv operator(): Chat format: peg-native
|
|
11.32.996.576 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.865 (> 0.100 thold), f_keep = 0.446
|
|
11.32.996.578 I srv get_availabl: updating prompt cache
|
|
11.32.996.607 W srv prompt_save: - saving prompt with length 101, total state size = 34.722 MiB (draft: 0.000 MiB)
|
|
11.33.027.227 I srv load: - looking for better prompt, base f_keep = 0.446, sim = 0.865
|
|
11.33.027.238 I srv update: - cache state: 17 prompts, 763.185 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
11.33.027.239 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
11.33.027.240 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
11.33.027.241 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
11.33.027.245 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
11.33.027.245 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
11.33.027.246 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
11.33.027.246 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
11.33.027.247 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
11.33.027.247 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
11.33.027.248 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
11.33.027.249 I srv update: - prompt 0x5b0d9a4043d0: 109 tokens, checkpoints: 0, 37.472 MiB
|
|
11.33.027.261 I srv update: - prompt 0x5b0d9a47eb40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
11.33.027.262 I srv update: - prompt 0x5b0d98ec5f30: 173 tokens, checkpoints: 0, 59.473 MiB
|
|
11.33.027.262 I srv update: - prompt 0x5b0d9a3fc940: 118 tokens, checkpoints: 0, 40.566 MiB
|
|
11.33.027.263 I srv update: - prompt 0x5b0d9a3c20e0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
11.33.027.263 I srv update: - prompt 0x5b0d9a58fee0: 191 tokens, checkpoints: 0, 65.660 MiB
|
|
11.33.027.264 I srv update: - prompt 0x5b0d9a3dae90: 101 tokens, checkpoints: 0, 34.722 MiB
|
|
11.33.027.264 I srv get_availabl: prompt cache update took 30.68 ms
|
|
11.33.027.312 I slot launch_slot_: id 3 | task 2177 | processing task, is_child = 0
|
|
11.33.027.313 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
11.33.027.314 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
11.33.027.322 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
11.33.027.322 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
11.33.027.326 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
11.33.027.327 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
12.01.260.160 I slot print_timing: id 3 | task 2177 | prompt eval time = 2058.13 ms / 7 tokens ( 294.02 ms per token, 3.40 tokens per second)
|
|
12.01.260.164 I slot print_timing: id 3 | task 2177 | eval time = 26174.69 ms / 89 tokens ( 294.10 ms per token, 3.40 tokens per second)
|
|
12.01.260.164 I slot print_timing: id 3 | task 2177 | total time = 28232.82 ms / 96 tokens
|
|
12.01.260.165 I slot print_timing: id 3 | task 2177 | graphs reused = 2163
|
|
12.01.260.196 I slot release: id 3 | task 2177 | stop processing: n_tokens = 140, truncated = 0
|
|
12.01.260.204 I srv update_slots: all slots are idle
|
|
12.01.262.959 I srv operator(): Chat format: peg-native
|
|
12.01.263.067 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.321
|
|
12.01.263.069 I srv get_availabl: updating prompt cache
|
|
12.01.263.095 W srv prompt_save: - saving prompt with length 140, total state size = 48.129 MiB (draft: 0.000 MiB)
|
|
12.01.302.856 I srv load: - looking for better prompt, base f_keep = 0.321, sim = 0.776
|
|
12.01.302.871 I srv update: - cache state: 18 prompts, 811.314 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
12.01.302.872 I srv update: - prompt 0x5b0d9a476e80: 160 tokens, checkpoints: 0, 55.004 MiB
|
|
12.01.302.876 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
12.01.302.876 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
12.01.302.877 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
12.01.302.877 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
12.01.302.878 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
12.01.302.879 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
12.01.302.879 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
12.01.302.880 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
12.01.302.884 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
12.01.302.884 I srv update: - prompt 0x5b0d9a4043d0: 109 tokens, checkpoints: 0, 37.472 MiB
|
|
12.01.302.885 I srv update: - prompt 0x5b0d9a47eb40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
12.01.302.885 I srv update: - prompt 0x5b0d98ec5f30: 173 tokens, checkpoints: 0, 59.473 MiB
|
|
12.01.302.886 I srv update: - prompt 0x5b0d9a3fc940: 118 tokens, checkpoints: 0, 40.566 MiB
|
|
12.01.302.886 I srv update: - prompt 0x5b0d9a3c20e0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
12.01.302.887 I srv update: - prompt 0x5b0d9a58fee0: 191 tokens, checkpoints: 0, 65.660 MiB
|
|
12.01.302.888 I srv update: - prompt 0x5b0d9a3dae90: 101 tokens, checkpoints: 0, 34.722 MiB
|
|
12.01.302.891 I srv update: - prompt 0x5b0d9a129270: 140 tokens, checkpoints: 0, 48.129 MiB
|
|
12.01.302.892 I srv get_availabl: prompt cache update took 39.82 ms
|
|
12.01.302.938 I slot launch_slot_: id 3 | task 2267 | processing task, is_child = 0
|
|
12.01.302.939 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
12.01.302.940 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
12.01.302.946 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
12.01.302.946 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
12.01.302.951 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
12.01.302.951 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
12.31.685.371 I slot print_timing: id 3 | task 2267 | n_decoded = 100, tg = 3.40 t/s, tg_3s = 3.40 t/s
|
|
12.34.363.005 I slot print_timing: id 3 | task 2267 | prompt eval time = 936.90 ms / 13 tokens ( 72.07 ms per token, 13.88 tokens per second)
|
|
12.34.363.009 I slot print_timing: id 3 | task 2267 | eval time = 32123.13 ms / 109 tokens ( 294.71 ms per token, 3.39 tokens per second)
|
|
12.34.363.009 I slot print_timing: id 3 | task 2267 | total time = 33060.04 ms / 122 tokens
|
|
12.34.363.010 I slot print_timing: id 3 | task 2267 | graphs reused = 2270
|
|
12.34.363.040 I slot release: id 3 | task 2267 | stop processing: n_tokens = 166, truncated = 0
|
|
12.34.363.048 I srv update_slots: all slots are idle
|
|
12.34.365.643 I srv operator(): Chat format: peg-native
|
|
12.34.365.745 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.804 (> 0.100 thold), f_keep = 0.271
|
|
12.34.365.746 I srv get_availabl: updating prompt cache
|
|
12.34.365.772 W srv prompt_save: - saving prompt with length 166, total state size = 57.066 MiB (draft: 0.000 MiB)
|
|
12.34.411.391 I srv load: - looking for better prompt, base f_keep = 0.271, sim = 0.804
|
|
12.34.411.404 I srv load: - found better prompt with f_keep = 0.300, sim = 0.857
|
|
12.34.436.016 I srv update: - cache state: 18 prompts, 813.376 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
12.34.436.020 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
12.34.436.020 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
12.34.436.021 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
12.34.436.021 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
12.34.436.022 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
12.34.436.022 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
12.34.436.023 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
12.34.436.023 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
12.34.436.024 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
12.34.436.024 I srv update: - prompt 0x5b0d9a4043d0: 109 tokens, checkpoints: 0, 37.472 MiB
|
|
12.34.436.025 I srv update: - prompt 0x5b0d9a47eb40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
12.34.436.026 I srv update: - prompt 0x5b0d98ec5f30: 173 tokens, checkpoints: 0, 59.473 MiB
|
|
12.34.436.026 I srv update: - prompt 0x5b0d9a3fc940: 118 tokens, checkpoints: 0, 40.566 MiB
|
|
12.34.436.027 I srv update: - prompt 0x5b0d9a3c20e0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
12.34.436.027 I srv update: - prompt 0x5b0d9a58fee0: 191 tokens, checkpoints: 0, 65.660 MiB
|
|
12.34.436.028 I srv update: - prompt 0x5b0d9a3dae90: 101 tokens, checkpoints: 0, 34.722 MiB
|
|
12.34.436.028 I srv update: - prompt 0x5b0d9a129270: 140 tokens, checkpoints: 0, 48.129 MiB
|
|
12.34.436.029 I srv update: - prompt 0x5b0d9b4a3360: 166 tokens, checkpoints: 0, 57.066 MiB
|
|
12.34.436.030 I srv get_availabl: prompt cache update took 70.28 ms
|
|
12.34.436.065 I slot launch_slot_: id 3 | task 2377 | processing task, is_child = 0
|
|
12.34.436.066 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
12.34.436.067 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
12.34.436.075 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
12.34.436.075 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
12.34.436.080 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
12.34.436.080 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
12.54.384.703 I slot print_timing: id 3 | task 2377 | prompt eval time = 2374.62 ms / 8 tokens ( 296.83 ms per token, 3.37 tokens per second)
|
|
12.54.384.706 I slot print_timing: id 3 | task 2377 | eval time = 17573.98 ms / 60 tokens ( 292.90 ms per token, 3.41 tokens per second)
|
|
12.54.384.707 I slot print_timing: id 3 | task 2377 | total time = 19948.60 ms / 68 tokens
|
|
12.54.384.707 I slot print_timing: id 3 | task 2377 | graphs reused = 2328
|
|
12.54.384.738 I slot release: id 3 | task 2377 | stop processing: n_tokens = 115, truncated = 0
|
|
12.54.384.745 I srv update_slots: all slots are idle
|
|
12.54.387.543 I srv operator(): Chat format: peg-native
|
|
12.54.387.660 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.836 (> 0.100 thold), f_keep = 0.400
|
|
12.54.387.662 I srv get_availabl: updating prompt cache
|
|
12.54.387.687 W srv prompt_save: - saving prompt with length 115, total state size = 39.535 MiB (draft: 0.000 MiB)
|
|
12.54.421.414 I srv load: - looking for better prompt, base f_keep = 0.400, sim = 0.836
|
|
12.54.421.430 I srv update: - cache state: 19 prompts, 852.911 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
12.54.421.432 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
12.54.421.435 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
12.54.421.436 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
12.54.421.436 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
12.54.421.437 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
12.54.421.437 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
12.54.421.438 I srv update: - prompt 0x5b0d9a473550: 116 tokens, checkpoints: 0, 39.878 MiB
|
|
12.54.421.438 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
12.54.421.441 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
12.54.421.441 I srv update: - prompt 0x5b0d9a4043d0: 109 tokens, checkpoints: 0, 37.472 MiB
|
|
12.54.421.442 I srv update: - prompt 0x5b0d9a47eb40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
12.54.421.442 I srv update: - prompt 0x5b0d98ec5f30: 173 tokens, checkpoints: 0, 59.473 MiB
|
|
12.54.421.443 I srv update: - prompt 0x5b0d9a3fc940: 118 tokens, checkpoints: 0, 40.566 MiB
|
|
12.54.421.443 I srv update: - prompt 0x5b0d9a3c20e0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
12.54.421.445 I srv update: - prompt 0x5b0d9a58fee0: 191 tokens, checkpoints: 0, 65.660 MiB
|
|
12.54.421.446 I srv update: - prompt 0x5b0d9a3dae90: 101 tokens, checkpoints: 0, 34.722 MiB
|
|
12.54.421.446 I srv update: - prompt 0x5b0d9a129270: 140 tokens, checkpoints: 0, 48.129 MiB
|
|
12.54.421.447 I srv update: - prompt 0x5b0d9b4a3360: 166 tokens, checkpoints: 0, 57.066 MiB
|
|
12.54.421.447 I srv update: - prompt 0x5b0d9b3e73c0: 115 tokens, checkpoints: 0, 39.535 MiB
|
|
12.54.421.449 I srv get_availabl: prompt cache update took 33.79 ms
|
|
12.54.421.496 I slot launch_slot_: id 3 | task 2438 | processing task, is_child = 0
|
|
12.54.421.497 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
12.54.421.498 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
12.54.421.506 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
12.54.421.506 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
12.54.421.510 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
12.54.421.511 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
13.17.665.161 I slot print_timing: id 3 | task 2438 | prompt eval time = 936.02 ms / 9 tokens ( 104.00 ms per token, 9.62 tokens per second)
|
|
13.17.665.165 I slot print_timing: id 3 | task 2438 | eval time = 22307.61 ms / 76 tokens ( 293.52 ms per token, 3.41 tokens per second)
|
|
13.17.665.165 I slot print_timing: id 3 | task 2438 | total time = 23243.63 ms / 85 tokens
|
|
13.17.665.166 I slot print_timing: id 3 | task 2438 | graphs reused = 2402
|
|
13.17.665.199 I slot release: id 3 | task 2438 | stop processing: n_tokens = 130, truncated = 0
|
|
13.17.665.205 I srv update_slots: all slots are idle
|
|
13.17.667.938 I srv operator(): Chat format: peg-native
|
|
13.17.668.045 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.346
|
|
13.17.668.047 I srv get_availabl: updating prompt cache
|
|
13.17.668.073 W srv prompt_save: - saving prompt with length 130, total state size = 44.691 MiB (draft: 0.000 MiB)
|
|
13.17.705.013 I srv load: - looking for better prompt, base f_keep = 0.346, sim = 0.776
|
|
13.17.705.024 I srv load: - found better prompt with f_keep = 0.397, sim = 0.793
|
|
13.17.724.530 I srv update: - cache state: 19 prompts, 857.724 MiB (limits: 8192.000 MiB, 8192 tokens, 23829 est)
|
|
13.17.724.534 I srv update: - prompt 0x5b0d9a3f4a60: 143 tokens, checkpoints: 0, 49.160 MiB
|
|
13.17.724.535 I srv update: - prompt 0x5b0d9a17b8a0: 103 tokens, checkpoints: 0, 35.409 MiB
|
|
13.17.724.535 I srv update: - prompt 0x5b0d9a192ed0: 129 tokens, checkpoints: 0, 44.347 MiB
|
|
13.17.724.536 I srv update: - prompt 0x5b0d9a0d8ee0: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
13.17.724.536 I srv update: - prompt 0x5b0d9b2799c0: 127 tokens, checkpoints: 0, 43.660 MiB
|
|
13.17.724.537 I srv update: - prompt 0x5b0d9b3bafd0: 104 tokens, checkpoints: 0, 35.753 MiB
|
|
13.17.724.537 I srv update: - prompt 0x5b0d9a490370: 125 tokens, checkpoints: 0, 42.972 MiB
|
|
13.17.724.537 I srv update: - prompt 0x5b0d9a44de50: 126 tokens, checkpoints: 0, 43.316 MiB
|
|
13.17.724.538 I srv update: - prompt 0x5b0d9a4043d0: 109 tokens, checkpoints: 0, 37.472 MiB
|
|
13.17.724.538 I srv update: - prompt 0x5b0d9a47eb40: 131 tokens, checkpoints: 0, 45.035 MiB
|
|
13.17.724.539 I srv update: - prompt 0x5b0d98ec5f30: 173 tokens, checkpoints: 0, 59.473 MiB
|
|
13.17.724.539 I srv update: - prompt 0x5b0d9a3fc940: 118 tokens, checkpoints: 0, 40.566 MiB
|
|
13.17.724.540 I srv update: - prompt 0x5b0d9a3c20e0: 133 tokens, checkpoints: 0, 45.722 MiB
|
|
13.17.724.540 I srv update: - prompt 0x5b0d9a58fee0: 191 tokens, checkpoints: 0, 65.660 MiB
|
|
13.17.724.541 I srv update: - prompt 0x5b0d9a3dae90: 101 tokens, checkpoints: 0, 34.722 MiB
|
|
13.17.724.541 I srv update: - prompt 0x5b0d9a129270: 140 tokens, checkpoints: 0, 48.129 MiB
|
|
13.17.724.542 I srv update: - prompt 0x5b0d9b4a3360: 166 tokens, checkpoints: 0, 57.066 MiB
|
|
13.17.724.543 I srv update: - prompt 0x5b0d9b3e73c0: 115 tokens, checkpoints: 0, 39.535 MiB
|
|
13.17.724.546 I srv update: - prompt 0x5b0d9a1bd0f0: 130 tokens, checkpoints: 0, 44.691 MiB
|
|
13.17.724.548 I srv get_availabl: prompt cache update took 56.50 ms
|
|
13.17.724.581 I slot launch_slot_: id 3 | task 2515 | processing task, is_child = 0
|
|
13.17.724.582 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
13.17.724.582 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
13.17.724.590 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
13.17.724.590 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
13.17.724.594 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
13.17.724.595 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
13.28.183.957 I slot print_timing: id 3 | task 2515 | prompt eval time = 937.48 ms / 12 tokens ( 78.12 ms per token, 12.80 tokens per second)
|
|
13.28.183.961 I slot print_timing: id 3 | task 2515 | eval time = 9521.87 ms / 33 tokens ( 288.54 ms per token, 3.47 tokens per second)
|
|
13.28.183.961 I slot print_timing: id 3 | task 2515 | total time = 10459.34 ms / 45 tokens
|
|
13.28.183.962 I slot print_timing: id 3 | task 2515 | graphs reused = 2433
|
|
13.28.183.995 I slot release: id 3 | task 2515 | stop processing: n_tokens = 90, truncated = 0
|
|
13.28.184.003 I srv update_slots: all slots are idle
|
|
13.28.186.793 I srv operator(): Chat format: peg-native
|
|
13.28.186.905 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.500
|
|
13.28.186.926 I slot launch_slot_: id 3 | task 2549 | processing task, is_child = 0
|
|
13.28.186.927 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
13.28.186.928 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
13.28.186.934 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
13.28.186.934 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
13.28.186.939 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
13.28.186.940 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
13.57.659.028 I slot print_timing: id 3 | task 2549 | prompt eval time = 936.10 ms / 9 tokens ( 104.01 ms per token, 9.61 tokens per second)
|
|
13.57.659.032 I slot print_timing: id 3 | task 2549 | eval time = 28535.98 ms / 97 tokens ( 294.19 ms per token, 3.40 tokens per second)
|
|
13.57.659.033 I slot print_timing: id 3 | task 2549 | total time = 29472.07 ms / 106 tokens
|
|
13.57.659.034 I slot print_timing: id 3 | task 2549 | graphs reused = 2528
|
|
13.57.659.078 I slot release: id 3 | task 2549 | stop processing: n_tokens = 150, truncated = 0
|
|
13.57.659.087 I srv update_slots: all slots are idle
|
|
13.57.659.590 I srv operator(): operator(): cleaning up before exit...
|