696 lines
67 KiB
Plaintext
696 lines
67 KiB
Plaintext
0.00.065.314 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
|
|
0.00.065.317 I device_info:
|
|
0.00.065.337 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
|
0.00.065.341 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
|
0.00.065.348 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
|
|
0.00.065.402 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
|
0.00.065.406 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
|
|
0.00.065.445 I srv init: running without SSL
|
|
0.00.065.506 I srv init: using 23 threads for HTTP server
|
|
0.00.065.838 I srv start: binding port with default address family
|
|
0.00.067.051 I srv llama_server: loading model
|
|
0.00.067.059 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/mistralai_Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf'
|
|
0.00.067.108 I common_init_result: fitting params to device memory ...
|
|
0.00.067.111 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
|
|
0.11.729.845 W llama_context: setting new yarn_attn_factor = 1.0000 (mscale == 1.0, mscale_all_dim = 1.0)
|
|
0.11.729.853 W llama_context: n_ctx_seq (24576) < n_ctx_train (393216) -- the full capacity of the model will not be utilized
|
|
0.11.808.276 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
|
0.11.972.561 I srv load_model: initializing slots, n_slots = 4
|
|
0.12.039.555 W common_speculative_init: no implementations specified for speculative decoding
|
|
0.12.039.558 I slot load_model: id 0 | task -1 | new slot, n_ctx = 24576
|
|
0.12.039.561 I slot load_model: id 1 | task -1 | new slot, n_ctx = 24576
|
|
0.12.039.561 I slot load_model: id 2 | task -1 | new slot, n_ctx = 24576
|
|
0.12.039.562 I slot load_model: id 3 | task -1 | new slot, n_ctx = 24576
|
|
0.12.039.727 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
|
|
0.12.039.728 I srv load_model: use `--cache-ram 0` to disable the prompt cache
|
|
0.12.039.728 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
|
0.12.039.729 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
|
|
0.12.039.743 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
|
|
0.12.042.716 I init: chat template, example_format: '[SYSTEM_PROMPT]You are a helpful assistant[/SYSTEM_PROMPT][INST]Hello[/INST]Hi there</s>[INST]How are you?[/INST]'
|
|
0.12.043.124 I srv init: init: chat template, thinking = 1
|
|
0.12.043.156 I srv llama_server: model loaded
|
|
0.12.043.160 I srv llama_server: server is listening on http://127.0.0.1:8090
|
|
0.12.043.166 I srv update_slots: all slots are idle
|
|
0.14.966.006 I srv operator(): Chat format: peg-native
|
|
0.14.966.181 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
|
|
0.14.966.184 I srv get_availabl: updating prompt cache
|
|
0.14.966.190 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
|
0.14.966.196 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 24576 tokens, 8589934592 est)
|
|
0.14.966.196 I srv get_availabl: prompt cache update took 0.01 ms
|
|
0.14.966.232 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
|
|
0.14.966.233 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.14.966.234 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.14.966.259 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.14.966.260 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.14.966.280 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.14.966.281 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.17.878.841 I slot print_timing: id 3 | task 0 | prompt eval time = 328.44 ms / 58 tokens ( 5.66 ms per token, 176.59 tokens per second)
|
|
0.17.878.845 I slot print_timing: id 3 | task 0 | eval time = 2584.08 ms / 64 tokens ( 40.38 ms per token, 24.77 tokens per second)
|
|
0.17.878.846 I slot print_timing: id 3 | task 0 | total time = 2912.52 ms / 122 tokens
|
|
0.17.878.849 I slot print_timing: id 3 | task 0 | graphs reused = 0
|
|
0.17.878.884 I slot release: id 3 | task 0 | stop processing: n_tokens = 121, truncated = 0
|
|
0.17.878.888 I srv update_slots: all slots are idle
|
|
0.17.881.421 I srv operator(): Chat format: peg-native
|
|
0.17.881.534 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.672 (> 0.100 thold), f_keep = 0.339
|
|
0.17.881.535 I srv get_availabl: updating prompt cache
|
|
0.17.881.574 W srv prompt_save: - saving prompt with length 121, total state size = 18.909 MiB (draft: 0.000 MiB)
|
|
0.17.898.222 I srv load: - looking for better prompt, base f_keep = 0.339, sim = 0.672
|
|
0.17.898.230 I srv update: - cache state: 1 prompts, 18.909 MiB (limits: 8192.000 MiB, 24576 tokens, 52422 est)
|
|
0.17.898.231 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
0.17.898.232 I srv get_availabl: prompt cache update took 16.70 ms
|
|
0.17.898.266 I slot launch_slot_: id 3 | task 65 | processing task, is_child = 0
|
|
0.17.898.267 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.17.898.267 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.17.898.287 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.17.898.287 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.17.898.299 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.17.898.299 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.18.152.614 I slot print_timing: id 3 | task 65 | prompt eval time = 213.34 ms / 20 tokens ( 10.67 ms per token, 93.74 tokens per second)
|
|
0.18.152.617 I slot print_timing: id 3 | task 65 | eval time = 40.95 ms / 2 tokens ( 20.47 ms per token, 48.85 tokens per second)
|
|
0.18.152.617 I slot print_timing: id 3 | task 65 | total time = 254.29 ms / 22 tokens
|
|
0.18.152.618 I slot print_timing: id 3 | task 65 | graphs reused = 0
|
|
0.18.152.638 I slot release: id 3 | task 65 | stop processing: n_tokens = 62, truncated = 0
|
|
0.18.152.646 I srv update_slots: all slots are idle
|
|
0.18.155.236 I srv operator(): Chat format: peg-native
|
|
0.18.155.359 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.506 (> 0.100 thold), f_keep = 0.661
|
|
0.18.155.373 I slot launch_slot_: id 3 | task 68 | processing task, is_child = 0
|
|
0.18.155.374 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.18.155.374 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.18.155.390 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.18.155.390 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.18.155.402 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.18.155.403 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.19.710.820 I slot print_timing: id 3 | task 68 | prompt eval time = 283.01 ms / 40 tokens ( 7.08 ms per token, 141.34 tokens per second)
|
|
0.19.710.823 I slot print_timing: id 3 | task 68 | eval time = 1272.38 ms / 32 tokens ( 39.76 ms per token, 25.15 tokens per second)
|
|
0.19.710.824 I slot print_timing: id 3 | task 68 | total time = 1555.40 ms / 72 tokens
|
|
0.19.710.825 I slot print_timing: id 3 | task 68 | graphs reused = 0
|
|
0.19.710.842 I slot release: id 3 | task 68 | stop processing: n_tokens = 112, truncated = 0
|
|
0.19.710.849 I srv update_slots: all slots are idle
|
|
0.19.713.219 I srv operator(): Chat format: peg-native
|
|
0.19.713.328 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.651 (> 0.100 thold), f_keep = 0.366
|
|
0.19.713.330 I srv get_availabl: updating prompt cache
|
|
0.19.713.354 W srv prompt_save: - saving prompt with length 112, total state size = 17.502 MiB (draft: 0.000 MiB)
|
|
0.19.728.675 I srv load: - looking for better prompt, base f_keep = 0.366, sim = 0.651
|
|
0.19.728.685 I srv update: - cache state: 2 prompts, 36.411 MiB (limits: 8192.000 MiB, 24576 tokens, 52422 est)
|
|
0.19.728.686 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
0.19.728.686 I srv update: - prompt 0x587509e27710: 112 tokens, checkpoints: 0, 17.502 MiB
|
|
0.19.728.688 I srv get_availabl: prompt cache update took 15.36 ms
|
|
0.19.728.719 I slot launch_slot_: id 3 | task 101 | processing task, is_child = 0
|
|
0.19.728.720 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.19.728.721 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.19.728.740 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.19.728.741 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.19.728.757 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.19.728.758 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.24.004.925 I slot print_timing: id 3 | task 101 | n_decoded = 100, tg = 24.61 t/s, tg_3s = 24.61 t/s
|
|
0.27.043.876 I slot print_timing: id 3 | task 101 | n_decoded = 174, tg = 24.50 t/s, tg_3s = 24.35 t/s
|
|
0.29.189.677 I slot print_timing: id 3 | task 101 | prompt eval time = 213.31 ms / 22 tokens ( 9.70 ms per token, 103.13 tokens per second)
|
|
0.29.189.680 I slot print_timing: id 3 | task 101 | eval time = 9247.58 ms / 226 tokens ( 40.92 ms per token, 24.44 tokens per second)
|
|
0.29.189.681 I slot print_timing: id 3 | task 101 | total time = 9460.90 ms / 248 tokens
|
|
0.29.189.682 I slot print_timing: id 3 | task 101 | graphs reused = 0
|
|
0.29.189.700 I slot release: id 3 | task 101 | stop processing: n_tokens = 288, truncated = 0
|
|
0.29.189.707 I srv update_slots: all slots are idle
|
|
0.29.192.282 I srv operator(): Chat format: peg-native
|
|
0.29.192.387 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.695 (> 0.100 thold), f_keep = 0.142
|
|
0.29.192.389 I srv get_availabl: updating prompt cache
|
|
0.29.192.419 W srv prompt_save: - saving prompt with length 288, total state size = 45.004 MiB (draft: 0.000 MiB)
|
|
0.29.228.076 I srv load: - looking for better prompt, base f_keep = 0.142, sim = 0.695
|
|
0.29.228.086 I srv update: - cache state: 3 prompts, 81.415 MiB (limits: 8192.000 MiB, 24576 tokens, 52423 est)
|
|
0.29.228.087 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
0.29.228.087 I srv update: - prompt 0x587509e27710: 112 tokens, checkpoints: 0, 17.502 MiB
|
|
0.29.228.088 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
0.29.228.089 I srv get_availabl: prompt cache update took 35.70 ms
|
|
0.29.228.127 I slot launch_slot_: id 3 | task 328 | processing task, is_child = 0
|
|
0.29.228.128 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.29.228.128 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.29.228.148 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.29.228.148 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.29.228.160 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.29.228.161 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.30.017.674 I slot print_timing: id 3 | task 328 | prompt eval time = 212.69 ms / 18 tokens ( 11.82 ms per token, 84.63 tokens per second)
|
|
0.30.017.677 I slot print_timing: id 3 | task 328 | eval time = 576.80 ms / 15 tokens ( 38.45 ms per token, 26.01 tokens per second)
|
|
0.30.017.677 I slot print_timing: id 3 | task 328 | total time = 789.49 ms / 33 tokens
|
|
0.30.017.678 I slot print_timing: id 3 | task 328 | graphs reused = 0
|
|
0.30.017.694 I slot release: id 3 | task 328 | stop processing: n_tokens = 73, truncated = 0
|
|
0.30.017.700 I srv update_slots: all slots are idle
|
|
0.30.020.213 I srv operator(): Chat format: peg-native
|
|
0.30.020.314 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.519 (> 0.100 thold), f_keep = 0.562
|
|
0.30.020.331 I slot launch_slot_: id 3 | task 344 | processing task, is_child = 0
|
|
0.30.020.331 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.30.020.332 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.30.020.349 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.30.020.349 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.30.020.361 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.30.020.361 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.31.951.949 I slot print_timing: id 3 | task 344 | prompt eval time = 282.87 ms / 38 tokens ( 7.44 ms per token, 134.34 tokens per second)
|
|
0.31.951.952 I slot print_timing: id 3 | task 344 | eval time = 1648.70 ms / 41 tokens ( 40.21 ms per token, 24.87 tokens per second)
|
|
0.31.951.953 I slot print_timing: id 3 | task 344 | total time = 1931.57 ms / 79 tokens
|
|
0.31.951.953 I slot print_timing: id 3 | task 344 | graphs reused = 0
|
|
0.31.951.973 I slot release: id 3 | task 344 | stop processing: n_tokens = 119, truncated = 0
|
|
0.31.951.979 I srv update_slots: all slots are idle
|
|
0.31.954.321 I srv operator(): Chat format: peg-native
|
|
0.31.954.425 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.547 (> 0.100 thold), f_keep = 0.345
|
|
0.31.954.427 I srv get_availabl: updating prompt cache
|
|
0.31.954.449 W srv prompt_save: - saving prompt with length 119, total state size = 18.596 MiB (draft: 0.000 MiB)
|
|
0.31.970.497 I srv load: - looking for better prompt, base f_keep = 0.345, sim = 0.547
|
|
0.31.970.503 I srv update: - cache state: 4 prompts, 100.011 MiB (limits: 8192.000 MiB, 24576 tokens, 52422 est)
|
|
0.31.970.504 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
0.31.970.505 I srv update: - prompt 0x587509e27710: 112 tokens, checkpoints: 0, 17.502 MiB
|
|
0.31.970.505 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
0.31.970.506 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
0.31.970.507 I srv get_availabl: prompt cache update took 16.08 ms
|
|
0.31.970.538 I slot launch_slot_: id 3 | task 386 | processing task, is_child = 0
|
|
0.31.970.539 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.31.970.540 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.31.970.559 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.31.970.560 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.31.970.572 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.31.970.572 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.34.147.611 I slot print_timing: id 3 | task 386 | prompt eval time = 282.35 ms / 34 tokens ( 8.30 ms per token, 120.42 tokens per second)
|
|
0.34.147.615 I slot print_timing: id 3 | task 386 | eval time = 1894.67 ms / 47 tokens ( 40.31 ms per token, 24.81 tokens per second)
|
|
0.34.147.615 I slot print_timing: id 3 | task 386 | total time = 2177.01 ms / 81 tokens
|
|
0.34.147.616 I slot print_timing: id 3 | task 386 | graphs reused = 0
|
|
0.34.147.634 I slot release: id 3 | task 386 | stop processing: n_tokens = 121, truncated = 0
|
|
0.34.147.640 I srv update_slots: all slots are idle
|
|
0.34.150.208 I srv operator(): Chat format: peg-native
|
|
0.34.150.307 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.672 (> 0.100 thold), f_keep = 0.339
|
|
0.34.150.308 I srv get_availabl: updating prompt cache
|
|
0.34.150.331 W srv prompt_save: - saving prompt with length 121, total state size = 18.909 MiB (draft: 0.000 MiB)
|
|
0.34.166.719 I srv load: - looking for better prompt, base f_keep = 0.339, sim = 0.672
|
|
0.34.166.725 I srv load: - found better prompt with f_keep = 0.393, sim = 0.721
|
|
0.34.175.727 I srv update: - cache state: 4 prompts, 101.417 MiB (limits: 8192.000 MiB, 24576 tokens, 52423 est)
|
|
0.34.175.730 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
0.34.175.731 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
0.34.175.731 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
0.34.175.732 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
0.34.175.733 I srv get_availabl: prompt cache update took 25.42 ms
|
|
0.34.175.767 I slot launch_slot_: id 3 | task 434 | processing task, is_child = 0
|
|
0.34.175.768 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.34.175.769 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.34.175.785 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.34.175.786 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.34.175.798 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.34.175.798 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.34.799.974 I slot print_timing: id 3 | task 434 | prompt eval time = 212.60 ms / 17 tokens ( 12.51 ms per token, 79.96 tokens per second)
|
|
0.34.799.977 I slot print_timing: id 3 | task 434 | eval time = 411.55 ms / 11 tokens ( 37.41 ms per token, 26.73 tokens per second)
|
|
0.34.799.978 I slot print_timing: id 3 | task 434 | total time = 624.15 ms / 28 tokens
|
|
0.34.799.978 I slot print_timing: id 3 | task 434 | graphs reused = 0
|
|
0.34.799.998 I slot release: id 3 | task 434 | stop processing: n_tokens = 71, truncated = 0
|
|
0.34.800.006 I srv update_slots: all slots are idle
|
|
0.34.802.416 I srv operator(): Chat format: peg-native
|
|
0.34.802.517 I slot get_availabl: id 2 | task -1 | selected slot by LRU, t_last = -1
|
|
0.34.802.518 I srv get_availabl: updating prompt cache
|
|
0.34.802.520 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
|
0.34.802.523 I srv update: - cache state: 4 prompts, 101.417 MiB (limits: 8192.000 MiB, 24576 tokens, 52423 est)
|
|
0.34.802.523 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
0.34.802.526 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
0.34.802.527 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
0.34.802.527 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
0.34.802.528 I srv get_availabl: prompt cache update took 0.01 ms
|
|
0.34.802.540 I slot launch_slot_: id 2 | task 446 | processing task, is_child = 0
|
|
0.34.802.540 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.34.802.541 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.34.802.557 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.34.802.558 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.34.802.570 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
0.34.802.596 W srv prompt_save: - saving prompt with length 71, total state size = 11.096 MiB (draft: 0.000 MiB)
|
|
0.34.812.966 I srv update: - cache state: 5 prompts, 112.513 MiB (limits: 8192.000 MiB, 24576 tokens, 52422 est)
|
|
0.34.812.969 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
0.34.812.970 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
0.34.812.970 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
0.34.812.971 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
0.34.812.971 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
0.34.812.972 I slot prompt_clear: id 3 | task -1 | clearing prompt with 71 tokens
|
|
0.39.309.683 I slot print_timing: id 2 | task 446 | n_decoded = 100, tg = 24.46 t/s, tg_3s = 24.46 t/s
|
|
0.42.326.997 I slot print_timing: id 2 | task 446 | n_decoded = 173, tg = 24.35 t/s, tg_3s = 24.19 t/s
|
|
0.45.358.511 I slot print_timing: id 2 | task 446 | n_decoded = 246, tg = 24.27 t/s, tg_3s = 24.08 t/s
|
|
0.48.390.088 I slot print_timing: id 2 | task 446 | n_decoded = 319, tg = 24.23 t/s, tg_3s = 24.08 t/s
|
|
0.50.013.051 I slot print_timing: id 2 | task 446 | prompt eval time = 409.03 ms / 90 tokens ( 4.54 ms per token, 220.03 tokens per second)
|
|
0.50.013.054 I slot print_timing: id 2 | task 446 | eval time = 14791.00 ms / 358 tokens ( 41.32 ms per token, 24.20 tokens per second)
|
|
0.50.013.055 I slot print_timing: id 2 | task 446 | total time = 15200.03 ms / 448 tokens
|
|
0.50.013.055 I slot print_timing: id 2 | task 446 | graphs reused = 0
|
|
0.50.013.074 I slot release: id 2 | task 446 | stop processing: n_tokens = 447, truncated = 0
|
|
0.50.013.079 I srv update_slots: all slots are idle
|
|
0.50.015.543 I srv operator(): Chat format: peg-native
|
|
0.50.015.642 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.513 (> 0.100 thold), f_keep = 0.132
|
|
0.50.015.643 I srv get_availabl: updating prompt cache
|
|
0.50.015.687 W srv prompt_save: - saving prompt with length 447, total state size = 69.850 MiB (draft: 0.000 MiB)
|
|
0.50.069.803 I srv load: - looking for better prompt, base f_keep = 0.132, sim = 0.513
|
|
0.50.069.812 I srv update: - cache state: 6 prompts, 182.363 MiB (limits: 8192.000 MiB, 24576 tokens, 52423 est)
|
|
0.50.069.813 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
0.50.069.814 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
0.50.069.814 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
0.50.069.815 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
0.50.069.815 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
0.50.069.816 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
0.50.069.817 I srv get_availabl: prompt cache update took 54.17 ms
|
|
0.50.069.857 I slot launch_slot_: id 2 | task 805 | processing task, is_child = 0
|
|
0.50.069.858 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.50.069.858 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.50.069.876 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.50.069.876 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.50.069.889 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
0.50.069.889 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
0.54.439.947 I slot print_timing: id 2 | task 805 | n_decoded = 100, tg = 24.44 t/s, tg_3s = 24.44 t/s
|
|
0.57.459.040 I slot print_timing: id 2 | task 805 | n_decoded = 173, tg = 24.33 t/s, tg_3s = 24.18 t/s
|
|
1.00.487.856 I slot print_timing: id 2 | task 805 | n_decoded = 246, tg = 24.26 t/s, tg_3s = 24.10 t/s
|
|
1.03.503.072 I slot print_timing: id 2 | task 805 | n_decoded = 319, tg = 24.25 t/s, tg_3s = 24.21 t/s
|
|
1.06.518.754 I slot print_timing: id 2 | task 805 | n_decoded = 392, tg = 24.24 t/s, tg_3s = 24.21 t/s
|
|
1.09.544.039 I slot print_timing: id 2 | task 805 | n_decoded = 465, tg = 24.22 t/s, tg_3s = 24.13 t/s
|
|
1.12.572.083 I slot print_timing: id 2 | task 805 | n_decoded = 538, tg = 24.21 t/s, tg_3s = 24.11 t/s
|
|
1.15.605.878 I slot print_timing: id 2 | task 805 | n_decoded = 611, tg = 24.19 t/s, tg_3s = 24.06 t/s
|
|
1.17.224.221 I slot print_timing: id 2 | task 805 | prompt eval time = 278.40 ms / 56 tokens ( 4.97 ms per token, 201.15 tokens per second)
|
|
1.17.224.226 I slot print_timing: id 2 | task 805 | eval time = 26875.91 ms / 650 tokens ( 41.35 ms per token, 24.19 tokens per second)
|
|
1.17.224.227 I slot print_timing: id 2 | task 805 | total time = 27154.31 ms / 706 tokens
|
|
1.17.224.227 I slot print_timing: id 2 | task 805 | graphs reused = 0
|
|
1.17.224.253 I slot release: id 2 | task 805 | stop processing: n_tokens = 764, truncated = 0
|
|
1.17.224.262 I srv update_slots: all slots are idle
|
|
1.17.227.040 I srv operator(): Chat format: peg-native
|
|
1.17.227.146 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.522 (> 0.100 thold), f_keep = 0.077
|
|
1.17.227.148 I srv get_availabl: updating prompt cache
|
|
1.17.227.194 W srv prompt_save: - saving prompt with length 764, total state size = 119.385 MiB (draft: 0.000 MiB)
|
|
1.17.358.874 I srv load: - looking for better prompt, base f_keep = 0.077, sim = 0.522
|
|
1.17.358.881 I srv update: - cache state: 7 prompts, 301.747 MiB (limits: 8192.000 MiB, 24576 tokens, 52423 est)
|
|
1.17.358.882 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
1.17.358.883 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
1.17.358.883 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
1.17.358.884 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
1.17.358.885 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
1.17.358.885 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
1.17.358.886 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
1.17.358.888 I srv get_availabl: prompt cache update took 131.74 ms
|
|
1.17.358.914 I slot launch_slot_: id 2 | task 1456 | processing task, is_child = 0
|
|
1.17.358.914 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.17.358.915 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.17.358.929 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.17.358.929 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.17.358.959 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
1.17.358.960 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
1.21.735.051 I slot print_timing: id 2 | task 1456 | n_decoded = 100, tg = 24.41 t/s, tg_3s = 24.41 t/s
|
|
1.24.761.866 I slot print_timing: id 2 | task 1456 | n_decoded = 173, tg = 24.28 t/s, tg_3s = 24.12 t/s
|
|
1.27.791.223 I slot print_timing: id 2 | task 1456 | n_decoded = 246, tg = 24.23 t/s, tg_3s = 24.10 t/s
|
|
1.30.821.963 I slot print_timing: id 2 | task 1456 | n_decoded = 319, tg = 24.20 t/s, tg_3s = 24.09 t/s
|
|
1.33.852.269 I slot print_timing: id 2 | task 1456 | n_decoded = 392, tg = 24.18 t/s, tg_3s = 24.09 t/s
|
|
1.36.854.086 I slot print_timing: id 2 | task 1456 | n_decoded = 464, tg = 24.15 t/s, tg_3s = 23.99 t/s
|
|
1.39.893.937 I slot print_timing: id 2 | task 1456 | n_decoded = 537, tg = 24.13 t/s, tg_3s = 24.01 t/s
|
|
1.42.933.149 I slot print_timing: id 2 | task 1456 | n_decoded = 610, tg = 24.12 t/s, tg_3s = 24.02 t/s
|
|
1.45.939.143 I slot print_timing: id 2 | task 1456 | n_decoded = 682, tg = 24.10 t/s, tg_3s = 23.95 t/s
|
|
1.46.065.209 I slot print_timing: id 2 | task 1456 | prompt eval time = 279.08 ms / 54 tokens ( 5.17 ms per token, 193.50 tokens per second)
|
|
1.46.065.211 I slot print_timing: id 2 | task 1456 | eval time = 28427.13 ms / 685 tokens ( 41.50 ms per token, 24.10 tokens per second)
|
|
1.46.065.212 I slot print_timing: id 2 | task 1456 | total time = 28706.21 ms / 739 tokens
|
|
1.46.065.213 I slot print_timing: id 2 | task 1456 | graphs reused = 0
|
|
1.46.065.236 I slot release: id 2 | task 1456 | stop processing: n_tokens = 797, truncated = 0
|
|
1.46.065.244 I srv update_slots: all slots are idle
|
|
1.46.068.116 I srv operator(): Chat format: peg-native
|
|
1.46.068.222 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.480 (> 0.100 thold), f_keep = 0.074
|
|
1.46.068.224 I srv get_availabl: updating prompt cache
|
|
1.46.068.265 W srv prompt_save: - saving prompt with length 797, total state size = 124.541 MiB (draft: 0.000 MiB)
|
|
1.46.195.141 I srv load: - looking for better prompt, base f_keep = 0.074, sim = 0.480
|
|
1.46.195.148 I srv update: - cache state: 8 prompts, 426.289 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
|
|
1.46.195.149 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
1.46.195.149 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
1.46.195.150 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
1.46.195.150 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
1.46.195.151 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
1.46.195.152 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
1.46.195.153 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
1.46.195.157 I srv update: - prompt 0x587509e12ef0: 797 tokens, checkpoints: 0, 124.541 MiB
|
|
1.46.195.158 I srv get_availabl: prompt cache update took 126.93 ms
|
|
1.46.195.184 I slot launch_slot_: id 2 | task 2142 | processing task, is_child = 0
|
|
1.46.195.185 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.46.195.186 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.46.195.201 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.46.195.201 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.46.195.214 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
1.46.195.215 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
1.50.582.047 I slot print_timing: id 2 | task 2142 | n_decoded = 100, tg = 24.55 t/s, tg_3s = 24.55 t/s
|
|
1.53.595.087 I slot print_timing: id 2 | task 2142 | n_decoded = 173, tg = 24.41 t/s, tg_3s = 24.23 t/s
|
|
1.56.619.402 I slot print_timing: id 2 | task 2142 | n_decoded = 246, tg = 24.33 t/s, tg_3s = 24.14 t/s
|
|
1.59.645.069 I slot print_timing: id 2 | task 2142 | n_decoded = 319, tg = 24.28 t/s, tg_3s = 24.13 t/s
|
|
2.02.672.481 I slot print_timing: id 2 | task 2142 | n_decoded = 392, tg = 24.25 t/s, tg_3s = 24.11 t/s
|
|
2.05.702.995 I slot print_timing: id 2 | task 2142 | n_decoded = 465, tg = 24.23 t/s, tg_3s = 24.09 t/s
|
|
2.08.741.554 I slot print_timing: id 2 | task 2142 | n_decoded = 538, tg = 24.20 t/s, tg_3s = 24.02 t/s
|
|
2.11.742.539 I slot print_timing: id 2 | task 2142 | n_decoded = 610, tg = 24.17 t/s, tg_3s = 23.99 t/s
|
|
2.12.908.509 I slot print_timing: id 2 | task 2142 | prompt eval time = 312.83 ms / 64 tokens ( 4.89 ms per token, 204.58 tokens per second)
|
|
2.12.908.512 I slot print_timing: id 2 | task 2142 | eval time = 26400.42 ms / 638 tokens ( 41.38 ms per token, 24.17 tokens per second)
|
|
2.12.908.513 I slot print_timing: id 2 | task 2142 | total time = 26713.25 ms / 702 tokens
|
|
2.12.908.513 I slot print_timing: id 2 | task 2142 | graphs reused = 0
|
|
2.12.908.535 I slot release: id 2 | task 2142 | stop processing: n_tokens = 760, truncated = 0
|
|
2.12.908.543 I srv update_slots: all slots are idle
|
|
2.12.911.201 I srv operator(): Chat format: peg-native
|
|
2.12.911.301 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.628 (> 0.100 thold), f_keep = 0.078
|
|
2.12.911.302 I srv get_availabl: updating prompt cache
|
|
2.12.911.353 W srv prompt_save: - saving prompt with length 760, total state size = 118.760 MiB (draft: 0.000 MiB)
|
|
2.13.042.741 I srv load: - looking for better prompt, base f_keep = 0.078, sim = 0.628
|
|
2.13.042.756 I srv update: - cache state: 9 prompts, 545.048 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
|
|
2.13.042.757 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.13.042.758 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
2.13.042.762 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
2.13.042.763 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.13.042.763 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
2.13.042.764 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
2.13.042.765 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
2.13.042.768 I srv update: - prompt 0x587509e12ef0: 797 tokens, checkpoints: 0, 124.541 MiB
|
|
2.13.042.768 I srv update: - prompt 0x58750abc84d0: 760 tokens, checkpoints: 0, 118.760 MiB
|
|
2.13.042.769 I srv get_availabl: prompt cache update took 131.47 ms
|
|
2.13.042.797 I slot launch_slot_: id 2 | task 2781 | processing task, is_child = 0
|
|
2.13.042.798 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.13.042.799 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.13.042.814 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.13.042.814 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.13.042.827 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
2.13.042.827 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
2.17.433.647 I slot print_timing: id 2 | task 2781 | n_decoded = 100, tg = 24.35 t/s, tg_3s = 24.35 t/s
|
|
2.20.465.082 I slot print_timing: id 2 | task 2781 | n_decoded = 173, tg = 24.23 t/s, tg_3s = 24.08 t/s
|
|
2.23.500.435 I slot print_timing: id 2 | task 2781 | n_decoded = 246, tg = 24.18 t/s, tg_3s = 24.05 t/s
|
|
2.26.538.058 I slot print_timing: id 2 | task 2781 | n_decoded = 319, tg = 24.14 t/s, tg_3s = 24.03 t/s
|
|
2.27.789.729 I slot print_timing: id 2 | task 2781 | prompt eval time = 283.19 ms / 35 tokens ( 8.09 ms per token, 123.59 tokens per second)
|
|
2.27.789.732 I slot print_timing: id 2 | task 2781 | eval time = 14463.68 ms / 349 tokens ( 41.44 ms per token, 24.13 tokens per second)
|
|
2.27.789.733 I slot print_timing: id 2 | task 2781 | total time = 14746.88 ms / 384 tokens
|
|
2.27.789.734 I slot print_timing: id 2 | task 2781 | graphs reused = 0
|
|
2.27.789.762 I slot release: id 2 | task 2781 | stop processing: n_tokens = 442, truncated = 0
|
|
2.27.789.769 I srv update_slots: all slots are idle
|
|
2.27.792.491 I srv operator(): Chat format: peg-native
|
|
2.27.792.595 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.541 (> 0.100 thold), f_keep = 0.133
|
|
2.27.792.596 I srv get_availabl: updating prompt cache
|
|
2.27.792.627 W srv prompt_save: - saving prompt with length 442, total state size = 69.068 MiB (draft: 0.000 MiB)
|
|
2.27.845.368 I srv load: - looking for better prompt, base f_keep = 0.133, sim = 0.541
|
|
2.27.845.378 I srv update: - cache state: 10 prompts, 614.117 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
|
|
2.27.845.379 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.27.845.380 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
2.27.845.380 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
2.27.845.381 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.27.845.393 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
2.27.845.394 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
2.27.845.395 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
2.27.845.395 I srv update: - prompt 0x587509e12ef0: 797 tokens, checkpoints: 0, 124.541 MiB
|
|
2.27.845.396 I srv update: - prompt 0x58750abc84d0: 760 tokens, checkpoints: 0, 118.760 MiB
|
|
2.27.845.396 I srv update: - prompt 0x587509e13bf0: 442 tokens, checkpoints: 0, 69.068 MiB
|
|
2.27.845.398 I srv get_availabl: prompt cache update took 52.80 ms
|
|
2.27.845.441 I slot launch_slot_: id 2 | task 3131 | processing task, is_child = 0
|
|
2.27.845.441 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.27.845.442 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.27.845.460 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.27.845.460 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.27.845.473 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
2.27.845.473 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
2.32.237.724 I slot print_timing: id 2 | task 3131 | n_decoded = 100, tg = 24.31 t/s, tg_3s = 24.31 t/s
|
|
2.35.259.742 I slot print_timing: id 2 | task 3131 | n_decoded = 173, tg = 24.24 t/s, tg_3s = 24.16 t/s
|
|
2.38.297.557 I slot print_timing: id 2 | task 3131 | n_decoded = 246, tg = 24.18 t/s, tg_3s = 24.03 t/s
|
|
2.41.333.956 I slot print_timing: id 2 | task 3131 | n_decoded = 319, tg = 24.15 t/s, tg_3s = 24.04 t/s
|
|
2.44.355.514 I slot print_timing: id 2 | task 3131 | n_decoded = 392, tg = 24.15 t/s, tg_3s = 24.16 t/s
|
|
2.46.600.054 I slot print_timing: id 2 | task 3131 | prompt eval time = 278.61 ms / 50 tokens ( 5.57 ms per token, 179.46 tokens per second)
|
|
2.46.600.058 I slot print_timing: id 2 | task 3131 | eval time = 18475.95 ms / 446 tokens ( 41.43 ms per token, 24.14 tokens per second)
|
|
2.46.600.059 I slot print_timing: id 2 | task 3131 | total time = 18754.56 ms / 496 tokens
|
|
2.46.600.059 I slot print_timing: id 2 | task 3131 | graphs reused = 0
|
|
2.46.600.083 I slot release: id 2 | task 3131 | stop processing: n_tokens = 554, truncated = 0
|
|
2.46.600.090 I srv update_slots: all slots are idle
|
|
2.46.602.966 I srv operator(): Chat format: peg-native
|
|
2.46.603.090 I slot get_availabl: id 1 | task -1 | selected slot by LRU, t_last = -1
|
|
2.46.603.091 I srv get_availabl: updating prompt cache
|
|
2.46.603.093 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
|
2.46.603.096 I srv update: - cache state: 10 prompts, 614.117 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
|
|
2.46.603.097 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.46.603.097 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
2.46.603.098 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
2.46.603.102 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.46.603.102 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
2.46.603.103 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
2.46.603.104 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
2.46.603.104 I srv update: - prompt 0x587509e12ef0: 797 tokens, checkpoints: 0, 124.541 MiB
|
|
2.46.603.105 I srv update: - prompt 0x58750abc84d0: 760 tokens, checkpoints: 0, 118.760 MiB
|
|
2.46.603.105 I srv update: - prompt 0x587509e13bf0: 442 tokens, checkpoints: 0, 69.068 MiB
|
|
2.46.603.106 I srv get_availabl: prompt cache update took 0.01 ms
|
|
2.46.603.118 I slot launch_slot_: id 1 | task 3578 | processing task, is_child = 0
|
|
2.46.603.119 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.46.603.119 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.46.603.134 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.46.603.179 W srv prompt_save: - saving prompt with length 554, total state size = 86.570 MiB (draft: 0.000 MiB)
|
|
2.46.717.833 I srv update: - cache state: 11 prompts, 700.687 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
|
|
2.46.717.837 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.46.717.838 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
2.46.717.838 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
2.46.717.839 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.46.717.839 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
2.46.717.840 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
2.46.717.840 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
2.46.717.841 I srv update: - prompt 0x587509e12ef0: 797 tokens, checkpoints: 0, 124.541 MiB
|
|
2.46.717.841 I srv update: - prompt 0x58750abc84d0: 760 tokens, checkpoints: 0, 118.760 MiB
|
|
2.46.717.842 I srv update: - prompt 0x587509e13bf0: 442 tokens, checkpoints: 0, 69.068 MiB
|
|
2.46.717.842 I srv update: - prompt 0x587509e23270: 554 tokens, checkpoints: 0, 86.570 MiB
|
|
2.46.717.843 I slot prompt_clear: id 2 | task -1 | clearing prompt with 554 tokens
|
|
2.46.717.930 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
2.46.717.930 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
2.50.138.643 I slot print_timing: id 1 | task 3578 | prompt eval time = 524.21 ms / 125 tokens ( 4.19 ms per token, 238.45 tokens per second)
|
|
2.50.138.647 I slot print_timing: id 1 | task 3578 | eval time = 2896.45 ms / 71 tokens ( 40.80 ms per token, 24.51 tokens per second)
|
|
2.50.138.647 I slot print_timing: id 1 | task 3578 | total time = 3420.67 ms / 196 tokens
|
|
2.50.138.648 I slot print_timing: id 1 | task 3578 | graphs reused = 0
|
|
2.50.138.670 I slot release: id 1 | task 3578 | stop processing: n_tokens = 195, truncated = 0
|
|
2.50.138.674 I srv update_slots: all slots are idle
|
|
2.50.202.698 I srv operator(): Chat format: peg-native
|
|
2.50.202.884 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.706 (> 0.100 thold), f_keep = 0.308
|
|
2.50.202.886 I srv get_availabl: updating prompt cache
|
|
2.50.202.921 W srv prompt_save: - saving prompt with length 195, total state size = 30.472 MiB (draft: 0.000 MiB)
|
|
2.50.228.109 I srv load: - looking for better prompt, base f_keep = 0.308, sim = 0.706
|
|
2.50.228.119 I srv update: - cache state: 12 prompts, 731.159 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
|
|
2.50.228.120 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.50.228.120 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
2.50.228.121 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
2.50.228.121 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.50.228.122 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
2.50.228.123 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
2.50.228.123 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
2.50.228.124 I srv update: - prompt 0x587509e12ef0: 797 tokens, checkpoints: 0, 124.541 MiB
|
|
2.50.228.125 I srv update: - prompt 0x58750abc84d0: 760 tokens, checkpoints: 0, 118.760 MiB
|
|
2.50.228.126 I srv update: - prompt 0x587509e13bf0: 442 tokens, checkpoints: 0, 69.068 MiB
|
|
2.50.228.127 I srv update: - prompt 0x587509e23270: 554 tokens, checkpoints: 0, 86.570 MiB
|
|
2.50.228.128 I srv update: - prompt 0x587506aa2e30: 195 tokens, checkpoints: 0, 30.472 MiB
|
|
2.50.228.129 I srv get_availabl: prompt cache update took 25.24 ms
|
|
2.50.228.171 I slot launch_slot_: id 1 | task 3650 | processing task, is_child = 0
|
|
2.50.228.172 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.50.228.173 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.50.228.189 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.50.228.189 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.50.228.202 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
2.50.228.202 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
2.52.131.180 I slot print_timing: id 1 | task 3650 | prompt eval time = 251.08 ms / 25 tokens ( 10.04 ms per token, 99.57 tokens per second)
|
|
2.52.131.184 I slot print_timing: id 1 | task 3650 | eval time = 1651.88 ms / 41 tokens ( 40.29 ms per token, 24.82 tokens per second)
|
|
2.52.131.185 I slot print_timing: id 1 | task 3650 | total time = 1902.95 ms / 66 tokens
|
|
2.52.131.186 I slot print_timing: id 1 | task 3650 | graphs reused = 0
|
|
2.52.131.207 I slot release: id 1 | task 3650 | stop processing: n_tokens = 125, truncated = 0
|
|
2.52.131.213 I srv update_slots: all slots are idle
|
|
2.52.178.876 I srv operator(): Chat format: peg-native
|
|
2.52.179.010 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.645 (> 0.100 thold), f_keep = 0.480
|
|
2.52.179.012 I srv get_availabl: updating prompt cache
|
|
2.52.179.058 W srv prompt_save: - saving prompt with length 125, total state size = 19.534 MiB (draft: 0.000 MiB)
|
|
2.52.196.092 I srv load: - looking for better prompt, base f_keep = 0.480, sim = 0.645
|
|
2.52.196.101 I srv update: - cache state: 13 prompts, 750.692 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
|
|
2.52.196.102 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.52.196.103 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
2.52.196.103 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
2.52.196.104 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.52.196.105 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
2.52.196.107 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
2.52.196.108 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
2.52.196.108 I srv update: - prompt 0x587509e12ef0: 797 tokens, checkpoints: 0, 124.541 MiB
|
|
2.52.196.109 I srv update: - prompt 0x58750abc84d0: 760 tokens, checkpoints: 0, 118.760 MiB
|
|
2.52.196.109 I srv update: - prompt 0x587509e13bf0: 442 tokens, checkpoints: 0, 69.068 MiB
|
|
2.52.196.110 I srv update: - prompt 0x587509e23270: 554 tokens, checkpoints: 0, 86.570 MiB
|
|
2.52.196.110 I srv update: - prompt 0x587506aa2e30: 195 tokens, checkpoints: 0, 30.472 MiB
|
|
2.52.196.111 I srv update: - prompt 0x587509f13940: 125 tokens, checkpoints: 0, 19.534 MiB
|
|
2.52.196.112 I srv get_availabl: prompt cache update took 17.10 ms
|
|
2.52.196.147 I slot launch_slot_: id 1 | task 3692 | processing task, is_child = 0
|
|
2.52.196.148 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.52.196.149 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.52.196.170 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.52.196.170 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.52.196.205 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
2.52.196.206 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
2.54.379.166 I slot print_timing: id 1 | task 3692 | prompt eval time = 282.51 ms / 33 tokens ( 8.56 ms per token, 116.81 tokens per second)
|
|
2.54.379.170 I slot print_timing: id 1 | task 3692 | eval time = 1900.42 ms / 47 tokens ( 40.43 ms per token, 24.73 tokens per second)
|
|
2.54.379.171 I slot print_timing: id 1 | task 3692 | total time = 2182.93 ms / 80 tokens
|
|
2.54.379.171 I slot print_timing: id 1 | task 3692 | graphs reused = 0
|
|
2.54.379.192 I slot release: id 1 | task 3692 | stop processing: n_tokens = 139, truncated = 0
|
|
2.54.379.199 I srv update_slots: all slots are idle
|
|
2.54.453.143 I srv operator(): Chat format: peg-native
|
|
2.54.453.260 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.484 (> 0.100 thold), f_keep = 0.432
|
|
2.54.453.262 I srv get_availabl: updating prompt cache
|
|
2.54.453.310 W srv prompt_save: - saving prompt with length 139, total state size = 21.721 MiB (draft: 0.000 MiB)
|
|
2.54.471.664 I srv load: - looking for better prompt, base f_keep = 0.432, sim = 0.484
|
|
2.54.471.673 I srv update: - cache state: 14 prompts, 772.413 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
|
|
2.54.471.674 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.54.471.675 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
2.54.471.679 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
2.54.471.679 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.54.471.680 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
2.54.471.681 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
2.54.471.681 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
2.54.471.683 I srv update: - prompt 0x587509e12ef0: 797 tokens, checkpoints: 0, 124.541 MiB
|
|
2.54.471.684 I srv update: - prompt 0x58750abc84d0: 760 tokens, checkpoints: 0, 118.760 MiB
|
|
2.54.471.684 I srv update: - prompt 0x587509e13bf0: 442 tokens, checkpoints: 0, 69.068 MiB
|
|
2.54.471.685 I srv update: - prompt 0x587509e23270: 554 tokens, checkpoints: 0, 86.570 MiB
|
|
2.54.471.685 I srv update: - prompt 0x587506aa2e30: 195 tokens, checkpoints: 0, 30.472 MiB
|
|
2.54.471.686 I srv update: - prompt 0x587509f13940: 125 tokens, checkpoints: 0, 19.534 MiB
|
|
2.54.471.686 I srv update: - prompt 0x587509df0bc0: 139 tokens, checkpoints: 0, 21.721 MiB
|
|
2.54.471.688 I srv get_availabl: prompt cache update took 18.43 ms
|
|
2.54.471.722 I slot launch_slot_: id 1 | task 3740 | processing task, is_child = 0
|
|
2.54.471.723 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.54.471.724 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.54.471.742 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.54.471.743 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.54.471.765 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
2.54.471.766 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
2.58.673.500 I slot print_timing: id 1 | task 3740 | prompt eval time = 312.63 ms / 64 tokens ( 4.88 ms per token, 204.72 tokens per second)
|
|
2.58.673.504 I slot print_timing: id 1 | task 3740 | eval time = 3889.08 ms / 95 tokens ( 40.94 ms per token, 24.43 tokens per second)
|
|
2.58.673.505 I slot print_timing: id 1 | task 3740 | total time = 4201.71 ms / 159 tokens
|
|
2.58.673.505 I slot print_timing: id 1 | task 3740 | graphs reused = 0
|
|
2.58.673.529 I slot release: id 1 | task 3740 | stop processing: n_tokens = 218, truncated = 0
|
|
2.58.673.535 I srv update_slots: all slots are idle
|
|
2.58.736.135 I srv operator(): Chat format: peg-native
|
|
2.58.736.254 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.612 (> 0.100 thold), f_keep = 0.275
|
|
2.58.736.256 I srv get_availabl: updating prompt cache
|
|
2.58.736.313 W srv prompt_save: - saving prompt with length 218, total state size = 34.066 MiB (draft: 0.000 MiB)
|
|
2.58.764.006 I srv load: - looking for better prompt, base f_keep = 0.275, sim = 0.612
|
|
2.58.764.017 I srv update: - cache state: 15 prompts, 806.479 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
|
|
2.58.764.018 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.58.764.019 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
2.58.764.019 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
2.58.764.020 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
2.58.764.020 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
2.58.764.021 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
2.58.764.022 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
2.58.764.023 I srv update: - prompt 0x587509e12ef0: 797 tokens, checkpoints: 0, 124.541 MiB
|
|
2.58.764.030 I srv update: - prompt 0x58750abc84d0: 760 tokens, checkpoints: 0, 118.760 MiB
|
|
2.58.764.030 I srv update: - prompt 0x587509e13bf0: 442 tokens, checkpoints: 0, 69.068 MiB
|
|
2.58.764.031 I srv update: - prompt 0x587509e23270: 554 tokens, checkpoints: 0, 86.570 MiB
|
|
2.58.764.031 I srv update: - prompt 0x587506aa2e30: 195 tokens, checkpoints: 0, 30.472 MiB
|
|
2.58.764.032 I srv update: - prompt 0x587509f13940: 125 tokens, checkpoints: 0, 19.534 MiB
|
|
2.58.764.032 I srv update: - prompt 0x587509df0bc0: 139 tokens, checkpoints: 0, 21.721 MiB
|
|
2.58.764.044 I srv update: - prompt 0x587506ca90d0: 218 tokens, checkpoints: 0, 34.066 MiB
|
|
2.58.764.046 I srv get_availabl: prompt cache update took 27.79 ms
|
|
2.58.764.083 I slot launch_slot_: id 1 | task 3836 | processing task, is_child = 0
|
|
2.58.764.084 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.58.764.085 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.58.764.103 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.58.764.104 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.58.764.138 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
2.58.764.138 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
3.00.660.665 I slot print_timing: id 1 | task 3836 | prompt eval time = 283.32 ms / 38 tokens ( 7.46 ms per token, 134.12 tokens per second)
|
|
3.00.660.668 I slot print_timing: id 1 | task 3836 | eval time = 1613.17 ms / 40 tokens ( 40.33 ms per token, 24.80 tokens per second)
|
|
3.00.660.669 I slot print_timing: id 1 | task 3836 | total time = 1896.49 ms / 78 tokens
|
|
3.00.660.669 I slot print_timing: id 1 | task 3836 | graphs reused = 0
|
|
3.00.660.691 I slot release: id 1 | task 3836 | stop processing: n_tokens = 137, truncated = 0
|
|
3.00.660.699 I srv update_slots: all slots are idle
|
|
3.00.708.300 I srv operator(): Chat format: peg-native
|
|
3.00.708.412 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.524 (> 0.100 thold), f_keep = 0.401
|
|
3.00.708.415 I srv get_availabl: updating prompt cache
|
|
3.00.708.467 W srv prompt_save: - saving prompt with length 137, total state size = 21.409 MiB (draft: 0.000 MiB)
|
|
3.00.726.802 I srv load: - looking for better prompt, base f_keep = 0.401, sim = 0.524
|
|
3.00.726.811 I srv update: - cache state: 16 prompts, 827.888 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
|
|
3.00.726.812 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
3.00.726.812 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
3.00.726.813 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
3.00.726.814 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
3.00.726.815 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
3.00.726.827 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
3.00.726.828 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
3.00.726.829 I srv update: - prompt 0x587509e12ef0: 797 tokens, checkpoints: 0, 124.541 MiB
|
|
3.00.726.829 I srv update: - prompt 0x58750abc84d0: 760 tokens, checkpoints: 0, 118.760 MiB
|
|
3.00.726.830 I srv update: - prompt 0x587509e13bf0: 442 tokens, checkpoints: 0, 69.068 MiB
|
|
3.00.726.831 I srv update: - prompt 0x587509e23270: 554 tokens, checkpoints: 0, 86.570 MiB
|
|
3.00.726.831 I srv update: - prompt 0x587506aa2e30: 195 tokens, checkpoints: 0, 30.472 MiB
|
|
3.00.726.831 I srv update: - prompt 0x587509f13940: 125 tokens, checkpoints: 0, 19.534 MiB
|
|
3.00.726.832 I srv update: - prompt 0x587509df0bc0: 139 tokens, checkpoints: 0, 21.721 MiB
|
|
3.00.726.832 I srv update: - prompt 0x587506ca90d0: 218 tokens, checkpoints: 0, 34.066 MiB
|
|
3.00.726.833 I srv update: - prompt 0x587509c8b170: 137 tokens, checkpoints: 0, 21.409 MiB
|
|
3.00.726.834 I srv get_availabl: prompt cache update took 18.42 ms
|
|
3.00.726.867 I slot launch_slot_: id 1 | task 3877 | processing task, is_child = 0
|
|
3.00.726.868 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.00.726.868 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.00.726.886 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.00.726.886 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.00.726.920 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
3.00.726.921 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
3.02.043.587 I slot print_timing: id 1 | task 3877 | prompt eval time = 278.04 ms / 50 tokens ( 5.56 ms per token, 179.83 tokens per second)
|
|
3.02.043.590 I slot print_timing: id 1 | task 3877 | eval time = 1038.59 ms / 26 tokens ( 39.95 ms per token, 25.03 tokens per second)
|
|
3.02.043.590 I slot print_timing: id 1 | task 3877 | total time = 1316.63 ms / 76 tokens
|
|
3.02.043.591 I slot print_timing: id 1 | task 3877 | graphs reused = 0
|
|
3.02.043.614 I slot release: id 1 | task 3877 | stop processing: n_tokens = 130, truncated = 0
|
|
3.02.043.621 I srv update_slots: all slots are idle
|
|
3.02.046.412 I srv operator(): Chat format: peg-native
|
|
3.02.046.517 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.591 (> 0.100 thold), f_keep = 0.423
|
|
3.02.046.519 I srv get_availabl: updating prompt cache
|
|
3.02.046.555 W srv prompt_save: - saving prompt with length 130, total state size = 20.315 MiB (draft: 0.000 MiB)
|
|
3.02.064.042 I srv load: - looking for better prompt, base f_keep = 0.423, sim = 0.591
|
|
3.02.064.049 I srv load: - found better prompt with f_keep = 0.480, sim = 0.645
|
|
3.02.073.793 I srv update: - cache state: 16 prompts, 828.669 MiB (limits: 8192.000 MiB, 24576 tokens, 52424 est)
|
|
3.02.073.796 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
3.02.073.797 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
3.02.073.797 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
3.02.073.798 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
3.02.073.798 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
3.02.073.799 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
3.02.073.800 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
3.02.073.800 I srv update: - prompt 0x587509e12ef0: 797 tokens, checkpoints: 0, 124.541 MiB
|
|
3.02.073.801 I srv update: - prompt 0x58750abc84d0: 760 tokens, checkpoints: 0, 118.760 MiB
|
|
3.02.073.801 I srv update: - prompt 0x587509e13bf0: 442 tokens, checkpoints: 0, 69.068 MiB
|
|
3.02.073.802 I srv update: - prompt 0x587509e23270: 554 tokens, checkpoints: 0, 86.570 MiB
|
|
3.02.073.803 I srv update: - prompt 0x587506aa2e30: 195 tokens, checkpoints: 0, 30.472 MiB
|
|
3.02.073.804 I srv update: - prompt 0x587509df0bc0: 139 tokens, checkpoints: 0, 21.721 MiB
|
|
3.02.073.804 I srv update: - prompt 0x587506ca90d0: 218 tokens, checkpoints: 0, 34.066 MiB
|
|
3.02.073.805 I srv update: - prompt 0x587509c8b170: 137 tokens, checkpoints: 0, 21.409 MiB
|
|
3.02.073.805 I srv update: - prompt 0x587509e134f0: 130 tokens, checkpoints: 0, 20.315 MiB
|
|
3.02.073.806 I srv get_availabl: prompt cache update took 27.29 ms
|
|
3.02.073.839 I slot launch_slot_: id 1 | task 3904 | processing task, is_child = 0
|
|
3.02.073.840 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.02.073.841 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.02.073.860 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.02.073.860 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.02.073.876 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
3.02.073.877 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
3.06.463.857 I slot print_timing: id 1 | task 3904 | n_decoded = 100, tg = 24.35 t/s, tg_3s = 24.35 t/s
|
|
3.07.459.151 I slot print_timing: id 1 | task 3904 | prompt eval time = 282.45 ms / 33 tokens ( 8.56 ms per token, 116.83 tokens per second)
|
|
3.07.459.154 I slot print_timing: id 1 | task 3904 | eval time = 5102.79 ms / 124 tokens ( 41.15 ms per token, 24.30 tokens per second)
|
|
3.07.459.155 I slot print_timing: id 1 | task 3904 | total time = 5385.24 ms / 157 tokens
|
|
3.07.459.155 I slot print_timing: id 1 | task 3904 | graphs reused = 0
|
|
3.07.459.178 I slot release: id 1 | task 3904 | stop processing: n_tokens = 216, truncated = 0
|
|
3.07.459.185 I srv update_slots: all slots are idle
|
|
3.07.522.656 I srv operator(): Chat format: peg-native
|
|
3.07.522.804 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.645 (> 0.100 thold), f_keep = 0.278
|
|
3.07.522.806 I srv get_availabl: updating prompt cache
|
|
3.07.522.858 W srv prompt_save: - saving prompt with length 216, total state size = 33.753 MiB (draft: 0.000 MiB)
|
|
3.07.550.469 I srv load: - looking for better prompt, base f_keep = 0.278, sim = 0.645
|
|
3.07.550.482 I srv update: - cache state: 17 prompts, 862.423 MiB (limits: 8192.000 MiB, 24576 tokens, 52423 est)
|
|
3.07.550.484 I srv update: - prompt 0x587509df1090: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
3.07.550.487 I srv update: - prompt 0x587509e18f50: 288 tokens, checkpoints: 0, 45.004 MiB
|
|
3.07.550.488 I srv update: - prompt 0x587509e19320: 119 tokens, checkpoints: 0, 18.596 MiB
|
|
3.07.550.488 I srv update: - prompt 0x587509c8d360: 121 tokens, checkpoints: 0, 18.909 MiB
|
|
3.07.550.489 I srv update: - prompt 0x587509e27710: 71 tokens, checkpoints: 0, 11.096 MiB
|
|
3.07.550.489 I srv update: - prompt 0x587509e014f0: 447 tokens, checkpoints: 0, 69.850 MiB
|
|
3.07.550.501 I srv update: - prompt 0x58750976acb0: 764 tokens, checkpoints: 0, 119.385 MiB
|
|
3.07.550.502 I srv update: - prompt 0x587509e12ef0: 797 tokens, checkpoints: 0, 124.541 MiB
|
|
3.07.550.503 I srv update: - prompt 0x58750abc84d0: 760 tokens, checkpoints: 0, 118.760 MiB
|
|
3.07.550.503 I srv update: - prompt 0x587509e13bf0: 442 tokens, checkpoints: 0, 69.068 MiB
|
|
3.07.550.504 I srv update: - prompt 0x587509e23270: 554 tokens, checkpoints: 0, 86.570 MiB
|
|
3.07.550.504 I srv update: - prompt 0x587506aa2e30: 195 tokens, checkpoints: 0, 30.472 MiB
|
|
3.07.550.504 I srv update: - prompt 0x587509df0bc0: 139 tokens, checkpoints: 0, 21.721 MiB
|
|
3.07.550.505 I srv update: - prompt 0x587506ca90d0: 218 tokens, checkpoints: 0, 34.066 MiB
|
|
3.07.550.505 I srv update: - prompt 0x587509c8b170: 137 tokens, checkpoints: 0, 21.409 MiB
|
|
3.07.550.506 I srv update: - prompt 0x587509e134f0: 130 tokens, checkpoints: 0, 20.315 MiB
|
|
3.07.550.507 I srv update: - prompt 0x587509de4800: 216 tokens, checkpoints: 0, 33.753 MiB
|
|
3.07.550.507 I srv get_availabl: prompt cache update took 27.70 ms
|
|
3.07.550.551 I slot launch_slot_: id 1 | task 4029 | processing task, is_child = 0
|
|
3.07.550.552 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.07.550.553 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.07.550.572 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.07.550.572 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.07.550.588 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
|
|
3.07.550.589 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
|
|
3.10.030.606 I slot print_timing: id 1 | task 4029 | prompt eval time = 282.84 ms / 33 tokens ( 8.57 ms per token, 116.67 tokens per second)
|
|
3.10.030.610 I slot print_timing: id 1 | task 4029 | eval time = 2197.14 ms / 54 tokens ( 40.69 ms per token, 24.58 tokens per second)
|
|
3.10.030.610 I slot print_timing: id 1 | task 4029 | total time = 2479.98 ms / 87 tokens
|
|
3.10.030.611 I slot print_timing: id 1 | task 4029 | graphs reused = 0
|
|
3.10.030.634 I slot release: id 1 | task 4029 | stop processing: n_tokens = 146, truncated = 0
|
|
3.10.030.643 I srv update_slots: all slots are idle
|
|
3.10.090.822 I srv operator(): operator(): cleaning up before exit...
|