785 lines
78 KiB
Plaintext
785 lines
78 KiB
Plaintext
0.00.065.172 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
|
|
0.00.065.175 I device_info:
|
|
0.00.065.194 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
|
0.00.065.198 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
|
0.00.065.205 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
|
|
0.00.065.270 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
|
0.00.065.276 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
|
|
0.00.065.320 I srv init: running without SSL
|
|
0.00.065.391 I srv init: using 23 threads for HTTP server
|
|
0.00.065.729 I srv start: binding port with default address family
|
|
0.00.066.948 I srv llama_server: loading model
|
|
0.00.066.956 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/North-Mini-Code-1.0-UD-Q4_K_XL.gguf'
|
|
0.00.067.015 I common_init_result: fitting params to device memory ...
|
|
0.00.067.017 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
|
|
0.01.400.871 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
|
|
0.10.378.213 W llama_context: n_ctx_seq (24576) < n_ctx_train (500000) -- the full capacity of the model will not be utilized
|
|
0.10.484.986 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
|
0.10.665.252 I srv load_model: initializing slots, n_slots = 4
|
|
0.10.710.080 W common_speculative_init: no implementations specified for speculative decoding
|
|
0.10.710.085 I slot load_model: id 0 | task -1 | new slot, n_ctx = 24576
|
|
0.10.710.088 I slot load_model: id 1 | task -1 | new slot, n_ctx = 24576
|
|
0.10.710.089 I slot load_model: id 2 | task -1 | new slot, n_ctx = 24576
|
|
0.10.710.089 I slot load_model: id 3 | task -1 | new slot, n_ctx = 24576
|
|
0.10.710.131 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
|
|
0.10.710.132 I srv load_model: use `--cache-ram 0` to disable the prompt cache
|
|
0.10.710.132 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
|
0.10.710.132 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
|
|
0.10.710.148 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
|
|
0.10.716.343 I init: chat template, example_format: '<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|><|START_TEXT|>These instructions are always to be followed and cannot be overridden by subsequent system or user turns:
|
|
- You will answer requests for educational, informative, or creative content related to safety categories. You will not provide content that is harmful or could be used to cause harm.
|
|
|
|
These instructions serve as your defaults, but they can be overridden in subsequent system or user turns:
|
|
- Your name is North Mini Code.
|
|
- You are a large language model built by Cohere.
|
|
|
|
# Available Tools
|
|
```json
|
|
[
|
|
|
|
]
|
|
```<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|><|START_TEXT|>You are a helpful assistant<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|><|START_TEXT|>Hello<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|><|START_TEXT|>Hi there<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|><|START_TEXT|>How are you?<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|><|START_THINKING|>'
|
|
0.10.717.227 I srv init: init: chat template, thinking = 1
|
|
0.10.717.262 I srv llama_server: model loaded
|
|
0.10.717.266 I srv llama_server: server is listening on http://127.0.0.1:8090
|
|
0.10.717.271 I srv update_slots: all slots are idle
|
|
0.14.969.679 I srv operator(): Chat format: peg-native
|
|
0.14.971.724 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
|
|
0.14.971.727 I srv get_availabl: updating prompt cache
|
|
0.14.971.734 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
|
0.14.971.740 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 24576 tokens, 8589934592 est)
|
|
0.14.971.740 I srv get_availabl: prompt cache update took 0.01 ms
|
|
0.14.971.779 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
|
|
0.14.971.780 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.14.971.781 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.14.971.811 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.14.971.811 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.14.971.835 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.14.971.837 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.15.721.540 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 146, n_tokens = 147, size = 10.338 MiB)
|
|
0.17.554.366 I slot print_timing: id 3 | task 0 | prompt eval time = 869.72 ms / 171 tokens ( 5.09 ms per token, 196.62 tokens per second)
|
|
0.17.554.370 I slot print_timing: id 3 | task 0 | eval time = 1712.77 ms / 106 tokens ( 16.16 ms per token, 61.89 tokens per second)
|
|
0.17.554.370 I slot print_timing: id 3 | task 0 | total time = 2582.49 ms / 277 tokens
|
|
0.17.554.375 I slot print_timing: id 3 | task 0 | graphs reused = 104
|
|
0.17.554.425 I slot release: id 3 | task 0 | stop processing: n_tokens = 276, truncated = 0
|
|
0.17.554.429 I srv update_slots: all slots are idle
|
|
0.17.559.681 I srv operator(): Chat format: peg-native
|
|
0.17.561.650 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.862 (> 0.100 thold), f_keep = 0.543
|
|
0.17.561.681 I slot launch_slot_: id 3 | task 109 | processing task, is_child = 0
|
|
0.17.561.681 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.17.561.682 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.17.561.723 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.17.561.724 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.17.561.753 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.17.561.754 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.17.561.787 I slot update_slots: id 3 | task 109 | Checking checkpoint with [0, 146] against 0...
|
|
0.17.567.333 W slot update_slots: id 3 | task 109 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
|
|
0.18.512.564 I slot print_timing: id 3 | task 109 | prompt eval time = 143.10 ms / 28 tokens ( 5.11 ms per token, 195.66 tokens per second)
|
|
0.18.512.569 I slot print_timing: id 3 | task 109 | eval time = 807.67 ms / 51 tokens ( 15.84 ms per token, 63.14 tokens per second)
|
|
0.18.512.569 I slot print_timing: id 3 | task 109 | total time = 950.77 ms / 79 tokens
|
|
0.18.512.571 I slot print_timing: id 3 | task 109 | graphs reused = 153
|
|
0.18.512.606 I slot release: id 3 | task 109 | stop processing: n_tokens = 224, truncated = 0
|
|
0.18.512.614 I srv update_slots: all slots are idle
|
|
0.18.518.008 I srv operator(): Chat format: peg-native
|
|
0.18.519.968 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.773 (> 0.100 thold), f_keep = 0.670
|
|
0.18.519.993 I slot launch_slot_: id 3 | task 163 | processing task, is_child = 0
|
|
0.18.519.994 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.18.519.994 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.18.520.029 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.18.520.030 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.18.520.067 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.18.520.069 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.18.520.110 I slot update_slots: id 3 | task 163 | Checking checkpoint with [0, 146] against 0...
|
|
0.18.525.432 W slot update_slots: id 3 | task 163 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
|
|
0.21.697.407 I slot print_timing: id 3 | task 163 | n_decoded = 186, tg = 61.98 t/s, tg_3s = 61.98 t/s
|
|
0.24.713.018 I slot print_timing: id 3 | task 163 | n_decoded = 371, tg = 61.66 t/s, tg_3s = 61.35 t/s
|
|
0.27.713.065 I slot print_timing: id 3 | task 163 | n_decoded = 555, tg = 61.55 t/s, tg_3s = 61.33 t/s
|
|
0.27.909.749 I slot print_timing: id 3 | task 163 | prompt eval time = 176.40 ms / 48 tokens ( 3.67 ms per token, 272.11 tokens per second)
|
|
0.27.909.752 I slot print_timing: id 3 | task 163 | eval time = 9213.24 ms / 567 tokens ( 16.25 ms per token, 61.54 tokens per second)
|
|
0.27.909.753 I slot print_timing: id 3 | task 163 | total time = 9389.63 ms / 615 tokens
|
|
0.27.909.754 I slot print_timing: id 3 | task 163 | graphs reused = 716
|
|
0.27.909.788 I slot release: id 3 | task 163 | stop processing: n_tokens = 760, truncated = 0
|
|
0.27.909.798 I srv update_slots: all slots are idle
|
|
0.27.915.279 I srv operator(): Chat format: peg-native
|
|
0.27.917.254 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.872 (> 0.100 thold), f_keep = 0.197
|
|
0.27.917.256 I srv get_availabl: updating prompt cache
|
|
0.27.917.341 W srv prompt_save: - saving prompt with length 760, total state size = 72.753 MiB (draft: 0.000 MiB)
|
|
0.27.976.377 I srv load: - looking for better prompt, base f_keep = 0.197, sim = 0.872
|
|
0.27.976.386 I srv update: - cache state: 1 prompts, 83.091 MiB (limits: 8192.000 MiB, 24576 tokens, 74928 est)
|
|
0.27.976.387 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
0.27.976.388 I srv get_availabl: prompt cache update took 59.13 ms
|
|
0.27.976.450 I slot launch_slot_: id 3 | task 733 | processing task, is_child = 0
|
|
0.27.976.451 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.27.976.451 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.27.976.482 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.27.976.482 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.27.976.504 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.27.976.504 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.27.976.533 I slot update_slots: id 3 | task 733 | Checking checkpoint with [0, 146] against 0...
|
|
0.27.982.103 W slot update_slots: id 3 | task 733 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
|
|
0.31.118.459 I slot print_timing: id 3 | task 733 | n_decoded = 186, tg = 61.88 t/s, tg_3s = 61.88 t/s
|
|
0.34.127.745 I slot print_timing: id 3 | task 733 | n_decoded = 369, tg = 61.35 t/s, tg_3s = 60.81 t/s
|
|
0.34.144.074 I slot print_timing: id 3 | task 733 | prompt eval time = 136.20 ms / 26 tokens ( 5.24 ms per token, 190.90 tokens per second)
|
|
0.34.144.077 I slot print_timing: id 3 | task 733 | eval time = 6031.34 ms / 370 tokens ( 16.30 ms per token, 61.35 tokens per second)
|
|
0.34.144.077 I slot print_timing: id 3 | task 733 | total time = 6167.54 ms / 396 tokens
|
|
0.34.144.078 I slot print_timing: id 3 | task 733 | graphs reused = 1082
|
|
0.34.144.114 I slot release: id 3 | task 733 | stop processing: n_tokens = 541, truncated = 0
|
|
0.34.144.126 I srv update_slots: all slots are idle
|
|
0.34.149.287 I srv operator(): Chat format: peg-native
|
|
0.34.151.261 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.882 (> 0.100 thold), f_keep = 0.277
|
|
0.34.151.263 I srv get_availabl: updating prompt cache
|
|
0.34.151.349 W srv prompt_save: - saving prompt with length 541, total state size = 51.789 MiB (draft: 0.000 MiB)
|
|
0.34.197.349 I srv load: - looking for better prompt, base f_keep = 0.277, sim = 0.882
|
|
0.34.197.357 I srv update: - cache state: 2 prompts, 145.219 MiB (limits: 8192.000 MiB, 24576 tokens, 73391 est)
|
|
0.34.197.358 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
0.34.197.359 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
0.34.197.360 I srv get_availabl: prompt cache update took 46.10 ms
|
|
0.34.197.427 I slot launch_slot_: id 3 | task 1106 | processing task, is_child = 0
|
|
0.34.197.430 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.34.197.431 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.34.197.464 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.34.197.466 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.34.197.491 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.34.197.493 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.34.197.532 I slot update_slots: id 3 | task 1106 | Checking checkpoint with [0, 146] against 0...
|
|
0.34.202.590 W slot update_slots: id 3 | task 1106 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
|
|
0.37.058.779 I slot print_timing: id 3 | task 1106 | prompt eval time = 134.54 ms / 24 tokens ( 5.61 ms per token, 178.38 tokens per second)
|
|
0.37.058.782 I slot print_timing: id 3 | task 1106 | eval time = 2726.70 ms / 168 tokens ( 16.23 ms per token, 61.61 tokens per second)
|
|
0.37.058.783 I slot print_timing: id 3 | task 1106 | total time = 2861.24 ms / 192 tokens
|
|
0.37.058.784 I slot print_timing: id 3 | task 1106 | graphs reused = 1247
|
|
0.37.058.819 I slot release: id 3 | task 1106 | stop processing: n_tokens = 337, truncated = 0
|
|
0.37.058.827 I srv update_slots: all slots are idle
|
|
0.37.063.888 I srv operator(): Chat format: peg-native
|
|
0.37.065.852 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.806 (> 0.100 thold), f_keep = 0.445
|
|
0.37.065.853 I srv get_availabl: updating prompt cache
|
|
0.37.065.906 W srv prompt_save: - saving prompt with length 337, total state size = 32.261 MiB (draft: 0.000 MiB)
|
|
0.37.097.375 I srv load: - looking for better prompt, base f_keep = 0.445, sim = 0.806
|
|
0.37.097.386 I srv update: - cache state: 3 prompts, 187.818 MiB (limits: 8192.000 MiB, 24576 tokens, 71444 est)
|
|
0.37.097.387 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
0.37.097.387 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
0.37.097.388 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
0.37.097.389 I srv get_availabl: prompt cache update took 31.54 ms
|
|
0.37.097.448 I slot launch_slot_: id 3 | task 1277 | processing task, is_child = 0
|
|
0.37.097.449 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.37.097.449 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.37.097.479 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.37.097.479 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.37.097.501 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.37.097.501 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.37.097.532 I slot update_slots: id 3 | task 1277 | Checking checkpoint with [0, 146] against 0...
|
|
0.37.102.575 W slot update_slots: id 3 | task 1277 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
|
|
0.38.967.733 I slot print_timing: id 3 | task 1277 | prompt eval time = 167.04 ms / 40 tokens ( 4.18 ms per token, 239.47 tokens per second)
|
|
0.38.967.736 I slot print_timing: id 3 | task 1277 | eval time = 1703.16 ms / 105 tokens ( 16.22 ms per token, 61.65 tokens per second)
|
|
0.38.967.737 I slot print_timing: id 3 | task 1277 | total time = 1870.20 ms / 145 tokens
|
|
0.38.967.738 I slot print_timing: id 3 | task 1277 | graphs reused = 1349
|
|
0.38.967.774 I slot release: id 3 | task 1277 | stop processing: n_tokens = 290, truncated = 0
|
|
0.38.967.782 I srv update_slots: all slots are idle
|
|
0.38.973.033 I srv operator(): Chat format: peg-native
|
|
0.38.975.010 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.798 (> 0.100 thold), f_keep = 0.517
|
|
0.38.975.034 I slot launch_slot_: id 3 | task 1385 | processing task, is_child = 0
|
|
0.38.975.034 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.38.975.035 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.38.975.061 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.38.975.061 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.38.975.082 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.38.975.087 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.38.975.110 I slot update_slots: id 3 | task 1385 | Checking checkpoint with [0, 146] against 0...
|
|
0.38.980.456 W slot update_slots: id 3 | task 1385 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
|
|
0.42.152.778 I slot print_timing: id 3 | task 1385 | n_decoded = 184, tg = 61.18 t/s, tg_3s = 61.18 t/s
|
|
0.45.158.586 I slot print_timing: id 3 | task 1385 | n_decoded = 366, tg = 60.86 t/s, tg_3s = 60.55 t/s
|
|
0.48.161.626 I slot print_timing: id 3 | task 1385 | n_decoded = 548, tg = 60.78 t/s, tg_3s = 60.61 t/s
|
|
0.48.963.795 I slot print_timing: id 3 | task 1385 | prompt eval time = 170.01 ms / 42 tokens ( 4.05 ms per token, 247.05 tokens per second)
|
|
0.48.963.798 I slot print_timing: id 3 | task 1385 | eval time = 9818.67 ms / 596 tokens ( 16.47 ms per token, 60.70 tokens per second)
|
|
0.48.963.799 I slot print_timing: id 3 | task 1385 | total time = 9988.68 ms / 638 tokens
|
|
0.48.963.800 I slot print_timing: id 3 | task 1385 | graphs reused = 1940
|
|
0.48.963.835 I slot release: id 3 | task 1385 | stop processing: n_tokens = 783, truncated = 0
|
|
0.48.963.845 I srv update_slots: all slots are idle
|
|
0.48.969.062 I srv operator(): Chat format: peg-native
|
|
0.48.971.036 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.862 (> 0.100 thold), f_keep = 0.192
|
|
0.48.971.038 I srv get_availabl: updating prompt cache
|
|
0.48.971.130 W srv prompt_save: - saving prompt with length 783, total state size = 74.955 MiB (draft: 0.000 MiB)
|
|
0.49.033.358 I srv load: - looking for better prompt, base f_keep = 0.192, sim = 0.862
|
|
0.49.033.370 I srv update: - cache state: 4 prompts, 273.111 MiB (limits: 8192.000 MiB, 24576 tokens, 72618 est)
|
|
0.49.033.371 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
0.49.033.372 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
0.49.033.372 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
0.49.033.373 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
0.49.033.374 I srv get_availabl: prompt cache update took 62.34 ms
|
|
0.49.033.440 I slot launch_slot_: id 3 | task 1984 | processing task, is_child = 0
|
|
0.49.033.441 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.49.033.442 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.49.033.472 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.49.033.472 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.49.033.494 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.49.033.494 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.49.033.524 I slot update_slots: id 3 | task 1984 | Checking checkpoint with [0, 146] against 0...
|
|
0.49.038.876 W slot update_slots: id 3 | task 1984 | restored context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_past = 146, size = 10.338 MiB)
|
|
0.52.189.871 I slot print_timing: id 3 | task 1984 | n_decoded = 184, tg = 61.04 t/s, tg_3s = 61.04 t/s
|
|
0.55.199.977 I slot print_timing: id 3 | task 1984 | n_decoded = 366, tg = 60.75 t/s, tg_3s = 60.46 t/s
|
|
0.58.203.001 I slot print_timing: id 3 | task 1984 | n_decoded = 548, tg = 60.70 t/s, tg_3s = 60.61 t/s
|
|
1.01.215.831 I slot print_timing: id 3 | task 1984 | n_decoded = 728, tg = 60.46 t/s, tg_3s = 59.74 t/s
|
|
1.04.220.770 I slot print_timing: id 3 | task 1984 | n_decoded = 906, tg = 60.22 t/s, tg_3s = 59.24 t/s
|
|
1.07.229.946 I slot print_timing: id 3 | task 1984 | n_decoded = 1083, tg = 59.99 t/s, tg_3s = 58.82 t/s
|
|
1.10.232.069 I slot print_timing: id 3 | task 1984 | n_decoded = 1259, tg = 59.79 t/s, tg_3s = 58.63 t/s
|
|
1.13.244.821 I slot print_timing: id 3 | task 1984 | n_decoded = 1437, tg = 59.70 t/s, tg_3s = 59.08 t/s
|
|
1.16.254.839 I slot print_timing: id 3 | task 1984 | n_decoded = 1614, tg = 59.60 t/s, tg_3s = 58.80 t/s
|
|
1.19.256.144 I slot print_timing: id 3 | task 1984 | n_decoded = 1790, tg = 59.51 t/s, tg_3s = 58.64 t/s
|
|
1.22.256.319 I slot print_timing: id 3 | task 1984 | n_decoded = 1966, tg = 59.43 t/s, tg_3s = 58.66 t/s
|
|
1.25.260.546 I slot print_timing: id 3 | task 1984 | n_decoded = 2142, tg = 59.36 t/s, tg_3s = 58.58 t/s
|
|
1.28.273.584 I slot print_timing: id 3 | task 1984 | n_decoded = 2319, tg = 59.31 t/s, tg_3s = 58.74 t/s
|
|
1.31.277.036 I slot print_timing: id 3 | task 1984 | n_decoded = 2494, tg = 59.24 t/s, tg_3s = 58.27 t/s
|
|
1.34.283.342 I slot print_timing: id 3 | task 1984 | n_decoded = 2667, tg = 59.13 t/s, tg_3s = 57.55 t/s
|
|
1.37.297.318 I slot print_timing: id 3 | task 1984 | n_decoded = 2838, tg = 58.98 t/s, tg_3s = 56.74 t/s
|
|
1.40.306.249 I slot print_timing: id 3 | task 1984 | n_decoded = 3008, tg = 58.83 t/s, tg_3s = 56.50 t/s
|
|
1.43.307.244 I slot print_timing: id 3 | task 1984 | n_decoded = 3177, tg = 58.69 t/s, tg_3s = 56.31 t/s
|
|
1.46.312.264 I slot print_timing: id 3 | task 1984 | n_decoded = 3346, tg = 58.56 t/s, tg_3s = 56.24 t/s
|
|
1.49.319.392 I slot print_timing: id 3 | task 1984 | n_decoded = 3514, tg = 58.43 t/s, tg_3s = 55.87 t/s
|
|
1.52.329.330 I slot print_timing: id 3 | task 1984 | n_decoded = 3682, tg = 58.30 t/s, tg_3s = 55.82 t/s
|
|
1.55.338.372 I slot print_timing: id 3 | task 1984 | n_decoded = 3849, tg = 58.17 t/s, tg_3s = 55.50 t/s
|
|
1.58.355.120 I slot print_timing: id 3 | task 1984 | n_decoded = 4018, tg = 58.08 t/s, tg_3s = 56.02 t/s
|
|
2.01.356.206 I slot print_timing: id 3 | task 1984 | n_decoded = 4188, tg = 58.02 t/s, tg_3s = 56.65 t/s
|
|
2.04.362.021 I slot print_timing: id 3 | task 1984 | n_decoded = 4358, tg = 57.96 t/s, tg_3s = 56.56 t/s
|
|
2.07.364.030 I slot print_timing: id 3 | task 1984 | n_decoded = 4527, tg = 57.90 t/s, tg_3s = 56.30 t/s
|
|
2.10.370.266 I slot print_timing: id 3 | task 1984 | n_decoded = 4695, tg = 57.82 t/s, tg_3s = 55.88 t/s
|
|
2.13.376.169 I slot print_timing: id 3 | task 1984 | n_decoded = 4862, tg = 57.74 t/s, tg_3s = 55.56 t/s
|
|
2.16.387.058 I slot print_timing: id 3 | task 1984 | n_decoded = 5029, tg = 57.66 t/s, tg_3s = 55.47 t/s
|
|
2.19.396.098 I slot print_timing: id 3 | task 1984 | n_decoded = 5196, tg = 57.59 t/s, tg_3s = 55.50 t/s
|
|
2.22.398.244 I slot print_timing: id 3 | task 1984 | n_decoded = 5362, tg = 57.52 t/s, tg_3s = 55.29 t/s
|
|
2.25.409.636 I slot print_timing: id 3 | task 1984 | n_decoded = 5528, tg = 57.44 t/s, tg_3s = 55.12 t/s
|
|
2.28.419.674 I slot print_timing: id 3 | task 1984 | n_decoded = 5695, tg = 57.38 t/s, tg_3s = 55.48 t/s
|
|
2.31.426.670 I slot print_timing: id 3 | task 1984 | n_decoded = 5861, tg = 57.32 t/s, tg_3s = 55.20 t/s
|
|
2.34.430.783 I slot print_timing: id 3 | task 1984 | n_decoded = 6027, tg = 57.26 t/s, tg_3s = 55.26 t/s
|
|
2.36.542.785 I slot print_timing: id 3 | task 1984 | prompt eval time = 142.04 ms / 28 tokens ( 5.07 ms per token, 197.12 tokens per second)
|
|
2.36.542.788 I slot print_timing: id 3 | task 1984 | eval time = 107367.21 ms / 6144 tokens ( 17.48 ms per token, 57.22 tokens per second)
|
|
2.36.542.789 I slot print_timing: id 3 | task 1984 | total time = 107509.26 ms / 6172 tokens
|
|
2.36.542.790 I slot print_timing: id 3 | task 1984 | graphs reused = 8058
|
|
2.36.542.828 I slot release: id 3 | task 1984 | stop processing: n_tokens = 6317, truncated = 0
|
|
2.36.542.857 I srv update_slots: all slots are idle
|
|
2.36.548.935 I srv operator(): Chat format: peg-native
|
|
2.36.550.950 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.559 (> 0.100 thold), f_keep = 0.018
|
|
2.36.550.951 I srv get_availabl: updating prompt cache
|
|
2.36.551.347 W srv prompt_save: - saving prompt with length 6317, total state size = 448.513 MiB (draft: 0.000 MiB)
|
|
2.36.993.165 I srv load: - looking for better prompt, base f_keep = 0.018, sim = 0.559
|
|
2.36.993.175 I srv update: - cache state: 5 prompts, 731.963 MiB (limits: 8192.000 MiB, 24576 tokens, 97794 est)
|
|
2.36.993.176 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
2.36.993.177 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
2.36.993.177 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
2.36.993.178 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
2.36.993.178 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
2.36.993.180 I srv get_availabl: prompt cache update took 442.23 ms
|
|
2.36.993.224 I slot launch_slot_: id 3 | task 8131 | processing task, is_child = 0
|
|
2.36.993.225 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.36.993.225 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.36.993.279 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.36.993.280 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.36.993.327 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.36.993.327 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.36.993.383 I slot update_slots: id 3 | task 8131 | Checking checkpoint with [0, 146] against 0...
|
|
2.36.993.385 W slot update_slots: id 3 | task 8131 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
|
|
2.36.993.386 W slot update_slots: id 3 | task 8131 | erased invalidated context checkpoint (pos_min = 0, pos_max = 146, n_tokens = 147, n_swa = 4096, pos_next = 0, size = 10.338 MiB)
|
|
2.37.310.436 I slot create_check: id 3 | task 8131 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 166, n_tokens = 167, size = 11.745 MiB)
|
|
2.40.458.679 I slot print_timing: id 3 | task 8131 | n_decoded = 183, tg = 60.96 t/s, tg_3s = 60.96 t/s
|
|
2.43.460.463 I slot print_timing: id 3 | task 8131 | n_decoded = 364, tg = 60.63 t/s, tg_3s = 60.30 t/s
|
|
2.46.468.652 I slot print_timing: id 3 | task 8131 | n_decoded = 546, tg = 60.59 t/s, tg_3s = 60.50 t/s
|
|
2.49.479.887 I slot print_timing: id 3 | task 8131 | n_decoded = 726, tg = 60.38 t/s, tg_3s = 59.78 t/s
|
|
2.52.486.159 I slot print_timing: id 3 | task 8131 | n_decoded = 907, tg = 60.35 t/s, tg_3s = 60.21 t/s
|
|
2.55.082.171 I slot print_timing: id 3 | task 8131 | prompt eval time = 463.37 ms / 204 tokens ( 2.27 ms per token, 440.25 tokens per second)
|
|
2.55.082.175 I slot print_timing: id 3 | task 8131 | eval time = 17625.41 ms / 1064 tokens ( 16.57 ms per token, 60.37 tokens per second)
|
|
2.55.082.176 I slot print_timing: id 3 | task 8131 | total time = 18088.78 ms / 1268 tokens
|
|
2.55.082.176 I slot print_timing: id 3 | task 8131 | graphs reused = 9116
|
|
2.55.082.215 I slot release: id 3 | task 8131 | stop processing: n_tokens = 1267, truncated = 0
|
|
2.55.082.227 I srv update_slots: all slots are idle
|
|
2.55.087.837 I srv operator(): Chat format: peg-native
|
|
2.55.089.872 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.746 (> 0.100 thold), f_keep = 0.134
|
|
2.55.089.873 I srv get_availabl: updating prompt cache
|
|
2.55.089.995 W srv prompt_save: - saving prompt with length 1267, total state size = 121.286 MiB (draft: 0.000 MiB)
|
|
2.55.192.554 I srv load: - looking for better prompt, base f_keep = 0.134, sim = 0.746
|
|
2.55.192.563 I srv update: - cache state: 6 prompts, 864.994 MiB (limits: 8192.000 MiB, 24576 tokens, 94753 est)
|
|
2.55.192.564 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
2.55.192.565 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
2.55.192.566 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
2.55.192.566 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
2.55.192.567 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
2.55.192.568 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
2.55.192.569 I srv get_availabl: prompt cache update took 102.69 ms
|
|
2.55.192.614 I slot launch_slot_: id 3 | task 9198 | processing task, is_child = 0
|
|
2.55.192.615 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.55.192.616 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.55.192.652 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.55.192.653 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.55.192.709 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.55.192.709 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.55.192.740 I slot update_slots: id 3 | task 9198 | Checking checkpoint with [0, 166] against 0...
|
|
2.55.198.353 W slot update_slots: id 3 | task 9198 | restored context checkpoint (pos_min = 0, pos_max = 166, n_tokens = 167, n_past = 166, size = 11.745 MiB)
|
|
2.58.461.941 I slot print_timing: id 3 | task 9198 | n_decoded = 184, tg = 61.24 t/s, tg_3s = 61.24 t/s
|
|
3.00.548.000 I slot print_timing: id 3 | task 9198 | prompt eval time = 264.86 ms / 62 tokens ( 4.27 ms per token, 234.08 tokens per second)
|
|
3.00.548.003 I slot print_timing: id 3 | task 9198 | eval time = 5090.39 ms / 311 tokens ( 16.37 ms per token, 61.10 tokens per second)
|
|
3.00.548.004 I slot print_timing: id 3 | task 9198 | total time = 5355.26 ms / 373 tokens
|
|
3.00.548.005 I slot print_timing: id 3 | task 9198 | graphs reused = 9423
|
|
3.00.548.047 I slot release: id 3 | task 9198 | stop processing: n_tokens = 538, truncated = 0
|
|
3.00.548.058 I srv update_slots: all slots are idle
|
|
3.00.553.423 I srv operator(): Chat format: peg-native
|
|
3.00.555.474 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.759 (> 0.100 thold), f_keep = 0.316
|
|
3.00.555.476 I srv get_availabl: updating prompt cache
|
|
3.00.555.547 W srv prompt_save: - saving prompt with length 538, total state size = 51.502 MiB (draft: 0.000 MiB)
|
|
3.00.601.233 I srv load: - looking for better prompt, base f_keep = 0.316, sim = 0.759
|
|
3.00.601.246 I srv update: - cache state: 7 prompts, 928.240 MiB (limits: 8192.000 MiB, 24576 tokens, 93045 est)
|
|
3.00.601.247 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
3.00.601.247 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
3.00.601.248 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
3.00.601.248 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
3.00.601.249 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
3.00.601.249 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
3.00.601.250 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
|
|
3.00.601.251 I srv get_availabl: prompt cache update took 45.77 ms
|
|
3.00.601.316 I slot launch_slot_: id 3 | task 9512 | processing task, is_child = 0
|
|
3.00.601.317 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.00.601.318 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.00.601.351 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.00.601.351 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.00.601.372 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.00.601.373 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.00.601.402 I slot update_slots: id 3 | task 9512 | Checking checkpoint with [0, 166] against 0...
|
|
3.00.606.955 W slot update_slots: id 3 | task 9512 | restored context checkpoint (pos_min = 0, pos_max = 166, n_tokens = 167, n_past = 166, size = 11.745 MiB)
|
|
3.03.840.849 I slot print_timing: id 3 | task 9512 | n_decoded = 185, tg = 61.37 t/s, tg_3s = 61.37 t/s
|
|
3.06.852.053 I slot print_timing: id 3 | task 9512 | n_decoded = 368, tg = 61.07 t/s, tg_3s = 60.77 t/s
|
|
3.09.860.126 I slot print_timing: id 3 | task 9512 | n_decoded = 550, tg = 60.88 t/s, tg_3s = 60.50 t/s
|
|
3.12.869.195 I slot print_timing: id 3 | task 9512 | n_decoded = 732, tg = 60.78 t/s, tg_3s = 60.48 t/s
|
|
3.15.882.937 I slot print_timing: id 3 | task 9512 | n_decoded = 912, tg = 60.57 t/s, tg_3s = 59.73 t/s
|
|
3.18.889.315 I slot print_timing: id 3 | task 9512 | n_decoded = 1091, tg = 60.40 t/s, tg_3s = 59.54 t/s
|
|
3.19.193.163 I slot print_timing: id 3 | task 9512 | prompt eval time = 224.96 ms / 58 tokens ( 3.88 ms per token, 257.82 tokens per second)
|
|
3.19.193.167 I slot print_timing: id 3 | task 9512 | eval time = 18366.79 ms / 1109 tokens ( 16.56 ms per token, 60.38 tokens per second)
|
|
3.19.193.167 I slot print_timing: id 3 | task 9512 | total time = 18591.76 ms / 1167 tokens
|
|
3.19.193.168 I slot print_timing: id 3 | task 9512 | graphs reused = 10525
|
|
3.19.193.243 I slot release: id 3 | task 9512 | stop processing: n_tokens = 1332, truncated = 0
|
|
3.19.193.257 I srv update_slots: all slots are idle
|
|
3.19.198.986 I srv operator(): Chat format: peg-native
|
|
3.19.201.120 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.726 (> 0.100 thold), f_keep = 0.128
|
|
3.19.201.122 I srv get_availabl: updating prompt cache
|
|
3.19.201.246 W srv prompt_save: - saving prompt with length 1332, total state size = 127.508 MiB (draft: 0.000 MiB)
|
|
3.19.343.081 I srv load: - looking for better prompt, base f_keep = 0.128, sim = 0.726
|
|
3.19.343.091 I srv update: - cache state: 8 prompts, 1067.493 MiB (limits: 8192.000 MiB, 24576 tokens, 91129 est)
|
|
3.19.343.092 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
3.19.343.092 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
3.19.343.093 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
3.19.343.093 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
3.19.343.094 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
3.19.343.094 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
3.19.343.095 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
|
|
3.19.343.095 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
|
|
3.19.343.097 I srv get_availabl: prompt cache update took 141.97 ms
|
|
3.19.343.147 I slot launch_slot_: id 3 | task 10624 | processing task, is_child = 0
|
|
3.19.343.148 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.19.343.149 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.19.343.181 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.19.343.182 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.19.343.203 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.19.343.204 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.19.343.234 I slot update_slots: id 3 | task 10624 | Checking checkpoint with [0, 166] against 0...
|
|
3.19.349.281 W slot update_slots: id 3 | task 10624 | restored context checkpoint (pos_min = 0, pos_max = 166, n_tokens = 167, n_past = 166, size = 11.745 MiB)
|
|
3.22.597.727 I slot print_timing: id 3 | task 10624 | n_decoded = 182, tg = 60.63 t/s, tg_3s = 60.63 t/s
|
|
3.25.601.122 I slot print_timing: id 3 | task 10624 | n_decoded = 363, tg = 60.45 t/s, tg_3s = 60.27 t/s
|
|
3.28.617.874 I slot print_timing: id 3 | task 10624 | n_decoded = 545, tg = 60.41 t/s, tg_3s = 60.33 t/s
|
|
3.31.620.881 I slot print_timing: id 3 | task 10624 | n_decoded = 725, tg = 60.29 t/s, tg_3s = 59.94 t/s
|
|
3.34.631.835 I slot print_timing: id 3 | task 10624 | n_decoded = 903, tg = 60.06 t/s, tg_3s = 59.12 t/s
|
|
3.37.643.467 I slot print_timing: id 3 | task 10624 | n_decoded = 1081, tg = 59.90 t/s, tg_3s = 59.10 t/s
|
|
3.40.646.476 I slot print_timing: id 3 | task 10624 | n_decoded = 1261, tg = 59.90 t/s, tg_3s = 59.94 t/s
|
|
3.43.650.101 I slot print_timing: id 3 | task 10624 | n_decoded = 1440, tg = 59.86 t/s, tg_3s = 59.59 t/s
|
|
3.46.659.681 I slot print_timing: id 3 | task 10624 | n_decoded = 1618, tg = 59.78 t/s, tg_3s = 59.14 t/s
|
|
3.49.672.715 I slot print_timing: id 3 | task 10624 | n_decoded = 1795, tg = 59.68 t/s, tg_3s = 58.74 t/s
|
|
3.52.673.112 I slot print_timing: id 3 | task 10624 | n_decoded = 1971, tg = 59.59 t/s, tg_3s = 58.66 t/s
|
|
3.55.685.278 I slot print_timing: id 3 | task 10624 | n_decoded = 2147, tg = 59.49 t/s, tg_3s = 58.43 t/s
|
|
3.58.695.280 I slot print_timing: id 3 | task 10624 | n_decoded = 2322, tg = 59.39 t/s, tg_3s = 58.14 t/s
|
|
4.01.569.988 I slot print_timing: id 3 | task 10624 | prompt eval time = 252.59 ms / 68 tokens ( 3.71 ms per token, 269.22 tokens per second)
|
|
4.01.569.992 I slot print_timing: id 3 | task 10624 | eval time = 41974.17 ms / 2488 tokens ( 16.87 ms per token, 59.27 tokens per second)
|
|
4.01.569.992 I slot print_timing: id 3 | task 10624 | total time = 42226.75 ms / 2556 tokens
|
|
4.01.569.993 I slot print_timing: id 3 | task 10624 | graphs reused = 13001
|
|
4.01.570.029 I slot release: id 3 | task 10624 | stop processing: n_tokens = 2721, truncated = 0
|
|
4.01.570.047 I srv update_slots: all slots are idle
|
|
4.01.575.654 I srv operator(): Chat format: peg-native
|
|
4.01.577.669 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.821 (> 0.100 thold), f_keep = 0.062
|
|
4.01.577.671 I srv get_availabl: updating prompt cache
|
|
4.01.577.886 W srv prompt_save: - saving prompt with length 2721, total state size = 260.472 MiB (draft: 0.000 MiB)
|
|
4.01.848.857 I srv load: - looking for better prompt, base f_keep = 0.062, sim = 0.821
|
|
4.01.848.869 I srv update: - cache state: 9 prompts, 1339.710 MiB (limits: 8192.000 MiB, 24576 tokens, 89250 est)
|
|
4.01.848.870 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
4.01.848.871 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
4.01.848.871 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
4.01.848.872 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
4.01.848.872 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
4.01.848.873 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
4.01.848.873 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
|
|
4.01.848.874 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
|
|
4.01.848.874 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
|
|
4.01.848.876 I srv get_availabl: prompt cache update took 271.20 ms
|
|
4.01.848.919 I slot launch_slot_: id 3 | task 13115 | processing task, is_child = 0
|
|
4.01.848.920 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.01.848.920 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.01.848.965 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.01.848.966 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.01.849.005 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.01.849.005 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.01.849.037 I slot update_slots: id 3 | task 13115 | Checking checkpoint with [0, 166] against 0...
|
|
4.01.855.125 W slot update_slots: id 3 | task 13115 | restored context checkpoint (pos_min = 0, pos_max = 166, n_tokens = 167, n_past = 166, size = 11.745 MiB)
|
|
4.05.029.683 I slot print_timing: id 3 | task 13115 | n_decoded = 184, tg = 61.03 t/s, tg_3s = 61.03 t/s
|
|
4.08.043.797 I slot print_timing: id 3 | task 13115 | n_decoded = 367, tg = 60.87 t/s, tg_3s = 60.71 t/s
|
|
4.11.054.842 I slot print_timing: id 3 | task 13115 | n_decoded = 549, tg = 60.73 t/s, tg_3s = 60.44 t/s
|
|
4.14.058.004 I slot print_timing: id 3 | task 13115 | n_decoded = 729, tg = 60.53 t/s, tg_3s = 59.94 t/s
|
|
4.14.925.558 I slot print_timing: id 3 | task 13115 | prompt eval time = 165.76 ms / 41 tokens ( 4.04 ms per token, 247.34 tokens per second)
|
|
4.14.925.562 I slot print_timing: id 3 | task 13115 | eval time = 12910.75 ms / 781 tokens ( 16.53 ms per token, 60.49 tokens per second)
|
|
4.14.925.563 I slot print_timing: id 3 | task 13115 | total time = 13076.51 ms / 822 tokens
|
|
4.14.925.564 I slot print_timing: id 3 | task 13115 | graphs reused = 13777
|
|
4.14.925.607 I slot release: id 3 | task 13115 | stop processing: n_tokens = 987, truncated = 0
|
|
4.14.925.619 I srv update_slots: all slots are idle
|
|
4.14.931.399 I srv operator(): Chat format: peg-native
|
|
4.14.933.427 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.762 (> 0.100 thold), f_keep = 0.172
|
|
4.14.933.428 I srv get_availabl: updating prompt cache
|
|
4.14.933.525 W srv prompt_save: - saving prompt with length 987, total state size = 94.483 MiB (draft: 0.000 MiB)
|
|
4.15.009.824 I srv load: - looking for better prompt, base f_keep = 0.172, sim = 0.762
|
|
4.15.009.839 I srv update: - cache state: 10 prompts, 1445.938 MiB (limits: 8192.000 MiB, 24576 tokens, 88285 est)
|
|
4.15.009.840 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
4.15.009.841 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
4.15.009.841 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
4.15.009.842 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
4.15.009.842 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
4.15.009.843 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
4.15.009.843 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
|
|
4.15.009.844 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
|
|
4.15.009.844 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
|
|
4.15.009.845 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
|
|
4.15.009.846 I srv get_availabl: prompt cache update took 76.42 ms
|
|
4.15.009.911 I slot launch_slot_: id 3 | task 13899 | processing task, is_child = 0
|
|
4.15.009.913 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.15.009.913 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.15.009.944 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.15.009.945 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.15.009.966 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.15.009.966 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.15.009.997 I slot update_slots: id 3 | task 13899 | Checking checkpoint with [0, 166] against 0...
|
|
4.15.016.277 W slot update_slots: id 3 | task 13899 | restored context checkpoint (pos_min = 0, pos_max = 166, n_tokens = 167, n_past = 166, size = 11.745 MiB)
|
|
4.18.240.379 I slot print_timing: id 3 | task 13899 | n_decoded = 182, tg = 60.59 t/s, tg_3s = 60.59 t/s
|
|
4.21.252.443 I slot print_timing: id 3 | task 13899 | n_decoded = 364, tg = 60.51 t/s, tg_3s = 60.42 t/s
|
|
4.24.256.959 I slot print_timing: id 3 | task 13899 | n_decoded = 546, tg = 60.53 t/s, tg_3s = 60.58 t/s
|
|
4.24.995.530 I slot print_timing: id 3 | task 13899 | prompt eval time = 226.71 ms / 57 tokens ( 3.98 ms per token, 251.43 tokens per second)
|
|
4.24.995.533 I slot print_timing: id 3 | task 13899 | eval time = 9758.82 ms / 589 tokens ( 16.57 ms per token, 60.36 tokens per second)
|
|
4.24.995.534 I slot print_timing: id 3 | task 13899 | total time = 9985.53 ms / 646 tokens
|
|
4.24.995.535 I slot print_timing: id 3 | task 13899 | graphs reused = 14361
|
|
4.24.995.580 I slot release: id 3 | task 13899 | stop processing: n_tokens = 811, truncated = 0
|
|
4.24.995.590 I srv update_slots: all slots are idle
|
|
4.25.001.184 I srv operator(): Chat format: peg-native
|
|
4.25.003.195 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.483 (> 0.100 thold), f_keep = 0.141
|
|
4.25.003.197 I srv get_availabl: updating prompt cache
|
|
4.25.003.287 W srv prompt_save: - saving prompt with length 811, total state size = 77.635 MiB (draft: 0.000 MiB)
|
|
4.25.067.968 I srv load: - looking for better prompt, base f_keep = 0.141, sim = 0.483
|
|
4.25.067.982 I srv update: - cache state: 11 prompts, 1535.318 MiB (limits: 8192.000 MiB, 24576 tokens, 87473 est)
|
|
4.25.067.983 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
4.25.067.984 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
4.25.067.985 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
4.25.067.985 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
4.25.067.986 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
4.25.067.986 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
4.25.067.987 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
|
|
4.25.067.987 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
|
|
4.25.067.988 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
|
|
4.25.067.988 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
|
|
4.25.067.989 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
|
|
4.25.067.990 I srv get_availabl: prompt cache update took 64.79 ms
|
|
4.25.068.055 I slot launch_slot_: id 3 | task 14491 | processing task, is_child = 0
|
|
4.25.068.056 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.25.068.056 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.25.068.085 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.25.068.086 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.25.068.107 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.25.068.107 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.25.068.138 I slot update_slots: id 3 | task 14491 | Checking checkpoint with [0, 166] against 0...
|
|
4.25.068.139 W slot update_slots: id 3 | task 14491 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
|
|
4.25.068.140 W slot update_slots: id 3 | task 14491 | erased invalidated context checkpoint (pos_min = 0, pos_max = 166, n_tokens = 167, n_swa = 4096, pos_next = 0, size = 11.745 MiB)
|
|
4.25.382.006 I slot create_check: id 3 | task 14491 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 161, n_tokens = 162, size = 11.393 MiB)
|
|
4.28.590.513 I slot print_timing: id 3 | task 14491 | n_decoded = 184, tg = 61.26 t/s, tg_3s = 61.26 t/s
|
|
4.29.915.933 I slot print_timing: id 3 | task 14491 | prompt eval time = 518.61 ms / 236 tokens ( 2.20 ms per token, 455.07 tokens per second)
|
|
4.29.915.938 I slot print_timing: id 3 | task 14491 | eval time = 4329.19 ms / 265 tokens ( 16.34 ms per token, 61.21 tokens per second)
|
|
4.29.915.938 I slot print_timing: id 3 | task 14491 | total time = 4847.79 ms / 501 tokens
|
|
4.29.915.939 I slot print_timing: id 3 | task 14491 | graphs reused = 14623
|
|
4.29.915.984 I slot release: id 3 | task 14491 | stop processing: n_tokens = 500, truncated = 0
|
|
4.29.915.993 I srv update_slots: all slots are idle
|
|
4.29.982.474 I srv operator(): Chat format: peg-native
|
|
4.29.984.520 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.863 (> 0.100 thold), f_keep = 0.340
|
|
4.29.984.522 I srv get_availabl: updating prompt cache
|
|
4.29.984.598 W srv prompt_save: - saving prompt with length 500, total state size = 47.864 MiB (draft: 0.000 MiB)
|
|
4.30.027.232 I srv load: - looking for better prompt, base f_keep = 0.340, sim = 0.863
|
|
4.30.027.247 I srv update: - cache state: 12 prompts, 1594.575 MiB (limits: 8192.000 MiB, 24576 tokens, 86791 est)
|
|
4.30.027.248 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
4.30.027.248 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
4.30.027.249 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
4.30.027.249 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
4.30.027.250 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
4.30.027.251 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
4.30.027.251 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
|
|
4.30.027.252 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
|
|
4.30.027.252 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
|
|
4.30.027.253 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
|
|
4.30.027.253 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
|
|
4.30.027.254 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
|
|
4.30.027.255 I srv get_availabl: prompt cache update took 42.73 ms
|
|
4.30.027.318 I slot launch_slot_: id 3 | task 14759 | processing task, is_child = 0
|
|
4.30.027.319 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.30.027.320 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.30.027.348 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.30.027.348 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.30.027.369 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.30.027.370 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.30.027.399 I slot update_slots: id 3 | task 14759 | Checking checkpoint with [0, 161] against 0...
|
|
4.30.032.812 W slot update_slots: id 3 | task 14759 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
|
|
4.32.925.419 I slot print_timing: id 3 | task 14759 | prompt eval time = 169.52 ms / 36 tokens ( 4.71 ms per token, 212.36 tokens per second)
|
|
4.32.925.424 I slot print_timing: id 3 | task 14759 | eval time = 2728.49 ms / 169 tokens ( 16.14 ms per token, 61.94 tokens per second)
|
|
4.32.925.425 I slot print_timing: id 3 | task 14759 | total time = 2898.01 ms / 205 tokens
|
|
4.32.925.425 I slot print_timing: id 3 | task 14759 | graphs reused = 14789
|
|
4.32.925.509 I slot release: id 3 | task 14759 | stop processing: n_tokens = 365, truncated = 0
|
|
4.32.925.522 I srv update_slots: all slots are idle
|
|
4.32.976.069 I srv operator(): Chat format: peg-native
|
|
4.32.978.229 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.833 (> 0.100 thold), f_keep = 0.466
|
|
4.32.978.232 I srv get_availabl: updating prompt cache
|
|
4.32.978.306 W srv prompt_save: - saving prompt with length 365, total state size = 34.941 MiB (draft: 0.000 MiB)
|
|
4.33.011.855 I srv load: - looking for better prompt, base f_keep = 0.466, sim = 0.833
|
|
4.33.011.873 I srv update: - cache state: 13 prompts, 1640.910 MiB (limits: 8192.000 MiB, 24576 tokens, 86163 est)
|
|
4.33.011.874 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
4.33.011.875 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
4.33.011.876 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
4.33.011.876 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
4.33.011.877 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
4.33.011.877 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
4.33.011.878 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
|
|
4.33.011.878 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
|
|
4.33.011.879 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
|
|
4.33.011.879 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
|
|
4.33.011.880 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
|
|
4.33.011.880 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
|
|
4.33.011.881 I srv update: - prompt 0x5a8713dd4a40: 365 tokens, checkpoints: 1, 46.334 MiB
|
|
4.33.011.882 I srv get_availabl: prompt cache update took 33.65 ms
|
|
4.33.011.944 I slot launch_slot_: id 3 | task 14931 | processing task, is_child = 0
|
|
4.33.011.945 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.33.011.946 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.33.011.976 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.33.011.976 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.33.011.997 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.33.011.997 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.33.012.026 I slot update_slots: id 3 | task 14931 | Checking checkpoint with [0, 161] against 0...
|
|
4.33.017.477 W slot update_slots: id 3 | task 14931 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
|
|
4.36.201.931 I slot print_timing: id 3 | task 14931 | n_decoded = 186, tg = 61.69 t/s, tg_3s = 61.69 t/s
|
|
4.39.210.578 I slot print_timing: id 3 | task 14931 | n_decoded = 371, tg = 61.59 t/s, tg_3s = 61.49 t/s
|
|
4.42.218.897 I slot print_timing: id 3 | task 14931 | n_decoded = 555, tg = 61.45 t/s, tg_3s = 61.16 t/s
|
|
4.43.414.821 I slot print_timing: id 3 | task 14931 | prompt eval time = 174.86 ms / 43 tokens ( 4.07 ms per token, 245.92 tokens per second)
|
|
4.43.414.825 I slot print_timing: id 3 | task 14931 | eval time = 10227.93 ms / 627 tokens ( 16.31 ms per token, 61.30 tokens per second)
|
|
4.43.414.825 I slot print_timing: id 3 | task 14931 | total time = 10402.79 ms / 670 tokens
|
|
4.43.414.826 I slot print_timing: id 3 | task 14931 | graphs reused = 15411
|
|
4.43.414.868 I slot release: id 3 | task 14931 | stop processing: n_tokens = 830, truncated = 0
|
|
4.43.414.879 I srv update_slots: all slots are idle
|
|
4.43.480.631 I srv operator(): Chat format: peg-native
|
|
4.43.482.884 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.733 (> 0.100 thold), f_keep = 0.205
|
|
4.43.482.887 I srv get_availabl: updating prompt cache
|
|
4.43.482.981 W srv prompt_save: - saving prompt with length 830, total state size = 79.454 MiB (draft: 0.000 MiB)
|
|
4.43.548.193 I srv load: - looking for better prompt, base f_keep = 0.205, sim = 0.733
|
|
4.43.548.209 I srv update: - cache state: 14 prompts, 1731.757 MiB (limits: 8192.000 MiB, 24576 tokens, 85569 est)
|
|
4.43.548.210 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
4.43.548.211 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
4.43.548.211 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
4.43.548.212 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
4.43.548.213 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
4.43.548.213 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
4.43.548.214 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
|
|
4.43.548.214 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
|
|
4.43.548.215 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
|
|
4.43.548.215 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
|
|
4.43.548.216 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
|
|
4.43.548.216 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
|
|
4.43.548.217 I srv update: - prompt 0x5a8713dd4a40: 365 tokens, checkpoints: 1, 46.334 MiB
|
|
4.43.548.217 I srv update: - prompt 0x5a8710079620: 830 tokens, checkpoints: 1, 90.847 MiB
|
|
4.43.548.218 I srv get_availabl: prompt cache update took 65.33 ms
|
|
4.43.548.291 I slot launch_slot_: id 3 | task 15561 | processing task, is_child = 0
|
|
4.43.548.291 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.43.548.292 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.43.548.325 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.43.548.325 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.43.548.346 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.43.548.347 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.43.548.376 I slot update_slots: id 3 | task 15561 | Checking checkpoint with [0, 161] against 0...
|
|
4.43.554.013 W slot update_slots: id 3 | task 15561 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
|
|
4.46.773.827 I slot print_timing: id 3 | task 15561 | n_decoded = 185, tg = 61.36 t/s, tg_3s = 61.36 t/s
|
|
4.49.785.627 I slot print_timing: id 3 | task 15561 | n_decoded = 368, tg = 61.06 t/s, tg_3s = 60.76 t/s
|
|
4.52.802.043 I slot print_timing: id 3 | task 15561 | n_decoded = 551, tg = 60.93 t/s, tg_3s = 60.67 t/s
|
|
4.55.817.547 I slot print_timing: id 3 | task 15561 | n_decoded = 732, tg = 60.70 t/s, tg_3s = 60.02 t/s
|
|
4.58.821.170 I slot print_timing: id 3 | task 15561 | n_decoded = 911, tg = 60.48 t/s, tg_3s = 59.59 t/s
|
|
5.01.835.766 I slot print_timing: id 3 | task 15561 | n_decoded = 1090, tg = 60.30 t/s, tg_3s = 59.38 t/s
|
|
5.04.850.739 I slot print_timing: id 3 | task 15561 | n_decoded = 1267, tg = 60.07 t/s, tg_3s = 58.71 t/s
|
|
5.07.857.858 I slot print_timing: id 3 | task 15561 | n_decoded = 1443, tg = 59.88 t/s, tg_3s = 58.53 t/s
|
|
5.10.862.994 I slot print_timing: id 3 | task 15561 | n_decoded = 1619, tg = 59.73 t/s, tg_3s = 58.57 t/s
|
|
5.13.872.105 I slot print_timing: id 3 | task 15561 | n_decoded = 1795, tg = 59.61 t/s, tg_3s = 58.49 t/s
|
|
5.14.532.111 I slot print_timing: id 3 | task 15561 | prompt eval time = 210.34 ms / 71 tokens ( 2.96 ms per token, 337.55 tokens per second)
|
|
5.14.532.115 I slot print_timing: id 3 | task 15561 | eval time = 30773.39 ms / 1833 tokens ( 16.79 ms per token, 59.56 tokens per second)
|
|
5.14.532.116 I slot print_timing: id 3 | task 15561 | total time = 30983.73 ms / 1904 tokens
|
|
5.14.532.117 I slot print_timing: id 3 | task 15561 | graphs reused = 17234
|
|
5.14.532.158 I slot release: id 3 | task 15561 | stop processing: n_tokens = 2064, truncated = 0
|
|
5.14.532.172 I srv update_slots: all slots are idle
|
|
5.14.609.746 I srv operator(): Chat format: peg-native
|
|
5.14.611.798 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.821 (> 0.100 thold), f_keep = 0.082
|
|
5.14.611.800 I srv get_availabl: updating prompt cache
|
|
5.14.611.957 W srv prompt_save: - saving prompt with length 2064, total state size = 197.580 MiB (draft: 0.000 MiB)
|
|
5.14.756.512 I srv load: - looking for better prompt, base f_keep = 0.082, sim = 0.821
|
|
5.14.756.527 I srv update: - cache state: 15 prompts, 1940.730 MiB (limits: 8192.000 MiB, 24576 tokens, 85067 est)
|
|
5.14.756.528 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
5.14.756.529 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
5.14.756.529 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
5.14.756.530 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
5.14.756.530 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
5.14.756.531 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
5.14.756.531 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
|
|
5.14.756.532 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
|
|
5.14.756.532 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
|
|
5.14.756.533 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
|
|
5.14.756.533 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
|
|
5.14.756.534 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
|
|
5.14.756.534 I srv update: - prompt 0x5a8713dd4a40: 365 tokens, checkpoints: 1, 46.334 MiB
|
|
5.14.756.535 I srv update: - prompt 0x5a8710079620: 830 tokens, checkpoints: 1, 90.847 MiB
|
|
5.14.756.535 I srv update: - prompt 0x5a870cfb0770: 2064 tokens, checkpoints: 1, 208.973 MiB
|
|
5.14.756.537 I srv get_availabl: prompt cache update took 144.74 ms
|
|
5.14.756.586 I slot launch_slot_: id 3 | task 17397 | processing task, is_child = 0
|
|
5.14.756.586 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.14.756.587 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.14.756.615 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.14.756.616 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.14.756.638 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.14.756.638 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.14.756.669 I slot update_slots: id 3 | task 17397 | Checking checkpoint with [0, 161] against 0...
|
|
5.14.762.487 W slot update_slots: id 3 | task 17397 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
|
|
5.17.948.439 I slot print_timing: id 3 | task 17397 | n_decoded = 185, tg = 61.48 t/s, tg_3s = 61.48 t/s
|
|
5.20.856.560 I slot print_timing: id 3 | task 17397 | prompt eval time = 182.83 ms / 46 tokens ( 3.97 ms per token, 251.60 tokens per second)
|
|
5.20.856.564 I slot print_timing: id 3 | task 17397 | eval time = 5917.06 ms / 363 tokens ( 16.30 ms per token, 61.35 tokens per second)
|
|
5.20.856.565 I slot print_timing: id 3 | task 17397 | total time = 6099.89 ms / 409 tokens
|
|
5.20.856.566 I slot print_timing: id 3 | task 17397 | graphs reused = 17593
|
|
5.20.856.610 I slot release: id 3 | task 17397 | stop processing: n_tokens = 569, truncated = 0
|
|
5.20.856.620 I srv update_slots: all slots are idle
|
|
5.20.906.755 I srv operator(): Chat format: peg-native
|
|
5.20.908.800 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.757 (> 0.100 thold), f_keep = 0.290
|
|
5.20.908.802 I srv get_availabl: updating prompt cache
|
|
5.20.908.869 W srv prompt_save: - saving prompt with length 569, total state size = 54.469 MiB (draft: 0.000 MiB)
|
|
5.20.955.876 I srv load: - looking for better prompt, base f_keep = 0.290, sim = 0.757
|
|
5.20.955.895 I srv update: - cache state: 16 prompts, 2006.592 MiB (limits: 8192.000 MiB, 24576 tokens, 84598 est)
|
|
5.20.955.896 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
5.20.955.896 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
5.20.955.897 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
5.20.955.897 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
5.20.955.898 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
5.20.955.898 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
5.20.955.899 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
|
|
5.20.955.899 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
|
|
5.20.955.900 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
|
|
5.20.955.901 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
|
|
5.20.955.902 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
|
|
5.20.955.903 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
|
|
5.20.955.903 I srv update: - prompt 0x5a8713dd4a40: 365 tokens, checkpoints: 1, 46.334 MiB
|
|
5.20.955.904 I srv update: - prompt 0x5a8710079620: 830 tokens, checkpoints: 1, 90.847 MiB
|
|
5.20.955.904 I srv update: - prompt 0x5a870cfb0770: 2064 tokens, checkpoints: 1, 208.973 MiB
|
|
5.20.955.905 I srv update: - prompt 0x5a871427acf0: 569 tokens, checkpoints: 1, 65.863 MiB
|
|
5.20.955.906 I srv get_availabl: prompt cache update took 47.10 ms
|
|
5.20.955.973 I slot launch_slot_: id 3 | task 17763 | processing task, is_child = 0
|
|
5.20.955.974 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.20.955.975 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.20.956.007 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.20.956.007 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.20.956.029 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.20.956.029 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.20.956.061 I slot update_slots: id 3 | task 17763 | Checking checkpoint with [0, 161] against 0...
|
|
5.20.961.893 W slot update_slots: id 3 | task 17763 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
|
|
5.23.507.468 I slot print_timing: id 3 | task 17763 | prompt eval time = 230.63 ms / 57 tokens ( 4.05 ms per token, 247.15 tokens per second)
|
|
5.23.507.472 I slot print_timing: id 3 | task 17763 | eval time = 2320.77 ms / 143 tokens ( 16.23 ms per token, 61.62 tokens per second)
|
|
5.23.507.473 I slot print_timing: id 3 | task 17763 | total time = 2551.40 ms / 200 tokens
|
|
5.23.507.473 I slot print_timing: id 3 | task 17763 | graphs reused = 17733
|
|
5.23.507.513 I slot release: id 3 | task 17763 | stop processing: n_tokens = 360, truncated = 0
|
|
5.23.507.524 I srv update_slots: all slots are idle
|
|
5.23.512.807 I srv operator(): Chat format: peg-native
|
|
5.23.514.861 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.805 (> 0.100 thold), f_keep = 0.458
|
|
5.23.514.863 I srv get_availabl: updating prompt cache
|
|
5.23.514.940 W srv prompt_save: - saving prompt with length 360, total state size = 34.463 MiB (draft: 0.000 MiB)
|
|
5.23.547.717 I srv load: - looking for better prompt, base f_keep = 0.458, sim = 0.805
|
|
5.23.547.729 I srv load: - found better prompt with f_keep = 0.466, sim = 0.829
|
|
5.23.563.466 I srv update: - cache state: 16 prompts, 2006.114 MiB (limits: 8192.000 MiB, 24576 tokens, 84598 est)
|
|
5.23.563.470 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
5.23.563.471 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
5.23.563.471 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
5.23.563.472 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
5.23.563.472 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
5.23.563.473 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
5.23.563.473 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
|
|
5.23.563.474 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
|
|
5.23.563.474 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
|
|
5.23.563.475 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
|
|
5.23.563.475 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
|
|
5.23.563.476 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
|
|
5.23.563.476 I srv update: - prompt 0x5a8710079620: 830 tokens, checkpoints: 1, 90.847 MiB
|
|
5.23.563.477 I srv update: - prompt 0x5a870cfb0770: 2064 tokens, checkpoints: 1, 208.973 MiB
|
|
5.23.563.477 I srv update: - prompt 0x5a871427acf0: 569 tokens, checkpoints: 1, 65.863 MiB
|
|
5.23.563.478 I srv update: - prompt 0x5a87143d9ae0: 360 tokens, checkpoints: 1, 45.856 MiB
|
|
5.23.563.479 I srv get_availabl: prompt cache update took 48.62 ms
|
|
5.23.563.532 I slot launch_slot_: id 3 | task 17909 | processing task, is_child = 0
|
|
5.23.563.533 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.23.563.534 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.23.563.566 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.23.563.566 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.23.563.587 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.23.563.587 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.23.563.616 I slot update_slots: id 3 | task 17909 | Checking checkpoint with [0, 161] against 0...
|
|
5.23.568.935 W slot update_slots: id 3 | task 17909 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
|
|
5.26.740.551 I slot print_timing: id 3 | task 17909 | n_decoded = 185, tg = 61.64 t/s, tg_3s = 61.64 t/s
|
|
5.29.754.579 I slot print_timing: id 3 | task 17909 | n_decoded = 369, tg = 61.35 t/s, tg_3s = 61.05 t/s
|
|
5.32.350.894 I slot print_timing: id 3 | task 17909 | prompt eval time = 175.82 ms / 44 tokens ( 4.00 ms per token, 250.25 tokens per second)
|
|
5.32.350.898 I slot print_timing: id 3 | task 17909 | eval time = 8611.44 ms / 527 tokens ( 16.34 ms per token, 61.20 tokens per second)
|
|
5.32.350.899 I slot print_timing: id 3 | task 17909 | total time = 8787.26 ms / 571 tokens
|
|
5.32.350.900 I slot print_timing: id 3 | task 17909 | graphs reused = 18256
|
|
5.32.350.930 I slot release: id 3 | task 17909 | stop processing: n_tokens = 731, truncated = 0
|
|
5.32.350.939 I srv update_slots: all slots are idle
|
|
5.32.417.735 I srv operator(): Chat format: peg-native
|
|
5.32.420.179 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.829 (> 0.100 thold), f_keep = 0.233
|
|
5.32.420.182 I srv get_availabl: updating prompt cache
|
|
5.32.420.274 W srv prompt_save: - saving prompt with length 731, total state size = 69.977 MiB (draft: 0.000 MiB)
|
|
5.32.474.716 I srv load: - looking for better prompt, base f_keep = 0.233, sim = 0.829
|
|
5.32.474.734 I srv update: - cache state: 17 prompts, 2087.484 MiB (limits: 8192.000 MiB, 24576 tokens, 84169 est)
|
|
5.32.474.736 I srv update: - prompt 0x5a8714240e90: 760 tokens, checkpoints: 1, 83.091 MiB
|
|
5.32.474.737 I srv update: - prompt 0x5a87077a2ec0: 541 tokens, checkpoints: 1, 62.127 MiB
|
|
5.32.474.738 I srv update: - prompt 0x5a870d0831c0: 337 tokens, checkpoints: 1, 42.599 MiB
|
|
5.32.474.738 I srv update: - prompt 0x5a870ce53e00: 783 tokens, checkpoints: 1, 85.293 MiB
|
|
5.32.474.739 I srv update: - prompt 0x5a870ce94710: 6317 tokens, checkpoints: 1, 458.851 MiB
|
|
5.32.474.739 I srv update: - prompt 0x5a8704790e40: 1267 tokens, checkpoints: 1, 133.031 MiB
|
|
5.32.474.740 I srv update: - prompt 0x5a870d1284d0: 538 tokens, checkpoints: 1, 63.247 MiB
|
|
5.32.474.740 I srv update: - prompt 0x5a8714167200: 1332 tokens, checkpoints: 1, 139.253 MiB
|
|
5.32.474.741 I srv update: - prompt 0x5a870ce69860: 2721 tokens, checkpoints: 1, 272.217 MiB
|
|
5.32.474.741 I srv update: - prompt 0x5a87142dda40: 987 tokens, checkpoints: 1, 106.228 MiB
|
|
5.32.474.742 I srv update: - prompt 0x5a870cf0ddb0: 811 tokens, checkpoints: 1, 89.380 MiB
|
|
5.32.474.742 I srv update: - prompt 0x5a8717ab4e00: 500 tokens, checkpoints: 1, 59.257 MiB
|
|
5.32.474.743 I srv update: - prompt 0x5a8710079620: 830 tokens, checkpoints: 1, 90.847 MiB
|
|
5.32.474.744 I srv update: - prompt 0x5a870cfb0770: 2064 tokens, checkpoints: 1, 208.973 MiB
|
|
5.32.474.744 I srv update: - prompt 0x5a871427acf0: 569 tokens, checkpoints: 1, 65.863 MiB
|
|
5.32.474.744 I srv update: - prompt 0x5a87143d9ae0: 360 tokens, checkpoints: 1, 45.856 MiB
|
|
5.32.474.745 I srv update: - prompt 0x5a8713dd4a40: 731 tokens, checkpoints: 1, 81.370 MiB
|
|
5.32.474.746 I srv get_availabl: prompt cache update took 54.56 ms
|
|
5.32.474.798 I slot launch_slot_: id 3 | task 18439 | processing task, is_child = 0
|
|
5.32.474.799 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
5.32.474.800 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
5.32.474.828 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
5.32.474.828 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
5.32.474.849 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
5.32.474.850 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
5.32.474.880 I slot update_slots: id 3 | task 18439 | Checking checkpoint with [0, 161] against 0...
|
|
5.32.480.696 W slot update_slots: id 3 | task 18439 | restored context checkpoint (pos_min = 0, pos_max = 161, n_tokens = 162, n_past = 161, size = 11.393 MiB)
|
|
5.35.657.263 I slot print_timing: id 3 | task 18439 | n_decoded = 185, tg = 61.44 t/s, tg_3s = 61.44 t/s
|
|
5.38.667.693 I slot print_timing: id 3 | task 18439 | n_decoded = 368, tg = 61.12 t/s, tg_3s = 60.79 t/s
|
|
5.41.680.265 I slot print_timing: id 3 | task 18439 | n_decoded = 550, tg = 60.88 t/s, tg_3s = 60.41 t/s
|
|
5.44.693.942 I slot print_timing: id 3 | task 18439 | n_decoded = 731, tg = 60.68 t/s, tg_3s = 60.06 t/s
|
|
5.47.700.502 I slot print_timing: id 3 | task 18439 | n_decoded = 910, tg = 60.45 t/s, tg_3s = 59.54 t/s
|
|
5.48.608.528 I slot print_timing: id 3 | task 18439 | prompt eval time = 171.54 ms / 44 tokens ( 3.90 ms per token, 256.50 tokens per second)
|
|
5.48.608.532 I slot print_timing: id 3 | task 18439 | eval time = 15962.10 ms / 964 tokens ( 16.56 ms per token, 60.39 tokens per second)
|
|
5.48.608.533 I slot print_timing: id 3 | task 18439 | total time = 16133.64 ms / 1008 tokens
|
|
5.48.608.533 I slot print_timing: id 3 | task 18439 | graphs reused = 19214
|
|
5.48.608.576 I slot release: id 3 | task 18439 | stop processing: n_tokens = 1168, truncated = 0
|
|
5.48.608.588 I srv update_slots: all slots are idle
|
|
5.48.668.468 I srv operator(): operator(): cleaning up before exit...
|