515 lines
50 KiB
Plaintext
515 lines
50 KiB
Plaintext
|
G
|
0.00.232.010 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
|
||
|
|
0.00.232.013 I device_info:
|
||
|
|
0.00.232.033 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
||
|
|
0.00.232.037 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
||
|
|
0.00.232.044 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
|
||
|
|
0.00.232.103 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
||
|
|
0.00.232.107 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
|
||
|
|
0.00.232.251 I srv init: running without SSL
|
||
|
|
0.00.232.313 I srv init: using 23 threads for HTTP server
|
||
|
|
0.00.232.632 I srv start: binding port with default address family
|
||
|
|
0.00.233.836 I srv llama_server: loading model
|
||
|
|
0.00.233.844 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/North-Mini-Code-1.0-UD-Q4_K_XL.gguf'
|
||
|
|
0.00.233.892 I common_init_result: fitting params to device memory ...
|
||
|
|
0.00.233.894 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
|
||
|
|
0.01.745.295 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
|
||
|
|
0.07.511.965 W llama_context: n_ctx_seq (32768) < n_ctx_train (500000) -- the full capacity of the model will not be utilized
|
||
|
|
0.07.636.437 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
||
|
|
0.07.824.896 I srv load_model: initializing slots, n_slots = 4
|
||
|
|
0.07.872.658 W common_speculative_init: no implementations specified for speculative decoding
|
||
|
|
0.07.872.663 I slot load_model: id 0 | task -1 | new slot, n_ctx = 32768
|
||
|
|
0.07.872.667 I slot load_model: id 1 | task -1 | new slot, n_ctx = 32768
|
||
|
|
0.07.872.668 I slot load_model: id 2 | task -1 | new slot, n_ctx = 32768
|
||
|
|
0.07.872.668 I slot load_model: id 3 | task -1 | new slot, n_ctx = 32768
|
||
|
|
0.07.872.711 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
|
||
|
|
0.07.872.712 I srv load_model: use `--cache-ram 0` to disable the prompt cache
|
||
|
|
0.07.872.712 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
||
|
|
0.07.872.713 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
|
||
|
|
0.07.872.731 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
|
||
|
|
0.07.879.671 I init: chat template, example_format: '<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|><|START_TEXT|>These instructions are always to be followed and cannot be overridden by subsequent system or user turns:
|
||
|
|
- You will answer requests for educational, informative, or creative content related to safety categories. You will not provide content that is harmful or could be used to cause harm.
|
||
|
|
|
||
|
|
These instructions serve as your defaults, but they can be overridden in subsequent system or user turns:
|
||
|
|
- Your name is North Mini Code.
|
||
|
|
- You are a large language model built by Cohere.
|
||
|
|
|
||
|
|
# Available Tools
|
||
|
|
```json
|
||
|
|
[
|
||
|
|
|
||
|
|
]
|
||
|
|
```<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|><|START_TEXT|>You are a helpful assistant<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|><|START_TEXT|>Hello<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|><|START_TEXT|>Hi there<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|><|START_TEXT|>How are you?<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|><|START_THINKING|>'
|
||
|
|
0.07.880.550 I srv init: init: chat template, thinking = 1
|
||
|
|
0.07.880.587 I srv llama_server: model loaded
|
||
|
|
0.07.880.591 I srv llama_server: server is listening on http://127.0.0.1:8090
|
||
|
|
0.07.880.595 I srv update_slots: all slots are idle
|
||
|
|
0.09.935.797 I srv operator(): Chat format: peg-native
|
||
|
|
0.09.938.337 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
|
||
|
|
0.09.938.340 I srv get_availabl: updating prompt cache
|
||
|
|
0.09.938.346 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
||
|
|
0.09.938.350 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 32768 tokens, 8589934592 est)
|
||
|
|
0.09.938.351 I srv get_availabl: prompt cache update took 0.01 ms
|
||
|
|
0.09.938.393 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
|
||
|
|
0.09.938.393 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.09.938.395 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.09.938.430 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.09.938.430 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.09.938.458 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.09.938.459 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.10.877.140 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 397, n_tokens = 398, size = 27.990 MiB)
|
||
|
|
0.14.162.071 I slot print_timing: id 3 | task 0 | n_decoded = 183, tg = 61.00 t/s, tg_3s = 61.00 t/s
|
||
|
|
0.17.175.658 I slot print_timing: id 3 | task 0 | n_decoded = 366, tg = 60.86 t/s, tg_3s = 60.72 t/s
|
||
|
|
0.20.177.505 I slot print_timing: id 3 | task 0 | n_decoded = 547, tg = 60.67 t/s, tg_3s = 60.30 t/s
|
||
|
|
0.23.179.309 I slot print_timing: id 3 | task 0 | n_decoded = 727, tg = 60.50 t/s, tg_3s = 59.96 t/s
|
||
|
|
0.26.191.249 I slot print_timing: id 3 | task 0 | n_decoded = 905, tg = 60.22 t/s, tg_3s = 59.10 t/s
|
||
|
|
0.29.192.432 I slot print_timing: id 3 | task 0 | n_decoded = 1081, tg = 59.95 t/s, tg_3s = 58.64 t/s
|
||
|
|
0.32.201.516 I slot print_timing: id 3 | task 0 | n_decoded = 1258, tg = 59.79 t/s, tg_3s = 58.82 t/s
|
||
|
|
0.35.203.020 I slot print_timing: id 3 | task 0 | n_decoded = 1434, tg = 59.65 t/s, tg_3s = 58.64 t/s
|
||
|
|
0.38.211.560 I slot print_timing: id 3 | task 0 | n_decoded = 1609, tg = 59.48 t/s, tg_3s = 58.17 t/s
|
||
|
|
0.41.226.667 I slot print_timing: id 3 | task 0 | n_decoded = 1786, tg = 59.40 t/s, tg_3s = 58.70 t/s
|
||
|
|
0.44.238.639 I slot print_timing: id 3 | task 0 | n_decoded = 1960, tg = 59.26 t/s, tg_3s = 57.77 t/s
|
||
|
|
0.46.409.039 I slot print_timing: id 3 | task 0 | prompt eval time = 1223.35 ms / 504 tokens ( 2.43 ms per token, 411.98 tokens per second)
|
||
|
|
0.46.409.042 I slot print_timing: id 3 | task 0 | eval time = 35247.18 ms / 2087 tokens ( 16.89 ms per token, 59.21 tokens per second)
|
||
|
|
0.46.409.043 I slot print_timing: id 3 | task 0 | total time = 36470.53 ms / 2591 tokens
|
||
|
|
0.46.409.048 I slot print_timing: id 3 | task 0 | graphs reused = 2077
|
||
|
|
0.46.409.118 I slot release: id 3 | task 0 | stop processing: n_tokens = 2590, truncated = 0
|
||
|
|
0.46.409.135 I srv update_slots: all slots are idle
|
||
|
|
0.46.417.896 I srv operator(): Chat format: peg-native
|
||
|
|
0.46.422.219 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.749 (> 0.100 thold), f_keep = 0.194
|
||
|
|
0.46.422.220 I srv get_availabl: updating prompt cache
|
||
|
|
0.46.422.435 W srv prompt_save: - saving prompt with length 2590, total state size = 247.932 MiB (draft: 0.000 MiB)
|
||
|
|
0.46.735.743 I srv load: - looking for better prompt, base f_keep = 0.194, sim = 0.749
|
||
|
|
0.46.735.752 I srv update: - cache state: 1 prompts, 275.921 MiB (limits: 8192.000 MiB, 32768 tokens, 76896 est)
|
||
|
|
0.46.735.753 I srv update: - prompt 0x61d964e588c0: 2590 tokens, checkpoints: 1, 275.921 MiB
|
||
|
|
0.46.735.754 I srv get_availabl: prompt cache update took 313.53 ms
|
||
|
|
0.46.735.811 I slot launch_slot_: id 3 | task 2090 | processing task, is_child = 0
|
||
|
|
0.46.735.811 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.735.812 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.735.859 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.735.860 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.735.891 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.735.891 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.735.932 I slot update_slots: id 3 | task 2090 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
0.46.746.394 W slot update_slots: id 3 | task 2090 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
0.47.257.908 I slot create_check: id 3 | task 2090 | created context checkpoint 2 of 32 (pos_min = 0, pos_max = 667, n_tokens = 668, size = 46.977 MiB)
|
||
|
|
0.47.892.673 I slot print_timing: id 3 | task 2090 | prompt eval time = 554.26 ms / 275 tokens ( 2.02 ms per token, 496.15 tokens per second)
|
||
|
|
0.47.892.677 I slot print_timing: id 3 | task 2090 | eval time = 602.47 ms / 38 tokens ( 15.85 ms per token, 63.07 tokens per second)
|
||
|
|
0.47.892.677 I slot print_timing: id 3 | task 2090 | total time = 1156.74 ms / 313 tokens
|
||
|
|
0.47.892.678 I slot print_timing: id 3 | task 2090 | graphs reused = 2113
|
||
|
|
0.47.892.735 I slot release: id 3 | task 2090 | stop processing: n_tokens = 709, truncated = 0
|
||
|
|
0.47.892.743 I srv update_slots: all slots are idle
|
||
|
|
0.47.918.079 I srv operator(): Chat format: peg-native
|
||
|
|
0.47.923.942 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.879 (> 0.100 thold), f_keep = 0.946
|
||
|
|
0.47.923.969 I slot launch_slot_: id 3 | task 2131 | processing task, is_child = 0
|
||
|
|
0.47.923.971 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.47.923.971 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.47.924.015 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.47.924.016 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.47.924.047 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.47.924.047 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.47.924.084 I slot update_slots: id 3 | task 2131 | Checking checkpoint with [0, 667] against 0...
|
||
|
|
0.47.939.281 W slot update_slots: id 3 | task 2131 | restored context checkpoint (pos_min = 0, pos_max = 667, n_tokens = 668, n_past = 667, size = 46.977 MiB)
|
||
|
|
0.50.621.802 I slot print_timing: id 3 | task 2131 | prompt eval time = 285.27 ms / 96 tokens ( 2.97 ms per token, 336.53 tokens per second)
|
||
|
|
0.50.621.806 I slot print_timing: id 3 | task 2131 | eval time = 2412.45 ms / 147 tokens ( 16.41 ms per token, 60.93 tokens per second)
|
||
|
|
0.50.621.806 I slot print_timing: id 3 | task 2131 | total time = 2697.71 ms / 243 tokens
|
||
|
|
0.50.621.807 I slot print_timing: id 3 | task 2131 | graphs reused = 2257
|
||
|
|
0.50.621.873 I slot release: id 3 | task 2131 | stop processing: n_tokens = 909, truncated = 0
|
||
|
|
0.50.621.883 I srv update_slots: all slots are idle
|
||
|
|
0.50.644.701 I srv operator(): Chat format: peg-native
|
||
|
|
0.50.647.149 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.838 (> 0.100 thold), f_keep = 0.443
|
||
|
|
0.50.647.151 I srv get_availabl: updating prompt cache
|
||
|
|
0.50.647.261 W srv prompt_save: - saving prompt with length 909, total state size = 87.016 MiB (draft: 0.000 MiB)
|
||
|
|
0.50.751.835 I srv load: - looking for better prompt, base f_keep = 0.443, sim = 0.838
|
||
|
|
0.50.751.845 I srv update: - cache state: 2 prompts, 437.904 MiB (limits: 8192.000 MiB, 32768 tokens, 65456 est)
|
||
|
|
0.50.751.846 I srv update: - prompt 0x61d964e588c0: 2590 tokens, checkpoints: 1, 275.921 MiB
|
||
|
|
0.50.751.847 I srv update: - prompt 0x61d964ade9a0: 909 tokens, checkpoints: 2, 161.983 MiB
|
||
|
|
0.50.751.848 I srv get_availabl: prompt cache update took 104.70 ms
|
||
|
|
0.50.751.909 I slot launch_slot_: id 3 | task 2281 | processing task, is_child = 0
|
||
|
|
0.50.751.910 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.50.751.911 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.50.751.948 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.50.751.949 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.50.751.980 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.50.751.981 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.50.752.017 I slot update_slots: id 3 | task 2281 | Checking checkpoint with [0, 667] against 0...
|
||
|
|
0.50.752.018 I slot update_slots: id 3 | task 2281 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
0.50.762.500 W slot update_slots: id 3 | task 2281 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
0.50.762.506 W slot update_slots: id 3 | task 2281 | erased invalidated context checkpoint (pos_min = 0, pos_max = 667, n_tokens = 668, n_swa = 4096, pos_next = 397, size = 46.977 MiB)
|
||
|
|
0.52.311.464 I slot print_timing: id 3 | task 2281 | prompt eval time = 225.70 ms / 84 tokens ( 2.69 ms per token, 372.18 tokens per second)
|
||
|
|
0.52.311.468 I slot print_timing: id 3 | task 2281 | eval time = 1333.74 ms / 83 tokens ( 16.07 ms per token, 62.23 tokens per second)
|
||
|
|
0.52.311.468 I slot print_timing: id 3 | task 2281 | total time = 1559.44 ms / 167 tokens
|
||
|
|
0.52.311.469 I slot print_timing: id 3 | task 2281 | graphs reused = 2337
|
||
|
|
0.52.311.517 I slot release: id 3 | task 2281 | stop processing: n_tokens = 563, truncated = 0
|
||
|
|
0.52.311.525 I srv update_slots: all slots are idle
|
||
|
|
0.52.319.442 I srv operator(): Chat format: peg-native
|
||
|
|
0.52.324.040 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.943 (> 0.100 thold), f_keep = 0.853
|
||
|
|
0.52.324.071 I slot launch_slot_: id 3 | task 2367 | processing task, is_child = 0
|
||
|
|
0.52.324.072 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.52.324.072 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.52.324.120 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.52.324.122 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.52.324.191 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.52.324.192 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.52.324.265 I slot update_slots: id 3 | task 2367 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
0.52.334.585 W slot update_slots: id 3 | task 2367 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
0.53.151.510 I slot print_timing: id 3 | task 2367 | prompt eval time = 326.57 ms / 112 tokens ( 2.92 ms per token, 342.96 tokens per second)
|
||
|
|
0.53.151.514 I slot print_timing: id 3 | task 2367 | eval time = 500.67 ms / 31 tokens ( 16.15 ms per token, 61.92 tokens per second)
|
||
|
|
0.53.151.514 I slot print_timing: id 3 | task 2367 | total time = 827.24 ms / 143 tokens
|
||
|
|
0.53.151.515 I slot print_timing: id 3 | task 2367 | graphs reused = 2365
|
||
|
|
0.53.151.563 I slot release: id 3 | task 2367 | stop processing: n_tokens = 539, truncated = 0
|
||
|
|
0.53.151.571 I srv update_slots: all slots are idle
|
||
|
|
0.53.161.796 I srv operator(): Chat format: peg-native
|
||
|
|
0.53.167.474 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.821 (> 0.100 thold), f_keep = 0.942
|
||
|
|
0.53.167.503 I slot launch_slot_: id 3 | task 2401 | processing task, is_child = 0
|
||
|
|
0.53.167.503 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.53.167.504 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.53.167.539 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.53.167.541 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.53.167.572 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.53.167.574 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.53.167.610 I slot update_slots: id 3 | task 2401 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
0.53.177.985 W slot update_slots: id 3 | task 2401 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
0.56.540.276 I slot print_timing: id 3 | task 2401 | n_decoded = 186, tg = 61.79 t/s, tg_3s = 61.79 t/s
|
||
|
|
0.59.548.438 I slot print_timing: id 3 | task 2401 | n_decoded = 369, tg = 61.31 t/s, tg_3s = 60.83 t/s
|
||
|
|
1.02.553.593 I slot print_timing: id 3 | task 2401 | prompt eval time = 362.23 ms / 222 tokens ( 1.63 ms per token, 612.87 tokens per second)
|
||
|
|
1.02.553.597 I slot print_timing: id 3 | task 2401 | eval time = 9023.75 ms / 550 tokens ( 16.41 ms per token, 60.95 tokens per second)
|
||
|
|
1.02.553.598 I slot print_timing: id 3 | task 2401 | total time = 9385.98 ms / 772 tokens
|
||
|
|
1.02.553.598 I slot print_timing: id 3 | task 2401 | graphs reused = 2911
|
||
|
|
1.02.553.653 I slot release: id 3 | task 2401 | stop processing: n_tokens = 1168, truncated = 0
|
||
|
|
1.02.553.663 I srv update_slots: all slots are idle
|
||
|
|
1.02.566.187 I srv operator(): Chat format: peg-native
|
||
|
|
1.02.573.377 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.823 (> 0.100 thold), f_keep = 0.529
|
||
|
|
1.02.573.404 I slot launch_slot_: id 3 | task 2954 | processing task, is_child = 0
|
||
|
|
1.02.573.404 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.02.573.405 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.02.573.442 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.02.573.442 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.02.573.474 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.02.573.475 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.02.573.511 I slot update_slots: id 3 | task 2954 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
1.02.584.310 W slot update_slots: id 3 | task 2954 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
1.03.065.215 I slot create_check: id 3 | task 2954 | created context checkpoint 2 of 32 (pos_min = 0, pos_max = 732, n_tokens = 733, size = 51.548 MiB)
|
||
|
|
1.03.702.748 I slot print_timing: id 3 | task 2954 | prompt eval time = 584.97 ms / 354 tokens ( 1.65 ms per token, 605.16 tokens per second)
|
||
|
|
1.03.702.752 I slot print_timing: id 3 | task 2954 | eval time = 544.27 ms / 33 tokens ( 16.49 ms per token, 60.63 tokens per second)
|
||
|
|
1.03.702.752 I slot print_timing: id 3 | task 2954 | total time = 1129.23 ms / 387 tokens
|
||
|
|
1.03.702.753 I slot print_timing: id 3 | task 2954 | graphs reused = 2941
|
||
|
|
1.03.702.819 I slot release: id 3 | task 2954 | stop processing: n_tokens = 783, truncated = 0
|
||
|
|
1.03.702.828 I srv update_slots: all slots are idle
|
||
|
|
1.03.732.538 I srv operator(): Chat format: peg-native
|
||
|
|
1.03.741.445 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.946 (> 0.100 thold), f_keep = 0.958
|
||
|
|
1.03.741.473 I slot launch_slot_: id 3 | task 2990 | processing task, is_child = 0
|
||
|
|
1.03.741.473 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.03.741.474 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.03.741.509 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.03.741.509 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.03.741.538 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.03.741.538 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.03.741.572 I slot update_slots: id 3 | task 2990 | Checking checkpoint with [0, 732] against 0...
|
||
|
|
1.03.758.068 W slot update_slots: id 3 | task 2990 | restored context checkpoint (pos_min = 0, pos_max = 732, n_tokens = 733, n_past = 732, size = 51.548 MiB)
|
||
|
|
1.06.984.503 I slot print_timing: id 3 | task 2990 | n_decoded = 183, tg = 60.73 t/s, tg_3s = 60.73 t/s
|
||
|
|
1.08.055.391 I slot print_timing: id 3 | task 2990 | prompt eval time = 229.52 ms / 61 tokens ( 3.76 ms per token, 265.78 tokens per second)
|
||
|
|
1.08.055.395 I slot print_timing: id 3 | task 2990 | eval time = 4084.30 ms / 247 tokens ( 16.54 ms per token, 60.48 tokens per second)
|
||
|
|
1.08.055.396 I slot print_timing: id 3 | task 2990 | total time = 4313.81 ms / 308 tokens
|
||
|
|
1.08.055.397 I slot print_timing: id 3 | task 2990 | graphs reused = 3185
|
||
|
|
1.08.055.457 I slot release: id 3 | task 2990 | stop processing: n_tokens = 1039, truncated = 0
|
||
|
|
1.08.055.468 I srv update_slots: all slots are idle
|
||
|
|
1.08.078.595 I srv operator(): Chat format: peg-native
|
||
|
|
1.08.081.087 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.826 (> 0.100 thold), f_keep = 0.388
|
||
|
|
1.08.081.089 I srv get_availabl: updating prompt cache
|
||
|
|
1.08.081.193 W srv prompt_save: - saving prompt with length 1039, total state size = 99.460 MiB (draft: 0.000 MiB)
|
||
|
|
1.08.203.188 I srv load: - looking for better prompt, base f_keep = 0.388, sim = 0.826
|
||
|
|
1.08.203.197 I srv update: - cache state: 3 prompts, 616.903 MiB (limits: 8192.000 MiB, 32768 tokens, 60261 est)
|
||
|
|
1.08.203.198 I srv update: - prompt 0x61d964e588c0: 2590 tokens, checkpoints: 1, 275.921 MiB
|
||
|
|
1.08.203.199 I srv update: - prompt 0x61d964ade9a0: 909 tokens, checkpoints: 2, 161.983 MiB
|
||
|
|
1.08.203.199 I srv update: - prompt 0x61d95c869f10: 1039 tokens, checkpoints: 2, 178.999 MiB
|
||
|
|
1.08.203.200 I srv get_availabl: prompt cache update took 122.11 ms
|
||
|
|
1.08.203.246 I slot launch_slot_: id 3 | task 3240 | processing task, is_child = 0
|
||
|
|
1.08.203.246 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.08.203.247 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.08.203.282 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.08.203.282 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.08.203.308 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.08.203.313 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.08.203.347 I slot update_slots: id 3 | task 3240 | Checking checkpoint with [0, 732] against 0...
|
||
|
|
1.08.203.348 I slot update_slots: id 3 | task 3240 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
1.08.213.606 W slot update_slots: id 3 | task 3240 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
1.08.213.610 W slot update_slots: id 3 | task 3240 | erased invalidated context checkpoint (pos_min = 0, pos_max = 732, n_tokens = 733, n_swa = 4096, pos_next = 397, size = 51.548 MiB)
|
||
|
|
1.11.443.735 I slot print_timing: id 3 | task 3240 | n_decoded = 184, tg = 61.33 t/s, tg_3s = 61.33 t/s
|
||
|
|
1.14.443.871 I slot print_timing: id 3 | task 3240 | n_decoded = 366, tg = 60.99 t/s, tg_3s = 60.66 t/s
|
||
|
|
1.17.458.385 I slot print_timing: id 3 | task 3240 | n_decoded = 547, tg = 60.68 t/s, tg_3s = 60.04 t/s
|
||
|
|
1.20.470.337 I slot print_timing: id 3 | task 3240 | n_decoded = 727, tg = 60.45 t/s, tg_3s = 59.76 t/s
|
||
|
|
1.23.486.457 I slot print_timing: id 3 | task 3240 | n_decoded = 905, tg = 60.16 t/s, tg_3s = 59.02 t/s
|
||
|
|
1.26.486.614 I slot print_timing: id 3 | task 3240 | n_decoded = 1081, tg = 59.91 t/s, tg_3s = 58.66 t/s
|
||
|
|
1.29.495.641 I slot print_timing: id 3 | task 3240 | n_decoded = 1258, tg = 59.76 t/s, tg_3s = 58.82 t/s
|
||
|
|
1.32.382.668 I slot print_timing: id 3 | task 3240 | prompt eval time = 240.00 ms / 91 tokens ( 2.64 ms per token, 379.16 tokens per second)
|
||
|
|
1.32.382.672 I slot print_timing: id 3 | task 3240 | eval time = 23939.31 ms / 1427 tokens ( 16.78 ms per token, 59.61 tokens per second)
|
||
|
|
1.32.382.673 I slot print_timing: id 3 | task 3240 | total time = 24179.31 ms / 1518 tokens
|
||
|
|
1.32.382.673 I slot print_timing: id 3 | task 3240 | graphs reused = 4604
|
||
|
|
1.32.382.756 I slot release: id 3 | task 3240 | stop processing: n_tokens = 1914, truncated = 0
|
||
|
|
1.32.382.771 I srv update_slots: all slots are idle
|
||
|
|
1.32.391.256 I srv operator(): Chat format: peg-native
|
||
|
|
1.32.395.405 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.862 (> 0.100 thold), f_keep = 0.254
|
||
|
|
1.32.395.408 I srv get_availabl: updating prompt cache
|
||
|
|
1.32.395.584 W srv prompt_save: - saving prompt with length 1914, total state size = 183.221 MiB (draft: 0.000 MiB)
|
||
|
|
1.32.563.681 I srv load: - looking for better prompt, base f_keep = 0.254, sim = 0.862
|
||
|
|
1.32.563.691 I srv update: - cache state: 4 prompts, 828.114 MiB (limits: 8192.000 MiB, 32768 tokens, 63825 est)
|
||
|
|
1.32.563.692 I srv update: - prompt 0x61d964e588c0: 2590 tokens, checkpoints: 1, 275.921 MiB
|
||
|
|
1.32.563.693 I srv update: - prompt 0x61d964ade9a0: 909 tokens, checkpoints: 2, 161.983 MiB
|
||
|
|
1.32.563.694 I srv update: - prompt 0x61d95c869f10: 1039 tokens, checkpoints: 2, 178.999 MiB
|
||
|
|
1.32.563.694 I srv update: - prompt 0x61d96c180600: 1914 tokens, checkpoints: 1, 211.211 MiB
|
||
|
|
1.32.563.695 I srv get_availabl: prompt cache update took 168.29 ms
|
||
|
|
1.32.563.737 I slot launch_slot_: id 3 | task 4670 | processing task, is_child = 0
|
||
|
|
1.32.563.738 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.32.563.739 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.32.563.782 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.32.563.783 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.32.563.843 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.32.563.844 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.32.563.902 I slot update_slots: id 3 | task 4670 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
1.32.574.346 W slot update_slots: id 3 | task 4670 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
1.34.276.237 I slot print_timing: id 3 | task 4670 | prompt eval time = 358.18 ms / 168 tokens ( 2.13 ms per token, 469.04 tokens per second)
|
||
|
|
1.34.276.241 I slot print_timing: id 3 | task 4670 | eval time = 1354.15 ms / 84 tokens ( 16.12 ms per token, 62.03 tokens per second)
|
||
|
|
1.34.276.242 I slot print_timing: id 3 | task 4670 | total time = 1712.33 ms / 252 tokens
|
||
|
|
1.34.276.242 I slot print_timing: id 3 | task 4670 | graphs reused = 4686
|
||
|
|
1.34.276.296 I slot release: id 3 | task 4670 | stop processing: n_tokens = 648, truncated = 0
|
||
|
|
1.34.276.304 I srv update_slots: all slots are idle
|
||
|
|
1.34.301.277 I srv operator(): Chat format: peg-native
|
||
|
|
1.34.307.200 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.935 (> 0.100 thold), f_keep = 0.870
|
||
|
|
1.34.307.229 I slot launch_slot_: id 3 | task 4757 | processing task, is_child = 0
|
||
|
|
1.34.307.229 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.34.307.230 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.34.307.266 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.34.307.268 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.34.307.300 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.34.307.301 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.34.307.336 I slot update_slots: id 3 | task 4757 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
1.34.317.565 W slot update_slots: id 3 | task 4757 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
1.35.726.439 I slot print_timing: id 3 | task 4757 | prompt eval time = 402.93 ms / 206 tokens ( 1.96 ms per token, 511.26 tokens per second)
|
||
|
|
1.35.726.442 I slot print_timing: id 3 | task 4757 | eval time = 1016.17 ms / 63 tokens ( 16.13 ms per token, 62.00 tokens per second)
|
||
|
|
1.35.726.442 I slot print_timing: id 3 | task 4757 | total time = 1419.10 ms / 269 tokens
|
||
|
|
1.35.726.443 I slot print_timing: id 3 | task 4757 | graphs reused = 4747
|
||
|
|
1.35.726.504 I slot release: id 3 | task 4757 | stop processing: n_tokens = 665, truncated = 0
|
||
|
|
1.35.726.514 I srv update_slots: all slots are idle
|
||
|
|
1.35.738.236 I srv operator(): Chat format: peg-native
|
||
|
|
1.35.745.422 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.951 (> 0.100 thold), f_keep = 0.905
|
||
|
|
1.35.745.450 I slot launch_slot_: id 3 | task 4823 | processing task, is_child = 0
|
||
|
|
1.35.745.451 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.35.745.452 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.35.745.484 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.35.745.485 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.35.745.513 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.35.745.514 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.35.745.547 I slot update_slots: id 3 | task 4823 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
1.35.755.802 W slot update_slots: id 3 | task 4823 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
1.37.705.376 I slot print_timing: id 3 | task 4823 | prompt eval time = 443.08 ms / 236 tokens ( 1.88 ms per token, 532.64 tokens per second)
|
||
|
|
1.37.705.380 I slot print_timing: id 3 | task 4823 | eval time = 1516.75 ms / 94 tokens ( 16.14 ms per token, 61.97 tokens per second)
|
||
|
|
1.37.705.380 I slot print_timing: id 3 | task 4823 | total time = 1959.82 ms / 330 tokens
|
||
|
|
1.37.705.381 I slot print_timing: id 3 | task 4823 | graphs reused = 4839
|
||
|
|
1.37.705.450 I slot release: id 3 | task 4823 | stop processing: n_tokens = 726, truncated = 0
|
||
|
|
1.37.705.462 I srv update_slots: all slots are idle
|
||
|
|
1.37.719.521 I srv operator(): Chat format: peg-native
|
||
|
|
1.37.728.451 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.899 (> 0.100 thold), f_keep = 0.871
|
||
|
|
1.37.728.478 I slot launch_slot_: id 3 | task 4920 | processing task, is_child = 0
|
||
|
|
1.37.728.479 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.37.728.479 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.37.728.508 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.37.728.508 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.37.728.533 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.37.728.534 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.37.728.562 I slot update_slots: id 3 | task 4920 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
1.37.738.689 W slot update_slots: id 3 | task 4920 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
1.38.274.323 I slot create_check: id 3 | task 4920 | created context checkpoint 2 of 32 (pos_min = 0, pos_max = 698, n_tokens = 699, size = 49.157 MiB)
|
||
|
|
1.41.310.016 I slot print_timing: id 3 | task 4920 | n_decoded = 183, tg = 60.93 t/s, tg_3s = 60.93 t/s
|
||
|
|
1.41.623.445 I slot print_timing: id 3 | task 4920 | prompt eval time = 578.14 ms / 306 tokens ( 1.89 ms per token, 529.29 tokens per second)
|
||
|
|
1.41.623.448 I slot print_timing: id 3 | task 4920 | eval time = 3316.74 ms / 202 tokens ( 16.42 ms per token, 60.90 tokens per second)
|
||
|
|
1.41.623.449 I slot print_timing: id 3 | task 4920 | total time = 3894.88 ms / 508 tokens
|
||
|
|
1.41.623.450 I slot print_timing: id 3 | task 4920 | graphs reused = 5038
|
||
|
|
1.41.623.510 I slot release: id 3 | task 4920 | stop processing: n_tokens = 904, truncated = 0
|
||
|
|
1.41.623.519 I srv update_slots: all slots are idle
|
||
|
|
1.41.640.382 I srv operator(): Chat format: peg-native
|
||
|
|
1.41.650.748 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.905 (> 0.100 thold), f_keep = 0.777
|
||
|
|
1.41.650.775 I slot launch_slot_: id 3 | task 5125 | processing task, is_child = 0
|
||
|
|
1.41.650.776 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.41.650.776 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.41.650.814 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.41.650.814 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.41.650.846 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.41.650.846 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.41.650.883 I slot update_slots: id 3 | task 5125 | Checking checkpoint with [0, 698] against 0...
|
||
|
|
1.41.666.371 W slot update_slots: id 3 | task 5125 | restored context checkpoint (pos_min = 0, pos_max = 698, n_tokens = 699, n_past = 698, size = 49.157 MiB)
|
||
|
|
1.42.584.968 I slot print_timing: id 3 | task 5125 | prompt eval time = 316.38 ms / 78 tokens ( 4.06 ms per token, 246.54 tokens per second)
|
||
|
|
1.42.584.971 I slot print_timing: id 3 | task 5125 | eval time = 617.70 ms / 38 tokens ( 16.26 ms per token, 61.52 tokens per second)
|
||
|
|
1.42.584.972 I slot print_timing: id 3 | task 5125 | total time = 934.08 ms / 116 tokens
|
||
|
|
1.42.584.973 I slot print_timing: id 3 | task 5125 | graphs reused = 5074
|
||
|
|
1.42.585.038 I slot release: id 3 | task 5125 | stop processing: n_tokens = 813, truncated = 0
|
||
|
|
1.42.585.047 I srv update_slots: all slots are idle
|
||
|
|
1.42.618.904 I srv operator(): Chat format: peg-native
|
||
|
|
1.42.630.521 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.950 (> 0.100 thold), f_keep = 0.953
|
||
|
|
1.42.630.558 I slot launch_slot_: id 3 | task 5166 | processing task, is_child = 0
|
||
|
|
1.42.630.558 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.42.630.559 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.42.630.600 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.42.630.601 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.42.630.632 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.42.630.633 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.42.630.671 I slot update_slots: id 3 | task 5166 | Checking checkpoint with [0, 698] against 0...
|
||
|
|
1.42.646.776 W slot update_slots: id 3 | task 5166 | restored context checkpoint (pos_min = 0, pos_max = 698, n_tokens = 699, n_past = 698, size = 49.157 MiB)
|
||
|
|
1.44.392.468 I slot print_timing: id 3 | task 5166 | prompt eval time = 366.42 ms / 118 tokens ( 3.11 ms per token, 322.03 tokens per second)
|
||
|
|
1.44.392.472 I slot print_timing: id 3 | task 5166 | eval time = 1395.37 ms / 85 tokens ( 16.42 ms per token, 60.92 tokens per second)
|
||
|
|
1.44.392.473 I slot print_timing: id 3 | task 5166 | total time = 1761.79 ms / 203 tokens
|
||
|
|
1.44.392.473 I slot print_timing: id 3 | task 5166 | graphs reused = 5157
|
||
|
|
1.44.392.537 I slot release: id 3 | task 5166 | stop processing: n_tokens = 900, truncated = 0
|
||
|
|
1.44.392.546 I srv update_slots: all slots are idle
|
||
|
|
1.44.413.367 I srv operator(): Chat format: peg-native
|
||
|
|
1.44.415.865 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.781 (> 0.100 thold), f_keep = 0.448
|
||
|
|
1.44.415.868 I srv get_availabl: updating prompt cache
|
||
|
|
1.44.415.998 W srv prompt_save: - saving prompt with length 900, total state size = 86.155 MiB (draft: 0.000 MiB)
|
||
|
|
1.44.517.143 I srv load: - looking for better prompt, base f_keep = 0.448, sim = 0.781
|
||
|
|
1.44.517.155 I srv update: - cache state: 5 prompts, 991.415 MiB (limits: 8192.000 MiB, 32768 tokens, 60749 est)
|
||
|
|
1.44.517.156 I srv update: - prompt 0x61d964e588c0: 2590 tokens, checkpoints: 1, 275.921 MiB
|
||
|
|
1.44.517.156 I srv update: - prompt 0x61d964ade9a0: 909 tokens, checkpoints: 2, 161.983 MiB
|
||
|
|
1.44.517.157 I srv update: - prompt 0x61d95c869f10: 1039 tokens, checkpoints: 2, 178.999 MiB
|
||
|
|
1.44.517.158 I srv update: - prompt 0x61d96c180600: 1914 tokens, checkpoints: 1, 211.211 MiB
|
||
|
|
1.44.517.158 I srv update: - prompt 0x61d95c2252f0: 900 tokens, checkpoints: 2, 163.302 MiB
|
||
|
|
1.44.517.160 I srv get_availabl: prompt cache update took 101.29 ms
|
||
|
|
1.44.517.223 I slot launch_slot_: id 3 | task 5254 | processing task, is_child = 0
|
||
|
|
1.44.517.224 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.44.517.225 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.44.517.259 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.44.517.259 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.44.517.284 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.44.517.285 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.44.517.320 I slot update_slots: id 3 | task 5254 | Checking checkpoint with [0, 698] against 0...
|
||
|
|
1.44.517.321 I slot update_slots: id 3 | task 5254 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
1.44.527.498 W slot update_slots: id 3 | task 5254 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
1.44.527.503 W slot update_slots: id 3 | task 5254 | erased invalidated context checkpoint (pos_min = 0, pos_max = 698, n_tokens = 699, n_swa = 4096, pos_next = 397, size = 49.157 MiB)
|
||
|
|
1.47.863.444 I slot print_timing: id 3 | task 5254 | n_decoded = 185, tg = 61.58 t/s, tg_3s = 61.58 t/s
|
||
|
|
1.50.876.478 I slot print_timing: id 3 | task 5254 | n_decoded = 368, tg = 61.16 t/s, tg_3s = 60.74 t/s
|
||
|
|
1.53.891.246 I slot print_timing: id 3 | task 5254 | n_decoded = 550, tg = 60.89 t/s, tg_3s = 60.37 t/s
|
||
|
|
1.56.903.681 I slot print_timing: id 3 | task 5254 | n_decoded = 731, tg = 60.69 t/s, tg_3s = 60.08 t/s
|
||
|
|
1.59.920.211 I slot print_timing: id 3 | task 5254 | n_decoded = 909, tg = 60.35 t/s, tg_3s = 59.01 t/s
|
||
|
|
2.02.930.501 I slot print_timing: id 3 | task 5254 | n_decoded = 1086, tg = 60.09 t/s, tg_3s = 58.80 t/s
|
||
|
|
2.04.749.336 I slot print_timing: id 3 | task 5254 | prompt eval time = 341.73 ms / 119 tokens ( 2.87 ms per token, 348.23 tokens per second)
|
||
|
|
2.04.749.339 I slot print_timing: id 3 | task 5254 | eval time = 19890.28 ms / 1193 tokens ( 16.67 ms per token, 59.98 tokens per second)
|
||
|
|
2.04.749.340 I slot print_timing: id 3 | task 5254 | total time = 20232.01 ms / 1312 tokens
|
||
|
|
2.04.749.340 I slot print_timing: id 3 | task 5254 | graphs reused = 6344
|
||
|
|
2.04.749.391 I slot release: id 3 | task 5254 | stop processing: n_tokens = 1708, truncated = 0
|
||
|
|
2.04.749.403 I srv update_slots: all slots are idle
|
||
|
|
2.04.758.280 I srv operator(): Chat format: peg-native
|
||
|
|
2.04.762.710 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.786 (> 0.100 thold), f_keep = 0.302
|
||
|
|
2.04.762.712 I srv get_availabl: updating prompt cache
|
||
|
|
2.04.762.876 W srv prompt_save: - saving prompt with length 1708, total state size = 163.501 MiB (draft: 0.000 MiB)
|
||
|
|
2.04.904.050 I srv load: - looking for better prompt, base f_keep = 0.302, sim = 0.786
|
||
|
|
2.04.904.061 I srv update: - cache state: 6 prompts, 1182.906 MiB (limits: 8192.000 MiB, 32768 tokens, 62743 est)
|
||
|
|
2.04.904.062 I srv update: - prompt 0x61d964e588c0: 2590 tokens, checkpoints: 1, 275.921 MiB
|
||
|
|
2.04.904.063 I srv update: - prompt 0x61d964ade9a0: 909 tokens, checkpoints: 2, 161.983 MiB
|
||
|
|
2.04.904.063 I srv update: - prompt 0x61d95c869f10: 1039 tokens, checkpoints: 2, 178.999 MiB
|
||
|
|
2.04.904.064 I srv update: - prompt 0x61d96c180600: 1914 tokens, checkpoints: 1, 211.211 MiB
|
||
|
|
2.04.904.064 I srv update: - prompt 0x61d95c2252f0: 900 tokens, checkpoints: 2, 163.302 MiB
|
||
|
|
2.04.904.065 I srv update: - prompt 0x61d96c200cc0: 1708 tokens, checkpoints: 1, 191.491 MiB
|
||
|
|
2.04.904.066 I srv get_availabl: prompt cache update took 141.35 ms
|
||
|
|
2.04.904.105 I slot launch_slot_: id 3 | task 6450 | processing task, is_child = 0
|
||
|
|
2.04.904.105 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.04.904.106 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.04.904.140 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.04.904.140 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.04.904.166 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.04.904.166 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.04.904.200 I slot update_slots: id 3 | task 6450 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
2.04.914.618 W slot update_slots: id 3 | task 6450 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
2.06.430.496 I slot print_timing: id 3 | task 6450 | prompt eval time = 494.36 ms / 258 tokens ( 1.92 ms per token, 521.88 tokens per second)
|
||
|
|
2.06.430.500 I slot print_timing: id 3 | task 6450 | eval time = 1031.93 ms / 63 tokens ( 16.38 ms per token, 61.05 tokens per second)
|
||
|
|
2.06.430.500 I slot print_timing: id 3 | task 6450 | total time = 1526.29 ms / 321 tokens
|
||
|
|
2.06.430.501 I slot print_timing: id 3 | task 6450 | graphs reused = 6405
|
||
|
|
2.06.430.557 I slot release: id 3 | task 6450 | stop processing: n_tokens = 717, truncated = 0
|
||
|
|
2.06.430.569 I srv update_slots: all slots are idle
|
||
|
|
2.06.455.288 I srv operator(): Chat format: peg-native
|
||
|
|
2.06.461.036 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.930 (> 0.100 thold), f_keep = 0.912
|
||
|
|
2.06.461.067 I slot launch_slot_: id 3 | task 6516 | processing task, is_child = 0
|
||
|
|
2.06.461.068 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.06.461.068 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.06.461.099 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.06.461.103 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.06.461.132 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.06.461.134 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.06.461.166 I slot update_slots: id 3 | task 6516 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
2.06.471.692 W slot update_slots: id 3 | task 6516 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
2.06.893.387 I slot create_check: id 3 | task 6516 | created context checkpoint 2 of 32 (pos_min = 0, pos_max = 673, n_tokens = 674, size = 47.399 MiB)
|
||
|
|
2.08.138.154 I slot print_timing: id 3 | task 6516 | prompt eval time = 579.06 ms / 306 tokens ( 1.89 ms per token, 528.44 tokens per second)
|
||
|
|
2.08.138.158 I slot print_timing: id 3 | task 6516 | eval time = 1097.91 ms / 67 tokens ( 16.39 ms per token, 61.02 tokens per second)
|
||
|
|
2.08.138.158 I slot print_timing: id 3 | task 6516 | total time = 1676.98 ms / 373 tokens
|
||
|
|
2.08.138.159 I slot print_timing: id 3 | task 6516 | graphs reused = 6469
|
||
|
|
2.08.138.218 I slot release: id 3 | task 6516 | stop processing: n_tokens = 769, truncated = 0
|
||
|
|
2.08.138.228 I srv update_slots: all slots are idle
|
||
|
|
2.08.159.007 I srv operator(): Chat format: peg-native
|
||
|
|
2.08.161.440 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.788 (> 0.100 thold), f_keep = 0.527
|
||
|
|
2.08.161.485 I slot launch_slot_: id 3 | task 6586 | processing task, is_child = 0
|
||
|
|
2.08.161.491 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.08.161.492 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.08.161.525 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.08.161.525 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.08.161.551 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.08.161.551 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.08.161.584 I slot update_slots: id 3 | task 6586 | Checking checkpoint with [0, 673] against 0...
|
||
|
|
2.08.161.585 I slot update_slots: id 3 | task 6586 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
2.08.172.108 W slot update_slots: id 3 | task 6586 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
2.08.172.113 W slot update_slots: id 3 | task 6586 | erased invalidated context checkpoint (pos_min = 0, pos_max = 673, n_tokens = 674, n_swa = 4096, pos_next = 397, size = 47.399 MiB)
|
||
|
|
2.11.481.111 I slot print_timing: id 3 | task 6586 | n_decoded = 183, tg = 60.74 t/s, tg_3s = 60.74 t/s
|
||
|
|
2.14.483.649 I slot print_timing: id 3 | task 6586 | n_decoded = 362, tg = 60.18 t/s, tg_3s = 59.62 t/s
|
||
|
|
2.17.496.152 I slot print_timing: id 3 | task 6586 | n_decoded = 542, tg = 60.04 t/s, tg_3s = 59.75 t/s
|
||
|
|
2.20.498.143 I slot print_timing: id 3 | task 6586 | n_decoded = 720, tg = 59.85 t/s, tg_3s = 59.29 t/s
|
||
|
|
2.20.872.159 I slot print_timing: id 3 | task 6586 | prompt eval time = 306.70 ms / 117 tokens ( 2.62 ms per token, 381.48 tokens per second)
|
||
|
|
2.20.872.162 I slot print_timing: id 3 | task 6586 | eval time = 12403.87 ms / 742 tokens ( 16.72 ms per token, 59.82 tokens per second)
|
||
|
|
2.20.872.162 I slot print_timing: id 3 | task 6586 | total time = 12710.57 ms / 859 tokens
|
||
|
|
2.20.872.163 I slot print_timing: id 3 | task 6586 | graphs reused = 7207
|
||
|
|
2.20.872.218 I slot release: id 3 | task 6586 | stop processing: n_tokens = 1255, truncated = 0
|
||
|
|
2.20.872.230 I srv update_slots: all slots are idle
|
||
|
|
2.20.881.671 I srv operator(): Chat format: peg-native
|
||
|
|
2.20.886.609 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.612 (> 0.100 thold), f_keep = 0.409
|
||
|
|
2.20.886.611 I srv get_availabl: updating prompt cache
|
||
|
|
2.20.886.752 W srv prompt_save: - saving prompt with length 1255, total state size = 120.137 MiB (draft: 0.000 MiB)
|
||
|
|
2.20.997.623 I srv load: - looking for better prompt, base f_keep = 0.409, sim = 0.612
|
||
|
|
2.20.997.635 I srv update: - cache state: 7 prompts, 1331.033 MiB (limits: 8192.000 MiB, 32768 tokens, 63484 est)
|
||
|
|
2.20.997.636 I srv update: - prompt 0x61d964e588c0: 2590 tokens, checkpoints: 1, 275.921 MiB
|
||
|
|
2.20.997.637 I srv update: - prompt 0x61d964ade9a0: 909 tokens, checkpoints: 2, 161.983 MiB
|
||
|
|
2.20.997.637 I srv update: - prompt 0x61d95c869f10: 1039 tokens, checkpoints: 2, 178.999 MiB
|
||
|
|
2.20.997.638 I srv update: - prompt 0x61d96c180600: 1914 tokens, checkpoints: 1, 211.211 MiB
|
||
|
|
2.20.997.638 I srv update: - prompt 0x61d95c2252f0: 900 tokens, checkpoints: 2, 163.302 MiB
|
||
|
|
2.20.997.639 I srv update: - prompt 0x61d96c200cc0: 1708 tokens, checkpoints: 1, 191.491 MiB
|
||
|
|
2.20.997.639 I srv update: - prompt 0x61d964d14ab0: 1255 tokens, checkpoints: 1, 148.127 MiB
|
||
|
|
2.20.997.640 I srv get_availabl: prompt cache update took 111.03 ms
|
||
|
|
2.20.997.680 I slot launch_slot_: id 3 | task 7331 | processing task, is_child = 0
|
||
|
|
2.20.997.681 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.20.997.682 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.20.997.721 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.20.997.722 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.20.997.782 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.20.997.782 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.20.997.850 I slot update_slots: id 3 | task 7331 | Checking checkpoint with [0, 397] against 0...
|
||
|
|
2.21.008.338 W slot update_slots: id 3 | task 7331 | restored context checkpoint (pos_min = 0, pos_max = 397, n_tokens = 398, n_past = 397, size = 27.990 MiB)
|
||
|
|
2.21.550.369 I slot create_check: id 3 | task 7331 | created context checkpoint 2 of 32 (pos_min = 0, pos_max = 818, n_tokens = 819, size = 57.596 MiB)
|
||
|
|
2.22.269.171 I slot print_timing: id 3 | task 7331 | prompt eval time = 646.86 ms / 441 tokens ( 1.47 ms per token, 681.75 tokens per second)
|
||
|
|
2.22.269.174 I slot print_timing: id 3 | task 7331 | eval time = 624.45 ms / 38 tokens ( 16.43 ms per token, 60.85 tokens per second)
|
||
|
|
2.22.269.175 I slot print_timing: id 3 | task 7331 | total time = 1271.32 ms / 479 tokens
|
||
|
|
2.22.269.175 I slot print_timing: id 3 | task 7331 | graphs reused = 7243
|
||
|
|
2.22.269.248 I slot release: id 3 | task 7331 | stop processing: n_tokens = 875, truncated = 0
|
||
|
|
2.22.269.262 I srv update_slots: all slots are idle
|
||
|
|
2.22.295.090 I srv operator(): Chat format: peg-native
|
||
|
|
2.22.301.400 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.951 (> 0.100 thold), f_keep = 0.957
|
||
|
|
2.22.301.425 I slot launch_slot_: id 3 | task 7372 | processing task, is_child = 0
|
||
|
|
2.22.301.425 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.22.301.426 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.22.301.461 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.22.301.461 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.22.301.491 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.22.301.492 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.22.301.526 I slot update_slots: id 3 | task 7372 | Checking checkpoint with [0, 818] against 0...
|
||
|
|
2.22.319.366 W slot update_slots: id 3 | task 7372 | restored context checkpoint (pos_min = 0, pos_max = 818, n_tokens = 819, n_past = 818, size = 57.596 MiB)
|
||
|
|
2.24.295.685 I slot print_timing: id 3 | task 7372 | prompt eval time = 228.22 ms / 62 tokens ( 3.68 ms per token, 271.67 tokens per second)
|
||
|
|
2.24.295.689 I slot print_timing: id 3 | task 7372 | eval time = 1765.93 ms / 107 tokens ( 16.50 ms per token, 60.59 tokens per second)
|
||
|
|
2.24.295.689 I slot print_timing: id 3 | task 7372 | total time = 1994.15 ms / 169 tokens
|
||
|
|
2.24.295.690 I slot print_timing: id 3 | task 7372 | graphs reused = 7348
|
||
|
|
2.24.295.759 I slot release: id 3 | task 7372 | stop processing: n_tokens = 986, truncated = 0
|
||
|
|
2.24.295.767 I srv update_slots: all slots are idle
|
||
|
|
2.24.313.604 I srv operator(): operator(): cleaning up before exit...
|