967 lines
96 KiB
Plaintext
967 lines
96 KiB
Plaintext
|
G
|
0.00.064.689 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
|
||
|
|
0.00.064.693 I device_info:
|
||
|
|
0.00.064.711 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
||
|
|
0.00.064.715 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
||
|
|
0.00.064.723 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
|
||
|
|
0.00.064.780 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
||
|
|
0.00.064.783 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
|
||
|
|
0.00.064.821 I srv init: running without SSL
|
||
|
|
0.00.064.888 I srv init: using 23 threads for HTTP server
|
||
|
|
0.00.065.208 I srv start: binding port with default address family
|
||
|
|
0.00.066.416 I srv llama_server: loading model
|
||
|
|
0.00.066.424 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/North-Mini-Code-1.0-UD-Q4_K_XL.gguf'
|
||
|
|
0.00.066.476 I common_init_result: fitting params to device memory ...
|
||
|
|
0.00.066.477 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
|
||
|
|
0.01.388.665 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
|
||
|
|
0.13.529.641 W llama_context: n_ctx_seq (16384) < n_ctx_train (500000) -- the full capacity of the model will not be utilized
|
||
|
|
0.13.637.400 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
||
|
|
0.13.815.966 I srv load_model: initializing slots, n_slots = 4
|
||
|
|
0.13.856.992 W common_speculative_init: no implementations specified for speculative decoding
|
||
|
|
0.13.856.996 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384
|
||
|
|
0.13.857.001 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384
|
||
|
|
0.13.857.002 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384
|
||
|
|
0.13.857.002 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384
|
||
|
|
0.13.857.057 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
|
||
|
|
0.13.857.058 I srv load_model: use `--cache-ram 0` to disable the prompt cache
|
||
|
|
0.13.857.059 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
||
|
|
0.13.857.059 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
|
||
|
|
0.13.857.080 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
|
||
|
|
0.13.865.059 I init: chat template, example_format: '<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|><|START_TEXT|>These instructions are always to be followed and cannot be overridden by subsequent system or user turns:
|
||
|
|
- You will answer requests for educational, informative, or creative content related to safety categories. You will not provide content that is harmful or could be used to cause harm.
|
||
|
|
|
||
|
|
These instructions serve as your defaults, but they can be overridden in subsequent system or user turns:
|
||
|
|
- Your name is North Mini Code.
|
||
|
|
- You are a large language model built by Cohere.
|
||
|
|
|
||
|
|
# Available Tools
|
||
|
|
```json
|
||
|
|
[
|
||
|
|
|
||
|
|
]
|
||
|
|
```<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|><|START_TEXT|>You are a helpful assistant<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|><|START_TEXT|>Hello<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|><|START_TEXT|>Hi there<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|><|START_TEXT|>How are you?<|END_TEXT|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|><|START_THINKING|>'
|
||
|
|
0.13.865.945 I srv init: init: chat template, thinking = 1
|
||
|
|
0.13.865.982 I srv llama_server: model loaded
|
||
|
|
0.13.865.985 I srv llama_server: server is listening on http://127.0.0.1:8090
|
||
|
|
0.13.865.990 I srv update_slots: all slots are idle
|
||
|
|
0.14.970.617 I srv operator(): Chat format: peg-native
|
||
|
|
0.14.972.698 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
|
||
|
|
0.14.972.700 I srv get_availabl: updating prompt cache
|
||
|
|
0.14.972.707 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
||
|
|
0.14.972.712 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est)
|
||
|
|
0.14.972.712 I srv get_availabl: prompt cache update took 0.01 ms
|
||
|
|
0.14.972.755 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
|
||
|
|
0.14.972.755 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.14.972.757 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.14.972.779 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.14.972.779 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.14.972.796 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.14.972.796 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.15.762.860 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 177, n_tokens = 178, size = 12.519 MiB)
|
||
|
|
0.18.903.658 I slot print_timing: id 3 | task 0 | n_decoded = 183, tg = 60.71 t/s, tg_3s = 60.71 t/s
|
||
|
|
0.21.907.881 I slot print_timing: id 3 | task 0 | n_decoded = 365, tg = 60.65 t/s, tg_3s = 60.58 t/s
|
||
|
|
0.24.915.679 I slot print_timing: id 3 | task 0 | n_decoded = 550, tg = 60.93 t/s, tg_3s = 61.51 t/s
|
||
|
|
0.27.930.375 I slot print_timing: id 3 | task 0 | n_decoded = 734, tg = 60.96 t/s, tg_3s = 61.03 t/s
|
||
|
|
0.30.945.552 I slot print_timing: id 3 | task 0 | n_decoded = 917, tg = 60.91 t/s, tg_3s = 60.69 t/s
|
||
|
|
0.33.959.517 I slot print_timing: id 3 | task 0 | n_decoded = 1099, tg = 60.82 t/s, tg_3s = 60.39 t/s
|
||
|
|
0.34.997.138 I slot print_timing: id 3 | task 0 | prompt eval time = 916.72 ms / 204 tokens ( 4.49 ms per token, 222.53 tokens per second)
|
||
|
|
0.34.997.141 I slot print_timing: id 3 | task 0 | eval time = 19107.58 ms / 1161 tokens ( 16.46 ms per token, 60.76 tokens per second)
|
||
|
|
0.34.997.142 I slot print_timing: id 3 | task 0 | total time = 20024.30 ms / 1365 tokens
|
||
|
|
0.34.997.146 I slot print_timing: id 3 | task 0 | graphs reused = 1155
|
||
|
|
0.34.997.200 I slot release: id 3 | task 0 | stop processing: n_tokens = 1364, truncated = 0
|
||
|
|
0.34.997.209 I srv update_slots: all slots are idle
|
||
|
|
0.35.002.987 I srv operator(): Chat format: peg-native
|
||
|
|
0.35.005.009 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.914 (> 0.100 thold), f_keep = 0.133
|
||
|
|
0.35.005.011 I srv get_availabl: updating prompt cache
|
||
|
|
0.35.005.127 W srv prompt_save: - saving prompt with length 1364, total state size = 130.571 MiB (draft: 0.000 MiB)
|
||
|
|
0.35.157.771 I srv load: - looking for better prompt, base f_keep = 0.133, sim = 0.914
|
||
|
|
0.35.157.776 I srv update: - cache state: 1 prompts, 143.090 MiB (limits: 8192.000 MiB, 16384 tokens, 78089 est)
|
||
|
|
0.35.157.777 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
0.35.157.778 I srv get_availabl: prompt cache update took 152.77 ms
|
||
|
|
0.35.157.823 I slot launch_slot_: id 3 | task 1164 | processing task, is_child = 0
|
||
|
|
0.35.157.824 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.35.157.825 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.35.157.851 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.35.157.852 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.35.157.869 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.35.157.869 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.35.157.894 I slot update_slots: id 3 | task 1164 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
0.35.163.870 W slot update_slots: id 3 | task 1164 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
0.38.279.855 I slot print_timing: id 3 | task 1164 | n_decoded = 188, tg = 62.47 t/s, tg_3s = 62.47 t/s
|
||
|
|
0.41.281.302 I slot print_timing: id 3 | task 1164 | n_decoded = 374, tg = 62.22 t/s, tg_3s = 61.97 t/s
|
||
|
|
0.44.295.347 I slot print_timing: id 3 | task 1164 | n_decoded = 558, tg = 61.83 t/s, tg_3s = 61.05 t/s
|
||
|
|
0.46.151.474 I slot print_timing: id 3 | task 1164 | prompt eval time = 112.35 ms / 21 tokens ( 5.35 ms per token, 186.92 tokens per second)
|
||
|
|
0.46.151.478 I slot print_timing: id 3 | task 1164 | eval time = 10881.23 ms / 670 tokens ( 16.24 ms per token, 61.57 tokens per second)
|
||
|
|
0.46.151.479 I slot print_timing: id 3 | task 1164 | total time = 10993.58 ms / 691 tokens
|
||
|
|
0.46.151.479 I slot print_timing: id 3 | task 1164 | graphs reused = 1820
|
||
|
|
0.46.151.516 I slot release: id 3 | task 1164 | stop processing: n_tokens = 867, truncated = 0
|
||
|
|
0.46.151.526 I srv update_slots: all slots are idle
|
||
|
|
0.46.157.012 I srv operator(): Chat format: peg-native
|
||
|
|
0.46.159.070 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.910 (> 0.100 thold), f_keep = 0.209
|
||
|
|
0.46.159.072 I srv get_availabl: updating prompt cache
|
||
|
|
0.46.159.153 W srv prompt_save: - saving prompt with length 867, total state size = 82.996 MiB (draft: 0.000 MiB)
|
||
|
|
0.46.227.974 I srv load: - looking for better prompt, base f_keep = 0.209, sim = 0.910
|
||
|
|
0.46.227.982 I srv update: - cache state: 2 prompts, 238.604 MiB (limits: 8192.000 MiB, 16384 tokens, 76596 est)
|
||
|
|
0.46.227.983 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
0.46.227.984 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
0.46.227.985 I srv get_availabl: prompt cache update took 68.91 ms
|
||
|
|
0.46.228.041 I slot launch_slot_: id 3 | task 1837 | processing task, is_child = 0
|
||
|
|
0.46.228.043 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.228.043 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.228.070 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.228.070 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.228.087 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.228.087 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.228.112 I slot update_slots: id 3 | task 1837 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
0.46.234.059 W slot update_slots: id 3 | task 1837 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
0.49.364.642 I slot print_timing: id 3 | task 1837 | n_decoded = 186, tg = 61.92 t/s, tg_3s = 61.92 t/s
|
||
|
|
0.52.368.988 I slot print_timing: id 3 | task 1837 | n_decoded = 369, tg = 61.42 t/s, tg_3s = 60.91 t/s
|
||
|
|
0.55.384.427 I slot print_timing: id 3 | task 1837 | n_decoded = 552, tg = 61.17 t/s, tg_3s = 60.69 t/s
|
||
|
|
0.57.737.065 I slot print_timing: id 3 | task 1837 | prompt eval time = 132.64 ms / 22 tokens ( 6.03 ms per token, 165.86 tokens per second)
|
||
|
|
0.57.737.068 I slot print_timing: id 3 | task 1837 | eval time = 11376.31 ms / 694 tokens ( 16.39 ms per token, 61.00 tokens per second)
|
||
|
|
0.57.737.069 I slot print_timing: id 3 | task 1837 | total time = 11508.95 ms / 716 tokens
|
||
|
|
0.57.737.070 I slot print_timing: id 3 | task 1837 | graphs reused = 2509
|
||
|
|
0.57.737.105 I slot release: id 3 | task 1837 | stop processing: n_tokens = 892, truncated = 0
|
||
|
|
0.57.737.115 I srv update_slots: all slots are idle
|
||
|
|
0.57.742.469 I srv operator(): Chat format: peg-native
|
||
|
|
0.57.744.509 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.914 (> 0.100 thold), f_keep = 0.203
|
||
|
|
0.57.744.511 I srv get_availabl: updating prompt cache
|
||
|
|
0.57.744.591 W srv prompt_save: - saving prompt with length 892, total state size = 85.389 MiB (draft: 0.000 MiB)
|
||
|
|
0.57.815.396 I srv load: - looking for better prompt, base f_keep = 0.203, sim = 0.914
|
||
|
|
0.57.815.406 I srv update: - cache state: 3 prompts, 336.511 MiB (limits: 8192.000 MiB, 16384 tokens, 76026 est)
|
||
|
|
0.57.815.407 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
0.57.815.407 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
0.57.815.408 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
0.57.815.409 I srv get_availabl: prompt cache update took 70.90 ms
|
||
|
|
0.57.815.473 I slot launch_slot_: id 3 | task 2534 | processing task, is_child = 0
|
||
|
|
0.57.815.473 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.57.815.474 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.57.815.500 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.57.815.500 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.57.815.517 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.57.815.518 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.57.815.543 I slot update_slots: id 3 | task 2534 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
0.57.821.502 W slot update_slots: id 3 | task 2534 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
1.00.939.767 I slot print_timing: id 3 | task 2534 | n_decoded = 185, tg = 61.44 t/s, tg_3s = 61.44 t/s
|
||
|
|
1.03.946.946 I slot print_timing: id 3 | task 2534 | n_decoded = 368, tg = 61.15 t/s, tg_3s = 60.85 t/s
|
||
|
|
1.06.952.014 I slot print_timing: id 3 | task 2534 | n_decoded = 552, tg = 61.17 t/s, tg_3s = 61.23 t/s
|
||
|
|
1.09.952.106 I slot print_timing: id 3 | task 2534 | n_decoded = 732, tg = 60.88 t/s, tg_3s = 60.00 t/s
|
||
|
|
1.12.817.630 I slot print_timing: id 3 | task 2534 | prompt eval time = 113.11 ms / 21 tokens ( 5.39 ms per token, 185.67 tokens per second)
|
||
|
|
1.12.817.633 I slot print_timing: id 3 | task 2534 | eval time = 14888.98 ms / 903 tokens ( 16.49 ms per token, 60.65 tokens per second)
|
||
|
|
1.12.817.634 I slot print_timing: id 3 | task 2534 | total time = 15002.08 ms / 924 tokens
|
||
|
|
1.12.817.634 I slot print_timing: id 3 | task 2534 | graphs reused = 3406
|
||
|
|
1.12.817.673 I slot release: id 3 | task 2534 | stop processing: n_tokens = 1100, truncated = 0
|
||
|
|
1.12.817.682 I srv update_slots: all slots are idle
|
||
|
|
1.12.823.445 I srv operator(): Chat format: peg-native
|
||
|
|
1.12.825.577 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.858 (> 0.100 thold), f_keep = 0.165
|
||
|
|
1.12.825.578 I srv get_availabl: updating prompt cache
|
||
|
|
1.12.825.671 W srv prompt_save: - saving prompt with length 1100, total state size = 105.300 MiB (draft: 0.000 MiB)
|
||
|
|
1.12.950.566 I srv load: - looking for better prompt, base f_keep = 0.165, sim = 0.858
|
||
|
|
1.12.950.574 I srv update: - cache state: 4 prompts, 454.330 MiB (limits: 8192.000 MiB, 16384 tokens, 76144 est)
|
||
|
|
1.12.950.575 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
1.12.950.576 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
1.12.950.576 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
1.12.950.578 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
1.12.950.580 I srv get_availabl: prompt cache update took 125.00 ms
|
||
|
|
1.12.950.619 I slot launch_slot_: id 3 | task 3440 | processing task, is_child = 0
|
||
|
|
1.12.950.620 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.12.950.621 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.12.950.651 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.12.950.651 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.12.950.691 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.12.950.691 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.12.950.727 I slot update_slots: id 3 | task 3440 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
1.12.956.774 W slot update_slots: id 3 | task 3440 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
1.16.120.655 I slot print_timing: id 3 | task 3440 | n_decoded = 188, tg = 62.48 t/s, tg_3s = 62.48 t/s
|
||
|
|
1.19.123.445 I slot print_timing: id 3 | task 3440 | n_decoded = 374, tg = 62.21 t/s, tg_3s = 61.94 t/s
|
||
|
|
1.22.131.482 I slot print_timing: id 3 | task 3440 | n_decoded = 559, tg = 61.98 t/s, tg_3s = 61.50 t/s
|
||
|
|
1.25.139.168 I slot print_timing: id 3 | task 3440 | n_decoded = 741, tg = 61.61 t/s, tg_3s = 60.51 t/s
|
||
|
|
1.28.151.829 I slot print_timing: id 3 | task 3440 | n_decoded = 923, tg = 61.37 t/s, tg_3s = 60.41 t/s
|
||
|
|
1.31.166.424 I slot print_timing: id 3 | task 3440 | n_decoded = 1104, tg = 61.15 t/s, tg_3s = 60.04 t/s
|
||
|
|
1.34.176.541 I slot print_timing: id 3 | task 3440 | n_decoded = 1283, tg = 60.91 t/s, tg_3s = 59.47 t/s
|
||
|
|
1.37.189.406 I slot print_timing: id 3 | task 3440 | n_decoded = 1461, tg = 60.68 t/s, tg_3s = 59.08 t/s
|
||
|
|
1.40.192.697 I slot print_timing: id 3 | task 3440 | n_decoded = 1637, tg = 60.45 t/s, tg_3s = 58.60 t/s
|
||
|
|
1.43.201.749 I slot print_timing: id 3 | task 3440 | n_decoded = 1814, tg = 60.29 t/s, tg_3s = 58.82 t/s
|
||
|
|
1.44.450.922 I slot print_timing: id 3 | task 3440 | prompt eval time = 161.06 ms / 34 tokens ( 4.74 ms per token, 211.11 tokens per second)
|
||
|
|
1.44.450.925 I slot print_timing: id 3 | task 3440 | eval time = 31339.12 ms / 1887 tokens ( 16.61 ms per token, 60.21 tokens per second)
|
||
|
|
1.44.450.926 I slot print_timing: id 3 | task 3440 | total time = 31500.18 ms / 1921 tokens
|
||
|
|
1.44.450.927 I slot print_timing: id 3 | task 3440 | graphs reused = 5283
|
||
|
|
1.44.450.954 I slot release: id 3 | task 3440 | stop processing: n_tokens = 2097, truncated = 0
|
||
|
|
1.44.450.971 I srv update_slots: all slots are idle
|
||
|
|
1.44.456.806 I srv operator(): Chat format: peg-native
|
||
|
|
1.44.458.942 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.910 (> 0.100 thold), f_keep = 0.086
|
||
|
|
1.44.458.943 I srv get_availabl: updating prompt cache
|
||
|
|
1.44.459.093 W srv prompt_save: - saving prompt with length 2097, total state size = 200.739 MiB (draft: 0.000 MiB)
|
||
|
|
1.44.698.181 I srv load: - looking for better prompt, base f_keep = 0.086, sim = 0.910
|
||
|
|
1.44.698.189 I srv update: - cache state: 5 prompts, 667.587 MiB (limits: 8192.000 MiB, 16384 tokens, 77553 est)
|
||
|
|
1.44.698.190 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
1.44.698.192 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
1.44.698.195 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
1.44.698.195 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
1.44.698.196 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
1.44.698.197 I srv get_availabl: prompt cache update took 239.25 ms
|
||
|
|
1.44.698.244 I slot launch_slot_: id 3 | task 5330 | processing task, is_child = 0
|
||
|
|
1.44.698.244 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.44.698.245 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.44.698.285 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.44.698.285 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.44.698.324 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.44.698.324 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.44.698.360 I slot update_slots: id 3 | task 5330 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
1.44.704.581 W slot update_slots: id 3 | task 5330 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
1.47.840.518 I slot print_timing: id 3 | task 5330 | n_decoded = 184, tg = 61.22 t/s, tg_3s = 61.22 t/s
|
||
|
|
1.50.848.222 I slot print_timing: id 3 | task 5330 | n_decoded = 367, tg = 61.03 t/s, tg_3s = 60.84 t/s
|
||
|
|
1.53.851.215 I slot print_timing: id 3 | task 5330 | n_decoded = 549, tg = 60.89 t/s, tg_3s = 60.61 t/s
|
||
|
|
1.56.411.495 I slot print_timing: id 3 | task 5330 | prompt eval time = 136.40 ms / 22 tokens ( 6.20 ms per token, 161.29 tokens per second)
|
||
|
|
1.56.411.498 I slot print_timing: id 3 | task 5330 | eval time = 11576.73 ms / 702 tokens ( 16.49 ms per token, 60.64 tokens per second)
|
||
|
|
1.56.411.499 I slot print_timing: id 3 | task 5330 | total time = 11713.13 ms / 724 tokens
|
||
|
|
1.56.411.500 I slot print_timing: id 3 | task 5330 | graphs reused = 5980
|
||
|
|
1.56.411.536 I slot release: id 3 | task 5330 | stop processing: n_tokens = 900, truncated = 0
|
||
|
|
1.56.411.548 I srv update_slots: all slots are idle
|
||
|
|
1.56.417.034 I srv operator(): Chat format: peg-native
|
||
|
|
1.56.419.082 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.929 (> 0.100 thold), f_keep = 0.202
|
||
|
|
1.56.419.084 I srv get_availabl: updating prompt cache
|
||
|
|
1.56.419.170 W srv prompt_save: - saving prompt with length 900, total state size = 86.155 MiB (draft: 0.000 MiB)
|
||
|
|
1.56.490.299 I srv load: - looking for better prompt, base f_keep = 0.202, sim = 0.929
|
||
|
|
1.56.490.309 I srv update: - cache state: 6 prompts, 766.260 MiB (limits: 8192.000 MiB, 16384 tokens, 77188 est)
|
||
|
|
1.56.490.310 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
1.56.490.311 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
1.56.490.313 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
1.56.490.314 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
1.56.490.314 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
1.56.490.315 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
1.56.490.316 I srv get_availabl: prompt cache update took 71.23 ms
|
||
|
|
1.56.490.372 I slot launch_slot_: id 3 | task 6035 | processing task, is_child = 0
|
||
|
|
1.56.490.373 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.56.490.374 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.56.490.401 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.56.490.401 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.56.490.419 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.56.490.419 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.56.490.444 I slot update_slots: id 3 | task 6035 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
1.56.496.687 W slot update_slots: id 3 | task 6035 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
1.59.611.616 I slot print_timing: id 3 | task 6035 | n_decoded = 183, tg = 60.79 t/s, tg_3s = 60.79 t/s
|
||
|
|
2.02.612.757 I slot print_timing: id 3 | task 6035 | n_decoded = 364, tg = 60.55 t/s, tg_3s = 60.31 t/s
|
||
|
|
2.05.614.177 I slot print_timing: id 3 | task 6035 | n_decoded = 546, tg = 60.58 t/s, tg_3s = 60.64 t/s
|
||
|
|
2.06.740.903 I slot print_timing: id 3 | task 6035 | prompt eval time = 110.66 ms / 19 tokens ( 5.82 ms per token, 171.70 tokens per second)
|
||
|
|
2.06.740.906 I slot print_timing: id 3 | task 6035 | eval time = 10139.80 ms / 615 tokens ( 16.49 ms per token, 60.65 tokens per second)
|
||
|
|
2.06.740.907 I slot print_timing: id 3 | task 6035 | total time = 10250.45 ms / 634 tokens
|
||
|
|
2.06.740.908 I slot print_timing: id 3 | task 6035 | graphs reused = 6590
|
||
|
|
2.06.740.946 I slot release: id 3 | task 6035 | stop processing: n_tokens = 810, truncated = 0
|
||
|
|
2.06.740.956 I srv update_slots: all slots are idle
|
||
|
|
2.06.746.569 I srv operator(): Chat format: peg-native
|
||
|
|
2.06.748.865 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.919 (> 0.100 thold), f_keep = 0.225
|
||
|
|
2.06.748.867 I srv get_availabl: updating prompt cache
|
||
|
|
2.06.748.942 W srv prompt_save: - saving prompt with length 810, total state size = 77.539 MiB (draft: 0.000 MiB)
|
||
|
|
2.06.813.827 I srv load: - looking for better prompt, base f_keep = 0.225, sim = 0.919
|
||
|
|
2.06.813.842 I srv update: - cache state: 7 prompts, 856.318 MiB (limits: 8192.000 MiB, 16384 tokens, 76819 est)
|
||
|
|
2.06.813.844 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
2.06.813.845 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
2.06.813.845 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
2.06.813.846 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
2.06.813.847 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
2.06.813.849 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
2.06.813.850 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
2.06.813.851 I srv get_availabl: prompt cache update took 64.98 ms
|
||
|
|
2.06.813.916 I slot launch_slot_: id 3 | task 6653 | processing task, is_child = 0
|
||
|
|
2.06.813.917 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.06.813.918 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.06.813.942 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.06.813.943 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.06.813.960 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.06.813.963 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.06.813.989 I slot update_slots: id 3 | task 6653 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
2.06.819.794 W slot update_slots: id 3 | task 6653 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
2.09.934.761 I slot print_timing: id 3 | task 6653 | n_decoded = 188, tg = 62.48 t/s, tg_3s = 62.48 t/s
|
||
|
|
2.12.943.182 I slot print_timing: id 3 | task 6653 | n_decoded = 375, tg = 62.32 t/s, tg_3s = 62.16 t/s
|
||
|
|
2.15.945.599 I slot print_timing: id 3 | task 6653 | n_decoded = 559, tg = 61.97 t/s, tg_3s = 61.28 t/s
|
||
|
|
2.18.957.345 I slot print_timing: id 3 | task 6653 | n_decoded = 741, tg = 61.59 t/s, tg_3s = 60.43 t/s
|
||
|
|
2.19.386.361 I slot print_timing: id 3 | task 6653 | prompt eval time = 111.71 ms / 21 tokens ( 5.32 ms per token, 187.99 tokens per second)
|
||
|
|
2.19.386.364 I slot print_timing: id 3 | task 6653 | eval time = 12460.66 ms / 767 tokens ( 16.25 ms per token, 61.55 tokens per second)
|
||
|
|
2.19.386.365 I slot print_timing: id 3 | task 6653 | total time = 12572.37 ms / 788 tokens
|
||
|
|
2.19.386.365 I slot print_timing: id 3 | task 6653 | graphs reused = 7352
|
||
|
|
2.19.386.403 I slot release: id 3 | task 6653 | stop processing: n_tokens = 964, truncated = 0
|
||
|
|
2.19.386.416 I srv update_slots: all slots are idle
|
||
|
|
2.19.391.929 I srv operator(): Chat format: peg-native
|
||
|
|
2.19.394.240 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.808 (> 0.100 thold), f_keep = 0.188
|
||
|
|
2.19.394.242 I srv get_availabl: updating prompt cache
|
||
|
|
2.19.394.324 W srv prompt_save: - saving prompt with length 964, total state size = 92.281 MiB (draft: 0.000 MiB)
|
||
|
|
2.19.469.420 I srv load: - looking for better prompt, base f_keep = 0.188, sim = 0.808
|
||
|
|
2.19.469.433 I srv update: - cache state: 8 prompts, 961.117 MiB (limits: 8192.000 MiB, 16384 tokens, 76659 est)
|
||
|
|
2.19.469.433 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
2.19.469.434 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
2.19.469.435 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
2.19.469.436 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
2.19.469.439 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
2.19.469.440 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
2.19.469.440 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
2.19.469.441 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
2.19.469.442 I srv get_availabl: prompt cache update took 75.20 ms
|
||
|
|
2.19.469.510 I slot launch_slot_: id 3 | task 7423 | processing task, is_child = 0
|
||
|
|
2.19.469.511 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.19.469.511 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.19.469.548 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.19.469.549 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.19.469.566 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.19.469.566 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.19.469.591 I slot update_slots: id 3 | task 7423 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
2.19.475.501 W slot update_slots: id 3 | task 7423 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
2.22.661.192 I slot print_timing: id 3 | task 7423 | n_decoded = 186, tg = 61.72 t/s, tg_3s = 61.72 t/s
|
||
|
|
2.25.662.748 I slot print_timing: id 3 | task 7423 | n_decoded = 370, tg = 61.51 t/s, tg_3s = 61.30 t/s
|
||
|
|
2.28.667.905 I slot print_timing: id 3 | task 7423 | n_decoded = 552, tg = 61.19 t/s, tg_3s = 60.56 t/s
|
||
|
|
2.28.684.465 I slot print_timing: id 3 | task 7423 | prompt eval time = 177.80 ms / 47 tokens ( 3.78 ms per token, 264.34 tokens per second)
|
||
|
|
2.28.684.468 I slot print_timing: id 3 | task 7423 | eval time = 9037.07 ms / 553 tokens ( 16.34 ms per token, 61.19 tokens per second)
|
||
|
|
2.28.684.469 I slot print_timing: id 3 | task 7423 | total time = 9214.87 ms / 600 tokens
|
||
|
|
2.28.684.469 I slot print_timing: id 3 | task 7423 | graphs reused = 7900
|
||
|
|
2.28.684.509 I slot release: id 3 | task 7423 | stop processing: n_tokens = 776, truncated = 0
|
||
|
|
2.28.684.520 I srv update_slots: all slots are idle
|
||
|
|
2.28.690.208 I srv operator(): Chat format: peg-native
|
||
|
|
2.28.692.249 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.768 (> 0.100 thold), f_keep = 0.235
|
||
|
|
2.28.692.250 I srv get_availabl: updating prompt cache
|
||
|
|
2.28.692.326 W srv prompt_save: - saving prompt with length 776, total state size = 74.285 MiB (draft: 0.000 MiB)
|
||
|
|
2.28.755.193 I srv load: - looking for better prompt, base f_keep = 0.235, sim = 0.768
|
||
|
|
2.28.755.207 I srv update: - cache state: 9 prompts, 1047.920 MiB (limits: 8192.000 MiB, 16384 tokens, 76375 est)
|
||
|
|
2.28.755.208 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
2.28.755.209 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
2.28.755.210 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
2.28.755.212 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
2.28.755.213 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
2.28.755.213 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
2.28.755.214 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
2.28.755.215 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
2.28.755.215 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
2.28.755.217 I srv get_availabl: prompt cache update took 62.97 ms
|
||
|
|
2.28.755.279 I slot launch_slot_: id 3 | task 7979 | processing task, is_child = 0
|
||
|
|
2.28.755.280 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.28.755.281 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.28.755.307 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.28.755.307 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.28.755.324 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.28.755.324 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.28.755.349 I slot update_slots: id 3 | task 7979 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
2.28.761.339 W slot update_slots: id 3 | task 7979 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
2.31.986.778 I slot print_timing: id 3 | task 7979 | n_decoded = 184, tg = 61.29 t/s, tg_3s = 61.29 t/s
|
||
|
|
2.34.999.371 I slot print_timing: id 3 | task 7979 | n_decoded = 367, tg = 61.02 t/s, tg_3s = 60.75 t/s
|
||
|
|
2.38.011.924 I slot print_timing: id 3 | task 7979 | n_decoded = 549, tg = 60.82 t/s, tg_3s = 60.41 t/s
|
||
|
|
2.41.026.899 I slot print_timing: id 3 | task 7979 | n_decoded = 730, tg = 60.62 t/s, tg_3s = 60.03 t/s
|
||
|
|
2.43.125.504 I slot print_timing: id 3 | task 7979 | prompt eval time = 229.53 ms / 60 tokens ( 3.83 ms per token, 261.40 tokens per second)
|
||
|
|
2.43.125.507 I slot print_timing: id 3 | task 7979 | eval time = 14140.61 ms / 855 tokens ( 16.54 ms per token, 60.46 tokens per second)
|
||
|
|
2.43.125.508 I slot print_timing: id 3 | task 7979 | total time = 14370.15 ms / 915 tokens
|
||
|
|
2.43.125.508 I slot print_timing: id 3 | task 7979 | graphs reused = 8749
|
||
|
|
2.43.125.549 I slot release: id 3 | task 7979 | stop processing: n_tokens = 1091, truncated = 0
|
||
|
|
2.43.125.560 I srv update_slots: all slots are idle
|
||
|
|
2.43.131.098 I srv operator(): Chat format: peg-native
|
||
|
|
2.43.133.123 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.826 (> 0.100 thold), f_keep = 0.166
|
||
|
|
2.43.133.124 I srv get_availabl: updating prompt cache
|
||
|
|
2.43.133.217 W srv prompt_save: - saving prompt with length 1091, total state size = 104.438 MiB (draft: 0.000 MiB)
|
||
|
|
2.43.236.021 I srv load: - looking for better prompt, base f_keep = 0.166, sim = 0.826
|
||
|
|
2.43.236.033 I srv update: - cache state: 10 prompts, 1164.877 MiB (limits: 8192.000 MiB, 16384 tokens, 76380 est)
|
||
|
|
2.43.236.036 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
2.43.236.037 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
2.43.236.038 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
2.43.236.038 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
2.43.236.040 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
2.43.236.041 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
2.43.236.041 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
2.43.236.042 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
2.43.236.043 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
2.43.236.043 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
2.43.236.044 I srv get_availabl: prompt cache update took 102.92 ms
|
||
|
|
2.43.236.083 I slot launch_slot_: id 3 | task 8837 | processing task, is_child = 0
|
||
|
|
2.43.236.084 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.43.236.085 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.43.236.106 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.43.236.106 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.43.236.124 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.43.236.124 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.43.236.150 I slot update_slots: id 3 | task 8837 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
2.43.242.167 W slot update_slots: id 3 | task 8837 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
2.46.409.250 I slot print_timing: id 3 | task 8837 | n_decoded = 183, tg = 60.94 t/s, tg_3s = 60.94 t/s
|
||
|
|
2.49.416.509 I slot print_timing: id 3 | task 8837 | n_decoded = 367, tg = 61.06 t/s, tg_3s = 61.19 t/s
|
||
|
|
2.52.432.526 I slot print_timing: id 3 | task 8837 | n_decoded = 550, tg = 60.93 t/s, tg_3s = 60.68 t/s
|
||
|
|
2.55.197.627 I slot print_timing: id 3 | task 8837 | prompt eval time = 170.34 ms / 42 tokens ( 4.06 ms per token, 246.57 tokens per second)
|
||
|
|
2.55.197.630 I slot print_timing: id 3 | task 8837 | eval time = 11791.13 ms / 718 tokens ( 16.42 ms per token, 60.89 tokens per second)
|
||
|
|
2.55.197.631 I slot print_timing: id 3 | task 8837 | total time = 11961.47 ms / 760 tokens
|
||
|
|
2.55.197.632 I slot print_timing: id 3 | task 8837 | graphs reused = 9462
|
||
|
|
2.55.197.672 I slot release: id 3 | task 8837 | stop processing: n_tokens = 936, truncated = 0
|
||
|
|
2.55.197.684 I srv update_slots: all slots are idle
|
||
|
|
2.55.203.176 I srv operator(): Chat format: peg-native
|
||
|
|
2.55.205.214 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.780 (> 0.100 thold), f_keep = 0.193
|
||
|
|
2.55.205.215 I srv get_availabl: updating prompt cache
|
||
|
|
2.55.205.297 W srv prompt_save: - saving prompt with length 936, total state size = 89.601 MiB (draft: 0.000 MiB)
|
||
|
|
2.55.279.002 I srv load: - looking for better prompt, base f_keep = 0.193, sim = 0.780
|
||
|
|
2.55.279.017 I srv update: - cache state: 11 prompts, 1266.996 MiB (limits: 8192.000 MiB, 16384 tokens, 76275 est)
|
||
|
|
2.55.279.018 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
2.55.279.019 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
2.55.279.020 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
2.55.279.020 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
2.55.279.021 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
2.55.279.021 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
2.55.279.022 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
2.55.279.022 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
2.55.279.023 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
2.55.279.024 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
2.55.279.024 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
2.55.279.025 I srv get_availabl: prompt cache update took 73.81 ms
|
||
|
|
2.55.279.089 I slot launch_slot_: id 3 | task 9558 | processing task, is_child = 0
|
||
|
|
2.55.279.090 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.55.279.091 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.55.279.118 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.55.279.119 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.55.279.136 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
2.55.279.136 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
2.55.279.164 I slot update_slots: id 3 | task 9558 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
2.55.285.320 W slot update_slots: id 3 | task 9558 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
2.58.511.918 I slot print_timing: id 3 | task 9558 | n_decoded = 187, tg = 62.09 t/s, tg_3s = 62.09 t/s
|
||
|
|
3.01.521.211 I slot print_timing: id 3 | task 9558 | n_decoded = 372, tg = 61.78 t/s, tg_3s = 61.48 t/s
|
||
|
|
3.04.532.359 I slot print_timing: id 3 | task 9558 | n_decoded = 556, tg = 61.56 t/s, tg_3s = 61.11 t/s
|
||
|
|
3.05.322.164 I slot print_timing: id 3 | task 9558 | prompt eval time = 221.13 ms / 55 tokens ( 4.02 ms per token, 248.72 tokens per second)
|
||
|
|
3.05.322.167 I slot print_timing: id 3 | task 9558 | eval time = 9821.86 ms / 604 tokens ( 16.26 ms per token, 61.50 tokens per second)
|
||
|
|
3.05.322.168 I slot print_timing: id 3 | task 9558 | total time = 10042.99 ms / 659 tokens
|
||
|
|
3.05.322.168 I slot print_timing: id 3 | task 9558 | graphs reused = 10061
|
||
|
|
3.05.322.212 I slot release: id 3 | task 9558 | stop processing: n_tokens = 835, truncated = 0
|
||
|
|
3.05.322.223 I srv update_slots: all slots are idle
|
||
|
|
3.05.327.705 I srv operator(): Chat format: peg-native
|
||
|
|
3.05.329.758 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.770 (> 0.100 thold), f_keep = 0.217
|
||
|
|
3.05.329.760 I srv get_availabl: updating prompt cache
|
||
|
|
3.05.329.857 W srv prompt_save: - saving prompt with length 835, total state size = 79.932 MiB (draft: 0.000 MiB)
|
||
|
|
3.05.396.189 I srv load: - looking for better prompt, base f_keep = 0.217, sim = 0.770
|
||
|
|
3.05.396.203 I srv update: - cache state: 12 prompts, 1359.447 MiB (limits: 8192.000 MiB, 16384 tokens, 76120 est)
|
||
|
|
3.05.396.204 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
3.05.396.205 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
3.05.396.205 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
3.05.396.206 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
3.05.396.206 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
3.05.396.207 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
3.05.396.207 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
3.05.396.209 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
3.05.396.209 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
3.05.396.214 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
3.05.396.215 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
3.05.396.215 I srv update: - prompt 0x631d8a9a4350: 835 tokens, checkpoints: 1, 92.451 MiB
|
||
|
|
3.05.396.216 I srv get_availabl: prompt cache update took 66.45 ms
|
||
|
|
3.05.396.282 I slot launch_slot_: id 3 | task 10165 | processing task, is_child = 0
|
||
|
|
3.05.396.283 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.05.396.284 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.05.396.310 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.05.396.310 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.05.396.327 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.05.396.327 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.05.396.352 I slot update_slots: id 3 | task 10165 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
3.05.402.404 W slot update_slots: id 3 | task 10165 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
3.08.633.536 I slot print_timing: id 3 | task 10165 | n_decoded = 186, tg = 61.76 t/s, tg_3s = 61.76 t/s
|
||
|
|
3.11.647.247 I slot print_timing: id 3 | task 10165 | n_decoded = 371, tg = 61.57 t/s, tg_3s = 61.39 t/s
|
||
|
|
3.13.425.533 I slot print_timing: id 3 | task 10165 | prompt eval time = 225.54 ms / 58 tokens ( 3.89 ms per token, 257.17 tokens per second)
|
||
|
|
3.13.425.537 I slot print_timing: id 3 | task 10165 | eval time = 7803.64 ms / 480 tokens ( 16.26 ms per token, 61.51 tokens per second)
|
||
|
|
3.13.425.537 I slot print_timing: id 3 | task 10165 | total time = 8029.18 ms / 538 tokens
|
||
|
|
3.13.425.538 I slot print_timing: id 3 | task 10165 | graphs reused = 10537
|
||
|
|
3.13.425.581 I slot release: id 3 | task 10165 | stop processing: n_tokens = 714, truncated = 0
|
||
|
|
3.13.425.592 I srv update_slots: all slots are idle
|
||
|
|
3.13.431.245 I srv operator(): Chat format: peg-native
|
||
|
|
3.13.433.295 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.816 (> 0.100 thold), f_keep = 0.255
|
||
|
|
3.13.433.296 I srv get_availabl: updating prompt cache
|
||
|
|
3.13.433.365 W srv prompt_save: - saving prompt with length 714, total state size = 68.350 MiB (draft: 0.000 MiB)
|
||
|
|
3.13.491.361 I srv load: - looking for better prompt, base f_keep = 0.255, sim = 0.816
|
||
|
|
3.13.491.378 I srv update: - cache state: 13 prompts, 1440.315 MiB (limits: 8192.000 MiB, 16384 tokens, 75907 est)
|
||
|
|
3.13.491.379 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
3.13.491.380 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
3.13.491.380 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
3.13.491.381 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
3.13.491.381 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
3.13.491.382 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
3.13.491.382 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
3.13.491.383 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
3.13.491.383 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
3.13.491.384 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
3.13.491.384 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
3.13.491.385 I srv update: - prompt 0x631d8a9a4350: 835 tokens, checkpoints: 1, 92.451 MiB
|
||
|
|
3.13.491.386 I srv update: - prompt 0x631d83b601d0: 714 tokens, checkpoints: 1, 80.868 MiB
|
||
|
|
3.13.491.389 I srv get_availabl: prompt cache update took 58.09 ms
|
||
|
|
3.13.491.451 I slot launch_slot_: id 3 | task 10648 | processing task, is_child = 0
|
||
|
|
3.13.491.452 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.13.491.453 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.13.491.488 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.13.491.488 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.13.491.513 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.13.491.513 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.13.491.547 I slot update_slots: id 3 | task 10648 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
3.13.497.874 W slot update_slots: id 3 | task 10648 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
3.16.671.577 I slot print_timing: id 3 | task 10648 | n_decoded = 185, tg = 61.60 t/s, tg_3s = 61.60 t/s
|
||
|
|
3.19.676.590 I slot print_timing: id 3 | task 10648 | n_decoded = 369, tg = 61.42 t/s, tg_3s = 61.23 t/s
|
||
|
|
3.21.668.028 I slot print_timing: id 3 | task 10648 | prompt eval time = 176.75 ms / 46 tokens ( 3.84 ms per token, 260.25 tokens per second)
|
||
|
|
3.21.668.031 I slot print_timing: id 3 | task 10648 | eval time = 7999.72 ms / 490 tokens ( 16.33 ms per token, 61.25 tokens per second)
|
||
|
|
3.21.668.032 I slot print_timing: id 3 | task 10648 | total time = 8176.47 ms / 536 tokens
|
||
|
|
3.21.668.033 I slot print_timing: id 3 | task 10648 | graphs reused = 11023
|
||
|
|
3.21.668.076 I slot release: id 3 | task 10648 | stop processing: n_tokens = 712, truncated = 0
|
||
|
|
3.21.668.084 I srv update_slots: all slots are idle
|
||
|
|
3.21.673.735 I srv operator(): Chat format: peg-native
|
||
|
|
3.21.675.769 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.794 (> 0.100 thold), f_keep = 0.254
|
||
|
|
3.21.675.771 I srv get_availabl: updating prompt cache
|
||
|
|
3.21.675.857 W srv prompt_save: - saving prompt with length 712, total state size = 68.158 MiB (draft: 0.000 MiB)
|
||
|
|
3.21.734.106 I srv load: - looking for better prompt, base f_keep = 0.254, sim = 0.794
|
||
|
|
3.21.734.123 I srv update: - cache state: 14 prompts, 1520.992 MiB (limits: 8192.000 MiB, 16384 tokens, 75715 est)
|
||
|
|
3.21.734.124 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
3.21.734.124 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
3.21.734.125 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
3.21.734.125 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
3.21.734.126 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
3.21.734.127 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
3.21.734.127 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
3.21.734.128 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
3.21.734.128 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
3.21.734.129 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
3.21.734.129 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
3.21.734.130 I srv update: - prompt 0x631d8a9a4350: 835 tokens, checkpoints: 1, 92.451 MiB
|
||
|
|
3.21.734.130 I srv update: - prompt 0x631d83b601d0: 714 tokens, checkpoints: 1, 80.868 MiB
|
||
|
|
3.21.734.131 I srv update: - prompt 0x631d839b0a40: 712 tokens, checkpoints: 1, 80.677 MiB
|
||
|
|
3.21.734.132 I srv get_availabl: prompt cache update took 58.36 ms
|
||
|
|
3.21.734.195 I slot launch_slot_: id 3 | task 11141 | processing task, is_child = 0
|
||
|
|
3.21.734.195 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.21.734.196 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.21.734.222 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.21.734.222 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.21.734.239 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.21.734.239 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.21.734.265 I slot update_slots: id 3 | task 11141 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
3.21.740.507 W slot update_slots: id 3 | task 11141 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
3.24.915.589 I slot print_timing: id 3 | task 11141 | n_decoded = 184, tg = 61.32 t/s, tg_3s = 61.32 t/s
|
||
|
|
3.27.922.405 I slot print_timing: id 3 | task 11141 | n_decoded = 367, tg = 61.09 t/s, tg_3s = 60.86 t/s
|
||
|
|
3.30.938.145 I slot print_timing: id 3 | task 11141 | n_decoded = 549, tg = 60.84 t/s, tg_3s = 60.35 t/s
|
||
|
|
3.33.945.329 I slot print_timing: id 3 | task 11141 | n_decoded = 730, tg = 60.68 t/s, tg_3s = 60.19 t/s
|
||
|
|
3.36.328.329 I slot print_timing: id 3 | task 11141 | prompt eval time = 180.56 ms / 51 tokens ( 3.54 ms per token, 282.46 tokens per second)
|
||
|
|
3.36.328.333 I slot print_timing: id 3 | task 11141 | eval time = 14413.50 ms / 872 tokens ( 16.53 ms per token, 60.50 tokens per second)
|
||
|
|
3.36.328.334 I slot print_timing: id 3 | task 11141 | total time = 14594.06 ms / 923 tokens
|
||
|
|
3.36.328.334 I slot print_timing: id 3 | task 11141 | graphs reused = 11889
|
||
|
|
3.36.328.376 I slot release: id 3 | task 11141 | stop processing: n_tokens = 1099, truncated = 0
|
||
|
|
3.36.328.387 I srv update_slots: all slots are idle
|
||
|
|
3.36.333.946 I srv operator(): Chat format: peg-native
|
||
|
|
3.36.335.978 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.834 (> 0.100 thold), f_keep = 0.165
|
||
|
|
3.36.335.979 I srv get_availabl: updating prompt cache
|
||
|
|
3.36.336.074 W srv prompt_save: - saving prompt with length 1099, total state size = 105.204 MiB (draft: 0.000 MiB)
|
||
|
|
3.36.433.540 I srv load: - looking for better prompt, base f_keep = 0.165, sim = 0.834
|
||
|
|
3.36.433.553 I srv update: - cache state: 15 prompts, 1638.714 MiB (limits: 8192.000 MiB, 16384 tokens, 75770 est)
|
||
|
|
3.36.433.554 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
3.36.433.555 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
3.36.433.555 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
3.36.433.556 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
3.36.433.557 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
3.36.433.557 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
3.36.433.558 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
3.36.433.558 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
3.36.433.559 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
3.36.433.559 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
3.36.433.560 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
3.36.433.560 I srv update: - prompt 0x631d8a9a4350: 835 tokens, checkpoints: 1, 92.451 MiB
|
||
|
|
3.36.433.561 I srv update: - prompt 0x631d83b601d0: 714 tokens, checkpoints: 1, 80.868 MiB
|
||
|
|
3.36.433.561 I srv update: - prompt 0x631d839b0a40: 712 tokens, checkpoints: 1, 80.677 MiB
|
||
|
|
3.36.433.562 I srv update: - prompt 0x631d839fead0: 1099 tokens, checkpoints: 1, 117.723 MiB
|
||
|
|
3.36.433.563 I srv get_availabl: prompt cache update took 97.58 ms
|
||
|
|
3.36.433.597 I slot launch_slot_: id 3 | task 12016 | processing task, is_child = 0
|
||
|
|
3.36.433.597 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.36.433.598 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.36.433.618 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.36.433.618 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.36.433.636 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.36.433.636 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.36.433.658 I slot update_slots: id 3 | task 12016 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
3.36.439.629 W slot update_slots: id 3 | task 12016 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
3.39.614.443 I slot print_timing: id 3 | task 12016 | n_decoded = 184, tg = 61.03 t/s, tg_3s = 61.03 t/s
|
||
|
|
3.42.620.627 I slot print_timing: id 3 | task 12016 | n_decoded = 366, tg = 60.78 t/s, tg_3s = 60.54 t/s
|
||
|
|
3.45.621.483 I slot print_timing: id 3 | task 12016 | n_decoded = 548, tg = 60.74 t/s, tg_3s = 60.65 t/s
|
||
|
|
3.48.631.154 I slot print_timing: id 3 | task 12016 | n_decoded = 731, tg = 60.76 t/s, tg_3s = 60.80 t/s
|
||
|
|
3.51.634.764 I slot print_timing: id 3 | task 12016 | n_decoded = 913, tg = 60.72 t/s, tg_3s = 60.59 t/s
|
||
|
|
3.54.265.537 I slot print_timing: id 3 | task 12016 | prompt eval time = 165.68 ms / 40 tokens ( 4.14 ms per token, 241.44 tokens per second)
|
||
|
|
3.54.265.540 I slot print_timing: id 3 | task 12016 | eval time = 17666.20 ms / 1071 tokens ( 16.50 ms per token, 60.62 tokens per second)
|
||
|
|
3.54.265.541 I slot print_timing: id 3 | task 12016 | total time = 17831.87 ms / 1111 tokens
|
||
|
|
3.54.265.542 I slot print_timing: id 3 | task 12016 | graphs reused = 12953
|
||
|
|
3.54.265.581 I slot release: id 3 | task 12016 | stop processing: n_tokens = 1287, truncated = 0
|
||
|
|
3.54.265.595 I srv update_slots: all slots are idle
|
||
|
|
3.54.271.106 I srv operator(): Chat format: peg-native
|
||
|
|
3.54.273.144 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.823 (> 0.100 thold), f_keep = 0.141
|
||
|
|
3.54.273.146 I srv get_availabl: updating prompt cache
|
||
|
|
3.54.273.250 W srv prompt_save: - saving prompt with length 1287, total state size = 123.201 MiB (draft: 0.000 MiB)
|
||
|
|
3.54.373.535 I srv load: - looking for better prompt, base f_keep = 0.141, sim = 0.823
|
||
|
|
3.54.373.548 I srv update: - cache state: 16 prompts, 1774.433 MiB (limits: 8192.000 MiB, 16384 tokens, 75916 est)
|
||
|
|
3.54.373.549 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
3.54.373.550 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
3.54.373.550 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
3.54.373.551 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
3.54.373.551 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
3.54.373.552 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
3.54.373.552 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
3.54.373.553 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
3.54.373.554 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
3.54.373.554 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
3.54.373.555 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
3.54.373.555 I srv update: - prompt 0x631d8a9a4350: 835 tokens, checkpoints: 1, 92.451 MiB
|
||
|
|
3.54.373.556 I srv update: - prompt 0x631d83b601d0: 714 tokens, checkpoints: 1, 80.868 MiB
|
||
|
|
3.54.373.556 I srv update: - prompt 0x631d839b0a40: 712 tokens, checkpoints: 1, 80.677 MiB
|
||
|
|
3.54.373.557 I srv update: - prompt 0x631d839fead0: 1099 tokens, checkpoints: 1, 117.723 MiB
|
||
|
|
3.54.373.557 I srv update: - prompt 0x631d83756070: 1287 tokens, checkpoints: 1, 135.719 MiB
|
||
|
|
3.54.373.559 I srv get_availabl: prompt cache update took 100.41 ms
|
||
|
|
3.54.373.607 I slot launch_slot_: id 3 | task 13090 | processing task, is_child = 0
|
||
|
|
3.54.373.608 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.54.373.609 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.54.373.640 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.54.373.641 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.54.373.666 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
3.54.373.666 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
3.54.373.699 I slot update_slots: id 3 | task 13090 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
3.54.379.462 W slot update_slots: id 3 | task 13090 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
3.57.548.226 I slot print_timing: id 3 | task 13090 | n_decoded = 186, tg = 61.84 t/s, tg_3s = 61.84 t/s
|
||
|
|
4.00.551.250 I slot print_timing: id 3 | task 13090 | n_decoded = 369, tg = 61.39 t/s, tg_3s = 60.94 t/s
|
||
|
|
4.03.551.439 I slot print_timing: id 3 | task 13090 | n_decoded = 552, tg = 61.26 t/s, tg_3s = 61.00 t/s
|
||
|
|
4.06.551.773 I slot print_timing: id 3 | task 13090 | n_decoded = 733, tg = 61.03 t/s, tg_3s = 60.33 t/s
|
||
|
|
4.09.552.428 I slot print_timing: id 3 | task 13090 | n_decoded = 913, tg = 60.82 t/s, tg_3s = 59.99 t/s
|
||
|
|
4.12.555.377 I slot print_timing: id 3 | task 13090 | n_decoded = 1092, tg = 60.62 t/s, tg_3s = 59.61 t/s
|
||
|
|
4.14.068.410 I slot print_timing: id 3 | task 13090 | prompt eval time = 166.99 ms / 43 tokens ( 3.88 ms per token, 257.50 tokens per second)
|
||
|
|
4.14.068.415 I slot print_timing: id 3 | task 13090 | eval time = 19527.72 ms / 1181 tokens ( 16.53 ms per token, 60.48 tokens per second)
|
||
|
|
4.14.068.416 I slot print_timing: id 3 | task 13090 | total time = 19694.71 ms / 1224 tokens
|
||
|
|
4.14.068.417 I slot print_timing: id 3 | task 13090 | graphs reused = 14127
|
||
|
|
4.14.068.462 I slot release: id 3 | task 13090 | stop processing: n_tokens = 1400, truncated = 0
|
||
|
|
4.14.068.481 I srv update_slots: all slots are idle
|
||
|
|
4.14.074.264 I srv operator(): Chat format: peg-native
|
||
|
|
4.14.076.585 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.862 (> 0.100 thold), f_keep = 0.129
|
||
|
|
4.14.076.586 I srv get_availabl: updating prompt cache
|
||
|
|
4.14.076.701 W srv prompt_save: - saving prompt with length 1400, total state size = 134.018 MiB (draft: 0.000 MiB)
|
||
|
|
4.14.192.979 I srv load: - looking for better prompt, base f_keep = 0.129, sim = 0.862
|
||
|
|
4.14.192.994 I srv update: - cache state: 17 prompts, 1920.969 MiB (limits: 8192.000 MiB, 16384 tokens, 76095 est)
|
||
|
|
4.14.192.994 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
4.14.192.995 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
4.14.192.996 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
4.14.192.996 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
4.14.192.997 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
4.14.192.998 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
4.14.192.998 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
4.14.192.999 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
4.14.192.999 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
4.14.193.000 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
4.14.193.000 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
4.14.193.001 I srv update: - prompt 0x631d8a9a4350: 835 tokens, checkpoints: 1, 92.451 MiB
|
||
|
|
4.14.193.001 I srv update: - prompt 0x631d83b601d0: 714 tokens, checkpoints: 1, 80.868 MiB
|
||
|
|
4.14.193.002 I srv update: - prompt 0x631d839b0a40: 712 tokens, checkpoints: 1, 80.677 MiB
|
||
|
|
4.14.193.002 I srv update: - prompt 0x631d839fead0: 1099 tokens, checkpoints: 1, 117.723 MiB
|
||
|
|
4.14.193.003 I srv update: - prompt 0x631d83756070: 1287 tokens, checkpoints: 1, 135.719 MiB
|
||
|
|
4.14.193.003 I srv update: - prompt 0x631d8376e200: 1400 tokens, checkpoints: 1, 146.536 MiB
|
||
|
|
4.14.193.005 I srv get_availabl: prompt cache update took 116.42 ms
|
||
|
|
4.14.193.051 I slot launch_slot_: id 3 | task 14274 | processing task, is_child = 0
|
||
|
|
4.14.193.052 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
4.14.193.053 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
4.14.193.082 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
4.14.193.082 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
4.14.193.108 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
4.14.193.108 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
4.14.193.142 I slot update_slots: id 3 | task 14274 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
4.14.198.940 W slot update_slots: id 3 | task 14274 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
4.17.359.397 I slot print_timing: id 3 | task 14274 | n_decoded = 184, tg = 61.27 t/s, tg_3s = 61.27 t/s
|
||
|
|
4.20.374.625 I slot print_timing: id 3 | task 14274 | n_decoded = 367, tg = 60.98 t/s, tg_3s = 60.69 t/s
|
||
|
|
4.23.384.307 I slot print_timing: id 3 | task 14274 | n_decoded = 549, tg = 60.81 t/s, tg_3s = 60.47 t/s
|
||
|
|
4.26.394.367 I slot print_timing: id 3 | task 14274 | n_decoded = 728, tg = 60.47 t/s, tg_3s = 59.47 t/s
|
||
|
|
4.29.407.383 I slot print_timing: id 3 | task 14274 | n_decoded = 906, tg = 60.19 t/s, tg_3s = 59.08 t/s
|
||
|
|
4.32.339.455 I slot print_timing: id 3 | task 14274 | prompt eval time = 162.96 ms / 33 tokens ( 4.94 ms per token, 202.51 tokens per second)
|
||
|
|
4.32.339.458 I slot print_timing: id 3 | task 14274 | eval time = 17983.35 ms / 1079 tokens ( 16.67 ms per token, 60.00 tokens per second)
|
||
|
|
4.32.339.459 I slot print_timing: id 3 | task 14274 | total time = 18146.31 ms / 1112 tokens
|
||
|
|
4.32.339.459 I slot print_timing: id 3 | task 14274 | graphs reused = 15199
|
||
|
|
4.32.339.504 I slot release: id 3 | task 14274 | stop processing: n_tokens = 1288, truncated = 0
|
||
|
|
4.32.339.516 I srv update_slots: all slots are idle
|
||
|
|
4.32.345.187 I srv operator(): Chat format: peg-native
|
||
|
|
4.32.347.221 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.900 (> 0.100 thold), f_keep = 0.141
|
||
|
|
4.32.347.224 I srv get_availabl: updating prompt cache
|
||
|
|
4.32.347.352 W srv prompt_save: - saving prompt with length 1288, total state size = 123.296 MiB (draft: 0.000 MiB)
|
||
|
|
4.32.453.150 I srv load: - looking for better prompt, base f_keep = 0.141, sim = 0.900
|
||
|
|
4.32.453.164 I srv update: - cache state: 18 prompts, 2056.784 MiB (limits: 8192.000 MiB, 16384 tokens, 76201 est)
|
||
|
|
4.32.453.165 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
4.32.453.166 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
4.32.453.166 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
4.32.453.167 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
4.32.453.167 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
4.32.453.168 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
4.32.453.168 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
4.32.453.169 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
4.32.453.170 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
4.32.453.170 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
4.32.453.171 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
4.32.453.171 I srv update: - prompt 0x631d8a9a4350: 835 tokens, checkpoints: 1, 92.451 MiB
|
||
|
|
4.32.453.172 I srv update: - prompt 0x631d83b601d0: 714 tokens, checkpoints: 1, 80.868 MiB
|
||
|
|
4.32.453.172 I srv update: - prompt 0x631d839b0a40: 712 tokens, checkpoints: 1, 80.677 MiB
|
||
|
|
4.32.453.173 I srv update: - prompt 0x631d839fead0: 1099 tokens, checkpoints: 1, 117.723 MiB
|
||
|
|
4.32.453.173 I srv update: - prompt 0x631d83756070: 1287 tokens, checkpoints: 1, 135.719 MiB
|
||
|
|
4.32.453.174 I srv update: - prompt 0x631d8376e200: 1400 tokens, checkpoints: 1, 146.536 MiB
|
||
|
|
4.32.453.174 I srv update: - prompt 0x631d8a98f960: 1288 tokens, checkpoints: 1, 135.815 MiB
|
||
|
|
4.32.453.175 I srv get_availabl: prompt cache update took 105.95 ms
|
||
|
|
4.32.453.217 I slot launch_slot_: id 3 | task 15356 | processing task, is_child = 0
|
||
|
|
4.32.453.218 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
4.32.453.218 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
4.32.453.241 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
4.32.453.241 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
4.32.453.259 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
4.32.453.259 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
4.32.453.285 I slot update_slots: id 3 | task 15356 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
4.32.459.589 W slot update_slots: id 3 | task 15356 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
4.35.598.431 I slot print_timing: id 3 | task 15356 | n_decoded = 188, tg = 62.45 t/s, tg_3s = 62.45 t/s
|
||
|
|
4.38.613.175 I slot print_timing: id 3 | task 15356 | n_decoded = 375, tg = 62.24 t/s, tg_3s = 62.03 t/s
|
||
|
|
4.41.624.844 I slot print_timing: id 3 | task 15356 | n_decoded = 561, tg = 62.08 t/s, tg_3s = 61.76 t/s
|
||
|
|
4.44.634.258 I slot print_timing: id 3 | task 15356 | n_decoded = 744, tg = 61.76 t/s, tg_3s = 60.81 t/s
|
||
|
|
4.47.649.113 I slot print_timing: id 3 | task 15356 | n_decoded = 926, tg = 61.48 t/s, tg_3s = 60.37 t/s
|
||
|
|
4.50.665.121 I slot print_timing: id 3 | task 15356 | n_decoded = 1108, tg = 61.29 t/s, tg_3s = 60.34 t/s
|
||
|
|
4.53.667.174 I slot print_timing: id 3 | task 15356 | n_decoded = 1288, tg = 61.10 t/s, tg_3s = 59.96 t/s
|
||
|
|
4.56.672.229 I slot print_timing: id 3 | task 15356 | n_decoded = 1466, tg = 60.87 t/s, tg_3s = 59.23 t/s
|
||
|
|
4.59.673.467 I slot print_timing: id 3 | task 15356 | n_decoded = 1642, tg = 60.62 t/s, tg_3s = 58.64 t/s
|
||
|
|
5.02.682.614 I slot print_timing: id 3 | task 15356 | n_decoded = 1819, tg = 60.44 t/s, tg_3s = 58.82 t/s
|
||
|
|
5.05.682.798 I slot print_timing: id 3 | task 15356 | n_decoded = 1995, tg = 60.28 t/s, tg_3s = 58.66 t/s
|
||
|
|
5.08.698.731 I slot print_timing: id 3 | task 15356 | n_decoded = 2171, tg = 60.12 t/s, tg_3s = 58.36 t/s
|
||
|
|
5.11.701.384 I slot print_timing: id 3 | task 15356 | n_decoded = 2345, tg = 59.95 t/s, tg_3s = 57.95 t/s
|
||
|
|
5.14.704.438 I slot print_timing: id 3 | task 15356 | n_decoded = 2517, tg = 59.76 t/s, tg_3s = 57.28 t/s
|
||
|
|
5.17.719.514 I slot print_timing: id 3 | task 15356 | n_decoded = 2688, tg = 59.56 t/s, tg_3s = 56.71 t/s
|
||
|
|
5.20.730.140 I slot print_timing: id 3 | task 15356 | n_decoded = 2859, tg = 59.39 t/s, tg_3s = 56.80 t/s
|
||
|
|
5.23.737.630 I slot print_timing: id 3 | task 15356 | n_decoded = 3029, tg = 59.22 t/s, tg_3s = 56.53 t/s
|
||
|
|
5.26.752.978 I slot print_timing: id 3 | task 15356 | n_decoded = 3200, tg = 59.08 t/s, tg_3s = 56.71 t/s
|
||
|
|
5.29.759.098 I slot print_timing: id 3 | task 15356 | n_decoded = 3371, tg = 58.96 t/s, tg_3s = 56.88 t/s
|
||
|
|
5.32.760.075 I slot print_timing: id 3 | task 15356 | n_decoded = 3542, tg = 58.86 t/s, tg_3s = 56.98 t/s
|
||
|
|
5.35.764.724 I slot print_timing: id 3 | task 15356 | n_decoded = 3713, tg = 58.77 t/s, tg_3s = 56.91 t/s
|
||
|
|
5.38.779.710 I slot print_timing: id 3 | task 15356 | n_decoded = 3885, tg = 58.69 t/s, tg_3s = 57.05 t/s
|
||
|
|
5.41.794.970 I slot print_timing: id 3 | task 15356 | n_decoded = 4056, tg = 58.61 t/s, tg_3s = 56.71 t/s
|
||
|
|
5.42.500.678 I slot print_timing: id 3 | task 15356 | prompt eval time = 134.93 ms / 24 tokens ( 5.62 ms per token, 177.87 tokens per second)
|
||
|
|
5.42.500.682 I slot print_timing: id 3 | task 15356 | eval time = 69912.46 ms / 4096 tokens ( 17.07 ms per token, 58.59 tokens per second)
|
||
|
|
5.42.500.683 I slot print_timing: id 3 | task 15356 | total time = 70047.39 ms / 4120 tokens
|
||
|
|
5.42.500.683 I slot print_timing: id 3 | task 15356 | graphs reused = 19277
|
||
|
|
5.42.500.722 I slot release: id 3 | task 15356 | stop processing: n_tokens = 4296, truncated = 0
|
||
|
|
5.42.500.750 I srv update_slots: all slots are idle
|
||
|
|
5.42.506.894 I srv operator(): Chat format: peg-native
|
||
|
|
5.42.508.954 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.830 (> 0.100 thold), f_keep = 0.042
|
||
|
|
5.42.508.956 I srv get_availabl: updating prompt cache
|
||
|
|
5.42.509.230 W srv prompt_save: - saving prompt with length 4296, total state size = 397.175 MiB (draft: 0.000 MiB)
|
||
|
|
5.42.799.254 I srv load: - looking for better prompt, base f_keep = 0.042, sim = 0.830
|
||
|
|
5.42.799.273 I srv update: - cache state: 19 prompts, 2466.478 MiB (limits: 8192.000 MiB, 16384 tokens, 77812 est)
|
||
|
|
5.42.799.274 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
5.42.799.275 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
5.42.799.276 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
5.42.799.277 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
5.42.799.278 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
5.42.799.278 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
5.42.799.279 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
5.42.799.280 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
5.42.799.281 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
5.42.799.281 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
5.42.799.282 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
5.42.799.283 I srv update: - prompt 0x631d8a9a4350: 835 tokens, checkpoints: 1, 92.451 MiB
|
||
|
|
5.42.799.283 I srv update: - prompt 0x631d83b601d0: 714 tokens, checkpoints: 1, 80.868 MiB
|
||
|
|
5.42.799.284 I srv update: - prompt 0x631d839b0a40: 712 tokens, checkpoints: 1, 80.677 MiB
|
||
|
|
5.42.799.285 I srv update: - prompt 0x631d839fead0: 1099 tokens, checkpoints: 1, 117.723 MiB
|
||
|
|
5.42.799.285 I srv update: - prompt 0x631d83756070: 1287 tokens, checkpoints: 1, 135.719 MiB
|
||
|
|
5.42.799.286 I srv update: - prompt 0x631d8376e200: 1400 tokens, checkpoints: 1, 146.536 MiB
|
||
|
|
5.42.799.286 I srv update: - prompt 0x631d8a98f960: 1288 tokens, checkpoints: 1, 135.815 MiB
|
||
|
|
5.42.799.287 I srv update: - prompt 0x631d839fcc10: 4296 tokens, checkpoints: 1, 409.694 MiB
|
||
|
|
5.42.799.288 I srv get_availabl: prompt cache update took 290.33 ms
|
||
|
|
5.42.799.337 I slot launch_slot_: id 3 | task 19455 | processing task, is_child = 0
|
||
|
|
5.42.799.338 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
5.42.799.339 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
5.42.799.392 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
5.42.799.392 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
5.42.799.431 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
5.42.799.431 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
5.42.799.470 I slot update_slots: id 3 | task 19455 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
5.42.806.292 W slot update_slots: id 3 | task 19455 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
5.45.980.505 I slot print_timing: id 3 | task 19455 | n_decoded = 185, tg = 61.43 t/s, tg_3s = 61.43 t/s
|
||
|
|
5.48.981.317 I slot print_timing: id 3 | task 19455 | n_decoded = 368, tg = 61.21 t/s, tg_3s = 60.98 t/s
|
||
|
|
5.51.985.052 I slot print_timing: id 3 | task 19455 | n_decoded = 551, tg = 61.11 t/s, tg_3s = 60.92 t/s
|
||
|
|
5.53.008.495 I slot print_timing: id 3 | task 19455 | prompt eval time = 169.62 ms / 41 tokens ( 4.14 ms per token, 241.71 tokens per second)
|
||
|
|
5.53.008.499 I slot print_timing: id 3 | task 19455 | eval time = 10039.40 ms / 612 tokens ( 16.40 ms per token, 60.96 tokens per second)
|
||
|
|
5.53.008.500 I slot print_timing: id 3 | task 19455 | total time = 10209.02 ms / 653 tokens
|
||
|
|
5.53.008.500 I slot print_timing: id 3 | task 19455 | graphs reused = 19884
|
||
|
|
5.53.008.543 I slot release: id 3 | task 19455 | stop processing: n_tokens = 829, truncated = 0
|
||
|
|
5.53.008.554 I srv update_slots: all slots are idle
|
||
|
|
5.53.014.107 I srv operator(): Chat format: peg-native
|
||
|
|
5.53.016.164 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.866 (> 0.100 thold), f_keep = 0.218
|
||
|
|
5.53.016.165 I srv get_availabl: updating prompt cache
|
||
|
|
5.53.016.249 W srv prompt_save: - saving prompt with length 829, total state size = 79.358 MiB (draft: 0.000 MiB)
|
||
|
|
5.53.081.909 I srv load: - looking for better prompt, base f_keep = 0.218, sim = 0.866
|
||
|
|
5.53.081.930 I srv update: - cache state: 20 prompts, 2558.354 MiB (limits: 8192.000 MiB, 16384 tokens, 77672 est)
|
||
|
|
5.53.081.931 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
5.53.081.932 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
5.53.081.933 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
5.53.081.933 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
5.53.081.934 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
5.53.081.934 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
5.53.081.935 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
5.53.081.936 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
5.53.081.937 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
5.53.081.937 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
5.53.081.938 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
5.53.081.938 I srv update: - prompt 0x631d8a9a4350: 835 tokens, checkpoints: 1, 92.451 MiB
|
||
|
|
5.53.081.939 I srv update: - prompt 0x631d83b601d0: 714 tokens, checkpoints: 1, 80.868 MiB
|
||
|
|
5.53.081.939 I srv update: - prompt 0x631d839b0a40: 712 tokens, checkpoints: 1, 80.677 MiB
|
||
|
|
5.53.081.940 I srv update: - prompt 0x631d839fead0: 1099 tokens, checkpoints: 1, 117.723 MiB
|
||
|
|
5.53.081.940 I srv update: - prompt 0x631d83756070: 1287 tokens, checkpoints: 1, 135.719 MiB
|
||
|
|
5.53.081.941 I srv update: - prompt 0x631d8376e200: 1400 tokens, checkpoints: 1, 146.536 MiB
|
||
|
|
5.53.081.941 I srv update: - prompt 0x631d8a98f960: 1288 tokens, checkpoints: 1, 135.815 MiB
|
||
|
|
5.53.081.942 I srv update: - prompt 0x631d839fcc10: 4296 tokens, checkpoints: 1, 409.694 MiB
|
||
|
|
5.53.081.942 I srv update: - prompt 0x631d83cb2e20: 829 tokens, checkpoints: 1, 91.877 MiB
|
||
|
|
5.53.081.943 I srv get_availabl: prompt cache update took 65.78 ms
|
||
|
|
5.53.082.014 I slot launch_slot_: id 3 | task 20070 | processing task, is_child = 0
|
||
|
|
5.53.082.014 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
5.53.082.015 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
5.53.082.042 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
5.53.082.042 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
5.53.082.059 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
5.53.082.059 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
5.53.082.085 I slot update_slots: id 3 | task 20070 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
5.53.088.077 W slot update_slots: id 3 | task 20070 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
5.56.250.284 I slot print_timing: id 3 | task 20070 | n_decoded = 184, tg = 61.26 t/s, tg_3s = 61.26 t/s
|
||
|
|
5.59.254.444 I slot print_timing: id 3 | task 20070 | n_decoded = 366, tg = 60.92 t/s, tg_3s = 60.58 t/s
|
||
|
|
6.02.257.423 I slot print_timing: id 3 | task 20070 | n_decoded = 548, tg = 60.82 t/s, tg_3s = 60.61 t/s
|
||
|
|
6.05.272.170 I slot print_timing: id 3 | task 20070 | n_decoded = 729, tg = 60.62 t/s, tg_3s = 60.04 t/s
|
||
|
|
6.08.286.291 I slot print_timing: id 3 | task 20070 | n_decoded = 910, tg = 60.51 t/s, tg_3s = 60.05 t/s
|
||
|
|
6.11.287.538 I slot print_timing: id 3 | task 20070 | n_decoded = 1088, tg = 60.31 t/s, tg_3s = 59.31 t/s
|
||
|
|
6.11.406.601 I slot print_timing: id 3 | task 20070 | prompt eval time = 164.60 ms / 32 tokens ( 5.14 ms per token, 194.41 tokens per second)
|
||
|
|
6.11.406.604 I slot print_timing: id 3 | task 20070 | eval time = 18159.91 ms / 1095 tokens ( 16.58 ms per token, 60.30 tokens per second)
|
||
|
|
6.11.406.604 I slot print_timing: id 3 | task 20070 | total time = 18324.51 ms / 1127 tokens
|
||
|
|
6.11.406.605 I slot print_timing: id 3 | task 20070 | graphs reused = 20972
|
||
|
|
6.11.406.657 I slot release: id 3 | task 20070 | stop processing: n_tokens = 1303, truncated = 0
|
||
|
|
6.11.406.672 I srv update_slots: all slots are idle
|
||
|
|
6.11.412.424 I srv operator(): Chat format: peg-native
|
||
|
|
6.11.414.466 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.842 (> 0.100 thold), f_keep = 0.139
|
||
|
|
6.11.414.468 I srv get_availabl: updating prompt cache
|
||
|
|
6.11.414.581 W srv prompt_save: - saving prompt with length 1303, total state size = 124.732 MiB (draft: 0.000 MiB)
|
||
|
|
6.11.518.292 I srv load: - looking for better prompt, base f_keep = 0.139, sim = 0.842
|
||
|
|
6.11.518.309 I srv update: - cache state: 21 prompts, 2695.605 MiB (limits: 8192.000 MiB, 16384 tokens, 77677 est)
|
||
|
|
6.11.518.310 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
6.11.518.311 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
6.11.518.311 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
6.11.518.312 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
6.11.518.312 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
6.11.518.313 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
6.11.518.314 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
6.11.518.314 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
6.11.518.315 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
6.11.518.315 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
6.11.518.316 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
6.11.518.317 I srv update: - prompt 0x631d8a9a4350: 835 tokens, checkpoints: 1, 92.451 MiB
|
||
|
|
6.11.518.317 I srv update: - prompt 0x631d83b601d0: 714 tokens, checkpoints: 1, 80.868 MiB
|
||
|
|
6.11.518.318 I srv update: - prompt 0x631d839b0a40: 712 tokens, checkpoints: 1, 80.677 MiB
|
||
|
|
6.11.518.318 I srv update: - prompt 0x631d839fead0: 1099 tokens, checkpoints: 1, 117.723 MiB
|
||
|
|
6.11.518.319 I srv update: - prompt 0x631d83756070: 1287 tokens, checkpoints: 1, 135.719 MiB
|
||
|
|
6.11.518.319 I srv update: - prompt 0x631d8376e200: 1400 tokens, checkpoints: 1, 146.536 MiB
|
||
|
|
6.11.518.320 I srv update: - prompt 0x631d8a98f960: 1288 tokens, checkpoints: 1, 135.815 MiB
|
||
|
|
6.11.518.320 I srv update: - prompt 0x631d839fcc10: 4296 tokens, checkpoints: 1, 409.694 MiB
|
||
|
|
6.11.518.320 I srv update: - prompt 0x631d83cb2e20: 829 tokens, checkpoints: 1, 91.877 MiB
|
||
|
|
6.11.518.321 I srv update: - prompt 0x631d8acf9680: 1303 tokens, checkpoints: 1, 137.251 MiB
|
||
|
|
6.11.518.322 I srv get_availabl: prompt cache update took 103.85 ms
|
||
|
|
6.11.518.367 I slot launch_slot_: id 3 | task 21168 | processing task, is_child = 0
|
||
|
|
6.11.518.368 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
6.11.518.369 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
6.11.518.407 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
6.11.518.408 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
6.11.518.433 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
6.11.518.433 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
6.11.518.465 I slot update_slots: id 3 | task 21168 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
6.11.524.228 W slot update_slots: id 3 | task 21168 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
6.14.693.147 I slot print_timing: id 3 | task 21168 | n_decoded = 188, tg = 62.50 t/s, tg_3s = 62.50 t/s
|
||
|
|
6.17.693.245 I slot print_timing: id 3 | task 21168 | n_decoded = 375, tg = 62.42 t/s, tg_3s = 62.33 t/s
|
||
|
|
6.20.705.276 I slot print_timing: id 3 | task 21168 | n_decoded = 562, tg = 62.31 t/s, tg_3s = 62.08 t/s
|
||
|
|
6.23.720.903 I slot print_timing: id 3 | task 21168 | n_decoded = 746, tg = 61.98 t/s, tg_3s = 61.02 t/s
|
||
|
|
6.26.730.092 I slot print_timing: id 3 | task 21168 | n_decoded = 928, tg = 61.68 t/s, tg_3s = 60.48 t/s
|
||
|
|
6.29.736.954 I slot print_timing: id 3 | task 21168 | n_decoded = 1109, tg = 61.43 t/s, tg_3s = 60.20 t/s
|
||
|
|
6.32.523.240 I slot print_timing: id 3 | task 21168 | prompt eval time = 166.84 ms / 38 tokens ( 4.39 ms per token, 227.76 tokens per second)
|
||
|
|
6.32.523.243 I slot print_timing: id 3 | task 21168 | eval time = 20837.92 ms / 1276 tokens ( 16.33 ms per token, 61.23 tokens per second)
|
||
|
|
6.32.523.244 I slot print_timing: id 3 | task 21168 | total time = 21004.77 ms / 1314 tokens
|
||
|
|
6.32.523.245 I slot print_timing: id 3 | task 21168 | graphs reused = 22241
|
||
|
|
6.32.523.290 I slot release: id 3 | task 21168 | stop processing: n_tokens = 1490, truncated = 0
|
||
|
|
6.32.523.305 I srv update_slots: all slots are idle
|
||
|
|
6.32.529.106 I srv operator(): Chat format: peg-native
|
||
|
|
6.32.531.172 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.808 (> 0.100 thold), f_keep = 0.121
|
||
|
|
6.32.531.174 I srv get_availabl: updating prompt cache
|
||
|
|
6.32.531.293 W srv prompt_save: - saving prompt with length 1490, total state size = 142.633 MiB (draft: 0.000 MiB)
|
||
|
|
6.32.642.350 I srv load: - looking for better prompt, base f_keep = 0.121, sim = 0.808
|
||
|
|
6.32.642.367 I srv update: - cache state: 22 prompts, 2850.757 MiB (limits: 8192.000 MiB, 16384 tokens, 77731 est)
|
||
|
|
6.32.642.368 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
6.32.642.369 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
6.32.642.370 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
6.32.642.370 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
6.32.642.371 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
6.32.642.371 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
6.32.642.372 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
6.32.642.372 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
6.32.642.373 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
6.32.642.374 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
6.32.642.374 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
6.32.642.375 I srv update: - prompt 0x631d8a9a4350: 835 tokens, checkpoints: 1, 92.451 MiB
|
||
|
|
6.32.642.375 I srv update: - prompt 0x631d83b601d0: 714 tokens, checkpoints: 1, 80.868 MiB
|
||
|
|
6.32.642.376 I srv update: - prompt 0x631d839b0a40: 712 tokens, checkpoints: 1, 80.677 MiB
|
||
|
|
6.32.642.376 I srv update: - prompt 0x631d839fead0: 1099 tokens, checkpoints: 1, 117.723 MiB
|
||
|
|
6.32.642.377 I srv update: - prompt 0x631d83756070: 1287 tokens, checkpoints: 1, 135.719 MiB
|
||
|
|
6.32.642.377 I srv update: - prompt 0x631d8376e200: 1400 tokens, checkpoints: 1, 146.536 MiB
|
||
|
|
6.32.642.378 I srv update: - prompt 0x631d8a98f960: 1288 tokens, checkpoints: 1, 135.815 MiB
|
||
|
|
6.32.642.378 I srv update: - prompt 0x631d839fcc10: 4296 tokens, checkpoints: 1, 409.694 MiB
|
||
|
|
6.32.642.379 I srv update: - prompt 0x631d83cb2e20: 829 tokens, checkpoints: 1, 91.877 MiB
|
||
|
|
6.32.642.379 I srv update: - prompt 0x631d8acf9680: 1303 tokens, checkpoints: 1, 137.251 MiB
|
||
|
|
6.32.642.380 I srv update: - prompt 0x631d8ae4a010: 1490 tokens, checkpoints: 1, 155.151 MiB
|
||
|
|
6.32.642.381 I srv get_availabl: prompt cache update took 111.21 ms
|
||
|
|
6.32.642.427 I slot launch_slot_: id 3 | task 22447 | processing task, is_child = 0
|
||
|
|
6.32.642.427 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
6.32.642.428 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
6.32.642.467 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
6.32.642.468 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
6.32.642.493 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
6.32.642.494 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
6.32.642.527 I slot update_slots: id 3 | task 22447 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
6.32.648.581 W slot update_slots: id 3 | task 22447 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
6.35.823.747 I slot print_timing: id 3 | task 22447 | n_decoded = 187, tg = 62.20 t/s, tg_3s = 62.20 t/s
|
||
|
|
6.38.834.270 I slot print_timing: id 3 | task 22447 | n_decoded = 372, tg = 61.83 t/s, tg_3s = 61.45 t/s
|
||
|
|
6.39.699.380 I slot print_timing: id 3 | task 22447 | prompt eval time = 175.03 ms / 47 tokens ( 3.72 ms per token, 268.53 tokens per second)
|
||
|
|
6.39.699.383 I slot print_timing: id 3 | task 22447 | eval time = 6881.82 ms / 425 tokens ( 16.19 ms per token, 61.76 tokens per second)
|
||
|
|
6.39.699.384 I slot print_timing: id 3 | task 22447 | total time = 7056.85 ms / 472 tokens
|
||
|
|
6.39.699.384 I slot print_timing: id 3 | task 22447 | graphs reused = 22662
|
||
|
|
6.39.699.435 I slot release: id 3 | task 22447 | stop processing: n_tokens = 648, truncated = 0
|
||
|
|
6.39.699.449 I srv update_slots: all slots are idle
|
||
|
|
6.39.704.877 I srv operator(): Chat format: peg-native
|
||
|
|
6.39.706.901 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.862 (> 0.100 thold), f_keep = 0.279
|
||
|
|
6.39.706.903 I srv get_availabl: updating prompt cache
|
||
|
|
6.39.706.985 W srv prompt_save: - saving prompt with length 648, total state size = 62.032 MiB (draft: 0.000 MiB)
|
||
|
|
6.39.760.442 I srv load: - looking for better prompt, base f_keep = 0.279, sim = 0.862
|
||
|
|
6.39.760.464 I srv update: - cache state: 23 prompts, 2925.307 MiB (limits: 8192.000 MiB, 16384 tokens, 77565 est)
|
||
|
|
6.39.760.464 I srv update: - prompt 0x631d8a96fd90: 1364 tokens, checkpoints: 1, 143.090 MiB
|
||
|
|
6.39.760.465 I srv update: - prompt 0x631d83b90070: 867 tokens, checkpoints: 1, 95.514 MiB
|
||
|
|
6.39.760.466 I srv update: - prompt 0x631d836f2830: 892 tokens, checkpoints: 1, 97.907 MiB
|
||
|
|
6.39.760.467 I srv update: - prompt 0x631d83c00df0: 1100 tokens, checkpoints: 1, 117.818 MiB
|
||
|
|
6.39.760.467 I srv update: - prompt 0x631d838b8960: 2097 tokens, checkpoints: 1, 213.257 MiB
|
||
|
|
6.39.760.468 I srv update: - prompt 0x631d7fa7fb70: 900 tokens, checkpoints: 1, 98.673 MiB
|
||
|
|
6.39.760.468 I srv update: - prompt 0x631d838b9910: 810 tokens, checkpoints: 1, 90.058 MiB
|
||
|
|
6.39.760.469 I srv update: - prompt 0x631d83768df0: 964 tokens, checkpoints: 1, 104.800 MiB
|
||
|
|
6.39.760.469 I srv update: - prompt 0x631d80f6f770: 776 tokens, checkpoints: 1, 86.803 MiB
|
||
|
|
6.39.760.470 I srv update: - prompt 0x631d8ae1f3b0: 1091 tokens, checkpoints: 1, 116.957 MiB
|
||
|
|
6.39.760.471 I srv update: - prompt 0x631d8ae06a00: 936 tokens, checkpoints: 1, 102.119 MiB
|
||
|
|
6.39.760.471 I srv update: - prompt 0x631d8a9a4350: 835 tokens, checkpoints: 1, 92.451 MiB
|
||
|
|
6.39.760.472 I srv update: - prompt 0x631d83b601d0: 714 tokens, checkpoints: 1, 80.868 MiB
|
||
|
|
6.39.760.472 I srv update: - prompt 0x631d839b0a40: 712 tokens, checkpoints: 1, 80.677 MiB
|
||
|
|
6.39.760.473 I srv update: - prompt 0x631d839fead0: 1099 tokens, checkpoints: 1, 117.723 MiB
|
||
|
|
6.39.760.473 I srv update: - prompt 0x631d83756070: 1287 tokens, checkpoints: 1, 135.719 MiB
|
||
|
|
6.39.760.474 I srv update: - prompt 0x631d8376e200: 1400 tokens, checkpoints: 1, 146.536 MiB
|
||
|
|
6.39.760.474 I srv update: - prompt 0x631d8a98f960: 1288 tokens, checkpoints: 1, 135.815 MiB
|
||
|
|
6.39.760.475 I srv update: - prompt 0x631d839fcc10: 4296 tokens, checkpoints: 1, 409.694 MiB
|
||
|
|
6.39.760.475 I srv update: - prompt 0x631d83cb2e20: 829 tokens, checkpoints: 1, 91.877 MiB
|
||
|
|
6.39.760.476 I srv update: - prompt 0x631d8acf9680: 1303 tokens, checkpoints: 1, 137.251 MiB
|
||
|
|
6.39.760.476 I srv update: - prompt 0x631d8ae4a010: 1490 tokens, checkpoints: 1, 155.151 MiB
|
||
|
|
6.39.760.477 I srv update: - prompt 0x631d8374ff90: 648 tokens, checkpoints: 1, 74.550 MiB
|
||
|
|
6.39.760.478 I srv get_availabl: prompt cache update took 53.57 ms
|
||
|
|
6.39.760.550 I slot launch_slot_: id 3 | task 22875 | processing task, is_child = 0
|
||
|
|
6.39.760.551 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
6.39.760.552 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
6.39.760.587 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
6.39.760.587 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
6.39.760.611 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
6.39.760.611 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
6.39.760.645 I slot update_slots: id 3 | task 22875 | Checking checkpoint with [0, 177] against 0...
|
||
|
|
6.39.766.637 W slot update_slots: id 3 | task 22875 | restored context checkpoint (pos_min = 0, pos_max = 177, n_tokens = 178, n_past = 177, size = 12.519 MiB)
|
||
|
|
6.42.930.304 I slot print_timing: id 3 | task 22875 | n_decoded = 186, tg = 61.82 t/s, tg_3s = 61.82 t/s
|
||
|
|
6.45.940.383 I slot print_timing: id 3 | task 22875 | n_decoded = 370, tg = 61.47 t/s, tg_3s = 61.13 t/s
|
||
|
|
6.48.947.952 I slot print_timing: id 3 | task 22875 | n_decoded = 554, tg = 61.38 t/s, tg_3s = 61.18 t/s
|
||
|
|
6.51.955.468 I slot print_timing: id 3 | task 22875 | n_decoded = 735, tg = 61.08 t/s, tg_3s = 60.18 t/s
|
||
|
|
6.54.961.519 I slot print_timing: id 3 | task 22875 | n_decoded = 915, tg = 60.84 t/s, tg_3s = 59.88 t/s
|
||
|
|
6.57.972.022 I slot print_timing: id 3 | task 22875 | prompt eval time = 160.90 ms / 33 tokens ( 4.88 ms per token, 205.10 tokens per second)
|
||
|
|
6.57.972.026 I slot print_timing: id 3 | task 22875 | eval time = 18050.48 ms / 1094 tokens ( 16.50 ms per token, 60.61 tokens per second)
|
||
|
|
6.57.972.026 I slot print_timing: id 3 | task 22875 | total time = 18211.37 ms / 1127 tokens
|
||
|
|
6.57.972.027 I slot print_timing: id 3 | task 22875 | graphs reused = 23749
|
||
|
|
6.57.972.073 I slot release: id 3 | task 22875 | stop processing: n_tokens = 1303, truncated = 0
|
||
|
|
6.57.972.085 I srv update_slots: all slots are idle
|
||
|
|
6.57.972.975 I srv operator(): operator(): cleaning up before exit...
|