910 lines
89 KiB
Plaintext
910 lines
89 KiB
Plaintext
0.00.115.917 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
|
|
0.00.115.921 I device_info:
|
|
0.00.115.941 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
|
0.00.115.945 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
|
0.00.115.953 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
|
|
0.00.116.015 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
|
0.00.116.021 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
|
|
0.00.116.240 I srv init: running without SSL
|
|
0.00.116.317 I srv init: using 23 threads for HTTP server
|
|
0.00.116.660 I srv start: binding port with default address family
|
|
0.00.117.869 I srv llama_server: loading model
|
|
0.00.117.878 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/Nemotron-3-Nano-30B-A3B-Q5_K_M.gguf'
|
|
0.00.117.938 I common_init_result: fitting params to device memory ...
|
|
0.00.117.939 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
|
|
0.15.192.545 W llama_context: n_ctx_seq (16384) < n_ctx_train (1048576) -- the full capacity of the model will not be utilized
|
|
0.15.306.657 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
|
0.15.493.949 I srv load_model: initializing slots, n_slots = 4
|
|
0.15.524.913 W srv load_model: speculative decoding will use checkpoints
|
|
0.15.524.933 W common_speculative_init: no implementations specified for speculative decoding
|
|
0.15.524.933 I slot load_model: id 0 | task -1 | new slot, n_ctx = 16384
|
|
0.15.524.938 I slot load_model: id 1 | task -1 | new slot, n_ctx = 16384
|
|
0.15.524.938 I slot load_model: id 2 | task -1 | new slot, n_ctx = 16384
|
|
0.15.524.938 I slot load_model: id 3 | task -1 | new slot, n_ctx = 16384
|
|
0.15.524.996 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
|
|
0.15.524.996 I srv load_model: use `--cache-ram 0` to disable the prompt cache
|
|
0.15.524.997 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
|
0.15.524.997 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
|
|
0.15.525.016 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
|
|
0.15.558.549 I init: chat template, example_format: '<|im_start|>system
|
|
You are a helpful assistant<|im_end|>
|
|
<|im_start|>user
|
|
Hello<|im_end|>
|
|
<|im_start|>assistant
|
|
<think></think>Hi there<|im_end|>
|
|
<|im_start|>user
|
|
How are you?<|im_end|>
|
|
<|im_start|>assistant
|
|
<think>
|
|
'
|
|
0.15.587.443 I srv init: init: chat template, thinking = 1
|
|
0.15.587.477 I srv llama_server: model loaded
|
|
0.15.587.480 I srv llama_server: server is listening on http://127.0.0.1:8090
|
|
0.15.587.485 I srv update_slots: all slots are idle
|
|
0.19.976.893 I srv operator(): Chat format: peg-native
|
|
0.19.977.186 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
|
|
0.19.977.189 I srv get_availabl: updating prompt cache
|
|
0.19.977.195 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
|
0.19.977.200 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 16384 tokens, 8589934592 est)
|
|
0.19.977.200 I srv get_availabl: prompt cache update took 0.01 ms
|
|
0.19.977.298 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
|
|
0.19.977.300 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.19.977.302 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.19.977.318 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.19.977.318 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.19.977.328 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.19.977.328 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.20.750.640 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 71, pos_max = 71, n_tokens = 72, size = 47.618 MiB)
|
|
0.23.911.752 I slot print_timing: id 3 | task 0 | n_decoded = 250, tg = 83.22 t/s, tg_3s = 83.22 t/s
|
|
0.26.916.820 I slot print_timing: id 3 | task 0 | n_decoded = 500, tg = 83.21 t/s, tg_3s = 83.19 t/s
|
|
0.29.921.857 I slot print_timing: id 3 | task 0 | n_decoded = 750, tg = 83.20 t/s, tg_3s = 83.19 t/s
|
|
0.32.926.900 I slot print_timing: id 3 | task 0 | n_decoded = 1000, tg = 83.20 t/s, tg_3s = 83.19 t/s
|
|
0.35.931.940 I slot print_timing: id 3 | task 0 | n_decoded = 1250, tg = 83.20 t/s, tg_3s = 83.19 t/s
|
|
0.38.612.981 I slot print_timing: id 3 | task 0 | prompt eval time = 930.40 ms / 101 tokens ( 9.21 ms per token, 108.56 tokens per second)
|
|
0.38.612.985 I slot print_timing: id 3 | task 0 | eval time = 17705.22 ms / 1473 tokens ( 12.02 ms per token, 83.20 tokens per second)
|
|
0.38.612.985 I slot print_timing: id 3 | task 0 | total time = 18635.62 ms / 1574 tokens
|
|
0.38.612.989 I slot print_timing: id 3 | task 0 | graphs reused = 1466
|
|
0.38.613.037 I slot release: id 3 | task 0 | stop processing: n_tokens = 1573, truncated = 0
|
|
0.38.613.048 I srv update_slots: all slots are idle
|
|
0.38.643.949 I srv operator(): Chat format: peg-native
|
|
0.38.644.197 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.781 (> 0.100 thold), f_keep = 0.048
|
|
0.38.644.199 I srv get_availabl: updating prompt cache
|
|
0.38.644.284 W srv prompt_save: - saving prompt with length 1573, total state size = 56.853 MiB (draft: 0.000 MiB)
|
|
0.38.747.389 I srv load: - looking for better prompt, base f_keep = 0.048, sim = 0.781
|
|
0.38.747.395 I srv update: - cache state: 1 prompts, 104.470 MiB (limits: 8192.000 MiB, 16384 tokens, 123346 est)
|
|
0.38.747.396 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
0.38.747.397 I srv get_availabl: prompt cache update took 103.20 ms
|
|
0.38.747.504 I slot launch_slot_: id 3 | task 1476 | processing task, is_child = 0
|
|
0.38.747.505 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.38.747.505 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.38.747.528 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.38.747.529 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.38.747.538 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.38.747.539 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.38.747.556 I slot update_slots: id 3 | task 1476 | Checking checkpoint with [71, 71] against 75...
|
|
0.38.759.612 W slot update_slots: id 3 | task 1476 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
0.41.900.952 I slot print_timing: id 3 | task 1476 | n_decoded = 251, tg = 83.38 t/s, tg_3s = 83.38 t/s
|
|
0.44.911.959 I slot print_timing: id 3 | task 1476 | n_decoded = 487, tg = 80.88 t/s, tg_3s = 78.38 t/s
|
|
0.47.920.042 I slot print_timing: id 3 | task 1476 | n_decoded = 737, tg = 81.62 t/s, tg_3s = 83.11 t/s
|
|
0.50.925.061 I slot print_timing: id 3 | task 1476 | n_decoded = 987, tg = 82.02 t/s, tg_3s = 83.19 t/s
|
|
0.52.773.060 I slot print_timing: id 3 | task 1476 | prompt eval time = 143.19 ms / 24 tokens ( 5.97 ms per token, 167.60 tokens per second)
|
|
0.52.773.064 I slot print_timing: id 3 | task 1476 | eval time = 13882.31 ms / 1141 tokens ( 12.17 ms per token, 82.19 tokens per second)
|
|
0.52.773.064 I slot print_timing: id 3 | task 1476 | total time = 14025.50 ms / 1165 tokens
|
|
0.52.773.065 I slot print_timing: id 3 | task 1476 | graphs reused = 2601
|
|
0.52.773.098 I slot release: id 3 | task 1476 | stop processing: n_tokens = 1236, truncated = 0
|
|
0.52.773.108 I srv update_slots: all slots are idle
|
|
0.52.803.596 I srv operator(): Chat format: peg-native
|
|
0.52.803.846 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.781 (> 0.100 thold), f_keep = 0.061
|
|
0.52.803.848 I srv get_availabl: updating prompt cache
|
|
0.52.803.913 W srv prompt_save: - saving prompt with length 1236, total state size = 54.874 MiB (draft: 0.000 MiB)
|
|
0.52.886.699 I srv load: - looking for better prompt, base f_keep = 0.061, sim = 0.781
|
|
0.52.886.705 I srv update: - cache state: 2 prompts, 206.962 MiB (limits: 8192.000 MiB, 16384 tokens, 111186 est)
|
|
0.52.886.706 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
0.52.886.707 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
0.52.886.708 I srv get_availabl: prompt cache update took 82.86 ms
|
|
0.52.886.801 I slot launch_slot_: id 3 | task 2619 | processing task, is_child = 0
|
|
0.52.886.801 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.52.886.802 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.52.886.816 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.52.886.816 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.52.886.825 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.52.886.825 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.52.886.843 I slot update_slots: id 3 | task 2619 | Checking checkpoint with [71, 71] against 75...
|
|
0.52.898.810 W slot update_slots: id 3 | task 2619 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
0.56.026.056 I slot print_timing: id 3 | task 2619 | n_decoded = 250, tg = 83.30 t/s, tg_3s = 83.30 t/s
|
|
0.58.254.201 I slot print_timing: id 3 | task 2619 | prompt eval time = 138.12 ms / 24 tokens ( 5.75 ms per token, 173.76 tokens per second)
|
|
0.58.254.205 I slot print_timing: id 3 | task 2619 | eval time = 5229.23 ms / 435 tokens ( 12.02 ms per token, 83.19 tokens per second)
|
|
0.58.254.205 I slot print_timing: id 3 | task 2619 | total time = 5367.35 ms / 459 tokens
|
|
0.58.254.206 I slot print_timing: id 3 | task 2619 | graphs reused = 3032
|
|
0.58.254.235 I slot release: id 3 | task 2619 | stop processing: n_tokens = 530, truncated = 0
|
|
0.58.254.246 I srv update_slots: all slots are idle
|
|
0.58.285.433 I srv operator(): Chat format: peg-native
|
|
0.58.285.698 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.142
|
|
0.58.285.700 I srv get_availabl: updating prompt cache
|
|
0.58.285.754 W srv prompt_save: - saving prompt with length 530, total state size = 50.729 MiB (draft: 0.000 MiB)
|
|
0.58.380.818 I srv load: - looking for better prompt, base f_keep = 0.142, sim = 0.789
|
|
0.58.380.825 I srv update: - cache state: 3 prompts, 305.310 MiB (limits: 8192.000 MiB, 16384 tokens, 89591 est)
|
|
0.58.380.826 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
0.58.380.827 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
0.58.380.827 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
0.58.380.828 I srv get_availabl: prompt cache update took 95.13 ms
|
|
0.58.380.918 I slot launch_slot_: id 3 | task 3056 | processing task, is_child = 0
|
|
0.58.380.924 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
0.58.380.925 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
0.58.380.937 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
0.58.380.937 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
0.58.380.958 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
0.58.380.958 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
0.58.380.987 I slot update_slots: id 3 | task 3056 | Checking checkpoint with [71, 71] against 75...
|
|
0.58.392.876 W slot update_slots: id 3 | task 3056 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
1.01.521.132 I slot print_timing: id 3 | task 3056 | n_decoded = 250, tg = 83.30 t/s, tg_3s = 83.30 t/s
|
|
1.04.525.950 I slot print_timing: id 3 | task 3056 | n_decoded = 498, tg = 82.92 t/s, tg_3s = 82.53 t/s
|
|
1.07.529.599 I slot print_timing: id 3 | task 3056 | n_decoded = 743, tg = 82.47 t/s, tg_3s = 81.57 t/s
|
|
1.10.190.186 I slot print_timing: id 3 | task 3056 | prompt eval time = 138.85 ms / 23 tokens ( 6.04 ms per token, 165.65 tokens per second)
|
|
1.10.190.190 I slot print_timing: id 3 | task 3056 | eval time = 11670.35 ms / 959 tokens ( 12.17 ms per token, 82.17 tokens per second)
|
|
1.10.190.191 I slot print_timing: id 3 | task 3056 | total time = 11809.19 ms / 982 tokens
|
|
1.10.190.191 I slot print_timing: id 3 | task 3056 | graphs reused = 3985
|
|
1.10.190.221 I slot release: id 3 | task 3056 | stop processing: n_tokens = 1053, truncated = 0
|
|
1.10.190.232 I srv update_slots: all slots are idle
|
|
1.10.220.694 I srv operator(): Chat format: peg-native
|
|
1.10.221.002 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.694 (> 0.100 thold), f_keep = 0.071
|
|
1.10.221.004 I srv get_availabl: updating prompt cache
|
|
1.10.221.055 W srv prompt_save: - saving prompt with length 1053, total state size = 53.800 MiB (draft: 0.000 MiB)
|
|
1.10.311.169 I srv load: - looking for better prompt, base f_keep = 0.071, sim = 0.694
|
|
1.10.311.178 I srv update: - cache state: 4 prompts, 406.727 MiB (limits: 8192.000 MiB, 16384 tokens, 88460 est)
|
|
1.10.311.179 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
1.10.311.180 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
1.10.311.180 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
1.10.311.181 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
1.10.311.182 I srv get_availabl: prompt cache update took 90.18 ms
|
|
1.10.311.280 I slot launch_slot_: id 3 | task 4017 | processing task, is_child = 0
|
|
1.10.311.283 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.10.311.283 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.10.311.298 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.10.311.298 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.10.311.307 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
1.10.311.308 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
1.10.311.325 I slot update_slots: id 3 | task 4017 | Checking checkpoint with [71, 71] against 75...
|
|
1.10.323.247 W slot update_slots: id 3 | task 4017 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
1.13.497.125 I slot print_timing: id 3 | task 4017 | n_decoded = 244, tg = 81.29 t/s, tg_3s = 81.29 t/s
|
|
1.16.498.931 I slot print_timing: id 3 | task 4017 | n_decoded = 490, tg = 81.62 t/s, tg_3s = 81.95 t/s
|
|
1.19.503.967 I slot print_timing: id 3 | task 4017 | n_decoded = 740, tg = 82.14 t/s, tg_3s = 83.19 t/s
|
|
1.22.509.011 I slot print_timing: id 3 | task 4017 | n_decoded = 990, tg = 82.41 t/s, tg_3s = 83.19 t/s
|
|
1.25.520.058 I slot print_timing: id 3 | task 4017 | n_decoded = 1240, tg = 82.53 t/s, tg_3s = 83.03 t/s
|
|
1.27.466.079 I slot print_timing: id 3 | task 4017 | prompt eval time = 184.10 ms / 36 tokens ( 5.11 ms per token, 195.55 tokens per second)
|
|
1.27.466.083 I slot print_timing: id 3 | task 4017 | eval time = 16970.65 ms / 1402 tokens ( 12.10 ms per token, 82.61 tokens per second)
|
|
1.27.466.084 I slot print_timing: id 3 | task 4017 | total time = 17154.75 ms / 1438 tokens
|
|
1.27.466.084 I slot print_timing: id 3 | task 4017 | graphs reused = 5380
|
|
1.27.466.116 I slot release: id 3 | task 4017 | stop processing: n_tokens = 1509, truncated = 0
|
|
1.27.466.131 I srv update_slots: all slots are idle
|
|
1.27.496.470 I srv operator(): Chat format: peg-native
|
|
1.27.496.729 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.773 (> 0.100 thold), f_keep = 0.050
|
|
1.27.496.732 I srv get_availabl: updating prompt cache
|
|
1.27.496.805 W srv prompt_save: - saving prompt with length 1509, total state size = 56.477 MiB (draft: 0.000 MiB)
|
|
1.27.580.059 I srv load: - looking for better prompt, base f_keep = 0.050, sim = 0.773
|
|
1.27.580.066 I srv update: - cache state: 5 prompts, 510.822 MiB (limits: 8192.000 MiB, 16384 tokens, 94633 est)
|
|
1.27.580.067 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
1.27.580.068 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
1.27.580.069 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
1.27.580.069 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
1.27.580.070 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
1.27.580.071 I srv get_availabl: prompt cache update took 83.34 ms
|
|
1.27.580.164 I slot launch_slot_: id 3 | task 5421 | processing task, is_child = 0
|
|
1.27.580.165 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.27.580.166 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.27.580.181 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.27.580.182 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.27.580.191 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
1.27.580.191 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
1.27.580.207 I slot update_slots: id 3 | task 5421 | Checking checkpoint with [71, 71] against 75...
|
|
1.27.592.071 W slot update_slots: id 3 | task 5421 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
1.30.730.029 I slot print_timing: id 3 | task 5421 | n_decoded = 251, tg = 83.38 t/s, tg_3s = 83.38 t/s
|
|
1.33.738.079 I slot print_timing: id 3 | task 5421 | n_decoded = 501, tg = 83.25 t/s, tg_3s = 83.11 t/s
|
|
1.36.743.140 I slot print_timing: id 3 | task 5421 | n_decoded = 751, tg = 83.23 t/s, tg_3s = 83.19 t/s
|
|
1.39.749.177 I slot print_timing: id 3 | task 5421 | n_decoded = 1001, tg = 83.21 t/s, tg_3s = 83.17 t/s
|
|
1.40.097.176 I slot print_timing: id 3 | task 5421 | prompt eval time = 139.55 ms / 25 tokens ( 5.58 ms per token, 179.14 tokens per second)
|
|
1.40.097.180 I slot print_timing: id 3 | task 5421 | eval time = 12377.41 ms / 1030 tokens ( 12.02 ms per token, 83.22 tokens per second)
|
|
1.40.097.181 I slot print_timing: id 3 | task 5421 | total time = 12516.97 ms / 1055 tokens
|
|
1.40.097.181 I slot print_timing: id 3 | task 5421 | graphs reused = 6404
|
|
1.40.097.213 I slot release: id 3 | task 5421 | stop processing: n_tokens = 1126, truncated = 0
|
|
1.40.097.224 I srv update_slots: all slots are idle
|
|
1.40.128.011 I srv operator(): Chat format: peg-native
|
|
1.40.128.338 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.817 (> 0.100 thold), f_keep = 0.067
|
|
1.40.128.341 I srv get_availabl: updating prompt cache
|
|
1.40.128.392 W srv prompt_save: - saving prompt with length 1126, total state size = 54.228 MiB (draft: 0.000 MiB)
|
|
1.40.210.279 I srv load: - looking for better prompt, base f_keep = 0.067, sim = 0.817
|
|
1.40.210.286 I srv update: - cache state: 6 prompts, 612.668 MiB (limits: 8192.000 MiB, 16384 tokens, 93958 est)
|
|
1.40.210.287 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
1.40.210.288 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
1.40.210.288 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
1.40.210.289 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
1.40.210.289 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
1.40.210.290 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
1.40.210.291 I srv get_availabl: prompt cache update took 81.95 ms
|
|
1.40.210.378 I slot launch_slot_: id 3 | task 6453 | processing task, is_child = 0
|
|
1.40.210.379 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.40.210.380 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.40.210.393 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.40.210.394 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.40.210.403 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
1.40.210.403 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
1.40.210.419 I slot update_slots: id 3 | task 6453 | Checking checkpoint with [71, 71] against 76...
|
|
1.40.222.432 W slot update_slots: id 3 | task 6453 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
1.43.356.141 I slot print_timing: id 3 | task 6453 | n_decoded = 251, tg = 83.38 t/s, tg_3s = 83.38 t/s
|
|
1.46.361.159 I slot print_timing: id 3 | task 6453 | n_decoded = 501, tg = 83.29 t/s, tg_3s = 83.19 t/s
|
|
1.48.933.192 I slot print_timing: id 3 | task 6453 | prompt eval time = 135.50 ms / 21 tokens ( 6.45 ms per token, 154.98 tokens per second)
|
|
1.48.933.195 I slot print_timing: id 3 | task 6453 | eval time = 8587.27 ms / 715 tokens ( 12.01 ms per token, 83.26 tokens per second)
|
|
1.48.933.196 I slot print_timing: id 3 | task 6453 | total time = 8722.77 ms / 736 tokens
|
|
1.48.933.196 I slot print_timing: id 3 | task 6453 | graphs reused = 7114
|
|
1.48.933.228 I slot release: id 3 | task 6453 | stop processing: n_tokens = 807, truncated = 0
|
|
1.48.933.238 I srv update_slots: all slots are idle
|
|
1.48.961.015 I srv operator(): Chat format: peg-native
|
|
1.48.961.246 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.800 (> 0.100 thold), f_keep = 0.094
|
|
1.48.961.246 I srv get_availabl: updating prompt cache
|
|
1.48.961.287 W srv prompt_save: - saving prompt with length 807, total state size = 52.356 MiB (draft: 0.000 MiB)
|
|
1.49.019.070 I srv load: - looking for better prompt, base f_keep = 0.094, sim = 0.800
|
|
1.49.019.078 I srv update: - cache state: 7 prompts, 712.641 MiB (limits: 8192.000 MiB, 16384 tokens, 90053 est)
|
|
1.49.019.079 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
1.49.019.080 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
1.49.019.080 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
1.49.019.081 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
1.49.019.081 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
1.49.019.082 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
1.49.019.082 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
1.49.019.083 I srv get_availabl: prompt cache update took 57.84 ms
|
|
1.49.019.177 I slot launch_slot_: id 3 | task 7170 | processing task, is_child = 0
|
|
1.49.019.177 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.49.019.178 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.49.019.189 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.49.019.190 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.49.019.198 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
1.49.019.198 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
1.49.019.215 I slot update_slots: id 3 | task 7170 | Checking checkpoint with [71, 71] against 76...
|
|
1.49.031.015 W slot update_slots: id 3 | task 7170 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
1.52.167.201 I slot print_timing: id 3 | task 7170 | n_decoded = 251, tg = 83.38 t/s, tg_3s = 83.38 t/s
|
|
1.55.174.079 I slot print_timing: id 3 | task 7170 | n_decoded = 501, tg = 83.26 t/s, tg_3s = 83.14 t/s
|
|
1.56.854.134 I slot print_timing: id 3 | task 7170 | prompt eval time = 137.82 ms / 23 tokens ( 5.99 ms per token, 166.89 tokens per second)
|
|
1.56.854.138 I slot print_timing: id 3 | task 7170 | eval time = 7697.10 ms / 641 tokens ( 12.01 ms per token, 83.28 tokens per second)
|
|
1.56.854.138 I slot print_timing: id 3 | task 7170 | total time = 7834.92 ms / 664 tokens
|
|
1.56.854.139 I slot print_timing: id 3 | task 7170 | graphs reused = 7751
|
|
1.56.854.169 I slot release: id 3 | task 7170 | stop processing: n_tokens = 735, truncated = 0
|
|
1.56.854.178 I srv update_slots: all slots are idle
|
|
1.56.884.587 I srv operator(): Chat format: peg-native
|
|
1.56.884.836 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.610 (> 0.100 thold), f_keep = 0.102
|
|
1.56.884.838 I srv get_availabl: updating prompt cache
|
|
1.56.884.882 W srv prompt_save: - saving prompt with length 735, total state size = 51.933 MiB (draft: 0.000 MiB)
|
|
1.56.946.838 I srv load: - looking for better prompt, base f_keep = 0.102, sim = 0.610
|
|
1.56.946.848 I srv update: - cache state: 8 prompts, 812.192 MiB (limits: 8192.000 MiB, 16384 tokens, 86429 est)
|
|
1.56.946.849 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
1.56.946.850 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
1.56.946.850 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
1.56.946.851 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
1.56.946.852 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
1.56.946.852 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
1.56.946.853 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
1.56.946.854 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
1.56.946.855 I srv get_availabl: prompt cache update took 62.02 ms
|
|
1.56.946.967 I slot launch_slot_: id 3 | task 7813 | processing task, is_child = 0
|
|
1.56.946.968 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
1.56.946.969 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
1.56.946.989 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
1.56.946.990 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
1.56.947.012 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
1.56.947.012 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
1.56.947.041 I slot update_slots: id 3 | task 7813 | Checking checkpoint with [71, 71] against 75...
|
|
1.56.958.921 W slot update_slots: id 3 | task 7813 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
2.00.214.153 I slot print_timing: id 3 | task 7813 | n_decoded = 251, tg = 83.35 t/s, tg_3s = 83.35 t/s
|
|
2.03.218.712 I slot print_timing: id 3 | task 7813 | n_decoded = 501, tg = 83.28 t/s, tg_3s = 83.21 t/s
|
|
2.03.342.231 I slot print_timing: id 3 | task 7813 | prompt eval time = 255.75 ms / 51 tokens ( 5.01 ms per token, 199.41 tokens per second)
|
|
2.03.342.235 I slot print_timing: id 3 | task 7813 | eval time = 6139.44 ms / 511 tokens ( 12.01 ms per token, 83.23 tokens per second)
|
|
2.03.342.235 I slot print_timing: id 3 | task 7813 | total time = 6395.19 ms / 562 tokens
|
|
2.03.342.236 I slot print_timing: id 3 | task 7813 | graphs reused = 8258
|
|
2.03.342.268 I slot release: id 3 | task 7813 | stop processing: n_tokens = 633, truncated = 0
|
|
2.03.342.280 I srv update_slots: all slots are idle
|
|
2.03.372.859 I srv operator(): Chat format: peg-native
|
|
2.03.373.110 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.531 (> 0.100 thold), f_keep = 0.120
|
|
2.03.373.113 I srv get_availabl: updating prompt cache
|
|
2.03.373.164 W srv prompt_save: - saving prompt with length 633, total state size = 51.334 MiB (draft: 0.000 MiB)
|
|
2.03.451.644 I srv load: - looking for better prompt, base f_keep = 0.120, sim = 0.531
|
|
2.03.451.653 I srv update: - cache state: 9 prompts, 911.144 MiB (limits: 8192.000 MiB, 16384 tokens, 82734 est)
|
|
2.03.451.654 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
2.03.451.654 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
2.03.451.655 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
2.03.451.655 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
2.03.451.656 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
2.03.451.656 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
2.03.451.657 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
2.03.451.657 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
2.03.451.658 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
2.03.451.659 I srv get_availabl: prompt cache update took 78.55 ms
|
|
2.03.451.742 I slot launch_slot_: id 3 | task 8326 | processing task, is_child = 0
|
|
2.03.451.743 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.03.451.744 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.03.451.755 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.03.451.755 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.03.451.764 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.03.451.764 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.03.451.780 I slot update_slots: id 3 | task 8326 | Checking checkpoint with [71, 71] against 76...
|
|
2.03.463.682 W slot update_slots: id 3 | task 8326 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
2.06.711.725 I slot print_timing: id 3 | task 8326 | n_decoded = 251, tg = 83.38 t/s, tg_3s = 83.38 t/s
|
|
2.09.524.739 I slot print_timing: id 3 | task 8326 | prompt eval time = 249.52 ms / 71 tokens ( 3.51 ms per token, 284.55 tokens per second)
|
|
2.09.524.742 I slot print_timing: id 3 | task 8326 | eval time = 5823.43 ms / 485 tokens ( 12.01 ms per token, 83.28 tokens per second)
|
|
2.09.524.743 I slot print_timing: id 3 | task 8326 | total time = 6072.95 ms / 556 tokens
|
|
2.09.524.744 I slot print_timing: id 3 | task 8326 | graphs reused = 8739
|
|
2.09.524.778 I slot release: id 3 | task 8326 | stop processing: n_tokens = 627, truncated = 0
|
|
2.09.524.787 I srv update_slots: all slots are idle
|
|
2.09.555.086 I srv operator(): Chat format: peg-native
|
|
2.09.555.336 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.636 (> 0.100 thold), f_keep = 0.120
|
|
2.09.555.338 I srv get_availabl: updating prompt cache
|
|
2.09.555.379 W srv prompt_save: - saving prompt with length 627, total state size = 51.299 MiB (draft: 0.000 MiB)
|
|
2.09.631.198 I srv load: - looking for better prompt, base f_keep = 0.120, sim = 0.636
|
|
2.09.631.209 I srv update: - cache state: 10 prompts, 1010.061 MiB (limits: 8192.000 MiB, 16384 tokens, 79717 est)
|
|
2.09.631.210 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
2.09.631.211 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
2.09.631.211 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
2.09.631.212 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
2.09.631.212 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
2.09.631.213 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
2.09.631.213 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
2.09.631.214 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
2.09.631.214 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
2.09.631.215 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
2.09.631.216 I srv get_availabl: prompt cache update took 75.88 ms
|
|
2.09.631.312 I slot launch_slot_: id 3 | task 8813 | processing task, is_child = 0
|
|
2.09.631.313 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.09.631.314 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.09.631.327 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.09.631.327 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.09.631.336 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.09.631.336 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.09.631.353 I slot update_slots: id 3 | task 8813 | Checking checkpoint with [71, 71] against 75...
|
|
2.09.643.184 W slot update_slots: id 3 | task 8813 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
2.12.895.786 I slot print_timing: id 3 | task 8813 | n_decoded = 251, tg = 83.40 t/s, tg_3s = 83.40 t/s
|
|
2.15.900.809 I slot print_timing: id 3 | task 8813 | n_decoded = 501, tg = 83.30 t/s, tg_3s = 83.19 t/s
|
|
2.18.413.878 I slot print_timing: id 3 | task 8813 | prompt eval time = 254.69 ms / 46 tokens ( 5.54 ms per token, 180.62 tokens per second)
|
|
2.18.413.882 I slot print_timing: id 3 | task 8813 | eval time = 8527.84 ms / 710 tokens ( 12.01 ms per token, 83.26 tokens per second)
|
|
2.18.413.883 I slot print_timing: id 3 | task 8813 | total time = 8782.52 ms / 756 tokens
|
|
2.18.413.883 I slot print_timing: id 3 | task 8813 | graphs reused = 9444
|
|
2.18.413.917 I slot release: id 3 | task 8813 | stop processing: n_tokens = 827, truncated = 0
|
|
2.18.413.927 I srv update_slots: all slots are idle
|
|
2.18.444.250 I srv operator(): Chat format: peg-native
|
|
2.18.444.504 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.528 (> 0.100 thold), f_keep = 0.091
|
|
2.18.444.507 I srv get_availabl: updating prompt cache
|
|
2.18.444.563 W srv prompt_save: - saving prompt with length 827, total state size = 52.473 MiB (draft: 0.000 MiB)
|
|
2.18.521.510 I srv load: - looking for better prompt, base f_keep = 0.091, sim = 0.528
|
|
2.18.521.524 I srv update: - cache state: 11 prompts, 1110.151 MiB (limits: 8192.000 MiB, 16384 tokens, 78632 est)
|
|
2.18.521.525 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
2.18.521.526 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
2.18.521.527 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
2.18.521.527 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
2.18.521.528 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
2.18.521.528 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
2.18.521.529 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
2.18.521.529 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
2.18.521.530 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
2.18.521.530 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
2.18.521.531 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
2.18.521.531 I srv get_availabl: prompt cache update took 77.02 ms
|
|
2.18.521.627 I slot launch_slot_: id 3 | task 9525 | processing task, is_child = 0
|
|
2.18.521.628 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.18.521.629 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.18.521.642 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.18.521.642 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.18.521.651 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.18.521.652 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.18.521.668 I slot update_slots: id 3 | task 9525 | Checking checkpoint with [71, 71] against 75...
|
|
2.18.533.645 W slot update_slots: id 3 | task 9525 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
2.21.780.871 I slot print_timing: id 3 | task 9525 | n_decoded = 250, tg = 83.12 t/s, tg_3s = 83.12 t/s
|
|
2.24.786.904 I slot print_timing: id 3 | task 9525 | n_decoded = 500, tg = 83.14 t/s, tg_3s = 83.17 t/s
|
|
2.24.882.900 I slot print_timing: id 3 | task 9525 | prompt eval time = 251.59 ms / 70 tokens ( 3.59 ms per token, 278.23 tokens per second)
|
|
2.24.882.903 I slot print_timing: id 3 | task 9525 | eval time = 6109.64 ms / 508 tokens ( 12.03 ms per token, 83.15 tokens per second)
|
|
2.24.882.904 I slot print_timing: id 3 | task 9525 | total time = 6361.23 ms / 578 tokens
|
|
2.24.882.904 I slot print_timing: id 3 | task 9525 | graphs reused = 9948
|
|
2.24.882.936 I slot release: id 3 | task 9525 | stop processing: n_tokens = 649, truncated = 0
|
|
2.24.882.946 I srv update_slots: all slots are idle
|
|
2.24.913.683 I srv operator(): Chat format: peg-native
|
|
2.24.913.936 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.556 (> 0.100 thold), f_keep = 0.116
|
|
2.24.913.938 I srv get_availabl: updating prompt cache
|
|
2.24.913.993 W srv prompt_save: - saving prompt with length 649, total state size = 51.428 MiB (draft: 0.000 MiB)
|
|
2.24.989.772 I srv load: - looking for better prompt, base f_keep = 0.116, sim = 0.556
|
|
2.24.989.782 I srv update: - cache state: 12 prompts, 1209.197 MiB (limits: 8192.000 MiB, 16384 tokens, 76588 est)
|
|
2.24.989.783 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
2.24.989.784 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
2.24.989.785 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
2.24.989.785 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
2.24.989.786 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
2.24.989.786 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
2.24.989.787 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
2.24.989.787 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
2.24.989.788 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
2.24.989.788 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
2.24.989.789 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
2.24.989.790 I srv update: - prompt 0x5d124d2a92c0: 649 tokens, checkpoints: 1, 99.046 MiB
|
|
2.24.989.790 I srv get_availabl: prompt cache update took 75.85 ms
|
|
2.24.989.876 I slot launch_slot_: id 3 | task 10035 | processing task, is_child = 0
|
|
2.24.989.876 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.24.989.877 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.24.989.890 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.24.989.890 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.24.989.899 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.24.989.899 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.24.989.915 I slot update_slots: id 3 | task 10035 | Checking checkpoint with [71, 71] against 75...
|
|
2.25.001.823 W slot update_slots: id 3 | task 10035 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
2.28.350.917 I slot print_timing: id 3 | task 10035 | n_decoded = 250, tg = 83.26 t/s, tg_3s = 83.26 t/s
|
|
2.31.359.973 I slot print_timing: id 3 | task 10035 | n_decoded = 500, tg = 83.17 t/s, tg_3s = 83.08 t/s
|
|
2.33.491.599 I slot print_timing: id 3 | task 10035 | prompt eval time = 358.49 ms / 63 tokens ( 5.69 ms per token, 175.74 tokens per second)
|
|
2.33.491.602 I slot print_timing: id 3 | task 10035 | eval time = 8143.19 ms / 676 tokens ( 12.05 ms per token, 83.01 tokens per second)
|
|
2.33.491.603 I slot print_timing: id 3 | task 10035 | total time = 8501.68 ms / 739 tokens
|
|
2.33.491.604 I slot print_timing: id 3 | task 10035 | graphs reused = 10619
|
|
2.33.491.636 I slot release: id 3 | task 10035 | stop processing: n_tokens = 810, truncated = 0
|
|
2.33.491.646 I srv update_slots: all slots are idle
|
|
2.33.522.157 I srv operator(): Chat format: peg-native
|
|
2.33.522.409 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.608 (> 0.100 thold), f_keep = 0.094
|
|
2.33.522.411 I srv get_availabl: updating prompt cache
|
|
2.33.522.459 W srv prompt_save: - saving prompt with length 810, total state size = 52.373 MiB (draft: 0.000 MiB)
|
|
2.33.595.361 I srv load: - looking for better prompt, base f_keep = 0.094, sim = 0.608
|
|
2.33.595.372 I srv update: - cache state: 13 prompts, 1309.188 MiB (limits: 8192.000 MiB, 16384 tokens, 75807 est)
|
|
2.33.595.373 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
2.33.595.373 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
2.33.595.374 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
2.33.595.374 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
2.33.595.375 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
2.33.595.375 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
2.33.595.376 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
2.33.595.376 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
2.33.595.377 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
2.33.595.377 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
2.33.595.378 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
2.33.595.378 I srv update: - prompt 0x5d124d2a92c0: 649 tokens, checkpoints: 1, 99.046 MiB
|
|
2.33.595.379 I srv update: - prompt 0x5d124d1657f0: 810 tokens, checkpoints: 1, 99.991 MiB
|
|
2.33.595.380 I srv get_availabl: prompt cache update took 72.97 ms
|
|
2.33.595.480 I slot launch_slot_: id 3 | task 10713 | processing task, is_child = 0
|
|
2.33.595.481 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.33.595.481 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.33.595.495 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.33.595.495 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.33.595.504 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.33.595.505 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.33.595.527 I slot update_slots: id 3 | task 10713 | Checking checkpoint with [71, 71] against 76...
|
|
2.33.607.555 W slot update_slots: id 3 | task 10713 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
2.36.905.591 I slot print_timing: id 3 | task 10713 | n_decoded = 248, tg = 82.66 t/s, tg_3s = 82.66 t/s
|
|
2.39.916.884 I slot print_timing: id 3 | task 10713 | n_decoded = 493, tg = 82.01 t/s, tg_3s = 81.36 t/s
|
|
2.40.274.001 I slot print_timing: id 3 | task 10713 | prompt eval time = 309.71 ms / 53 tokens ( 5.84 ms per token, 171.13 tokens per second)
|
|
2.40.274.004 I slot print_timing: id 3 | task 10713 | eval time = 6368.76 ms / 522 tokens ( 12.20 ms per token, 81.96 tokens per second)
|
|
2.40.274.005 I slot print_timing: id 3 | task 10713 | total time = 6678.47 ms / 575 tokens
|
|
2.40.274.006 I slot print_timing: id 3 | task 10713 | graphs reused = 11137
|
|
2.40.274.038 I slot release: id 3 | task 10713 | stop processing: n_tokens = 646, truncated = 0
|
|
2.40.274.049 I srv update_slots: all slots are idle
|
|
2.40.304.664 I srv operator(): Chat format: peg-native
|
|
2.40.304.918 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.573 (> 0.100 thold), f_keep = 0.116
|
|
2.40.304.920 I srv get_availabl: updating prompt cache
|
|
2.40.304.959 W srv prompt_save: - saving prompt with length 646, total state size = 51.410 MiB (draft: 0.000 MiB)
|
|
2.40.366.972 I srv load: - looking for better prompt, base f_keep = 0.116, sim = 0.573
|
|
2.40.366.984 I srv update: - cache state: 14 prompts, 1408.216 MiB (limits: 8192.000 MiB, 16384 tokens, 74234 est)
|
|
2.40.366.985 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
2.40.366.986 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
2.40.366.986 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
2.40.366.987 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
2.40.366.987 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
2.40.366.988 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
2.40.366.988 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
2.40.366.989 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
2.40.366.989 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
2.40.366.990 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
2.40.366.990 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
2.40.366.991 I srv update: - prompt 0x5d124d2a92c0: 649 tokens, checkpoints: 1, 99.046 MiB
|
|
2.40.366.991 I srv update: - prompt 0x5d124d1657f0: 810 tokens, checkpoints: 1, 99.991 MiB
|
|
2.40.366.992 I srv update: - prompt 0x5d124d25eb70: 646 tokens, checkpoints: 1, 99.028 MiB
|
|
2.40.366.993 I srv get_availabl: prompt cache update took 62.07 ms
|
|
2.40.367.092 I slot launch_slot_: id 3 | task 11237 | processing task, is_child = 0
|
|
2.40.367.093 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.40.367.093 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.40.367.106 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.40.367.106 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.40.367.115 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.40.367.115 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.40.367.132 I slot update_slots: id 3 | task 11237 | Checking checkpoint with [71, 71] against 75...
|
|
2.40.378.969 W slot update_slots: id 3 | task 11237 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
2.43.699.848 I slot print_timing: id 3 | task 11237 | n_decoded = 244, tg = 81.11 t/s, tg_3s = 81.11 t/s
|
|
2.46.706.882 I slot print_timing: id 3 | task 11237 | n_decoded = 487, tg = 80.96 t/s, tg_3s = 80.81 t/s
|
|
2.49.718.478 I slot print_timing: id 3 | task 11237 | n_decoded = 729, tg = 80.76 t/s, tg_3s = 80.36 t/s
|
|
2.52.450.545 I slot print_timing: id 3 | task 11237 | prompt eval time = 324.38 ms / 59 tokens ( 5.50 ms per token, 181.89 tokens per second)
|
|
2.52.450.550 I slot print_timing: id 3 | task 11237 | eval time = 11759.03 ms / 947 tokens ( 12.42 ms per token, 80.53 tokens per second)
|
|
2.52.450.551 I slot print_timing: id 3 | task 11237 | total time = 12083.41 ms / 1006 tokens
|
|
2.52.450.551 I slot print_timing: id 3 | task 11237 | graphs reused = 12078
|
|
2.52.450.585 I slot release: id 3 | task 11237 | stop processing: n_tokens = 1077, truncated = 0
|
|
2.52.450.597 I srv update_slots: all slots are idle
|
|
2.52.481.043 I srv operator(): Chat format: peg-native
|
|
2.52.481.295 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.641 (> 0.100 thold), f_keep = 0.070
|
|
2.52.481.298 I srv get_availabl: updating prompt cache
|
|
2.52.481.355 W srv prompt_save: - saving prompt with length 1077, total state size = 53.941 MiB (draft: 0.000 MiB)
|
|
2.52.559.440 I srv load: - looking for better prompt, base f_keep = 0.070, sim = 0.641
|
|
2.52.559.453 I srv update: - cache state: 15 prompts, 1509.775 MiB (limits: 8192.000 MiB, 16384 tokens, 75084 est)
|
|
2.52.559.454 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
2.52.559.455 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
2.52.559.455 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
2.52.559.456 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
2.52.559.456 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
2.52.559.457 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
2.52.559.457 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
2.52.559.458 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
2.52.559.458 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
2.52.559.459 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
2.52.559.459 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
2.52.559.460 I srv update: - prompt 0x5d124d2a92c0: 649 tokens, checkpoints: 1, 99.046 MiB
|
|
2.52.559.460 I srv update: - prompt 0x5d124d1657f0: 810 tokens, checkpoints: 1, 99.991 MiB
|
|
2.52.559.461 I srv update: - prompt 0x5d124d25eb70: 646 tokens, checkpoints: 1, 99.028 MiB
|
|
2.52.559.461 I srv update: - prompt 0x5d1246f61e60: 1077 tokens, checkpoints: 1, 101.559 MiB
|
|
2.52.559.462 I srv get_availabl: prompt cache update took 78.16 ms
|
|
2.52.559.566 I slot launch_slot_: id 3 | task 12186 | processing task, is_child = 0
|
|
2.52.559.567 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
2.52.559.568 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
2.52.559.580 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
2.52.559.581 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
2.52.559.590 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
2.52.559.590 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
2.52.559.606 I slot update_slots: id 3 | task 12186 | Checking checkpoint with [71, 71] against 75...
|
|
2.52.571.536 W slot update_slots: id 3 | task 12186 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
2.55.806.849 I slot print_timing: id 3 | task 12186 | n_decoded = 243, tg = 80.92 t/s, tg_3s = 80.92 t/s
|
|
2.58.811.890 I slot print_timing: id 3 | task 12186 | n_decoded = 493, tg = 82.06 t/s, tg_3s = 83.19 t/s
|
|
3.01.820.919 I slot print_timing: id 3 | task 12186 | n_decoded = 743, tg = 82.40 t/s, tg_3s = 83.08 t/s
|
|
3.04.826.982 I slot print_timing: id 3 | task 12186 | n_decoded = 993, tg = 82.59 t/s, tg_3s = 83.17 t/s
|
|
3.07.460.008 I slot print_timing: id 3 | task 12186 | prompt eval time = 244.19 ms / 45 tokens ( 5.43 ms per token, 184.28 tokens per second)
|
|
3.07.460.012 I slot print_timing: id 3 | task 12186 | eval time = 14656.20 ms / 1212 tokens ( 12.09 ms per token, 82.70 tokens per second)
|
|
3.07.460.013 I slot print_timing: id 3 | task 12186 | total time = 14900.40 ms / 1257 tokens
|
|
3.07.460.014 I slot print_timing: id 3 | task 12186 | graphs reused = 13283
|
|
3.07.460.048 I slot release: id 3 | task 12186 | stop processing: n_tokens = 1328, truncated = 0
|
|
3.07.460.059 I srv update_slots: all slots are idle
|
|
3.07.490.401 I srv operator(): Chat format: peg-native
|
|
3.07.490.662 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.641 (> 0.100 thold), f_keep = 0.056
|
|
3.07.490.664 I srv get_availabl: updating prompt cache
|
|
3.07.490.723 W srv prompt_save: - saving prompt with length 1328, total state size = 55.414 MiB (draft: 0.000 MiB)
|
|
3.07.560.953 I srv load: - looking for better prompt, base f_keep = 0.056, sim = 0.641
|
|
3.07.560.964 I srv update: - cache state: 16 prompts, 1612.807 MiB (limits: 8192.000 MiB, 16384 tokens, 77033 est)
|
|
3.07.560.965 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
3.07.560.966 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
3.07.560.967 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
3.07.560.967 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
3.07.560.968 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
3.07.560.968 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
3.07.560.969 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
3.07.560.969 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
3.07.560.970 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
3.07.560.970 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
3.07.560.971 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
3.07.560.971 I srv update: - prompt 0x5d124d2a92c0: 649 tokens, checkpoints: 1, 99.046 MiB
|
|
3.07.560.972 I srv update: - prompt 0x5d124d1657f0: 810 tokens, checkpoints: 1, 99.991 MiB
|
|
3.07.560.972 I srv update: - prompt 0x5d124d25eb70: 646 tokens, checkpoints: 1, 99.028 MiB
|
|
3.07.560.973 I srv update: - prompt 0x5d1246f61e60: 1077 tokens, checkpoints: 1, 101.559 MiB
|
|
3.07.560.973 I srv update: - prompt 0x5d1246f79540: 1328 tokens, checkpoints: 1, 103.032 MiB
|
|
3.07.560.974 I srv get_availabl: prompt cache update took 70.31 ms
|
|
3.07.561.069 I slot launch_slot_: id 3 | task 13400 | processing task, is_child = 0
|
|
3.07.561.070 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.07.561.071 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.07.561.086 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.07.561.086 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.07.561.095 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.07.561.095 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.07.561.111 I slot update_slots: id 3 | task 13400 | Checking checkpoint with [71, 71] against 75...
|
|
3.07.572.942 W slot update_slots: id 3 | task 13400 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
3.10.817.976 I slot print_timing: id 3 | task 13400 | n_decoded = 251, tg = 83.37 t/s, tg_3s = 83.37 t/s
|
|
3.13.823.011 I slot print_timing: id 3 | task 13400 | n_decoded = 501, tg = 83.28 t/s, tg_3s = 83.19 t/s
|
|
3.16.828.066 I slot print_timing: id 3 | task 13400 | n_decoded = 751, tg = 83.25 t/s, tg_3s = 83.19 t/s
|
|
3.19.331.071 I slot print_timing: id 3 | task 13400 | prompt eval time = 246.18 ms / 45 tokens ( 5.47 ms per token, 182.79 tokens per second)
|
|
3.19.331.075 I slot print_timing: id 3 | task 13400 | eval time = 11523.77 ms / 959 tokens ( 12.02 ms per token, 83.22 tokens per second)
|
|
3.19.331.075 I slot print_timing: id 3 | task 13400 | total time = 11769.95 ms / 1004 tokens
|
|
3.19.331.076 I slot print_timing: id 3 | task 13400 | graphs reused = 14236
|
|
3.19.331.110 I slot release: id 3 | task 13400 | stop processing: n_tokens = 1075, truncated = 0
|
|
3.19.331.121 I srv update_slots: all slots are idle
|
|
3.19.361.651 I srv operator(): Chat format: peg-native
|
|
3.19.361.899 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.688 (> 0.100 thold), f_keep = 0.070
|
|
3.19.361.901 I srv get_availabl: updating prompt cache
|
|
3.19.361.960 W srv prompt_save: - saving prompt with length 1075, total state size = 53.929 MiB (draft: 0.000 MiB)
|
|
3.19.433.006 I srv load: - looking for better prompt, base f_keep = 0.070, sim = 0.688
|
|
3.19.433.018 I srv update: - cache state: 17 prompts, 1714.354 MiB (limits: 8192.000 MiB, 16384 tokens, 77607 est)
|
|
3.19.433.019 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
3.19.433.020 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
3.19.433.020 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
3.19.433.021 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
3.19.433.021 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
3.19.433.022 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
3.19.433.022 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
3.19.433.023 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
3.19.433.023 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
3.19.433.024 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
3.19.433.024 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
3.19.433.025 I srv update: - prompt 0x5d124d2a92c0: 649 tokens, checkpoints: 1, 99.046 MiB
|
|
3.19.433.025 I srv update: - prompt 0x5d124d1657f0: 810 tokens, checkpoints: 1, 99.991 MiB
|
|
3.19.433.026 I srv update: - prompt 0x5d124d25eb70: 646 tokens, checkpoints: 1, 99.028 MiB
|
|
3.19.433.027 I srv update: - prompt 0x5d1246f61e60: 1077 tokens, checkpoints: 1, 101.559 MiB
|
|
3.19.433.027 I srv update: - prompt 0x5d1246f79540: 1328 tokens, checkpoints: 1, 103.032 MiB
|
|
3.19.433.028 I srv update: - prompt 0x5d12470ce770: 1075 tokens, checkpoints: 1, 101.547 MiB
|
|
3.19.433.029 I srv get_availabl: prompt cache update took 71.13 ms
|
|
3.19.433.118 I slot launch_slot_: id 3 | task 14361 | processing task, is_child = 0
|
|
3.19.433.119 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.19.433.120 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.19.433.133 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.19.433.133 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.19.433.142 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.19.433.142 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.19.433.158 I slot update_slots: id 3 | task 14361 | Checking checkpoint with [71, 71] against 75...
|
|
3.19.445.060 W slot update_slots: id 3 | task 14361 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
3.22.645.092 I slot print_timing: id 3 | task 14361 | n_decoded = 251, tg = 83.35 t/s, tg_3s = 83.35 t/s
|
|
3.25.650.120 I slot print_timing: id 3 | task 14361 | n_decoded = 501, tg = 83.27 t/s, tg_3s = 83.19 t/s
|
|
3.27.406.075 I slot print_timing: id 3 | task 14361 | prompt eval time = 200.49 ms / 37 tokens ( 5.42 ms per token, 184.55 tokens per second)
|
|
3.27.406.079 I slot print_timing: id 3 | task 14361 | eval time = 7772.43 ms / 647 tokens ( 12.01 ms per token, 83.24 tokens per second)
|
|
3.27.406.080 I slot print_timing: id 3 | task 14361 | total time = 7972.91 ms / 684 tokens
|
|
3.27.406.081 I slot print_timing: id 3 | task 14361 | graphs reused = 14879
|
|
3.27.406.112 I slot release: id 3 | task 14361 | stop processing: n_tokens = 755, truncated = 0
|
|
3.27.406.122 I srv update_slots: all slots are idle
|
|
3.27.436.344 I srv operator(): Chat format: peg-native
|
|
3.27.436.605 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.765 (> 0.100 thold), f_keep = 0.099
|
|
3.27.436.605 I srv get_availabl: updating prompt cache
|
|
3.27.436.654 W srv prompt_save: - saving prompt with length 755, total state size = 52.050 MiB (draft: 0.000 MiB)
|
|
3.27.503.998 I srv load: - looking for better prompt, base f_keep = 0.099, sim = 0.765
|
|
3.27.504.010 I srv update: - cache state: 18 prompts, 1814.022 MiB (limits: 8192.000 MiB, 16384 tokens, 76752 est)
|
|
3.27.504.011 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
3.27.504.011 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
3.27.504.012 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
3.27.504.012 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
3.27.504.013 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
3.27.504.014 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
3.27.504.014 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
3.27.504.015 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
3.27.504.015 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
3.27.504.016 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
3.27.504.016 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
3.27.504.017 I srv update: - prompt 0x5d124d2a92c0: 649 tokens, checkpoints: 1, 99.046 MiB
|
|
3.27.504.017 I srv update: - prompt 0x5d124d1657f0: 810 tokens, checkpoints: 1, 99.991 MiB
|
|
3.27.504.018 I srv update: - prompt 0x5d124d25eb70: 646 tokens, checkpoints: 1, 99.028 MiB
|
|
3.27.504.018 I srv update: - prompt 0x5d1246f61e60: 1077 tokens, checkpoints: 1, 101.559 MiB
|
|
3.27.504.019 I srv update: - prompt 0x5d1246f79540: 1328 tokens, checkpoints: 1, 103.032 MiB
|
|
3.27.504.019 I srv update: - prompt 0x5d12470ce770: 1075 tokens, checkpoints: 1, 101.547 MiB
|
|
3.27.504.020 I srv update: - prompt 0x5d124708c4c0: 755 tokens, checkpoints: 1, 99.668 MiB
|
|
3.27.504.021 I srv get_availabl: prompt cache update took 67.41 ms
|
|
3.27.504.108 I slot launch_slot_: id 3 | task 15010 | processing task, is_child = 0
|
|
3.27.504.109 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.27.504.110 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.27.504.120 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.27.504.121 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.27.504.130 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.27.504.130 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.27.504.144 I slot update_slots: id 3 | task 15010 | Checking checkpoint with [71, 71] against 75...
|
|
3.27.516.003 W slot update_slots: id 3 | task 15010 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
3.30.656.991 I slot print_timing: id 3 | task 15010 | n_decoded = 249, tg = 82.75 t/s, tg_3s = 82.75 t/s
|
|
3.33.662.013 I slot print_timing: id 3 | task 15010 | n_decoded = 493, tg = 81.97 t/s, tg_3s = 81.20 t/s
|
|
3.35.923.647 I slot print_timing: id 3 | task 15010 | prompt eval time = 143.66 ms / 26 tokens ( 5.53 ms per token, 180.98 tokens per second)
|
|
3.35.923.650 I slot print_timing: id 3 | task 15010 | eval time = 8275.84 ms / 676 tokens ( 12.24 ms per token, 81.68 tokens per second)
|
|
3.35.923.651 I slot print_timing: id 3 | task 15010 | total time = 8419.50 ms / 702 tokens
|
|
3.35.923.651 I slot print_timing: id 3 | task 15010 | graphs reused = 15550
|
|
3.35.923.680 I slot release: id 3 | task 15010 | stop processing: n_tokens = 773, truncated = 0
|
|
3.35.923.689 I srv update_slots: all slots are idle
|
|
3.35.954.127 I srv operator(): Chat format: peg-native
|
|
3.35.954.377 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.625 (> 0.100 thold), f_keep = 0.097
|
|
3.35.954.378 I srv get_availabl: updating prompt cache
|
|
3.35.954.427 W srv prompt_save: - saving prompt with length 773, total state size = 52.156 MiB (draft: 0.000 MiB)
|
|
3.36.022.859 I srv load: - looking for better prompt, base f_keep = 0.097, sim = 0.625
|
|
3.36.022.873 I srv update: - cache state: 19 prompts, 1913.796 MiB (limits: 8192.000 MiB, 16384 tokens, 76060 est)
|
|
3.36.022.874 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
3.36.022.874 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
3.36.022.875 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
3.36.022.875 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
3.36.022.876 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
3.36.022.877 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
3.36.022.877 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
3.36.022.878 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
3.36.022.878 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
3.36.022.879 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
3.36.022.879 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
3.36.022.880 I srv update: - prompt 0x5d124d2a92c0: 649 tokens, checkpoints: 1, 99.046 MiB
|
|
3.36.022.880 I srv update: - prompt 0x5d124d1657f0: 810 tokens, checkpoints: 1, 99.991 MiB
|
|
3.36.022.881 I srv update: - prompt 0x5d124d25eb70: 646 tokens, checkpoints: 1, 99.028 MiB
|
|
3.36.022.882 I srv update: - prompt 0x5d1246f61e60: 1077 tokens, checkpoints: 1, 101.559 MiB
|
|
3.36.022.883 I srv update: - prompt 0x5d1246f79540: 1328 tokens, checkpoints: 1, 103.032 MiB
|
|
3.36.022.885 I srv update: - prompt 0x5d12470ce770: 1075 tokens, checkpoints: 1, 101.547 MiB
|
|
3.36.022.885 I srv update: - prompt 0x5d124708c4c0: 755 tokens, checkpoints: 1, 99.668 MiB
|
|
3.36.022.885 I srv update: - prompt 0x5d124d2a0a20: 773 tokens, checkpoints: 1, 99.774 MiB
|
|
3.36.022.887 I srv get_availabl: prompt cache update took 68.51 ms
|
|
3.36.022.989 I slot launch_slot_: id 3 | task 15688 | processing task, is_child = 0
|
|
3.36.022.990 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.36.022.990 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.36.023.002 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.36.023.003 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.36.023.011 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.36.023.012 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.36.023.028 I slot update_slots: id 3 | task 15688 | Checking checkpoint with [71, 71] against 75...
|
|
3.36.034.929 W slot update_slots: id 3 | task 15688 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
3.39.282.167 I slot print_timing: id 3 | task 15688 | n_decoded = 244, tg = 81.20 t/s, tg_3s = 81.20 t/s
|
|
3.40.872.173 I slot print_timing: id 3 | task 15688 | prompt eval time = 254.20 ms / 48 tokens ( 5.30 ms per token, 188.83 tokens per second)
|
|
3.40.872.177 I slot print_timing: id 3 | task 15688 | eval time = 4594.94 ms / 373 tokens ( 12.32 ms per token, 81.18 tokens per second)
|
|
3.40.872.178 I slot print_timing: id 3 | task 15688 | total time = 4849.14 ms / 421 tokens
|
|
3.40.872.178 I slot print_timing: id 3 | task 15688 | graphs reused = 15920
|
|
3.40.872.208 I slot release: id 3 | task 15688 | stop processing: n_tokens = 492, truncated = 0
|
|
3.40.872.218 I srv update_slots: all slots are idle
|
|
3.40.902.648 I srv operator(): Chat format: peg-native
|
|
3.40.902.898 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.694 (> 0.100 thold), f_keep = 0.152
|
|
3.40.902.901 I srv get_availabl: updating prompt cache
|
|
3.40.902.937 W srv prompt_save: - saving prompt with length 492, total state size = 50.506 MiB (draft: 0.000 MiB)
|
|
3.40.971.873 I srv load: - looking for better prompt, base f_keep = 0.152, sim = 0.694
|
|
3.40.971.886 I srv update: - cache state: 20 prompts, 2011.920 MiB (limits: 8192.000 MiB, 16384 tokens, 74353 est)
|
|
3.40.971.887 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
3.40.971.887 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
3.40.971.888 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
3.40.971.888 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
3.40.971.889 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
3.40.971.890 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
3.40.971.890 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
3.40.971.891 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
3.40.971.891 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
3.40.971.892 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
3.40.971.892 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
3.40.971.893 I srv update: - prompt 0x5d124d2a92c0: 649 tokens, checkpoints: 1, 99.046 MiB
|
|
3.40.971.893 I srv update: - prompt 0x5d124d1657f0: 810 tokens, checkpoints: 1, 99.991 MiB
|
|
3.40.971.894 I srv update: - prompt 0x5d124d25eb70: 646 tokens, checkpoints: 1, 99.028 MiB
|
|
3.40.971.894 I srv update: - prompt 0x5d1246f61e60: 1077 tokens, checkpoints: 1, 101.559 MiB
|
|
3.40.971.895 I srv update: - prompt 0x5d1246f79540: 1328 tokens, checkpoints: 1, 103.032 MiB
|
|
3.40.971.895 I srv update: - prompt 0x5d12470ce770: 1075 tokens, checkpoints: 1, 101.547 MiB
|
|
3.40.971.896 I srv update: - prompt 0x5d124708c4c0: 755 tokens, checkpoints: 1, 99.668 MiB
|
|
3.40.971.896 I srv update: - prompt 0x5d124d2a0a20: 773 tokens, checkpoints: 1, 99.774 MiB
|
|
3.40.971.897 I srv update: - prompt 0x5d1246f72650: 492 tokens, checkpoints: 1, 98.124 MiB
|
|
3.40.971.898 I srv get_availabl: prompt cache update took 69.00 ms
|
|
3.40.971.984 I slot launch_slot_: id 3 | task 16063 | processing task, is_child = 0
|
|
3.40.971.985 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.40.971.985 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.40.971.997 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.40.971.997 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.40.972.005 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.40.972.006 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.40.972.021 I slot update_slots: id 3 | task 16063 | Checking checkpoint with [71, 71] against 75...
|
|
3.40.983.931 W slot update_slots: id 3 | task 16063 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
3.44.166.297 I slot print_timing: id 3 | task 16063 | n_decoded = 250, tg = 83.10 t/s, tg_3s = 83.10 t/s
|
|
3.47.175.313 I slot print_timing: id 3 | task 16063 | n_decoded = 500, tg = 83.09 t/s, tg_3s = 83.08 t/s
|
|
3.49.257.735 I slot print_timing: id 3 | task 16063 | prompt eval time = 185.75 ms / 36 tokens ( 5.16 ms per token, 193.81 tokens per second)
|
|
3.49.257.738 I slot print_timing: id 3 | task 16063 | eval time = 8099.96 ms / 673 tokens ( 12.04 ms per token, 83.09 tokens per second)
|
|
3.49.257.739 I slot print_timing: id 3 | task 16063 | total time = 8285.71 ms / 709 tokens
|
|
3.49.257.740 I slot print_timing: id 3 | task 16063 | graphs reused = 16588
|
|
3.49.257.771 I slot release: id 3 | task 16063 | stop processing: n_tokens = 780, truncated = 0
|
|
3.49.257.781 I srv update_slots: all slots are idle
|
|
3.49.288.562 I srv operator(): Chat format: peg-native
|
|
3.49.288.824 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.647 (> 0.100 thold), f_keep = 0.096
|
|
3.49.288.827 I srv get_availabl: updating prompt cache
|
|
3.49.288.885 W srv prompt_save: - saving prompt with length 780, total state size = 52.197 MiB (draft: 0.000 MiB)
|
|
3.49.351.039 I srv load: - looking for better prompt, base f_keep = 0.096, sim = 0.647
|
|
3.49.351.053 I srv update: - cache state: 21 prompts, 2111.735 MiB (limits: 8192.000 MiB, 16384 tokens, 73865 est)
|
|
3.49.351.054 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
3.49.351.055 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
3.49.351.055 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
3.49.351.056 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
3.49.351.056 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
3.49.351.057 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
3.49.351.057 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
3.49.351.058 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
3.49.351.058 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
3.49.351.059 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
3.49.351.059 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
3.49.351.060 I srv update: - prompt 0x5d124d2a92c0: 649 tokens, checkpoints: 1, 99.046 MiB
|
|
3.49.351.060 I srv update: - prompt 0x5d124d1657f0: 810 tokens, checkpoints: 1, 99.991 MiB
|
|
3.49.351.061 I srv update: - prompt 0x5d124d25eb70: 646 tokens, checkpoints: 1, 99.028 MiB
|
|
3.49.351.061 I srv update: - prompt 0x5d1246f61e60: 1077 tokens, checkpoints: 1, 101.559 MiB
|
|
3.49.351.062 I srv update: - prompt 0x5d1246f79540: 1328 tokens, checkpoints: 1, 103.032 MiB
|
|
3.49.351.062 I srv update: - prompt 0x5d12470ce770: 1075 tokens, checkpoints: 1, 101.547 MiB
|
|
3.49.351.063 I srv update: - prompt 0x5d124708c4c0: 755 tokens, checkpoints: 1, 99.668 MiB
|
|
3.49.351.064 I srv update: - prompt 0x5d124d2a0a20: 773 tokens, checkpoints: 1, 99.774 MiB
|
|
3.49.351.070 I srv update: - prompt 0x5d1246f72650: 492 tokens, checkpoints: 1, 98.124 MiB
|
|
3.49.351.070 I srv update: - prompt 0x5d124d2b2390: 780 tokens, checkpoints: 1, 99.815 MiB
|
|
3.49.351.072 I srv get_availabl: prompt cache update took 62.24 ms
|
|
3.49.351.159 I slot launch_slot_: id 3 | task 16738 | processing task, is_child = 0
|
|
3.49.351.160 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
3.49.351.160 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
3.49.351.177 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
3.49.351.177 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
3.49.351.198 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
3.49.351.198 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
3.49.351.227 I slot update_slots: id 3 | task 16738 | Checking checkpoint with [71, 71] against 75...
|
|
3.49.363.156 W slot update_slots: id 3 | task 16738 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
3.52.565.732 I slot print_timing: id 3 | task 16738 | n_decoded = 251, tg = 83.38 t/s, tg_3s = 83.38 t/s
|
|
3.55.570.752 I slot print_timing: id 3 | task 16738 | n_decoded = 501, tg = 83.29 t/s, tg_3s = 83.19 t/s
|
|
3.58.579.799 I slot print_timing: id 3 | task 16738 | n_decoded = 751, tg = 83.22 t/s, tg_3s = 83.08 t/s
|
|
4.01.581.730 I slot print_timing: id 3 | task 16738 | n_decoded = 999, tg = 83.07 t/s, tg_3s = 82.61 t/s
|
|
4.03.312.449 I slot print_timing: id 3 | task 16738 | prompt eval time = 204.13 ms / 44 tokens ( 4.64 ms per token, 215.55 tokens per second)
|
|
4.03.312.452 I slot print_timing: id 3 | task 16738 | eval time = 13757.09 ms / 1141 tokens ( 12.06 ms per token, 82.94 tokens per second)
|
|
4.03.312.453 I slot print_timing: id 3 | task 16738 | total time = 13961.22 ms / 1185 tokens
|
|
4.03.312.454 I slot print_timing: id 3 | task 16738 | graphs reused = 17723
|
|
4.03.312.488 I slot release: id 3 | task 16738 | stop processing: n_tokens = 1256, truncated = 0
|
|
4.03.312.499 I srv update_slots: all slots are idle
|
|
4.03.343.120 I srv operator(): Chat format: peg-native
|
|
4.03.343.371 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.615 (> 0.100 thold), f_keep = 0.060
|
|
4.03.343.373 I srv get_availabl: updating prompt cache
|
|
4.03.343.424 W srv prompt_save: - saving prompt with length 1256, total state size = 54.992 MiB (draft: 0.000 MiB)
|
|
4.03.416.843 I srv load: - looking for better prompt, base f_keep = 0.060, sim = 0.615
|
|
4.03.416.856 I srv update: - cache state: 22 prompts, 2214.344 MiB (limits: 8192.000 MiB, 16384 tokens, 75089 est)
|
|
4.03.416.859 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
4.03.416.860 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
4.03.416.860 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
4.03.416.861 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
4.03.416.861 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
4.03.416.862 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
4.03.416.863 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
4.03.416.864 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
4.03.416.864 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
4.03.416.865 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
4.03.416.865 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
4.03.416.866 I srv update: - prompt 0x5d124d2a92c0: 649 tokens, checkpoints: 1, 99.046 MiB
|
|
4.03.416.866 I srv update: - prompt 0x5d124d1657f0: 810 tokens, checkpoints: 1, 99.991 MiB
|
|
4.03.416.867 I srv update: - prompt 0x5d124d25eb70: 646 tokens, checkpoints: 1, 99.028 MiB
|
|
4.03.416.867 I srv update: - prompt 0x5d1246f61e60: 1077 tokens, checkpoints: 1, 101.559 MiB
|
|
4.03.416.868 I srv update: - prompt 0x5d1246f79540: 1328 tokens, checkpoints: 1, 103.032 MiB
|
|
4.03.416.868 I srv update: - prompt 0x5d12470ce770: 1075 tokens, checkpoints: 1, 101.547 MiB
|
|
4.03.416.869 I srv update: - prompt 0x5d124708c4c0: 755 tokens, checkpoints: 1, 99.668 MiB
|
|
4.03.416.869 I srv update: - prompt 0x5d124d2a0a20: 773 tokens, checkpoints: 1, 99.774 MiB
|
|
4.03.416.870 I srv update: - prompt 0x5d1246f72650: 492 tokens, checkpoints: 1, 98.124 MiB
|
|
4.03.416.870 I srv update: - prompt 0x5d124d2b2390: 780 tokens, checkpoints: 1, 99.815 MiB
|
|
4.03.416.871 I srv update: - prompt 0x5d1247010110: 1256 tokens, checkpoints: 1, 102.609 MiB
|
|
4.03.416.872 I srv get_availabl: prompt cache update took 73.50 ms
|
|
4.03.416.964 I slot launch_slot_: id 3 | task 17881 | processing task, is_child = 0
|
|
4.03.416.965 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.03.416.966 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.03.416.980 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.03.416.981 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.03.416.990 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.03.416.990 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.03.417.007 I slot update_slots: id 3 | task 17881 | Checking checkpoint with [71, 71] against 75...
|
|
4.03.428.853 W slot update_slots: id 3 | task 17881 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
4.06.677.410 I slot print_timing: id 3 | task 17881 | n_decoded = 250, tg = 83.24 t/s, tg_3s = 83.24 t/s
|
|
4.07.930.267 I slot print_timing: id 3 | task 17881 | prompt eval time = 257.06 ms / 50 tokens ( 5.14 ms per token, 194.50 tokens per second)
|
|
4.07.930.271 I slot print_timing: id 3 | task 17881 | eval time = 4256.19 ms / 353 tokens ( 12.06 ms per token, 82.94 tokens per second)
|
|
4.07.930.272 I slot print_timing: id 3 | task 17881 | total time = 4513.26 ms / 403 tokens
|
|
4.07.930.273 I slot print_timing: id 3 | task 17881 | graphs reused = 18073
|
|
4.07.930.307 I slot release: id 3 | task 17881 | stop processing: n_tokens = 474, truncated = 0
|
|
4.07.930.318 I srv update_slots: all slots are idle
|
|
4.07.960.511 I srv operator(): Chat format: peg-native
|
|
4.07.960.768 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.701 (> 0.100 thold), f_keep = 0.158
|
|
4.07.960.771 I srv get_availabl: updating prompt cache
|
|
4.07.960.802 W srv prompt_save: - saving prompt with length 474, total state size = 50.401 MiB (draft: 0.000 MiB)
|
|
4.08.024.702 I srv load: - looking for better prompt, base f_keep = 0.158, sim = 0.701
|
|
4.08.024.717 I srv update: - cache state: 23 prompts, 2312.362 MiB (limits: 8192.000 MiB, 16384 tokens, 73585 est)
|
|
4.08.024.718 I srv update: - prompt 0x5d1246f6dd50: 1573 tokens, checkpoints: 1, 104.470 MiB
|
|
4.08.024.718 I srv update: - prompt 0x5d1246f4cd00: 1236 tokens, checkpoints: 1, 102.492 MiB
|
|
4.08.024.719 I srv update: - prompt 0x5d124d161fb0: 530 tokens, checkpoints: 1, 98.347 MiB
|
|
4.08.024.719 I srv update: - prompt 0x5d1246f52790: 1053 tokens, checkpoints: 1, 101.418 MiB
|
|
4.08.024.720 I srv update: - prompt 0x5d124700bd30: 1509 tokens, checkpoints: 1, 104.095 MiB
|
|
4.08.024.720 I srv update: - prompt 0x5d1246ff14b0: 1126 tokens, checkpoints: 1, 101.846 MiB
|
|
4.08.024.721 I srv update: - prompt 0x5d124d2628f0: 807 tokens, checkpoints: 1, 99.973 MiB
|
|
4.08.024.722 I srv update: - prompt 0x5d1246f7d810: 735 tokens, checkpoints: 1, 99.551 MiB
|
|
4.08.024.722 I srv update: - prompt 0x5d124d2ad330: 633 tokens, checkpoints: 1, 98.952 MiB
|
|
4.08.024.723 I srv update: - prompt 0x5d124708bc30: 627 tokens, checkpoints: 1, 98.917 MiB
|
|
4.08.024.723 I srv update: - prompt 0x5d12471a6a80: 827 tokens, checkpoints: 1, 100.091 MiB
|
|
4.08.024.724 I srv update: - prompt 0x5d124d2a92c0: 649 tokens, checkpoints: 1, 99.046 MiB
|
|
4.08.024.724 I srv update: - prompt 0x5d124d1657f0: 810 tokens, checkpoints: 1, 99.991 MiB
|
|
4.08.024.725 I srv update: - prompt 0x5d124d25eb70: 646 tokens, checkpoints: 1, 99.028 MiB
|
|
4.08.024.725 I srv update: - prompt 0x5d1246f61e60: 1077 tokens, checkpoints: 1, 101.559 MiB
|
|
4.08.024.726 I srv update: - prompt 0x5d1246f79540: 1328 tokens, checkpoints: 1, 103.032 MiB
|
|
4.08.024.726 I srv update: - prompt 0x5d12470ce770: 1075 tokens, checkpoints: 1, 101.547 MiB
|
|
4.08.024.727 I srv update: - prompt 0x5d124708c4c0: 755 tokens, checkpoints: 1, 99.668 MiB
|
|
4.08.024.727 I srv update: - prompt 0x5d124d2a0a20: 773 tokens, checkpoints: 1, 99.774 MiB
|
|
4.08.024.728 I srv update: - prompt 0x5d1246f72650: 492 tokens, checkpoints: 1, 98.124 MiB
|
|
4.08.024.728 I srv update: - prompt 0x5d124d2b2390: 780 tokens, checkpoints: 1, 99.815 MiB
|
|
4.08.024.729 I srv update: - prompt 0x5d1247010110: 1256 tokens, checkpoints: 1, 102.609 MiB
|
|
4.08.024.729 I srv update: - prompt 0x5d1245588ce0: 474 tokens, checkpoints: 1, 98.018 MiB
|
|
4.08.024.731 I srv get_availabl: prompt cache update took 63.96 ms
|
|
4.08.024.825 I slot launch_slot_: id 3 | task 18236 | processing task, is_child = 0
|
|
4.08.024.826 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
|
4.08.024.827 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
|
4.08.024.838 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
|
4.08.024.838 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
|
4.08.024.846 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
|
4.08.024.847 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
|
4.08.024.863 I slot update_slots: id 3 | task 18236 | Checking checkpoint with [71, 71] against 75...
|
|
4.08.036.679 W slot update_slots: id 3 | task 18236 | restored context checkpoint (pos_min = 71, pos_max = 71, n_tokens = 72, n_past = 72, size = 47.618 MiB)
|
|
4.11.216.280 I slot print_timing: id 3 | task 18236 | n_decoded = 246, tg = 81.74 t/s, tg_3s = 81.74 t/s
|
|
4.14.223.524 I slot print_timing: id 3 | task 18236 | n_decoded = 490, tg = 81.44 t/s, tg_3s = 81.14 t/s
|
|
4.17.227.744 I slot print_timing: id 3 | task 18236 | n_decoded = 733, tg = 81.25 t/s, tg_3s = 80.89 t/s
|
|
4.20.235.877 I slot print_timing: id 3 | task 18236 | n_decoded = 975, tg = 81.05 t/s, tg_3s = 80.45 t/s
|
|
4.21.807.903 I slot print_timing: id 3 | task 18236 | prompt eval time = 181.80 ms / 35 tokens ( 5.19 ms per token, 192.52 tokens per second)
|
|
4.21.807.907 I slot print_timing: id 3 | task 18236 | eval time = 13601.24 ms / 1102 tokens ( 12.34 ms per token, 81.02 tokens per second)
|
|
4.21.807.907 I slot print_timing: id 3 | task 18236 | total time = 13783.04 ms / 1137 tokens
|
|
4.21.807.908 I slot print_timing: id 3 | task 18236 | graphs reused = 19169
|
|
4.21.807.939 I slot release: id 3 | task 18236 | stop processing: n_tokens = 1208, truncated = 0
|
|
4.21.807.951 I srv update_slots: all slots are idle
|
|
4.21.808.536 I srv operator(): operator(): cleaning up before exit...
|