0.00.065.681 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.065.684 I device_info:
0.00.065.704 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.709 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.716 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.065.772 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.065.776 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.065.815 I srv init: running without SSL
0.00.065.875 I srv init: using 23 threads for HTTP server
0.00.066.206 I srv start: binding port with default address family
0.00.067.404 I srv llama_server: loading model
0.00.067.414 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/nvidia_Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf'
0.00.067.461 I common_init_result: fitting params to device memory ...
0.00.067.464 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.22.612.553 W llama_context: n_ctx_seq (24576) < n_ctx_train (1048576) -- the full capacity of the model will not be utilized
0.22.722.753 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.22.907.089 I srv load_model: initializing slots, n_slots = 4
0.22.938.777 W srv load_model: speculative decoding will use checkpoints
0.22.938.798 W common_speculative_init: no implementations specified for speculative decoding
0.22.938.799 I slot load_model: id 0 | task -1 | new slot, n_ctx = 24576
0.22.938.803 I slot load_model: id 1 | task -1 | new slot, n_ctx = 24576
0.22.938.803 I slot load_model: id 2 | task -1 | new slot, n_ctx = 24576
0.22.938.804 I slot load_model: id 3 | task -1 | new slot, n_ctx = 24576
0.22.938.873 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.22.938.873 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.22.938.873 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.22.938.874 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.22.938.892 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.22.973.329 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
'
0.23.002.855 I srv init: init: chat template, thinking = 1
0.23.002.893 I srv llama_server: model loaded
0.23.002.897 I srv llama_server: server is listening on http://127.0.0.1:8090
0.23.002.903 I srv update_slots: all slots are idle
0.24.997.788 I srv operator(): Chat format: peg-native
0.25.000.661 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.25.000.664 I srv get_availabl: updating prompt cache
0.25.000.669 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.25.000.673 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 24576 tokens, 8589934592 est)
0.25.000.674 I srv get_availabl: prompt cache update took 0.01 ms
0.25.001.621 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.25.001.623 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.25.001.625 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.25.001.645 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.25.001.645 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.25.001.659 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.25.001.660 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.25.725.546 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 41, pos_max = 41, n_tokens = 42, size = 47.618 MiB)
0.27.915.825 I slot print_timing: id 3 | task 0 | prompt eval time = 867.32 ms / 69 tokens ( 12.57 ms per token, 79.56 tokens per second)
0.27.915.829 I slot print_timing: id 3 | task 0 | eval time = 2046.81 ms / 162 tokens ( 12.63 ms per token, 79.15 tokens per second)
0.27.915.829 I slot print_timing: id 3 | task 0 | total time = 2914.13 ms / 231 tokens
0.27.915.833 I slot print_timing: id 3 | task 0 | graphs reused = 161
0.27.915.873 I slot release: id 3 | task 0 | stop processing: n_tokens = 230, truncated = 0
0.27.915.877 I srv update_slots: all slots are idle
0.27.949.506 I srv operator(): Chat format: peg-native
0.27.952.333 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.625 (> 0.100 thold), f_keep = 0.196
0.27.952.335 I srv get_availabl: updating prompt cache
0.27.952.372 W srv prompt_save: - saving prompt with length 230, total state size = 48.968 MiB (draft: 0.000 MiB)
0.28.011.736 I srv load: - looking for better prompt, base f_keep = 0.196, sim = 0.625
0.28.011.743 I srv update: - cache state: 1 prompts, 96.586 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
0.28.011.743 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
0.28.011.745 I srv get_availabl: prompt cache update took 59.41 ms
0.28.012.644 I slot launch_slot_: id 3 | task 165 | processing task, is_child = 0
0.28.012.646 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.28.012.646 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.28.012.661 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.28.012.661 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.28.012.674 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.28.012.674 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.28.012.692 I slot update_slots: id 3 | task 165 | Checking checkpoint with [41, 41] against 45...
0.28.024.571 W slot update_slots: id 3 | task 165 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 47.618 MiB)
0.29.160.844 I slot print_timing: id 3 | task 165 | prompt eval time = 177.26 ms / 30 tokens ( 5.91 ms per token, 169.24 tokens per second)
0.29.160.847 I slot print_timing: id 3 | task 165 | eval time = 970.88 ms / 77 tokens ( 12.61 ms per token, 79.31 tokens per second)
0.29.160.848 I slot print_timing: id 3 | task 165 | total time = 1148.15 ms / 107 tokens
0.29.160.848 I slot print_timing: id 3 | task 165 | graphs reused = 236
0.29.160.874 I slot release: id 3 | task 165 | stop processing: n_tokens = 148, truncated = 0
0.29.160.880 I srv update_slots: all slots are idle
0.29.194.005 I srv operator(): Chat format: peg-native
0.29.196.872 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.489 (> 0.100 thold), f_keep = 0.304
0.29.196.874 I srv get_availabl: updating prompt cache
0.29.196.907 W srv prompt_save: - saving prompt with length 148, total state size = 48.487 MiB (draft: 0.000 MiB)
0.29.282.730 I srv load: - looking for better prompt, base f_keep = 0.304, sim = 0.489
0.29.282.736 I srv update: - cache state: 2 prompts, 192.690 MiB (limits: 8192.000 MiB, 24576 tokens, 24576 est)
0.29.282.737 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
0.29.282.738 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
0.29.282.739 I srv get_availabl: prompt cache update took 85.86 ms
0.29.283.640 I slot launch_slot_: id 3 | task 244 | processing task, is_child = 0
0.29.283.642 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.283.643 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.283.657 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.283.657 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.283.670 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.283.670 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.29.283.690 I slot update_slots: id 3 | task 244 | Checking checkpoint with [41, 41] against 45...
0.29.295.565 W slot update_slots: id 3 | task 244 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 47.618 MiB)
0.32.553.379 I slot print_timing: id 3 | task 244 | n_decoded = 237, tg = 78.97 t/s, tg_3s = 78.96 t/s
0.35.558.370 I slot print_timing: id 3 | task 244 | n_decoded = 472, tg = 78.58 t/s, tg_3s = 78.20 t/s
0.36.936.387 I slot print_timing: id 3 | task 244 | prompt eval time = 268.36 ms / 50 tokens ( 5.37 ms per token, 186.32 tokens per second)
0.36.936.390 I slot print_timing: id 3 | task 244 | eval time = 7384.34 ms / 578 tokens ( 12.78 ms per token, 78.27 tokens per second)
0.36.936.391 I slot print_timing: id 3 | task 244 | total time = 7652.69 ms / 628 tokens
0.36.936.392 I slot print_timing: id 3 | task 244 | graphs reused = 810
0.36.936.420 I slot release: id 3 | task 244 | stop processing: n_tokens = 669, truncated = 0
0.36.936.430 I srv update_slots: all slots are idle
0.36.969.618 I srv operator(): Chat format: peg-native
0.36.972.470 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.608 (> 0.100 thold), f_keep = 0.067
0.36.972.472 I srv get_availabl: updating prompt cache
0.36.972.521 W srv prompt_save: - saving prompt with length 669, total state size = 51.545 MiB (draft: 0.000 MiB)
0.37.062.751 I srv load: - looking for better prompt, base f_keep = 0.067, sim = 0.608
0.37.062.760 I srv update: - cache state: 3 prompts, 291.854 MiB (limits: 8192.000 MiB, 24576 tokens, 29388 est)
0.37.062.761 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
0.37.062.761 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
0.37.062.764 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
0.37.062.765 I srv get_availabl: prompt cache update took 90.29 ms
0.37.063.656 I slot launch_slot_: id 3 | task 824 | processing task, is_child = 0
0.37.063.658 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.37.063.659 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.37.063.674 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.37.063.674 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.37.063.687 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.37.063.687 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.37.063.708 I slot update_slots: id 3 | task 824 | Checking checkpoint with [41, 41] against 45...
0.37.075.617 W slot update_slots: id 3 | task 824 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 47.618 MiB)
0.40.255.378 I slot print_timing: id 3 | task 824 | n_decoded = 234, tg = 77.79 t/s, tg_3s = 77.79 t/s
0.42.803.426 I slot print_timing: id 3 | task 824 | prompt eval time = 183.57 ms / 32 tokens ( 5.74 ms per token, 174.32 tokens per second)
0.42.803.430 I slot print_timing: id 3 | task 824 | eval time = 5556.15 ms / 430 tokens ( 12.92 ms per token, 77.39 tokens per second)
0.42.803.430 I slot print_timing: id 3 | task 824 | total time = 5739.72 ms / 462 tokens
0.42.803.431 I slot print_timing: id 3 | task 824 | graphs reused = 1237
0.42.803.459 I slot release: id 3 | task 824 | stop processing: n_tokens = 503, truncated = 0
0.42.803.470 I srv update_slots: all slots are idle
0.42.836.766 I srv operator(): Chat format: peg-native
0.42.839.591 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.643 (> 0.100 thold), f_keep = 0.089
0.42.839.593 I srv get_availabl: updating prompt cache
0.42.839.636 W srv prompt_save: - saving prompt with length 503, total state size = 50.571 MiB (draft: 0.000 MiB)
0.42.917.490 I srv load: - looking for better prompt, base f_keep = 0.089, sim = 0.643
0.42.917.498 I srv update: - cache state: 4 prompts, 390.042 MiB (limits: 8192.000 MiB, 24576 tokens, 32554 est)
0.42.917.499 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
0.42.917.500 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
0.42.917.501 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
0.42.917.502 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
0.42.917.505 I srv get_availabl: prompt cache update took 77.91 ms
0.42.918.404 I slot launch_slot_: id 3 | task 1256 | processing task, is_child = 0
0.42.918.406 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.42.918.407 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.42.918.423 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.42.918.424 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.42.918.436 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.42.918.437 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.42.918.456 I slot update_slots: id 3 | task 1256 | Checking checkpoint with [41, 41] against 45...
0.42.930.315 W slot update_slots: id 3 | task 1256 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 47.618 MiB)
0.46.083.462 I slot print_timing: id 3 | task 1256 | n_decoded = 232, tg = 77.14 t/s, tg_3s = 77.14 t/s
0.46.265.450 I slot print_timing: id 3 | task 1256 | prompt eval time = 157.67 ms / 28 tokens ( 5.63 ms per token, 177.58 tokens per second)
0.46.265.454 I slot print_timing: id 3 | task 1256 | eval time = 3189.32 ms / 246 tokens ( 12.96 ms per token, 77.13 tokens per second)
0.46.265.454 I slot print_timing: id 3 | task 1256 | total time = 3346.99 ms / 274 tokens
0.46.265.455 I slot print_timing: id 3 | task 1256 | graphs reused = 1480
0.46.265.484 I slot release: id 3 | task 1256 | stop processing: n_tokens = 315, truncated = 0
0.46.265.491 I srv update_slots: all slots are idle
0.46.298.891 I srv operator(): Chat format: peg-native
0.46.301.746 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.500 (> 0.100 thold), f_keep = 0.143
0.46.301.748 I srv get_availabl: updating prompt cache
0.46.301.790 W srv prompt_save: - saving prompt with length 315, total state size = 49.467 MiB (draft: 0.000 MiB)
0.46.389.544 I srv load: - looking for better prompt, base f_keep = 0.143, sim = 0.500
0.46.389.551 I srv update: - cache state: 5 prompts, 487.127 MiB (limits: 8192.000 MiB, 24576 tokens, 31363 est)
0.46.389.553 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
0.46.389.554 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
0.46.389.555 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
0.46.389.555 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
0.46.389.556 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
0.46.389.557 I srv get_availabl: prompt cache update took 87.81 ms
0.46.390.440 I slot launch_slot_: id 3 | task 1504 | processing task, is_child = 0
0.46.390.442 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.46.390.443 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.46.390.459 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.46.390.459 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.46.390.472 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.46.390.472 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.46.390.493 I slot update_slots: id 3 | task 1504 | Checking checkpoint with [41, 41] against 45...
0.46.402.336 W slot update_slots: id 3 | task 1504 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 47.618 MiB)
0.49.659.071 I slot print_timing: id 3 | task 1504 | n_decoded = 238, tg = 79.16 t/s, tg_3s = 79.16 t/s
0.50.046.526 I slot print_timing: id 3 | task 1504 | prompt eval time = 261.87 ms / 48 tokens ( 5.46 ms per token, 183.30 tokens per second)
0.50.046.530 I slot print_timing: id 3 | task 1504 | eval time = 3394.16 ms / 269 tokens ( 12.62 ms per token, 79.25 tokens per second)
0.50.046.531 I slot print_timing: id 3 | task 1504 | total time = 3656.03 ms / 317 tokens
0.50.046.531 I slot print_timing: id 3 | task 1504 | graphs reused = 1746
0.50.046.561 I slot release: id 3 | task 1504 | stop processing: n_tokens = 358, truncated = 0
0.50.046.570 I srv update_slots: all slots are idle
0.50.079.839 I srv operator(): Chat format: peg-native
0.50.082.677 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.523 (> 0.100 thold), f_keep = 0.126
0.50.082.680 I srv get_availabl: updating prompt cache
0.50.082.716 W srv prompt_save: - saving prompt with length 358, total state size = 49.720 MiB (draft: 0.000 MiB)
0.50.168.816 I srv load: - looking for better prompt, base f_keep = 0.126, sim = 0.523
0.50.168.823 I srv update: - cache state: 6 prompts, 584.465 MiB (limits: 8192.000 MiB, 24576 tokens, 31158 est)
0.50.168.824 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
0.50.168.825 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
0.50.168.826 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
0.50.168.829 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
0.50.168.830 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
0.50.168.830 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
0.50.168.831 I srv get_availabl: prompt cache update took 86.15 ms
0.50.169.715 I slot launch_slot_: id 3 | task 1775 | processing task, is_child = 0
0.50.169.717 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.50.169.717 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.50.169.733 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.50.169.733 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.50.169.745 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.50.169.746 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.50.169.766 I slot update_slots: id 3 | task 1775 | Checking checkpoint with [41, 41] against 45...
0.50.181.776 W slot update_slots: id 3 | task 1775 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 47.618 MiB)
0.53.394.111 I slot print_timing: id 3 | task 1775 | n_decoded = 240, tg = 80.00 t/s, tg_3s = 80.00 t/s
0.56.401.601 I slot print_timing: id 3 | task 1775 | n_decoded = 480, tg = 79.90 t/s, tg_3s = 79.80 t/s
0.59.411.206 I slot print_timing: id 3 | task 1775 | n_decoded = 718, tg = 79.63 t/s, tg_3s = 79.08 t/s
1.01.568.217 I slot print_timing: id 3 | task 1775 | prompt eval time = 224.22 ms / 44 tokens ( 5.10 ms per token, 196.23 tokens per second)
1.01.568.220 I slot print_timing: id 3 | task 1775 | eval time = 11174.22 ms / 888 tokens ( 12.58 ms per token, 79.47 tokens per second)
1.01.568.221 I slot print_timing: id 3 | task 1775 | total time = 11398.45 ms / 932 tokens
1.01.568.221 I slot print_timing: id 3 | task 1775 | graphs reused = 2629
1.01.568.249 I slot release: id 3 | task 1775 | stop processing: n_tokens = 973, truncated = 0
1.01.568.259 I srv update_slots: all slots are idle
1.01.602.092 I srv operator(): Chat format: peg-native
1.01.604.938 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.625 (> 0.100 thold), f_keep = 0.046
1.01.604.940 I srv get_availabl: updating prompt cache
1.01.604.996 W srv prompt_save: - saving prompt with length 973, total state size = 53.330 MiB (draft: 0.000 MiB)
1.01.693.793 I srv load: - looking for better prompt, base f_keep = 0.046, sim = 0.625
1.01.693.801 I srv update: - cache state: 7 prompts, 685.413 MiB (limits: 8192.000 MiB, 24576 tokens, 38198 est)
1.01.693.802 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
1.01.693.804 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
1.01.693.805 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
1.01.693.805 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
1.01.693.806 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
1.01.693.806 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
1.01.693.807 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
1.01.693.808 I srv get_availabl: prompt cache update took 88.87 ms
1.01.694.686 I slot launch_slot_: id 3 | task 2665 | processing task, is_child = 0
1.01.694.689 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.01.694.690 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.01.694.707 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.01.694.707 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.01.694.720 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.01.694.720 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
1.01.694.741 I slot update_slots: id 3 | task 2665 | Checking checkpoint with [41, 41] against 45...
1.01.706.947 W slot update_slots: id 3 | task 2665 | restored context checkpoint (pos_min = 41, pos_max = 41, n_tokens = 42, n_past = 42, size = 47.618 MiB)
1.03.771.595 I slot print_timing: id 3 | task 2665 | prompt eval time = 179.66 ms / 30 tokens ( 5.99 ms per token, 166.99 tokens per second)
1.03.771.598 I slot print_timing: id 3 | task 2665 | eval time = 1897.19 ms / 151 tokens ( 12.56 ms per token, 79.59 tokens per second)
1.03.771.599 I slot print_timing: id 3 | task 2665 | total time = 2076.85 ms / 181 tokens
1.03.771.600 I slot print_timing: id 3 | task 2665 | graphs reused = 2778
1.03.771.629 I slot release: id 3 | task 2665 | stop processing: n_tokens = 222, truncated = 0
1.03.771.636 I srv update_slots: all slots are idle
1.03.805.932 I srv operator(): Chat format: peg-native
1.03.808.801 I slot get_availabl: id 2 | task -1 | selected slot by LRU, t_last = -1
1.03.808.802 I srv get_availabl: updating prompt cache
1.03.808.804 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
1.03.808.806 I srv update: - cache state: 7 prompts, 685.413 MiB (limits: 8192.000 MiB, 24576 tokens, 38198 est)
1.03.808.807 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
1.03.808.808 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
1.03.808.811 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
1.03.808.812 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
1.03.808.813 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
1.03.808.813 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
1.03.808.814 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
1.03.808.814 I srv get_availabl: prompt cache update took 0.01 ms
1.03.809.758 I slot launch_slot_: id 2 | task 2818 | processing task, is_child = 0
1.03.809.760 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.03.809.761 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.03.809.779 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.03.809.779 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.03.809.793 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
1.03.809.828 W srv prompt_save: - saving prompt with length 222, total state size = 48.921 MiB (draft: 0.000 MiB)
1.03.873.514 I srv update: - cache state: 8 prompts, 781.952 MiB (limits: 8192.000 MiB, 24576 tokens, 35808 est)
1.03.873.518 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
1.03.873.519 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
1.03.873.519 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
1.03.873.520 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
1.03.873.520 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
1.03.873.521 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
1.03.873.521 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
1.03.873.522 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
1.03.873.524 I slot prompt_clear: id 3 | task -1 | clearing prompt with 222 tokens
1.04.240.125 I slot create_check: id 2 | task 2818 | created context checkpoint 1 of 32 (pos_min = 59, pos_max = 59, n_tokens = 60, size = 47.618 MiB)
1.07.450.638 I slot print_timing: id 2 | task 2818 | n_decoded = 237, tg = 78.96 t/s, tg_3s = 78.96 t/s
1.10.451.240 I slot print_timing: id 2 | task 2818 | n_decoded = 471, tg = 78.47 t/s, tg_3s = 77.98 t/s
1.13.459.119 I slot print_timing: id 2 | task 2818 | n_decoded = 702, tg = 77.91 t/s, tg_3s = 76.80 t/s
1.16.466.177 I slot print_timing: id 2 | task 2818 | n_decoded = 933, tg = 77.64 t/s, tg_3s = 76.82 t/s
1.19.469.190 I slot print_timing: id 2 | task 2818 | n_decoded = 1164, tg = 77.50 t/s, tg_3s = 76.92 t/s
1.22.479.240 I slot print_timing: id 2 | task 2818 | n_decoded = 1395, tg = 77.37 t/s, tg_3s = 76.74 t/s
1.25.487.265 I slot print_timing: id 2 | task 2818 | n_decoded = 1626, tg = 77.29 t/s, tg_3s = 76.79 t/s
1.28.496.300 I slot print_timing: id 2 | task 2818 | n_decoded = 1857, tg = 77.22 t/s, tg_3s = 76.77 t/s
1.29.549.330 I slot print_timing: id 2 | task 2818 | prompt eval time = 575.37 ms / 101 tokens ( 5.70 ms per token, 175.54 tokens per second)
1.29.549.334 I slot print_timing: id 2 | task 2818 | eval time = 25100.35 ms / 1938 tokens ( 12.95 ms per token, 77.21 tokens per second)
1.29.549.334 I slot print_timing: id 2 | task 2818 | total time = 25675.72 ms / 2039 tokens
1.29.549.335 I slot print_timing: id 2 | task 2818 | graphs reused = 4707
1.29.549.368 I slot release: id 2 | task 2818 | stop processing: n_tokens = 2038, truncated = 0
1.29.549.378 I srv update_slots: all slots are idle
1.29.583.115 I srv operator(): Chat format: peg-native
1.29.585.940 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.500 (> 0.100 thold), f_keep = 0.031
1.29.585.942 I srv get_availabl: updating prompt cache
1.29.586.039 W srv prompt_save: - saving prompt with length 2038, total state size = 59.583 MiB (draft: 0.000 MiB)
1.29.657.741 I srv load: - looking for better prompt, base f_keep = 0.031, sim = 0.500
1.29.657.748 I srv update: - cache state: 9 prompts, 889.152 MiB (limits: 8192.000 MiB, 24576 tokens, 50267 est)
1.29.657.749 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
1.29.657.750 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
1.29.657.750 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
1.29.657.751 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
1.29.657.751 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
1.29.657.752 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
1.29.657.756 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
1.29.657.757 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
1.29.657.757 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
1.29.657.758 I srv get_availabl: prompt cache update took 71.81 ms
1.29.658.640 I slot launch_slot_: id 2 | task 4759 | processing task, is_child = 0
1.29.658.642 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.29.658.643 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.29.658.666 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.29.658.666 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.29.658.680 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
1.29.658.680 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
1.29.658.700 I slot update_slots: id 2 | task 4759 | Checking checkpoint with [59, 59] against 63...
1.29.670.737 W slot update_slots: id 2 | task 4759 | restored context checkpoint (pos_min = 59, pos_max = 59, n_tokens = 60, n_past = 60, size = 47.618 MiB)
1.33.057.305 I slot print_timing: id 2 | task 4759 | n_decoded = 232, tg = 77.04 t/s, tg_3s = 77.04 t/s
1.36.065.333 I slot print_timing: id 2 | task 4759 | n_decoded = 463, tg = 76.92 t/s, tg_3s = 76.79 t/s
1.38.616.375 I slot print_timing: id 2 | task 4759 | prompt eval time = 387.30 ms / 66 tokens ( 5.87 ms per token, 170.41 tokens per second)
1.38.616.379 I slot print_timing: id 2 | task 4759 | eval time = 8570.36 ms / 659 tokens ( 13.01 ms per token, 76.89 tokens per second)
1.38.616.379 I slot print_timing: id 2 | task 4759 | total time = 8957.67 ms / 725 tokens
1.38.616.380 I slot print_timing: id 2 | task 4759 | graphs reused = 5361
1.38.616.413 I slot release: id 2 | task 4759 | stop processing: n_tokens = 784, truncated = 0
1.38.616.422 I srv update_slots: all slots are idle
1.38.650.098 I srv operator(): Chat format: peg-native
1.38.652.960 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.508 (> 0.100 thold), f_keep = 0.080
1.38.652.962 I srv get_availabl: updating prompt cache
1.38.653.021 W srv prompt_save: - saving prompt with length 784, total state size = 52.221 MiB (draft: 0.000 MiB)
1.38.716.445 I srv load: - looking for better prompt, base f_keep = 0.080, sim = 0.508
1.38.716.454 I srv update: - cache state: 10 prompts, 988.990 MiB (limits: 8192.000 MiB, 24576 tokens, 51687 est)
1.38.716.455 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
1.38.716.455 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
1.38.716.456 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
1.38.716.457 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
1.38.716.457 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
1.38.716.458 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
1.38.716.462 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
1.38.716.462 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
1.38.716.463 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
1.38.716.463 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
1.38.716.465 I srv get_availabl: prompt cache update took 63.50 ms
1.38.717.371 I slot launch_slot_: id 2 | task 5420 | processing task, is_child = 0
1.38.717.373 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.38.717.374 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
1.38.717.390 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
1.38.717.390 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
1.38.717.403 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
1.38.717.403 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
1.38.717.423 I slot update_slots: id 2 | task 5420 | Checking checkpoint with [59, 59] against 63...
1.38.729.386 W slot update_slots: id 2 | task 5420 | restored context checkpoint (pos_min = 59, pos_max = 59, n_tokens = 60, n_past = 60, size = 47.618 MiB)
1.42.085.649 I slot print_timing: id 2 | task 5420 | n_decoded = 234, tg = 77.91 t/s, tg_3s = 77.91 t/s
1.45.090.700 I slot print_timing: id 2 | task 5420 | n_decoded = 474, tg = 78.89 t/s, tg_3s = 79.87 t/s
1.48.093.204 I slot print_timing: id 2 | task 5420 | n_decoded = 713, tg = 79.12 t/s, tg_3s = 79.60 t/s
1.51.093.336 I slot print_timing: id 2 | task 5420 | n_decoded = 950, tg = 79.09 t/s, tg_3s = 79.00 t/s
1.54.100.248 I slot print_timing: id 2 | task 5420 | n_decoded = 1187, tg = 79.04 t/s, tg_3s = 78.82 t/s
1.57.108.253 I slot print_timing: id 2 | task 5420 | n_decoded = 1422, tg = 78.89 t/s, tg_3s = 78.12 t/s
2.00.118.260 I slot print_timing: id 2 | task 5420 | n_decoded = 1655, tg = 78.67 t/s, tg_3s = 77.41 t/s
2.03.126.307 I slot print_timing: id 2 | task 5420 | n_decoded = 1886, tg = 78.44 t/s, tg_3s = 76.79 t/s
2.06.132.356 I slot print_timing: id 2 | task 5420 | n_decoded = 2117, tg = 78.26 t/s, tg_3s = 76.85 t/s
2.09.140.375 I slot print_timing: id 2 | task 5420 | n_decoded = 2348, tg = 78.11 t/s, tg_3s = 76.79 t/s
2.12.149.416 I slot print_timing: id 2 | task 5420 | n_decoded = 2579, tg = 77.99 t/s, tg_3s = 76.77 t/s
2.12.955.453 I slot print_timing: id 2 | task 5420 | prompt eval time = 364.64 ms / 64 tokens ( 5.70 ms per token, 175.52 tokens per second)
2.12.955.457 I slot print_timing: id 2 | task 5420 | eval time = 33873.39 ms / 2641 tokens ( 12.83 ms per token, 77.97 tokens per second)
2.12.955.458 I slot print_timing: id 2 | task 5420 | total time = 34238.03 ms / 2705 tokens
2.12.955.459 I slot print_timing: id 2 | task 5420 | graphs reused = 7990
2.12.955.493 I slot release: id 2 | task 5420 | stop processing: n_tokens = 2764, truncated = 0
2.12.955.514 I srv update_slots: all slots are idle
2.12.989.473 I srv operator(): Chat format: peg-native
2.12.992.357 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.470 (> 0.100 thold), f_keep = 0.023
2.12.992.359 I srv get_availabl: updating prompt cache
2.12.992.459 W srv prompt_save: - saving prompt with length 2764, total state size = 63.845 MiB (draft: 0.000 MiB)
2.13.085.004 I srv load: - looking for better prompt, base f_keep = 0.023, sim = 0.470
2.13.085.013 I srv update: - cache state: 11 prompts, 1100.453 MiB (limits: 8192.000 MiB, 24576 tokens, 67027 est)
2.13.085.013 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
2.13.085.014 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
2.13.085.015 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
2.13.085.016 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
2.13.085.020 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
2.13.085.020 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
2.13.085.021 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
2.13.085.021 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
2.13.085.022 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
2.13.085.022 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
2.13.085.025 I srv update: - prompt 0x5653ddc6df80: 2764 tokens, checkpoints: 1, 111.463 MiB
2.13.085.026 I srv get_availabl: prompt cache update took 92.67 ms
2.13.085.898 I slot launch_slot_: id 2 | task 8063 | processing task, is_child = 0
2.13.085.901 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.13.085.901 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.13.085.920 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.13.085.921 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.13.085.934 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
2.13.085.935 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
2.13.085.955 I slot update_slots: id 2 | task 8063 | Checking checkpoint with [59, 59] against 63...
2.13.097.928 W slot update_slots: id 2 | task 8063 | restored context checkpoint (pos_min = 59, pos_max = 59, n_tokens = 60, n_past = 60, size = 47.618 MiB)
2.16.361.359 I slot print_timing: id 2 | task 8063 | n_decoded = 232, tg = 77.01 t/s, tg_3s = 77.01 t/s
2.19.370.382 I slot print_timing: id 2 | task 8063 | n_decoded = 463, tg = 76.89 t/s, tg_3s = 76.77 t/s
2.22.378.444 I slot print_timing: id 2 | task 8063 | n_decoded = 694, tg = 76.86 t/s, tg_3s = 76.79 t/s
2.25.386.470 I slot print_timing: id 2 | task 8063 | n_decoded = 925, tg = 76.84 t/s, tg_3s = 76.79 t/s
2.28.398.994 I slot print_timing: id 2 | task 8063 | n_decoded = 1158, tg = 76.94 t/s, tg_3s = 77.34 t/s
2.31.405.290 I slot print_timing: id 2 | task 8063 | n_decoded = 1397, tg = 77.37 t/s, tg_3s = 79.50 t/s
2.32.103.954 I slot print_timing: id 2 | task 8063 | prompt eval time = 262.75 ms / 74 tokens ( 3.55 ms per token, 281.64 tokens per second)
2.32.103.957 I slot print_timing: id 2 | task 8063 | eval time = 18755.24 ms / 1452 tokens ( 12.92 ms per token, 77.42 tokens per second)
2.32.103.958 I slot print_timing: id 2 | task 8063 | total time = 19017.99 ms / 1526 tokens
2.32.103.958 I slot print_timing: id 2 | task 8063 | graphs reused = 9434
2.32.103.994 I slot release: id 2 | task 8063 | stop processing: n_tokens = 1585, truncated = 0
2.32.104.007 I srv update_slots: all slots are idle
2.32.138.596 I srv operator(): Chat format: peg-native
2.32.141.449 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.600 (> 0.100 thold), f_keep = 0.040
2.32.141.451 I srv get_availabl: updating prompt cache
2.32.141.533 W srv prompt_save: - saving prompt with length 1585, total state size = 56.923 MiB (draft: 0.000 MiB)
2.32.221.732 I srv load: - looking for better prompt, base f_keep = 0.040, sim = 0.600
2.32.221.742 I srv update: - cache state: 12 prompts, 1204.994 MiB (limits: 8192.000 MiB, 24576 tokens, 71988 est)
2.32.221.745 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
2.32.221.746 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
2.32.221.747 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
2.32.221.747 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
2.32.221.748 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
2.32.221.750 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
2.32.221.751 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
2.32.221.751 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
2.32.221.752 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
2.32.221.753 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
2.32.221.754 I srv update: - prompt 0x5653ddc6df80: 2764 tokens, checkpoints: 1, 111.463 MiB
2.32.221.755 I srv update: - prompt 0x5653ddac56c0: 1585 tokens, checkpoints: 1, 104.541 MiB
2.32.221.756 I srv get_availabl: prompt cache update took 80.30 ms
2.32.222.643 I slot launch_slot_: id 2 | task 9517 | processing task, is_child = 0
2.32.222.645 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.32.222.646 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.32.222.666 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.32.222.666 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.32.222.679 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
2.32.222.679 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
2.32.222.699 I slot update_slots: id 2 | task 9517 | Checking checkpoint with [59, 59] against 63...
2.32.234.803 W slot update_slots: id 2 | task 9517 | restored context checkpoint (pos_min = 59, pos_max = 59, n_tokens = 60, n_past = 60, size = 47.618 MiB)
2.35.481.002 I slot print_timing: id 2 | task 9517 | n_decoded = 239, tg = 79.64 t/s, tg_3s = 79.64 t/s
2.38.492.410 I slot print_timing: id 2 | task 9517 | n_decoded = 478, tg = 79.50 t/s, tg_3s = 79.36 t/s
2.41.492.619 I slot print_timing: id 2 | task 9517 | n_decoded = 716, tg = 79.44 t/s, tg_3s = 79.33 t/s
2.44.504.131 I slot print_timing: id 2 | task 9517 | n_decoded = 952, tg = 79.17 t/s, tg_3s = 78.37 t/s
2.44.669.110 I slot print_timing: id 2 | task 9517 | prompt eval time = 257.19 ms / 45 tokens ( 5.72 ms per token, 174.97 tokens per second)
2.44.669.113 I slot print_timing: id 2 | task 9517 | eval time = 12189.22 ms / 965 tokens ( 12.63 ms per token, 79.17 tokens per second)
2.44.669.114 I slot print_timing: id 2 | task 9517 | total time = 12446.41 ms / 1010 tokens
2.44.669.115 I slot print_timing: id 2 | task 9517 | graphs reused = 10393
2.44.669.149 I slot release: id 2 | task 9517 | stop processing: n_tokens = 1069, truncated = 0
2.44.669.167 I srv update_slots: all slots are idle
2.44.703.016 I srv operator(): Chat format: peg-native
2.44.705.902 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.525 (> 0.100 thold), f_keep = 0.059
2.44.705.904 I srv get_availabl: updating prompt cache
2.44.705.971 W srv prompt_save: - saving prompt with length 1069, total state size = 53.894 MiB (draft: 0.000 MiB)
2.44.770.188 I srv load: - looking for better prompt, base f_keep = 0.059, sim = 0.525
2.44.770.197 I srv update: - cache state: 13 prompts, 1306.505 MiB (limits: 8192.000 MiB, 24576 tokens, 73097 est)
2.44.770.200 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
2.44.770.201 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
2.44.770.201 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
2.44.770.202 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
2.44.770.203 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
2.44.770.204 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
2.44.770.204 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
2.44.770.205 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
2.44.770.206 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
2.44.770.206 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
2.44.770.207 I srv update: - prompt 0x5653ddc6df80: 2764 tokens, checkpoints: 1, 111.463 MiB
2.44.770.207 I srv update: - prompt 0x5653ddac56c0: 1585 tokens, checkpoints: 1, 104.541 MiB
2.44.770.208 I srv update: - prompt 0x5653ddc5e170: 1069 tokens, checkpoints: 1, 101.512 MiB
2.44.770.209 I srv get_availabl: prompt cache update took 64.30 ms
2.44.771.103 I slot launch_slot_: id 2 | task 10484 | processing task, is_child = 0
2.44.771.105 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.44.771.106 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.44.771.129 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
2.44.771.130 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
2.44.771.149 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
2.44.771.150 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
2.44.771.186 I slot update_slots: id 2 | task 10484 | Checking checkpoint with [59, 59] against 63...
2.44.783.238 W slot update_slots: id 2 | task 10484 | restored context checkpoint (pos_min = 59, pos_max = 59, n_tokens = 60, n_past = 60, size = 47.618 MiB)
2.48.124.357 I slot print_timing: id 2 | task 10484 | n_decoded = 237, tg = 78.90 t/s, tg_3s = 78.90 t/s
2.51.131.075 I slot print_timing: id 2 | task 10484 | n_decoded = 473, tg = 78.70 t/s, tg_3s = 78.49 t/s
2.54.134.120 I slot print_timing: id 2 | task 10484 | n_decoded = 706, tg = 78.33 t/s, tg_3s = 77.59 t/s
2.56.358.168 I slot print_timing: id 2 | task 10484 | prompt eval time = 349.39 ms / 60 tokens ( 5.82 ms per token, 171.73 tokens per second)
2.56.358.171 I slot print_timing: id 2 | task 10484 | eval time = 11237.58 ms / 877 tokens ( 12.81 ms per token, 78.04 tokens per second)
2.56.358.172 I slot print_timing: id 2 | task 10484 | total time = 11586.97 ms / 937 tokens
2.56.358.172 I slot print_timing: id 2 | task 10484 | graphs reused = 11265
2.56.358.208 I slot release: id 2 | task 10484 | stop processing: n_tokens = 996, truncated = 0
2.56.358.220 I srv update_slots: all slots are idle
2.56.392.002 I srv operator(): Chat format: peg-native
2.56.394.883 I slot get_availabl: id 1 | task -1 | selected slot by LRU, t_last = -1
2.56.394.885 I srv get_availabl: updating prompt cache
2.56.394.887 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
2.56.394.891 I srv update: - cache state: 13 prompts, 1306.505 MiB (limits: 8192.000 MiB, 24576 tokens, 73097 est)
2.56.394.892 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
2.56.394.896 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
2.56.394.896 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
2.56.394.897 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
2.56.394.898 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
2.56.394.898 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
2.56.394.899 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
2.56.394.900 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
2.56.394.902 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
2.56.394.902 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
2.56.394.903 I srv update: - prompt 0x5653ddc6df80: 2764 tokens, checkpoints: 1, 111.463 MiB
2.56.394.904 I srv update: - prompt 0x5653ddac56c0: 1585 tokens, checkpoints: 1, 104.541 MiB
2.56.394.904 I srv update: - prompt 0x5653ddc5e170: 1069 tokens, checkpoints: 1, 101.512 MiB
2.56.394.905 I srv get_availabl: prompt cache update took 0.02 ms
2.56.395.922 I slot launch_slot_: id 1 | task 11363 | processing task, is_child = 0
2.56.395.924 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
2.56.395.925 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
2.56.395.944 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
2.56.396.017 W srv prompt_save: - saving prompt with length 996, total state size = 53.465 MiB (draft: 0.000 MiB)
2.56.472.337 I srv update: - cache state: 14 prompts, 1407.588 MiB (limits: 8192.000 MiB, 24576 tokens, 73644 est)
2.56.472.341 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
2.56.472.341 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
2.56.472.342 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
2.56.472.342 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
2.56.472.343 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
2.56.472.343 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
2.56.472.344 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
2.56.472.345 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
2.56.472.346 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
2.56.472.349 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
2.56.472.350 I srv update: - prompt 0x5653ddc6df80: 2764 tokens, checkpoints: 1, 111.463 MiB
2.56.472.350 I srv update: - prompt 0x5653ddac56c0: 1585 tokens, checkpoints: 1, 104.541 MiB
2.56.472.351 I srv update: - prompt 0x5653ddc5e170: 1069 tokens, checkpoints: 1, 101.512 MiB
2.56.472.351 I srv update: - prompt 0x5653ddc5d290: 996 tokens, checkpoints: 1, 101.083 MiB
2.56.472.353 I slot prompt_clear: id 2 | task -1 | clearing prompt with 996 tokens
2.56.472.557 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
2.56.472.559 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
2.56.798.932 I slot create_check: id 1 | task 11363 | created context checkpoint 1 of 32 (pos_min = 55, pos_max = 55, n_tokens = 56, size = 47.618 MiB)
3.00.067.118 I slot print_timing: id 1 | task 11363 | n_decoded = 234, tg = 77.94 t/s, tg_3s = 77.94 t/s
3.00.611.101 I slot print_timing: id 1 | task 11363 | prompt eval time = 592.14 ms / 136 tokens ( 4.35 ms per token, 229.68 tokens per second)
3.00.611.104 I slot print_timing: id 1 | task 11363 | eval time = 3546.30 ms / 276 tokens ( 12.85 ms per token, 77.83 tokens per second)
3.00.611.105 I slot print_timing: id 1 | task 11363 | total time = 4138.44 ms / 412 tokens
3.00.611.106 I slot print_timing: id 1 | task 11363 | graphs reused = 11538
3.00.611.138 I slot release: id 1 | task 11363 | stop processing: n_tokens = 411, truncated = 0
3.00.611.142 I srv update_slots: all slots are idle
3.00.707.048 I srv operator(): Chat format: peg-native
3.00.709.957 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.667 (> 0.100 thold), f_keep = 0.156
3.00.709.960 I srv get_availabl: updating prompt cache
3.00.710.010 W srv prompt_save: - saving prompt with length 411, total state size = 50.031 MiB (draft: 0.000 MiB)
3.00.778.324 I srv load: - looking for better prompt, base f_keep = 0.156, sim = 0.667
3.00.778.333 I srv update: - cache state: 15 prompts, 1505.237 MiB (limits: 8192.000 MiB, 24576 tokens, 71104 est)
3.00.778.334 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
3.00.778.335 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
3.00.778.336 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
3.00.778.340 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
3.00.778.340 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
3.00.778.341 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
3.00.778.341 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
3.00.778.342 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
3.00.778.342 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
3.00.778.345 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
3.00.778.346 I srv update: - prompt 0x5653ddc6df80: 2764 tokens, checkpoints: 1, 111.463 MiB
3.00.778.346 I srv update: - prompt 0x5653ddac56c0: 1585 tokens, checkpoints: 1, 104.541 MiB
3.00.778.347 I srv update: - prompt 0x5653ddc5e170: 1069 tokens, checkpoints: 1, 101.512 MiB
3.00.778.347 I srv update: - prompt 0x5653ddc5d290: 996 tokens, checkpoints: 1, 101.083 MiB
3.00.778.348 I srv update: - prompt 0x5653e3f4ce40: 411 tokens, checkpoints: 1, 97.649 MiB
3.00.778.351 I srv get_availabl: prompt cache update took 68.39 ms
3.00.779.257 I slot launch_slot_: id 1 | task 11642 | processing task, is_child = 0
3.00.779.259 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.00.779.260 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.00.779.275 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.00.779.275 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.00.779.288 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.00.779.288 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
3.00.779.307 I slot update_slots: id 1 | task 11642 | Checking checkpoint with [55, 55] against 64...
3.00.791.351 W slot update_slots: id 1 | task 11642 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 47.618 MiB)
3.03.994.166 I slot print_timing: id 1 | task 11642 | n_decoded = 233, tg = 77.47 t/s, tg_3s = 77.47 t/s
3.07.001.200 I slot print_timing: id 1 | task 11642 | n_decoded = 464, tg = 77.14 t/s, tg_3s = 76.82 t/s
3.08.366.206 I slot print_timing: id 1 | task 11642 | prompt eval time = 207.20 ms / 40 tokens ( 5.18 ms per token, 193.05 tokens per second)
3.08.366.209 I slot print_timing: id 1 | task 11642 | eval time = 7379.69 ms / 569 tokens ( 12.97 ms per token, 77.10 tokens per second)
3.08.366.210 I slot print_timing: id 1 | task 11642 | total time = 7586.89 ms / 609 tokens
3.08.366.211 I slot print_timing: id 1 | task 11642 | graphs reused = 12103
3.08.366.243 I slot release: id 1 | task 11642 | stop processing: n_tokens = 664, truncated = 0
3.08.366.255 I srv update_slots: all slots are idle
3.08.443.930 I srv operator(): Chat format: peg-native
3.08.446.798 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.615 (> 0.100 thold), f_keep = 0.096
3.08.446.800 I srv get_availabl: updating prompt cache
3.08.446.843 W srv prompt_save: - saving prompt with length 664, total state size = 51.516 MiB (draft: 0.000 MiB)
3.08.515.908 I srv load: - looking for better prompt, base f_keep = 0.096, sim = 0.615
3.08.515.919 I srv update: - cache state: 16 prompts, 1604.371 MiB (limits: 8192.000 MiB, 24576 tokens, 70100 est)
3.08.515.920 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
3.08.515.921 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
3.08.515.922 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
3.08.515.922 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
3.08.515.924 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
3.08.515.925 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
3.08.515.926 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
3.08.515.927 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
3.08.515.928 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
3.08.515.928 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
3.08.515.929 I srv update: - prompt 0x5653ddc6df80: 2764 tokens, checkpoints: 1, 111.463 MiB
3.08.515.929 I srv update: - prompt 0x5653ddac56c0: 1585 tokens, checkpoints: 1, 104.541 MiB
3.08.515.930 I srv update: - prompt 0x5653ddc5e170: 1069 tokens, checkpoints: 1, 101.512 MiB
3.08.515.931 I srv update: - prompt 0x5653ddc5d290: 996 tokens, checkpoints: 1, 101.083 MiB
3.08.515.931 I srv update: - prompt 0x5653e3f4ce40: 411 tokens, checkpoints: 1, 97.649 MiB
3.08.515.932 I srv update: - prompt 0x5653e3e2d170: 664 tokens, checkpoints: 1, 99.134 MiB
3.08.515.934 I srv get_availabl: prompt cache update took 69.13 ms
3.08.516.851 I slot launch_slot_: id 1 | task 12213 | processing task, is_child = 0
3.08.516.854 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.08.516.855 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.08.516.872 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.08.516.872 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.08.516.887 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.08.516.888 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
3.08.516.912 I slot update_slots: id 1 | task 12213 | Checking checkpoint with [55, 55] against 64...
3.08.529.183 W slot update_slots: id 1 | task 12213 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 47.618 MiB)
3.11.790.236 I slot print_timing: id 1 | task 12213 | n_decoded = 232, tg = 77.05 t/s, tg_3s = 77.05 t/s
3.12.947.255 I slot print_timing: id 1 | task 12213 | prompt eval time = 262.33 ms / 48 tokens ( 5.47 ms per token, 182.97 tokens per second)
3.12.947.258 I slot print_timing: id 1 | task 12213 | eval time = 4168.01 ms / 321 tokens ( 12.98 ms per token, 77.02 tokens per second)
3.12.947.259 I slot print_timing: id 1 | task 12213 | total time = 4430.34 ms / 369 tokens
3.12.947.260 I slot print_timing: id 1 | task 12213 | graphs reused = 12421
3.12.947.318 I slot release: id 1 | task 12213 | stop processing: n_tokens = 424, truncated = 0
3.12.947.328 I srv update_slots: all slots are idle
3.13.052.653 I srv operator(): Chat format: peg-native
3.13.055.542 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.474 (> 0.100 thold), f_keep = 0.151
3.13.055.544 I srv get_availabl: updating prompt cache
3.13.055.587 W srv prompt_save: - saving prompt with length 424, total state size = 50.107 MiB (draft: 0.000 MiB)
3.13.118.122 I srv load: - looking for better prompt, base f_keep = 0.151, sim = 0.474
3.13.118.130 I srv update: - cache state: 17 prompts, 1702.095 MiB (limits: 8192.000 MiB, 24576 tokens, 68116 est)
3.13.118.131 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
3.13.118.132 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
3.13.118.133 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
3.13.118.137 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
3.13.118.137 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
3.13.118.138 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
3.13.118.138 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
3.13.118.139 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
3.13.118.139 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
3.13.118.142 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
3.13.118.143 I srv update: - prompt 0x5653ddc6df80: 2764 tokens, checkpoints: 1, 111.463 MiB
3.13.118.143 I srv update: - prompt 0x5653ddac56c0: 1585 tokens, checkpoints: 1, 104.541 MiB
3.13.118.144 I srv update: - prompt 0x5653ddc5e170: 1069 tokens, checkpoints: 1, 101.512 MiB
3.13.118.144 I srv update: - prompt 0x5653ddc5d290: 996 tokens, checkpoints: 1, 101.083 MiB
3.13.118.145 I srv update: - prompt 0x5653e3f4ce40: 411 tokens, checkpoints: 1, 97.649 MiB
3.13.118.148 I srv update: - prompt 0x5653e3e2d170: 664 tokens, checkpoints: 1, 99.134 MiB
3.13.118.148 I srv update: - prompt 0x5653ddbf3fc0: 424 tokens, checkpoints: 1, 97.725 MiB
3.13.118.149 I srv get_availabl: prompt cache update took 62.60 ms
3.13.119.009 I slot launch_slot_: id 1 | task 12536 | processing task, is_child = 0
3.13.119.011 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.13.119.012 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.13.119.027 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.13.119.027 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.13.119.040 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.13.119.041 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
3.13.119.061 I slot update_slots: id 1 | task 12536 | Checking checkpoint with [55, 55] against 64...
3.13.131.202 W slot update_slots: id 1 | task 12536 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 47.618 MiB)
3.16.394.266 I slot print_timing: id 1 | task 12536 | n_decoded = 231, tg = 77.00 t/s, tg_3s = 77.00 t/s
3.19.402.321 I slot print_timing: id 1 | task 12536 | n_decoded = 462, tg = 76.90 t/s, tg_3s = 76.79 t/s
3.22.412.630 I slot print_timing: id 1 | task 12536 | n_decoded = 696, tg = 77.18 t/s, tg_3s = 77.73 t/s
3.23.424.107 I slot print_timing: id 1 | task 12536 | prompt eval time = 275.18 ms / 79 tokens ( 3.48 ms per token, 287.09 tokens per second)
3.23.424.111 I slot print_timing: id 1 | task 12536 | eval time = 10029.86 ms / 776 tokens ( 12.93 ms per token, 77.37 tokens per second)
3.23.424.112 I slot print_timing: id 1 | task 12536 | total time = 10305.04 ms / 855 tokens
3.23.424.112 I slot print_timing: id 1 | task 12536 | graphs reused = 13192
3.23.424.149 I slot release: id 1 | task 12536 | stop processing: n_tokens = 910, truncated = 0
3.23.424.165 I srv update_slots: all slots are idle
3.23.529.124 I srv operator(): Chat format: peg-native
3.23.531.997 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.587 (> 0.100 thold), f_keep = 0.070
3.23.531.999 I srv get_availabl: updating prompt cache
3.23.532.083 W srv prompt_save: - saving prompt with length 910, total state size = 52.960 MiB (draft: 0.000 MiB)
3.23.595.326 I srv load: - looking for better prompt, base f_keep = 0.070, sim = 0.587
3.23.595.338 I srv update: - cache state: 18 prompts, 1802.674 MiB (limits: 8192.000 MiB, 24576 tokens, 68451 est)
3.23.595.339 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
3.23.595.339 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
3.23.595.340 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
3.23.595.340 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
3.23.595.341 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
3.23.595.341 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
3.23.595.342 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
3.23.595.343 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
3.23.595.343 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
3.23.595.344 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
3.23.595.344 I srv update: - prompt 0x5653ddc6df80: 2764 tokens, checkpoints: 1, 111.463 MiB
3.23.595.345 I srv update: - prompt 0x5653ddac56c0: 1585 tokens, checkpoints: 1, 104.541 MiB
3.23.595.345 I srv update: - prompt 0x5653ddc5e170: 1069 tokens, checkpoints: 1, 101.512 MiB
3.23.595.346 I srv update: - prompt 0x5653ddc5d290: 996 tokens, checkpoints: 1, 101.083 MiB
3.23.595.346 I srv update: - prompt 0x5653e3f4ce40: 411 tokens, checkpoints: 1, 97.649 MiB
3.23.595.347 I srv update: - prompt 0x5653e3e2d170: 664 tokens, checkpoints: 1, 99.134 MiB
3.23.595.347 I srv update: - prompt 0x5653ddbf3fc0: 424 tokens, checkpoints: 1, 97.725 MiB
3.23.595.348 I srv update: - prompt 0x5653e3f724a0: 910 tokens, checkpoints: 1, 100.578 MiB
3.23.595.349 I srv get_availabl: prompt cache update took 63.35 ms
3.23.596.235 I slot launch_slot_: id 1 | task 13314 | processing task, is_child = 0
3.23.596.237 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.23.596.238 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.23.596.254 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.23.596.254 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.23.596.267 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.23.596.267 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
3.23.596.287 I slot update_slots: id 1 | task 13314 | Checking checkpoint with [55, 55] against 64...
3.23.608.198 W slot update_slots: id 1 | task 13314 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 47.618 MiB)
3.26.934.461 I slot print_timing: id 1 | task 13314 | n_decoded = 239, tg = 79.49 t/s, tg_3s = 79.49 t/s
3.28.264.439 I slot print_timing: id 1 | task 13314 | prompt eval time = 331.44 ms / 53 tokens ( 6.25 ms per token, 159.91 tokens per second)
3.28.264.443 I slot print_timing: id 1 | task 13314 | eval time = 4336.70 ms / 344 tokens ( 12.61 ms per token, 79.32 tokens per second)
3.28.264.443 I slot print_timing: id 1 | task 13314 | total time = 4668.15 ms / 397 tokens
3.28.264.444 I slot print_timing: id 1 | task 13314 | graphs reused = 13533
3.28.264.478 I slot release: id 1 | task 13314 | stop processing: n_tokens = 452, truncated = 0
3.28.264.487 I srv update_slots: all slots are idle
3.28.343.078 I srv operator(): Chat format: peg-native
3.28.345.955 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.509 (> 0.100 thold), f_keep = 0.131
3.28.345.957 I srv get_availabl: updating prompt cache
3.28.346.009 W srv prompt_save: - saving prompt with length 452, total state size = 50.272 MiB (draft: 0.000 MiB)
3.28.410.292 I srv load: - looking for better prompt, base f_keep = 0.131, sim = 0.509
3.28.410.303 I srv update: - cache state: 19 prompts, 1900.563 MiB (limits: 8192.000 MiB, 24576 tokens, 66874 est)
3.28.410.304 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
3.28.410.306 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
3.28.410.306 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
3.28.410.307 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
3.28.410.308 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
3.28.410.309 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
3.28.410.313 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
3.28.410.314 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
3.28.410.315 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
3.28.410.315 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
3.28.410.316 I srv update: - prompt 0x5653ddc6df80: 2764 tokens, checkpoints: 1, 111.463 MiB
3.28.410.319 I srv update: - prompt 0x5653ddac56c0: 1585 tokens, checkpoints: 1, 104.541 MiB
3.28.410.320 I srv update: - prompt 0x5653ddc5e170: 1069 tokens, checkpoints: 1, 101.512 MiB
3.28.410.321 I srv update: - prompt 0x5653ddc5d290: 996 tokens, checkpoints: 1, 101.083 MiB
3.28.410.322 I srv update: - prompt 0x5653e3f4ce40: 411 tokens, checkpoints: 1, 97.649 MiB
3.28.410.322 I srv update: - prompt 0x5653e3e2d170: 664 tokens, checkpoints: 1, 99.134 MiB
3.28.410.323 I srv update: - prompt 0x5653ddbf3fc0: 424 tokens, checkpoints: 1, 97.725 MiB
3.28.410.326 I srv update: - prompt 0x5653e3f724a0: 910 tokens, checkpoints: 1, 100.578 MiB
3.28.410.327 I srv update: - prompt 0x5653e3f4c080: 452 tokens, checkpoints: 1, 97.889 MiB
3.28.410.328 I srv get_availabl: prompt cache update took 64.37 ms
3.28.411.256 I slot launch_slot_: id 1 | task 13660 | processing task, is_child = 0
3.28.411.259 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.28.411.259 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.28.411.277 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.28.411.277 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.28.411.292 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.28.411.292 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
3.28.411.315 I slot update_slots: id 1 | task 13660 | Checking checkpoint with [55, 55] against 59...
3.28.423.324 W slot update_slots: id 1 | task 13660 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 47.618 MiB)
3.30.472.647 I slot print_timing: id 1 | task 13660 | prompt eval time = 330.02 ms / 60 tokens ( 5.50 ms per token, 181.80 tokens per second)
3.30.472.651 I slot print_timing: id 1 | task 13660 | eval time = 1731.30 ms / 138 tokens ( 12.55 ms per token, 79.71 tokens per second)
3.30.472.651 I slot print_timing: id 1 | task 13660 | total time = 2061.33 ms / 198 tokens
3.30.472.652 I slot print_timing: id 1 | task 13660 | graphs reused = 13669
3.30.472.688 I slot release: id 1 | task 13660 | stop processing: n_tokens = 253, truncated = 0
3.30.472.697 I srv update_slots: all slots are idle
3.30.506.135 I srv operator(): Chat format: peg-native
3.30.509.050 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.567 (> 0.100 thold), f_keep = 0.233
3.30.509.052 I srv get_availabl: updating prompt cache
3.30.509.101 W srv prompt_save: - saving prompt with length 253, total state size = 49.103 MiB (draft: 0.000 MiB)
3.30.572.608 I srv load: - looking for better prompt, base f_keep = 0.233, sim = 0.567
3.30.572.619 I srv update: - cache state: 20 prompts, 1997.284 MiB (limits: 8192.000 MiB, 24576 tokens, 64673 est)
3.30.572.620 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
3.30.572.621 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
3.30.572.621 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
3.30.572.622 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
3.30.572.622 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
3.30.572.623 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
3.30.572.623 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
3.30.572.624 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
3.30.572.624 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
3.30.572.625 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
3.30.572.625 I srv update: - prompt 0x5653ddc6df80: 2764 tokens, checkpoints: 1, 111.463 MiB
3.30.572.626 I srv update: - prompt 0x5653ddac56c0: 1585 tokens, checkpoints: 1, 104.541 MiB
3.30.572.626 I srv update: - prompt 0x5653ddc5e170: 1069 tokens, checkpoints: 1, 101.512 MiB
3.30.572.627 I srv update: - prompt 0x5653ddc5d290: 996 tokens, checkpoints: 1, 101.083 MiB
3.30.572.627 I srv update: - prompt 0x5653e3f4ce40: 411 tokens, checkpoints: 1, 97.649 MiB
3.30.572.628 I srv update: - prompt 0x5653e3e2d170: 664 tokens, checkpoints: 1, 99.134 MiB
3.30.572.628 I srv update: - prompt 0x5653ddbf3fc0: 424 tokens, checkpoints: 1, 97.725 MiB
3.30.572.629 I srv update: - prompt 0x5653e3f724a0: 910 tokens, checkpoints: 1, 100.578 MiB
3.30.572.629 I srv update: - prompt 0x5653e3f4c080: 452 tokens, checkpoints: 1, 97.889 MiB
3.30.572.630 I srv update: - prompt 0x5653e3e72630: 253 tokens, checkpoints: 1, 96.721 MiB
3.30.572.631 I srv get_availabl: prompt cache update took 63.58 ms
3.30.573.513 I slot launch_slot_: id 1 | task 13800 | processing task, is_child = 0
3.30.573.515 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.30.573.516 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.30.573.531 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.30.573.531 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.30.573.544 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.30.573.544 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
3.30.573.564 I slot update_slots: id 1 | task 13800 | Checking checkpoint with [55, 55] against 59...
3.30.585.453 W slot update_slots: id 1 | task 13800 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 47.618 MiB)
3.33.834.302 I slot print_timing: id 1 | task 13800 | n_decoded = 237, tg = 78.95 t/s, tg_3s = 78.95 t/s
3.35.628.132 I slot print_timing: id 1 | task 13800 | prompt eval time = 258.98 ms / 48 tokens ( 5.40 ms per token, 185.34 tokens per second)
3.35.628.136 I slot print_timing: id 1 | task 13800 | eval time = 4795.59 ms / 378 tokens ( 12.69 ms per token, 78.82 tokens per second)
3.35.628.137 I slot print_timing: id 1 | task 13800 | total time = 5054.56 ms / 426 tokens
3.35.628.138 I slot print_timing: id 1 | task 13800 | graphs reused = 14044
3.35.628.180 I slot release: id 1 | task 13800 | stop processing: n_tokens = 481, truncated = 0
3.35.628.190 I srv update_slots: all slots are idle
3.35.720.328 I srv operator(): Chat format: peg-native
3.35.723.192 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.615 (> 0.100 thold), f_keep = 0.133
3.35.723.193 I srv get_availabl: updating prompt cache
3.35.723.238 W srv prompt_save: - saving prompt with length 481, total state size = 50.442 MiB (draft: 0.000 MiB)
3.35.787.488 I srv load: - looking for better prompt, base f_keep = 0.133, sim = 0.615
3.35.787.501 I srv update: - cache state: 21 prompts, 2095.343 MiB (limits: 8192.000 MiB, 24576 tokens, 63527 est)
3.35.787.503 I srv update: - prompt 0x5653ddb5ee90: 230 tokens, checkpoints: 1, 96.586 MiB
3.35.787.508 I srv update: - prompt 0x5653ddcb7c40: 148 tokens, checkpoints: 1, 96.105 MiB
3.35.787.508 I srv update: - prompt 0x5653e3e32a50: 669 tokens, checkpoints: 1, 99.163 MiB
3.35.787.509 I srv update: - prompt 0x5653ddb17a20: 503 tokens, checkpoints: 1, 98.189 MiB
3.35.787.509 I srv update: - prompt 0x5653ddbf8330: 315 tokens, checkpoints: 1, 97.085 MiB
3.35.787.510 I srv update: - prompt 0x5653e3e32bd0: 358 tokens, checkpoints: 1, 97.337 MiB
3.35.787.511 I srv update: - prompt 0x5653ddbd0440: 973 tokens, checkpoints: 1, 100.948 MiB
3.35.787.512 I srv update: - prompt 0x5653e3fa0c70: 222 tokens, checkpoints: 1, 96.539 MiB
3.35.787.512 I srv update: - prompt 0x5653ddcac250: 2038 tokens, checkpoints: 1, 107.200 MiB
3.35.787.513 I srv update: - prompt 0x5653ddb47f40: 784 tokens, checkpoints: 1, 99.838 MiB
3.35.787.513 I srv update: - prompt 0x5653ddc6df80: 2764 tokens, checkpoints: 1, 111.463 MiB
3.35.787.514 I srv update: - prompt 0x5653ddac56c0: 1585 tokens, checkpoints: 1, 104.541 MiB
3.35.787.514 I srv update: - prompt 0x5653ddc5e170: 1069 tokens, checkpoints: 1, 101.512 MiB
3.35.787.515 I srv update: - prompt 0x5653ddc5d290: 996 tokens, checkpoints: 1, 101.083 MiB
3.35.787.515 I srv update: - prompt 0x5653e3f4ce40: 411 tokens, checkpoints: 1, 97.649 MiB
3.35.787.516 I srv update: - prompt 0x5653e3e2d170: 664 tokens, checkpoints: 1, 99.134 MiB
3.35.787.516 I srv update: - prompt 0x5653ddbf3fc0: 424 tokens, checkpoints: 1, 97.725 MiB
3.35.787.517 I srv update: - prompt 0x5653e3f724a0: 910 tokens, checkpoints: 1, 100.578 MiB
3.35.787.517 I srv update: - prompt 0x5653e3f4c080: 452 tokens, checkpoints: 1, 97.889 MiB
3.35.787.518 I srv update: - prompt 0x5653e3e72630: 253 tokens, checkpoints: 1, 96.721 MiB
3.35.787.518 I srv update: - prompt 0x5653ddc21380: 481 tokens, checkpoints: 1, 98.059 MiB
3.35.787.520 I srv get_availabl: prompt cache update took 64.33 ms
3.35.788.430 I slot launch_slot_: id 1 | task 14180 | processing task, is_child = 0
3.35.788.432 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
3.35.788.433 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
3.35.788.449 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
3.35.788.449 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
3.35.788.466 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
3.35.788.467 I slot prompt_clear: id 3 | task -1 | clearing prompt with 0 tokens
3.35.788.492 I slot update_slots: id 1 | task 14180 | Checking checkpoint with [55, 55] against 64...
3.35.800.383 W slot update_slots: id 1 | task 14180 | restored context checkpoint (pos_min = 55, pos_max = 55, n_tokens = 56, n_past = 56, size = 47.618 MiB)
3.39.055.518 I slot print_timing: id 1 | task 14180 | n_decoded = 236, tg = 78.52 t/s, tg_3s = 78.52 t/s
3.40.033.277 I slot print_timing: id 1 | task 14180 | prompt eval time = 261.31 ms / 48 tokens ( 5.44 ms per token, 183.69 tokens per second)
3.40.033.281 I slot print_timing: id 1 | task 14180 | eval time = 3983.47 ms / 312 tokens ( 12.77 ms per token, 78.32 tokens per second)
3.40.033.281 I slot print_timing: id 1 | task 14180 | total time = 4244.78 ms / 360 tokens
3.40.033.282 I slot print_timing: id 1 | task 14180 | graphs reused = 14353
3.40.033.312 I slot release: id 1 | task 14180 | stop processing: n_tokens = 415, truncated = 0
3.40.033.320 I srv update_slots: all slots are idle
3.40.094.094 I srv operator(): operator(): cleaning up before exit...