qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results/nemotron-cascade-30b.server.log

1119 lines
108 KiB
Plaintext
Raw Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.065.379 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.065.383 I device_info:
0.00.065.401 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.405 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.411 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.065.470 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.065.473 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.065.513 I srv init: running without SSL
0.00.065.572 I srv init: using 23 threads for HTTP server
0.00.065.905 I srv start: binding port with default address family
0.00.067.120 I srv llama_server: loading model
0.00.067.127 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/nvidia_Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf'
0.00.067.173 I common_init_result: fitting params to device memory ...
0.00.067.175 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.15.860.594 W llama_context: n_ctx_seq (8192) < n_ctx_train (1048576) -- the full capacity of the model will not be utilized
0.15.963.830 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.16.152.338 I srv load_model: initializing slots, n_slots = 4
0.16.183.748 W srv load_model: speculative decoding will use checkpoints
0.16.183.768 W common_speculative_init: no implementations specified for speculative decoding
0.16.183.769 I slot load_model: id 0 | task -1 | new slot, n_ctx = 8192
0.16.183.773 I slot load_model: id 1 | task -1 | new slot, n_ctx = 8192
0.16.183.773 I slot load_model: id 2 | task -1 | new slot, n_ctx = 8192
0.16.183.773 I slot load_model: id 3 | task -1 | new slot, n_ctx = 8192
0.16.183.828 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.16.183.828 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.16.183.829 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.16.183.829 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.16.183.846 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.16.218.437 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
<think></think>Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
<think>
'
0.16.247.958 I srv init: init: chat template, thinking = 1
0.16.247.995 I srv llama_server: model loaded
0.16.247.999 I srv llama_server: server is listening on http://127.0.0.1:8090
0.16.248.003 I srv update_slots: all slots are idle
0.19.987.882 I srv operator(): Chat format: peg-native
0.19.990.748 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.19.990.751 I srv get_availabl: updating prompt cache
0.19.990.758 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.19.990.763 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.19.990.763 I srv get_availabl: prompt cache update took 0.01 ms
0.19.991.313 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.19.991.315 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.19.991.316 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.19.991.328 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.19.991.328 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.19.991.333 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.19.991.333 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.20.780.540 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 45, pos_max = 45, n_tokens = 46, size = 47.618 MiB)
0.21.510.403 I slot print_timing: id 3 | task 0 | prompt eval time = 951.43 ms / 75 tokens ( 12.69 ms per token, 78.83 tokens per second)
0.21.510.406 I slot print_timing: id 3 | task 0 | eval time = 567.62 ms / 44 tokens ( 12.90 ms per token, 77.52 tokens per second)
0.21.510.407 I slot print_timing: id 3 | task 0 | total time = 1519.05 ms / 119 tokens
0.21.510.410 I slot print_timing: id 3 | task 0 | graphs reused = 43
0.21.510.448 I slot release: id 3 | task 0 | stop processing: n_tokens = 118, truncated = 0
0.21.510.452 I srv update_slots: all slots are idle
0.21.543.353 I srv operator(): Chat format: peg-native
0.21.546.195 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.415
0.21.546.198 I srv get_availabl: updating prompt cache
0.21.546.221 W srv prompt_save: - saving prompt with length 118, total state size = 48.311 MiB (draft: 0.000 MiB)
0.21.642.945 I srv load: - looking for better prompt, base f_keep = 0.415, sim = 0.742
0.21.642.952 I srv update: - cache state: 1 prompts, 95.928 MiB (limits: 8192.000 MiB, 8192 tokens, 10076 est)
0.21.642.953 I srv update: - prompt 0x589e59904d20: 118 tokens, checkpoints: 1, 95.928 MiB
0.21.642.955 I srv get_availabl: prompt cache update took 96.76 ms
0.21.643.474 I slot launch_slot_: id 3 | task 47 | processing task, is_child = 0
0.21.643.476 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.21.643.477 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.21.643.483 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.21.643.483 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.21.643.488 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.21.643.488 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.21.643.499 I slot update_slots: id 3 | task 47 | Checking checkpoint with [45, 45] against 49...
0.21.655.530 W slot update_slots: id 3 | task 47 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.22.350.459 I slot print_timing: id 3 | task 47 | prompt eval time = 118.62 ms / 20 tokens ( 5.93 ms per token, 168.61 tokens per second)
0.22.350.463 I slot print_timing: id 3 | task 47 | eval time = 588.34 ms / 46 tokens ( 12.79 ms per token, 78.19 tokens per second)
0.22.350.464 I slot print_timing: id 3 | task 47 | total time = 706.95 ms / 66 tokens
0.22.350.464 I slot print_timing: id 3 | task 47 | graphs reused = 87
0.22.350.492 I slot release: id 3 | task 47 | stop processing: n_tokens = 111, truncated = 0
0.22.350.500 I srv update_slots: all slots are idle
0.22.383.251 I srv operator(): Chat format: peg-native
0.22.386.070 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.441
0.22.386.073 I srv get_availabl: updating prompt cache
0.22.386.094 W srv prompt_save: - saving prompt with length 111, total state size = 48.270 MiB (draft: 0.000 MiB)
0.22.481.981 I srv load: - looking for better prompt, base f_keep = 0.441, sim = 0.710
0.22.481.987 I srv update: - cache state: 2 prompts, 191.816 MiB (limits: 8192.000 MiB, 8192 tokens, 9780 est)
0.22.481.988 I srv update: - prompt 0x589e59904d20: 118 tokens, checkpoints: 1, 95.928 MiB
0.22.481.988 I srv update: - prompt 0x589e5a239440: 111 tokens, checkpoints: 1, 95.887 MiB
0.22.481.989 I srv get_availabl: prompt cache update took 95.92 ms
0.22.482.500 I slot launch_slot_: id 3 | task 95 | processing task, is_child = 0
0.22.482.502 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.22.482.502 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.22.482.509 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.22.482.509 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.22.482.514 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.22.482.514 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.22.482.529 I slot update_slots: id 3 | task 95 | Checking checkpoint with [45, 45] against 49...
0.22.494.567 W slot update_slots: id 3 | task 95 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.23.068.411 I slot print_timing: id 3 | task 95 | prompt eval time = 140.00 ms / 23 tokens ( 6.09 ms per token, 164.29 tokens per second)
0.23.068.415 I slot print_timing: id 3 | task 95 | eval time = 445.88 ms / 35 tokens ( 12.74 ms per token, 78.50 tokens per second)
0.23.068.415 I slot print_timing: id 3 | task 95 | total time = 585.88 ms / 58 tokens
0.23.068.416 I slot print_timing: id 3 | task 95 | graphs reused = 120
0.23.068.442 I slot release: id 3 | task 95 | stop processing: n_tokens = 103, truncated = 0
0.23.068.450 I srv update_slots: all slots are idle
0.23.101.299 I srv operator(): Chat format: peg-native
0.23.104.110 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.735 (> 0.100 thold), f_keep = 0.485
0.23.104.112 I srv get_availabl: updating prompt cache
0.23.104.131 W srv prompt_save: - saving prompt with length 103, total state size = 48.223 MiB (draft: 0.000 MiB)
0.23.171.579 I srv load: - looking for better prompt, base f_keep = 0.485, sim = 0.735
0.23.171.586 I srv update: - cache state: 3 prompts, 287.656 MiB (limits: 8192.000 MiB, 8192 tokens, 9454 est)
0.23.171.587 I srv update: - prompt 0x589e59904d20: 118 tokens, checkpoints: 1, 95.928 MiB
0.23.171.587 I srv update: - prompt 0x589e5a239440: 111 tokens, checkpoints: 1, 95.887 MiB
0.23.171.588 I srv update: - prompt 0x589e5a1543d0: 103 tokens, checkpoints: 1, 95.840 MiB
0.23.171.589 I srv get_availabl: prompt cache update took 67.48 ms
0.23.172.107 I slot launch_slot_: id 3 | task 132 | processing task, is_child = 0
0.23.172.109 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.23.172.110 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.23.172.117 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.23.172.117 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.23.172.122 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.23.172.122 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.23.172.133 I slot update_slots: id 3 | task 132 | Checking checkpoint with [45, 45] against 50...
0.23.184.016 W slot update_slots: id 3 | task 132 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.24.850.436 I slot print_timing: id 3 | task 132 | prompt eval time = 139.86 ms / 22 tokens ( 6.36 ms per token, 157.30 tokens per second)
0.24.850.440 I slot print_timing: id 3 | task 132 | eval time = 1538.44 ms / 119 tokens ( 12.93 ms per token, 77.35 tokens per second)
0.24.850.440 I slot print_timing: id 3 | task 132 | total time = 1678.30 ms / 141 tokens
0.24.850.441 I slot print_timing: id 3 | task 132 | graphs reused = 237
0.24.850.466 I slot release: id 3 | task 132 | stop processing: n_tokens = 186, truncated = 0
0.24.850.474 I srv update_slots: all slots are idle
0.24.883.321 I srv operator(): Chat format: peg-native
0.24.886.151 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.269
0.24.886.153 I srv get_availabl: updating prompt cache
0.24.886.175 W srv prompt_save: - saving prompt with length 186, total state size = 48.710 MiB (draft: 0.000 MiB)
0.24.949.359 I srv load: - looking for better prompt, base f_keep = 0.269, sim = 0.714
0.24.949.363 I srv load: - found better prompt with f_keep = 0.441, sim = 0.743
0.24.973.503 I srv update: - cache state: 3 prompts, 288.055 MiB (limits: 8192.000 MiB, 8192 tokens, 11375 est)
0.24.973.506 I srv update: - prompt 0x589e5a239440: 111 tokens, checkpoints: 1, 95.887 MiB
0.24.973.506 I srv update: - prompt 0x589e5a1543d0: 103 tokens, checkpoints: 1, 95.840 MiB
0.24.973.507 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.24.973.508 I srv get_availabl: prompt cache update took 87.35 ms
0.24.974.000 I slot launch_slot_: id 3 | task 253 | processing task, is_child = 0
0.24.974.003 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.24.974.003 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.24.974.019 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.24.974.019 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.24.974.031 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.24.974.031 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.24.974.044 I slot update_slots: id 3 | task 253 | Checking checkpoint with [45, 45] against 52...
0.25.008.725 W slot update_slots: id 3 | task 253 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.26.091.446 I slot print_timing: id 3 | task 253 | prompt eval time = 164.22 ms / 24 tokens ( 6.84 ms per token, 146.15 tokens per second)
0.26.091.449 I slot print_timing: id 3 | task 253 | eval time = 953.18 ms / 74 tokens ( 12.88 ms per token, 77.63 tokens per second)
0.26.091.450 I slot print_timing: id 3 | task 253 | total time = 1117.40 ms / 98 tokens
0.26.091.451 I slot print_timing: id 3 | task 253 | graphs reused = 309
0.26.091.478 I slot release: id 3 | task 253 | stop processing: n_tokens = 143, truncated = 0
0.26.091.489 I srv update_slots: all slots are idle
0.26.124.290 I srv operator(): Chat format: peg-native
0.26.127.111 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.343
0.26.127.113 I srv get_availabl: updating prompt cache
0.26.127.130 W srv prompt_save: - saving prompt with length 143, total state size = 48.457 MiB (draft: 0.000 MiB)
0.26.189.425 I srv load: - looking for better prompt, base f_keep = 0.343, sim = 0.721
0.26.189.429 I srv load: - found better prompt with f_keep = 0.450, sim = 0.735
0.26.214.035 I srv update: - cache state: 3 prompts, 288.243 MiB (limits: 8192.000 MiB, 8192 tokens, 12277 est)
0.26.214.039 I srv update: - prompt 0x589e5a1543d0: 103 tokens, checkpoints: 1, 95.840 MiB
0.26.214.040 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.26.214.040 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.26.214.042 I srv get_availabl: prompt cache update took 86.93 ms
0.26.214.644 I slot launch_slot_: id 3 | task 329 | processing task, is_child = 0
0.26.214.647 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.26.214.647 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.26.214.653 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.26.214.653 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.26.214.658 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.26.214.658 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.26.214.670 I slot update_slots: id 3 | task 329 | Checking checkpoint with [45, 45] against 50...
0.26.248.610 W slot update_slots: id 3 | task 329 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.27.017.445 I slot print_timing: id 3 | task 329 | prompt eval time = 161.37 ms / 22 tokens ( 7.33 ms per token, 136.34 tokens per second)
0.27.017.449 I slot print_timing: id 3 | task 329 | eval time = 641.40 ms / 50 tokens ( 12.83 ms per token, 77.95 tokens per second)
0.27.017.450 I slot print_timing: id 3 | task 329 | total time = 802.77 ms / 72 tokens
0.27.017.450 I slot print_timing: id 3 | task 329 | graphs reused = 357
0.27.017.478 I slot release: id 3 | task 329 | stop processing: n_tokens = 117, truncated = 0
0.27.017.487 I srv update_slots: all slots are idle
0.27.050.209 I srv operator(): Chat format: peg-native
0.27.053.057 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.662 (> 0.100 thold), f_keep = 0.419
0.27.053.059 I srv get_availabl: updating prompt cache
0.27.053.079 W srv prompt_save: - saving prompt with length 117, total state size = 48.305 MiB (draft: 0.000 MiB)
0.27.112.509 I srv load: - looking for better prompt, base f_keep = 0.419, sim = 0.662
0.27.112.513 I srv load: - found better prompt with f_keep = 0.485, sim = 0.676
0.27.136.910 I srv update: - cache state: 3 prompts, 288.325 MiB (limits: 8192.000 MiB, 8192 tokens, 12671 est)
0.27.136.913 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.27.136.914 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.27.136.914 I srv update: - prompt 0x589e60509ca0: 117 tokens, checkpoints: 1, 95.923 MiB
0.27.136.916 I srv get_availabl: prompt cache update took 83.86 ms
0.27.137.420 I slot launch_slot_: id 3 | task 381 | processing task, is_child = 0
0.27.137.422 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.27.137.423 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.27.137.428 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.27.137.428 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.27.137.433 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.27.137.433 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.27.137.444 I slot update_slots: id 3 | task 381 | Checking checkpoint with [45, 45] against 50...
0.27.157.525 W slot update_slots: id 3 | task 381 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.27.692.450 I slot print_timing: id 3 | task 381 | prompt eval time = 160.46 ms / 28 tokens ( 5.73 ms per token, 174.50 tokens per second)
0.27.692.453 I slot print_timing: id 3 | task 381 | eval time = 394.54 ms / 31 tokens ( 12.73 ms per token, 78.57 tokens per second)
0.27.692.454 I slot print_timing: id 3 | task 381 | total time = 555.00 ms / 59 tokens
0.27.692.455 I slot print_timing: id 3 | task 381 | graphs reused = 386
0.27.692.482 I slot release: id 3 | task 381 | stop processing: n_tokens = 104, truncated = 0
0.27.692.491 I srv update_slots: all slots are idle
0.27.725.155 I srv operator(): Chat format: peg-native
0.27.727.971 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.781 (> 0.100 thold), f_keep = 0.481
0.27.727.973 I srv get_availabl: updating prompt cache
0.27.727.993 W srv prompt_save: - saving prompt with length 104, total state size = 48.228 MiB (draft: 0.000 MiB)
0.27.787.280 I srv load: - looking for better prompt, base f_keep = 0.481, sim = 0.781
0.27.787.288 I srv update: - cache state: 4 prompts, 384.171 MiB (limits: 8192.000 MiB, 8192 tokens, 11728 est)
0.27.787.289 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.27.787.290 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.27.787.290 I srv update: - prompt 0x589e60509ca0: 117 tokens, checkpoints: 1, 95.923 MiB
0.27.787.291 I srv update: - prompt 0x589e5a239440: 104 tokens, checkpoints: 1, 95.846 MiB
0.27.787.292 I srv get_availabl: prompt cache update took 59.32 ms
0.27.787.792 I slot launch_slot_: id 3 | task 414 | processing task, is_child = 0
0.27.787.794 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.27.787.795 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.27.787.801 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.27.787.802 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.27.787.806 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.27.787.807 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.27.787.819 I slot update_slots: id 3 | task 414 | Checking checkpoint with [45, 45] against 50...
0.27.799.648 W slot update_slots: id 3 | task 414 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.28.946.470 I slot print_timing: id 3 | task 414 | prompt eval time = 114.08 ms / 18 tokens ( 6.34 ms per token, 157.78 tokens per second)
0.28.946.473 I slot print_timing: id 3 | task 414 | eval time = 1044.56 ms / 81 tokens ( 12.90 ms per token, 77.54 tokens per second)
0.28.946.474 I slot print_timing: id 3 | task 414 | total time = 1158.64 ms / 99 tokens
0.28.946.474 I slot print_timing: id 3 | task 414 | graphs reused = 465
0.28.946.499 I slot release: id 3 | task 414 | stop processing: n_tokens = 144, truncated = 0
0.28.946.508 I srv update_slots: all slots are idle
0.28.979.301 I srv operator(): Chat format: peg-native
0.28.982.075 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.662 (> 0.100 thold), f_keep = 0.340
0.28.982.077 I srv get_availabl: updating prompt cache
0.28.982.096 W srv prompt_save: - saving prompt with length 144, total state size = 48.463 MiB (draft: 0.000 MiB)
0.29.058.828 I srv load: - looking for better prompt, base f_keep = 0.340, sim = 0.662
0.29.058.832 I srv load: - found better prompt with f_keep = 0.427, sim = 0.676
0.29.082.912 I srv update: - cache state: 4 prompts, 384.330 MiB (limits: 8192.000 MiB, 8192 tokens, 12298 est)
0.29.082.915 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.29.082.916 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.29.082.916 I srv update: - prompt 0x589e5a239440: 104 tokens, checkpoints: 1, 95.846 MiB
0.29.082.917 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.29.082.918 I srv get_availabl: prompt cache update took 100.84 ms
0.29.083.408 I slot launch_slot_: id 3 | task 497 | processing task, is_child = 0
0.29.083.410 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.083.411 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.083.416 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.083.417 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.083.421 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.083.421 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.29.083.432 I slot update_slots: id 3 | task 497 | Checking checkpoint with [45, 45] against 50...
0.29.122.581 W slot update_slots: id 3 | task 497 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.29.658.483 I slot print_timing: id 3 | task 497 | prompt eval time = 180.16 ms / 28 tokens ( 6.43 ms per token, 155.41 tokens per second)
0.29.658.486 I slot print_timing: id 3 | task 497 | eval time = 394.88 ms / 31 tokens ( 12.74 ms per token, 78.50 tokens per second)
0.29.658.487 I slot print_timing: id 3 | task 497 | total time = 575.05 ms / 59 tokens
0.29.658.488 I slot print_timing: id 3 | task 497 | graphs reused = 494
0.29.658.513 I slot release: id 3 | task 497 | stop processing: n_tokens = 104, truncated = 0
0.29.658.521 I srv update_slots: all slots are idle
0.29.691.485 I srv operator(): Chat format: peg-native
0.29.694.299 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.471
0.29.694.301 I srv get_availabl: updating prompt cache
0.29.694.321 W srv prompt_save: - saving prompt with length 104, total state size = 48.228 MiB (draft: 0.000 MiB)
0.29.752.936 I srv load: - looking for better prompt, base f_keep = 0.471, sim = 0.710
0.29.752.941 I srv load: - found better prompt with f_keep = 0.481, sim = 0.725
0.29.778.665 I srv update: - cache state: 4 prompts, 384.330 MiB (limits: 8192.000 MiB, 8192 tokens, 12298 est)
0.29.778.668 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.29.778.669 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.29.778.670 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.29.778.670 I srv update: - prompt 0x589e5a195f90: 104 tokens, checkpoints: 1, 95.846 MiB
0.29.778.672 I srv get_availabl: prompt cache update took 84.37 ms
0.29.779.194 I slot launch_slot_: id 3 | task 530 | processing task, is_child = 0
0.29.779.196 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.779.197 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.779.204 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.779.204 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.779.209 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.779.209 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.29.779.220 I slot update_slots: id 3 | task 530 | Checking checkpoint with [45, 45] against 50...
0.29.815.017 W slot update_slots: id 3 | task 530 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.31.108.498 I slot print_timing: id 3 | task 530 | prompt eval time = 167.65 ms / 23 tokens ( 7.29 ms per token, 137.19 tokens per second)
0.31.108.501 I slot print_timing: id 3 | task 530 | eval time = 1161.62 ms / 90 tokens ( 12.91 ms per token, 77.48 tokens per second)
0.31.108.502 I slot print_timing: id 3 | task 530 | total time = 1329.27 ms / 113 tokens
0.31.108.503 I slot print_timing: id 3 | task 530 | graphs reused = 582
0.31.108.531 I slot release: id 3 | task 530 | stop processing: n_tokens = 158, truncated = 0
0.31.108.538 I srv update_slots: all slots are idle
0.31.141.471 I srv operator(): Chat format: peg-native
0.31.144.251 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.662 (> 0.100 thold), f_keep = 0.310
0.31.144.254 I srv get_availabl: updating prompt cache
0.31.144.272 W srv prompt_save: - saving prompt with length 158, total state size = 48.545 MiB (draft: 0.000 MiB)
0.31.239.570 I srv load: - looking for better prompt, base f_keep = 0.310, sim = 0.662
0.31.239.577 I srv update: - cache state: 5 prompts, 480.493 MiB (limits: 8192.000 MiB, 8192 tokens, 12531 est)
0.31.239.578 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.31.239.579 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.31.239.580 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.31.239.580 I srv update: - prompt 0x589e5a195f90: 104 tokens, checkpoints: 1, 95.846 MiB
0.31.239.581 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.31.239.582 I srv get_availabl: prompt cache update took 95.33 ms
0.31.240.096 I slot launch_slot_: id 3 | task 622 | processing task, is_child = 0
0.31.240.098 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.31.240.099 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.31.240.105 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.31.240.106 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.31.240.110 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.31.240.110 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.31.240.122 I slot update_slots: id 3 | task 622 | Checking checkpoint with [45, 45] against 49...
0.31.252.085 W slot update_slots: id 3 | task 622 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.31.732.481 I slot print_timing: id 3 | task 622 | prompt eval time = 150.41 ms / 28 tokens ( 5.37 ms per token, 186.16 tokens per second)
0.31.732.485 I slot print_timing: id 3 | task 622 | eval time = 341.95 ms / 27 tokens ( 12.66 ms per token, 78.96 tokens per second)
0.31.732.486 I slot print_timing: id 3 | task 622 | total time = 492.36 ms / 55 tokens
0.31.732.486 I slot print_timing: id 3 | task 622 | graphs reused = 607
0.31.732.512 I slot release: id 3 | task 622 | stop processing: n_tokens = 100, truncated = 0
0.31.732.520 I srv update_slots: all slots are idle
0.31.765.445 I srv operator(): Chat format: peg-native
0.31.768.240 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.490
0.31.768.243 I srv get_availabl: updating prompt cache
0.31.768.263 W srv prompt_save: - saving prompt with length 100, total state size = 48.205 MiB (draft: 0.000 MiB)
0.31.845.906 I srv load: - looking for better prompt, base f_keep = 0.490, sim = 0.710
0.31.845.913 I srv update: - cache state: 6 prompts, 576.316 MiB (limits: 8192.000 MiB, 8192 tokens, 11869 est)
0.31.845.914 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.31.845.915 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.31.845.915 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.31.845.916 I srv update: - prompt 0x589e5a195f90: 104 tokens, checkpoints: 1, 95.846 MiB
0.31.845.916 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.31.845.917 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.31.845.918 I srv get_availabl: prompt cache update took 77.67 ms
0.31.846.416 I slot launch_slot_: id 3 | task 651 | processing task, is_child = 0
0.31.846.418 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.31.846.419 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.31.846.425 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.31.846.425 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.31.846.430 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.31.846.430 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.31.846.443 I slot update_slots: id 3 | task 651 | Checking checkpoint with [45, 45] against 49...
0.31.858.452 W slot update_slots: id 3 | task 651 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.32.629.550 I slot print_timing: id 3 | task 651 | prompt eval time = 142.06 ms / 23 tokens ( 6.18 ms per token, 161.90 tokens per second)
0.32.629.553 I slot print_timing: id 3 | task 651 | eval time = 641.04 ms / 50 tokens ( 12.82 ms per token, 78.00 tokens per second)
0.32.629.554 I slot print_timing: id 3 | task 651 | total time = 783.10 ms / 73 tokens
0.32.629.555 I slot print_timing: id 3 | task 651 | graphs reused = 655
0.32.629.602 I slot release: id 3 | task 651 | stop processing: n_tokens = 118, truncated = 0
0.32.629.611 I srv update_slots: all slots are idle
0.32.662.588 I srv operator(): Chat format: peg-native
0.32.665.418 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.690 (> 0.100 thold), f_keep = 0.415
0.32.665.421 I srv get_availabl: updating prompt cache
0.32.665.440 W srv prompt_save: - saving prompt with length 118, total state size = 48.311 MiB (draft: 0.000 MiB)
0.32.739.409 I srv load: - looking for better prompt, base f_keep = 0.415, sim = 0.690
0.32.739.416 I srv update: - cache state: 7 prompts, 672.244 MiB (limits: 8192.000 MiB, 8192 tokens, 11613 est)
0.32.739.417 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.32.739.418 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.32.739.419 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.32.739.419 I srv update: - prompt 0x589e5a195f90: 104 tokens, checkpoints: 1, 95.846 MiB
0.32.739.420 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.32.739.420 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.32.739.421 I srv update: - prompt 0x589e5a0240a0: 118 tokens, checkpoints: 1, 95.928 MiB
0.32.739.422 I srv get_availabl: prompt cache update took 74.00 ms
0.32.739.932 I slot launch_slot_: id 3 | task 703 | processing task, is_child = 0
0.32.739.934 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.32.739.935 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.32.739.942 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.32.739.943 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.32.739.947 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.32.739.947 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.32.739.959 I slot update_slots: id 3 | task 703 | Checking checkpoint with [45, 45] against 49...
0.32.751.978 W slot update_slots: id 3 | task 703 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.33.099.515 I slot print_timing: id 3 | task 703 | prompt eval time = 146.93 ms / 25 tokens ( 5.88 ms per token, 170.15 tokens per second)
0.33.099.518 I slot print_timing: id 3 | task 703 | eval time = 212.63 ms / 17 tokens ( 12.51 ms per token, 79.95 tokens per second)
0.33.099.519 I slot print_timing: id 3 | task 703 | total time = 359.55 ms / 42 tokens
0.33.099.520 I slot print_timing: id 3 | task 703 | graphs reused = 670
0.33.099.544 I slot release: id 3 | task 703 | stop processing: n_tokens = 87, truncated = 0
0.33.099.550 I srv update_slots: all slots are idle
0.33.132.381 I srv operator(): Chat format: peg-native
0.33.135.198 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.700 (> 0.100 thold), f_keep = 0.563
0.33.135.711 I slot launch_slot_: id 3 | task 722 | processing task, is_child = 0
0.33.135.713 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.33.135.713 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.33.135.720 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.33.135.720 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.33.135.725 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.33.135.726 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.33.135.734 I slot update_slots: id 3 | task 722 | Checking checkpoint with [45, 45] against 49...
0.33.147.979 W slot update_slots: id 3 | task 722 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.33.882.534 I slot print_timing: id 3 | task 722 | prompt eval time = 144.47 ms / 24 tokens ( 6.02 ms per token, 166.12 tokens per second)
0.33.882.538 I slot print_timing: id 3 | task 722 | eval time = 602.32 ms / 47 tokens ( 12.82 ms per token, 78.03 tokens per second)
0.33.882.539 I slot print_timing: id 3 | task 722 | total time = 746.80 ms / 71 tokens
0.33.882.539 I slot print_timing: id 3 | task 722 | graphs reused = 715
0.33.882.565 I slot release: id 3 | task 722 | stop processing: n_tokens = 116, truncated = 0
0.33.882.573 I srv update_slots: all slots are idle
0.33.915.907 I srv operator(): Chat format: peg-native
0.33.918.728 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.681 (> 0.100 thold), f_keep = 0.422
0.33.918.730 I srv get_availabl: updating prompt cache
0.33.918.749 W srv prompt_save: - saving prompt with length 116, total state size = 48.299 MiB (draft: 0.000 MiB)
0.33.992.539 I srv load: - looking for better prompt, base f_keep = 0.422, sim = 0.681
0.33.992.544 I srv load: - found better prompt with f_keep = 0.424, sim = 0.694
0.34.016.346 I srv update: - cache state: 7 prompts, 672.233 MiB (limits: 8192.000 MiB, 8192 tokens, 11589 est)
0.34.016.349 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.34.016.350 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.34.016.351 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.34.016.351 I srv update: - prompt 0x589e5a195f90: 104 tokens, checkpoints: 1, 95.846 MiB
0.34.016.352 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.34.016.352 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.34.016.353 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.34.016.354 I srv get_availabl: prompt cache update took 97.62 ms
0.34.016.865 I slot launch_slot_: id 3 | task 771 | processing task, is_child = 0
0.34.016.867 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.34.016.868 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.34.016.875 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.34.016.875 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.34.016.879 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.34.016.880 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.34.016.891 I slot update_slots: id 3 | task 771 | Checking checkpoint with [45, 45] against 50...
0.34.033.973 W slot update_slots: id 3 | task 771 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.34.456.504 I slot print_timing: id 3 | task 771 | prompt eval time = 150.37 ms / 26 tokens ( 5.78 ms per token, 172.91 tokens per second)
0.34.456.508 I slot print_timing: id 3 | task 771 | eval time = 289.24 ms / 23 tokens ( 12.58 ms per token, 79.52 tokens per second)
0.34.456.508 I slot print_timing: id 3 | task 771 | total time = 439.61 ms / 49 tokens
0.34.456.509 I slot print_timing: id 3 | task 771 | graphs reused = 736
0.34.456.536 I slot release: id 3 | task 771 | stop processing: n_tokens = 94, truncated = 0
0.34.456.544 I srv update_slots: all slots are idle
0.34.489.344 I srv operator(): Chat format: peg-native
0.34.492.190 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.521
0.34.492.624 I slot launch_slot_: id 3 | task 796 | processing task, is_child = 0
0.34.492.626 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.34.492.627 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.34.492.633 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.34.492.633 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.34.492.638 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.34.492.638 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.34.492.645 I slot update_slots: id 3 | task 796 | Checking checkpoint with [45, 45] against 49...
0.34.504.756 W slot update_slots: id 3 | task 796 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.35.456.568 I slot print_timing: id 3 | task 796 | prompt eval time = 139.93 ms / 22 tokens ( 6.36 ms per token, 157.22 tokens per second)
0.35.456.572 I slot print_timing: id 3 | task 796 | eval time = 823.99 ms / 60 tokens ( 13.73 ms per token, 72.82 tokens per second)
0.35.456.573 I slot print_timing: id 3 | task 796 | total time = 963.92 ms / 82 tokens
0.35.456.573 I slot print_timing: id 3 | task 796 | graphs reused = 794
0.35.456.600 I slot release: id 3 | task 796 | stop processing: n_tokens = 127, truncated = 0
0.35.456.608 I srv update_slots: all slots are idle
0.35.489.722 I srv operator(): Chat format: peg-native
0.35.492.534 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.681 (> 0.100 thold), f_keep = 0.386
0.35.492.536 I srv get_availabl: updating prompt cache
0.35.492.554 W srv prompt_save: - saving prompt with length 127, total state size = 48.363 MiB (draft: 0.000 MiB)
0.35.554.879 I srv load: - looking for better prompt, base f_keep = 0.386, sim = 0.681
0.35.554.887 I srv update: - cache state: 8 prompts, 768.214 MiB (limits: 8192.000 MiB, 8192 tokens, 11495 est)
0.35.554.888 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.35.554.889 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.35.554.889 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.35.554.890 I srv update: - prompt 0x589e5a195f90: 104 tokens, checkpoints: 1, 95.846 MiB
0.35.554.891 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.35.554.891 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.35.554.892 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.35.554.892 I srv update: - prompt 0x589e5a03d180: 127 tokens, checkpoints: 1, 95.981 MiB
0.35.554.894 I srv get_availabl: prompt cache update took 62.36 ms
0.35.555.391 I slot launch_slot_: id 3 | task 858 | processing task, is_child = 0
0.35.555.393 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.35.555.394 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.35.555.400 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.35.555.400 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.35.555.405 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.35.555.405 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.35.555.417 I slot update_slots: id 3 | task 858 | Checking checkpoint with [45, 45] against 49...
0.35.567.479 W slot update_slots: id 3 | task 858 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.36.316.529 I slot print_timing: id 3 | task 858 | prompt eval time = 145.54 ms / 26 tokens ( 5.60 ms per token, 178.64 tokens per second)
0.36.316.532 I slot print_timing: id 3 | task 858 | eval time = 615.57 ms / 48 tokens ( 12.82 ms per token, 77.98 tokens per second)
0.36.316.533 I slot print_timing: id 3 | task 858 | total time = 761.11 ms / 74 tokens
0.36.316.534 I slot print_timing: id 3 | task 858 | graphs reused = 840
0.36.316.560 I slot release: id 3 | task 858 | stop processing: n_tokens = 119, truncated = 0
0.36.316.568 I srv update_slots: all slots are idle
0.36.349.784 I srv operator(): Chat format: peg-native
0.36.352.602 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.412
0.36.352.604 I srv get_availabl: updating prompt cache
0.36.352.627 W srv prompt_save: - saving prompt with length 119, total state size = 48.317 MiB (draft: 0.000 MiB)
0.36.412.961 I srv load: - looking for better prompt, base f_keep = 0.412, sim = 0.731
0.36.412.967 I srv load: - found better prompt with f_keep = 0.481, sim = 0.746
0.36.437.176 I srv update: - cache state: 8 prompts, 768.302 MiB (limits: 8192.000 MiB, 8192 tokens, 11654 est)
0.36.437.178 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.36.437.179 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.36.437.180 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.36.437.180 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.36.437.181 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.36.437.181 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.36.437.182 I srv update: - prompt 0x589e5a03d180: 127 tokens, checkpoints: 1, 95.981 MiB
0.36.437.182 I srv update: - prompt 0x589e5a276720: 119 tokens, checkpoints: 1, 95.934 MiB
0.36.437.184 I srv get_availabl: prompt cache update took 84.58 ms
0.36.437.670 I slot launch_slot_: id 3 | task 908 | processing task, is_child = 0
0.36.437.673 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.36.437.674 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.36.437.680 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.36.437.680 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.36.437.684 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.36.437.685 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.36.437.697 I slot update_slots: id 3 | task 908 | Checking checkpoint with [45, 45] against 50...
0.36.478.225 W slot update_slots: id 3 | task 908 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.37.544.534 I slot print_timing: id 3 | task 908 | prompt eval time = 166.72 ms / 21 tokens ( 7.94 ms per token, 125.96 tokens per second)
0.37.544.538 I slot print_timing: id 3 | task 908 | eval time = 940.11 ms / 73 tokens ( 12.88 ms per token, 77.65 tokens per second)
0.37.544.538 I slot print_timing: id 3 | task 908 | total time = 1106.83 ms / 94 tokens
0.37.544.539 I slot print_timing: id 3 | task 908 | graphs reused = 911
0.37.544.565 I slot release: id 3 | task 908 | stop processing: n_tokens = 139, truncated = 0
0.37.544.574 I srv update_slots: all slots are idle
0.37.577.781 I srv operator(): Chat format: peg-native
0.37.580.595 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.353
0.37.580.597 I srv get_availabl: updating prompt cache
0.37.580.616 W srv prompt_save: - saving prompt with length 139, total state size = 48.434 MiB (draft: 0.000 MiB)
0.37.638.759 I srv load: - looking for better prompt, base f_keep = 0.353, sim = 0.742
0.37.638.765 I srv load: - found better prompt with f_keep = 0.420, sim = 0.758
0.37.661.934 I srv update: - cache state: 8 prompts, 768.419 MiB (limits: 8192.000 MiB, 8192 tokens, 11865 est)
0.37.661.936 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.37.661.937 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.37.661.938 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.37.661.938 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.37.661.939 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.37.661.939 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.37.661.940 I srv update: - prompt 0x589e5a03d180: 127 tokens, checkpoints: 1, 95.981 MiB
0.37.661.940 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.37.661.942 I srv get_availabl: prompt cache update took 81.34 ms
0.37.662.580 I slot launch_slot_: id 3 | task 983 | processing task, is_child = 0
0.37.662.584 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.37.662.584 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.37.662.592 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.37.662.592 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.37.662.597 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.37.662.597 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.37.662.610 I slot update_slots: id 3 | task 983 | Checking checkpoint with [45, 45] against 50...
0.37.699.519 W slot update_slots: id 3 | task 983 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.37.941.544 I slot print_timing: id 3 | task 983 | prompt eval time = 144.18 ms / 20 tokens ( 7.21 ms per token, 138.71 tokens per second)
0.37.941.547 I slot print_timing: id 3 | task 983 | eval time = 134.75 ms / 11 tokens ( 12.25 ms per token, 81.63 tokens per second)
0.37.941.548 I slot print_timing: id 3 | task 983 | total time = 278.93 ms / 31 tokens
0.37.941.549 I slot print_timing: id 3 | task 983 | graphs reused = 920
0.37.941.576 I slot release: id 3 | task 983 | stop processing: n_tokens = 76, truncated = 0
0.37.941.583 I srv update_slots: all slots are idle
0.37.974.872 I srv operator(): Chat format: peg-native
0.37.977.683 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.769 (> 0.100 thold), f_keep = 0.658
0.37.978.206 I slot launch_slot_: id 3 | task 996 | processing task, is_child = 0
0.37.978.208 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.37.978.209 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.37.978.216 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.37.978.216 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.37.978.221 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.37.978.222 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.37.978.230 I slot update_slots: id 3 | task 996 | Checking checkpoint with [45, 45] against 50...
0.37.990.370 W slot update_slots: id 3 | task 996 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.38.242.544 I slot print_timing: id 3 | task 996 | prompt eval time = 116.29 ms / 19 tokens ( 6.12 ms per token, 163.38 tokens per second)
0.38.242.547 I slot print_timing: id 3 | task 996 | eval time = 148.02 ms / 12 tokens ( 12.33 ms per token, 81.07 tokens per second)
0.38.242.548 I slot print_timing: id 3 | task 996 | total time = 264.31 ms / 31 tokens
0.38.242.548 I slot print_timing: id 3 | task 996 | graphs reused = 930
0.38.242.575 I slot release: id 3 | task 996 | stop processing: n_tokens = 76, truncated = 0
0.38.242.581 I srv update_slots: all slots are idle
0.38.276.225 I srv operator(): Chat format: peg-native
0.38.279.044 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.766 (> 0.100 thold), f_keep = 0.645
0.38.279.475 I slot launch_slot_: id 3 | task 1010 | processing task, is_child = 0
0.38.279.477 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.279.477 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.38.279.483 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.279.483 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.38.279.488 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.279.488 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.38.279.495 I slot update_slots: id 3 | task 1010 | Checking checkpoint with [45, 45] against 49...
0.38.291.562 W slot update_slots: id 3 | task 1010 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.38.580.537 I slot print_timing: id 3 | task 1010 | prompt eval time = 115.62 ms / 18 tokens ( 6.42 ms per token, 155.68 tokens per second)
0.38.580.541 I slot print_timing: id 3 | task 1010 | eval time = 185.42 ms / 15 tokens ( 12.36 ms per token, 80.90 tokens per second)
0.38.580.542 I slot print_timing: id 3 | task 1010 | total time = 301.04 ms / 33 tokens
0.38.580.542 I slot print_timing: id 3 | task 1010 | graphs reused = 943
0.38.580.569 I slot release: id 3 | task 1010 | stop processing: n_tokens = 78, truncated = 0
0.38.580.577 I srv update_slots: all slots are idle
0.38.613.690 I srv operator(): Chat format: peg-native
0.38.616.632 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.628
0.38.617.100 I slot launch_slot_: id 3 | task 1027 | processing task, is_child = 0
0.38.617.102 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.617.103 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.38.617.110 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.617.110 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.38.617.115 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.617.115 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.38.617.123 I slot update_slots: id 3 | task 1027 | Checking checkpoint with [45, 45] against 49...
0.38.629.130 W slot update_slots: id 3 | task 1027 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.39.087.562 I slot print_timing: id 3 | task 1027 | prompt eval time = 141.06 ms / 23 tokens ( 6.13 ms per token, 163.05 tokens per second)
0.39.087.565 I slot print_timing: id 3 | task 1027 | eval time = 329.38 ms / 26 tokens ( 12.67 ms per token, 78.94 tokens per second)
0.39.087.566 I slot print_timing: id 3 | task 1027 | total time = 470.43 ms / 49 tokens
0.39.087.567 I slot print_timing: id 3 | task 1027 | graphs reused = 967
0.39.087.594 I slot release: id 3 | task 1027 | stop processing: n_tokens = 94, truncated = 0
0.39.087.603 I srv update_slots: all slots are idle
0.39.120.729 I srv operator(): Chat format: peg-native
0.39.123.576 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.521
0.39.124.126 I slot launch_slot_: id 3 | task 1055 | processing task, is_child = 0
0.39.124.128 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.39.124.129 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.39.124.136 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.39.124.136 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.39.124.142 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.39.124.142 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.39.124.151 I slot update_slots: id 3 | task 1055 | Checking checkpoint with [45, 45] against 49...
0.39.136.186 W slot update_slots: id 3 | task 1055 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.39.646.542 I slot print_timing: id 3 | task 1055 | prompt eval time = 141.12 ms / 22 tokens ( 6.41 ms per token, 155.90 tokens per second)
0.39.646.545 I slot print_timing: id 3 | task 1055 | eval time = 381.27 ms / 30 tokens ( 12.71 ms per token, 78.68 tokens per second)
0.39.646.546 I slot print_timing: id 3 | task 1055 | total time = 522.39 ms / 52 tokens
0.39.646.547 I slot print_timing: id 3 | task 1055 | graphs reused = 995
0.39.646.574 I slot release: id 3 | task 1055 | stop processing: n_tokens = 97, truncated = 0
0.39.646.582 I srv update_slots: all slots are idle
0.39.679.258 I srv operator(): Chat format: peg-native
0.39.682.055 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.662 (> 0.100 thold), f_keep = 0.505
0.39.682.486 I slot launch_slot_: id 3 | task 1087 | processing task, is_child = 0
0.39.682.488 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.39.682.489 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.39.682.495 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.39.682.495 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.39.682.500 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.39.682.500 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.39.682.507 I slot update_slots: id 3 | task 1087 | Checking checkpoint with [45, 45] against 49...
0.39.694.581 W slot update_slots: id 3 | task 1087 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.40.141.575 I slot print_timing: id 3 | task 1087 | prompt eval time = 155.02 ms / 28 tokens ( 5.54 ms per token, 180.63 tokens per second)
0.40.141.579 I slot print_timing: id 3 | task 1087 | eval time = 304.05 ms / 24 tokens ( 12.67 ms per token, 78.94 tokens per second)
0.40.141.580 I slot print_timing: id 3 | task 1087 | total time = 459.06 ms / 52 tokens
0.40.141.580 I slot print_timing: id 3 | task 1087 | graphs reused = 1017
0.40.141.608 I slot release: id 3 | task 1087 | stop processing: n_tokens = 97, truncated = 0
0.40.141.616 I srv update_slots: all slots are idle
0.40.174.287 I srv operator(): Chat format: peg-native
0.40.177.108 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.671 (> 0.100 thold), f_keep = 0.505
0.40.177.539 I slot launch_slot_: id 3 | task 1113 | processing task, is_child = 0
0.40.177.541 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.40.177.542 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.40.177.548 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.40.177.548 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.40.177.553 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.40.177.553 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.40.177.561 I slot update_slots: id 3 | task 1113 | Checking checkpoint with [45, 45] against 49...
0.40.189.657 W slot update_slots: id 3 | task 1113 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.41.089.565 I slot print_timing: id 3 | task 1113 | prompt eval time = 154.09 ms / 27 tokens ( 5.71 ms per token, 175.22 tokens per second)
0.41.089.568 I slot print_timing: id 3 | task 1113 | eval time = 757.91 ms / 59 tokens ( 12.85 ms per token, 77.85 tokens per second)
0.41.089.569 I slot print_timing: id 3 | task 1113 | total time = 912.00 ms / 86 tokens
0.41.089.569 I slot print_timing: id 3 | task 1113 | graphs reused = 1074
0.41.089.598 I slot release: id 3 | task 1113 | stop processing: n_tokens = 131, truncated = 0
0.41.089.605 I srv update_slots: all slots are idle
0.41.122.181 I srv operator(): Chat format: peg-native
0.41.124.942 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.766 (> 0.100 thold), f_keep = 0.374
0.41.124.945 I srv get_availabl: updating prompt cache
0.41.124.967 W srv prompt_save: - saving prompt with length 131, total state size = 48.387 MiB (draft: 0.000 MiB)
0.41.198.216 I srv load: - looking for better prompt, base f_keep = 0.374, sim = 0.766
0.41.198.223 I srv load: - found better prompt with f_keep = 0.394, sim = 0.781
0.41.222.515 I srv update: - cache state: 8 prompts, 768.443 MiB (limits: 8192.000 MiB, 8192 tokens, 11907 est)
0.41.222.518 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.41.222.519 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.41.222.519 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.41.222.520 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.41.222.520 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.41.222.521 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.41.222.521 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.41.222.522 I srv update: - prompt 0x589e5a03f4d0: 131 tokens, checkpoints: 1, 96.005 MiB
0.41.222.523 I srv get_availabl: prompt cache update took 97.58 ms
0.41.223.024 I slot launch_slot_: id 3 | task 1174 | processing task, is_child = 0
0.41.223.026 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.41.223.027 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.41.223.033 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.41.223.033 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.41.223.038 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.41.223.038 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.41.223.049 I slot update_slots: id 3 | task 1174 | Checking checkpoint with [45, 45] against 50...
0.41.255.941 W slot update_slots: id 3 | task 1174 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.41.464.478 I slot print_timing: id 3 | task 1174 | prompt eval time = 135.51 ms / 18 tokens ( 7.53 ms per token, 132.83 tokens per second)
0.41.464.482 I slot print_timing: id 3 | task 1174 | eval time = 105.92 ms / 9 tokens ( 11.77 ms per token, 84.97 tokens per second)
0.41.464.482 I slot print_timing: id 3 | task 1174 | total time = 241.43 ms / 27 tokens
0.41.464.483 I slot print_timing: id 3 | task 1174 | graphs reused = 1081
0.41.464.507 I slot release: id 3 | task 1174 | stop processing: n_tokens = 72, truncated = 0
0.41.464.515 I srv update_slots: all slots are idle
0.41.497.522 I srv operator(): Chat format: peg-native
0.41.500.334 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.721 (> 0.100 thold), f_keep = 0.681
0.41.500.866 I slot launch_slot_: id 3 | task 1185 | processing task, is_child = 0
0.41.500.869 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.41.500.869 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.41.500.876 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.41.500.876 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.41.500.882 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.41.500.882 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.41.500.891 I slot update_slots: id 3 | task 1185 | Checking checkpoint with [45, 45] against 49...
0.41.513.068 W slot update_slots: id 3 | task 1185 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.42.434.505 I slot print_timing: id 3 | task 1185 | prompt eval time = 142.45 ms / 22 tokens ( 6.48 ms per token, 154.44 tokens per second)
0.42.434.508 I slot print_timing: id 3 | task 1185 | eval time = 791.15 ms / 64 tokens ( 12.36 ms per token, 80.89 tokens per second)
0.42.434.509 I slot print_timing: id 3 | task 1185 | total time = 933.61 ms / 86 tokens
0.42.434.509 I slot print_timing: id 3 | task 1185 | graphs reused = 1143
0.42.434.537 I slot release: id 3 | task 1185 | stop processing: n_tokens = 131, truncated = 0
0.42.434.544 I srv update_slots: all slots are idle
0.42.467.276 I srv operator(): Chat format: peg-native
0.42.470.135 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.700 (> 0.100 thold), f_keep = 0.374
0.42.470.137 I srv get_availabl: updating prompt cache
0.42.470.156 W srv prompt_save: - saving prompt with length 131, total state size = 48.387 MiB (draft: 0.000 MiB)
0.42.527.935 I srv load: - looking for better prompt, base f_keep = 0.374, sim = 0.700
0.42.527.945 I srv update: - cache state: 9 prompts, 864.447 MiB (limits: 8192.000 MiB, 8192 tokens, 11826 est)
0.42.527.945 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.42.527.946 I srv update: - prompt 0x589e6020c7e0: 143 tokens, checkpoints: 1, 96.075 MiB
0.42.527.947 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.42.527.947 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.42.527.948 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.42.527.948 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.42.527.949 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.42.527.949 I srv update: - prompt 0x589e5a03f4d0: 131 tokens, checkpoints: 1, 96.005 MiB
0.42.527.950 I srv update: - prompt 0x589e5a181210: 131 tokens, checkpoints: 1, 96.005 MiB
0.42.527.951 I srv get_availabl: prompt cache update took 57.81 ms
0.42.528.464 I slot launch_slot_: id 3 | task 1251 | processing task, is_child = 0
0.42.528.466 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.42.528.467 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.42.528.472 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.42.528.473 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.42.528.477 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.42.528.477 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.42.528.489 I slot update_slots: id 3 | task 1251 | Checking checkpoint with [45, 45] against 49...
0.42.540.456 W slot update_slots: id 3 | task 1251 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.43.541.453 I slot print_timing: id 3 | task 1251 | prompt eval time = 146.35 ms / 24 tokens ( 6.10 ms per token, 163.99 tokens per second)
0.43.541.456 I slot print_timing: id 3 | task 1251 | eval time = 866.61 ms / 70 tokens ( 12.38 ms per token, 80.77 tokens per second)
0.43.541.457 I slot print_timing: id 3 | task 1251 | total time = 1012.96 ms / 94 tokens
0.43.541.458 I slot print_timing: id 3 | task 1251 | graphs reused = 1211
0.43.541.505 I slot release: id 3 | task 1251 | stop processing: n_tokens = 139, truncated = 0
0.43.541.514 I srv update_slots: all slots are idle
0.43.574.220 I srv operator(): Chat format: peg-native
0.43.577.004 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.605 (> 0.100 thold), f_keep = 0.353
0.43.577.007 I srv get_availabl: updating prompt cache
0.43.577.044 W srv prompt_save: - saving prompt with length 139, total state size = 48.434 MiB (draft: 0.000 MiB)
0.43.636.366 I srv load: - looking for better prompt, base f_keep = 0.353, sim = 0.605
0.43.636.371 I srv load: - found better prompt with f_keep = 0.364, sim = 0.642
0.43.659.929 I srv update: - cache state: 9 prompts, 864.424 MiB (limits: 8192.000 MiB, 8192 tokens, 11789 est)
0.43.659.931 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.43.659.932 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.43.659.933 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.43.659.933 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.43.659.934 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.43.659.934 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.43.659.935 I srv update: - prompt 0x589e5a03f4d0: 131 tokens, checkpoints: 1, 96.005 MiB
0.43.659.935 I srv update: - prompt 0x589e5a181210: 131 tokens, checkpoints: 1, 96.005 MiB
0.43.659.936 I srv update: - prompt 0x589e5a120ca0: 139 tokens, checkpoints: 1, 96.052 MiB
0.43.659.937 I srv get_availabl: prompt cache update took 82.93 ms
0.43.660.431 I slot launch_slot_: id 3 | task 1323 | processing task, is_child = 0
0.43.660.433 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.43.660.434 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.43.660.440 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.43.660.440 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.43.660.445 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.43.660.445 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.43.660.458 I slot update_slots: id 3 | task 1323 | Checking checkpoint with [45, 45] against 52...
0.43.703.428 W slot update_slots: id 3 | task 1323 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.45.464.479 I slot print_timing: id 3 | task 1323 | prompt eval time = 211.25 ms / 35 tokens ( 6.04 ms per token, 165.68 tokens per second)
0.45.464.482 I slot print_timing: id 3 | task 1323 | eval time = 1592.77 ms / 128 tokens ( 12.44 ms per token, 80.36 tokens per second)
0.45.464.483 I slot print_timing: id 3 | task 1323 | total time = 1804.02 ms / 163 tokens
0.45.464.483 I slot print_timing: id 3 | task 1323 | graphs reused = 1337
0.45.464.512 I slot release: id 3 | task 1323 | stop processing: n_tokens = 208, truncated = 0
0.45.464.520 I srv update_slots: all slots are idle
0.45.497.093 I srv operator(): Chat format: peg-native
0.45.499.909 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.694 (> 0.100 thold), f_keep = 0.240
0.45.499.912 I srv get_availabl: updating prompt cache
0.45.499.936 W srv prompt_save: - saving prompt with length 208, total state size = 48.839 MiB (draft: 0.000 MiB)
0.45.559.331 I srv load: - looking for better prompt, base f_keep = 0.240, sim = 0.694
0.45.559.343 I srv update: - cache state: 10 prompts, 960.881 MiB (limits: 8192.000 MiB, 8192 tokens, 12379 est)
0.45.559.343 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.45.559.344 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.45.559.345 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.45.559.345 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.45.559.346 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.45.559.346 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.45.559.347 I srv update: - prompt 0x589e5a03f4d0: 131 tokens, checkpoints: 1, 96.005 MiB
0.45.559.347 I srv update: - prompt 0x589e5a181210: 131 tokens, checkpoints: 1, 96.005 MiB
0.45.559.348 I srv update: - prompt 0x589e5a120ca0: 139 tokens, checkpoints: 1, 96.052 MiB
0.45.559.348 I srv update: - prompt 0x589e5a287110: 208 tokens, checkpoints: 1, 96.457 MiB
0.45.559.350 I srv get_availabl: prompt cache update took 59.44 ms
0.45.559.848 I slot launch_slot_: id 3 | task 1453 | processing task, is_child = 0
0.45.559.850 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.45.559.851 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.45.559.857 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.45.559.858 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.45.559.862 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.45.559.862 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.45.559.874 I slot update_slots: id 3 | task 1453 | Checking checkpoint with [45, 45] against 50...
0.45.571.739 W slot update_slots: id 3 | task 1453 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.46.840.505 I slot print_timing: id 3 | task 1453 | prompt eval time = 150.85 ms / 26 tokens ( 5.80 ms per token, 172.36 tokens per second)
0.46.840.509 I slot print_timing: id 3 | task 1453 | eval time = 1129.78 ms / 91 tokens ( 12.42 ms per token, 80.55 tokens per second)
0.46.840.509 I slot print_timing: id 3 | task 1453 | total time = 1280.63 ms / 117 tokens
0.46.840.510 I slot print_timing: id 3 | task 1453 | graphs reused = 1426
0.46.840.535 I slot release: id 3 | task 1453 | stop processing: n_tokens = 162, truncated = 0
0.46.840.543 I srv update_slots: all slots are idle
0.46.873.583 I srv operator(): Chat format: peg-native
0.46.876.419 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.302
0.46.876.421 I srv get_availabl: updating prompt cache
0.46.876.447 W srv prompt_save: - saving prompt with length 162, total state size = 48.569 MiB (draft: 0.000 MiB)
0.46.949.571 I srv load: - looking for better prompt, base f_keep = 0.302, sim = 0.731
0.46.949.582 I srv update: - cache state: 11 prompts, 1057.067 MiB (limits: 8192.000 MiB, 8192 tokens, 12508 est)
0.46.949.584 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.46.949.585 I srv update: - prompt 0x589e5f8ff560: 144 tokens, checkpoints: 1, 96.081 MiB
0.46.949.586 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.46.949.587 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.46.949.587 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.46.949.588 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.46.949.588 I srv update: - prompt 0x589e5a03f4d0: 131 tokens, checkpoints: 1, 96.005 MiB
0.46.949.589 I srv update: - prompt 0x589e5a181210: 131 tokens, checkpoints: 1, 96.005 MiB
0.46.949.589 I srv update: - prompt 0x589e5a120ca0: 139 tokens, checkpoints: 1, 96.052 MiB
0.46.949.590 I srv update: - prompt 0x589e5a287110: 208 tokens, checkpoints: 1, 96.457 MiB
0.46.949.590 I srv update: - prompt 0x589e5a2754f0: 162 tokens, checkpoints: 1, 96.187 MiB
0.46.949.591 I srv get_availabl: prompt cache update took 73.17 ms
0.46.950.100 I slot launch_slot_: id 3 | task 1546 | processing task, is_child = 0
0.46.950.102 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.46.950.103 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.46.950.109 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.46.950.109 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.46.950.113 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.46.950.114 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.46.950.126 I slot update_slots: id 3 | task 1546 | Checking checkpoint with [45, 45] against 49...
0.46.961.850 W slot update_slots: id 3 | task 1546 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.48.106.921 I slot print_timing: id 3 | task 1546 | prompt eval time = 139.36 ms / 21 tokens ( 6.64 ms per token, 150.69 tokens per second)
0.48.106.924 I slot print_timing: id 3 | task 1546 | eval time = 1017.43 ms / 82 tokens ( 12.41 ms per token, 80.60 tokens per second)
0.48.106.925 I slot print_timing: id 3 | task 1546 | total time = 1156.79 ms / 103 tokens
0.48.106.926 I slot print_timing: id 3 | task 1546 | graphs reused = 1506
0.48.106.951 I slot release: id 3 | task 1546 | stop processing: n_tokens = 148, truncated = 0
0.48.106.960 I srv update_slots: all slots are idle
0.48.139.597 I srv operator(): Chat format: peg-native
0.48.142.540 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.681 (> 0.100 thold), f_keep = 0.331
0.48.142.542 I srv get_availabl: updating prompt cache
0.48.142.564 W srv prompt_save: - saving prompt with length 148, total state size = 48.487 MiB (draft: 0.000 MiB)
0.48.213.262 I srv load: - looking for better prompt, base f_keep = 0.331, sim = 0.681
0.48.213.270 I srv load: - found better prompt with f_keep = 0.347, sim = 0.694
0.48.236.522 I srv update: - cache state: 11 prompts, 1057.091 MiB (limits: 8192.000 MiB, 8192 tokens, 12538 est)
0.48.236.524 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.48.236.525 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.48.236.525 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.48.236.526 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.48.236.526 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.48.236.527 I srv update: - prompt 0x589e5a03f4d0: 131 tokens, checkpoints: 1, 96.005 MiB
0.48.236.527 I srv update: - prompt 0x589e5a181210: 131 tokens, checkpoints: 1, 96.005 MiB
0.48.236.528 I srv update: - prompt 0x589e5a120ca0: 139 tokens, checkpoints: 1, 96.052 MiB
0.48.236.528 I srv update: - prompt 0x589e5a287110: 208 tokens, checkpoints: 1, 96.457 MiB
0.48.236.529 I srv update: - prompt 0x589e5a2754f0: 162 tokens, checkpoints: 1, 96.187 MiB
0.48.236.529 I srv update: - prompt 0x589e5a11f700: 148 tokens, checkpoints: 1, 96.105 MiB
0.48.236.531 I srv get_availabl: prompt cache update took 93.99 ms
0.48.237.065 I slot launch_slot_: id 3 | task 1630 | processing task, is_child = 0
0.48.237.067 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.48.237.068 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.48.237.074 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.48.237.074 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.48.237.079 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.48.237.079 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.48.237.091 I slot update_slots: id 3 | task 1630 | Checking checkpoint with [45, 45] against 50...
0.48.278.613 W slot update_slots: id 3 | task 1630 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.48.957.949 I slot print_timing: id 3 | task 1630 | prompt eval time = 178.76 ms / 26 tokens ( 6.88 ms per token, 145.44 tokens per second)
0.48.957.952 I slot print_timing: id 3 | task 1630 | eval time = 542.09 ms / 44 tokens ( 12.32 ms per token, 81.17 tokens per second)
0.48.957.953 I slot print_timing: id 3 | task 1630 | total time = 720.85 ms / 70 tokens
0.48.957.954 I slot print_timing: id 3 | task 1630 | graphs reused = 1548
0.48.957.984 I slot release: id 3 | task 1630 | stop processing: n_tokens = 115, truncated = 0
0.48.957.992 I srv update_slots: all slots are idle
0.48.990.658 I srv operator(): Chat format: peg-native
0.48.993.504 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.694 (> 0.100 thold), f_keep = 0.435
0.48.993.506 I srv get_availabl: updating prompt cache
0.48.993.527 W srv prompt_save: - saving prompt with length 115, total state size = 48.293 MiB (draft: 0.000 MiB)
0.49.068.594 I srv load: - looking for better prompt, base f_keep = 0.435, sim = 0.694
0.49.068.604 I srv update: - cache state: 12 prompts, 1153.002 MiB (limits: 8192.000 MiB, 8192 tokens, 12312 est)
0.49.068.605 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.49.068.605 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.49.068.606 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.49.068.610 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.49.068.611 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.49.068.611 I srv update: - prompt 0x589e5a03f4d0: 131 tokens, checkpoints: 1, 96.005 MiB
0.49.068.611 I srv update: - prompt 0x589e5a181210: 131 tokens, checkpoints: 1, 96.005 MiB
0.49.068.612 I srv update: - prompt 0x589e5a120ca0: 139 tokens, checkpoints: 1, 96.052 MiB
0.49.068.612 I srv update: - prompt 0x589e5a287110: 208 tokens, checkpoints: 1, 96.457 MiB
0.49.068.613 I srv update: - prompt 0x589e5a2754f0: 162 tokens, checkpoints: 1, 96.187 MiB
0.49.068.616 I srv update: - prompt 0x589e5a11f700: 148 tokens, checkpoints: 1, 96.105 MiB
0.49.068.616 I srv update: - prompt 0x589e603aafb0: 115 tokens, checkpoints: 1, 95.911 MiB
0.49.068.617 I srv get_availabl: prompt cache update took 75.11 ms
0.49.069.139 I slot launch_slot_: id 3 | task 1676 | processing task, is_child = 0
0.49.069.141 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.49.069.141 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.49.069.148 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.49.069.148 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.49.069.153 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.49.069.153 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.49.069.164 I slot update_slots: id 3 | task 1676 | Checking checkpoint with [45, 45] against 50...
0.49.081.000 W slot update_slots: id 3 | task 1676 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.50.093.521 I slot print_timing: id 3 | task 1676 | prompt eval time = 152.48 ms / 26 tokens ( 5.86 ms per token, 170.51 tokens per second)
0.50.093.525 I slot print_timing: id 3 | task 1676 | eval time = 871.87 ms / 70 tokens ( 12.46 ms per token, 80.29 tokens per second)
0.50.093.526 I slot print_timing: id 3 | task 1676 | total time = 1024.35 ms / 96 tokens
0.50.093.526 I slot print_timing: id 3 | task 1676 | graphs reused = 1616
0.50.093.556 I slot release: id 3 | task 1676 | stop processing: n_tokens = 141, truncated = 0
0.50.093.564 I srv update_slots: all slots are idle
0.50.126.310 I srv operator(): Chat format: peg-native
0.50.129.107 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.348
0.50.129.108 I srv get_availabl: updating prompt cache
0.50.129.127 W srv prompt_save: - saving prompt with length 141, total state size = 48.446 MiB (draft: 0.000 MiB)
0.50.193.788 I srv load: - looking for better prompt, base f_keep = 0.348, sim = 0.742
0.50.193.798 I srv update: - cache state: 13 prompts, 1249.065 MiB (limits: 8192.000 MiB, 8192 tokens, 12290 est)
0.50.193.800 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.50.193.800 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.50.193.802 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.50.193.802 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.50.193.803 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.50.193.803 I srv update: - prompt 0x589e5a03f4d0: 131 tokens, checkpoints: 1, 96.005 MiB
0.50.193.803 I srv update: - prompt 0x589e5a181210: 131 tokens, checkpoints: 1, 96.005 MiB
0.50.193.804 I srv update: - prompt 0x589e5a120ca0: 139 tokens, checkpoints: 1, 96.052 MiB
0.50.193.804 I srv update: - prompt 0x589e5a287110: 208 tokens, checkpoints: 1, 96.457 MiB
0.50.193.805 I srv update: - prompt 0x589e5a2754f0: 162 tokens, checkpoints: 1, 96.187 MiB
0.50.193.806 I srv update: - prompt 0x589e5a11f700: 148 tokens, checkpoints: 1, 96.105 MiB
0.50.193.806 I srv update: - prompt 0x589e603aafb0: 115 tokens, checkpoints: 1, 95.911 MiB
0.50.193.807 I srv update: - prompt 0x589e5a136d50: 141 tokens, checkpoints: 1, 96.063 MiB
0.50.193.808 I srv get_availabl: prompt cache update took 64.70 ms
0.50.194.323 I slot launch_slot_: id 3 | task 1748 | processing task, is_child = 0
0.50.194.324 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.50.194.325 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.50.194.331 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.50.194.332 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.50.194.336 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.50.194.336 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.50.194.347 I slot update_slots: id 3 | task 1748 | Checking checkpoint with [45, 45] against 49...
0.50.206.354 W slot update_slots: id 3 | task 1748 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.51.044.936 I slot print_timing: id 3 | task 1748 | prompt eval time = 119.29 ms / 20 tokens ( 5.96 ms per token, 167.66 tokens per second)
0.51.044.939 I slot print_timing: id 3 | task 1748 | eval time = 731.29 ms / 59 tokens ( 12.39 ms per token, 80.68 tokens per second)
0.51.044.940 I slot print_timing: id 3 | task 1748 | total time = 850.58 ms / 79 tokens
0.51.044.940 I slot print_timing: id 3 | task 1748 | graphs reused = 1673
0.51.044.966 I slot release: id 3 | task 1748 | stop processing: n_tokens = 124, truncated = 0
0.51.044.973 I srv update_slots: all slots are idle
0.51.077.615 I srv operator(): Chat format: peg-native
0.51.080.424 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.778 (> 0.100 thold), f_keep = 0.395
0.51.080.426 I srv get_availabl: updating prompt cache
0.51.080.447 W srv prompt_save: - saving prompt with length 124, total state size = 48.346 MiB (draft: 0.000 MiB)
0.51.140.130 I srv load: - looking for better prompt, base f_keep = 0.395, sim = 0.778
0.51.140.141 I srv update: - cache state: 14 prompts, 1345.029 MiB (limits: 8192.000 MiB, 8192 tokens, 12168 est)
0.51.140.142 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.51.140.142 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.51.140.143 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.51.140.144 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.51.140.144 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.51.140.145 I srv update: - prompt 0x589e5a03f4d0: 131 tokens, checkpoints: 1, 96.005 MiB
0.51.140.150 I srv update: - prompt 0x589e5a181210: 131 tokens, checkpoints: 1, 96.005 MiB
0.51.140.150 I srv update: - prompt 0x589e5a120ca0: 139 tokens, checkpoints: 1, 96.052 MiB
0.51.140.151 I srv update: - prompt 0x589e5a287110: 208 tokens, checkpoints: 1, 96.457 MiB
0.51.140.151 I srv update: - prompt 0x589e5a2754f0: 162 tokens, checkpoints: 1, 96.187 MiB
0.51.140.152 I srv update: - prompt 0x589e5a11f700: 148 tokens, checkpoints: 1, 96.105 MiB
0.51.140.152 I srv update: - prompt 0x589e603aafb0: 115 tokens, checkpoints: 1, 95.911 MiB
0.51.140.155 I srv update: - prompt 0x589e5a136d50: 141 tokens, checkpoints: 1, 96.063 MiB
0.51.140.156 I srv update: - prompt 0x589e5a159f80: 124 tokens, checkpoints: 1, 95.964 MiB
0.51.140.157 I srv get_availabl: prompt cache update took 59.73 ms
0.51.140.655 I slot launch_slot_: id 3 | task 1809 | processing task, is_child = 0
0.51.140.657 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.51.140.657 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.51.140.664 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.51.140.664 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.51.140.669 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.51.140.669 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.51.140.681 I slot update_slots: id 3 | task 1809 | Checking checkpoint with [45, 45] against 49...
0.51.152.713 W slot update_slots: id 3 | task 1809 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.51.406.529 I slot print_timing: id 3 | task 1809 | prompt eval time = 110.68 ms / 17 tokens ( 6.51 ms per token, 153.59 tokens per second)
0.51.406.532 I slot print_timing: id 3 | task 1809 | eval time = 155.16 ms / 13 tokens ( 11.94 ms per token, 83.79 tokens per second)
0.51.406.533 I slot print_timing: id 3 | task 1809 | total time = 265.84 ms / 30 tokens
0.51.406.533 I slot print_timing: id 3 | task 1809 | graphs reused = 1684
0.51.406.559 I slot release: id 3 | task 1809 | stop processing: n_tokens = 75, truncated = 0
0.51.406.567 I srv update_slots: all slots are idle
0.51.439.092 I srv operator(): Chat format: peg-native
0.51.441.905 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.653
0.51.442.435 I slot launch_slot_: id 3 | task 1824 | processing task, is_child = 0
0.51.442.437 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.51.442.437 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.51.442.444 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.51.442.445 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.51.442.450 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.51.442.450 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.51.442.458 I slot update_slots: id 3 | task 1824 | Checking checkpoint with [45, 45] against 49...
0.51.454.602 W slot update_slots: id 3 | task 1824 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.52.489.550 I slot print_timing: id 3 | task 1824 | prompt eval time = 142.05 ms / 23 tokens ( 6.18 ms per token, 161.91 tokens per second)
0.52.489.554 I slot print_timing: id 3 | task 1824 | eval time = 905.03 ms / 73 tokens ( 12.40 ms per token, 80.66 tokens per second)
0.52.489.555 I slot print_timing: id 3 | task 1824 | total time = 1047.09 ms / 96 tokens
0.52.489.556 I slot print_timing: id 3 | task 1824 | graphs reused = 1755
0.52.489.582 I slot release: id 3 | task 1824 | stop processing: n_tokens = 141, truncated = 0
0.52.489.589 I srv update_slots: all slots are idle
0.52.522.760 I srv operator(): Chat format: peg-native
0.52.525.555 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.731 (> 0.100 thold), f_keep = 0.348
0.52.525.557 I srv get_availabl: updating prompt cache
0.52.525.575 W srv prompt_save: - saving prompt with length 141, total state size = 48.446 MiB (draft: 0.000 MiB)
0.52.586.612 I srv load: - looking for better prompt, base f_keep = 0.348, sim = 0.731
0.52.586.619 I srv load: - found better prompt with f_keep = 0.382, sim = 0.746
0.52.610.452 I srv update: - cache state: 14 prompts, 1345.087 MiB (limits: 8192.000 MiB, 8192 tokens, 12229 est)
0.52.610.455 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.52.610.455 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.52.610.456 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.52.610.457 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.52.610.457 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.52.610.458 I srv update: - prompt 0x589e5a181210: 131 tokens, checkpoints: 1, 96.005 MiB
0.52.610.458 I srv update: - prompt 0x589e5a120ca0: 139 tokens, checkpoints: 1, 96.052 MiB
0.52.610.459 I srv update: - prompt 0x589e5a287110: 208 tokens, checkpoints: 1, 96.457 MiB
0.52.610.459 I srv update: - prompt 0x589e5a2754f0: 162 tokens, checkpoints: 1, 96.187 MiB
0.52.610.460 I srv update: - prompt 0x589e5a11f700: 148 tokens, checkpoints: 1, 96.105 MiB
0.52.610.461 I srv update: - prompt 0x589e603aafb0: 115 tokens, checkpoints: 1, 95.911 MiB
0.52.610.462 I srv update: - prompt 0x589e5a136d50: 141 tokens, checkpoints: 1, 96.063 MiB
0.52.610.462 I srv update: - prompt 0x589e5a159f80: 124 tokens, checkpoints: 1, 95.964 MiB
0.52.610.463 I srv update: - prompt 0x589e53138640: 141 tokens, checkpoints: 1, 96.063 MiB
0.52.610.465 I srv get_availabl: prompt cache update took 84.91 ms
0.52.610.969 I slot launch_slot_: id 3 | task 1899 | processing task, is_child = 0
0.52.610.971 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.52.610.972 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.52.610.978 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.52.610.978 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.52.610.982 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.52.610.983 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.52.610.994 I slot update_slots: id 3 | task 1899 | Checking checkpoint with [45, 45] against 50...
0.52.637.988 W slot update_slots: id 3 | task 1899 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.53.681.973 I slot print_timing: id 3 | task 1899 | prompt eval time = 153.26 ms / 21 tokens ( 7.30 ms per token, 137.02 tokens per second)
0.53.681.976 I slot print_timing: id 3 | task 1899 | eval time = 917.71 ms / 74 tokens ( 12.40 ms per token, 80.64 tokens per second)
0.53.681.977 I slot print_timing: id 3 | task 1899 | total time = 1070.97 ms / 95 tokens
0.53.681.977 I slot print_timing: id 3 | task 1899 | graphs reused = 1827
0.53.682.001 I slot release: id 3 | task 1899 | stop processing: n_tokens = 140, truncated = 0
0.53.682.018 I srv update_slots: all slots are idle
0.53.714.575 I srv operator(): Chat format: peg-native
0.53.717.420 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.758 (> 0.100 thold), f_keep = 0.357
0.53.717.422 I srv get_availabl: updating prompt cache
0.53.717.444 W srv prompt_save: - saving prompt with length 140, total state size = 48.440 MiB (draft: 0.000 MiB)
0.53.776.040 I srv load: - looking for better prompt, base f_keep = 0.357, sim = 0.758
0.53.776.051 I srv update: - cache state: 15 prompts, 1441.145 MiB (limits: 8192.000 MiB, 8192 tokens, 12210 est)
0.53.776.055 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.53.776.055 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.53.776.056 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.53.776.056 I srv update: - prompt 0x589e52f81f70: 116 tokens, checkpoints: 1, 95.917 MiB
0.53.776.057 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.53.776.057 I srv update: - prompt 0x589e5a181210: 131 tokens, checkpoints: 1, 96.005 MiB
0.53.776.060 I srv update: - prompt 0x589e5a120ca0: 139 tokens, checkpoints: 1, 96.052 MiB
0.53.776.061 I srv update: - prompt 0x589e5a287110: 208 tokens, checkpoints: 1, 96.457 MiB
0.53.776.061 I srv update: - prompt 0x589e5a2754f0: 162 tokens, checkpoints: 1, 96.187 MiB
0.53.776.062 I srv update: - prompt 0x589e5a11f700: 148 tokens, checkpoints: 1, 96.105 MiB
0.53.776.062 I srv update: - prompt 0x589e603aafb0: 115 tokens, checkpoints: 1, 95.911 MiB
0.53.776.063 I srv update: - prompt 0x589e5a136d50: 141 tokens, checkpoints: 1, 96.063 MiB
0.53.776.066 I srv update: - prompt 0x589e5a159f80: 124 tokens, checkpoints: 1, 95.964 MiB
0.53.776.066 I srv update: - prompt 0x589e53138640: 141 tokens, checkpoints: 1, 96.063 MiB
0.53.776.066 I srv update: - prompt 0x589e59904d20: 140 tokens, checkpoints: 1, 96.058 MiB
0.53.776.068 I srv get_availabl: prompt cache update took 58.65 ms
0.53.776.593 I slot launch_slot_: id 3 | task 1975 | processing task, is_child = 0
0.53.776.595 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.53.776.596 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.53.776.603 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.53.776.604 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.53.776.609 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.53.776.609 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.53.776.623 I slot update_slots: id 3 | task 1975 | Checking checkpoint with [45, 45] against 50...
0.53.788.617 W slot update_slots: id 3 | task 1975 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.54.613.580 I slot print_timing: id 3 | task 1975 | prompt eval time = 118.84 ms / 20 tokens ( 5.94 ms per token, 168.30 tokens per second)
0.54.613.583 I slot print_timing: id 3 | task 1975 | eval time = 718.12 ms / 58 tokens ( 12.38 ms per token, 80.77 tokens per second)
0.54.613.584 I slot print_timing: id 3 | task 1975 | total time = 836.95 ms / 78 tokens
0.54.613.585 I slot print_timing: id 3 | task 1975 | graphs reused = 1883
0.54.613.610 I slot release: id 3 | task 1975 | stop processing: n_tokens = 123, truncated = 0
0.54.613.618 I srv update_slots: all slots are idle
0.54.646.328 I srv operator(): Chat format: peg-native
0.54.649.166 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.710 (> 0.100 thold), f_keep = 0.398
0.54.649.168 I srv get_availabl: updating prompt cache
0.54.649.191 W srv prompt_save: - saving prompt with length 123, total state size = 48.340 MiB (draft: 0.000 MiB)
0.54.714.079 I srv load: - looking for better prompt, base f_keep = 0.398, sim = 0.710
0.54.714.086 I srv load: - found better prompt with f_keep = 0.431, sim = 0.725
0.54.737.357 I srv update: - cache state: 15 prompts, 1441.186 MiB (limits: 8192.000 MiB, 8192 tokens, 12249 est)
0.54.737.360 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.54.737.360 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.54.737.361 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.54.737.361 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.54.737.362 I srv update: - prompt 0x589e5a181210: 131 tokens, checkpoints: 1, 96.005 MiB
0.54.737.362 I srv update: - prompt 0x589e5a120ca0: 139 tokens, checkpoints: 1, 96.052 MiB
0.54.737.363 I srv update: - prompt 0x589e5a287110: 208 tokens, checkpoints: 1, 96.457 MiB
0.54.737.363 I srv update: - prompt 0x589e5a2754f0: 162 tokens, checkpoints: 1, 96.187 MiB
0.54.737.364 I srv update: - prompt 0x589e5a11f700: 148 tokens, checkpoints: 1, 96.105 MiB
0.54.737.364 I srv update: - prompt 0x589e603aafb0: 115 tokens, checkpoints: 1, 95.911 MiB
0.54.737.365 I srv update: - prompt 0x589e5a136d50: 141 tokens, checkpoints: 1, 96.063 MiB
0.54.737.365 I srv update: - prompt 0x589e5a159f80: 124 tokens, checkpoints: 1, 95.964 MiB
0.54.737.366 I srv update: - prompt 0x589e53138640: 141 tokens, checkpoints: 1, 96.063 MiB
0.54.737.367 I srv update: - prompt 0x589e59904d20: 140 tokens, checkpoints: 1, 96.058 MiB
0.54.737.368 I srv update: - prompt 0x589e604b7500: 123 tokens, checkpoints: 1, 95.958 MiB
0.54.737.369 I srv get_availabl: prompt cache update took 88.20 ms
0.54.737.996 I slot launch_slot_: id 3 | task 2035 | processing task, is_child = 0
0.54.738.000 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.54.738.001 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.54.738.031 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.54.738.032 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.54.738.038 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.54.738.039 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.54.738.055 I slot update_slots: id 3 | task 2035 | Checking checkpoint with [45, 45] against 50...
0.54.764.044 W slot update_slots: id 3 | task 2035 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.55.361.564 I slot print_timing: id 3 | task 2035 | prompt eval time = 156.18 ms / 23 tokens ( 6.79 ms per token, 147.26 tokens per second)
0.55.361.568 I slot print_timing: id 3 | task 2035 | eval time = 467.32 ms / 38 tokens ( 12.30 ms per token, 81.31 tokens per second)
0.55.361.569 I slot print_timing: id 3 | task 2035 | total time = 623.51 ms / 61 tokens
0.55.361.570 I slot print_timing: id 3 | task 2035 | graphs reused = 1919
0.55.361.597 I slot release: id 3 | task 2035 | stop processing: n_tokens = 106, truncated = 0
0.55.361.604 I srv update_slots: all slots are idle
0.55.394.464 I srv operator(): Chat format: peg-native
0.55.397.298 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.754 (> 0.100 thold), f_keep = 0.462
0.55.397.300 I srv get_availabl: updating prompt cache
0.55.397.320 W srv prompt_save: - saving prompt with length 106, total state size = 48.240 MiB (draft: 0.000 MiB)
0.55.455.941 I srv load: - looking for better prompt, base f_keep = 0.462, sim = 0.754
0.55.455.953 I srv update: - cache state: 16 prompts, 1537.044 MiB (limits: 8192.000 MiB, 8192 tokens, 12050 est)
0.55.455.954 I srv update: - prompt 0x589e5a1264b0: 186 tokens, checkpoints: 1, 96.328 MiB
0.55.455.955 I srv update: - prompt 0x589e5a1839b0: 158 tokens, checkpoints: 1, 96.163 MiB
0.55.455.956 I srv update: - prompt 0x589e5a14eaf0: 100 tokens, checkpoints: 1, 95.823 MiB
0.55.455.956 I srv update: - prompt 0x589e604c5140: 139 tokens, checkpoints: 1, 96.052 MiB
0.55.455.962 I srv update: - prompt 0x589e5a181210: 131 tokens, checkpoints: 1, 96.005 MiB
0.55.455.962 I srv update: - prompt 0x589e5a120ca0: 139 tokens, checkpoints: 1, 96.052 MiB
0.55.455.963 I srv update: - prompt 0x589e5a287110: 208 tokens, checkpoints: 1, 96.457 MiB
0.55.455.963 I srv update: - prompt 0x589e5a2754f0: 162 tokens, checkpoints: 1, 96.187 MiB
0.55.455.964 I srv update: - prompt 0x589e5a11f700: 148 tokens, checkpoints: 1, 96.105 MiB
0.55.455.964 I srv update: - prompt 0x589e603aafb0: 115 tokens, checkpoints: 1, 95.911 MiB
0.55.455.965 I srv update: - prompt 0x589e5a136d50: 141 tokens, checkpoints: 1, 96.063 MiB
0.55.455.966 I srv update: - prompt 0x589e5a159f80: 124 tokens, checkpoints: 1, 95.964 MiB
0.55.455.969 I srv update: - prompt 0x589e53138640: 141 tokens, checkpoints: 1, 96.063 MiB
0.55.455.969 I srv update: - prompt 0x589e59904d20: 140 tokens, checkpoints: 1, 96.058 MiB
0.55.455.970 I srv update: - prompt 0x589e604b7500: 123 tokens, checkpoints: 1, 95.958 MiB
0.55.455.970 I srv update: - prompt 0x589e5a0771b0: 106 tokens, checkpoints: 1, 95.858 MiB
0.55.455.971 I srv get_availabl: prompt cache update took 58.67 ms
0.55.456.505 I slot launch_slot_: id 3 | task 2075 | processing task, is_child = 0
0.55.456.507 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.55.456.507 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.55.456.513 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.55.456.514 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.55.456.519 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.55.456.519 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.55.456.531 I slot update_slots: id 3 | task 2075 | Checking checkpoint with [45, 45] against 49...
0.55.468.597 W slot update_slots: id 3 | task 2075 | restored context checkpoint (pos_min = 45, pos_max = 45, n_tokens = 46, n_past = 46, size = 47.618 MiB)
0.56.829.995 I slot print_timing: id 3 | task 2075 | prompt eval time = 117.73 ms / 19 tokens ( 6.20 ms per token, 161.38 tokens per second)
0.56.829.999 I slot print_timing: id 3 | task 2075 | eval time = 1255.72 ms / 101 tokens ( 12.43 ms per token, 80.43 tokens per second)
0.56.829.999 I slot print_timing: id 3 | task 2075 | total time = 1373.46 ms / 120 tokens
0.56.830.000 I slot print_timing: id 3 | task 2075 | graphs reused = 2018
0.56.830.035 I slot release: id 3 | task 2075 | stop processing: n_tokens = 165, truncated = 0
0.56.830.045 I srv update_slots: all slots are idle
0.56.830.508 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink