qoherent/experiments-07-07-12-local-llm-knowledge-2
23
0
Fork 0
Code Issues Pull Requests Workflows Packages Projects Releases Wiki Activity
Push Tracker
experiments-07-07-12-local-.../results/aya-23-8b.server.log

779 lines
73 KiB
Plaintext
Raw Normal View History

G
readme report
2026-07-12 21:35:10 -04:00
0.00.065.683 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.065.686 I device_info:
0.00.065.706 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.711 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
0.00.065.718 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
0.00.065.773 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.065.776 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.065.817 I srv init: running without SSL
0.00.065.882 I srv init: using 23 threads for HTTP server
0.00.066.207 I srv start: binding port with default address family
0.00.067.417 I srv llama_server: loading model
0.00.067.424 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/aya-23-8B-Q5_K_M.gguf'
0.00.067.472 I common_init_result: fitting params to device memory ...
0.00.067.474 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.01.416.936 W load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
0.04.235.562 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.04.375.921 I srv load_model: initializing slots, n_slots = 4
0.04.429.272 W common_speculative_init: no implementations specified for speculative decoding
0.04.429.283 I slot load_model: id 0 | task -1 | new slot, n_ctx = 8192
0.04.429.289 I slot load_model: id 1 | task -1 | new slot, n_ctx = 8192
0.04.429.289 I slot load_model: id 2 | task -1 | new slot, n_ctx = 8192
0.04.429.290 I slot load_model: id 3 | task -1 | new slot, n_ctx = 8192
0.04.429.348 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.04.429.350 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.04.429.350 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.04.429.351 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
0.04.429.369 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.04.433.964 I init: chat template, example_format: '<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>You are a helpful assistant<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Hello<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>Hi there<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>How are you?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>'
0.04.435.545 I srv init: init: chat template, thinking = 0
0.04.435.586 I srv llama_server: model loaded
0.04.435.593 I srv llama_server: server is listening on http://127.0.0.1:8090
0.04.435.598 I srv update_slots: all slots are idle
0.04.965.526 I srv operator(): Chat format: peg-native
0.04.965.927 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.04.965.929 I srv get_availabl: updating prompt cache
0.04.965.935 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.04.965.939 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.04.965.940 I srv get_availabl: prompt cache update took 0.01 ms
0.04.965.982 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
0.04.965.983 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.04.965.985 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.04.965.993 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.04.965.994 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.04.965.999 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.04.965.999 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.06.465.813 I slot print_timing: id 3 | task 0 | prompt eval time = 212.66 ms / 66 tokens ( 3.22 ms per token, 310.36 tokens per second)
0.06.465.817 I slot print_timing: id 3 | task 0 | eval time = 1287.13 ms / 62 tokens ( 20.76 ms per token, 48.17 tokens per second)
0.06.465.817 I slot print_timing: id 3 | task 0 | total time = 1499.79 ms / 128 tokens
0.06.465.822 I slot print_timing: id 3 | task 0 | graphs reused = 61
0.06.465.860 I slot release: id 3 | task 0 | stop processing: n_tokens = 127, truncated = 0
0.06.465.865 I srv update_slots: all slots are idle
0.06.469.182 I srv operator(): Chat format: peg-native
0.06.469.523 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.797 (> 0.100 thold), f_keep = 0.370
0.06.469.525 I srv get_availabl: updating prompt cache
0.06.469.547 W srv prompt_save: - saving prompt with length 127, total state size = 15.877 MiB (draft: 0.000 MiB)
0.06.483.382 I srv load: - looking for better prompt, base f_keep = 0.370, sim = 0.797
0.06.483.389 I srv update: - cache state: 1 prompts, 15.877 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.06.483.390 I srv update: - prompt 0x644e41e619a0: 127 tokens, checkpoints: 0, 15.877 MiB
0.06.483.391 I srv get_availabl: prompt cache update took 13.87 ms
0.06.483.441 I slot launch_slot_: id 3 | task 64 | processing task, is_child = 0
0.06.483.442 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.06.483.443 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.06.483.450 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.06.483.450 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.06.483.455 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.06.483.455 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.06.837.781 I slot print_timing: id 3 | task 64 | prompt eval time = 100.04 ms / 12 tokens ( 8.34 ms per token, 119.95 tokens per second)
0.06.837.784 I slot print_timing: id 3 | task 64 | eval time = 254.27 ms / 13 tokens ( 19.56 ms per token, 51.13 tokens per second)
0.06.837.785 I slot print_timing: id 3 | task 64 | total time = 354.31 ms / 25 tokens
0.06.837.785 I slot print_timing: id 3 | task 64 | graphs reused = 72
0.06.837.806 I slot release: id 3 | task 64 | stop processing: n_tokens = 71, truncated = 0
0.06.837.812 I srv update_slots: all slots are idle
0.06.840.891 I srv operator(): Chat format: peg-native
0.06.841.270 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.758 (> 0.100 thold), f_keep = 0.662
0.06.841.302 I slot launch_slot_: id 3 | task 78 | processing task, is_child = 0
0.06.841.303 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.06.841.303 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.06.841.309 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.06.841.309 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.06.841.314 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.06.841.314 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.07.951.792 I slot print_timing: id 3 | task 78 | prompt eval time = 99.94 ms / 15 tokens ( 6.66 ms per token, 150.10 tokens per second)
0.07.951.796 I slot print_timing: id 3 | task 78 | eval time = 1010.53 ms / 49 tokens ( 20.62 ms per token, 48.49 tokens per second)
0.07.951.797 I slot print_timing: id 3 | task 78 | total time = 1110.46 ms / 64 tokens
0.07.951.797 I slot print_timing: id 3 | task 78 | graphs reused = 119
0.07.951.817 I slot release: id 3 | task 78 | stop processing: n_tokens = 110, truncated = 0
0.07.951.823 I srv update_slots: all slots are idle
0.07.954.974 I srv operator(): Chat format: peg-native
0.07.955.170 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.787 (> 0.100 thold), f_keep = 0.436
0.07.955.172 I srv get_availabl: updating prompt cache
0.07.955.189 W srv prompt_save: - saving prompt with length 110, total state size = 13.752 MiB (draft: 0.000 MiB)
0.07.967.274 I srv load: - looking for better prompt, base f_keep = 0.436, sim = 0.787
0.07.967.287 I srv update: - cache state: 2 prompts, 29.629 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.07.967.288 I srv update: - prompt 0x644e41e619a0: 127 tokens, checkpoints: 0, 15.877 MiB
0.07.967.289 I srv update: - prompt 0x644e41f89010: 110 tokens, checkpoints: 0, 13.752 MiB
0.07.967.290 I srv get_availabl: prompt cache update took 12.12 ms
0.07.967.332 I slot launch_slot_: id 3 | task 128 | processing task, is_child = 0
0.07.967.332 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.07.967.333 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.07.967.340 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.07.967.340 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.07.967.344 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.07.967.345 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.08.615.795 I slot print_timing: id 3 | task 128 | prompt eval time = 100.27 ms / 13 tokens ( 7.71 ms per token, 129.65 tokens per second)
0.08.615.799 I slot print_timing: id 3 | task 128 | eval time = 548.16 ms / 27 tokens ( 20.30 ms per token, 49.26 tokens per second)
0.08.615.799 I slot print_timing: id 3 | task 128 | total time = 648.43 ms / 40 tokens
0.08.615.800 I slot print_timing: id 3 | task 128 | graphs reused = 144
0.08.615.822 I slot release: id 3 | task 128 | stop processing: n_tokens = 87, truncated = 0
0.08.615.829 I srv update_slots: all slots are idle
0.08.618.646 I srv operator(): Chat format: peg-native
0.08.618.838 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.762 (> 0.100 thold), f_keep = 0.552
0.08.618.861 I slot launch_slot_: id 3 | task 156 | processing task, is_child = 0
0.08.618.862 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.08.618.862 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.08.618.871 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.08.618.871 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.08.618.878 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.08.618.878 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.08.765.529 I slot print_timing: id 3 | task 156 | prompt eval time = 99.94 ms / 15 tokens ( 6.66 ms per token, 150.09 tokens per second)
0.08.765.532 I slot print_timing: id 3 | task 156 | eval time = 46.70 ms / 3 tokens ( 15.57 ms per token, 64.24 tokens per second)
0.08.765.533 I slot print_timing: id 3 | task 156 | total time = 146.63 ms / 18 tokens
0.08.765.534 I slot print_timing: id 3 | task 156 | graphs reused = 145
0.08.765.568 I slot release: id 3 | task 156 | stop processing: n_tokens = 65, truncated = 0
0.08.765.575 I srv update_slots: all slots are idle
0.08.768.889 I srv operator(): Chat format: peg-native
0.08.769.083 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.758 (> 0.100 thold), f_keep = 0.723
0.08.769.106 I slot launch_slot_: id 3 | task 160 | processing task, is_child = 0
0.08.769.106 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.08.769.107 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.08.769.113 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.08.769.114 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.08.769.118 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.08.769.118 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.08.936.799 I slot print_timing: id 3 | task 160 | prompt eval time = 101.06 ms / 15 tokens ( 6.74 ms per token, 148.42 tokens per second)
0.08.936.802 I slot print_timing: id 3 | task 160 | eval time = 66.61 ms / 4 tokens ( 16.65 ms per token, 60.05 tokens per second)
0.08.936.802 I slot print_timing: id 3 | task 160 | total time = 167.67 ms / 19 tokens
0.08.936.803 I slot print_timing: id 3 | task 160 | graphs reused = 147
0.08.936.822 I slot release: id 3 | task 160 | stop processing: n_tokens = 65, truncated = 0
0.08.936.828 I srv update_slots: all slots are idle
0.08.939.573 I srv operator(): Chat format: peg-native
0.08.939.763 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.701 (> 0.100 thold), f_keep = 0.723
0.08.939.793 I slot launch_slot_: id 3 | task 165 | processing task, is_child = 0
0.08.939.794 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.08.939.795 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.08.939.800 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.08.939.801 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.08.939.806 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.08.939.807 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.09.970.832 I slot print_timing: id 3 | task 165 | prompt eval time = 125.36 ms / 20 tokens ( 6.27 ms per token, 159.54 tokens per second)
0.09.970.835 I slot print_timing: id 3 | task 165 | eval time = 905.65 ms / 44 tokens ( 20.58 ms per token, 48.58 tokens per second)
0.09.970.836 I slot print_timing: id 3 | task 165 | total time = 1031.01 ms / 64 tokens
0.09.970.836 I slot print_timing: id 3 | task 165 | graphs reused = 189
0.09.970.859 I slot release: id 3 | task 165 | stop processing: n_tokens = 110, truncated = 0
0.09.970.866 I srv update_slots: all slots are idle
0.09.973.934 I srv operator(): Chat format: peg-native
0.09.974.130 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.842 (> 0.100 thold), f_keep = 0.436
0.09.974.132 I srv get_availabl: updating prompt cache
0.09.974.150 W srv prompt_save: - saving prompt with length 110, total state size = 13.752 MiB (draft: 0.000 MiB)
0.09.986.158 I srv load: - looking for better prompt, base f_keep = 0.436, sim = 0.842
0.09.986.162 I srv load: - found better prompt with f_keep = 0.445, sim = 0.860
0.09.993.165 I srv update: - cache state: 2 prompts, 29.629 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.09.993.168 I srv update: - prompt 0x644e41e619a0: 127 tokens, checkpoints: 0, 15.877 MiB
0.09.993.169 I srv update: - prompt 0x644e41e5c2c0: 110 tokens, checkpoints: 0, 13.752 MiB
0.09.993.170 I srv get_availabl: prompt cache update took 19.04 ms
0.09.993.214 I slot launch_slot_: id 3 | task 210 | processing task, is_child = 0
0.09.993.215 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.09.993.216 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.09.993.223 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.09.993.224 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.09.993.228 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.09.993.229 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.10.504.824 I slot print_timing: id 3 | task 210 | prompt eval time = 109.80 ms / 8 tokens ( 13.73 ms per token, 72.86 tokens per second)
0.10.504.828 I slot print_timing: id 3 | task 210 | eval time = 401.77 ms / 20 tokens ( 20.09 ms per token, 49.78 tokens per second)
0.10.504.829 I slot print_timing: id 3 | task 210 | total time = 511.58 ms / 28 tokens
0.10.504.829 I slot print_timing: id 3 | task 210 | graphs reused = 207
0.10.504.852 I slot release: id 3 | task 210 | stop processing: n_tokens = 76, truncated = 0
0.10.504.859 I srv update_slots: all slots are idle
0.10.507.813 I srv operator(): Chat format: peg-native
0.10.508.006 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.701 (> 0.100 thold), f_keep = 0.618
0.10.508.032 I slot launch_slot_: id 3 | task 231 | processing task, is_child = 0
0.10.508.033 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.10.508.034 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.10.508.042 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.10.508.043 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.10.508.050 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.10.508.051 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.11.370.840 I slot print_timing: id 3 | task 231 | prompt eval time = 124.40 ms / 20 tokens ( 6.22 ms per token, 160.78 tokens per second)
0.11.370.843 I slot print_timing: id 3 | task 231 | eval time = 738.38 ms / 36 tokens ( 20.51 ms per token, 48.76 tokens per second)
0.11.370.844 I slot print_timing: id 3 | task 231 | total time = 862.77 ms / 56 tokens
0.11.370.844 I slot print_timing: id 3 | task 231 | graphs reused = 241
0.11.370.866 I slot release: id 3 | task 231 | stop processing: n_tokens = 102, truncated = 0
0.11.370.873 I srv update_slots: all slots are idle
0.11.374.041 I srv operator(): Chat format: peg-native
0.11.374.240 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.758 (> 0.100 thold), f_keep = 0.461
0.11.374.242 I srv get_availabl: updating prompt cache
0.11.374.264 W srv prompt_save: - saving prompt with length 102, total state size = 12.752 MiB (draft: 0.000 MiB)
0.11.385.683 I srv load: - looking for better prompt, base f_keep = 0.461, sim = 0.758
0.11.385.690 I srv update: - cache state: 3 prompts, 42.381 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.11.385.691 I srv update: - prompt 0x644e41e619a0: 127 tokens, checkpoints: 0, 15.877 MiB
0.11.385.691 I srv update: - prompt 0x644e41e5c2c0: 110 tokens, checkpoints: 0, 13.752 MiB
0.11.385.692 I srv update: - prompt 0x644e41f0a560: 102 tokens, checkpoints: 0, 12.752 MiB
0.11.385.693 I srv get_availabl: prompt cache update took 11.45 ms
0.11.385.733 I slot launch_slot_: id 3 | task 268 | processing task, is_child = 0
0.11.385.734 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.11.385.734 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.11.385.741 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.11.385.741 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.11.385.745 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.11.385.746 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.12.223.832 I slot print_timing: id 3 | task 268 | prompt eval time = 99.40 ms / 15 tokens ( 6.63 ms per token, 150.91 tokens per second)
0.12.223.835 I slot print_timing: id 3 | task 268 | eval time = 738.67 ms / 36 tokens ( 20.52 ms per token, 48.74 tokens per second)
0.12.223.836 I slot print_timing: id 3 | task 268 | total time = 838.07 ms / 51 tokens
0.12.223.836 I slot print_timing: id 3 | task 268 | graphs reused = 275
0.12.223.857 I slot release: id 3 | task 268 | stop processing: n_tokens = 97, truncated = 0
0.12.223.864 I srv update_slots: all slots are idle
0.12.227.180 I srv operator(): Chat format: peg-native
0.12.227.425 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.701 (> 0.100 thold), f_keep = 0.485
0.12.227.427 I srv get_availabl: updating prompt cache
0.12.227.445 W srv prompt_save: - saving prompt with length 97, total state size = 12.127 MiB (draft: 0.000 MiB)
0.12.238.336 I srv load: - looking for better prompt, base f_keep = 0.485, sim = 0.701
0.12.238.343 I srv update: - cache state: 4 prompts, 54.508 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.12.238.344 I srv update: - prompt 0x644e41e619a0: 127 tokens, checkpoints: 0, 15.877 MiB
0.12.238.345 I srv update: - prompt 0x644e41e5c2c0: 110 tokens, checkpoints: 0, 13.752 MiB
0.12.238.345 I srv update: - prompt 0x644e41f0a560: 102 tokens, checkpoints: 0, 12.752 MiB
0.12.238.346 I srv update: - prompt 0x644e41f89010: 97 tokens, checkpoints: 0, 12.127 MiB
0.12.238.347 I srv get_availabl: prompt cache update took 10.92 ms
0.12.238.391 I slot launch_slot_: id 3 | task 305 | processing task, is_child = 0
0.12.238.392 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.12.238.392 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.12.238.398 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.12.238.398 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.12.238.403 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.12.238.403 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.12.932.860 I slot print_timing: id 3 | task 305 | prompt eval time = 124.10 ms / 20 tokens ( 6.21 ms per token, 161.16 tokens per second)
0.12.932.864 I slot print_timing: id 3 | task 305 | eval time = 570.34 ms / 28 tokens ( 20.37 ms per token, 49.09 tokens per second)
0.12.932.865 I slot print_timing: id 3 | task 305 | total time = 694.44 ms / 48 tokens
0.12.932.866 I slot print_timing: id 3 | task 305 | graphs reused = 301
0.12.932.888 I slot release: id 3 | task 305 | stop processing: n_tokens = 94, truncated = 0
0.12.932.895 I srv update_slots: all slots are idle
0.12.936.030 I srv operator(): Chat format: peg-native
0.12.936.260 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.770 (> 0.100 thold), f_keep = 0.500
0.12.936.293 I slot launch_slot_: id 3 | task 334 | processing task, is_child = 0
0.12.936.294 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.12.936.294 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.12.936.300 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.12.936.300 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.12.936.305 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.12.936.305 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.13.983.851 I slot print_timing: id 3 | task 334 | prompt eval time = 99.48 ms / 14 tokens ( 7.11 ms per token, 140.73 tokens per second)
0.13.983.855 I slot print_timing: id 3 | task 334 | eval time = 948.05 ms / 46 tokens ( 20.61 ms per token, 48.52 tokens per second)
0.13.983.855 I slot print_timing: id 3 | task 334 | total time = 1047.53 ms / 60 tokens
0.13.983.856 I slot print_timing: id 3 | task 334 | graphs reused = 345
0.13.983.876 I slot release: id 3 | task 334 | stop processing: n_tokens = 106, truncated = 0
0.13.983.882 I srv update_slots: all slots are idle
0.13.986.716 I srv operator(): Chat format: peg-native
0.13.986.973 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.758 (> 0.100 thold), f_keep = 0.443
0.13.986.975 I srv get_availabl: updating prompt cache
0.13.986.995 W srv prompt_save: - saving prompt with length 106, total state size = 13.252 MiB (draft: 0.000 MiB)
0.13.998.621 I srv load: - looking for better prompt, base f_keep = 0.443, sim = 0.758
0.13.998.631 I srv update: - cache state: 5 prompts, 67.760 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.13.998.632 I srv update: - prompt 0x644e41e619a0: 127 tokens, checkpoints: 0, 15.877 MiB
0.13.998.633 I srv update: - prompt 0x644e41e5c2c0: 110 tokens, checkpoints: 0, 13.752 MiB
0.13.998.634 I srv update: - prompt 0x644e41f0a560: 102 tokens, checkpoints: 0, 12.752 MiB
0.13.998.634 I srv update: - prompt 0x644e41f89010: 97 tokens, checkpoints: 0, 12.127 MiB
0.13.998.635 I srv update: - prompt 0x644e41db5c70: 106 tokens, checkpoints: 0, 13.252 MiB
0.13.998.636 I srv get_availabl: prompt cache update took 11.66 ms
0.13.998.680 I slot launch_slot_: id 3 | task 381 | processing task, is_child = 0
0.13.998.681 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.13.998.682 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.13.998.688 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.13.998.688 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.13.998.693 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.13.998.693 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.14.394.861 I slot print_timing: id 3 | task 381 | prompt eval time = 99.48 ms / 15 tokens ( 6.63 ms per token, 150.78 tokens per second)
0.14.394.864 I slot print_timing: id 3 | task 381 | eval time = 296.67 ms / 15 tokens ( 19.78 ms per token, 50.56 tokens per second)
0.14.394.865 I slot print_timing: id 3 | task 381 | total time = 396.15 ms / 30 tokens
0.14.394.865 I slot print_timing: id 3 | task 381 | graphs reused = 358
0.14.394.889 I slot release: id 3 | task 381 | stop processing: n_tokens = 76, truncated = 0
0.14.394.895 I srv update_slots: all slots are idle
0.14.397.761 I srv operator(): Chat format: peg-native
0.14.397.951 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.746 (> 0.100 thold), f_keep = 0.618
0.14.397.975 I slot launch_slot_: id 3 | task 397 | processing task, is_child = 0
0.14.397.976 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.14.397.976 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.14.397.981 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.14.397.981 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.14.397.986 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.14.397.986 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.14.564.238 I slot print_timing: id 3 | task 397 | prompt eval time = 100.17 ms / 16 tokens ( 6.26 ms per token, 159.73 tokens per second)
0.14.564.242 I slot print_timing: id 3 | task 397 | eval time = 66.07 ms / 4 tokens ( 16.52 ms per token, 60.54 tokens per second)
0.14.564.242 I slot print_timing: id 3 | task 397 | total time = 166.24 ms / 20 tokens
0.14.564.243 I slot print_timing: id 3 | task 397 | graphs reused = 360
0.14.564.266 I slot release: id 3 | task 397 | stop processing: n_tokens = 66, truncated = 0
0.14.564.272 I srv update_slots: all slots are idle
0.14.567.209 I srv operator(): Chat format: peg-native
0.14.567.454 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.723 (> 0.100 thold), f_keep = 0.712
0.14.567.485 I slot launch_slot_: id 3 | task 402 | processing task, is_child = 0
0.14.567.485 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.14.567.486 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.14.567.494 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.14.567.494 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.14.567.500 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.14.567.500 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.15.135.858 I slot print_timing: id 3 | task 402 | prompt eval time = 123.99 ms / 18 tokens ( 6.89 ms per token, 145.18 tokens per second)
0.15.135.862 I slot print_timing: id 3 | task 402 | eval time = 444.36 ms / 22 tokens ( 20.20 ms per token, 49.51 tokens per second)
0.15.135.862 I slot print_timing: id 3 | task 402 | total time = 568.34 ms / 40 tokens
0.15.135.863 I slot print_timing: id 3 | task 402 | graphs reused = 380
0.15.135.884 I slot release: id 3 | task 402 | stop processing: n_tokens = 86, truncated = 0
0.15.135.890 I srv update_slots: all slots are idle
0.15.138.685 I srv operator(): Chat format: peg-native
0.15.138.873 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.770 (> 0.100 thold), f_keep = 0.547
0.15.138.905 I slot launch_slot_: id 3 | task 425 | processing task, is_child = 0
0.15.138.905 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.15.138.906 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.15.138.913 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.15.138.914 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.15.138.919 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.15.138.919 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.15.325.831 I slot print_timing: id 3 | task 425 | prompt eval time = 99.59 ms / 14 tokens ( 7.11 ms per token, 140.58 tokens per second)
0.15.325.835 I slot print_timing: id 3 | task 425 | eval time = 87.31 ms / 5 tokens ( 17.46 ms per token, 57.27 tokens per second)
0.15.325.836 I slot print_timing: id 3 | task 425 | total time = 186.90 ms / 19 tokens
0.15.325.836 I slot print_timing: id 3 | task 425 | graphs reused = 383
0.15.325.857 I slot release: id 3 | task 425 | stop processing: n_tokens = 65, truncated = 0
0.15.325.863 I srv update_slots: all slots are idle
0.15.328.846 I srv operator(): Chat format: peg-native
0.15.329.036 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.723 (> 0.100 thold), f_keep = 0.723
0.15.329.068 I slot launch_slot_: id 3 | task 431 | processing task, is_child = 0
0.15.329.068 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.15.329.069 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.15.329.076 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.15.329.077 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.15.329.082 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.15.329.083 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.16.128.954 I slot print_timing: id 3 | task 431 | prompt eval time = 124.03 ms / 18 tokens ( 6.89 ms per token, 145.12 tokens per second)
0.16.128.958 I slot print_timing: id 3 | task 431 | eval time = 675.82 ms / 33 tokens ( 20.48 ms per token, 48.83 tokens per second)
0.16.128.958 I slot print_timing: id 3 | task 431 | total time = 799.86 ms / 51 tokens
0.16.128.959 I slot print_timing: id 3 | task 431 | graphs reused = 414
0.16.128.982 I slot release: id 3 | task 431 | stop processing: n_tokens = 97, truncated = 0
0.16.128.989 I srv update_slots: all slots are idle
0.16.131.796 I srv operator(): Chat format: peg-native
0.16.131.987 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.783 (> 0.100 thold), f_keep = 0.485
0.16.131.989 I srv get_availabl: updating prompt cache
0.16.132.004 W srv prompt_save: - saving prompt with length 97, total state size = 12.127 MiB (draft: 0.000 MiB)
0.16.142.774 I srv load: - looking for better prompt, base f_keep = 0.485, sim = 0.783
0.16.142.781 I srv update: - cache state: 6 prompts, 79.887 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.16.142.782 I srv update: - prompt 0x644e41e619a0: 127 tokens, checkpoints: 0, 15.877 MiB
0.16.142.782 I srv update: - prompt 0x644e41e5c2c0: 110 tokens, checkpoints: 0, 13.752 MiB
0.16.142.783 I srv update: - prompt 0x644e41f0a560: 102 tokens, checkpoints: 0, 12.752 MiB
0.16.142.784 I srv update: - prompt 0x644e41f89010: 97 tokens, checkpoints: 0, 12.127 MiB
0.16.142.784 I srv update: - prompt 0x644e41db5c70: 106 tokens, checkpoints: 0, 13.252 MiB
0.16.142.785 I srv update: - prompt 0x644e41ed63e0: 97 tokens, checkpoints: 0, 12.127 MiB
0.16.142.785 I srv get_availabl: prompt cache update took 10.80 ms
0.16.142.828 I slot launch_slot_: id 3 | task 465 | processing task, is_child = 0
0.16.142.829 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.16.142.829 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.16.142.837 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.16.142.837 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.16.142.841 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.16.142.842 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.16.497.876 I slot print_timing: id 3 | task 465 | prompt eval time = 99.69 ms / 13 tokens ( 7.67 ms per token, 130.41 tokens per second)
0.16.497.879 I slot print_timing: id 3 | task 465 | eval time = 255.33 ms / 13 tokens ( 19.64 ms per token, 50.91 tokens per second)
0.16.497.880 I slot print_timing: id 3 | task 465 | total time = 355.02 ms / 26 tokens
0.16.497.881 I slot print_timing: id 3 | task 465 | graphs reused = 425
0.16.497.904 I slot release: id 3 | task 465 | stop processing: n_tokens = 72, truncated = 0
0.16.497.910 I srv update_slots: all slots are idle
0.16.501.237 I srv operator(): Chat format: peg-native
0.16.501.443 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.810 (> 0.100 thold), f_keep = 0.653
0.16.501.472 I slot launch_slot_: id 3 | task 479 | processing task, is_child = 0
0.16.501.473 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.16.501.474 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.16.501.480 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.16.501.480 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.16.501.485 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.16.501.485 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.16.708.888 I slot print_timing: id 3 | task 479 | prompt eval time = 99.39 ms / 11 tokens ( 9.04 ms per token, 110.67 tokens per second)
0.16.708.890 I slot print_timing: id 3 | task 479 | eval time = 108.00 ms / 6 tokens ( 18.00 ms per token, 55.56 tokens per second)
0.16.708.891 I slot print_timing: id 3 | task 479 | total time = 207.39 ms / 17 tokens
0.16.708.892 I slot print_timing: id 3 | task 479 | graphs reused = 429
0.16.708.913 I slot release: id 3 | task 479 | stop processing: n_tokens = 63, truncated = 0
0.16.708.918 I srv update_slots: all slots are idle
0.16.711.991 I srv operator(): Chat format: peg-native
0.16.712.183 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.828 (> 0.100 thold), f_keep = 0.762
0.16.712.208 I slot launch_slot_: id 3 | task 486 | processing task, is_child = 0
0.16.712.209 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.16.712.209 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.16.712.215 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.16.712.215 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.16.712.219 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.16.712.220 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.16.982.877 I slot print_timing: id 3 | task 486 | prompt eval time = 99.09 ms / 10 tokens ( 9.91 ms per token, 100.92 tokens per second)
0.16.982.880 I slot print_timing: id 3 | task 486 | eval time = 171.55 ms / 9 tokens ( 19.06 ms per token, 52.46 tokens per second)
0.16.982.881 I slot print_timing: id 3 | task 486 | total time = 270.64 ms / 19 tokens
0.16.982.881 I slot print_timing: id 3 | task 486 | graphs reused = 436
0.16.982.903 I slot release: id 3 | task 486 | stop processing: n_tokens = 66, truncated = 0
0.16.982.908 I srv update_slots: all slots are idle
0.16.985.900 I srv operator(): Chat format: peg-native
0.16.986.093 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.825 (> 0.100 thold), f_keep = 0.712
0.16.986.115 I slot launch_slot_: id 3 | task 496 | processing task, is_child = 0
0.16.986.116 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.16.986.116 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.16.986.123 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.16.986.123 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.16.986.127 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.16.986.128 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.17.319.871 I slot print_timing: id 3 | task 496 | prompt eval time = 99.44 ms / 10 tokens ( 9.94 ms per token, 100.56 tokens per second)
0.17.319.875 I slot print_timing: id 3 | task 496 | eval time = 234.29 ms / 12 tokens ( 19.52 ms per token, 51.22 tokens per second)
0.17.319.877 I slot print_timing: id 3 | task 496 | total time = 333.73 ms / 22 tokens
0.17.319.877 I slot print_timing: id 3 | task 496 | graphs reused = 446
0.17.319.898 I slot release: id 3 | task 496 | stop processing: n_tokens = 68, truncated = 0
0.17.319.905 I srv update_slots: all slots are idle
0.17.322.722 I srv operator(): Chat format: peg-native
0.17.323.001 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.758 (> 0.100 thold), f_keep = 0.691
0.17.323.026 I slot launch_slot_: id 3 | task 509 | processing task, is_child = 0
0.17.323.026 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.17.323.027 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.17.323.033 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.17.323.033 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.17.323.037 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.17.323.038 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.17.467.668 I slot print_timing: id 3 | task 509 | prompt eval time = 99.41 ms / 15 tokens ( 6.63 ms per token, 150.89 tokens per second)
0.17.467.672 I slot print_timing: id 3 | task 509 | eval time = 45.20 ms / 3 tokens ( 15.07 ms per token, 66.37 tokens per second)
0.17.467.672 I slot print_timing: id 3 | task 509 | total time = 144.62 ms / 18 tokens
0.17.467.673 I slot print_timing: id 3 | task 509 | graphs reused = 447
0.17.467.696 I slot release: id 3 | task 509 | stop processing: n_tokens = 64, truncated = 0
0.17.467.705 I srv update_slots: all slots are idle
0.17.470.546 I srv operator(): Chat format: peg-native
0.17.470.736 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.770 (> 0.100 thold), f_keep = 0.734
0.17.470.766 I slot launch_slot_: id 3 | task 513 | processing task, is_child = 0
0.17.470.766 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.17.470.767 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.17.470.773 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.17.470.774 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.17.470.779 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.17.470.780 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.17.929.886 I slot print_timing: id 3 | task 513 | prompt eval time = 99.27 ms / 14 tokens ( 7.09 ms per token, 141.03 tokens per second)
0.17.929.889 I slot print_timing: id 3 | task 513 | eval time = 359.82 ms / 18 tokens ( 19.99 ms per token, 50.02 tokens per second)
0.17.929.890 I slot print_timing: id 3 | task 513 | total time = 459.09 ms / 32 tokens
0.17.929.891 I slot print_timing: id 3 | task 513 | graphs reused = 463
0.17.929.911 I slot release: id 3 | task 513 | stop processing: n_tokens = 78, truncated = 0
0.17.929.917 I srv update_slots: all slots are idle
0.17.932.895 I srv operator(): Chat format: peg-native
0.17.933.093 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.712 (> 0.100 thold), f_keep = 0.603
0.17.933.118 I slot launch_slot_: id 3 | task 532 | processing task, is_child = 0
0.17.933.118 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.17.933.119 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.17.933.125 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.17.933.125 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.17.933.129 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.17.933.130 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.18.521.903 I slot print_timing: id 3 | task 532 | prompt eval time = 124.23 ms / 19 tokens ( 6.54 ms per token, 152.94 tokens per second)
0.18.521.906 I slot print_timing: id 3 | task 532 | eval time = 464.53 ms / 23 tokens ( 20.20 ms per token, 49.51 tokens per second)
0.18.521.907 I slot print_timing: id 3 | task 532 | total time = 588.76 ms / 42 tokens
0.18.521.908 I slot print_timing: id 3 | task 532 | graphs reused = 484
0.18.521.930 I slot release: id 3 | task 532 | stop processing: n_tokens = 88, truncated = 0
0.18.521.937 I srv update_slots: all slots are idle
0.18.524.809 I srv operator(): Chat format: peg-native
0.18.525.005 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.712 (> 0.100 thold), f_keep = 0.534
0.18.525.029 I slot launch_slot_: id 3 | task 556 | processing task, is_child = 0
0.18.525.030 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.18.525.030 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.18.525.036 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.18.525.037 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.18.525.041 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.18.525.041 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.19.135.899 I slot print_timing: id 3 | task 556 | prompt eval time = 124.30 ms / 19 tokens ( 6.54 ms per token, 152.86 tokens per second)
0.19.135.903 I slot print_timing: id 3 | task 556 | eval time = 486.55 ms / 24 tokens ( 20.27 ms per token, 49.33 tokens per second)
0.19.135.904 I slot print_timing: id 3 | task 556 | total time = 610.84 ms / 43 tokens
0.19.135.904 I slot print_timing: id 3 | task 556 | graphs reused = 506
0.19.135.925 I slot release: id 3 | task 556 | stop processing: n_tokens = 89, truncated = 0
0.19.135.932 I srv update_slots: all slots are idle
0.19.138.752 I srv operator(): Chat format: peg-native
0.19.138.947 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.825 (> 0.100 thold), f_keep = 0.528
0.19.138.970 I slot launch_slot_: id 3 | task 581 | processing task, is_child = 0
0.19.138.971 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.19.138.971 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.19.138.977 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.19.138.977 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.19.138.982 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.19.138.982 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.19.324.898 I slot print_timing: id 3 | task 581 | prompt eval time = 99.34 ms / 10 tokens ( 9.93 ms per token, 100.66 tokens per second)
0.19.324.901 I slot print_timing: id 3 | task 581 | eval time = 86.56 ms / 5 tokens ( 17.31 ms per token, 57.76 tokens per second)
0.19.324.901 I slot print_timing: id 3 | task 581 | total time = 185.90 ms / 15 tokens
0.19.324.902 I slot print_timing: id 3 | task 581 | graphs reused = 509
0.19.324.924 I slot release: id 3 | task 581 | stop processing: n_tokens = 61, truncated = 0
0.19.324.930 I srv update_slots: all slots are idle
0.19.327.677 I srv operator(): Chat format: peg-native
0.19.327.873 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.770 (> 0.100 thold), f_keep = 0.770
0.19.327.898 I slot launch_slot_: id 3 | task 587 | processing task, is_child = 0
0.19.327.899 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.19.327.899 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.19.327.906 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.19.327.906 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.19.327.911 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.19.327.911 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.19.493.772 I slot print_timing: id 3 | task 587 | prompt eval time = 99.42 ms / 14 tokens ( 7.10 ms per token, 140.82 tokens per second)
0.19.493.775 I slot print_timing: id 3 | task 587 | eval time = 66.43 ms / 4 tokens ( 16.61 ms per token, 60.21 tokens per second)
0.19.493.776 I slot print_timing: id 3 | task 587 | total time = 165.85 ms / 18 tokens
0.19.493.776 I slot print_timing: id 3 | task 587 | graphs reused = 511
0.19.493.799 I slot release: id 3 | task 587 | stop processing: n_tokens = 64, truncated = 0
0.19.493.806 I srv update_slots: all slots are idle
0.19.496.571 I srv operator(): Chat format: peg-native
0.19.496.759 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.746 (> 0.100 thold), f_keep = 0.734
0.19.496.780 I slot launch_slot_: id 3 | task 592 | processing task, is_child = 0
0.19.496.781 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.19.496.781 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.19.496.787 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.19.496.788 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.19.496.792 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.19.496.792 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.19.979.925 I slot print_timing: id 3 | task 592 | prompt eval time = 99.73 ms / 16 tokens ( 6.23 ms per token, 160.43 tokens per second)
0.19.979.928 I slot print_timing: id 3 | task 592 | eval time = 383.39 ms / 19 tokens ( 20.18 ms per token, 49.56 tokens per second)
0.19.979.929 I slot print_timing: id 3 | task 592 | total time = 483.12 ms / 35 tokens
0.19.979.930 I slot print_timing: id 3 | task 592 | graphs reused = 528
0.19.979.951 I slot release: id 3 | task 592 | stop processing: n_tokens = 81, truncated = 0
0.19.979.957 I srv update_slots: all slots are idle
0.19.982.771 I srv operator(): Chat format: peg-native
0.19.982.957 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.635 (> 0.100 thold), f_keep = 0.580
0.19.982.982 I slot launch_slot_: id 3 | task 612 | processing task, is_child = 0
0.19.982.982 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.19.982.983 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.19.982.988 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.19.982.988 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.19.982.993 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.19.982.993 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.20.686.904 I slot print_timing: id 3 | task 612 | prompt eval time = 133.87 ms / 27 tokens ( 4.96 ms per token, 201.69 tokens per second)
0.20.686.908 I slot print_timing: id 3 | task 612 | eval time = 570.03 ms / 28 tokens ( 20.36 ms per token, 49.12 tokens per second)
0.20.686.908 I slot print_timing: id 3 | task 612 | total time = 703.90 ms / 55 tokens
0.20.686.909 I slot print_timing: id 3 | task 612 | graphs reused = 554
0.20.686.935 I slot release: id 3 | task 612 | stop processing: n_tokens = 101, truncated = 0
0.20.686.941 I srv update_slots: all slots are idle
0.20.689.843 I srv operator(): Chat format: peg-native
0.20.690.032 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.738 (> 0.100 thold), f_keep = 0.475
0.20.690.034 I srv get_availabl: updating prompt cache
0.20.690.055 W srv prompt_save: - saving prompt with length 101, total state size = 12.627 MiB (draft: 0.000 MiB)
0.20.701.063 I srv load: - looking for better prompt, base f_keep = 0.475, sim = 0.738
0.20.701.070 I srv update: - cache state: 7 prompts, 92.514 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.20.701.071 I srv update: - prompt 0x644e41e619a0: 127 tokens, checkpoints: 0, 15.877 MiB
0.20.701.072 I srv update: - prompt 0x644e41e5c2c0: 110 tokens, checkpoints: 0, 13.752 MiB
0.20.701.072 I srv update: - prompt 0x644e41f0a560: 102 tokens, checkpoints: 0, 12.752 MiB
0.20.701.073 I srv update: - prompt 0x644e41f89010: 97 tokens, checkpoints: 0, 12.127 MiB
0.20.701.073 I srv update: - prompt 0x644e41db5c70: 106 tokens, checkpoints: 0, 13.252 MiB
0.20.701.074 I srv update: - prompt 0x644e41ed63e0: 97 tokens, checkpoints: 0, 12.127 MiB
0.20.701.074 I srv update: - prompt 0x644e41ee8f00: 101 tokens, checkpoints: 0, 12.627 MiB
0.20.701.075 I srv get_availabl: prompt cache update took 11.04 ms
0.20.701.119 I slot launch_slot_: id 3 | task 641 | processing task, is_child = 0
0.20.701.119 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.20.701.120 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.20.701.125 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.20.701.126 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.20.701.130 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.20.701.130 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.21.478.935 I slot print_timing: id 3 | task 641 | prompt eval time = 124.00 ms / 17 tokens ( 7.29 ms per token, 137.10 tokens per second)
0.21.478.939 I slot print_timing: id 3 | task 641 | eval time = 653.79 ms / 32 tokens ( 20.43 ms per token, 48.95 tokens per second)
0.21.478.940 I slot print_timing: id 3 | task 641 | total time = 777.79 ms / 49 tokens
0.21.478.940 I slot print_timing: id 3 | task 641 | graphs reused = 584
0.21.478.964 I slot release: id 3 | task 641 | stop processing: n_tokens = 96, truncated = 0
0.21.478.970 I srv update_slots: all slots are idle
0.21.481.857 I srv operator(): Chat format: peg-native
0.21.482.060 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.783 (> 0.100 thold), f_keep = 0.490
0.21.482.062 I srv get_availabl: updating prompt cache
0.21.482.080 W srv prompt_save: - saving prompt with length 96, total state size = 12.002 MiB (draft: 0.000 MiB)
0.21.492.686 I srv load: - looking for better prompt, base f_keep = 0.490, sim = 0.783
0.21.492.692 I srv update: - cache state: 8 prompts, 104.516 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.21.492.693 I srv update: - prompt 0x644e41e619a0: 127 tokens, checkpoints: 0, 15.877 MiB
0.21.492.694 I srv update: - prompt 0x644e41e5c2c0: 110 tokens, checkpoints: 0, 13.752 MiB
0.21.492.695 I srv update: - prompt 0x644e41f0a560: 102 tokens, checkpoints: 0, 12.752 MiB
0.21.492.695 I srv update: - prompt 0x644e41f89010: 97 tokens, checkpoints: 0, 12.127 MiB
0.21.492.696 I srv update: - prompt 0x644e41db5c70: 106 tokens, checkpoints: 0, 13.252 MiB
0.21.492.697 I srv update: - prompt 0x644e41ed63e0: 97 tokens, checkpoints: 0, 12.127 MiB
0.21.492.698 I srv update: - prompt 0x644e41ee8f00: 101 tokens, checkpoints: 0, 12.627 MiB
0.21.492.698 I srv update: - prompt 0x644e41dbebb0: 96 tokens, checkpoints: 0, 12.002 MiB
0.21.492.699 I srv get_availabl: prompt cache update took 10.64 ms
0.21.492.738 I slot launch_slot_: id 3 | task 674 | processing task, is_child = 0
0.21.492.739 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.21.492.740 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.21.492.746 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.21.492.746 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.21.492.750 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.21.492.751 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.24.429.987 I slot print_timing: id 3 | task 674 | prompt eval time = 99.28 ms / 13 tokens ( 7.64 ms per token, 130.95 tokens per second)
0.24.429.990 I slot print_timing: id 3 | task 674 | eval time = 2837.94 ms / 136 tokens ( 20.87 ms per token, 47.92 tokens per second)
0.24.429.991 I slot print_timing: id 3 | task 674 | total time = 2937.22 ms / 149 tokens
0.24.429.992 I slot print_timing: id 3 | task 674 | graphs reused = 718
0.24.430.012 I slot release: id 3 | task 674 | stop processing: n_tokens = 195, truncated = 0
0.24.430.021 I srv update_slots: all slots are idle
0.24.432.861 I srv operator(): Chat format: peg-native
0.24.433.057 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.712 (> 0.100 thold), f_keep = 0.241
0.24.433.060 I srv get_availabl: updating prompt cache
0.24.433.085 W srv prompt_save: - saving prompt with length 195, total state size = 24.378 MiB (draft: 0.000 MiB)
0.24.453.057 I srv load: - looking for better prompt, base f_keep = 0.241, sim = 0.712
0.24.453.064 I srv load: - found better prompt with f_keep = 0.378, sim = 0.727
0.24.461.103 I srv update: - cache state: 8 prompts, 113.016 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.24.461.107 I srv update: - prompt 0x644e41e5c2c0: 110 tokens, checkpoints: 0, 13.752 MiB
0.24.461.107 I srv update: - prompt 0x644e41f0a560: 102 tokens, checkpoints: 0, 12.752 MiB
0.24.461.108 I srv update: - prompt 0x644e41f89010: 97 tokens, checkpoints: 0, 12.127 MiB
0.24.461.108 I srv update: - prompt 0x644e41db5c70: 106 tokens, checkpoints: 0, 13.252 MiB
0.24.461.109 I srv update: - prompt 0x644e41ed63e0: 97 tokens, checkpoints: 0, 12.127 MiB
0.24.461.110 I srv update: - prompt 0x644e41ee8f00: 101 tokens, checkpoints: 0, 12.627 MiB
0.24.461.110 I srv update: - prompt 0x644e41dbebb0: 96 tokens, checkpoints: 0, 12.002 MiB
0.24.461.111 I srv update: - prompt 0x644e41f19370: 195 tokens, checkpoints: 0, 24.378 MiB
0.24.461.112 I srv get_availabl: prompt cache update took 28.05 ms
0.24.461.157 I slot launch_slot_: id 3 | task 811 | processing task, is_child = 0
0.24.461.157 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.24.461.158 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.24.461.165 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.24.461.165 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.24.461.170 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.24.461.170 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.25.469.962 I slot print_timing: id 3 | task 811 | prompt eval time = 124.11 ms / 18 tokens ( 6.90 ms per token, 145.03 tokens per second)
0.25.469.966 I slot print_timing: id 3 | task 811 | eval time = 884.66 ms / 43 tokens ( 20.57 ms per token, 48.61 tokens per second)
0.25.469.967 I slot print_timing: id 3 | task 811 | total time = 1008.77 ms / 61 tokens
0.25.469.967 I slot print_timing: id 3 | task 811 | graphs reused = 759
0.25.469.990 I slot release: id 3 | task 811 | stop processing: n_tokens = 108, truncated = 0
0.25.469.996 I srv update_slots: all slots are idle
0.25.473.198 I srv operator(): Chat format: peg-native
0.25.473.401 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.727 (> 0.100 thold), f_keep = 0.444
0.25.473.403 I srv get_availabl: updating prompt cache
0.25.473.425 W srv prompt_save: - saving prompt with length 108, total state size = 13.502 MiB (draft: 0.000 MiB)
0.25.485.529 I srv load: - looking for better prompt, base f_keep = 0.444, sim = 0.727
0.25.485.535 I srv load: - found better prompt with f_keep = 0.515, sim = 0.758
0.25.490.744 I srv update: - cache state: 8 prompts, 114.392 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.25.490.748 I srv update: - prompt 0x644e41e5c2c0: 110 tokens, checkpoints: 0, 13.752 MiB
0.25.490.749 I srv update: - prompt 0x644e41f0a560: 102 tokens, checkpoints: 0, 12.752 MiB
0.25.490.749 I srv update: - prompt 0x644e41db5c70: 106 tokens, checkpoints: 0, 13.252 MiB
0.25.490.750 I srv update: - prompt 0x644e41ed63e0: 97 tokens, checkpoints: 0, 12.127 MiB
0.25.490.750 I srv update: - prompt 0x644e41ee8f00: 101 tokens, checkpoints: 0, 12.627 MiB
0.25.490.751 I srv update: - prompt 0x644e41dbebb0: 96 tokens, checkpoints: 0, 12.002 MiB
0.25.490.751 I srv update: - prompt 0x644e41f19370: 195 tokens, checkpoints: 0, 24.378 MiB
0.25.490.752 I srv update: - prompt 0x644e41e6c220: 108 tokens, checkpoints: 0, 13.502 MiB
0.25.490.753 I srv get_availabl: prompt cache update took 17.35 ms
0.25.490.794 I slot launch_slot_: id 3 | task 855 | processing task, is_child = 0
0.25.490.794 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.25.490.795 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.25.490.801 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.25.490.802 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.25.490.806 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.25.490.807 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.27.525.653 I slot print_timing: id 3 | task 855 | prompt eval time = 100.14 ms / 16 tokens ( 6.26 ms per token, 159.77 tokens per second)
0.27.525.657 I slot print_timing: id 3 | task 855 | eval time = 1934.69 ms / 93 tokens ( 20.80 ms per token, 48.07 tokens per second)
0.27.525.657 I slot print_timing: id 3 | task 855 | total time = 2034.83 ms / 109 tokens
0.27.525.658 I slot print_timing: id 3 | task 855 | graphs reused = 850
0.27.525.680 I slot release: id 3 | task 855 | stop processing: n_tokens = 158, truncated = 0
0.27.525.686 I srv update_slots: all slots are idle
0.27.528.558 I srv operator(): Chat format: peg-native
0.27.528.750 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.797 (> 0.100 thold), f_keep = 0.297
0.27.528.751 I srv get_availabl: updating prompt cache
0.27.528.772 W srv prompt_save: - saving prompt with length 158, total state size = 19.753 MiB (draft: 0.000 MiB)
0.27.545.201 I srv load: - looking for better prompt, base f_keep = 0.297, sim = 0.797
0.27.545.211 I srv update: - cache state: 9 prompts, 134.144 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.27.545.211 I srv update: - prompt 0x644e41e5c2c0: 110 tokens, checkpoints: 0, 13.752 MiB
0.27.545.212 I srv update: - prompt 0x644e41f0a560: 102 tokens, checkpoints: 0, 12.752 MiB
0.27.545.212 I srv update: - prompt 0x644e41db5c70: 106 tokens, checkpoints: 0, 13.252 MiB
0.27.545.213 I srv update: - prompt 0x644e41ed63e0: 97 tokens, checkpoints: 0, 12.127 MiB
0.27.545.214 I srv update: - prompt 0x644e41ee8f00: 101 tokens, checkpoints: 0, 12.627 MiB
0.27.545.214 I srv update: - prompt 0x644e41dbebb0: 96 tokens, checkpoints: 0, 12.002 MiB
0.27.545.215 I srv update: - prompt 0x644e41f19370: 195 tokens, checkpoints: 0, 24.378 MiB
0.27.545.215 I srv update: - prompt 0x644e41e6c220: 108 tokens, checkpoints: 0, 13.502 MiB
0.27.545.216 I srv update: - prompt 0x644e4201b9e0: 158 tokens, checkpoints: 0, 19.753 MiB
0.27.545.217 I srv get_availabl: prompt cache update took 16.46 ms
0.27.545.265 I slot launch_slot_: id 3 | task 949 | processing task, is_child = 0
0.27.545.266 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.27.545.266 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.27.545.274 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.27.545.274 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.27.545.290 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.27.545.291 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.28.256.973 I slot print_timing: id 3 | task 949 | prompt eval time = 99.72 ms / 12 tokens ( 8.31 ms per token, 120.34 tokens per second)
0.28.256.977 I slot print_timing: id 3 | task 949 | eval time = 611.95 ms / 30 tokens ( 20.40 ms per token, 49.02 tokens per second)
0.28.256.977 I slot print_timing: id 3 | task 949 | total time = 711.66 ms / 42 tokens
0.28.256.978 I slot print_timing: id 3 | task 949 | graphs reused = 878
0.28.256.999 I slot release: id 3 | task 949 | stop processing: n_tokens = 88, truncated = 0
0.28.257.005 I srv update_slots: all slots are idle
0.28.259.688 I srv operator(): Chat format: peg-native
0.28.259.873 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.839 (> 0.100 thold), f_keep = 0.534
0.28.259.901 I slot launch_slot_: id 3 | task 980 | processing task, is_child = 0
0.28.259.902 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.28.259.903 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.28.259.908 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.28.259.908 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.28.259.912 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.28.259.913 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.29.011.966 I slot print_timing: id 3 | task 980 | prompt eval time = 98.91 ms / 9 tokens ( 10.99 ms per token, 90.99 tokens per second)
0.29.011.970 I slot print_timing: id 3 | task 980 | eval time = 653.13 ms / 32 tokens ( 20.41 ms per token, 48.99 tokens per second)
0.29.011.971 I slot print_timing: id 3 | task 980 | total time = 752.04 ms / 41 tokens
0.29.011.971 I slot print_timing: id 3 | task 980 | graphs reused = 908
0.29.011.992 I slot release: id 3 | task 980 | stop processing: n_tokens = 87, truncated = 0
0.29.011.998 I srv update_slots: all slots are idle
0.29.014.928 I srv operator(): Chat format: peg-native
0.29.015.128 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.746 (> 0.100 thold), f_keep = 0.540
0.29.015.153 I slot launch_slot_: id 3 | task 1013 | processing task, is_child = 0
0.29.015.154 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.015.155 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.015.161 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.015.161 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.015.166 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.015.166 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.29.600.642 I slot print_timing: id 3 | task 1013 | prompt eval time = 99.98 ms / 16 tokens ( 6.25 ms per token, 160.04 tokens per second)
0.29.600.645 I slot print_timing: id 3 | task 1013 | eval time = 485.48 ms / 24 tokens ( 20.23 ms per token, 49.44 tokens per second)
0.29.600.646 I slot print_timing: id 3 | task 1013 | total time = 585.46 ms / 40 tokens
0.29.600.646 I slot print_timing: id 3 | task 1013 | graphs reused = 930
0.29.600.668 I slot release: id 3 | task 1013 | stop processing: n_tokens = 86, truncated = 0
0.29.600.676 I srv update_slots: all slots are idle
0.29.603.441 I srv operator(): Chat format: peg-native
0.29.603.638 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.783 (> 0.100 thold), f_keep = 0.547
0.29.603.664 I slot launch_slot_: id 3 | task 1038 | processing task, is_child = 0
0.29.603.664 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.29.603.665 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.29.603.671 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.29.603.671 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.29.603.676 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.29.603.676 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.30.734.656 I slot print_timing: id 3 | task 1038 | prompt eval time = 99.34 ms / 13 tokens ( 7.64 ms per token, 130.87 tokens per second)
0.30.734.660 I slot print_timing: id 3 | task 1038 | eval time = 1031.63 ms / 50 tokens ( 20.63 ms per token, 48.47 tokens per second)
0.30.734.660 I slot print_timing: id 3 | task 1038 | total time = 1130.97 ms / 63 tokens
0.30.734.661 I slot print_timing: id 3 | task 1038 | graphs reused = 978
0.30.734.684 I slot release: id 3 | task 1038 | stop processing: n_tokens = 109, truncated = 0
0.30.734.692 I srv update_slots: all slots are idle
0.30.737.454 I srv operator(): Chat format: peg-native
0.30.737.643 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.828 (> 0.100 thold), f_keep = 0.440
0.30.737.644 I srv get_availabl: updating prompt cache
0.30.737.664 W srv prompt_save: - saving prompt with length 109, total state size = 13.627 MiB (draft: 0.000 MiB)
0.30.749.598 I srv load: - looking for better prompt, base f_keep = 0.440, sim = 0.828
0.30.749.607 I srv update: - cache state: 10 prompts, 147.771 MiB (limits: 8192.000 MiB, 8192 tokens, 65526 est)
0.30.749.607 I srv update: - prompt 0x644e41e5c2c0: 110 tokens, checkpoints: 0, 13.752 MiB
0.30.749.608 I srv update: - prompt 0x644e41f0a560: 102 tokens, checkpoints: 0, 12.752 MiB
0.30.749.610 I srv update: - prompt 0x644e41db5c70: 106 tokens, checkpoints: 0, 13.252 MiB
0.30.749.611 I srv update: - prompt 0x644e41ed63e0: 97 tokens, checkpoints: 0, 12.127 MiB
0.30.749.612 I srv update: - prompt 0x644e41ee8f00: 101 tokens, checkpoints: 0, 12.627 MiB
0.30.749.612 I srv update: - prompt 0x644e41dbebb0: 96 tokens, checkpoints: 0, 12.002 MiB
0.30.749.613 I srv update: - prompt 0x644e41f19370: 195 tokens, checkpoints: 0, 24.378 MiB
0.30.749.613 I srv update: - prompt 0x644e41e6c220: 108 tokens, checkpoints: 0, 13.502 MiB
0.30.749.614 I srv update: - prompt 0x644e4201b9e0: 158 tokens, checkpoints: 0, 19.753 MiB
0.30.749.614 I srv update: - prompt 0x644e41f23150: 109 tokens, checkpoints: 0, 13.627 MiB
0.30.749.615 I srv get_availabl: prompt cache update took 11.97 ms
0.30.749.657 I slot launch_slot_: id 3 | task 1089 | processing task, is_child = 0
0.30.749.657 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.30.749.658 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.30.749.665 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.30.749.665 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.30.749.669 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.30.749.670 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.31.586.647 I slot print_timing: id 3 | task 1089 | prompt eval time = 99.56 ms / 10 tokens ( 9.96 ms per token, 100.45 tokens per second)
0.31.586.650 I slot print_timing: id 3 | task 1089 | eval time = 737.41 ms / 36 tokens ( 20.48 ms per token, 48.82 tokens per second)
0.31.586.651 I slot print_timing: id 3 | task 1089 | total time = 836.96 ms / 46 tokens
0.31.586.651 I slot print_timing: id 3 | task 1089 | graphs reused = 1012
0.31.586.673 I slot release: id 3 | task 1089 | stop processing: n_tokens = 93, truncated = 0
0.31.586.680 I srv update_slots: all slots are idle
0.31.589.613 I srv operator(): Chat format: peg-native
0.31.589.806 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.758 (> 0.100 thold), f_keep = 0.505
0.31.589.836 I slot launch_slot_: id 3 | task 1126 | processing task, is_child = 0
0.31.589.837 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.31.589.838 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.31.589.844 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.31.589.844 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.31.589.849 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.31.589.849 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.32.238.541 I slot print_timing: id 3 | task 1126 | prompt eval time = 99.42 ms / 15 tokens ( 6.63 ms per token, 150.88 tokens per second)
0.32.238.544 I slot print_timing: id 3 | task 1126 | eval time = 549.26 ms / 27 tokens ( 20.34 ms per token, 49.16 tokens per second)
0.32.238.545 I slot print_timing: id 3 | task 1126 | total time = 648.68 ms / 42 tokens
0.32.238.546 I slot print_timing: id 3 | task 1126 | graphs reused = 1037
0.32.238.567 I slot release: id 3 | task 1126 | stop processing: n_tokens = 88, truncated = 0
0.32.238.573 I srv update_slots: all slots are idle
0.32.241.348 I srv operator(): Chat format: peg-native
0.32.241.546 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.839 (> 0.100 thold), f_keep = 0.534
0.32.241.568 I slot launch_slot_: id 3 | task 1154 | processing task, is_child = 0
0.32.241.569 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.32.241.570 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
0.32.241.576 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.32.241.576 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
0.32.241.580 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.32.241.581 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
0.33.245.680 I slot print_timing: id 3 | task 1154 | prompt eval time = 99.16 ms / 9 tokens ( 11.02 ms per token, 90.76 tokens per second)
0.33.245.684 I slot print_timing: id 3 | task 1154 | eval time = 904.92 ms / 44 tokens ( 20.57 ms per token, 48.62 tokens per second)
0.33.245.684 I slot print_timing: id 3 | task 1154 | total time = 1004.09 ms / 53 tokens
0.33.245.685 I slot print_timing: id 3 | task 1154 | graphs reused = 1079
0.33.245.704 I slot release: id 3 | task 1154 | stop processing: n_tokens = 99, truncated = 0
0.33.245.712 I srv update_slots: all slots are idle
0.33.246.184 I srv operator(): operator(): cleaning up before exit...
Reference in New Issue Copy Permalink