1153 lines
112 KiB
Plaintext
1153 lines
112 KiB
Plaintext
|
G
|
0.00.067.256 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
|
||
|
|
0.00.067.259 I device_info:
|
||
|
|
0.00.067.279 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
||
|
|
0.00.067.283 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
||
|
|
0.00.067.290 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
|
||
|
|
0.00.067.354 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
||
|
|
0.00.067.358 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
|
||
|
|
0.00.067.396 I srv init: running without SSL
|
||
|
|
0.00.067.458 I srv init: using 23 threads for HTTP server
|
||
|
|
0.00.067.786 I srv start: binding port with default address family
|
||
|
|
0.00.068.997 I srv llama_server: loading model
|
||
|
|
0.00.069.006 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/openai_gpt-oss-20b-Q5_K_M.gguf'
|
||
|
|
0.00.069.056 I common_init_result: fitting params to device memory ...
|
||
|
|
0.00.069.058 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
|
||
|
|
0.01.535.378 W load: setting token '<|message|>' (200008) attribute to USER_DEFINED (16), old attributes: 8
|
||
|
|
0.01.535.381 W load: setting token '<|start|>' (200006) attribute to USER_DEFINED (16), old attributes: 8
|
||
|
|
0.01.535.382 W load: setting token '<|constrain|>' (200003) attribute to USER_DEFINED (16), old attributes: 8
|
||
|
|
0.01.536.532 W load: setting token '<|channel|>' (200005) attribute to USER_DEFINED (16), old attributes: 8
|
||
|
|
0.01.541.695 W load: special_eog_ids contains both '<|return|>' and '<|call|>', or '<|calls|>' and '<|flush|>' tokens, removing '<|end|>' token from EOG list
|
||
|
|
0.07.057.032 W llama_context: n_ctx_seq (8192) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
|
||
|
|
0.07.112.222 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
||
|
|
0.07.293.729 I srv load_model: initializing slots, n_slots = 4
|
||
|
|
0.07.326.591 W common_speculative_init: no implementations specified for speculative decoding
|
||
|
|
0.07.326.597 I slot load_model: id 0 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.07.326.602 I slot load_model: id 1 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.07.326.603 I slot load_model: id 2 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.07.326.603 I slot load_model: id 3 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.07.326.681 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
|
||
|
|
0.07.326.682 I srv load_model: use `--cache-ram 0` to disable the prompt cache
|
||
|
|
0.07.326.682 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
||
|
|
0.07.326.683 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
|
||
|
|
0.07.326.702 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
|
||
|
|
0.07.330.332 I init: chat template, example_format: '<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.
|
||
|
|
Knowledge cutoff: 2024-06
|
||
|
|
Current date: 2026-07-11
|
||
|
|
|
||
|
|
Reasoning: medium
|
||
|
|
|
||
|
|
# Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>developer<|message|># Instructions
|
||
|
|
|
||
|
|
You are a helpful assistant<|end|><|start|>user<|message|>Hello<|end|><|start|>assistant<|channel|>final<|message|>Hi there<|end|><|start|>user<|message|>How are you?<|end|><|start|>assistant'
|
||
|
|
0.07.330.897 I srv init: init: chat template, thinking = 1
|
||
|
|
0.07.330.930 I srv llama_server: model loaded
|
||
|
|
0.07.330.933 I srv llama_server: server is listening on http://127.0.0.1:8090
|
||
|
|
0.07.330.938 I srv update_slots: all slots are idle
|
||
|
|
0.09.969.994 I srv operator(): Chat format: peg-native
|
||
|
|
0.09.973.818 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
|
||
|
|
0.09.973.821 I srv get_availabl: updating prompt cache
|
||
|
|
0.09.973.826 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
||
|
|
0.09.973.831 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
|
||
|
|
0.09.973.832 I srv get_availabl: prompt cache update took 0.01 ms
|
||
|
|
0.09.974.816 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
|
||
|
|
0.09.974.818 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.09.974.819 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.09.974.829 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.09.974.829 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.09.974.834 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.09.974.834 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.10.657.729 I slot create_check: id 3 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 108, n_tokens = 109, size = 2.556 MiB)
|
||
|
|
0.12.150.869 I slot print_timing: id 3 | task 0 | prompt eval time = 794.83 ms / 133 tokens ( 5.98 ms per token, 167.33 tokens per second)
|
||
|
|
0.12.150.872 I slot print_timing: id 3 | task 0 | eval time = 1381.19 ms / 122 tokens ( 11.32 ms per token, 88.33 tokens per second)
|
||
|
|
0.12.150.873 I slot print_timing: id 3 | task 0 | total time = 2176.01 ms / 255 tokens
|
||
|
|
0.12.150.878 I slot print_timing: id 3 | task 0 | graphs reused = 121
|
||
|
|
0.12.150.923 I slot release: id 3 | task 0 | stop processing: n_tokens = 254, truncated = 0
|
||
|
|
0.12.150.928 I srv update_slots: all slots are idle
|
||
|
|
0.12.159.057 I srv operator(): Chat format: peg-native
|
||
|
|
0.12.162.776 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.896 (> 0.100 thold), f_keep = 0.441
|
||
|
|
0.12.162.778 I srv get_availabl: updating prompt cache
|
||
|
|
0.12.162.806 W srv prompt_save: - saving prompt with length 254, total state size = 8.958 MiB (draft: 0.000 MiB)
|
||
|
|
0.12.168.032 I srv load: - looking for better prompt, base f_keep = 0.441, sim = 0.896
|
||
|
|
0.12.168.039 I srv update: - cache state: 1 prompts, 11.514 MiB (limits: 8192.000 MiB, 8192 tokens, 180711 est)
|
||
|
|
0.12.168.040 I srv update: - prompt 0x5bea71423ba0: 254 tokens, checkpoints: 1, 11.514 MiB
|
||
|
|
0.12.168.041 I srv get_availabl: prompt cache update took 5.26 ms
|
||
|
|
0.12.168.958 I slot launch_slot_: id 3 | task 125 | processing task, is_child = 0
|
||
|
|
0.12.168.960 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.12.168.961 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.12.168.968 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.12.168.968 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.12.168.973 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.12.168.974 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.12.168.985 I slot update_slots: id 3 | task 125 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.12.170.539 W slot update_slots: id 3 | task 125 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.13.306.389 I slot print_timing: id 3 | task 125 | prompt eval time = 90.44 ms / 17 tokens ( 5.32 ms per token, 187.97 tokens per second)
|
||
|
|
0.13.306.393 I slot print_timing: id 3 | task 125 | eval time = 1046.96 ms / 93 tokens ( 11.26 ms per token, 88.83 tokens per second)
|
||
|
|
0.13.306.393 I slot print_timing: id 3 | task 125 | total time = 1137.40 ms / 110 tokens
|
||
|
|
0.13.306.394 I slot print_timing: id 3 | task 125 | graphs reused = 212
|
||
|
|
0.13.306.424 I slot release: id 3 | task 125 | stop processing: n_tokens = 217, truncated = 0
|
||
|
|
0.13.306.433 I srv update_slots: all slots are idle
|
||
|
|
0.13.314.841 I srv operator(): Chat format: peg-native
|
||
|
|
0.13.318.628 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.882 (> 0.100 thold), f_keep = 0.516
|
||
|
|
0.13.319.680 I slot launch_slot_: id 3 | task 221 | processing task, is_child = 0
|
||
|
|
0.13.319.682 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.13.319.683 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.13.319.690 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.13.319.691 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.13.319.696 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.13.319.696 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.13.319.705 I slot update_slots: id 3 | task 221 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.13.321.564 W slot update_slots: id 3 | task 221 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.14.411.921 I slot print_timing: id 3 | task 221 | prompt eval time = 95.89 ms / 19 tokens ( 5.05 ms per token, 198.15 tokens per second)
|
||
|
|
0.14.411.924 I slot print_timing: id 3 | task 221 | eval time = 996.32 ms / 88 tokens ( 11.32 ms per token, 88.32 tokens per second)
|
||
|
|
0.14.411.925 I slot print_timing: id 3 | task 221 | total time = 1092.21 ms / 107 tokens
|
||
|
|
0.14.411.925 I slot print_timing: id 3 | task 221 | graphs reused = 298
|
||
|
|
0.14.411.952 I slot release: id 3 | task 221 | stop processing: n_tokens = 214, truncated = 0
|
||
|
|
0.14.411.961 I srv update_slots: all slots are idle
|
||
|
|
0.14.420.057 I srv operator(): Chat format: peg-native
|
||
|
|
0.14.423.736 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.904 (> 0.100 thold), f_keep = 0.528
|
||
|
|
0.14.424.804 I slot launch_slot_: id 3 | task 312 | processing task, is_child = 0
|
||
|
|
0.14.424.809 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.14.424.809 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.14.424.816 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.14.424.817 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.14.424.823 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.14.424.823 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.14.424.833 I slot update_slots: id 3 | task 312 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.14.426.708 W slot update_slots: id 3 | task 312 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.16.484.639 I slot print_timing: id 3 | task 312 | prompt eval time = 91.27 ms / 17 tokens ( 5.37 ms per token, 186.26 tokens per second)
|
||
|
|
0.16.484.642 I slot print_timing: id 3 | task 312 | eval time = 1968.53 ms / 173 tokens ( 11.38 ms per token, 87.88 tokens per second)
|
||
|
|
0.16.484.643 I slot print_timing: id 3 | task 312 | total time = 2059.80 ms / 190 tokens
|
||
|
|
0.16.484.644 I slot print_timing: id 3 | task 312 | graphs reused = 468
|
||
|
|
0.16.484.674 I slot release: id 3 | task 312 | stop processing: n_tokens = 297, truncated = 0
|
||
|
|
0.16.484.682 I srv update_slots: all slots are idle
|
||
|
|
0.16.492.895 I srv operator(): Chat format: peg-native
|
||
|
|
0.16.496.610 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.883 (> 0.100 thold), f_keep = 0.380
|
||
|
|
0.16.496.612 I srv get_availabl: updating prompt cache
|
||
|
|
0.16.496.640 W srv prompt_save: - saving prompt with length 297, total state size = 9.966 MiB (draft: 0.000 MiB)
|
||
|
|
0.16.503.795 I srv load: - looking for better prompt, base f_keep = 0.380, sim = 0.883
|
||
|
|
0.16.503.799 I srv load: - found better prompt with f_keep = 0.453, sim = 0.898
|
||
|
|
0.16.507.720 I srv update: - cache state: 1 prompts, 12.523 MiB (limits: 8192.000 MiB, 8192 tokens, 194290 est)
|
||
|
|
0.16.507.723 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.16.507.724 I srv get_availabl: prompt cache update took 11.11 ms
|
||
|
|
0.16.508.631 I slot launch_slot_: id 3 | task 488 | processing task, is_child = 0
|
||
|
|
0.16.508.633 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.16.508.633 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.16.508.640 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.16.508.640 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.16.508.645 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.16.508.646 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.16.508.656 I slot update_slots: id 3 | task 488 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.16.510.419 W slot update_slots: id 3 | task 488 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.19.614.449 I slot print_timing: id 3 | task 488 | n_decoded = 265, tg = 88.02 t/s, tg_3s = 88.02 t/s
|
||
|
|
0.20.172.945 I slot print_timing: id 3 | task 488 | prompt eval time = 95.01 ms / 20 tokens ( 4.75 ms per token, 210.50 tokens per second)
|
||
|
|
0.20.172.949 I slot print_timing: id 3 | task 488 | eval time = 3569.27 ms / 314 tokens ( 11.37 ms per token, 87.97 tokens per second)
|
||
|
|
0.20.172.950 I slot print_timing: id 3 | task 488 | total time = 3664.29 ms / 334 tokens
|
||
|
|
0.20.172.951 I slot print_timing: id 3 | task 488 | graphs reused = 779
|
||
|
|
0.20.172.982 I slot release: id 3 | task 488 | stop processing: n_tokens = 441, truncated = 0
|
||
|
|
0.20.172.990 I srv update_slots: all slots are idle
|
||
|
|
0.20.181.260 I srv operator(): Chat format: peg-native
|
||
|
|
0.20.185.081 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.875 (> 0.100 thold), f_keep = 0.254
|
||
|
|
0.20.185.083 I srv get_availabl: updating prompt cache
|
||
|
|
0.20.185.120 W srv prompt_save: - saving prompt with length 441, total state size = 13.343 MiB (draft: 0.000 MiB)
|
||
|
|
0.20.196.152 I srv load: - looking for better prompt, base f_keep = 0.254, sim = 0.875
|
||
|
|
0.20.196.159 I srv update: - cache state: 2 prompts, 28.422 MiB (limits: 8192.000 MiB, 8192 tokens, 212712 est)
|
||
|
|
0.20.196.160 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.20.196.161 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.20.196.163 I srv get_availabl: prompt cache update took 11.08 ms
|
||
|
|
0.20.197.033 I slot launch_slot_: id 3 | task 805 | processing task, is_child = 0
|
||
|
|
0.20.197.035 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.20.197.035 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.20.197.044 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.20.197.044 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.20.197.049 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.20.197.049 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.20.197.060 I slot update_slots: id 3 | task 805 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.20.198.582 W slot update_slots: id 3 | task 805 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.21.914.333 I slot print_timing: id 3 | task 805 | prompt eval time = 96.37 ms / 20 tokens ( 4.82 ms per token, 207.54 tokens per second)
|
||
|
|
0.21.914.336 I slot print_timing: id 3 | task 805 | eval time = 1620.89 ms / 143 tokens ( 11.33 ms per token, 88.22 tokens per second)
|
||
|
|
0.21.914.337 I slot print_timing: id 3 | task 805 | total time = 1717.26 ms / 163 tokens
|
||
|
|
0.21.914.338 I slot print_timing: id 3 | task 805 | graphs reused = 919
|
||
|
|
0.21.914.366 I slot release: id 3 | task 805 | stop processing: n_tokens = 270, truncated = 0
|
||
|
|
0.21.914.373 I srv update_slots: all slots are idle
|
||
|
|
0.21.922.318 I srv operator(): Chat format: peg-native
|
||
|
|
0.21.925.969 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.848 (> 0.100 thold), f_keep = 0.415
|
||
|
|
0.21.925.972 I srv get_availabl: updating prompt cache
|
||
|
|
0.21.926.008 W srv prompt_save: - saving prompt with length 270, total state size = 9.333 MiB (draft: 0.000 MiB)
|
||
|
|
0.21.934.938 I srv load: - looking for better prompt, base f_keep = 0.415, sim = 0.848
|
||
|
|
0.21.934.946 I srv update: - cache state: 3 prompts, 40.311 MiB (limits: 8192.000 MiB, 8192 tokens, 204843 est)
|
||
|
|
0.21.934.947 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.21.934.948 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.21.934.948 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.21.934.949 I srv get_availabl: prompt cache update took 8.98 ms
|
||
|
|
0.21.936.058 I slot launch_slot_: id 3 | task 951 | processing task, is_child = 0
|
||
|
|
0.21.936.060 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.21.936.061 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.21.936.070 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.21.936.070 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.21.936.076 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.21.936.076 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.21.936.090 I slot update_slots: id 3 | task 951 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.21.937.729 W slot update_slots: id 3 | task 951 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.23.356.459 I slot print_timing: id 3 | task 951 | prompt eval time = 115.22 ms / 24 tokens ( 4.80 ms per token, 208.29 tokens per second)
|
||
|
|
0.23.356.462 I slot print_timing: id 3 | task 951 | eval time = 1305.14 ms / 115 tokens ( 11.35 ms per token, 88.11 tokens per second)
|
||
|
|
0.23.356.463 I slot print_timing: id 3 | task 951 | total time = 1420.37 ms / 139 tokens
|
||
|
|
0.23.356.464 I slot print_timing: id 3 | task 951 | graphs reused = 1032
|
||
|
|
0.23.356.496 I slot release: id 3 | task 951 | stop processing: n_tokens = 246, truncated = 0
|
||
|
|
0.23.356.505 I srv update_slots: all slots are idle
|
||
|
|
0.23.364.373 I srv operator(): Chat format: peg-native
|
||
|
|
0.23.368.107 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.926 (> 0.100 thold), f_keep = 0.459
|
||
|
|
0.23.368.109 I srv get_availabl: updating prompt cache
|
||
|
|
0.23.368.133 W srv prompt_save: - saving prompt with length 246, total state size = 8.770 MiB (draft: 0.000 MiB)
|
||
|
|
0.23.376.145 I srv load: - looking for better prompt, base f_keep = 0.459, sim = 0.926
|
||
|
|
0.23.376.155 I srv update: - cache state: 4 prompts, 51.638 MiB (limits: 8192.000 MiB, 8192 tokens, 198937 est)
|
||
|
|
0.23.376.155 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.23.376.156 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.23.376.157 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.23.376.157 I srv update: - prompt 0x5bea7141f020: 246 tokens, checkpoints: 1, 11.327 MiB
|
||
|
|
0.23.376.158 I srv get_availabl: prompt cache update took 8.05 ms
|
||
|
|
0.23.377.069 I slot launch_slot_: id 3 | task 1069 | processing task, is_child = 0
|
||
|
|
0.23.377.071 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.23.377.072 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.23.377.080 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.23.377.080 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.23.377.086 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.23.377.086 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.23.377.096 I slot update_slots: id 3 | task 1069 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.23.378.553 W slot update_slots: id 3 | task 1069 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.24.987.350 I slot print_timing: id 3 | task 1069 | prompt eval time = 88.31 ms / 14 tokens ( 6.31 ms per token, 158.53 tokens per second)
|
||
|
|
0.24.987.353 I slot print_timing: id 3 | task 1069 | eval time = 1521.94 ms / 133 tokens ( 11.44 ms per token, 87.39 tokens per second)
|
||
|
|
0.24.987.353 I slot print_timing: id 3 | task 1069 | total time = 1610.25 ms / 147 tokens
|
||
|
|
0.24.987.354 I slot print_timing: id 3 | task 1069 | graphs reused = 1163
|
||
|
|
0.24.987.387 I slot release: id 3 | task 1069 | stop processing: n_tokens = 254, truncated = 0
|
||
|
|
0.24.987.395 I srv update_slots: all slots are idle
|
||
|
|
0.24.995.372 I srv operator(): Chat format: peg-native
|
||
|
|
0.24.999.068 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.848 (> 0.100 thold), f_keep = 0.441
|
||
|
|
0.24.999.071 I srv get_availabl: updating prompt cache
|
||
|
|
0.24.999.100 W srv prompt_save: - saving prompt with length 254, total state size = 8.958 MiB (draft: 0.000 MiB)
|
||
|
|
0.25.007.528 I srv load: - looking for better prompt, base f_keep = 0.441, sim = 0.848
|
||
|
|
0.25.007.535 I srv update: - cache state: 5 prompts, 63.152 MiB (limits: 8192.000 MiB, 8192 tokens, 195614 est)
|
||
|
|
0.25.007.539 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.25.007.539 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.25.007.540 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.25.007.540 I srv update: - prompt 0x5bea7141f020: 246 tokens, checkpoints: 1, 11.327 MiB
|
||
|
|
0.25.007.541 I srv update: - prompt 0x5bea712fb8d0: 254 tokens, checkpoints: 1, 11.514 MiB
|
||
|
|
0.25.007.544 I srv get_availabl: prompt cache update took 8.47 ms
|
||
|
|
0.25.008.462 I slot launch_slot_: id 3 | task 1205 | processing task, is_child = 0
|
||
|
|
0.25.008.464 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.25.008.465 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.25.008.472 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.25.008.472 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.25.008.477 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.25.008.477 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.25.008.488 I slot update_slots: id 3 | task 1205 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.25.010.152 W slot update_slots: id 3 | task 1205 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.26.310.418 I slot print_timing: id 3 | task 1205 | prompt eval time = 111.86 ms / 24 tokens ( 4.66 ms per token, 214.55 tokens per second)
|
||
|
|
0.26.310.421 I slot print_timing: id 3 | task 1205 | eval time = 1190.06 ms / 104 tokens ( 11.44 ms per token, 87.39 tokens per second)
|
||
|
|
0.26.310.422 I slot print_timing: id 3 | task 1205 | total time = 1301.93 ms / 128 tokens
|
||
|
|
0.26.310.423 I slot print_timing: id 3 | task 1205 | graphs reused = 1265
|
||
|
|
0.26.310.470 I slot release: id 3 | task 1205 | stop processing: n_tokens = 235, truncated = 0
|
||
|
|
0.26.310.478 I srv update_slots: all slots are idle
|
||
|
|
0.26.318.494 I srv operator(): Chat format: peg-native
|
||
|
|
0.26.322.139 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.882 (> 0.100 thold), f_keep = 0.477
|
||
|
|
0.26.322.141 I srv get_availabl: updating prompt cache
|
||
|
|
0.26.322.167 W srv prompt_save: - saving prompt with length 235, total state size = 8.513 MiB (draft: 0.000 MiB)
|
||
|
|
0.26.331.224 I srv load: - looking for better prompt, base f_keep = 0.477, sim = 0.882
|
||
|
|
0.26.331.230 I srv update: - cache state: 6 prompts, 74.221 MiB (limits: 8192.000 MiB, 8192 tokens, 192379 est)
|
||
|
|
0.26.331.231 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.26.331.234 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.26.331.235 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.26.331.235 I srv update: - prompt 0x5bea7141f020: 246 tokens, checkpoints: 1, 11.327 MiB
|
||
|
|
0.26.331.236 I srv update: - prompt 0x5bea712fb8d0: 254 tokens, checkpoints: 1, 11.514 MiB
|
||
|
|
0.26.331.236 I srv update: - prompt 0x5bea7855c070: 235 tokens, checkpoints: 1, 11.069 MiB
|
||
|
|
0.26.331.238 I srv get_availabl: prompt cache update took 9.10 ms
|
||
|
|
0.26.332.158 I slot launch_slot_: id 3 | task 1312 | processing task, is_child = 0
|
||
|
|
0.26.332.160 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.26.332.160 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.26.332.167 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.26.332.167 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.26.332.173 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.26.332.173 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.26.332.185 I slot update_slots: id 3 | task 1312 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.26.333.754 W slot update_slots: id 3 | task 1312 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.27.721.677 I slot print_timing: id 3 | task 1312 | prompt eval time = 94.87 ms / 19 tokens ( 4.99 ms per token, 200.27 tokens per second)
|
||
|
|
0.27.721.680 I slot print_timing: id 3 | task 1312 | eval time = 1294.62 ms / 113 tokens ( 11.46 ms per token, 87.28 tokens per second)
|
||
|
|
0.27.721.681 I slot print_timing: id 3 | task 1312 | total time = 1389.49 ms / 132 tokens
|
||
|
|
0.27.721.682 I slot print_timing: id 3 | task 1312 | graphs reused = 1376
|
||
|
|
0.27.721.711 I slot release: id 3 | task 1312 | stop processing: n_tokens = 239, truncated = 0
|
||
|
|
0.27.721.718 I srv update_slots: all slots are idle
|
||
|
|
0.27.729.981 I srv operator(): Chat format: peg-native
|
||
|
|
0.27.733.709 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.848 (> 0.100 thold), f_keep = 0.469
|
||
|
|
0.27.733.710 I srv get_availabl: updating prompt cache
|
||
|
|
0.27.733.734 W srv prompt_save: - saving prompt with length 239, total state size = 8.606 MiB (draft: 0.000 MiB)
|
||
|
|
0.27.742.636 I srv load: - looking for better prompt, base f_keep = 0.469, sim = 0.848
|
||
|
|
0.27.742.645 I srv update: - cache state: 7 prompts, 85.384 MiB (limits: 8192.000 MiB, 8192 tokens, 190159 est)
|
||
|
|
0.27.742.648 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.27.742.649 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.27.742.649 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.27.742.650 I srv update: - prompt 0x5bea7141f020: 246 tokens, checkpoints: 1, 11.327 MiB
|
||
|
|
0.27.742.650 I srv update: - prompt 0x5bea712fb8d0: 254 tokens, checkpoints: 1, 11.514 MiB
|
||
|
|
0.27.742.653 I srv update: - prompt 0x5bea7855c070: 235 tokens, checkpoints: 1, 11.069 MiB
|
||
|
|
0.27.742.653 I srv update: - prompt 0x5bea6846efc0: 239 tokens, checkpoints: 1, 11.163 MiB
|
||
|
|
0.27.742.654 I srv get_availabl: prompt cache update took 8.94 ms
|
||
|
|
0.27.743.605 I slot launch_slot_: id 3 | task 1428 | processing task, is_child = 0
|
||
|
|
0.27.743.607 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.27.743.608 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.27.743.615 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.27.743.616 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.27.743.621 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.27.743.621 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.27.743.632 I slot update_slots: id 3 | task 1428 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.27.745.268 W slot update_slots: id 3 | task 1428 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.29.418.431 I slot print_timing: id 3 | task 1428 | prompt eval time = 114.50 ms / 24 tokens ( 4.77 ms per token, 209.61 tokens per second)
|
||
|
|
0.29.418.435 I slot print_timing: id 3 | task 1428 | eval time = 1560.30 ms / 136 tokens ( 11.47 ms per token, 87.16 tokens per second)
|
||
|
|
0.29.418.435 I slot print_timing: id 3 | task 1428 | total time = 1674.79 ms / 160 tokens
|
||
|
|
0.29.418.436 I slot print_timing: id 3 | task 1428 | graphs reused = 1509
|
||
|
|
0.29.418.468 I slot release: id 3 | task 1428 | stop processing: n_tokens = 267, truncated = 0
|
||
|
|
0.29.418.476 I srv update_slots: all slots are idle
|
||
|
|
0.29.426.598 I srv operator(): Chat format: peg-native
|
||
|
|
0.29.430.297 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.875 (> 0.100 thold), f_keep = 0.419
|
||
|
|
0.29.430.299 I srv get_availabl: updating prompt cache
|
||
|
|
0.29.430.333 W srv prompt_save: - saving prompt with length 267, total state size = 9.263 MiB (draft: 0.000 MiB)
|
||
|
|
0.29.439.742 I srv load: - looking for better prompt, base f_keep = 0.419, sim = 0.875
|
||
|
|
0.29.439.748 I srv load: - found better prompt with f_keep = 0.459, sim = 0.883
|
||
|
|
0.29.443.655 I srv update: - cache state: 7 prompts, 85.876 MiB (limits: 8192.000 MiB, 8192 tokens, 191072 est)
|
||
|
|
0.29.443.657 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.29.443.658 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.29.443.659 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.29.443.659 I srv update: - prompt 0x5bea712fb8d0: 254 tokens, checkpoints: 1, 11.514 MiB
|
||
|
|
0.29.443.660 I srv update: - prompt 0x5bea7855c070: 235 tokens, checkpoints: 1, 11.069 MiB
|
||
|
|
0.29.443.660 I srv update: - prompt 0x5bea6846efc0: 239 tokens, checkpoints: 1, 11.163 MiB
|
||
|
|
0.29.443.661 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.29.443.662 I srv get_availabl: prompt cache update took 13.36 ms
|
||
|
|
0.29.444.597 I slot launch_slot_: id 3 | task 1567 | processing task, is_child = 0
|
||
|
|
0.29.444.599 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.29.444.600 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.29.444.607 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.29.444.607 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.29.444.613 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.29.444.613 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.29.444.628 I slot update_slots: id 3 | task 1567 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.29.446.259 W slot update_slots: id 3 | task 1567 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.30.852.627 I slot print_timing: id 3 | task 1567 | prompt eval time = 91.78 ms / 20 tokens ( 4.59 ms per token, 217.91 tokens per second)
|
||
|
|
0.30.852.630 I slot print_timing: id 3 | task 1567 | eval time = 1316.21 ms / 115 tokens ( 11.45 ms per token, 87.37 tokens per second)
|
||
|
|
0.30.852.631 I slot print_timing: id 3 | task 1567 | total time = 1407.99 ms / 135 tokens
|
||
|
|
0.30.852.631 I slot print_timing: id 3 | task 1567 | graphs reused = 1622
|
||
|
|
0.30.852.663 I slot release: id 3 | task 1567 | stop processing: n_tokens = 242, truncated = 0
|
||
|
|
0.30.852.674 I srv update_slots: all slots are idle
|
||
|
|
0.30.860.569 I srv operator(): Chat format: peg-native
|
||
|
|
0.30.864.568 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.868 (> 0.100 thold), f_keep = 0.463
|
||
|
|
0.30.864.570 I srv get_availabl: updating prompt cache
|
||
|
|
0.30.864.610 W srv prompt_save: - saving prompt with length 242, total state size = 8.677 MiB (draft: 0.000 MiB)
|
||
|
|
0.30.869.512 I srv load: - looking for better prompt, base f_keep = 0.463, sim = 0.868
|
||
|
|
0.30.869.521 I srv update: - cache state: 8 prompts, 97.109 MiB (limits: 8192.000 MiB, 8192 tokens, 189385 est)
|
||
|
|
0.30.869.522 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.30.869.522 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.30.869.523 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.30.869.523 I srv update: - prompt 0x5bea712fb8d0: 254 tokens, checkpoints: 1, 11.514 MiB
|
||
|
|
0.30.869.524 I srv update: - prompt 0x5bea7855c070: 235 tokens, checkpoints: 1, 11.069 MiB
|
||
|
|
0.30.869.524 I srv update: - prompt 0x5bea6846efc0: 239 tokens, checkpoints: 1, 11.163 MiB
|
||
|
|
0.30.869.525 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.30.869.525 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.30.869.527 I srv get_availabl: prompt cache update took 4.96 ms
|
||
|
|
0.30.870.449 I slot launch_slot_: id 3 | task 1685 | processing task, is_child = 0
|
||
|
|
0.30.870.451 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.30.870.452 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.30.870.460 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.30.870.460 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.30.870.466 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.30.870.466 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.30.870.481 I slot update_slots: id 3 | task 1685 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.30.872.079 W slot update_slots: id 3 | task 1685 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.31.656.383 I slot print_timing: id 3 | task 1685 | prompt eval time = 91.77 ms / 21 tokens ( 4.37 ms per token, 228.83 tokens per second)
|
||
|
|
0.31.656.387 I slot print_timing: id 3 | task 1685 | eval time = 694.13 ms / 61 tokens ( 11.38 ms per token, 87.88 tokens per second)
|
||
|
|
0.31.656.388 I slot print_timing: id 3 | task 1685 | total time = 785.90 ms / 82 tokens
|
||
|
|
0.31.656.389 I slot print_timing: id 3 | task 1685 | graphs reused = 1681
|
||
|
|
0.31.656.421 I slot release: id 3 | task 1685 | stop processing: n_tokens = 189, truncated = 0
|
||
|
|
0.31.656.430 I srv update_slots: all slots are idle
|
||
|
|
0.31.664.444 I srv operator(): Chat format: peg-native
|
||
|
|
0.31.668.626 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.875 (> 0.100 thold), f_keep = 0.593
|
||
|
|
0.31.669.460 I slot launch_slot_: id 3 | task 1749 | processing task, is_child = 0
|
||
|
|
0.31.669.462 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.31.669.463 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.31.669.469 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.31.669.469 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.31.669.475 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.31.669.476 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.31.669.486 I slot update_slots: id 3 | task 1749 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.31.671.430 W slot update_slots: id 3 | task 1749 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.33.008.426 I slot print_timing: id 3 | task 1749 | prompt eval time = 91.35 ms / 20 tokens ( 4.57 ms per token, 218.93 tokens per second)
|
||
|
|
0.33.008.429 I slot print_timing: id 3 | task 1749 | eval time = 1247.58 ms / 109 tokens ( 11.45 ms per token, 87.37 tokens per second)
|
||
|
|
0.33.008.430 I slot print_timing: id 3 | task 1749 | total time = 1338.94 ms / 129 tokens
|
||
|
|
0.33.008.431 I slot print_timing: id 3 | task 1749 | graphs reused = 1788
|
||
|
|
0.33.008.476 I slot release: id 3 | task 1749 | stop processing: n_tokens = 236, truncated = 0
|
||
|
|
0.33.008.484 I srv update_slots: all slots are idle
|
||
|
|
0.33.016.572 I srv operator(): Chat format: peg-native
|
||
|
|
0.33.020.269 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.862 (> 0.100 thold), f_keep = 0.475
|
||
|
|
0.33.020.271 I srv get_availabl: updating prompt cache
|
||
|
|
0.33.020.297 W srv prompt_save: - saving prompt with length 236, total state size = 8.536 MiB (draft: 0.000 MiB)
|
||
|
|
0.33.029.705 I srv load: - looking for better prompt, base f_keep = 0.475, sim = 0.862
|
||
|
|
0.33.029.715 I srv update: - cache state: 9 prompts, 108.201 MiB (limits: 8192.000 MiB, 8192 tokens, 187838 est)
|
||
|
|
0.33.029.716 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.33.029.717 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.33.029.717 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.33.029.718 I srv update: - prompt 0x5bea712fb8d0: 254 tokens, checkpoints: 1, 11.514 MiB
|
||
|
|
0.33.029.718 I srv update: - prompt 0x5bea7855c070: 235 tokens, checkpoints: 1, 11.069 MiB
|
||
|
|
0.33.029.719 I srv update: - prompt 0x5bea6846efc0: 239 tokens, checkpoints: 1, 11.163 MiB
|
||
|
|
0.33.029.719 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.33.029.720 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.33.029.721 I srv update: - prompt 0x5bea714338e0: 236 tokens, checkpoints: 1, 11.092 MiB
|
||
|
|
0.33.029.722 I srv get_availabl: prompt cache update took 9.45 ms
|
||
|
|
0.33.030.654 I slot launch_slot_: id 3 | task 1861 | processing task, is_child = 0
|
||
|
|
0.33.030.656 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.33.030.656 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.33.030.664 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.33.030.664 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.33.030.669 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.33.030.669 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.33.030.682 I slot update_slots: id 3 | task 1861 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.33.032.366 W slot update_slots: id 3 | task 1861 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.34.605.781 I slot print_timing: id 3 | task 1861 | prompt eval time = 109.76 ms / 22 tokens ( 4.99 ms per token, 200.44 tokens per second)
|
||
|
|
0.34.605.784 I slot print_timing: id 3 | task 1861 | eval time = 1465.33 ms / 128 tokens ( 11.45 ms per token, 87.35 tokens per second)
|
||
|
|
0.34.605.785 I slot print_timing: id 3 | task 1861 | total time = 1575.09 ms / 150 tokens
|
||
|
|
0.34.605.786 I slot print_timing: id 3 | task 1861 | graphs reused = 1913
|
||
|
|
0.34.605.812 I slot release: id 3 | task 1861 | stop processing: n_tokens = 257, truncated = 0
|
||
|
|
0.34.605.823 I srv update_slots: all slots are idle
|
||
|
|
0.34.613.976 I srv operator(): Chat format: peg-native
|
||
|
|
0.34.617.706 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.889 (> 0.100 thold), f_keep = 0.436
|
||
|
|
0.34.617.707 I srv get_availabl: updating prompt cache
|
||
|
|
0.34.617.732 W srv prompt_save: - saving prompt with length 257, total state size = 9.028 MiB (draft: 0.000 MiB)
|
||
|
|
0.34.626.878 I srv load: - looking for better prompt, base f_keep = 0.436, sim = 0.889
|
||
|
|
0.34.626.883 I srv load: - found better prompt with f_keep = 0.481, sim = 0.897
|
||
|
|
0.34.630.635 I srv update: - cache state: 9 prompts, 108.717 MiB (limits: 8192.000 MiB, 8192 tokens, 188604 est)
|
||
|
|
0.34.630.637 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.34.630.638 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.34.630.638 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.34.630.639 I srv update: - prompt 0x5bea712fb8d0: 254 tokens, checkpoints: 1, 11.514 MiB
|
||
|
|
0.34.630.640 I srv update: - prompt 0x5bea6846efc0: 239 tokens, checkpoints: 1, 11.163 MiB
|
||
|
|
0.34.630.640 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.34.630.640 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.34.630.641 I srv update: - prompt 0x5bea714338e0: 236 tokens, checkpoints: 1, 11.092 MiB
|
||
|
|
0.34.630.641 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
0.34.630.643 I srv get_availabl: prompt cache update took 12.93 ms
|
||
|
|
0.34.631.563 I slot launch_slot_: id 3 | task 1992 | processing task, is_child = 0
|
||
|
|
0.34.631.565 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.34.631.566 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.34.631.572 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.34.631.573 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.34.631.578 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.34.631.578 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.34.631.589 I slot update_slots: id 3 | task 1992 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.34.633.159 W slot update_slots: id 3 | task 1992 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.35.794.421 I slot print_timing: id 3 | task 1992 | prompt eval time = 89.65 ms / 18 tokens ( 4.98 ms per token, 200.78 tokens per second)
|
||
|
|
0.35.794.424 I slot print_timing: id 3 | task 1992 | eval time = 1073.17 ms / 94 tokens ( 11.42 ms per token, 87.59 tokens per second)
|
||
|
|
0.35.794.424 I slot print_timing: id 3 | task 1992 | total time = 1162.83 ms / 112 tokens
|
||
|
|
0.35.794.425 I slot print_timing: id 3 | task 1992 | graphs reused = 2005
|
||
|
|
0.35.794.454 I slot release: id 3 | task 1992 | stop processing: n_tokens = 219, truncated = 0
|
||
|
|
0.35.794.462 I srv update_slots: all slots are idle
|
||
|
|
0.35.802.418 I srv operator(): Chat format: peg-native
|
||
|
|
0.35.806.298 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.862 (> 0.100 thold), f_keep = 0.511
|
||
|
|
0.35.807.231 I slot launch_slot_: id 3 | task 2089 | processing task, is_child = 0
|
||
|
|
0.35.807.233 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.35.807.234 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.35.807.241 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.35.807.241 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.35.807.246 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.35.807.246 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.35.807.254 I slot update_slots: id 3 | task 2089 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.35.809.058 W slot update_slots: id 3 | task 2089 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.37.131.417 I slot print_timing: id 3 | task 2089 | prompt eval time = 112.80 ms / 22 tokens ( 5.13 ms per token, 195.03 tokens per second)
|
||
|
|
0.37.131.420 I slot print_timing: id 3 | task 2089 | eval time = 1211.35 ms / 106 tokens ( 11.43 ms per token, 87.51 tokens per second)
|
||
|
|
0.37.131.421 I slot print_timing: id 3 | task 2089 | total time = 1324.16 ms / 128 tokens
|
||
|
|
0.37.131.422 I slot print_timing: id 3 | task 2089 | graphs reused = 2109
|
||
|
|
0.37.131.452 I slot release: id 3 | task 2089 | stop processing: n_tokens = 235, truncated = 0
|
||
|
|
0.37.131.461 I srv update_slots: all slots are idle
|
||
|
|
0.37.139.411 I srv operator(): Chat format: peg-native
|
||
|
|
0.37.143.054 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.896 (> 0.100 thold), f_keep = 0.477
|
||
|
|
0.37.143.056 I srv get_availabl: updating prompt cache
|
||
|
|
0.37.143.085 W srv prompt_save: - saving prompt with length 235, total state size = 8.513 MiB (draft: 0.000 MiB)
|
||
|
|
0.37.148.127 I srv load: - looking for better prompt, base f_keep = 0.477, sim = 0.896
|
||
|
|
0.37.148.137 I srv update: - cache state: 10 prompts, 119.786 MiB (limits: 8192.000 MiB, 8192 tokens, 187247 est)
|
||
|
|
0.37.148.138 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.37.148.139 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.37.148.140 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.37.148.141 I srv update: - prompt 0x5bea712fb8d0: 254 tokens, checkpoints: 1, 11.514 MiB
|
||
|
|
0.37.148.142 I srv update: - prompt 0x5bea6846efc0: 239 tokens, checkpoints: 1, 11.163 MiB
|
||
|
|
0.37.148.143 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.37.148.144 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.37.148.144 I srv update: - prompt 0x5bea714338e0: 236 tokens, checkpoints: 1, 11.092 MiB
|
||
|
|
0.37.148.145 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
0.37.148.146 I srv update: - prompt 0x5bea7144c930: 235 tokens, checkpoints: 1, 11.069 MiB
|
||
|
|
0.37.148.146 I srv get_availabl: prompt cache update took 5.09 ms
|
||
|
|
0.37.149.076 I slot launch_slot_: id 3 | task 2198 | processing task, is_child = 0
|
||
|
|
0.37.149.078 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.149.079 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.149.088 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.149.088 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.149.094 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.149.095 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.149.108 I slot update_slots: id 3 | task 2198 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.37.150.610 W slot update_slots: id 3 | task 2198 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.38.500.716 I slot print_timing: id 3 | task 2198 | prompt eval time = 91.03 ms / 17 tokens ( 5.35 ms per token, 186.76 tokens per second)
|
||
|
|
0.38.500.719 I slot print_timing: id 3 | task 2198 | eval time = 1260.58 ms / 110 tokens ( 11.46 ms per token, 87.26 tokens per second)
|
||
|
|
0.38.500.720 I slot print_timing: id 3 | task 2198 | total time = 1351.60 ms / 127 tokens
|
||
|
|
0.38.500.721 I slot print_timing: id 3 | task 2198 | graphs reused = 2217
|
||
|
|
0.38.500.753 I slot release: id 3 | task 2198 | stop processing: n_tokens = 234, truncated = 0
|
||
|
|
0.38.500.760 I srv update_slots: all slots are idle
|
||
|
|
0.38.508.615 I srv operator(): Chat format: peg-native
|
||
|
|
0.38.512.294 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.903 (> 0.100 thold), f_keep = 0.479
|
||
|
|
0.38.512.296 I srv get_availabl: updating prompt cache
|
||
|
|
0.38.512.318 W srv prompt_save: - saving prompt with length 234, total state size = 8.489 MiB (draft: 0.000 MiB)
|
||
|
|
0.38.521.405 I srv load: - looking for better prompt, base f_keep = 0.479, sim = 0.903
|
||
|
|
0.38.521.412 I srv load: - found better prompt with f_keep = 0.481, sim = 0.911
|
||
|
|
0.38.525.231 I srv update: - cache state: 10 prompts, 119.763 MiB (limits: 8192.000 MiB, 8192 tokens, 187216 est)
|
||
|
|
0.38.525.233 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.38.525.234 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.38.525.234 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.38.525.235 I srv update: - prompt 0x5bea712fb8d0: 254 tokens, checkpoints: 1, 11.514 MiB
|
||
|
|
0.38.525.235 I srv update: - prompt 0x5bea6846efc0: 239 tokens, checkpoints: 1, 11.163 MiB
|
||
|
|
0.38.525.236 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.38.525.236 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.38.525.237 I srv update: - prompt 0x5bea714338e0: 236 tokens, checkpoints: 1, 11.092 MiB
|
||
|
|
0.38.525.237 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
0.38.525.238 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
0.38.525.239 I srv get_availabl: prompt cache update took 12.94 ms
|
||
|
|
0.38.526.163 I slot launch_slot_: id 3 | task 2311 | processing task, is_child = 0
|
||
|
|
0.38.526.165 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.38.526.166 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.38.526.173 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.38.526.173 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.38.526.178 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.38.526.178 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.38.526.190 I slot update_slots: id 3 | task 2311 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.38.527.657 W slot update_slots: id 3 | task 2311 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.39.494.739 I slot print_timing: id 3 | task 2311 | prompt eval time = 90.35 ms / 16 tokens ( 5.65 ms per token, 177.08 tokens per second)
|
||
|
|
0.39.494.743 I slot print_timing: id 3 | task 2311 | eval time = 878.19 ms / 77 tokens ( 11.41 ms per token, 87.68 tokens per second)
|
||
|
|
0.39.494.743 I slot print_timing: id 3 | task 2311 | total time = 968.54 ms / 93 tokens
|
||
|
|
0.39.494.744 I slot print_timing: id 3 | task 2311 | graphs reused = 2292
|
||
|
|
0.39.494.775 I slot release: id 3 | task 2311 | stop processing: n_tokens = 200, truncated = 0
|
||
|
|
0.39.494.784 I srv update_slots: all slots are idle
|
||
|
|
0.39.502.686 I srv operator(): Chat format: peg-native
|
||
|
|
0.39.506.421 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.919 (> 0.100 thold), f_keep = 0.565
|
||
|
|
0.39.507.458 I slot launch_slot_: id 3 | task 2391 | processing task, is_child = 0
|
||
|
|
0.39.507.459 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.39.507.460 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.39.507.467 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.39.507.468 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.39.507.474 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.39.507.474 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.39.507.483 I slot update_slots: id 3 | task 2391 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.39.509.422 W slot update_slots: id 3 | task 2391 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.40.353.487 I slot print_timing: id 3 | task 2391 | prompt eval time = 88.69 ms / 15 tokens ( 5.91 ms per token, 169.13 tokens per second)
|
||
|
|
0.40.353.490 I slot print_timing: id 3 | task 2391 | eval time = 757.31 ms / 66 tokens ( 11.47 ms per token, 87.15 tokens per second)
|
||
|
|
0.40.353.491 I slot print_timing: id 3 | task 2391 | total time = 846.00 ms / 81 tokens
|
||
|
|
0.40.353.492 I slot print_timing: id 3 | task 2391 | graphs reused = 2356
|
||
|
|
0.40.353.537 I slot release: id 3 | task 2391 | stop processing: n_tokens = 188, truncated = 0
|
||
|
|
0.40.353.547 I srv update_slots: all slots are idle
|
||
|
|
0.40.361.013 I srv operator(): Chat format: peg-native
|
||
|
|
0.40.364.743 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.918 (> 0.100 thold), f_keep = 0.596
|
||
|
|
0.40.365.735 I slot launch_slot_: id 3 | task 2460 | processing task, is_child = 0
|
||
|
|
0.40.365.737 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.40.365.737 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.40.365.744 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.40.365.745 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.40.365.750 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.40.365.750 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.40.365.759 I slot update_slots: id 3 | task 2460 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.40.367.678 W slot update_slots: id 3 | task 2460 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.41.097.495 I slot print_timing: id 3 | task 2460 | prompt eval time = 83.20 ms / 14 tokens ( 5.94 ms per token, 168.27 tokens per second)
|
||
|
|
0.41.097.498 I slot print_timing: id 3 | task 2460 | eval time = 648.53 ms / 57 tokens ( 11.38 ms per token, 87.89 tokens per second)
|
||
|
|
0.41.097.499 I slot print_timing: id 3 | task 2460 | total time = 731.73 ms / 71 tokens
|
||
|
|
0.41.097.499 I slot print_timing: id 3 | task 2460 | graphs reused = 2411
|
||
|
|
0.41.097.528 I slot release: id 3 | task 2460 | stop processing: n_tokens = 178, truncated = 0
|
||
|
|
0.41.097.537 I srv update_slots: all slots are idle
|
||
|
|
0.41.105.833 I srv operator(): Chat format: peg-native
|
||
|
|
0.41.109.883 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.882 (> 0.100 thold), f_keep = 0.629
|
||
|
|
0.41.110.866 I slot launch_slot_: id 3 | task 2520 | processing task, is_child = 0
|
||
|
|
0.41.110.867 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.41.110.868 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.41.110.875 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.41.110.875 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.41.110.880 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.41.110.880 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.41.110.888 I slot update_slots: id 3 | task 2520 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.41.112.788 W slot update_slots: id 3 | task 2520 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.43.150.489 I slot print_timing: id 3 | task 2520 | prompt eval time = 89.52 ms / 19 tokens ( 4.71 ms per token, 212.24 tokens per second)
|
||
|
|
0.43.150.493 I slot print_timing: id 3 | task 2520 | eval time = 1950.08 ms / 169 tokens ( 11.54 ms per token, 86.66 tokens per second)
|
||
|
|
0.43.150.494 I slot print_timing: id 3 | task 2520 | total time = 2039.60 ms / 188 tokens
|
||
|
|
0.43.150.494 I slot print_timing: id 3 | task 2520 | graphs reused = 2577
|
||
|
|
0.43.150.524 I slot release: id 3 | task 2520 | stop processing: n_tokens = 295, truncated = 0
|
||
|
|
0.43.150.536 I srv update_slots: all slots are idle
|
||
|
|
0.43.158.299 I srv operator(): Chat format: peg-native
|
||
|
|
0.43.162.055 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.889 (> 0.100 thold), f_keep = 0.380
|
||
|
|
0.43.162.057 I srv get_availabl: updating prompt cache
|
||
|
|
0.43.162.091 W srv prompt_save: - saving prompt with length 295, total state size = 9.919 MiB (draft: 0.000 MiB)
|
||
|
|
0.43.168.473 I srv load: - looking for better prompt, base f_keep = 0.380, sim = 0.889
|
||
|
|
0.43.168.479 I srv load: - found better prompt with f_keep = 0.481, sim = 0.913
|
||
|
|
0.43.172.388 I srv update: - cache state: 10 prompts, 121.076 MiB (limits: 8192.000 MiB, 8192 tokens, 188974 est)
|
||
|
|
0.43.172.391 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.43.172.391 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.43.172.392 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.43.172.393 I srv update: - prompt 0x5bea712fb8d0: 254 tokens, checkpoints: 1, 11.514 MiB
|
||
|
|
0.43.172.393 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.43.172.394 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.43.172.394 I srv update: - prompt 0x5bea714338e0: 236 tokens, checkpoints: 1, 11.092 MiB
|
||
|
|
0.43.172.395 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
0.43.172.395 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
0.43.172.395 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
0.43.172.397 I srv get_availabl: prompt cache update took 10.34 ms
|
||
|
|
0.43.173.320 I slot launch_slot_: id 3 | task 2692 | processing task, is_child = 0
|
||
|
|
0.43.173.329 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.43.173.329 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.43.173.337 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.43.173.337 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.43.173.343 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.43.173.343 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.43.173.359 I slot update_slots: id 3 | task 2692 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.43.174.838 W slot update_slots: id 3 | task 2692 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.44.823.844 I slot print_timing: id 3 | task 2692 | prompt eval time = 90.00 ms / 18 tokens ( 5.00 ms per token, 199.99 tokens per second)
|
||
|
|
0.44.823.848 I slot print_timing: id 3 | task 2692 | eval time = 1560.48 ms / 136 tokens ( 11.47 ms per token, 87.15 tokens per second)
|
||
|
|
0.44.823.849 I slot print_timing: id 3 | task 2692 | total time = 1650.48 ms / 154 tokens
|
||
|
|
0.44.823.850 I slot print_timing: id 3 | task 2692 | graphs reused = 2710
|
||
|
|
0.44.823.881 I slot release: id 3 | task 2692 | stop processing: n_tokens = 261, truncated = 0
|
||
|
|
0.44.823.892 I srv update_slots: all slots are idle
|
||
|
|
0.44.831.803 I srv operator(): Chat format: peg-native
|
||
|
|
0.44.835.518 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.848 (> 0.100 thold), f_keep = 0.429
|
||
|
|
0.44.835.520 I srv get_availabl: updating prompt cache
|
||
|
|
0.44.835.548 W srv prompt_save: - saving prompt with length 261, total state size = 9.122 MiB (draft: 0.000 MiB)
|
||
|
|
0.44.840.702 I srv load: - looking for better prompt, base f_keep = 0.429, sim = 0.848
|
||
|
|
0.44.840.708 I srv load: - found better prompt with f_keep = 0.479, sim = 0.856
|
||
|
|
0.44.844.526 I srv update: - cache state: 10 prompts, 121.662 MiB (limits: 8192.000 MiB, 8192 tokens, 189747 est)
|
||
|
|
0.44.844.529 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.44.844.529 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.44.844.530 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.44.844.531 I srv update: - prompt 0x5bea712fb8d0: 254 tokens, checkpoints: 1, 11.514 MiB
|
||
|
|
0.44.844.531 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.44.844.531 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.44.844.532 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
0.44.844.532 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
0.44.844.533 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
0.44.844.533 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
0.44.844.534 I srv get_availabl: prompt cache update took 9.01 ms
|
||
|
|
0.44.845.484 I slot launch_slot_: id 3 | task 2831 | processing task, is_child = 0
|
||
|
|
0.44.845.487 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.44.845.487 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.44.845.495 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.44.845.496 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.44.845.502 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.44.845.502 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.44.845.518 I slot update_slots: id 3 | task 2831 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.44.847.091 W slot update_slots: id 3 | task 2831 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.46.429.667 I slot print_timing: id 3 | task 2831 | prompt eval time = 109.72 ms / 24 tokens ( 4.57 ms per token, 218.73 tokens per second)
|
||
|
|
0.46.429.670 I slot print_timing: id 3 | task 2831 | eval time = 1474.42 ms / 127 tokens ( 11.61 ms per token, 86.14 tokens per second)
|
||
|
|
0.46.429.671 I slot print_timing: id 3 | task 2831 | total time = 1584.14 ms / 151 tokens
|
||
|
|
0.46.429.672 I slot print_timing: id 3 | task 2831 | graphs reused = 2834
|
||
|
|
0.46.429.700 I slot release: id 3 | task 2831 | stop processing: n_tokens = 258, truncated = 0
|
||
|
|
0.46.429.710 I srv update_slots: all slots are idle
|
||
|
|
0.46.437.600 I srv operator(): Chat format: peg-native
|
||
|
|
0.46.441.389 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.855 (> 0.100 thold), f_keep = 0.434
|
||
|
|
0.46.441.392 I srv get_availabl: updating prompt cache
|
||
|
|
0.46.441.433 W srv prompt_save: - saving prompt with length 258, total state size = 9.052 MiB (draft: 0.000 MiB)
|
||
|
|
0.46.450.213 I srv load: - looking for better prompt, base f_keep = 0.434, sim = 0.855
|
||
|
|
0.46.450.219 I srv load: - found better prompt with f_keep = 0.445, sim = 0.863
|
||
|
|
0.46.454.254 I srv update: - cache state: 10 prompts, 121.756 MiB (limits: 8192.000 MiB, 8192 tokens, 189870 est)
|
||
|
|
0.46.454.256 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.46.454.257 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.46.454.257 I srv update: - prompt 0x5bea715006a0: 270 tokens, checkpoints: 1, 11.889 MiB
|
||
|
|
0.46.454.258 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.46.454.258 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.46.454.259 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
0.46.454.259 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
0.46.454.260 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
0.46.454.260 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
0.46.454.260 I srv update: - prompt 0x5bea712ba060: 258 tokens, checkpoints: 1, 11.608 MiB
|
||
|
|
0.46.454.262 I srv get_availabl: prompt cache update took 12.87 ms
|
||
|
|
0.46.455.159 I slot launch_slot_: id 3 | task 2961 | processing task, is_child = 0
|
||
|
|
0.46.455.162 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.455.162 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.455.170 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.455.170 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.455.175 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.46.455.175 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.46.455.187 I slot update_slots: id 3 | task 2961 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.46.456.673 W slot update_slots: id 3 | task 2961 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.48.438.854 I slot print_timing: id 3 | task 2961 | prompt eval time = 109.41 ms / 23 tokens ( 4.76 ms per token, 210.21 tokens per second)
|
||
|
|
0.48.438.858 I slot print_timing: id 3 | task 2961 | eval time = 1874.25 ms / 161 tokens ( 11.64 ms per token, 85.90 tokens per second)
|
||
|
|
0.48.438.858 I slot print_timing: id 3 | task 2961 | total time = 1983.66 ms / 184 tokens
|
||
|
|
0.48.438.859 I slot print_timing: id 3 | task 2961 | graphs reused = 2992
|
||
|
|
0.48.438.893 I slot release: id 3 | task 2961 | stop processing: n_tokens = 291, truncated = 0
|
||
|
|
0.48.438.903 I srv update_slots: all slots are idle
|
||
|
|
0.48.446.717 I srv operator(): Chat format: peg-native
|
||
|
|
0.48.450.420 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.911 (> 0.100 thold), f_keep = 0.385
|
||
|
|
0.48.450.422 I srv get_availabl: updating prompt cache
|
||
|
|
0.48.450.447 W srv prompt_save: - saving prompt with length 291, total state size = 9.826 MiB (draft: 0.000 MiB)
|
||
|
|
0.48.456.046 I srv load: - looking for better prompt, base f_keep = 0.385, sim = 0.911
|
||
|
|
0.48.456.051 I srv load: - found better prompt with f_keep = 0.419, sim = 0.919
|
||
|
|
0.48.460.209 I srv update: - cache state: 10 prompts, 122.248 MiB (limits: 8192.000 MiB, 8192 tokens, 190512 est)
|
||
|
|
0.48.460.212 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.48.460.212 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.48.460.213 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.48.460.213 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.48.460.214 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
0.48.460.214 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
0.48.460.215 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
0.48.460.215 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
0.48.460.216 I srv update: - prompt 0x5bea712ba060: 258 tokens, checkpoints: 1, 11.608 MiB
|
||
|
|
0.48.460.216 I srv update: - prompt 0x5bea7867aee0: 291 tokens, checkpoints: 1, 12.382 MiB
|
||
|
|
0.48.460.218 I srv get_availabl: prompt cache update took 9.79 ms
|
||
|
|
0.48.461.123 I slot launch_slot_: id 3 | task 3125 | processing task, is_child = 0
|
||
|
|
0.48.461.124 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.48.461.125 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.48.461.133 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.48.461.133 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.48.461.138 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.48.461.138 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.48.461.149 I slot update_slots: id 3 | task 3125 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.48.462.645 W slot update_slots: id 3 | task 3125 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.49.201.787 I slot print_timing: id 3 | task 3125 | prompt eval time = 84.73 ms / 15 tokens ( 5.65 ms per token, 177.02 tokens per second)
|
||
|
|
0.49.201.790 I slot print_timing: id 3 | task 3125 | eval time = 655.90 ms / 57 tokens ( 11.51 ms per token, 86.90 tokens per second)
|
||
|
|
0.49.201.791 I slot print_timing: id 3 | task 3125 | total time = 740.63 ms / 72 tokens
|
||
|
|
0.49.201.791 I slot print_timing: id 3 | task 3125 | graphs reused = 3047
|
||
|
|
0.49.201.822 I slot release: id 3 | task 3125 | stop processing: n_tokens = 179, truncated = 0
|
||
|
|
0.49.201.834 I srv update_slots: all slots are idle
|
||
|
|
0.49.209.580 I srv operator(): Chat format: peg-native
|
||
|
|
0.49.213.305 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.889 (> 0.100 thold), f_keep = 0.626
|
||
|
|
0.49.214.502 I slot launch_slot_: id 3 | task 3185 | processing task, is_child = 0
|
||
|
|
0.49.214.507 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.49.214.508 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.49.214.518 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.49.214.518 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.49.214.524 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.49.214.525 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.49.214.538 I slot update_slots: id 3 | task 3185 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.49.216.494 W slot update_slots: id 3 | task 3185 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.51.491.501 I slot print_timing: id 3 | task 3185 | prompt eval time = 89.90 ms / 18 tokens ( 4.99 ms per token, 200.23 tokens per second)
|
||
|
|
0.51.491.504 I slot print_timing: id 3 | task 3185 | eval time = 2187.06 ms / 188 tokens ( 11.63 ms per token, 85.96 tokens per second)
|
||
|
|
0.51.491.505 I slot print_timing: id 3 | task 3185 | total time = 2276.96 ms / 206 tokens
|
||
|
|
0.51.491.505 I slot print_timing: id 3 | task 3185 | graphs reused = 3232
|
||
|
|
0.51.491.537 I slot release: id 3 | task 3185 | stop processing: n_tokens = 313, truncated = 0
|
||
|
|
0.51.491.549 I srv update_slots: all slots are idle
|
||
|
|
0.51.499.635 I srv operator(): Chat format: peg-native
|
||
|
|
0.51.503.399 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.875 (> 0.100 thold), f_keep = 0.358
|
||
|
|
0.51.503.401 I srv get_availabl: updating prompt cache
|
||
|
|
0.51.503.434 W srv prompt_save: - saving prompt with length 313, total state size = 10.342 MiB (draft: 0.000 MiB)
|
||
|
|
0.51.513.051 I srv load: - looking for better prompt, base f_keep = 0.358, sim = 0.875
|
||
|
|
0.51.513.061 I srv update: - cache state: 11 prompts, 135.146 MiB (limits: 8192.000 MiB, 8192 tokens, 191303 est)
|
||
|
|
0.51.513.062 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.51.513.063 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.51.513.063 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.51.513.064 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.51.513.064 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
0.51.513.065 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
0.51.513.065 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
0.51.513.066 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
0.51.513.066 I srv update: - prompt 0x5bea712ba060: 258 tokens, checkpoints: 1, 11.608 MiB
|
||
|
|
0.51.513.067 I srv update: - prompt 0x5bea7867aee0: 291 tokens, checkpoints: 1, 12.382 MiB
|
||
|
|
0.51.513.067 I srv update: - prompt 0x5bea7148a050: 313 tokens, checkpoints: 1, 12.898 MiB
|
||
|
|
0.51.513.068 I srv get_availabl: prompt cache update took 9.67 ms
|
||
|
|
0.51.513.974 I slot launch_slot_: id 3 | task 3376 | processing task, is_child = 0
|
||
|
|
0.51.513.976 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.51.513.977 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.51.513.984 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.51.513.984 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.51.513.989 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.51.513.989 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.51.514.001 I slot update_slots: id 3 | task 3376 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.51.515.634 W slot update_slots: id 3 | task 3376 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.52.336.590 I slot print_timing: id 3 | task 3376 | prompt eval time = 95.77 ms / 20 tokens ( 4.79 ms per token, 208.83 tokens per second)
|
||
|
|
0.52.336.594 I slot print_timing: id 3 | task 3376 | eval time = 726.81 ms / 63 tokens ( 11.54 ms per token, 86.68 tokens per second)
|
||
|
|
0.52.336.594 I slot print_timing: id 3 | task 3376 | total time = 822.58 ms / 83 tokens
|
||
|
|
0.52.336.595 I slot print_timing: id 3 | task 3376 | graphs reused = 3293
|
||
|
|
0.52.336.626 I slot release: id 3 | task 3376 | stop processing: n_tokens = 190, truncated = 0
|
||
|
|
0.52.336.635 I srv update_slots: all slots are idle
|
||
|
|
0.52.344.429 I srv operator(): Chat format: peg-native
|
||
|
|
0.52.348.147 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.806 (> 0.100 thold), f_keep = 0.589
|
||
|
|
0.52.349.002 I slot launch_slot_: id 3 | task 3442 | processing task, is_child = 0
|
||
|
|
0.52.349.005 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.52.349.005 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.52.349.013 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.52.349.013 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.52.349.018 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.52.349.019 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.52.349.027 I slot update_slots: id 3 | task 3442 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.52.350.924 W slot update_slots: id 3 | task 3442 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.54.950.357 I slot print_timing: id 3 | task 3442 | prompt eval time = 139.82 ms / 31 tokens ( 4.51 ms per token, 221.71 tokens per second)
|
||
|
|
0.54.950.360 I slot print_timing: id 3 | task 3442 | eval time = 2461.50 ms / 214 tokens ( 11.50 ms per token, 86.94 tokens per second)
|
||
|
|
0.54.950.361 I slot print_timing: id 3 | task 3442 | total time = 2601.32 ms / 245 tokens
|
||
|
|
0.54.950.361 I slot print_timing: id 3 | task 3442 | graphs reused = 3504
|
||
|
|
0.54.950.392 I slot release: id 3 | task 3442 | stop processing: n_tokens = 352, truncated = 0
|
||
|
|
0.54.950.400 I srv update_slots: all slots are idle
|
||
|
|
0.54.958.209 I srv operator(): Chat format: peg-native
|
||
|
|
0.54.962.037 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.869 (> 0.100 thold), f_keep = 0.321
|
||
|
|
0.54.962.038 I srv get_availabl: updating prompt cache
|
||
|
|
0.54.962.084 W srv prompt_save: - saving prompt with length 352, total state size = 11.256 MiB (draft: 0.000 MiB)
|
||
|
|
0.54.972.054 I srv load: - looking for better prompt, base f_keep = 0.321, sim = 0.869
|
||
|
|
0.54.972.065 I srv update: - cache state: 12 prompts, 148.958 MiB (limits: 8192.000 MiB, 8192 tokens, 192923 est)
|
||
|
|
0.54.972.066 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.54.972.066 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.54.972.067 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.54.972.067 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.54.972.068 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
0.54.972.068 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
0.54.972.069 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
0.54.972.069 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
0.54.972.070 I srv update: - prompt 0x5bea712ba060: 258 tokens, checkpoints: 1, 11.608 MiB
|
||
|
|
0.54.972.070 I srv update: - prompt 0x5bea7867aee0: 291 tokens, checkpoints: 1, 12.382 MiB
|
||
|
|
0.54.972.071 I srv update: - prompt 0x5bea7148a050: 313 tokens, checkpoints: 1, 12.898 MiB
|
||
|
|
0.54.972.072 I srv update: - prompt 0x5bea7855c740: 352 tokens, checkpoints: 1, 13.812 MiB
|
||
|
|
0.54.972.073 I srv get_availabl: prompt cache update took 10.03 ms
|
||
|
|
0.54.973.023 I slot launch_slot_: id 3 | task 3659 | processing task, is_child = 0
|
||
|
|
0.54.973.026 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.54.973.027 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.54.973.036 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.54.973.037 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.54.973.043 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.54.973.043 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.54.973.058 I slot update_slots: id 3 | task 3659 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.54.974.724 W slot update_slots: id 3 | task 3659 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.57.094.123 I slot print_timing: id 3 | task 3659 | prompt eval time = 115.11 ms / 22 tokens ( 5.23 ms per token, 191.12 tokens per second)
|
||
|
|
0.57.094.126 I slot print_timing: id 3 | task 3659 | eval time = 2005.95 ms / 181 tokens ( 11.08 ms per token, 90.23 tokens per second)
|
||
|
|
0.57.094.127 I slot print_timing: id 3 | task 3659 | total time = 2121.06 ms / 203 tokens
|
||
|
|
0.57.094.128 I slot print_timing: id 3 | task 3659 | graphs reused = 3682
|
||
|
|
0.57.094.159 I slot release: id 3 | task 3659 | stop processing: n_tokens = 310, truncated = 0
|
||
|
|
0.57.094.168 I srv update_slots: all slots are idle
|
||
|
|
0.57.102.125 I srv operator(): Chat format: peg-native
|
||
|
|
0.57.105.832 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.896 (> 0.100 thold), f_keep = 0.361
|
||
|
|
0.57.105.834 I srv get_availabl: updating prompt cache
|
||
|
|
0.57.105.873 W srv prompt_save: - saving prompt with length 310, total state size = 10.271 MiB (draft: 0.000 MiB)
|
||
|
|
0.57.114.945 I srv load: - looking for better prompt, base f_keep = 0.361, sim = 0.896
|
||
|
|
0.57.114.956 I srv update: - cache state: 13 prompts, 161.786 MiB (limits: 8192.000 MiB, 8192 tokens, 193323 est)
|
||
|
|
0.57.114.957 I srv update: - prompt 0x5bea712eb690: 297 tokens, checkpoints: 1, 12.523 MiB
|
||
|
|
0.57.114.957 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.57.114.958 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.57.114.958 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.57.114.959 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
0.57.114.959 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
0.57.114.960 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
0.57.114.960 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
0.57.114.961 I srv update: - prompt 0x5bea712ba060: 258 tokens, checkpoints: 1, 11.608 MiB
|
||
|
|
0.57.114.961 I srv update: - prompt 0x5bea7867aee0: 291 tokens, checkpoints: 1, 12.382 MiB
|
||
|
|
0.57.114.962 I srv update: - prompt 0x5bea7148a050: 313 tokens, checkpoints: 1, 12.898 MiB
|
||
|
|
0.57.114.962 I srv update: - prompt 0x5bea7855c740: 352 tokens, checkpoints: 1, 13.812 MiB
|
||
|
|
0.57.114.963 I srv update: - prompt 0x5bea7137e380: 310 tokens, checkpoints: 1, 12.827 MiB
|
||
|
|
0.57.114.963 I srv get_availabl: prompt cache update took 9.13 ms
|
||
|
|
0.57.115.919 I slot launch_slot_: id 3 | task 3843 | processing task, is_child = 0
|
||
|
|
0.57.115.921 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.57.115.922 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.57.115.930 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.57.115.930 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.57.115.936 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.57.115.936 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.57.115.949 I slot update_slots: id 3 | task 3843 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.57.117.647 W slot update_slots: id 3 | task 3843 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
0.59.777.541 I slot print_timing: id 3 | task 3843 | prompt eval time = 94.88 ms / 17 tokens ( 5.58 ms per token, 179.18 tokens per second)
|
||
|
|
0.59.777.545 I slot print_timing: id 3 | task 3843 | eval time = 2566.71 ms / 230 tokens ( 11.16 ms per token, 89.61 tokens per second)
|
||
|
|
0.59.777.546 I slot print_timing: id 3 | task 3843 | total time = 2661.59 ms / 247 tokens
|
||
|
|
0.59.777.546 I slot print_timing: id 3 | task 3843 | graphs reused = 3909
|
||
|
|
0.59.777.577 I slot release: id 3 | task 3843 | stop processing: n_tokens = 354, truncated = 0
|
||
|
|
0.59.777.589 I srv update_slots: all slots are idle
|
||
|
|
0.59.785.457 I srv operator(): Chat format: peg-native
|
||
|
|
0.59.789.188 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.862 (> 0.100 thold), f_keep = 0.316
|
||
|
|
0.59.789.189 I srv get_availabl: updating prompt cache
|
||
|
|
0.59.789.224 W srv prompt_save: - saving prompt with length 354, total state size = 11.303 MiB (draft: 0.000 MiB)
|
||
|
|
0.59.799.024 I srv load: - looking for better prompt, base f_keep = 0.316, sim = 0.862
|
||
|
|
0.59.799.032 I srv load: - found better prompt with f_keep = 0.380, sim = 0.869
|
||
|
|
0.59.803.486 I srv update: - cache state: 13 prompts, 163.122 MiB (limits: 8192.000 MiB, 8192 tokens, 194602 est)
|
||
|
|
0.59.803.489 I srv update: - prompt 0x5bea78567de0: 441 tokens, checkpoints: 1, 15.899 MiB
|
||
|
|
0.59.803.489 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
0.59.803.490 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
0.59.803.490 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
0.59.803.491 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
0.59.803.491 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
0.59.803.492 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
0.59.803.492 I srv update: - prompt 0x5bea712ba060: 258 tokens, checkpoints: 1, 11.608 MiB
|
||
|
|
0.59.803.493 I srv update: - prompt 0x5bea7867aee0: 291 tokens, checkpoints: 1, 12.382 MiB
|
||
|
|
0.59.803.493 I srv update: - prompt 0x5bea7148a050: 313 tokens, checkpoints: 1, 12.898 MiB
|
||
|
|
0.59.803.494 I srv update: - prompt 0x5bea7855c740: 352 tokens, checkpoints: 1, 13.812 MiB
|
||
|
|
0.59.803.497 I srv update: - prompt 0x5bea7137e380: 310 tokens, checkpoints: 1, 12.827 MiB
|
||
|
|
0.59.803.498 I srv update: - prompt 0x5bea714ca9c0: 354 tokens, checkpoints: 1, 13.859 MiB
|
||
|
|
0.59.803.499 I srv get_availabl: prompt cache update took 14.31 ms
|
||
|
|
0.59.804.456 I slot launch_slot_: id 3 | task 4076 | processing task, is_child = 0
|
||
|
|
0.59.804.458 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.59.804.459 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.59.804.466 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.59.804.466 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.59.804.471 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.59.804.471 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.59.804.485 I slot update_slots: id 3 | task 4076 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
0.59.805.943 W slot update_slots: id 3 | task 4076 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
1.02.917.145 I slot print_timing: id 3 | task 4076 | n_decoded = 267, tg = 88.96 t/s, tg_3s = 88.96 t/s
|
||
|
|
1.05.923.520 I slot print_timing: id 3 | task 4076 | n_decoded = 530, tg = 88.22 t/s, tg_3s = 87.48 t/s
|
||
|
|
1.08.929.354 I slot print_timing: id 3 | task 4076 | n_decoded = 791, tg = 87.76 t/s, tg_3s = 86.83 t/s
|
||
|
|
1.09.101.675 I slot print_timing: id 3 | task 4076 | prompt eval time = 111.22 ms / 22 tokens ( 5.06 ms per token, 197.80 tokens per second)
|
||
|
|
1.09.101.678 I slot print_timing: id 3 | task 4076 | eval time = 9185.96 ms / 806 tokens ( 11.40 ms per token, 87.74 tokens per second)
|
||
|
|
1.09.101.678 I slot print_timing: id 3 | task 4076 | total time = 9297.19 ms / 828 tokens
|
||
|
|
1.09.101.679 I slot print_timing: id 3 | task 4076 | graphs reused = 4710
|
||
|
|
1.09.101.711 I slot release: id 3 | task 4076 | stop processing: n_tokens = 935, truncated = 0
|
||
|
|
1.09.101.722 I srv update_slots: all slots are idle
|
||
|
|
1.09.109.737 I srv operator(): Chat format: peg-native
|
||
|
|
1.09.113.512 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.850 (> 0.100 thold), f_keep = 0.121
|
||
|
|
1.09.113.514 I srv get_availabl: updating prompt cache
|
||
|
|
1.09.113.560 W srv prompt_save: - saving prompt with length 935, total state size = 24.927 MiB (draft: 0.000 MiB)
|
||
|
|
1.09.133.394 I srv load: - looking for better prompt, base f_keep = 0.121, sim = 0.850
|
||
|
|
1.09.133.402 I srv load: - found better prompt with f_keep = 0.261, sim = 0.865
|
||
|
|
1.09.138.836 I srv update: - cache state: 13 prompts, 174.706 MiB (limits: 8192.000 MiB, 8192 tokens, 204863 est)
|
||
|
|
1.09.138.838 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
1.09.138.839 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
1.09.138.840 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
1.09.138.840 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
1.09.138.841 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
1.09.138.841 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
1.09.138.842 I srv update: - prompt 0x5bea712ba060: 258 tokens, checkpoints: 1, 11.608 MiB
|
||
|
|
1.09.138.842 I srv update: - prompt 0x5bea7867aee0: 291 tokens, checkpoints: 1, 12.382 MiB
|
||
|
|
1.09.138.842 I srv update: - prompt 0x5bea7148a050: 313 tokens, checkpoints: 1, 12.898 MiB
|
||
|
|
1.09.138.844 I srv update: - prompt 0x5bea7855c740: 352 tokens, checkpoints: 1, 13.812 MiB
|
||
|
|
1.09.138.845 I srv update: - prompt 0x5bea7137e380: 310 tokens, checkpoints: 1, 12.827 MiB
|
||
|
|
1.09.138.845 I srv update: - prompt 0x5bea714ca9c0: 354 tokens, checkpoints: 1, 13.859 MiB
|
||
|
|
1.09.138.845 I srv update: - prompt 0x5bea712d6920: 935 tokens, checkpoints: 1, 27.483 MiB
|
||
|
|
1.09.138.847 I srv get_availabl: prompt cache update took 25.33 ms
|
||
|
|
1.09.139.792 I slot launch_slot_: id 3 | task 4885 | processing task, is_child = 0
|
||
|
|
1.09.139.793 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.09.139.794 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.09.139.802 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.09.139.803 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.09.139.808 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.09.139.808 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.09.139.819 I slot update_slots: id 3 | task 4885 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
1.09.141.420 W slot update_slots: id 3 | task 4885 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
1.10.268.573 I slot print_timing: id 3 | task 4885 | prompt eval time = 121.87 ms / 25 tokens ( 4.87 ms per token, 205.13 tokens per second)
|
||
|
|
1.10.268.577 I slot print_timing: id 3 | task 4885 | eval time = 1006.88 ms / 90 tokens ( 11.19 ms per token, 89.39 tokens per second)
|
||
|
|
1.10.268.577 I slot print_timing: id 3 | task 4885 | total time = 1128.75 ms / 115 tokens
|
||
|
|
1.10.268.578 I slot print_timing: id 3 | task 4885 | graphs reused = 4798
|
||
|
|
1.10.268.610 I slot release: id 3 | task 4885 | stop processing: n_tokens = 222, truncated = 0
|
||
|
|
1.10.268.617 I srv update_slots: all slots are idle
|
||
|
|
1.10.276.352 I srv operator(): Chat format: peg-native
|
||
|
|
1.10.280.044 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.896 (> 0.100 thold), f_keep = 0.505
|
||
|
|
1.10.281.039 I slot launch_slot_: id 3 | task 4978 | processing task, is_child = 0
|
||
|
|
1.10.281.040 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.10.281.041 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.10.281.048 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.10.281.048 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.10.281.053 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.10.281.053 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.10.281.061 I slot update_slots: id 3 | task 4978 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
1.10.282.840 W slot update_slots: id 3 | task 4978 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
1.11.722.414 I slot print_timing: id 3 | task 4978 | prompt eval time = 94.10 ms / 17 tokens ( 5.54 ms per token, 180.66 tokens per second)
|
||
|
|
1.11.722.417 I slot print_timing: id 3 | task 4978 | eval time = 1347.25 ms / 120 tokens ( 11.23 ms per token, 89.07 tokens per second)
|
||
|
|
1.11.722.417 I slot print_timing: id 3 | task 4978 | total time = 1441.35 ms / 137 tokens
|
||
|
|
1.11.722.418 I slot print_timing: id 3 | task 4978 | graphs reused = 4916
|
||
|
|
1.11.722.447 I slot release: id 3 | task 4978 | stop processing: n_tokens = 244, truncated = 0
|
||
|
|
1.11.722.456 I srv update_slots: all slots are idle
|
||
|
|
1.11.730.297 I srv operator(): Chat format: peg-native
|
||
|
|
1.11.734.219 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.926 (> 0.100 thold), f_keep = 0.459
|
||
|
|
1.11.734.221 I srv get_availabl: updating prompt cache
|
||
|
|
1.11.734.248 W srv prompt_save: - saving prompt with length 244, total state size = 8.724 MiB (draft: 0.000 MiB)
|
||
|
|
1.11.739.118 I srv load: - looking for better prompt, base f_keep = 0.459, sim = 0.926
|
||
|
|
1.11.739.132 I srv update: - cache state: 14 prompts, 185.986 MiB (limits: 8192.000 MiB, 8192 tokens, 203185 est)
|
||
|
|
1.11.739.133 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
1.11.739.133 I srv update: - prompt 0x5bea714456a0: 242 tokens, checkpoints: 1, 11.233 MiB
|
||
|
|
1.11.739.134 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
1.11.739.134 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
1.11.739.135 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
1.11.739.137 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
1.11.739.138 I srv update: - prompt 0x5bea712ba060: 258 tokens, checkpoints: 1, 11.608 MiB
|
||
|
|
1.11.739.138 I srv update: - prompt 0x5bea7867aee0: 291 tokens, checkpoints: 1, 12.382 MiB
|
||
|
|
1.11.739.139 I srv update: - prompt 0x5bea7148a050: 313 tokens, checkpoints: 1, 12.898 MiB
|
||
|
|
1.11.739.139 I srv update: - prompt 0x5bea7855c740: 352 tokens, checkpoints: 1, 13.812 MiB
|
||
|
|
1.11.739.140 I srv update: - prompt 0x5bea7137e380: 310 tokens, checkpoints: 1, 12.827 MiB
|
||
|
|
1.11.739.140 I srv update: - prompt 0x5bea714ca9c0: 354 tokens, checkpoints: 1, 13.859 MiB
|
||
|
|
1.11.739.140 I srv update: - prompt 0x5bea712d6920: 935 tokens, checkpoints: 1, 27.483 MiB
|
||
|
|
1.11.739.141 I srv update: - prompt 0x5bea7853a2d0: 244 tokens, checkpoints: 1, 11.280 MiB
|
||
|
|
1.11.739.142 I srv get_availabl: prompt cache update took 4.92 ms
|
||
|
|
1.11.740.070 I slot launch_slot_: id 3 | task 5101 | processing task, is_child = 0
|
||
|
|
1.11.740.072 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.11.740.073 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.11.740.081 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.11.740.081 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.11.740.086 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.11.740.087 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.11.740.099 I slot update_slots: id 3 | task 5101 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
1.11.741.613 W slot update_slots: id 3 | task 5101 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
1.12.718.539 I slot print_timing: id 3 | task 5101 | prompt eval time = 58.77 ms / 13 tokens ( 4.52 ms per token, 221.19 tokens per second)
|
||
|
|
1.12.718.543 I slot print_timing: id 3 | task 5101 | eval time = 919.66 ms / 82 tokens ( 11.22 ms per token, 89.16 tokens per second)
|
||
|
|
1.12.718.543 I slot print_timing: id 3 | task 5101 | total time = 978.44 ms / 95 tokens
|
||
|
|
1.12.718.544 I slot print_timing: id 3 | task 5101 | graphs reused = 4996
|
||
|
|
1.12.718.574 I slot release: id 3 | task 5101 | stop processing: n_tokens = 202, truncated = 0
|
||
|
|
1.12.718.581 I srv update_slots: all slots are idle
|
||
|
|
1.12.726.618 I srv operator(): Chat format: peg-native
|
||
|
|
1.12.730.353 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.868 (> 0.100 thold), f_keep = 0.554
|
||
|
|
1.12.731.246 I slot launch_slot_: id 3 | task 5186 | processing task, is_child = 0
|
||
|
|
1.12.731.248 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.12.731.248 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.12.731.256 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.12.731.256 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.12.731.261 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.12.731.261 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.12.731.269 I slot update_slots: id 3 | task 5186 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
1.12.733.075 W slot update_slots: id 3 | task 5186 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
1.15.097.236 I slot print_timing: id 3 | task 5186 | prompt eval time = 98.79 ms / 21 tokens ( 4.70 ms per token, 212.57 tokens per second)
|
||
|
|
1.15.097.240 I slot print_timing: id 3 | task 5186 | eval time = 2267.17 ms / 200 tokens ( 11.34 ms per token, 88.22 tokens per second)
|
||
|
|
1.15.097.240 I slot print_timing: id 3 | task 5186 | total time = 2365.96 ms / 221 tokens
|
||
|
|
1.15.097.241 I slot print_timing: id 3 | task 5186 | graphs reused = 5193
|
||
|
|
1.15.097.273 I slot release: id 3 | task 5186 | stop processing: n_tokens = 328, truncated = 0
|
||
|
|
1.15.097.284 I srv update_slots: all slots are idle
|
||
|
|
1.15.105.254 I srv operator(): Chat format: peg-native
|
||
|
|
1.15.109.245 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.896 (> 0.100 thold), f_keep = 0.341
|
||
|
|
1.15.109.247 I srv get_availabl: updating prompt cache
|
||
|
|
1.15.109.277 W srv prompt_save: - saving prompt with length 328, total state size = 10.693 MiB (draft: 0.000 MiB)
|
||
|
|
1.15.114.768 I srv load: - looking for better prompt, base f_keep = 0.341, sim = 0.896
|
||
|
|
1.15.114.777 I srv load: - found better prompt with f_keep = 0.467, sim = 0.904
|
||
|
|
1.15.118.598 I srv update: - cache state: 14 prompts, 188.003 MiB (limits: 8192.000 MiB, 8192 tokens, 204753 est)
|
||
|
|
1.15.118.600 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
1.15.118.601 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
1.15.118.601 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
1.15.118.602 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
1.15.118.602 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
1.15.118.603 I srv update: - prompt 0x5bea712ba060: 258 tokens, checkpoints: 1, 11.608 MiB
|
||
|
|
1.15.118.603 I srv update: - prompt 0x5bea7867aee0: 291 tokens, checkpoints: 1, 12.382 MiB
|
||
|
|
1.15.118.604 I srv update: - prompt 0x5bea7148a050: 313 tokens, checkpoints: 1, 12.898 MiB
|
||
|
|
1.15.118.605 I srv update: - prompt 0x5bea7855c740: 352 tokens, checkpoints: 1, 13.812 MiB
|
||
|
|
1.15.118.606 I srv update: - prompt 0x5bea7137e380: 310 tokens, checkpoints: 1, 12.827 MiB
|
||
|
|
1.15.118.606 I srv update: - prompt 0x5bea714ca9c0: 354 tokens, checkpoints: 1, 13.859 MiB
|
||
|
|
1.15.118.607 I srv update: - prompt 0x5bea712d6920: 935 tokens, checkpoints: 1, 27.483 MiB
|
||
|
|
1.15.118.607 I srv update: - prompt 0x5bea7853a2d0: 244 tokens, checkpoints: 1, 11.280 MiB
|
||
|
|
1.15.118.608 I srv update: - prompt 0x5bea784f00e0: 328 tokens, checkpoints: 1, 13.250 MiB
|
||
|
|
1.15.118.609 I srv get_availabl: prompt cache update took 9.36 ms
|
||
|
|
1.15.119.531 I slot launch_slot_: id 3 | task 5389 | processing task, is_child = 0
|
||
|
|
1.15.119.533 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.15.119.533 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.15.119.541 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.15.119.541 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.15.119.546 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.15.119.546 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.15.119.557 I slot update_slots: id 3 | task 5389 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
1.15.121.028 W slot update_slots: id 3 | task 5389 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
1.17.143.068 I slot print_timing: id 3 | task 5389 | prompt eval time = 93.25 ms / 17 tokens ( 5.49 ms per token, 182.30 tokens per second)
|
||
|
|
1.17.143.072 I slot print_timing: id 3 | task 5389 | eval time = 1930.25 ms / 170 tokens ( 11.35 ms per token, 88.07 tokens per second)
|
||
|
|
1.17.143.073 I slot print_timing: id 3 | task 5389 | total time = 2023.50 ms / 187 tokens
|
||
|
|
1.17.143.073 I slot print_timing: id 3 | task 5389 | graphs reused = 5360
|
||
|
|
1.17.143.122 I slot release: id 3 | task 5389 | stop processing: n_tokens = 294, truncated = 0
|
||
|
|
1.17.143.131 I srv update_slots: all slots are idle
|
||
|
|
1.17.150.682 I srv operator(): Chat format: peg-native
|
||
|
|
1.17.154.414 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.911 (> 0.100 thold), f_keep = 0.384
|
||
|
|
1.17.154.417 I srv get_availabl: updating prompt cache
|
||
|
|
1.17.154.450 W srv prompt_save: - saving prompt with length 294, total state size = 9.896 MiB (draft: 0.000 MiB)
|
||
|
|
1.17.159.762 I srv load: - looking for better prompt, base f_keep = 0.384, sim = 0.911
|
||
|
|
1.17.159.775 I srv update: - cache state: 15 prompts, 200.455 MiB (limits: 8192.000 MiB, 8192 tokens, 204049 est)
|
||
|
|
1.17.159.776 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
1.17.159.777 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
1.17.159.778 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
1.17.159.778 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
1.17.159.779 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
1.17.159.779 I srv update: - prompt 0x5bea712ba060: 258 tokens, checkpoints: 1, 11.608 MiB
|
||
|
|
1.17.159.780 I srv update: - prompt 0x5bea7867aee0: 291 tokens, checkpoints: 1, 12.382 MiB
|
||
|
|
1.17.159.780 I srv update: - prompt 0x5bea7148a050: 313 tokens, checkpoints: 1, 12.898 MiB
|
||
|
|
1.17.159.781 I srv update: - prompt 0x5bea7855c740: 352 tokens, checkpoints: 1, 13.812 MiB
|
||
|
|
1.17.159.781 I srv update: - prompt 0x5bea7137e380: 310 tokens, checkpoints: 1, 12.827 MiB
|
||
|
|
1.17.159.782 I srv update: - prompt 0x5bea714ca9c0: 354 tokens, checkpoints: 1, 13.859 MiB
|
||
|
|
1.17.159.782 I srv update: - prompt 0x5bea712d6920: 935 tokens, checkpoints: 1, 27.483 MiB
|
||
|
|
1.17.159.783 I srv update: - prompt 0x5bea7853a2d0: 244 tokens, checkpoints: 1, 11.280 MiB
|
||
|
|
1.17.159.783 I srv update: - prompt 0x5bea784f00e0: 328 tokens, checkpoints: 1, 13.250 MiB
|
||
|
|
1.17.159.785 I srv update: - prompt 0x5bea71433430: 294 tokens, checkpoints: 1, 12.452 MiB
|
||
|
|
1.17.159.786 I srv get_availabl: prompt cache update took 5.37 ms
|
||
|
|
1.17.160.670 I slot launch_slot_: id 3 | task 5562 | processing task, is_child = 0
|
||
|
|
1.17.160.672 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.17.160.673 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.17.160.681 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.17.160.681 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.17.160.687 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.17.160.687 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.17.160.698 I slot update_slots: id 3 | task 5562 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
1.17.162.508 W slot update_slots: id 3 | task 5562 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
1.18.557.595 I slot print_timing: id 3 | task 5562 | prompt eval time = 88.33 ms / 16 tokens ( 5.52 ms per token, 181.15 tokens per second)
|
||
|
|
1.18.557.599 I slot print_timing: id 3 | task 5562 | eval time = 1308.57 ms / 116 tokens ( 11.28 ms per token, 88.65 tokens per second)
|
||
|
|
1.18.557.599 I slot print_timing: id 3 | task 5562 | total time = 1396.89 ms / 132 tokens
|
||
|
|
1.18.557.600 I slot print_timing: id 3 | task 5562 | graphs reused = 5474
|
||
|
|
1.18.557.633 I slot release: id 3 | task 5562 | stop processing: n_tokens = 239, truncated = 0
|
||
|
|
1.18.557.641 I srv update_slots: all slots are idle
|
||
|
|
1.18.565.748 I srv operator(): Chat format: peg-native
|
||
|
|
1.18.569.625 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.882 (> 0.100 thold), f_keep = 0.469
|
||
|
|
1.18.569.627 I srv get_availabl: updating prompt cache
|
||
|
|
1.18.569.657 W srv prompt_save: - saving prompt with length 239, total state size = 8.606 MiB (draft: 0.000 MiB)
|
||
|
|
1.18.577.789 I srv load: - looking for better prompt, base f_keep = 0.469, sim = 0.882
|
||
|
|
1.18.577.802 I srv update: - cache state: 16 prompts, 211.617 MiB (limits: 8192.000 MiB, 8192 tokens, 202537 est)
|
||
|
|
1.18.577.806 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
1.18.577.807 I srv update: - prompt 0x5bea78582370: 257 tokens, checkpoints: 1, 11.585 MiB
|
||
|
|
1.18.577.807 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
1.18.577.808 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
1.18.577.808 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
1.18.577.809 I srv update: - prompt 0x5bea712ba060: 258 tokens, checkpoints: 1, 11.608 MiB
|
||
|
|
1.18.577.812 I srv update: - prompt 0x5bea7867aee0: 291 tokens, checkpoints: 1, 12.382 MiB
|
||
|
|
1.18.577.812 I srv update: - prompt 0x5bea7148a050: 313 tokens, checkpoints: 1, 12.898 MiB
|
||
|
|
1.18.577.813 I srv update: - prompt 0x5bea7855c740: 352 tokens, checkpoints: 1, 13.812 MiB
|
||
|
|
1.18.577.813 I srv update: - prompt 0x5bea7137e380: 310 tokens, checkpoints: 1, 12.827 MiB
|
||
|
|
1.18.577.814 I srv update: - prompt 0x5bea714ca9c0: 354 tokens, checkpoints: 1, 13.859 MiB
|
||
|
|
1.18.577.814 I srv update: - prompt 0x5bea712d6920: 935 tokens, checkpoints: 1, 27.483 MiB
|
||
|
|
1.18.577.816 I srv update: - prompt 0x5bea7853a2d0: 244 tokens, checkpoints: 1, 11.280 MiB
|
||
|
|
1.18.577.817 I srv update: - prompt 0x5bea784f00e0: 328 tokens, checkpoints: 1, 13.250 MiB
|
||
|
|
1.18.577.817 I srv update: - prompt 0x5bea71433430: 294 tokens, checkpoints: 1, 12.452 MiB
|
||
|
|
1.18.577.818 I srv update: - prompt 0x5bea7142b210: 239 tokens, checkpoints: 1, 11.163 MiB
|
||
|
|
1.18.577.819 I srv get_availabl: prompt cache update took 8.19 ms
|
||
|
|
1.18.578.706 I slot launch_slot_: id 3 | task 5681 | processing task, is_child = 0
|
||
|
|
1.18.578.707 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.18.578.708 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.18.578.714 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.18.578.715 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.18.578.720 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.18.578.720 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.18.578.730 I slot update_slots: id 3 | task 5681 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
1.18.580.492 W slot update_slots: id 3 | task 5681 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
1.21.678.285 I slot print_timing: id 3 | task 5681 | n_decoded = 263, tg = 87.46 t/s, tg_3s = 87.46 t/s
|
||
|
|
1.24.685.668 I slot print_timing: id 3 | task 5681 | n_decoded = 524, tg = 87.12 t/s, tg_3s = 86.79 t/s
|
||
|
|
1.27.697.171 I slot print_timing: id 3 | task 5681 | n_decoded = 784, tg = 86.86 t/s, tg_3s = 86.34 t/s
|
||
|
|
1.30.697.590 I slot print_timing: id 3 | task 5681 | n_decoded = 1040, tg = 86.48 t/s, tg_3s = 85.32 t/s
|
||
|
|
1.32.586.832 I slot print_timing: id 3 | task 5681 | prompt eval time = 92.39 ms / 19 tokens ( 4.86 ms per token, 205.66 tokens per second)
|
||
|
|
1.32.586.836 I slot print_timing: id 3 | task 5681 | eval time = 13915.71 ms / 1200 tokens ( 11.60 ms per token, 86.23 tokens per second)
|
||
|
|
1.32.586.836 I slot print_timing: id 3 | task 5681 | total time = 14008.10 ms / 1219 tokens
|
||
|
|
1.32.586.837 I slot print_timing: id 3 | task 5681 | graphs reused = 6667
|
||
|
|
1.32.586.872 I slot release: id 3 | task 5681 | stop processing: n_tokens = 1326, truncated = 0
|
||
|
|
1.32.586.887 I srv update_slots: all slots are idle
|
||
|
|
1.32.594.806 I srv operator(): Chat format: peg-native
|
||
|
|
1.32.598.493 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.911 (> 0.100 thold), f_keep = 0.084
|
||
|
|
1.32.598.495 I srv get_availabl: updating prompt cache
|
||
|
|
1.32.598.564 W srv prompt_save: - saving prompt with length 1326, total state size = 34.095 MiB (draft: 0.000 MiB)
|
||
|
|
1.32.628.235 I srv load: - looking for better prompt, base f_keep = 0.084, sim = 0.911
|
||
|
|
1.32.628.244 I srv load: - found better prompt with f_keep = 0.440, sim = 0.919
|
||
|
|
1.32.632.493 I srv update: - cache state: 16 prompts, 236.684 MiB (limits: 8192.000 MiB, 8192 tokens, 218087 est)
|
||
|
|
1.32.632.496 I srv update: - prompt 0x5bea71309a30: 267 tokens, checkpoints: 1, 11.819 MiB
|
||
|
|
1.32.632.496 I srv update: - prompt 0x5bea78544900: 234 tokens, checkpoints: 1, 11.045 MiB
|
||
|
|
1.32.632.497 I srv update: - prompt 0x5bea6d11d1e0: 295 tokens, checkpoints: 1, 12.476 MiB
|
||
|
|
1.32.632.498 I srv update: - prompt 0x5bea713a1740: 261 tokens, checkpoints: 1, 11.678 MiB
|
||
|
|
1.32.632.498 I srv update: - prompt 0x5bea712ba060: 258 tokens, checkpoints: 1, 11.608 MiB
|
||
|
|
1.32.632.499 I srv update: - prompt 0x5bea7867aee0: 291 tokens, checkpoints: 1, 12.382 MiB
|
||
|
|
1.32.632.499 I srv update: - prompt 0x5bea7148a050: 313 tokens, checkpoints: 1, 12.898 MiB
|
||
|
|
1.32.632.500 I srv update: - prompt 0x5bea7855c740: 352 tokens, checkpoints: 1, 13.812 MiB
|
||
|
|
1.32.632.500 I srv update: - prompt 0x5bea7137e380: 310 tokens, checkpoints: 1, 12.827 MiB
|
||
|
|
1.32.632.500 I srv update: - prompt 0x5bea714ca9c0: 354 tokens, checkpoints: 1, 13.859 MiB
|
||
|
|
1.32.632.501 I srv update: - prompt 0x5bea712d6920: 935 tokens, checkpoints: 1, 27.483 MiB
|
||
|
|
1.32.632.502 I srv update: - prompt 0x5bea7853a2d0: 244 tokens, checkpoints: 1, 11.280 MiB
|
||
|
|
1.32.632.503 I srv update: - prompt 0x5bea784f00e0: 328 tokens, checkpoints: 1, 13.250 MiB
|
||
|
|
1.32.632.504 I srv update: - prompt 0x5bea71433430: 294 tokens, checkpoints: 1, 12.452 MiB
|
||
|
|
1.32.632.504 I srv update: - prompt 0x5bea7142b210: 239 tokens, checkpoints: 1, 11.163 MiB
|
||
|
|
1.32.632.505 I srv update: - prompt 0x5bea6de9bc70: 1326 tokens, checkpoints: 1, 36.652 MiB
|
||
|
|
1.32.632.506 I srv get_availabl: prompt cache update took 34.01 ms
|
||
|
|
1.32.633.440 I slot launch_slot_: id 3 | task 6884 | processing task, is_child = 0
|
||
|
|
1.32.633.442 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.32.633.443 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.32.633.450 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.32.633.450 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.32.633.455 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.32.633.455 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.32.633.467 I slot update_slots: id 3 | task 6884 | Checking checkpoint with [0, 108] against 0...
|
||
|
|
1.32.634.958 W slot update_slots: id 3 | task 6884 | restored context checkpoint (pos_min = 0, pos_max = 108, n_tokens = 109, n_past = 108, size = 2.556 MiB)
|
||
|
|
1.35.727.379 I slot print_timing: id 3 | task 6884 | n_decoded = 262, tg = 87.08 t/s, tg_3s = 87.08 t/s
|
||
|
|
1.38.734.145 I slot print_timing: id 3 | task 6884 | n_decoded = 521, tg = 86.61 t/s, tg_3s = 86.14 t/s
|
||
|
|
1.41.738.541 I slot print_timing: id 3 | task 6884 | n_decoded = 776, tg = 86.03 t/s, tg_3s = 84.88 t/s
|
||
|
|
1.44.743.558 I slot print_timing: id 3 | task 6884 | n_decoded = 1026, tg = 85.32 t/s, tg_3s = 83.19 t/s
|
||
|
|
1.46.834.633 I slot print_timing: id 3 | task 6884 | prompt eval time = 85.26 ms / 15 tokens ( 5.68 ms per token, 175.93 tokens per second)
|
||
|
|
1.46.834.637 I slot print_timing: id 3 | task 6884 | eval time = 14115.90 ms / 1200 tokens ( 11.76 ms per token, 85.01 tokens per second)
|
||
|
|
1.46.834.638 I slot print_timing: id 3 | task 6884 | total time = 14201.16 ms / 1215 tokens
|
||
|
|
1.46.834.638 I slot print_timing: id 3 | task 6884 | graphs reused = 7860
|
||
|
|
1.46.834.666 I slot release: id 3 | task 6884 | stop processing: n_tokens = 1322, truncated = 0
|
||
|
|
1.46.834.679 I srv update_slots: all slots are idle
|
||
|
|
1.46.835.279 I srv operator(): operator(): cleaning up before exit...
|