1101 lines
105 KiB
Plaintext
1101 lines
105 KiB
Plaintext
|
G
|
0.00.065.631 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
|
||
|
|
0.00.065.634 I device_info:
|
||
|
|
0.00.065.653 I - ROCm0 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
||
|
|
0.00.065.657 I - ROCm1 : AMD Radeon Graphics (32752 MiB, 32728 MiB free)
|
||
|
|
0.00.065.663 I - CPU : AMD Ryzen 9 3900X 12-Core Processor (128730 MiB, 128730 MiB free)
|
||
|
|
0.00.065.717 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
||
|
|
0.00.065.721 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
|
||
|
|
0.00.065.761 I srv init: running without SSL
|
||
|
|
0.00.065.820 I srv init: using 23 threads for HTTP server
|
||
|
|
0.00.066.155 I srv start: binding port with default address family
|
||
|
|
0.00.067.353 I srv llama_server: loading model
|
||
|
|
0.00.067.363 I srv load_model: loading model '/home/qrf/.cache/llama.cpp/zai-org_GLM-4.7-Flash-Q5_K_M.gguf'
|
||
|
|
0.00.067.412 I common_init_result: fitting params to device memory ...
|
||
|
|
0.00.067.417 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
|
||
|
|
0.01.167.191 W load: special_eot_id is not in special_eog_ids - the tokenizer config may be incorrect
|
||
|
|
0.01.167.196 W load: special_eom_id is not in special_eog_ids - the tokenizer config may be incorrect
|
||
|
|
0.25.094.074 W llama_context: n_ctx_seq (8192) < n_ctx_train (202752) -- the full capacity of the model will not be utilized
|
||
|
|
0.25.188.402 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
||
|
|
0.25.397.668 I srv load_model: initializing slots, n_slots = 4
|
||
|
|
0.25.450.125 W common_speculative_init: no implementations specified for speculative decoding
|
||
|
|
0.25.450.130 I slot load_model: id 0 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.25.450.134 I slot load_model: id 1 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.25.450.135 I slot load_model: id 2 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.25.450.135 I slot load_model: id 3 | task -1 | new slot, n_ctx = 8192
|
||
|
|
0.25.450.184 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
|
||
|
|
0.25.450.184 I srv load_model: use `--cache-ram 0` to disable the prompt cache
|
||
|
|
0.25.450.185 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
||
|
|
0.25.450.185 I srv load_model: context checkpoints enabled, max = 32, min spacing = 256
|
||
|
|
0.25.450.202 I srv init: idle slots will be saved to prompt cache and cleared upon starting a new task
|
||
|
|
0.25.462.273 I init: chat template, example_format: '[gMASK]<sop><|system|>You are a helpful assistant<|user|>Hello<|assistant|></think>Hi there<|user|>How are you?<|assistant|><think>'
|
||
|
|
0.25.471.502 I srv init: init: chat template, thinking = 1
|
||
|
|
0.25.471.544 I srv llama_server: model loaded
|
||
|
|
0.25.471.548 I srv llama_server: server is listening on http://127.0.0.1:8090
|
||
|
|
0.25.471.554 I srv update_slots: all slots are idle
|
||
|
|
0.29.978.638 I srv operator(): Chat format: peg-native
|
||
|
|
0.29.978.840 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
|
||
|
|
0.29.978.843 I srv get_availabl: updating prompt cache
|
||
|
|
0.29.978.849 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
|
||
|
|
0.29.978.854 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
|
||
|
|
0.29.978.859 I srv get_availabl: prompt cache update took 0.01 ms
|
||
|
|
0.29.978.895 I slot launch_slot_: id 3 | task 0 | processing task, is_child = 0
|
||
|
|
0.29.978.896 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.29.978.899 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.29.978.908 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.29.978.909 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.29.978.915 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.29.978.915 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.31.899.699 I slot print_timing: id 3 | task 0 | prompt eval time = 822.13 ms / 65 tokens ( 12.65 ms per token, 79.06 tokens per second)
|
||
|
|
0.31.899.704 I slot print_timing: id 3 | task 0 | eval time = 1098.63 ms / 50 tokens ( 21.97 ms per token, 45.51 tokens per second)
|
||
|
|
0.31.899.705 I slot print_timing: id 3 | task 0 | total time = 1920.76 ms / 115 tokens
|
||
|
|
0.31.899.709 I slot print_timing: id 3 | task 0 | graphs reused = 49
|
||
|
|
0.31.899.765 I slot release: id 3 | task 0 | stop processing: n_tokens = 114, truncated = 0
|
||
|
|
0.31.899.773 I srv update_slots: all slots are idle
|
||
|
|
0.31.910.479 I srv operator(): Chat format: peg-native
|
||
|
|
0.31.910.653 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.395
|
||
|
|
0.31.910.654 I srv get_availabl: updating prompt cache
|
||
|
|
0.31.910.681 W srv prompt_save: - saving prompt with length 114, total state size = 5.888 MiB (draft: 0.000 MiB)
|
||
|
|
0.31.914.703 I srv load: - looking for better prompt, base f_keep = 0.395, sim = 0.789
|
||
|
|
0.31.914.710 I srv update: - cache state: 1 prompts, 5.888 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
0.31.914.711 I srv update: - prompt 0x569a51d72a50: 114 tokens, checkpoints: 0, 5.888 MiB
|
||
|
|
0.31.914.712 I srv get_availabl: prompt cache update took 4.06 ms
|
||
|
|
0.31.914.764 I slot launch_slot_: id 3 | task 52 | processing task, is_child = 0
|
||
|
|
0.31.914.766 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.31.914.767 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.31.914.774 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.31.914.774 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.31.914.779 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.31.914.779 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.32.856.892 I slot print_timing: id 3 | task 52 | prompt eval time = 137.75 ms / 12 tokens ( 11.48 ms per token, 87.11 tokens per second)
|
||
|
|
0.32.856.895 I slot print_timing: id 3 | task 52 | eval time = 804.35 ms / 37 tokens ( 21.74 ms per token, 46.00 tokens per second)
|
||
|
|
0.32.856.896 I slot print_timing: id 3 | task 52 | total time = 942.10 ms / 49 tokens
|
||
|
|
0.32.856.897 I slot print_timing: id 3 | task 52 | graphs reused = 84
|
||
|
|
0.32.856.955 I slot release: id 3 | task 52 | stop processing: n_tokens = 93, truncated = 0
|
||
|
|
0.32.856.965 I srv update_slots: all slots are idle
|
||
|
|
0.32.867.888 I srv operator(): Chat format: peg-native
|
||
|
|
0.32.868.062 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.763 (> 0.100 thold), f_keep = 0.484
|
||
|
|
0.32.868.064 I srv get_availabl: updating prompt cache
|
||
|
|
0.32.868.080 W srv prompt_save: - saving prompt with length 93, total state size = 4.804 MiB (draft: 0.000 MiB)
|
||
|
|
0.32.871.352 I srv load: - looking for better prompt, base f_keep = 0.484, sim = 0.763
|
||
|
|
0.32.871.355 I srv update: - cache state: 2 prompts, 10.692 MiB (limits: 8192.000 MiB, 8192 tokens, 158598 est)
|
||
|
|
0.32.871.356 I srv update: - prompt 0x569a51d72a50: 114 tokens, checkpoints: 0, 5.888 MiB
|
||
|
|
0.32.871.356 I srv update: - prompt 0x569a517d2a10: 93 tokens, checkpoints: 0, 4.804 MiB
|
||
|
|
0.32.871.357 I srv get_availabl: prompt cache update took 3.29 ms
|
||
|
|
0.32.871.379 I slot launch_slot_: id 3 | task 90 | processing task, is_child = 0
|
||
|
|
0.32.871.380 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.32.871.380 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.32.871.385 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.32.871.385 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.32.871.390 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.32.871.390 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.34.280.680 I slot print_timing: id 3 | task 90 | prompt eval time = 144.17 ms / 14 tokens ( 10.30 ms per token, 97.10 tokens per second)
|
||
|
|
0.34.280.683 I slot print_timing: id 3 | task 90 | eval time = 1265.10 ms / 58 tokens ( 21.81 ms per token, 45.85 tokens per second)
|
||
|
|
0.34.280.684 I slot print_timing: id 3 | task 90 | total time = 1409.28 ms / 72 tokens
|
||
|
|
0.34.280.684 I slot print_timing: id 3 | task 90 | graphs reused = 140
|
||
|
|
0.34.280.719 I slot release: id 3 | task 90 | stop processing: n_tokens = 116, truncated = 0
|
||
|
|
0.34.280.728 I srv update_slots: all slots are idle
|
||
|
|
0.34.291.644 I srv operator(): Chat format: peg-native
|
||
|
|
0.34.291.811 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.793 (> 0.100 thold), f_keep = 0.397
|
||
|
|
0.34.291.813 I srv get_availabl: updating prompt cache
|
||
|
|
0.34.291.832 W srv prompt_save: - saving prompt with length 116, total state size = 5.992 MiB (draft: 0.000 MiB)
|
||
|
|
0.34.295.456 I srv load: - looking for better prompt, base f_keep = 0.397, sim = 0.793
|
||
|
|
0.34.295.460 I srv update: - cache state: 3 prompts, 16.684 MiB (limits: 8192.000 MiB, 8192 tokens, 158598 est)
|
||
|
|
0.34.295.461 I srv update: - prompt 0x569a51d72a50: 114 tokens, checkpoints: 0, 5.888 MiB
|
||
|
|
0.34.295.461 I srv update: - prompt 0x569a517d2a10: 93 tokens, checkpoints: 0, 4.804 MiB
|
||
|
|
0.34.295.462 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.34.295.463 I srv get_availabl: prompt cache update took 3.65 ms
|
||
|
|
0.34.295.491 I slot launch_slot_: id 3 | task 149 | processing task, is_child = 0
|
||
|
|
0.34.295.492 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.34.295.492 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.34.295.497 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.34.295.497 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.34.295.502 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.34.295.502 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.35.921.905 I slot print_timing: id 3 | task 149 | prompt eval time = 139.03 ms / 12 tokens ( 11.59 ms per token, 86.31 tokens per second)
|
||
|
|
0.35.921.908 I slot print_timing: id 3 | task 149 | eval time = 1487.36 ms / 68 tokens ( 21.87 ms per token, 45.72 tokens per second)
|
||
|
|
0.35.921.909 I slot print_timing: id 3 | task 149 | total time = 1626.39 ms / 80 tokens
|
||
|
|
0.35.921.910 I slot print_timing: id 3 | task 149 | graphs reused = 206
|
||
|
|
0.35.921.943 I slot release: id 3 | task 149 | stop processing: n_tokens = 125, truncated = 0
|
||
|
|
0.35.921.952 I srv update_slots: all slots are idle
|
||
|
|
0.35.932.850 I srv operator(): Chat format: peg-native
|
||
|
|
0.35.933.012 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.767 (> 0.100 thold), f_keep = 0.368
|
||
|
|
0.35.933.014 I srv get_availabl: updating prompt cache
|
||
|
|
0.35.933.040 W srv prompt_save: - saving prompt with length 125, total state size = 6.456 MiB (draft: 0.000 MiB)
|
||
|
|
0.35.937.399 I srv load: - looking for better prompt, base f_keep = 0.368, sim = 0.767
|
||
|
|
0.35.937.401 I srv load: - found better prompt with f_keep = 0.421, sim = 0.800
|
||
|
|
0.35.940.413 I srv update: - cache state: 3 prompts, 17.252 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
0.35.940.415 I srv update: - prompt 0x569a517d2a10: 93 tokens, checkpoints: 0, 4.804 MiB
|
||
|
|
0.35.940.416 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.35.940.416 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.35.940.417 I srv get_availabl: prompt cache update took 7.40 ms
|
||
|
|
0.35.940.452 I slot launch_slot_: id 3 | task 218 | processing task, is_child = 0
|
||
|
|
0.35.940.452 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.35.940.453 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.35.940.458 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.35.940.459 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.35.940.463 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.35.940.464 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.738.610 I slot print_timing: id 3 | task 218 | prompt eval time = 128.26 ms / 12 tokens ( 10.69 ms per token, 93.56 tokens per second)
|
||
|
|
0.37.738.614 I slot print_timing: id 3 | task 218 | eval time = 1669.87 ms / 76 tokens ( 21.97 ms per token, 45.51 tokens per second)
|
||
|
|
0.37.738.615 I slot print_timing: id 3 | task 218 | total time = 1798.13 ms / 88 tokens
|
||
|
|
0.37.738.615 I slot print_timing: id 3 | task 218 | graphs reused = 280
|
||
|
|
0.37.738.648 I slot release: id 3 | task 218 | stop processing: n_tokens = 135, truncated = 0
|
||
|
|
0.37.738.655 I srv update_slots: all slots are idle
|
||
|
|
0.37.749.545 I srv operator(): Chat format: peg-native
|
||
|
|
0.37.749.709 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.750 (> 0.100 thold), f_keep = 0.333
|
||
|
|
0.37.749.711 I srv get_availabl: updating prompt cache
|
||
|
|
0.37.749.733 W srv prompt_save: - saving prompt with length 135, total state size = 6.973 MiB (draft: 0.000 MiB)
|
||
|
|
0.37.756.413 I srv load: - looking for better prompt, base f_keep = 0.333, sim = 0.750
|
||
|
|
0.37.756.416 I srv load: - found better prompt with f_keep = 0.495, sim = 0.767
|
||
|
|
0.37.759.277 I srv update: - cache state: 3 prompts, 19.421 MiB (limits: 8192.000 MiB, 8192 tokens, 158600 est)
|
||
|
|
0.37.759.279 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.37.759.279 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.37.759.280 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.37.759.281 I srv get_availabl: prompt cache update took 9.57 ms
|
||
|
|
0.37.759.314 I slot launch_slot_: id 3 | task 295 | processing task, is_child = 0
|
||
|
|
0.37.759.314 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.759.315 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.759.320 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.759.320 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.37.759.325 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.37.759.325 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.38.224.644 I slot print_timing: id 3 | task 295 | prompt eval time = 143.58 ms / 14 tokens ( 10.26 ms per token, 97.51 tokens per second)
|
||
|
|
0.38.224.647 I slot print_timing: id 3 | task 295 | eval time = 321.73 ms / 15 tokens ( 21.45 ms per token, 46.62 tokens per second)
|
||
|
|
0.38.224.648 I slot print_timing: id 3 | task 295 | total time = 465.31 ms / 29 tokens
|
||
|
|
0.38.224.649 I slot print_timing: id 3 | task 295 | graphs reused = 293
|
||
|
|
0.38.224.707 I slot release: id 3 | task 295 | stop processing: n_tokens = 74, truncated = 0
|
||
|
|
0.38.224.714 I srv update_slots: all slots are idle
|
||
|
|
0.38.235.364 I srv operator(): Chat format: peg-native
|
||
|
|
0.38.235.554 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.703 (> 0.100 thold), f_keep = 0.608
|
||
|
|
0.38.235.578 I slot launch_slot_: id 3 | task 311 | processing task, is_child = 0
|
||
|
|
0.38.235.579 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.38.235.579 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.38.235.586 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.38.235.586 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.38.235.592 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.38.235.592 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.39.051.735 I slot print_timing: id 3 | task 311 | prompt eval time = 186.15 ms / 19 tokens ( 9.80 ms per token, 102.07 tokens per second)
|
||
|
|
0.39.051.738 I slot print_timing: id 3 | task 311 | eval time = 629.98 ms / 29 tokens ( 21.72 ms per token, 46.03 tokens per second)
|
||
|
|
0.39.051.739 I slot print_timing: id 3 | task 311 | total time = 816.13 ms / 48 tokens
|
||
|
|
0.39.051.739 I slot print_timing: id 3 | task 311 | graphs reused = 320
|
||
|
|
0.39.051.772 I slot release: id 3 | task 311 | stop processing: n_tokens = 92, truncated = 0
|
||
|
|
0.39.051.778 I srv update_slots: all slots are idle
|
||
|
|
0.39.062.942 I srv operator(): Chat format: peg-native
|
||
|
|
0.39.063.115 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.852 (> 0.100 thold), f_keep = 0.500
|
||
|
|
0.39.063.136 I slot launch_slot_: id 3 | task 341 | processing task, is_child = 0
|
||
|
|
0.39.063.137 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.39.063.142 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.39.063.147 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.39.063.148 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.39.063.152 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.39.063.152 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.40.390.135 I slot print_timing: id 3 | task 341 | prompt eval time = 78.13 ms / 8 tokens ( 9.77 ms per token, 102.40 tokens per second)
|
||
|
|
0.40.390.138 I slot print_timing: id 3 | task 341 | eval time = 1248.84 ms / 57 tokens ( 21.91 ms per token, 45.64 tokens per second)
|
||
|
|
0.40.390.139 I slot print_timing: id 3 | task 341 | total time = 1326.97 ms / 65 tokens
|
||
|
|
0.40.390.139 I slot print_timing: id 3 | task 341 | graphs reused = 375
|
||
|
|
0.40.390.172 I slot release: id 3 | task 341 | stop processing: n_tokens = 110, truncated = 0
|
||
|
|
0.40.390.180 I srv update_slots: all slots are idle
|
||
|
|
0.40.401.859 I srv operator(): Chat format: peg-native
|
||
|
|
0.40.402.020 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.682 (> 0.100 thold), f_keep = 0.409
|
||
|
|
0.40.402.021 I srv get_availabl: updating prompt cache
|
||
|
|
0.40.402.045 W srv prompt_save: - saving prompt with length 110, total state size = 5.682 MiB (draft: 0.000 MiB)
|
||
|
|
0.40.405.567 I srv load: - looking for better prompt, base f_keep = 0.409, sim = 0.682
|
||
|
|
0.40.405.572 I srv update: - cache state: 4 prompts, 25.103 MiB (limits: 8192.000 MiB, 8192 tokens, 158600 est)
|
||
|
|
0.40.405.573 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.40.405.573 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.40.405.574 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.40.405.574 I srv update: - prompt 0x569a51cda8f0: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
0.40.405.574 I srv get_availabl: prompt cache update took 3.55 ms
|
||
|
|
0.40.405.600 I slot launch_slot_: id 3 | task 399 | processing task, is_child = 0
|
||
|
|
0.40.405.600 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.40.405.601 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.40.405.606 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.40.405.607 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.40.405.611 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.40.405.611 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.41.265.739 I slot print_timing: id 3 | task 399 | prompt eval time = 184.90 ms / 21 tokens ( 8.80 ms per token, 113.58 tokens per second)
|
||
|
|
0.41.265.743 I slot print_timing: id 3 | task 399 | eval time = 675.22 ms / 31 tokens ( 21.78 ms per token, 45.91 tokens per second)
|
||
|
|
0.41.265.744 I slot print_timing: id 3 | task 399 | total time = 860.12 ms / 52 tokens
|
||
|
|
0.41.265.744 I slot print_timing: id 3 | task 399 | graphs reused = 404
|
||
|
|
0.41.265.779 I slot release: id 3 | task 399 | stop processing: n_tokens = 96, truncated = 0
|
||
|
|
0.41.265.787 I srv update_slots: all slots are idle
|
||
|
|
0.41.276.763 I srv operator(): Chat format: peg-native
|
||
|
|
0.41.276.932 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.763 (> 0.100 thold), f_keep = 0.469
|
||
|
|
0.41.276.933 I srv get_availabl: updating prompt cache
|
||
|
|
0.41.276.949 W srv prompt_save: - saving prompt with length 96, total state size = 4.959 MiB (draft: 0.000 MiB)
|
||
|
|
0.41.280.325 I srv load: - looking for better prompt, base f_keep = 0.469, sim = 0.763
|
||
|
|
0.41.280.329 I srv update: - cache state: 5 prompts, 30.061 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
0.41.280.330 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.41.280.330 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.41.280.331 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.41.280.331 I srv update: - prompt 0x569a51cda8f0: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
0.41.280.332 I srv update: - prompt 0x569a58a3bdb0: 96 tokens, checkpoints: 0, 4.959 MiB
|
||
|
|
0.41.280.332 I srv get_availabl: prompt cache update took 3.40 ms
|
||
|
|
0.41.280.359 I slot launch_slot_: id 3 | task 431 | processing task, is_child = 0
|
||
|
|
0.41.280.360 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.41.280.360 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.41.280.365 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.41.280.365 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.41.280.370 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.41.280.370 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.43.141.125 I slot print_timing: id 3 | task 431 | prompt eval time = 147.64 ms / 14 tokens ( 10.55 ms per token, 94.82 tokens per second)
|
||
|
|
0.43.141.128 I slot print_timing: id 3 | task 431 | eval time = 1713.10 ms / 78 tokens ( 21.96 ms per token, 45.53 tokens per second)
|
||
|
|
0.43.141.129 I slot print_timing: id 3 | task 431 | total time = 1860.74 ms / 92 tokens
|
||
|
|
0.43.141.129 I slot print_timing: id 3 | task 431 | graphs reused = 480
|
||
|
|
0.43.141.161 I slot release: id 3 | task 431 | stop processing: n_tokens = 136, truncated = 0
|
||
|
|
0.43.141.168 I srv update_slots: all slots are idle
|
||
|
|
0.43.151.923 I srv operator(): Chat format: peg-native
|
||
|
|
0.43.152.087 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.692 (> 0.100 thold), f_keep = 0.331
|
||
|
|
0.43.152.088 I srv get_availabl: updating prompt cache
|
||
|
|
0.43.152.106 W srv prompt_save: - saving prompt with length 136, total state size = 7.025 MiB (draft: 0.000 MiB)
|
||
|
|
0.43.158.645 I srv load: - looking for better prompt, base f_keep = 0.331, sim = 0.692
|
||
|
|
0.43.158.651 I srv update: - cache state: 6 prompts, 37.086 MiB (limits: 8192.000 MiB, 8192 tokens, 158600 est)
|
||
|
|
0.43.158.652 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.43.158.652 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.43.158.653 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.43.158.653 I srv update: - prompt 0x569a51cda8f0: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
0.43.158.654 I srv update: - prompt 0x569a58a3bdb0: 96 tokens, checkpoints: 0, 4.959 MiB
|
||
|
|
0.43.158.654 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.43.158.654 I srv get_availabl: prompt cache update took 6.57 ms
|
||
|
|
0.43.158.689 I slot launch_slot_: id 3 | task 510 | processing task, is_child = 0
|
||
|
|
0.43.158.689 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.43.158.690 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.43.158.695 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.43.158.696 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.43.158.700 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.43.158.700 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.44.202.371 I slot print_timing: id 3 | task 510 | prompt eval time = 190.23 ms / 20 tokens ( 9.51 ms per token, 105.14 tokens per second)
|
||
|
|
0.44.202.374 I slot print_timing: id 3 | task 510 | eval time = 853.43 ms / 39 tokens ( 21.88 ms per token, 45.70 tokens per second)
|
||
|
|
0.44.202.375 I slot print_timing: id 3 | task 510 | total time = 1043.66 ms / 59 tokens
|
||
|
|
0.44.202.376 I slot print_timing: id 3 | task 510 | graphs reused = 517
|
||
|
|
0.44.202.410 I slot release: id 3 | task 510 | stop processing: n_tokens = 103, truncated = 0
|
||
|
|
0.44.202.417 I srv update_slots: all slots are idle
|
||
|
|
0.44.212.933 I srv operator(): Chat format: peg-native
|
||
|
|
0.44.213.099 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.750 (> 0.100 thold), f_keep = 0.437
|
||
|
|
0.44.213.100 I srv get_availabl: updating prompt cache
|
||
|
|
0.44.213.119 W srv prompt_save: - saving prompt with length 103, total state size = 5.320 MiB (draft: 0.000 MiB)
|
||
|
|
0.44.218.879 I srv load: - looking for better prompt, base f_keep = 0.437, sim = 0.750
|
||
|
|
0.44.218.886 I srv update: - cache state: 7 prompts, 42.406 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
0.44.218.887 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.44.218.887 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.44.218.888 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.44.218.888 I srv update: - prompt 0x569a51cda8f0: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
0.44.218.889 I srv update: - prompt 0x569a58a3bdb0: 96 tokens, checkpoints: 0, 4.959 MiB
|
||
|
|
0.44.218.889 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.44.218.890 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
0.44.218.890 I srv get_availabl: prompt cache update took 5.79 ms
|
||
|
|
0.44.218.919 I slot launch_slot_: id 3 | task 550 | processing task, is_child = 0
|
||
|
|
0.44.218.920 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.44.218.920 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.44.218.925 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.44.218.926 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.44.218.930 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.44.218.930 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.45.284.363 I slot print_timing: id 3 | task 550 | prompt eval time = 143.65 ms / 15 tokens ( 9.58 ms per token, 104.42 tokens per second)
|
||
|
|
0.45.284.366 I slot print_timing: id 3 | task 550 | eval time = 921.77 ms / 42 tokens ( 21.95 ms per token, 45.56 tokens per second)
|
||
|
|
0.45.284.367 I slot print_timing: id 3 | task 550 | total time = 1065.42 ms / 57 tokens
|
||
|
|
0.45.284.368 I slot print_timing: id 3 | task 550 | graphs reused = 557
|
||
|
|
0.45.284.402 I slot release: id 3 | task 550 | stop processing: n_tokens = 101, truncated = 0
|
||
|
|
0.45.284.409 I srv update_slots: all slots are idle
|
||
|
|
0.45.295.273 I srv operator(): Chat format: peg-native
|
||
|
|
0.45.295.433 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.738 (> 0.100 thold), f_keep = 0.446
|
||
|
|
0.45.295.434 I srv get_availabl: updating prompt cache
|
||
|
|
0.45.295.455 W srv prompt_save: - saving prompt with length 101, total state size = 5.217 MiB (draft: 0.000 MiB)
|
||
|
|
0.45.300.885 I srv load: - looking for better prompt, base f_keep = 0.446, sim = 0.738
|
||
|
|
0.45.300.891 I srv update: - cache state: 8 prompts, 47.623 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
0.45.300.892 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.45.300.892 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.45.300.893 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.45.300.893 I srv update: - prompt 0x569a51cda8f0: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
0.45.300.894 I srv update: - prompt 0x569a58a3bdb0: 96 tokens, checkpoints: 0, 4.959 MiB
|
||
|
|
0.45.300.894 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.45.300.894 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
0.45.300.895 I srv update: - prompt 0x569a552036c0: 101 tokens, checkpoints: 0, 5.217 MiB
|
||
|
|
0.45.300.895 I srv get_availabl: prompt cache update took 5.46 ms
|
||
|
|
0.45.300.921 I slot launch_slot_: id 3 | task 593 | processing task, is_child = 0
|
||
|
|
0.45.300.922 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.45.300.923 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.45.300.928 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.45.300.928 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.45.300.932 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.45.300.933 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.45.792.379 I slot print_timing: id 3 | task 593 | prompt eval time = 147.01 ms / 16 tokens ( 9.19 ms per token, 108.84 tokens per second)
|
||
|
|
0.45.792.383 I slot print_timing: id 3 | task 593 | eval time = 344.43 ms / 16 tokens ( 21.53 ms per token, 46.45 tokens per second)
|
||
|
|
0.45.792.383 I slot print_timing: id 3 | task 593 | total time = 491.43 ms / 32 tokens
|
||
|
|
0.45.792.384 I slot print_timing: id 3 | task 593 | graphs reused = 571
|
||
|
|
0.45.792.418 I slot release: id 3 | task 593 | stop processing: n_tokens = 76, truncated = 0
|
||
|
|
0.45.792.425 I srv update_slots: all slots are idle
|
||
|
|
0.45.802.956 I srv operator(): Chat format: peg-native
|
||
|
|
0.45.803.120 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.750 (> 0.100 thold), f_keep = 0.592
|
||
|
|
0.45.803.145 I slot launch_slot_: id 3 | task 610 | processing task, is_child = 0
|
||
|
|
0.45.803.145 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.45.803.146 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.45.803.153 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.45.803.153 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.45.803.158 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.45.803.158 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.47.107.850 I slot print_timing: id 3 | task 610 | prompt eval time = 140.47 ms / 15 tokens ( 9.36 ms per token, 106.78 tokens per second)
|
||
|
|
0.47.107.854 I slot print_timing: id 3 | task 610 | eval time = 1164.21 ms / 53 tokens ( 21.97 ms per token, 45.52 tokens per second)
|
||
|
|
0.47.107.855 I slot print_timing: id 3 | task 610 | total time = 1304.68 ms / 68 tokens
|
||
|
|
0.47.107.855 I slot print_timing: id 3 | task 610 | graphs reused = 622
|
||
|
|
0.47.107.890 I slot release: id 3 | task 610 | stop processing: n_tokens = 112, truncated = 0
|
||
|
|
0.47.107.897 I srv update_slots: all slots are idle
|
||
|
|
0.47.118.709 I srv operator(): Chat format: peg-native
|
||
|
|
0.47.118.876 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.726 (> 0.100 thold), f_keep = 0.402
|
||
|
|
0.47.118.877 I srv get_availabl: updating prompt cache
|
||
|
|
0.47.118.893 W srv prompt_save: - saving prompt with length 112, total state size = 5.785 MiB (draft: 0.000 MiB)
|
||
|
|
0.47.124.580 I srv load: - looking for better prompt, base f_keep = 0.402, sim = 0.726
|
||
|
|
0.47.124.584 I srv load: - found better prompt with f_keep = 0.418, sim = 0.742
|
||
|
|
0.47.127.898 I srv update: - cache state: 8 prompts, 47.726 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
0.47.127.901 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.47.127.901 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.47.127.902 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.47.127.902 I srv update: - prompt 0x569a58a3bdb0: 96 tokens, checkpoints: 0, 4.959 MiB
|
||
|
|
0.47.127.902 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.47.127.903 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
0.47.127.903 I srv update: - prompt 0x569a552036c0: 101 tokens, checkpoints: 0, 5.217 MiB
|
||
|
|
0.47.127.904 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
0.47.127.904 I srv get_availabl: prompt cache update took 9.03 ms
|
||
|
|
0.47.127.935 I slot launch_slot_: id 3 | task 664 | processing task, is_child = 0
|
||
|
|
0.47.127.936 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.47.127.937 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.47.127.942 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.47.127.943 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.47.127.947 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.47.127.949 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.48.597.444 I slot print_timing: id 3 | task 664 | prompt eval time = 148.75 ms / 16 tokens ( 9.30 ms per token, 107.56 tokens per second)
|
||
|
|
0.48.597.448 I slot print_timing: id 3 | task 664 | eval time = 1320.73 ms / 60 tokens ( 22.01 ms per token, 45.43 tokens per second)
|
||
|
|
0.48.597.448 I slot print_timing: id 3 | task 664 | total time = 1469.48 ms / 76 tokens
|
||
|
|
0.48.597.449 I slot print_timing: id 3 | task 664 | graphs reused = 680
|
||
|
|
0.48.597.485 I slot release: id 3 | task 664 | stop processing: n_tokens = 121, truncated = 0
|
||
|
|
0.48.597.494 I srv update_slots: all slots are idle
|
||
|
|
0.48.608.577 I srv operator(): Chat format: peg-native
|
||
|
|
0.48.608.749 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.763 (> 0.100 thold), f_keep = 0.372
|
||
|
|
0.48.608.751 I srv get_availabl: updating prompt cache
|
||
|
|
0.48.608.768 W srv prompt_save: - saving prompt with length 121, total state size = 6.250 MiB (draft: 0.000 MiB)
|
||
|
|
0.48.615.190 I srv load: - looking for better prompt, base f_keep = 0.372, sim = 0.763
|
||
|
|
0.48.615.193 I srv load: - found better prompt with f_keep = 0.479, sim = 0.780
|
||
|
|
0.48.618.170 I srv update: - cache state: 8 prompts, 49.018 MiB (limits: 8192.000 MiB, 8192 tokens, 158600 est)
|
||
|
|
0.48.618.173 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.48.618.174 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.48.618.174 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.48.618.175 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.48.618.175 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
0.48.618.176 I srv update: - prompt 0x569a552036c0: 101 tokens, checkpoints: 0, 5.217 MiB
|
||
|
|
0.48.618.176 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
0.48.618.177 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
0.48.618.177 I srv get_availabl: prompt cache update took 9.43 ms
|
||
|
|
0.48.618.211 I slot launch_slot_: id 3 | task 725 | processing task, is_child = 0
|
||
|
|
0.48.618.212 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.48.618.212 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.48.618.217 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.48.618.218 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.48.618.222 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.48.618.222 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.49.678.997 I slot print_timing: id 3 | task 725 | prompt eval time = 136.46 ms / 13 tokens ( 10.50 ms per token, 95.27 tokens per second)
|
||
|
|
0.49.679.001 I slot print_timing: id 3 | task 725 | eval time = 924.30 ms / 42 tokens ( 22.01 ms per token, 45.44 tokens per second)
|
||
|
|
0.49.679.002 I slot print_timing: id 3 | task 725 | total time = 1060.76 ms / 55 tokens
|
||
|
|
0.49.679.003 I slot print_timing: id 3 | task 725 | graphs reused = 720
|
||
|
|
0.49.679.042 I slot release: id 3 | task 725 | stop processing: n_tokens = 100, truncated = 0
|
||
|
|
0.49.679.048 I srv update_slots: all slots are idle
|
||
|
|
0.49.690.088 I srv operator(): Chat format: peg-native
|
||
|
|
0.49.690.249 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.726 (> 0.100 thold), f_keep = 0.450
|
||
|
|
0.49.690.251 I srv get_availabl: updating prompt cache
|
||
|
|
0.49.690.269 W srv prompt_save: - saving prompt with length 100, total state size = 5.165 MiB (draft: 0.000 MiB)
|
||
|
|
0.49.693.838 I srv load: - looking for better prompt, base f_keep = 0.450, sim = 0.726
|
||
|
|
0.49.693.842 I srv load: - found better prompt with f_keep = 0.455, sim = 0.742
|
||
|
|
0.49.696.910 I srv update: - cache state: 8 prompts, 48.966 MiB (limits: 8192.000 MiB, 8192 tokens, 158600 est)
|
||
|
|
0.49.696.913 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.49.696.913 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.49.696.913 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.49.696.914 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.49.696.914 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
0.49.696.915 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
0.49.696.915 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
0.49.696.915 I srv update: - prompt 0x569a49876a20: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
0.49.696.916 I srv get_availabl: prompt cache update took 6.66 ms
|
||
|
|
0.49.696.946 I slot launch_slot_: id 3 | task 768 | processing task, is_child = 0
|
||
|
|
0.49.696.947 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.49.696.947 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.49.696.953 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.49.696.953 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.49.696.957 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.49.696.957 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.50.922.908 I slot print_timing: id 3 | task 768 | prompt eval time = 146.68 ms / 16 tokens ( 9.17 ms per token, 109.08 tokens per second)
|
||
|
|
0.50.922.912 I slot print_timing: id 3 | task 768 | eval time = 1079.25 ms / 49 tokens ( 22.03 ms per token, 45.40 tokens per second)
|
||
|
|
0.50.922.912 I slot print_timing: id 3 | task 768 | total time = 1225.93 ms / 65 tokens
|
||
|
|
0.50.922.913 I slot print_timing: id 3 | task 768 | graphs reused = 767
|
||
|
|
0.50.922.946 I slot release: id 3 | task 768 | stop processing: n_tokens = 110, truncated = 0
|
||
|
|
0.50.922.955 I srv update_slots: all slots are idle
|
||
|
|
0.50.934.069 I srv operator(): Chat format: peg-native
|
||
|
|
0.50.934.237 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.409
|
||
|
|
0.50.934.239 I srv get_availabl: updating prompt cache
|
||
|
|
0.50.934.258 W srv prompt_save: - saving prompt with length 110, total state size = 5.682 MiB (draft: 0.000 MiB)
|
||
|
|
0.50.940.054 I srv load: - looking for better prompt, base f_keep = 0.409, sim = 0.789
|
||
|
|
0.50.940.057 I srv load: - found better prompt with f_keep = 0.460, sim = 0.807
|
||
|
|
0.50.942.865 I srv update: - cache state: 8 prompts, 49.482 MiB (limits: 8192.000 MiB, 8192 tokens, 158600 est)
|
||
|
|
0.50.942.867 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.50.942.868 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.50.942.868 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.50.942.868 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.50.942.869 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
0.50.942.869 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
0.50.942.870 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
0.50.942.870 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
0.50.942.871 I srv get_availabl: prompt cache update took 8.63 ms
|
||
|
|
0.50.942.899 I slot launch_slot_: id 3 | task 818 | processing task, is_child = 0
|
||
|
|
0.50.942.900 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.50.942.900 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.50.942.905 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.50.942.905 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.50.942.910 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.50.942.910 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.52.041.420 I slot print_timing: id 3 | task 818 | prompt eval time = 129.59 ms / 11 tokens ( 11.78 ms per token, 84.89 tokens per second)
|
||
|
|
0.52.041.423 I slot print_timing: id 3 | task 818 | eval time = 968.91 ms / 44 tokens ( 22.02 ms per token, 45.41 tokens per second)
|
||
|
|
0.52.041.424 I slot print_timing: id 3 | task 818 | total time = 1098.49 ms / 55 tokens
|
||
|
|
0.52.041.425 I slot print_timing: id 3 | task 818 | graphs reused = 809
|
||
|
|
0.52.041.457 I slot release: id 3 | task 818 | stop processing: n_tokens = 100, truncated = 0
|
||
|
|
0.52.041.465 I srv update_slots: all slots are idle
|
||
|
|
0.52.052.833 I srv operator(): Chat format: peg-native
|
||
|
|
0.52.052.996 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.804 (> 0.100 thold), f_keep = 0.450
|
||
|
|
0.52.052.998 I srv get_availabl: updating prompt cache
|
||
|
|
0.52.053.016 W srv prompt_save: - saving prompt with length 100, total state size = 5.165 MiB (draft: 0.000 MiB)
|
||
|
|
0.52.056.504 I srv load: - looking for better prompt, base f_keep = 0.450, sim = 0.804
|
||
|
|
0.52.056.509 I srv update: - cache state: 9 prompts, 54.648 MiB (limits: 8192.000 MiB, 8192 tokens, 158600 est)
|
||
|
|
0.52.056.509 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.52.056.510 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.52.056.510 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.52.056.511 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.52.056.512 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
0.52.056.512 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
0.52.056.513 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
0.52.056.513 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
0.52.056.513 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
0.52.056.514 I srv get_availabl: prompt cache update took 3.52 ms
|
||
|
|
0.52.056.538 I slot launch_slot_: id 3 | task 863 | processing task, is_child = 0
|
||
|
|
0.52.056.539 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.52.056.539 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.52.056.544 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.52.056.545 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.52.056.549 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.52.056.549 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.53.154.146 I slot print_timing: id 3 | task 863 | prompt eval time = 129.21 ms / 11 tokens ( 11.75 ms per token, 85.13 tokens per second)
|
||
|
|
0.53.154.150 I slot print_timing: id 3 | task 863 | eval time = 968.37 ms / 44 tokens ( 22.01 ms per token, 45.44 tokens per second)
|
||
|
|
0.53.154.150 I slot print_timing: id 3 | task 863 | total time = 1097.58 ms / 55 tokens
|
||
|
|
0.53.154.151 I slot print_timing: id 3 | task 863 | graphs reused = 851
|
||
|
|
0.53.154.182 I slot release: id 3 | task 863 | stop processing: n_tokens = 99, truncated = 0
|
||
|
|
0.53.154.190 I srv update_slots: all slots are idle
|
||
|
|
0.53.165.435 I srv operator(): Chat format: peg-native
|
||
|
|
0.53.165.597 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.807 (> 0.100 thold), f_keep = 0.465
|
||
|
|
0.53.165.599 I srv get_availabl: updating prompt cache
|
||
|
|
0.53.165.617 W srv prompt_save: - saving prompt with length 99, total state size = 5.114 MiB (draft: 0.000 MiB)
|
||
|
|
0.53.169.247 I srv load: - looking for better prompt, base f_keep = 0.465, sim = 0.807
|
||
|
|
0.53.169.251 I srv update: - cache state: 10 prompts, 59.761 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
0.53.169.251 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.53.169.252 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.53.169.252 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.53.169.253 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.53.169.253 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
0.53.169.254 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
0.53.169.254 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
0.53.169.254 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
0.53.169.255 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
0.53.169.255 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
0.53.169.256 I srv get_availabl: prompt cache update took 3.66 ms
|
||
|
|
0.53.169.280 I slot launch_slot_: id 3 | task 908 | processing task, is_child = 0
|
||
|
|
0.53.169.281 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.53.169.281 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.53.169.286 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.53.169.286 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.53.169.291 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.53.169.291 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.54.202.400 I slot print_timing: id 3 | task 908 | prompt eval time = 131.34 ms / 11 tokens ( 11.94 ms per token, 83.75 tokens per second)
|
||
|
|
0.54.202.403 I slot print_timing: id 3 | task 908 | eval time = 901.75 ms / 41 tokens ( 21.99 ms per token, 45.47 tokens per second)
|
||
|
|
0.54.202.404 I slot print_timing: id 3 | task 908 | total time = 1033.10 ms / 52 tokens
|
||
|
|
0.54.202.405 I slot print_timing: id 3 | task 908 | graphs reused = 890
|
||
|
|
0.54.202.467 I slot release: id 3 | task 908 | stop processing: n_tokens = 97, truncated = 0
|
||
|
|
0.54.202.474 I srv update_slots: all slots are idle
|
||
|
|
0.54.213.229 I srv operator(): Chat format: peg-native
|
||
|
|
0.54.213.389 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.804 (> 0.100 thold), f_keep = 0.464
|
||
|
|
0.54.213.391 I srv get_availabl: updating prompt cache
|
||
|
|
0.54.213.407 W srv prompt_save: - saving prompt with length 97, total state size = 5.010 MiB (draft: 0.000 MiB)
|
||
|
|
0.54.216.977 I srv load: - looking for better prompt, base f_keep = 0.464, sim = 0.804
|
||
|
|
0.54.216.981 I srv update: - cache state: 11 prompts, 64.772 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
0.54.216.982 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.54.216.983 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.54.216.983 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.54.216.983 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.54.216.984 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
0.54.216.984 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
0.54.216.985 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
0.54.216.985 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
0.54.216.985 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
0.54.216.986 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
0.54.216.986 I srv update: - prompt 0x569a58e820f0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
0.54.216.986 I srv get_availabl: prompt cache update took 3.60 ms
|
||
|
|
0.54.217.015 I slot launch_slot_: id 3 | task 950 | processing task, is_child = 0
|
||
|
|
0.54.217.016 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.54.217.016 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.54.217.021 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.54.217.022 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.54.217.026 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.54.217.026 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.55.382.903 I slot print_timing: id 3 | task 950 | prompt eval time = 131.32 ms / 11 tokens ( 11.94 ms per token, 83.76 tokens per second)
|
||
|
|
0.55.382.907 I slot print_timing: id 3 | task 950 | eval time = 1034.54 ms / 47 tokens ( 22.01 ms per token, 45.43 tokens per second)
|
||
|
|
0.55.382.908 I slot print_timing: id 3 | task 950 | total time = 1165.86 ms / 58 tokens
|
||
|
|
0.55.382.908 I slot print_timing: id 3 | task 950 | graphs reused = 935
|
||
|
|
0.55.382.943 I slot release: id 3 | task 950 | stop processing: n_tokens = 102, truncated = 0
|
||
|
|
0.55.382.951 I srv update_slots: all slots are idle
|
||
|
|
0.55.394.001 I srv operator(): Chat format: peg-native
|
||
|
|
0.55.394.169 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.763 (> 0.100 thold), f_keep = 0.441
|
||
|
|
0.55.394.171 I srv get_availabl: updating prompt cache
|
||
|
|
0.55.394.187 W srv prompt_save: - saving prompt with length 102, total state size = 5.269 MiB (draft: 0.000 MiB)
|
||
|
|
0.55.400.036 I srv load: - looking for better prompt, base f_keep = 0.441, sim = 0.763
|
||
|
|
0.55.400.044 I srv update: - cache state: 12 prompts, 70.040 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
0.55.400.044 I srv update: - prompt 0x569a58d9ad40: 116 tokens, checkpoints: 0, 5.992 MiB
|
||
|
|
0.55.400.045 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.55.400.045 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.55.400.046 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.55.400.046 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
0.55.400.046 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
0.55.400.047 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
0.55.400.047 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
0.55.400.047 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
0.55.400.048 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
0.55.400.049 I srv update: - prompt 0x569a58e820f0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
0.55.400.050 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
0.55.400.051 I srv get_availabl: prompt cache update took 5.88 ms
|
||
|
|
0.55.400.080 I slot launch_slot_: id 3 | task 998 | processing task, is_child = 0
|
||
|
|
0.55.400.081 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.55.400.081 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.55.400.087 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.55.400.087 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.55.400.092 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.55.400.092 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.56.749.427 I slot print_timing: id 3 | task 998 | prompt eval time = 137.43 ms / 14 tokens ( 9.82 ms per token, 101.87 tokens per second)
|
||
|
|
0.56.749.431 I slot print_timing: id 3 | task 998 | eval time = 1211.89 ms / 55 tokens ( 22.03 ms per token, 45.38 tokens per second)
|
||
|
|
0.56.749.431 I slot print_timing: id 3 | task 998 | total time = 1349.32 ms / 69 tokens
|
||
|
|
0.56.749.432 I slot print_timing: id 3 | task 998 | graphs reused = 988
|
||
|
|
0.56.749.465 I slot release: id 3 | task 998 | stop processing: n_tokens = 113, truncated = 0
|
||
|
|
0.56.749.472 I srv update_slots: all slots are idle
|
||
|
|
0.56.760.420 I srv operator(): Chat format: peg-native
|
||
|
|
0.56.760.581 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.398
|
||
|
|
0.56.760.583 I srv get_availabl: updating prompt cache
|
||
|
|
0.56.760.599 W srv prompt_save: - saving prompt with length 113, total state size = 5.837 MiB (draft: 0.000 MiB)
|
||
|
|
0.56.766.414 I srv load: - looking for better prompt, base f_keep = 0.398, sim = 0.776
|
||
|
|
0.56.766.418 I srv load: - found better prompt with f_keep = 0.414, sim = 0.828
|
||
|
|
0.56.769.416 I srv update: - cache state: 12 prompts, 69.885 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
0.56.769.418 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.56.769.418 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.56.769.419 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.56.769.419 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
0.56.769.420 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
0.56.769.420 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
0.56.769.421 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
0.56.769.421 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
0.56.769.421 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
0.56.769.422 I srv update: - prompt 0x569a58e820f0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
0.56.769.422 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
0.56.769.422 I srv update: - prompt 0x569a58e83740: 113 tokens, checkpoints: 0, 5.837 MiB
|
||
|
|
0.56.769.423 I srv get_availabl: prompt cache update took 8.84 ms
|
||
|
|
0.56.769.455 I slot launch_slot_: id 3 | task 1054 | processing task, is_child = 0
|
||
|
|
0.56.769.456 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.56.769.457 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.56.769.462 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.56.769.462 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.56.769.466 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.56.769.467 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.57.287.399 I slot print_timing: id 3 | task 1054 | prompt eval time = 129.32 ms / 10 tokens ( 12.93 ms per token, 77.33 tokens per second)
|
||
|
|
0.57.287.402 I slot print_timing: id 3 | task 1054 | eval time = 388.60 ms / 18 tokens ( 21.59 ms per token, 46.32 tokens per second)
|
||
|
|
0.57.287.403 I slot print_timing: id 3 | task 1054 | total time = 517.91 ms / 28 tokens
|
||
|
|
0.57.287.404 I slot print_timing: id 3 | task 1054 | graphs reused = 1004
|
||
|
|
0.57.287.438 I slot release: id 3 | task 1054 | stop processing: n_tokens = 75, truncated = 0
|
||
|
|
0.57.287.445 I srv update_slots: all slots are idle
|
||
|
|
0.57.297.886 I srv operator(): Chat format: peg-native
|
||
|
|
0.57.298.059 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.703 (> 0.100 thold), f_keep = 0.600
|
||
|
|
0.57.298.082 I slot launch_slot_: id 3 | task 1073 | processing task, is_child = 0
|
||
|
|
0.57.298.083 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.57.298.084 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.57.298.090 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.57.298.090 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.57.298.096 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.57.298.096 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.58.031.392 I slot print_timing: id 3 | task 1073 | prompt eval time = 188.84 ms / 19 tokens ( 9.94 ms per token, 100.61 tokens per second)
|
||
|
|
0.58.031.395 I slot print_timing: id 3 | task 1073 | eval time = 544.44 ms / 25 tokens ( 21.78 ms per token, 45.92 tokens per second)
|
||
|
|
0.58.031.396 I slot print_timing: id 3 | task 1073 | total time = 733.28 ms / 44 tokens
|
||
|
|
0.58.031.397 I slot print_timing: id 3 | task 1073 | graphs reused = 1027
|
||
|
|
0.58.031.433 I slot release: id 3 | task 1073 | stop processing: n_tokens = 88, truncated = 0
|
||
|
|
0.58.031.440 I srv update_slots: all slots are idle
|
||
|
|
0.58.042.025 I srv operator(): Chat format: peg-native
|
||
|
|
0.58.042.199 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.714 (> 0.100 thold), f_keep = 0.511
|
||
|
|
0.58.042.222 I slot launch_slot_: id 3 | task 1099 | processing task, is_child = 0
|
||
|
|
0.58.042.223 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.58.042.224 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.58.042.230 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.58.042.230 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.58.042.236 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.58.042.236 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.58.614.084 I slot print_timing: id 3 | task 1099 | prompt eval time = 183.56 ms / 18 tokens ( 10.20 ms per token, 98.06 tokens per second)
|
||
|
|
0.58.614.087 I slot print_timing: id 3 | task 1099 | eval time = 388.27 ms / 18 tokens ( 21.57 ms per token, 46.36 tokens per second)
|
||
|
|
0.58.614.088 I slot print_timing: id 3 | task 1099 | total time = 571.83 ms / 36 tokens
|
||
|
|
0.58.614.089 I slot print_timing: id 3 | task 1099 | graphs reused = 1043
|
||
|
|
0.58.614.122 I slot release: id 3 | task 1099 | stop processing: n_tokens = 80, truncated = 0
|
||
|
|
0.58.614.130 I srv update_slots: all slots are idle
|
||
|
|
0.58.624.976 I srv operator(): Chat format: peg-native
|
||
|
|
0.58.625.151 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.818 (> 0.100 thold), f_keep = 0.562
|
||
|
|
0.58.625.176 I slot launch_slot_: id 3 | task 1118 | processing task, is_child = 0
|
||
|
|
0.58.625.176 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.58.625.177 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.58.625.185 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.58.625.185 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.58.625.191 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.58.625.191 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.59.698.790 I slot print_timing: id 3 | task 1118 | prompt eval time = 129.42 ms / 10 tokens ( 12.94 ms per token, 77.27 tokens per second)
|
||
|
|
0.59.698.794 I slot print_timing: id 3 | task 1118 | eval time = 944.16 ms / 43 tokens ( 21.96 ms per token, 45.54 tokens per second)
|
||
|
|
0.59.698.794 I slot print_timing: id 3 | task 1118 | total time = 1073.58 ms / 53 tokens
|
||
|
|
0.59.698.795 I slot print_timing: id 3 | task 1118 | graphs reused = 1084
|
||
|
|
0.59.698.828 I slot release: id 3 | task 1118 | stop processing: n_tokens = 97, truncated = 0
|
||
|
|
0.59.698.834 I srv update_slots: all slots are idle
|
||
|
|
0.59.709.495 I srv operator(): Chat format: peg-native
|
||
|
|
0.59.709.671 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.776 (> 0.100 thold), f_keep = 0.464
|
||
|
|
0.59.709.674 I srv get_availabl: updating prompt cache
|
||
|
|
0.59.709.701 W srv prompt_save: - saving prompt with length 97, total state size = 5.010 MiB (draft: 0.000 MiB)
|
||
|
|
0.59.713.290 I srv load: - looking for better prompt, base f_keep = 0.464, sim = 0.776
|
||
|
|
0.59.713.295 I srv update: - cache state: 13 prompts, 74.896 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
0.59.713.295 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
0.59.713.296 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
0.59.713.296 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
0.59.713.297 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
0.59.713.297 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
0.59.713.297 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
0.59.713.298 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
0.59.713.298 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
0.59.713.299 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
0.59.713.299 I srv update: - prompt 0x569a58e820f0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
0.59.713.299 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
0.59.713.300 I srv update: - prompt 0x569a58e83740: 113 tokens, checkpoints: 0, 5.837 MiB
|
||
|
|
0.59.713.300 I srv update: - prompt 0x569a552036c0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
0.59.713.301 I srv get_availabl: prompt cache update took 3.63 ms
|
||
|
|
0.59.713.328 I slot launch_slot_: id 3 | task 1162 | processing task, is_child = 0
|
||
|
|
0.59.713.328 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.59.713.329 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.59.713.334 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.59.713.334 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
0.59.713.339 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
0.59.713.339 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.00.310.070 I slot print_timing: id 3 | task 1162 | prompt eval time = 142.13 ms / 13 tokens ( 10.93 ms per token, 91.47 tokens per second)
|
||
|
|
1.00.310.073 I slot print_timing: id 3 | task 1162 | eval time = 454.59 ms / 21 tokens ( 21.65 ms per token, 46.20 tokens per second)
|
||
|
|
1.00.310.074 I slot print_timing: id 3 | task 1162 | total time = 596.72 ms / 34 tokens
|
||
|
|
1.00.310.074 I slot print_timing: id 3 | task 1162 | graphs reused = 1103
|
||
|
|
1.00.310.110 I slot release: id 3 | task 1162 | stop processing: n_tokens = 78, truncated = 0
|
||
|
|
1.00.310.117 I srv update_slots: all slots are idle
|
||
|
|
1.00.320.774 I srv operator(): Chat format: peg-native
|
||
|
|
1.00.320.934 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.750 (> 0.100 thold), f_keep = 0.577
|
||
|
|
1.00.320.957 I slot launch_slot_: id 3 | task 1184 | processing task, is_child = 0
|
||
|
|
1.00.320.957 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.00.320.958 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.00.320.964 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.00.320.964 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.00.320.969 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.00.320.969 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.01.167.649 I slot print_timing: id 3 | task 1184 | prompt eval time = 146.84 ms / 15 tokens ( 9.79 ms per token, 102.15 tokens per second)
|
||
|
|
1.01.167.653 I slot print_timing: id 3 | task 1184 | eval time = 699.83 ms / 32 tokens ( 21.87 ms per token, 45.73 tokens per second)
|
||
|
|
1.01.167.653 I slot print_timing: id 3 | task 1184 | total time = 846.67 ms / 47 tokens
|
||
|
|
1.01.167.654 I slot print_timing: id 3 | task 1184 | graphs reused = 1133
|
||
|
|
1.01.167.686 I slot release: id 3 | task 1184 | stop processing: n_tokens = 91, truncated = 0
|
||
|
|
1.01.167.695 I srv update_slots: all slots are idle
|
||
|
|
1.01.178.300 I srv operator(): Chat format: peg-native
|
||
|
|
1.01.178.467 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.643 (> 0.100 thold), f_keep = 0.495
|
||
|
|
1.01.178.468 I srv get_availabl: updating prompt cache
|
||
|
|
1.01.178.483 W srv prompt_save: - saving prompt with length 91, total state size = 4.700 MiB (draft: 0.000 MiB)
|
||
|
|
1.01.183.520 I srv load: - looking for better prompt, base f_keep = 0.495, sim = 0.643
|
||
|
|
1.01.183.523 I srv load: - found better prompt with f_keep = 0.495, sim = 0.686
|
||
|
|
1.01.186.253 I srv update: - cache state: 13 prompts, 74.586 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
1.01.186.255 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
1.01.186.255 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
1.01.186.256 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
1.01.186.256 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.01.186.257 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
1.01.186.257 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
1.01.186.258 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
1.01.186.258 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
1.01.186.258 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
1.01.186.259 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
1.01.186.259 I srv update: - prompt 0x569a58e83740: 113 tokens, checkpoints: 0, 5.837 MiB
|
||
|
|
1.01.186.260 I srv update: - prompt 0x569a552036c0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
1.01.186.260 I srv update: - prompt 0x569a51a3b720: 91 tokens, checkpoints: 0, 4.700 MiB
|
||
|
|
1.01.186.261 I srv get_availabl: prompt cache update took 7.79 ms
|
||
|
|
1.01.186.292 I slot launch_slot_: id 3 | task 1217 | processing task, is_child = 0
|
||
|
|
1.01.186.292 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.01.186.293 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.01.186.298 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.01.186.298 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.01.186.303 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.01.186.303 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.03.144.678 I slot print_timing: id 3 | task 1217 | prompt eval time = 189.38 ms / 22 tokens ( 8.61 ms per token, 116.17 tokens per second)
|
||
|
|
1.03.144.681 I slot print_timing: id 3 | task 1217 | eval time = 1768.98 ms / 80 tokens ( 22.11 ms per token, 45.22 tokens per second)
|
||
|
|
1.03.144.682 I slot print_timing: id 3 | task 1217 | total time = 1958.36 ms / 102 tokens
|
||
|
|
1.03.144.683 I slot print_timing: id 3 | task 1217 | graphs reused = 1211
|
||
|
|
1.03.144.719 I slot release: id 3 | task 1217 | stop processing: n_tokens = 149, truncated = 0
|
||
|
|
1.03.144.730 I srv update_slots: all slots are idle
|
||
|
|
1.03.155.211 I srv operator(): Chat format: peg-native
|
||
|
|
1.03.155.377 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.742 (> 0.100 thold), f_keep = 0.309
|
||
|
|
1.03.155.379 I srv get_availabl: updating prompt cache
|
||
|
|
1.03.155.395 W srv prompt_save: - saving prompt with length 149, total state size = 7.696 MiB (draft: 0.000 MiB)
|
||
|
|
1.03.162.374 I srv load: - looking for better prompt, base f_keep = 0.309, sim = 0.742
|
||
|
|
1.03.162.380 I srv update: - cache state: 14 prompts, 82.282 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
1.03.162.381 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
1.03.162.381 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
1.03.162.382 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
1.03.162.382 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.03.162.383 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
1.03.162.383 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
1.03.162.384 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
1.03.162.384 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
1.03.162.384 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
1.03.162.385 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
1.03.162.385 I srv update: - prompt 0x569a58e83740: 113 tokens, checkpoints: 0, 5.837 MiB
|
||
|
|
1.03.162.385 I srv update: - prompt 0x569a552036c0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
1.03.162.387 I srv update: - prompt 0x569a51a3b720: 91 tokens, checkpoints: 0, 4.700 MiB
|
||
|
|
1.03.162.388 I srv update: - prompt 0x569a517d0820: 149 tokens, checkpoints: 0, 7.696 MiB
|
||
|
|
1.03.162.388 I srv get_availabl: prompt cache update took 7.01 ms
|
||
|
|
1.03.162.426 I slot launch_slot_: id 3 | task 1298 | processing task, is_child = 0
|
||
|
|
1.03.162.426 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.03.162.427 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.03.162.432 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.03.162.432 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.03.162.437 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.03.162.437 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.04.057.009 I slot print_timing: id 3 | task 1298 | prompt eval time = 156.96 ms / 16 tokens ( 9.81 ms per token, 101.94 tokens per second)
|
||
|
|
1.04.057.014 I slot print_timing: id 3 | task 1298 | eval time = 737.60 ms / 34 tokens ( 21.69 ms per token, 46.10 tokens per second)
|
||
|
|
1.04.057.014 I slot print_timing: id 3 | task 1298 | total time = 894.56 ms / 50 tokens
|
||
|
|
1.04.057.015 I slot print_timing: id 3 | task 1298 | graphs reused = 1243
|
||
|
|
1.04.057.063 I slot release: id 3 | task 1298 | stop processing: n_tokens = 95, truncated = 0
|
||
|
|
1.04.057.072 I srv update_slots: all slots are idle
|
||
|
|
1.04.067.970 I srv operator(): Chat format: peg-native
|
||
|
|
1.04.068.152 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.474
|
||
|
|
1.04.068.154 I srv get_availabl: updating prompt cache
|
||
|
|
1.04.068.171 W srv prompt_save: - saving prompt with length 95, total state size = 4.907 MiB (draft: 0.000 MiB)
|
||
|
|
1.04.071.454 I srv load: - looking for better prompt, base f_keep = 0.474, sim = 0.789
|
||
|
|
1.04.071.462 I srv update: - cache state: 15 prompts, 87.189 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
1.04.071.463 I srv update: - prompt 0x569a51df7a50: 125 tokens, checkpoints: 0, 6.456 MiB
|
||
|
|
1.04.071.463 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
1.04.071.464 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
1.04.071.464 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.04.071.464 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
1.04.071.465 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
1.04.071.465 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
1.04.071.465 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
1.04.071.466 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
1.04.071.466 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
1.04.071.467 I srv update: - prompt 0x569a58e83740: 113 tokens, checkpoints: 0, 5.837 MiB
|
||
|
|
1.04.071.471 I srv update: - prompt 0x569a552036c0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
1.04.071.471 I srv update: - prompt 0x569a51a3b720: 91 tokens, checkpoints: 0, 4.700 MiB
|
||
|
|
1.04.071.471 I srv update: - prompt 0x569a517d0820: 149 tokens, checkpoints: 0, 7.696 MiB
|
||
|
|
1.04.071.472 I srv update: - prompt 0x569a517c7ca0: 95 tokens, checkpoints: 0, 4.907 MiB
|
||
|
|
1.04.071.472 I srv get_availabl: prompt cache update took 3.32 ms
|
||
|
|
1.04.071.495 I slot launch_slot_: id 3 | task 1333 | processing task, is_child = 0
|
||
|
|
1.04.071.496 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.04.071.496 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.04.071.502 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.04.071.502 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.04.071.506 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.04.071.507 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.05.871.011 I slot print_timing: id 3 | task 1333 | prompt eval time = 138.26 ms / 12 tokens ( 11.52 ms per token, 86.79 tokens per second)
|
||
|
|
1.05.871.015 I slot print_timing: id 3 | task 1333 | eval time = 1661.23 ms / 76 tokens ( 21.86 ms per token, 45.75 tokens per second)
|
||
|
|
1.05.871.016 I slot print_timing: id 3 | task 1333 | total time = 1799.49 ms / 88 tokens
|
||
|
|
1.05.871.016 I slot print_timing: id 3 | task 1333 | graphs reused = 1317
|
||
|
|
1.05.871.064 I slot release: id 3 | task 1333 | stop processing: n_tokens = 132, truncated = 0
|
||
|
|
1.05.871.073 I srv update_slots: all slots are idle
|
||
|
|
1.05.881.641 I srv operator(): Chat format: peg-native
|
||
|
|
1.05.881.803 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.726 (> 0.100 thold), f_keep = 0.341
|
||
|
|
1.05.881.805 I srv get_availabl: updating prompt cache
|
||
|
|
1.05.881.825 W srv prompt_save: - saving prompt with length 132, total state size = 6.818 MiB (draft: 0.000 MiB)
|
||
|
|
1.05.888.145 I srv load: - looking for better prompt, base f_keep = 0.341, sim = 0.726
|
||
|
|
1.05.888.150 I srv load: - found better prompt with f_keep = 0.368, sim = 0.742
|
||
|
|
1.05.891.376 I srv update: - cache state: 15 prompts, 87.550 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
1.05.891.378 I srv update: - prompt 0x569a51a7e9b0: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
1.05.891.379 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
1.05.891.379 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.05.891.380 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
1.05.891.380 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
1.05.891.381 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
1.05.891.381 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
1.05.891.382 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
1.05.891.382 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
1.05.891.382 I srv update: - prompt 0x569a58e83740: 113 tokens, checkpoints: 0, 5.837 MiB
|
||
|
|
1.05.891.383 I srv update: - prompt 0x569a552036c0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
1.05.891.383 I srv update: - prompt 0x569a51a3b720: 91 tokens, checkpoints: 0, 4.700 MiB
|
||
|
|
1.05.891.384 I srv update: - prompt 0x569a517d0820: 149 tokens, checkpoints: 0, 7.696 MiB
|
||
|
|
1.05.891.384 I srv update: - prompt 0x569a517c7ca0: 95 tokens, checkpoints: 0, 4.907 MiB
|
||
|
|
1.05.891.385 I srv update: - prompt 0x569a51dd5030: 132 tokens, checkpoints: 0, 6.818 MiB
|
||
|
|
1.05.891.386 I srv get_availabl: prompt cache update took 9.58 ms
|
||
|
|
1.05.891.421 I slot launch_slot_: id 3 | task 1410 | processing task, is_child = 0
|
||
|
|
1.05.891.421 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.05.891.422 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.05.891.427 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.05.891.428 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.05.891.432 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.05.891.432 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.08.120.104 I slot print_timing: id 3 | task 1410 | prompt eval time = 149.93 ms / 16 tokens ( 9.37 ms per token, 106.72 tokens per second)
|
||
|
|
1.08.120.107 I slot print_timing: id 3 | task 1410 | eval time = 2078.72 ms / 95 tokens ( 21.88 ms per token, 45.70 tokens per second)
|
||
|
|
1.08.120.108 I slot print_timing: id 3 | task 1410 | total time = 2228.66 ms / 111 tokens
|
||
|
|
1.08.120.109 I slot print_timing: id 3 | task 1410 | graphs reused = 1410
|
||
|
|
1.08.120.142 I slot release: id 3 | task 1410 | stop processing: n_tokens = 156, truncated = 0
|
||
|
|
1.08.120.149 I srv update_slots: all slots are idle
|
||
|
|
1.08.130.756 I srv operator(): Chat format: peg-native
|
||
|
|
1.08.130.917 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.708 (> 0.100 thold), f_keep = 0.295
|
||
|
|
1.08.130.917 I srv get_availabl: updating prompt cache
|
||
|
|
1.08.130.940 W srv prompt_save: - saving prompt with length 156, total state size = 8.058 MiB (draft: 0.000 MiB)
|
||
|
|
1.08.138.488 I srv load: - looking for better prompt, base f_keep = 0.295, sim = 0.708
|
||
|
|
1.08.138.494 I srv load: - found better prompt with f_keep = 0.356, sim = 0.738
|
||
|
|
1.08.142.038 I srv update: - cache state: 15 prompts, 88.635 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
1.08.142.040 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
1.08.142.041 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.08.142.041 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
1.08.142.042 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
1.08.142.042 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
1.08.142.043 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
1.08.142.043 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
1.08.142.044 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
1.08.142.044 I srv update: - prompt 0x569a58e83740: 113 tokens, checkpoints: 0, 5.837 MiB
|
||
|
|
1.08.142.044 I srv update: - prompt 0x569a552036c0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
1.08.142.045 I srv update: - prompt 0x569a51a3b720: 91 tokens, checkpoints: 0, 4.700 MiB
|
||
|
|
1.08.142.045 I srv update: - prompt 0x569a517d0820: 149 tokens, checkpoints: 0, 7.696 MiB
|
||
|
|
1.08.142.045 I srv update: - prompt 0x569a517c7ca0: 95 tokens, checkpoints: 0, 4.907 MiB
|
||
|
|
1.08.142.046 I srv update: - prompt 0x569a51dd5030: 132 tokens, checkpoints: 0, 6.818 MiB
|
||
|
|
1.08.142.047 I srv update: - prompt 0x569a49824f50: 156 tokens, checkpoints: 0, 8.058 MiB
|
||
|
|
1.08.142.048 I srv get_availabl: prompt cache update took 11.13 ms
|
||
|
|
1.08.142.086 I slot launch_slot_: id 3 | task 1506 | processing task, is_child = 0
|
||
|
|
1.08.142.087 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.08.142.088 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.08.142.094 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.08.142.094 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.08.142.098 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.08.142.098 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.09.289.130 I slot print_timing: id 3 | task 1506 | prompt eval time = 190.34 ms / 17 tokens ( 11.20 ms per token, 89.31 tokens per second)
|
||
|
|
1.09.289.134 I slot print_timing: id 3 | task 1506 | eval time = 956.67 ms / 44 tokens ( 21.74 ms per token, 45.99 tokens per second)
|
||
|
|
1.09.289.134 I slot print_timing: id 3 | task 1506 | total time = 1147.02 ms / 61 tokens
|
||
|
|
1.09.289.135 I slot print_timing: id 3 | task 1506 | graphs reused = 1452
|
||
|
|
1.09.289.168 I slot release: id 3 | task 1506 | stop processing: n_tokens = 108, truncated = 0
|
||
|
|
1.09.289.176 I srv update_slots: all slots are idle
|
||
|
|
1.09.300.270 I srv operator(): Chat format: peg-native
|
||
|
|
1.09.300.433 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.417
|
||
|
|
1.09.300.434 I srv get_availabl: updating prompt cache
|
||
|
|
1.09.300.456 W srv prompt_save: - saving prompt with length 108, total state size = 5.578 MiB (draft: 0.000 MiB)
|
||
|
|
1.09.304.116 I srv load: - looking for better prompt, base f_keep = 0.417, sim = 0.789
|
||
|
|
1.09.304.122 I srv update: - cache state: 16 prompts, 94.213 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
1.09.304.122 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
1.09.304.123 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.09.304.123 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
1.09.304.124 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
1.09.304.124 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
1.09.304.125 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
1.09.304.125 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
1.09.304.125 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
1.09.304.126 I srv update: - prompt 0x569a58e83740: 113 tokens, checkpoints: 0, 5.837 MiB
|
||
|
|
1.09.304.126 I srv update: - prompt 0x569a552036c0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
1.09.304.127 I srv update: - prompt 0x569a51a3b720: 91 tokens, checkpoints: 0, 4.700 MiB
|
||
|
|
1.09.304.127 I srv update: - prompt 0x569a517d0820: 149 tokens, checkpoints: 0, 7.696 MiB
|
||
|
|
1.09.304.128 I srv update: - prompt 0x569a517c7ca0: 95 tokens, checkpoints: 0, 4.907 MiB
|
||
|
|
1.09.304.128 I srv update: - prompt 0x569a51dd5030: 132 tokens, checkpoints: 0, 6.818 MiB
|
||
|
|
1.09.304.129 I srv update: - prompt 0x569a49824f50: 156 tokens, checkpoints: 0, 8.058 MiB
|
||
|
|
1.09.304.129 I srv update: - prompt 0x569a58a3a2d0: 108 tokens, checkpoints: 0, 5.578 MiB
|
||
|
|
1.09.304.129 I srv get_availabl: prompt cache update took 3.69 ms
|
||
|
|
1.09.304.151 I slot launch_slot_: id 3 | task 1551 | processing task, is_child = 0
|
||
|
|
1.09.304.152 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.09.304.152 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.09.304.157 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.09.304.157 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.09.304.162 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.09.304.162 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.10.178.118 I slot print_timing: id 3 | task 1551 | prompt eval time = 136.97 ms / 12 tokens ( 11.41 ms per token, 87.61 tokens per second)
|
||
|
|
1.10.178.122 I slot print_timing: id 3 | task 1551 | eval time = 736.98 ms / 34 tokens ( 21.68 ms per token, 46.13 tokens per second)
|
||
|
|
1.10.178.123 I slot print_timing: id 3 | task 1551 | total time = 873.94 ms / 46 tokens
|
||
|
|
1.10.178.123 I slot print_timing: id 3 | task 1551 | graphs reused = 1484
|
||
|
|
1.10.178.157 I slot release: id 3 | task 1551 | stop processing: n_tokens = 90, truncated = 0
|
||
|
|
1.10.178.164 I srv update_slots: all slots are idle
|
||
|
|
1.10.188.706 I srv operator(): Chat format: peg-native
|
||
|
|
1.10.188.866 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.849 (> 0.100 thold), f_keep = 0.500
|
||
|
|
1.10.188.893 I slot launch_slot_: id 3 | task 1586 | processing task, is_child = 0
|
||
|
|
1.10.188.893 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.10.188.894 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.10.188.905 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.10.188.905 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.10.188.910 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.10.188.910 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.11.376.141 I slot print_timing: id 3 | task 1586 | prompt eval time = 75.85 ms / 8 tokens ( 9.48 ms per token, 105.47 tokens per second)
|
||
|
|
1.11.376.144 I slot print_timing: id 3 | task 1586 | eval time = 1111.37 ms / 51 tokens ( 21.79 ms per token, 45.89 tokens per second)
|
||
|
|
1.11.376.145 I slot print_timing: id 3 | task 1586 | total time = 1187.22 ms / 59 tokens
|
||
|
|
1.11.376.145 I slot print_timing: id 3 | task 1586 | graphs reused = 1533
|
||
|
|
1.11.376.179 I slot release: id 3 | task 1586 | stop processing: n_tokens = 103, truncated = 0
|
||
|
|
1.11.376.188 I srv update_slots: all slots are idle
|
||
|
|
1.11.386.530 I srv operator(): Chat format: peg-native
|
||
|
|
1.11.386.688 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.738 (> 0.100 thold), f_keep = 0.437
|
||
|
|
1.11.386.690 I srv get_availabl: updating prompt cache
|
||
|
|
1.11.386.704 W srv prompt_save: - saving prompt with length 103, total state size = 5.320 MiB (draft: 0.000 MiB)
|
||
|
|
1.11.390.374 I srv load: - looking for better prompt, base f_keep = 0.437, sim = 0.738
|
||
|
|
1.11.390.379 I srv update: - cache state: 17 prompts, 99.534 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
1.11.390.380 I srv update: - prompt 0x569a51ba1d90: 136 tokens, checkpoints: 0, 7.025 MiB
|
||
|
|
1.11.390.380 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.11.390.381 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
1.11.390.381 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
1.11.390.382 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
1.11.390.382 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
1.11.390.382 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
1.11.390.383 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
1.11.390.383 I srv update: - prompt 0x569a58e83740: 113 tokens, checkpoints: 0, 5.837 MiB
|
||
|
|
1.11.390.384 I srv update: - prompt 0x569a552036c0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
1.11.390.385 I srv update: - prompt 0x569a51a3b720: 91 tokens, checkpoints: 0, 4.700 MiB
|
||
|
|
1.11.390.385 I srv update: - prompt 0x569a517d0820: 149 tokens, checkpoints: 0, 7.696 MiB
|
||
|
|
1.11.390.386 I srv update: - prompt 0x569a517c7ca0: 95 tokens, checkpoints: 0, 4.907 MiB
|
||
|
|
1.11.390.386 I srv update: - prompt 0x569a51dd5030: 132 tokens, checkpoints: 0, 6.818 MiB
|
||
|
|
1.11.390.387 I srv update: - prompt 0x569a49824f50: 156 tokens, checkpoints: 0, 8.058 MiB
|
||
|
|
1.11.390.387 I srv update: - prompt 0x569a58a3a2d0: 108 tokens, checkpoints: 0, 5.578 MiB
|
||
|
|
1.11.390.387 I srv update: - prompt 0x569a51a7e9b0: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.11.390.388 I srv get_availabl: prompt cache update took 3.70 ms
|
||
|
|
1.11.390.414 I slot launch_slot_: id 3 | task 1638 | processing task, is_child = 0
|
||
|
|
1.11.390.414 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.11.390.414 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.11.390.420 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.11.390.420 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.11.390.424 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.11.390.425 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.13.186.484 I slot print_timing: id 3 | task 1638 | prompt eval time = 154.52 ms / 16 tokens ( 9.66 ms per token, 103.55 tokens per second)
|
||
|
|
1.13.186.487 I slot print_timing: id 3 | task 1638 | eval time = 1641.53 ms / 75 tokens ( 21.89 ms per token, 45.69 tokens per second)
|
||
|
|
1.13.186.488 I slot print_timing: id 3 | task 1638 | total time = 1796.04 ms / 91 tokens
|
||
|
|
1.13.186.488 I slot print_timing: id 3 | task 1638 | graphs reused = 1606
|
||
|
|
1.13.186.523 I slot release: id 3 | task 1638 | stop processing: n_tokens = 135, truncated = 0
|
||
|
|
1.13.186.531 I srv update_slots: all slots are idle
|
||
|
|
1.13.197.044 I srv operator(): Chat format: peg-native
|
||
|
|
1.13.197.206 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.789 (> 0.100 thold), f_keep = 0.333
|
||
|
|
1.13.197.206 I srv get_availabl: updating prompt cache
|
||
|
|
1.13.197.224 W srv prompt_save: - saving prompt with length 135, total state size = 6.973 MiB (draft: 0.000 MiB)
|
||
|
|
1.13.203.679 I srv load: - looking for better prompt, base f_keep = 0.333, sim = 0.789
|
||
|
|
1.13.203.684 I srv load: - found better prompt with f_keep = 0.353, sim = 0.842
|
||
|
|
1.13.206.987 I srv update: - cache state: 17 prompts, 99.482 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
1.13.206.989 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.13.206.990 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
1.13.206.990 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
1.13.206.990 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
1.13.206.991 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
1.13.206.991 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
1.13.206.992 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
1.13.206.992 I srv update: - prompt 0x569a58e83740: 113 tokens, checkpoints: 0, 5.837 MiB
|
||
|
|
1.13.206.992 I srv update: - prompt 0x569a552036c0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
1.13.206.993 I srv update: - prompt 0x569a51a3b720: 91 tokens, checkpoints: 0, 4.700 MiB
|
||
|
|
1.13.206.993 I srv update: - prompt 0x569a517d0820: 149 tokens, checkpoints: 0, 7.696 MiB
|
||
|
|
1.13.206.993 I srv update: - prompt 0x569a517c7ca0: 95 tokens, checkpoints: 0, 4.907 MiB
|
||
|
|
1.13.206.994 I srv update: - prompt 0x569a51dd5030: 132 tokens, checkpoints: 0, 6.818 MiB
|
||
|
|
1.13.206.994 I srv update: - prompt 0x569a49824f50: 156 tokens, checkpoints: 0, 8.058 MiB
|
||
|
|
1.13.206.995 I srv update: - prompt 0x569a58a3a2d0: 108 tokens, checkpoints: 0, 5.578 MiB
|
||
|
|
1.13.206.996 I srv update: - prompt 0x569a51a7e9b0: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.13.206.996 I srv update: - prompt 0x569a51bb0720: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
1.13.206.997 I srv get_availabl: prompt cache update took 9.79 ms
|
||
|
|
1.13.207.039 I slot launch_slot_: id 3 | task 1714 | processing task, is_child = 0
|
||
|
|
1.13.207.039 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.13.207.040 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.13.207.045 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.13.207.046 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.13.207.050 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.13.207.050 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.14.554.084 I slot print_timing: id 3 | task 1714 | prompt eval time = 125.63 ms / 9 tokens ( 13.96 ms per token, 71.64 tokens per second)
|
||
|
|
1.14.554.088 I slot print_timing: id 3 | task 1714 | eval time = 1221.38 ms / 56 tokens ( 21.81 ms per token, 45.85 tokens per second)
|
||
|
|
1.14.554.089 I slot print_timing: id 3 | task 1714 | total time = 1347.02 ms / 65 tokens
|
||
|
|
1.14.554.089 I slot print_timing: id 3 | task 1714 | graphs reused = 1660
|
||
|
|
1.14.554.125 I slot release: id 3 | task 1714 | stop processing: n_tokens = 112, truncated = 0
|
||
|
|
1.14.554.134 I srv update_slots: all slots are idle
|
||
|
|
1.14.564.711 I srv operator(): Chat format: peg-native
|
||
|
|
1.14.564.875 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.821 (> 0.100 thold), f_keep = 0.411
|
||
|
|
1.14.564.877 I srv get_availabl: updating prompt cache
|
||
|
|
1.14.564.894 W srv prompt_save: - saving prompt with length 112, total state size = 5.785 MiB (draft: 0.000 MiB)
|
||
|
|
1.14.568.502 I srv load: - looking for better prompt, base f_keep = 0.411, sim = 0.821
|
||
|
|
1.14.568.508 I srv update: - cache state: 18 prompts, 105.267 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
1.14.568.509 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.14.568.510 I srv update: - prompt 0x569a51d226e0: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
1.14.568.510 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
1.14.568.511 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
1.14.568.512 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
1.14.568.512 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
1.14.568.513 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
1.14.568.514 I srv update: - prompt 0x569a58e83740: 113 tokens, checkpoints: 0, 5.837 MiB
|
||
|
|
1.14.568.515 I srv update: - prompt 0x569a552036c0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
1.14.568.515 I srv update: - prompt 0x569a51a3b720: 91 tokens, checkpoints: 0, 4.700 MiB
|
||
|
|
1.14.568.515 I srv update: - prompt 0x569a517d0820: 149 tokens, checkpoints: 0, 7.696 MiB
|
||
|
|
1.14.568.516 I srv update: - prompt 0x569a517c7ca0: 95 tokens, checkpoints: 0, 4.907 MiB
|
||
|
|
1.14.568.516 I srv update: - prompt 0x569a51dd5030: 132 tokens, checkpoints: 0, 6.818 MiB
|
||
|
|
1.14.568.517 I srv update: - prompt 0x569a49824f50: 156 tokens, checkpoints: 0, 8.058 MiB
|
||
|
|
1.14.568.517 I srv update: - prompt 0x569a58a3a2d0: 108 tokens, checkpoints: 0, 5.578 MiB
|
||
|
|
1.14.568.517 I srv update: - prompt 0x569a51a7e9b0: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.14.568.518 I srv update: - prompt 0x569a51bb0720: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
1.14.568.518 I srv update: - prompt 0x569a51b8bb80: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
1.14.568.519 I srv get_availabl: prompt cache update took 3.64 ms
|
||
|
|
1.14.568.540 I slot launch_slot_: id 3 | task 1771 | processing task, is_child = 0
|
||
|
|
1.14.568.541 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.14.568.541 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.14.568.546 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.14.568.547 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.14.568.551 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.14.568.551 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.16.381.503 I slot print_timing: id 3 | task 1771 | prompt eval time = 129.68 ms / 10 tokens ( 12.97 ms per token, 77.11 tokens per second)
|
||
|
|
1.16.381.507 I slot print_timing: id 3 | task 1771 | eval time = 1683.26 ms / 77 tokens ( 21.86 ms per token, 45.74 tokens per second)
|
||
|
|
1.16.381.507 I slot print_timing: id 3 | task 1771 | total time = 1812.94 ms / 87 tokens
|
||
|
|
1.16.381.508 I slot print_timing: id 3 | task 1771 | graphs reused = 1735
|
||
|
|
1.16.381.544 I slot release: id 3 | task 1771 | stop processing: n_tokens = 132, truncated = 0
|
||
|
|
1.16.381.551 I srv update_slots: all slots are idle
|
||
|
|
1.16.392.181 I srv operator(): Chat format: peg-native
|
||
|
|
1.16.392.350 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.763 (> 0.100 thold), f_keep = 0.341
|
||
|
|
1.16.392.352 I srv get_availabl: updating prompt cache
|
||
|
|
1.16.392.368 W srv prompt_save: - saving prompt with length 132, total state size = 6.818 MiB (draft: 0.000 MiB)
|
||
|
|
1.16.398.661 I srv load: - looking for better prompt, base f_keep = 0.341, sim = 0.763
|
||
|
|
1.16.398.665 I srv load: - found better prompt with f_keep = 0.411, sim = 0.780
|
||
|
|
1.16.401.599 I srv update: - cache state: 18 prompts, 106.300 MiB (limits: 8192.000 MiB, 8192 tokens, 158599 est)
|
||
|
|
1.16.401.600 I srv update: - prompt 0x569a51d2a280: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.16.401.601 I srv update: - prompt 0x569a51aa0c80: 121 tokens, checkpoints: 0, 6.250 MiB
|
||
|
|
1.16.401.601 I srv update: - prompt 0x569a517cd690: 110 tokens, checkpoints: 0, 5.682 MiB
|
||
|
|
1.16.401.602 I srv update: - prompt 0x569a58a3bdb0: 100 tokens, checkpoints: 0, 5.165 MiB
|
||
|
|
1.16.401.602 I srv update: - prompt 0x569a58a37930: 99 tokens, checkpoints: 0, 5.114 MiB
|
||
|
|
1.16.401.603 I srv update: - prompt 0x569a51b8e980: 102 tokens, checkpoints: 0, 5.269 MiB
|
||
|
|
1.16.401.603 I srv update: - prompt 0x569a58e83740: 113 tokens, checkpoints: 0, 5.837 MiB
|
||
|
|
1.16.401.603 I srv update: - prompt 0x569a552036c0: 97 tokens, checkpoints: 0, 5.010 MiB
|
||
|
|
1.16.401.604 I srv update: - prompt 0x569a51a3b720: 91 tokens, checkpoints: 0, 4.700 MiB
|
||
|
|
1.16.401.604 I srv update: - prompt 0x569a517d0820: 149 tokens, checkpoints: 0, 7.696 MiB
|
||
|
|
1.16.401.605 I srv update: - prompt 0x569a517c7ca0: 95 tokens, checkpoints: 0, 4.907 MiB
|
||
|
|
1.16.401.605 I srv update: - prompt 0x569a51dd5030: 132 tokens, checkpoints: 0, 6.818 MiB
|
||
|
|
1.16.401.606 I srv update: - prompt 0x569a49824f50: 156 tokens, checkpoints: 0, 8.058 MiB
|
||
|
|
1.16.401.606 I srv update: - prompt 0x569a58a3a2d0: 108 tokens, checkpoints: 0, 5.578 MiB
|
||
|
|
1.16.401.607 I srv update: - prompt 0x569a51a7e9b0: 103 tokens, checkpoints: 0, 5.320 MiB
|
||
|
|
1.16.401.607 I srv update: - prompt 0x569a51bb0720: 135 tokens, checkpoints: 0, 6.973 MiB
|
||
|
|
1.16.401.608 I srv update: - prompt 0x569a51b8bb80: 112 tokens, checkpoints: 0, 5.785 MiB
|
||
|
|
1.16.401.608 I srv update: - prompt 0x569a51d42770: 132 tokens, checkpoints: 0, 6.818 MiB
|
||
|
|
1.16.401.609 I srv get_availabl: prompt cache update took 9.26 ms
|
||
|
|
1.16.401.642 I slot launch_slot_: id 3 | task 1849 | processing task, is_child = 0
|
||
|
|
1.16.401.643 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.16.401.643 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.16.401.648 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.16.401.649 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.16.401.653 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.16.401.653 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.17.163.612 I slot print_timing: id 3 | task 1849 | prompt eval time = 134.39 ms / 13 tokens ( 10.34 ms per token, 96.73 tokens per second)
|
||
|
|
1.17.163.616 I slot print_timing: id 3 | task 1849 | eval time = 627.55 ms / 29 tokens ( 21.64 ms per token, 46.21 tokens per second)
|
||
|
|
1.17.163.616 I slot print_timing: id 3 | task 1849 | total time = 761.94 ms / 42 tokens
|
||
|
|
1.17.163.617 I slot print_timing: id 3 | task 1849 | graphs reused = 1762
|
||
|
|
1.17.163.653 I slot release: id 3 | task 1849 | stop processing: n_tokens = 87, truncated = 0
|
||
|
|
1.17.163.662 I srv update_slots: all slots are idle
|
||
|
|
1.17.174.158 I srv operator(): Chat format: peg-native
|
||
|
|
1.17.174.325 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.818 (> 0.100 thold), f_keep = 0.517
|
||
|
|
1.17.174.347 I slot launch_slot_: id 3 | task 1879 | processing task, is_child = 0
|
||
|
|
1.17.174.348 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.17.174.348 I slot prompt_clear: id 0 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.17.174.354 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.17.174.354 I slot prompt_clear: id 1 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.17.174.358 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
|
||
|
|
1.17.174.359 I slot prompt_clear: id 2 | task -1 | clearing prompt with 0 tokens
|
||
|
|
1.18.967.504 I slot print_timing: id 3 | task 1879 | prompt eval time = 132.80 ms / 10 tokens ( 13.28 ms per token, 75.30 tokens per second)
|
||
|
|
1.18.967.507 I slot print_timing: id 3 | task 1879 | eval time = 1660.34 ms / 76 tokens ( 21.85 ms per token, 45.77 tokens per second)
|
||
|
|
1.18.967.508 I slot print_timing: id 3 | task 1879 | total time = 1793.13 ms / 86 tokens
|
||
|
|
1.18.967.509 I slot print_timing: id 3 | task 1879 | graphs reused = 1836
|
||
|
|
1.18.967.544 I slot release: id 3 | task 1879 | stop processing: n_tokens = 130, truncated = 0
|
||
|
|
1.18.967.552 I srv update_slots: all slots are idle
|
||
|
|
1.18.968.011 I srv operator(): operator(): cleaning up before exit...
|