experiments-07-07-12-local-llm-knowledge-2/POWER_ANALYSIS_V2.md

Dual-MI50 Power × Context Sweep — V2 Audit

Recommendation

Do not set a production power cap from this CSV. The v2 sweep is still invalid. The file records llama-bench's stddev_ts field—not mean throughput—because the harness extracts the final CSV column. In llama-bench CSV, the final columns are:

..., avg_ns, stddev_ns, avg_ts, stddev_ts

The harness uses $NF, so the reported pp_mean≈1–8 and tg_mean≈0.2–0.4 are standard deviations in tok/s. They are not throughput. This exactly explains why they are orders of magnitude below the known sane values (~470 pp512 and ~60 tg) and why their context/power shape is physically incoherent.

Until the extraction is fixed and the sweep rerun, retain the existing known-safe cap—225 W—rather than interpreting these numbers as evidence for 200/180/160 W. No chat-versus-RAG split can be supported by this dataset.

Source audited: mi50_pxc_v2_20260712_225435.csv (most recent).

Grid completeness

The intended grid was 4 powers × 3 contexts = 12 cells. The CSV contains only 9 cells:

• Present: 225, 200, and 180 W × 512, 8192, and 32768 tokens.
• Missing: all three 160 W cells.

The script's 90-minute budget stopped the run before 160 W. Even with correct extraction, this file could not establish a four-level optimum.

Recorded pivot tables

These tables reproduce the requested CSV fields, but the values are misidentified stddev_ts, not mean tok/s.

Recorded pp_mean (invalid as throughput)

Set cap	512	8,192	32,768
225 W	3.04	1.53	7.51
200 W	3.20	1.58	7.71
180 W	3.68	3.45	5.81
160 W	—	—	—

Recorded tg_mean (invalid as throughput)

Set cap	512	8,192	32,768
225 W	0.37	0.17	0.24
200 W	0.31	0.29	0.24
180 W	0.32	0.27	0.17
160 W	—	—	—

The physically impossible indicators requested in the prompt are present:

• pp_mean at 32K is 2.5–4.9× the 512/8K values despite being reported as throughput.
• Decode is ~0.2–0.4 tok/s, roughly two orders of magnitude below the expected ~60 tok/s.
• Prefill is ~1.5–7.7 tok/s, roughly two orders of magnitude below the expected ~470 tok/s at pp512.

These are not subtle noisy cells; the entire throughput dataset is the wrong metric.

Spread audit

Flag criterion: (max-min)/mean > 10%. Because each outer repetition captured llama-bench's internal throughput standard deviation, these spreads describe variation in standard deviation, not variation in actual mean throughput. They are included only to show that the file is additionally noisy.

Cap	Context	PP recorded range	PP spread	TG recorded range	TG spread	Flag
225 W	512	2.26–3.82	51.3%	0.23–0.51	75.7%	PP + TG noisy
225 W	8,192	1.48–1.58	6.5%	0.10–0.24	82.4%	TG noisy
225 W	32,768	7.46–7.56	1.3%	0.20–0.28	33.3%	TG noisy
200 W	512	3.20–3.20	0.0%	0.29–0.33	12.9%	TG noisy
200 W	8,192	1.41–1.75	21.5%	0.26–0.33	24.1%	PP + TG noisy
200 W	32,768	7.69–7.74	0.6%	0.21–0.27	25.0%	TG noisy
180 W	512	3.51–3.84	9.0%	0.30–0.33	9.4%	Tight by threshold
180 W	8,192	3.44–3.45	0.3%	0.16–0.38	81.5%	TG noisy
180 W	32,768	5.34–6.29	16.4%	0.14–0.19	29.4%	PP + TG noisy

Two outer reps are also too few for reliable distribution estimates, particularly when the underlying field is already the wrong one.

Percent-retention audit

True pp/tg retention versus 225 W cannot be computed from this file. Dividing standard deviations by baseline standard deviations does not measure throughput retention.

For transparency, the ratios one would obtain from the mislabeled fields are shown below and explicitly rejected:

Ratio of recorded PP-standard-deviation fields to 225 W

Cap	512	8,192	32,768
225 W	100.0%	100.0%	100.0%
200 W	105.3%	103.3%	102.7%
180 W	121.1%	225.5%	77.4%
160 W	—	—	—

Ratio of recorded TG-standard-deviation fields to 225 W

Cap	512	8,192	32,768
225 W	100.0%	100.0%	100.0%
200 W	83.8%	170.6%	100.0%
180 W	86.5%	158.8%	70.8%
160 W	—	—	—

Applying <95% or <90% throughput flags to these ratios would be misleading, so no retention verdict is assigned.

Hypothesis test

Not testable with this CSV.

• The recorded TG values do not tell whether decode remains flat with lower caps; they measure run-to-run/internal dispersion, not decode rate.
• The recorded PP values do not tell whether low-power prefill degradation worsens at 32K; they measure dispersion, not prefill rate.
• Therefore the operator's exact question—“is the optimal low-power config useless at the largest context?”—has no defensible answer from v2.

The apparent 180 W “retention” at 32K must not be interpreted. A lower standard deviation can coexist with much lower, equal, or higher mean throughput.

Thermal read

Set cap	512	8,192	32,768
225 W	39°C	42°C	44°C
200 W	40°C	43°C	44°C
180 W	40°C	43°C	47°C
160 W	—	—	—

• Maximum edge temperature was 47°C; no 225 W cell exceeded 44°C.
• There is no evidence of edge-temperature throttling at 225 W.
• Lower caps being as warm or warmer is consistent with sequential run order/heat soak and does not indicate that lower power creates more heat.
• These temperatures are credible, but they cannot rescue the invalid throughput extraction.

Root cause in the harness

Current code in better_power_test.sh:

v=$(echo "$out" | grep ... | tail -1 \
      | awk -F',' '{gsub(/"/,"",$NF); print $NF}')

llama-bench documents the CSV suffix as:

..., avg_ns, stddev_ns, avg_ts, stddev_ts

Therefore:

• $NF = stddev_ts — what v2 captured.
• $(NF-1) = avg_ts — the requested mean throughput.

Minimal correction:

extract_toks() {
  local out="$1" tag="$2"
  echo "$out" \
    | grep -iE "(^|[\",])${tag}([\",]|$)" \
    | tail -1 \
    | awk -F',' '{gsub(/"/,"",$(NF-1)); print $(NF-1)}'
}

Safer correction: request JSON/JSONL and extract the named avg_ts field rather than relying on column position. Also retain llama-bench's stddev_ts separately as the internal variance metric instead of calling it an outer-rep throughput sample.

Rerun requirements

1. Extract avg_ts, not stddev_ts.
2. Add a sanity gate before accepting a rep, e.g. pp512 >100 tok/s and TG >10 tok/s for this known configuration.
3. Preserve raw llama-bench output for at least one rep per cell so schema errors are auditable.
4. Complete all 12 cells, including 160 W, or explicitly mark the sweep partial.
5. Use at least 3 outer reps if wall-clock permits; two reps cannot characterize noise well.
6. Randomize/alternate cap order or perform a steady-state soak to reduce thermal-order confounding.
7. Record both avg_ts and stddev_ts; calculate retention from avg_ts only.

Final answer

• Is v2 trustworthy? No. Power capping and temperatures appear valid, but throughput extraction is wrong.
• Does decode stay flat as power falls? Unknown.
• Does large-context prefill degrade more? Unknown.
• Is a low-power optimum useless at 32K? Unknown; the current data cannot answer it.
• What cap should be set from this sweep? None. Keep 225 W as the conservative existing setting until a corrected v3 sweep captures avg_ts.

The prior invalidation was attributed to -ngl 999; v2 corrected that flag but introduced/retained a separate CSV-field bug. The sane throughput likely existed in the penultimate field of every raw llama-bench row, but it was discarded when only $NF was written to the summary CSV.