experiments-07-07-12-local-llm-knowledge-2/POWER_ANALYSIS_V2.md

# Dual-MI50 Power × Context Sweep — V2 Audit

## Recommendation

**Do not set a production power cap from this CSV. The v2 sweep is still invalid.** The file records llama-bench's `stddev_ts` field—not mean throughput—because the harness extracts the final CSV column. In llama-bench CSV, the final columns are:

```text
..., avg_ns, stddev_ns, avg_ts, stddev_ts
```

The harness uses `$NF`, so the reported `pp_mean≈1–8` and `tg_mean≈0.2–0.4` are standard deviations in tok/s. They are not throughput. This exactly explains why they are orders of magnitude below the known sane values (~470 pp512 and ~60 tg) and why their context/power shape is physically incoherent.

Until the extraction is fixed and the sweep rerun, retain the existing known-safe cap—**225 W**—rather than interpreting these numbers as evidence for 200/180/160 W. No chat-versus-RAG split can be supported by this dataset.

Source audited: `mi50_pxc_v2_20260712_225435.csv` (most recent).

## Grid completeness

The intended grid was 4 powers × 3 contexts = 12 cells. The CSV contains only **9 cells**:

- Present: 225, 200, and 180 W × 512, 8192, and 32768 tokens.
- Missing: **all three 160 W cells**.

The script's 90-minute budget stopped the run before 160 W. Even with correct extraction, this file could not establish a four-level optimum.

## Recorded pivot tables

These tables reproduce the requested CSV fields, but the values are **misidentified `stddev_ts`, not mean tok/s**.

### Recorded `pp_mean` (invalid as throughput)

| Set cap | 512 | 8,192 | 32,768 |
|---:|---:|---:|---:|
| 225 W | 3.04 | 1.53 | 7.51 |
| 200 W | 3.20 | 1.58 | 7.71 |
| 180 W | 3.68 | 3.45 | 5.81 |
| 160 W | — | — | — |

### Recorded `tg_mean` (invalid as throughput)

| Set cap | 512 | 8,192 | 32,768 |
|---:|---:|---:|---:|
| 225 W | 0.37 | 0.17 | 0.24 |
| 200 W | 0.31 | 0.29 | 0.24 |
| 180 W | 0.32 | 0.27 | 0.17 |
| 160 W | — | — | — |

The physically impossible indicators requested in the prompt are present:

- `pp_mean` at 32K is 2.5–4.9× the 512/8K values despite being reported as throughput.
- Decode is ~0.2–0.4 tok/s, roughly two orders of magnitude below the expected ~60 tok/s.
- Prefill is ~1.5–7.7 tok/s, roughly two orders of magnitude below the expected ~470 tok/s at pp512.

These are not subtle noisy cells; the entire throughput dataset is the wrong metric.

## Spread audit

Flag criterion: `(max-min)/mean > 10%`. Because each outer repetition captured llama-bench's internal throughput standard deviation, these spreads describe variation in **standard deviation**, not variation in actual mean throughput. They are included only to show that the file is additionally noisy.

| Cap | Context | PP recorded range | PP spread | TG recorded range | TG spread | Flag |
|---:|---:|---:|---:|---:|---:|---|
| 225 W | 512 | 2.26–3.82 | 51.3% | 0.23–0.51 | 75.7% | PP + TG noisy |
| 225 W | 8,192 | 1.48–1.58 | 6.5% | 0.10–0.24 | 82.4% | TG noisy |
| 225 W | 32,768 | 7.46–7.56 | 1.3% | 0.20–0.28 | 33.3% | TG noisy |
| 200 W | 512 | 3.20–3.20 | 0.0% | 0.29–0.33 | 12.9% | TG noisy |
| 200 W | 8,192 | 1.41–1.75 | 21.5% | 0.26–0.33 | 24.1% | PP + TG noisy |
| 200 W | 32,768 | 7.69–7.74 | 0.6% | 0.21–0.27 | 25.0% | TG noisy |
| 180 W | 512 | 3.51–3.84 | 9.0% | 0.30–0.33 | 9.4% | Tight by threshold |
| 180 W | 8,192 | 3.44–3.45 | 0.3% | 0.16–0.38 | 81.5% | TG noisy |
| 180 W | 32,768 | 5.34–6.29 | 16.4% | 0.14–0.19 | 29.4% | PP + TG noisy |

Two outer reps are also too few for reliable distribution estimates, particularly when the underlying field is already the wrong one.

## Percent-retention audit

True pp/tg retention versus 225 W **cannot be computed** from this file. Dividing standard deviations by baseline standard deviations does not measure throughput retention.

For transparency, the ratios one would obtain from the mislabeled fields are shown below and explicitly rejected:

### Ratio of recorded PP-standard-deviation fields to 225 W

| Cap | 512 | 8,192 | 32,768 |
|---:|---:|---:|---:|
| 225 W | 100.0% | 100.0% | 100.0% |
| 200 W | 105.3% | 103.3% | 102.7% |
| 180 W | 121.1% | 225.5% | 77.4% |
| 160 W | — | — | — |

### Ratio of recorded TG-standard-deviation fields to 225 W

| Cap | 512 | 8,192 | 32,768 |
|---:|---:|---:|---:|
| 225 W | 100.0% | 100.0% | 100.0% |
| 200 W | 83.8% | 170.6% | 100.0% |
| 180 W | 86.5% | 158.8% | 70.8% |
| 160 W | — | — | — |

Applying <95% or <90% throughput flags to these ratios would be misleading, so no retention verdict is assigned.

## Hypothesis test

**Not testable with this CSV.**

- The recorded TG values do not tell whether decode remains flat with lower caps; they measure run-to-run/internal dispersion, not decode rate.
- The recorded PP values do not tell whether low-power prefill degradation worsens at 32K; they measure dispersion, not prefill rate.
- Therefore the operator's exact question—“is the optimal low-power config useless at the largest context?”—has no defensible answer from v2.

The apparent 180 W “retention” at 32K must not be interpreted. A lower standard deviation can coexist with much lower, equal, or higher mean throughput.

## Thermal read

| Set cap | 512 | 8,192 | 32,768 |
|---:|---:|---:|---:|
| 225 W | 39°C | 42°C | 44°C |
| 200 W | 40°C | 43°C | 44°C |
| 180 W | 40°C | 43°C | 47°C |
| 160 W | — | — | — |

- Maximum edge temperature was **47°C**; no 225 W cell exceeded 44°C.
- There is no evidence of edge-temperature throttling at 225 W.
- Lower caps being as warm or warmer is consistent with sequential run order/heat soak and does not indicate that lower power creates more heat.
- These temperatures are credible, but they cannot rescue the invalid throughput extraction.

## Root cause in the harness

Current code in `better_power_test.sh`:

```bash
v=$(echo "$out" | grep ... | tail -1 \
      | awk -F',' '{gsub(/"/,"",$NF); print $NF}')
```

llama-bench documents the CSV suffix as:

```text
..., avg_ns, stddev_ns, avg_ts, stddev_ts
```

Therefore:

- `$NF` = `stddev_ts` — what v2 captured.
- `$(NF-1)` = `avg_ts` — the requested mean throughput.

Minimal correction:

```bash
extract_toks() {
  local out="$1" tag="$2"
  echo "$out" \
    | grep -iE "(^|[\",])${tag}([\",]|$)" \
    | tail -1 \
    | awk -F',' '{gsub(/"/,"",$(NF-1)); print $(NF-1)}'
}
```

Safer correction: request JSON/JSONL and extract the named `avg_ts` field rather than relying on column position. Also retain llama-bench's `stddev_ts` separately as the **internal** variance metric instead of calling it an outer-rep throughput sample.

## Rerun requirements

1. Extract `avg_ts`, not `stddev_ts`.
2. Add a sanity gate before accepting a rep, e.g. pp512 >100 tok/s and TG >10 tok/s for this known configuration.
3. Preserve raw llama-bench output for at least one rep per cell so schema errors are auditable.
4. Complete all 12 cells, including 160 W, or explicitly mark the sweep partial.
5. Use at least 3 outer reps if wall-clock permits; two reps cannot characterize noise well.
6. Randomize/alternate cap order or perform a steady-state soak to reduce thermal-order confounding.
7. Record both `avg_ts` and `stddev_ts`; calculate retention from `avg_ts` only.

## Final answer

- **Is v2 trustworthy?** No. Power capping and temperatures appear valid, but throughput extraction is wrong.
- **Does decode stay flat as power falls?** Unknown.
- **Does large-context prefill degrade more?** Unknown.
- **Is a low-power optimum useless at 32K?** Unknown; the current data cannot answer it.
- **What cap should be set from this sweep?** None. Keep **225 W** as the conservative existing setting until a corrected v3 sweep captures `avg_ts`.

The prior invalidation was attributed to `-ngl 999`; v2 corrected that flag but introduced/retained a separate CSV-field bug. The sane throughput likely existed in the penultimate field of every raw llama-bench row, but it was discarded when only `$NF` was written to the summary CSV.