experiments-07-07-12-local-llm-knowledge-2/SCORES.md

# Local-LLM Knowledge Benchmark Scores

Scoring follows the requested 1.0 / 0.5 / 0.0 rubric. No model log contained `FAILED TO LOAD`, so there are no DNFs. “Said” below is a concise faithful summary of the emitted answer; runaway reasoning tails were graded on the factual answer actually present in the log.

## Ranking

| Rank | Model | Score /40 | Verdict |
|---:|---|---:|---|
| 1 | gemma4-31b | 40.0 | Flawless |
| 2= | gpt-oss-120b | 39.5 | Excellent |
| 2= | nemotron3-super-120b | 39.5 | Excellent |
| 2= | north-mini-code | 39.5 | Excellent |
| 2= | qwen3next-80b-a3b | 39.5 | Excellent |
| 6= | gemma4-26b-a4b | 39.0 | Excellent |
| 6= | glm4.6v-iq2 | 39.0 | Excellent |
| 8 | devstral-small2-24b | 38.5 | Strong |
| 9= | devstral2-123b-iq2 | 38.0 | Strong |
| 9= | ministral3-14b | 38.0 | Strong |
| 9= | qwen3.6-27b | 38.0 | Strong |
| 12 | qwen3.6-35b-a3b | 37.5 | Strong |
| 13= | gpt-oss-20b | 36.5 | Mixed |
| 13= | minimax-m2-iq2 | 36.5 | Mixed |
| 13= | nemotron-cascade-30b | 36.5 | Mixed |
| 16 | nemotron3-nano-30b | 36.0 | Mixed |
| 17 | granite4-micro | 35.5 | Efficient |
| 18 | glm4.7-flash | 33.5 | Uneven |
| 19 | aya-23-8b | 26.5 | Weak |

## Per-subject breakdown

| Model | physics | chemistry | biology | history | geography | literature | music | rf_engineering | math | econ_law |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| gemma4-31b | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 |
| gpt-oss-120b | 3.5 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 |
| nemotron3-super-120b | 3.5 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 |
| north-mini-code | 3.5 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 |
| qwen3next-80b-a3b | 3.5 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 |
| gemma4-26b-a4b | 3.5 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 3.5 |
| glm4.6v-iq2 | 3.5 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 3.5 |
| devstral-small2-24b | 3.5 | 4 | 4 | 4 | 4 | 4 | 4 | 3 | 4 | 4 |
| devstral2-123b-iq2 | 3.5 | 3.5 | 4 | 3.5 | 4 | 4 | 3.5 | 4 | 4 | 4 |
| ministral3-14b | 3.5 | 4 | 4 | 4 | 4 | 4 | 4 | 3 | 3.5 | 4 |
| qwen3.6-27b | 3.5 | 4 | 3.5 | 4 | 4 | 4 | 4 | 3 | 4 | 4 |
| qwen3.6-35b-a3b | 3.5 | 4 | 4 | 4 | 3.5 | 4 | 4 | 3 | 4 | 3.5 |
| gpt-oss-20b | 3.5 | 4 | 4 | 4 | 4 | 3.5 | 4 | 3 | 4 | 2.5 |
| minimax-m2-iq2 | 3.5 | 3 | 4 | 4 | 4 | 4 | 3 | 4 | 3 | 4 |
| nemotron-cascade-30b | 3.5 | 4 | 4 | 4 | 4 | 4 | 3 | 2.5 | 4 | 3.5 |
| nemotron3-nano-30b | 4 | 4 | 3.5 | 4 | 4 | 3 | 3 | 2.5 | 4 | 4 |
| granite4-micro | 4 | 4 | 3.5 | 4 | 4 | 3.5 | 3 | 2.5 | 4 | 3 |
| glm4.7-flash | 4 | 3 | 4 | 4 | 3 | 3 | 2.5 | 3 | 4 | 3 |
| aya-23-8b | 3 | 3.5 | 2 | 4 | 3 | 3 | 3 | 0 | 1 | 4 |

## Every answer scored below 1.0

### aya-23-8b

- **Q01 — 0.0.** Said about 1.4 solar masses, but described a generic stellar core/fusion limit and neutron-star/black-hole remnant. Correct: about 1.4–1.44 solar masses, the maximum electron-degeneracy-supported white-dwarf mass; the classic accreting-white-dwarf outcome is a Type Ia supernova.
- **Q07 — 0.5.** Correctly named Haber–Bosch and iron, but called it an “iron and aluminum” catalyst. Correct: an iron-based catalyst, commonly with oxide promoters such as K₂O and Al₂O₃; aluminum is not the co-catalyst metal.
- **Q09 — 0.0.** Said the phrenic nerve was cranial and arose at T1–T5. Correct: phrenic nerve from C3–C5.
- **Q10 — 0.0.** Said its primary role was reabsorbing glucose and water. Correct: countercurrent multiplication establishing the medullary osmotic gradient, enabling urine concentration; glucose is reabsorbed chiefly in the proximal tubule.
- **Q17 — 0.0.** Named Baikal but gave 1,642 feet / 500 m. Correct: Lake Baikal, about 1,642 metres (5,387 ft).
- **Q22 — 0.0.** Said Virgil. Correct: Beatrice (with St Bernard taking over at the very end).
- **Q27 — 0.0.** Said “Not sure.” Correct: Rick Rubin produced; Daron Malakian co-produced.
- **Q29 — 0.0.** Said 35 dB. Correct: about 100 dB.
- **Q30 — 0.0.** Said a Smith chart plots two-port S-parameters. Correct: it plots complex reflection coefficient / normalized impedance or admittance and is used for transmission-line analysis and impedance matching.
- **Q31 — 0.0.** Omitted bandwidth and gave `C = log2(1+S/N)`. Correct: `C = B log2(1+S/N)`, with C capacity, B bandwidth, S signal power, and N noise power.
- **Q32 — 0.0.** Claimed a 150-chip PN code and “pulse-code modulation.” Correct: BPSK with DSSS using the 1023-chip C/A Gold code at 1.023 Mcps.
- **Q33 — 0.0.** Claimed ζ(2) lacked a simple value and supplied a purported divergent expansion. Correct: ζ(2) = π²/6.
- **Q35 — 0.0.** Gave an incorrect `(1+i)^n` equation. Correct: `e^(iπ) + 1 = 0`.
- **Q36 — 0.0.** Said two groups. Correct: five groups of order 8 up to isomorphism.

### devstral-small2-24b

- **Q01 — 0.5.** Correct limit and white-dwarf meaning, but added neutron-star formation as the classic over-limit outcome. Correct: about 1.4–1.44 M☉; for the classic accreting white dwarf, exceeding it triggers a Type Ia supernova.
- **Q29 — 0.0.** Said 92.45 dB even though its stated km/MHz formula evaluates to about 100.05 dB. Correct: about 100 dB.

### devstral2-123b-iq2

- **Q01 — 0.5.** Correctly gave 1.44 M☉ and the white-dwarf support limit, but presented neutron-star/black-hole formation as an alternative classic outcome. Correct classic case: Type Ia supernova.
- **Q06 — 0.5.** Guessed oxygen correctly but explicitly said “Not sure.” Under the requested rule, a correct uncertain guess receives 0.5.
- **Q16 — 0.5.** Correctly said Ming, but attributed the 1405–1433 expeditions to Yongle and Hongxi while omitting Xuande; the final voyage was under Xuande, while Hongxi suspended the voyages.
- **Q27 — 0.5.** Included required producer Rick Rubin, but incorrectly said David Bottrill co-produced. Correct: Rubin produced and Daron Malakian co-produced.

### gemma4-26b-a4b

- **Q01 — 0.5.** Correct white-dwarf limit and Type Ia outcome, but also added neutron-star formation as the standard alternative. Correct classic case: Type Ia supernova.
- **Q39 — 0.5.** Correctly named Daniel Kahneman but falsely said he shared the prize with Amos Tversky. Correct: Kahneman received it (with Vernon Smith receiving the other half for unrelated work); Tversky was deceased and ineligible.

### glm4.6v-iq2

- **Q01 — 0.5.** Correct limit/support mechanism, but said exceeding it leads to a neutron star or black hole and omitted the classic Type Ia result.
- **Q39 — 0.5.** Correctly named Kahneman but said the work/prize was “shared with Amos Tversky” before noting Tversky was ineligible. Correct: only Kahneman received the Nobel recognition for this work.

### glm4.7-flash

- **Q06 — 0.0.** Said chlorine. Correct: oxygen.
- **Q18 — 0.0.** Said Russia and Kazakhstan. Correct: Canada and the United States (including Alaska–Canada).
- **Q22 — 0.0.** Initially and directly said Virgil guides Dante through Paradise, then contradicted itself by mentioning Beatrice. Correct: Beatrice; Virgil cannot enter Paradise.
- **Q27 — 0.0.** Said the band self-produced and Rick Rubin was merely executive producer. Correct: Rick Rubin produced, with Daron Malakian co-producing.
- **Q28 — 0.5.** Described the sixth/relative-minor relationship but did not supply the asked-for mode name. Correct: Aeolian.
- **Q29 — 0.0.** Said 40.1 dB while stating the correct km/MHz formula, whose terms total about 100.05 dB. Correct: about 100 dB.
- **Q39 — 0.0.** Said Kahneman and Tversky won in 2002. Correct: Daniel Kahneman; Tversky was ineligible posthumously.

### gpt-oss-120b

- **Q01 — 0.5.** Correctly defined the 1.44 M☉ white-dwarf support limit, but gave neutron-star/black-hole collapse instead of the classic Type Ia-supernova outcome.

### gpt-oss-20b

- **Q01 — 0.5.** Correctly defined the 1.44 M☉ white-dwarf limit, but gave neutron-star/black-hole collapse instead of the classic Type Ia result.
- **Q23 — 0.5.** Gave only “All happy families are alike; each unhappy family is unhappy,” omitting “in its own way.” Correct full line includes that final clause.
- **Q32 — 0.0.** Claimed BPSK with a 50 Hz PN/chipping code. Correct: BPSK/DSSS, 1023-chip C/A Gold code at 1.023 Mcps; 50 bps is the navigation-data rate.
- **Q39 — 0.5.** Its runaway tail repeatedly paired Daniel Kahneman with Vernon Smith without cleanly distinguishing why each won. Correct answer to prospect theory is Kahneman alone; Smith’s half recognized experimental economics.
- **Q40 — 0.0.** The runaway answer said it was unsure and speculated about light speed in a medium. Correct: under a fixed legal exchange value, overvalued “bad” money circulates while undervalued “good” money is hoarded/exported.

### granite4-micro

- **Q12 — 0.5.** Correctly named *Trypanosoma cruzi* and triatomine bugs, but incorrectly added mosquitoes as occasional vectors. Mosquitoes do not transmit Chagas disease.
- **Q23 — 0.5.** Gave the essentially correct opening line but explicitly prefaced it with “Not sure”; per the requested uncertainty rule this is 0.5.
- **Q27 — 0.0.** Said Scott Litt. Correct: Rick Rubin produced; Daron Malakian co-produced.
- **Q29 — 0.0.** Said 116 dB and mixed incompatible units in the constant-based formula. Correct: about 100 dB.
- **Q32 — 0.5.** Correctly identified DSSS and the 1.023 MHz PN code, but omitted the asked-for carrier modulation. Correct: BPSK plus DSSS/Gold-code spreading.
- **Q40 — 0.0.** Reversed/garbled the consequence, saying the more intrinsically valuable currency is devalued and replaced by “better quality” currency. Correct: bad money circulates and good money is hoarded/exported when law fixes them at the same face value.

### minimax-m2-iq2

- **Q01 — 0.5.** Correctly defined the 1.44 M☉ white-dwarf limit but added neutron-star collapse rather than identifying the classic Type Ia outcome cleanly.
- **Q06 — 0.0.** The truncated reasoning compared electronegativities but emitted no answer. Correct: oxygen.
- **Q27 — 0.0.** Said only Daron Malakian. Full credit requires Rick Rubin: Rubin produced and Malakian co-produced.
- **Q36 — 0.0.** The truncated reasoning never supplied the number and ended while incorrectly wondering whether there were three nonabelian groups. Correct: five total (three abelian, two nonabelian).

### ministral3-14b

- **Q01 — 0.5.** Correct limit and white-dwarf subject, but stated neutron-star/black-hole collapse instead of the classic Type Ia outcome.
- **Q29 — 0.0.** Said approximately 90 dB. Correct: approximately 100 dB (the 10 dB miss is material here).
- **Q36 — 0.5.** Correctly said five, which answers the question, but its supporting list substituted the order-4 Klein group and a dicyclic group for the actual `C4×C2` and `C2×C2×C2` entries.

### nemotron-cascade-30b

- **Q01 — 0.5.** Correct limit and white-dwarf subject, but gave neutron-star/black-hole formation and omitted the classic Type Ia outcome.
- **Q27 — 0.0.** Said the band/Daron Malakian/Shavo Odadjian produced it and omitted Rick Rubin. Correct: Rick Rubin produced; Daron Malakian co-produced.
- **Q29 — 0.0.** Said 92 dB while its own km/MHz formula evaluates to about 100 dB.
- **Q32 — 0.5.** Correctly gave DSSS, Gold code, and chip rate, but omitted the asked-for BPSK modulation.
- **Q39 — 0.5.** Guessed Daniel Kahneman correctly but began “not sure”; per the requested rule, a correct uncertain guess earns 0.5.

### nemotron3-nano-30b

- **Q09 — 0.5.** Correctly gave phrenic nerve C3–C5 but added an “occasional C6” contribution, a minor anatomical inaccuracy for the standard answer.
- **Q23 — 0.0.** Gave Dickens’s “It was the best of times...” and said “not sure.” Correct: “All happy families are alike; each unhappy family is unhappy in its own way.”
- **Q25 — 0.0.** Correct album title but said 1982. Correct: *Iron Maiden*, 1980.
- **Q29 — 0.0.** Said 112 dB and mangled units in the constant form. Correct: about 100 dB.
- **Q32 — 0.5.** Correctly gave BPSK, the 1023-chip code, and 1.023 MHz, but incorrectly claimed 1023 chips per navigation-data bit; at 50 bps there are 20,460 chips per bit.

### nemotron3-super-120b

- **Q01 — 0.5.** Correctly defined the white-dwarf limit but added neutron-star formation rather than cleanly stating the classic Type Ia outcome.

### north-mini-code

- **Q01 — 0.5.** Correctly gave the limit and Type Ia possibility, but also presented neutron-star collapse as the alternative classic white-dwarf outcome.

### qwen3.6-27b

- **Q01 — 0.5.** Correctly defined the limit and mentioned Type Ia, but also added neutron-star formation for the classic white-dwarf case.
- **Q10 — 0.5.** Correct core function (medullary gradient/urine concentration), but incorrectly said the descending limb prevents water reabsorption; it is water-permeable, while the ascending limb is water-impermeable.
- **Q29 — 0.0.** Said 92.45 dB even though the supplied km/MHz formula evaluates to about 100.05 dB.

### qwen3.6-35b-a3b

- **Q01 — 0.5.** Correctly defined the limit but added neutron-star formation instead of identifying the classic Type Ia outcome cleanly.
- **Q19 — 0.5.** Correctly named the Bering Strait, but reversed its adjoining seas: the Bering Sea is south and the Chukchi Sea north.
- **Q29 — 0.0.** Said about 92 dB despite using metre/Hz variables in the full formula; correct substitution gives about 100 dB.
- **Q40 — 0.5.** Correctly explained hoarding higher-intrinsic-value money, then reversed the valuation labels in its conclusion. Correct: overvalued bad money circulates; undervalued good money disappears from circulation.

### qwen3next-80b-a3b

- **Q01 — 0.5.** Correctly defined the white-dwarf limit but added neutron-star formation rather than cleanly giving the classic Type Ia outcome.

**Best overall:** gemma4-31b — 40.0/40.

**Best knowledge-per-GB:** granite4-micro — 35.5/40 from an approximately 2.0 GB model file, by far the strongest score per stored GB; for a higher absolute-score efficiency option, gpt-oss-20b scores 36.5/40 at about 10.9 GiB.