Which local models to use for which jobs, and the experiments that led us there. This is hardware-agnostic, about the models themselves, not any particular machine or serving setup. The underlying benchmark reports (`SCORES*.md`) contain the raw per-question data.
The central finding, worth internalizing before the rest: **there is no single best model.** Knowledge, reasoning, instruction-following, tool-protocol compatibility, and clean autonomous termination are *separate* capabilities. A model that tops the knowledge benchmark can be a poor autonomous agent; a model that runs tool loops flawlessly can be a weak writer; a tiny model can be a perfect bounded worker and a disaster the moment a task turns open-ended. The right approach is a role-specialized portfolio, route each kind of work to the model that is actually good at it, rather than crowning one winner and using it for everything.
---
## Recommendations at a glance
### General assistant work (chat, coding, knowledge, writing)
| Job | Model | Why |
|---|---|---|
| Default for nearly everything | **Qwen3.6-27B** | Best all-rounder: broad knowledge, strong technical reasoning, concise, strict formats |
| Writing, docs, explanations, synthesis | **Gemma4-31B** | Strongest knowledge + writing profile; verbose, so instruct it to be concise |
| Hard problems, architecture, second opinions | **GPT-OSS-120B** | Deepest reasoning, best at catching subtle errors; slower and heavier |
| Fast, lightweight chat/coding | **GPT-OSS-20B** | Capable and quick for ordinary work (not for tool/agent use, see below) |
### Autonomous agent / tool-loop work
Here the priorities invert: **clean termination and valid tool calls matter more than raw intelligence.** A smarter model that occasionally hangs, loops, or emits unparseable output is worse in a cluster than a slightly less capable model that always finishes cleanly and emits valid calls every time.
| Role | Model | Notes |
|---|---|---|
| Orchestrator / planner | **Qwen3.6-27B** | Proven clean across real multi-turn tool loops |
| Primary bounded worker | **North-Mini-Code** | Excellent with named files, explicit tests, a clear stopping condition |
| Worker backup / overflow | **Devstral-Small2-24B** | Clean multi-turn behavior; different model family = failure-mode diversity |
| Cheap micro-worker | **Qwen3.5-9B** | Fast and efficient, but ONLY on concrete, mechanically-verified tasks |
| Independent reviewer | **GPT-OSS-120B** | Reviews consequential work; deliberately a different family from the workers |
If only one model is deployed anywhere, use **Qwen3.6-27B**, it had the best combined evidence across knowledge, reasoning, premise-resistance, formatting, and multi-turn tool use, and it is the least likely to be the wrong choice for an unknown task.
---
## What each model is for
### Qwen3.6-27B — the default brain
The best all-purpose model in the set and the safe default for any client app that doesn't have a specific reason to pick something else. It combines broad knowledge, strong technical reasoning, reliable instruction-following, good premise-resistance, and, uniquely among the mid-size models, proven clean behavior in multi-turn tool loops. It was the only model to score a perfect main-role result (calibration, strict formats, concise helpfulness) *and* tie for the top on hard reasoning *and* complete every agentic task cleanly.
It can credibly fill both the general-assistant and the orchestrator roles at once. In a small deployment that's fine; where independent review matters, pair it with a different-family reviewer rather than letting it check its own work (correlated errors are the risk). Its one blemish across all testing was a single malformed tool call that it immediately corrected by switching to a compact JSON form, competent recovery, not a red flag.
Mild downside: it can over-explain. A terse system instruction fixes that.
### Gemma4-31B — knowledge and writing specialist
The strongest broad-knowledge and writing model tested. It was the only model to score a perfect 40/40 on the broad knowledge round, and it reasoned very well and passed every single-shot worker task. For reports, documentation, explanatory writing, research synthesis, and teaching-style explanations, it's the best choice in the set.
Two caveats. First, it's verbose, it tends to produce more text than asked, so lead it with an explicit "be concise, lead with the answer" instruction for interactive use. Second, and more important for anyone considering it for automation: **it was never run through the multi-turn agentic loop.** Its excellent single-shot behavior does not prove it can drive a tool loop, recover from malformed calls, or terminate cleanly across many turns. Treat it as a proven chat/knowledge/writing model and an *unproven* agent until it's actually tested in a loop.
### GPT-OSS-120B — the skeptical reviewer
The deepest reasoner in the set and the best at catching subtle errors, exactly what you want in a reviewer. It solved every hard reasoning problem, resisted every false premise, and made the cleanest, most controlled arguments. Its ideal use is high-consequence review: checking a worker's output, challenging assumptions, difficult debugging, architecture decisions, and any judgment call where a confident wrong answer costs more than the extra inference time.
It is *not* the right choice as a bulk file-writing worker. In the agentic round it solved everything but had the weakest tool-protocol discipline of the successful models, including three consecutive malformed writes on one task before recovering. That inefficiency is fine for a reviewer that runs occasionally on important work; it would generate excessive retry traffic as a swarm of workers. Spend its intelligence on judgment, not throughput.
### North-Mini-Code — the primary hands
The best bounded implementation worker tested, and the round-4 agentic results are what validate it. Given a concrete task, a named file scope, and objective tests, it read, wrote, executed, verified, and stopped cleanly across a full multi-turn trajectory, 5/5 tasks solved, 5/5 clean terminations, zero bad calls, no runaway over 21 agent turns. Its coding and technical performance is genuinely strong, including good instincts on unsafe patterns (it correctly rejected `gets()` and SQL-injection framings).
The important nuance: North is the **hands, not the executive.** Its one documented failure mode is an ambiguity-driven runaway, on an open-ended factual prompt with no clear stopping condition, it once consumed its entire token budget without producing an answer. Concrete tool tasks with external state and objective progress stabilize it; open-ended prompts destabilize it. So route bounded, verifiable implementation to it, and never make it the open-ended planner or give it final sign-off on destructive changes.
### Devstral-Small2-24B — the dependable backup worker
Less flashy than North but predictable, which is its own virtue. Clean live code execution, solid error-detection in review, and clean multi-turn behavior with immediate recovery from a single malformed call. It's a good overflow/parallel worker precisely because it's a different model family than North, so it doesn't share the exact same failure modes, useful diversity when you're running workers in parallel. Its broad daily-driver and recall scores are weaker than Qwen's, so it earns worker duty, not promotion to general assistant.
### Qwen3.5-9B — cheap, tightly-bounded worker
The surprise of the agentic round: the most turn-efficient perfect agent tested (5/5 solved, 5/5 clean, 18 turns, zero bad calls). For cheap, concrete, mechanically-verified microtasks it's excellent value. But this is *highly conditional on task shape*, the same model produced multiple empty-final runaways on warm, open-ended prompts in the role round. It is reliable when the environment supplies a concrete objective, a tiny action space, immediate tool feedback, and a clear completion condition; it is unsafe the moment the task is open-ended. Deploy it only as a narrow worker behind strict controls, and never promote it to a planning or conversational role on the strength of its worker score.
---
## Agent strategy (for client apps driving these autonomously)
The models are capable; the safety comes from the harness around them. Any client app running a model in an autonomous tool loop should enforce a control contract rather than trusting the model to behave:
1. Hard turn cap and wall-clock timeout per task.
2. Abort or switch model after two consecutive malformed tool calls.
3. Extract and validate exactly one allowed tool call per turn.
4. Validate tool name, argument schema, permissions, and scope before executing.
5. Treat tests and observable state as ground truth, never the model's own claim of success.
6. Force a clean termination immediately after verified success, or terminate automatically.
7. Detect repeated calls, repeated writes, and no-progress loops, and break them.
8. Route consequential changes to an independent reviewer of a different model family.
A note on parsing that bit us: several models that scored "zero bad calls" still emitted prose before the JSON, a trailing `</think>`, or a stray extra brace that our lenient benchmark parser happened to tolerate. Production parsing must **normalize and strictly validate** output, and the component that parses should not also hold execution authority. Zero counted bad calls in testing does not mean byte-perfect protocol compliance.
**When to escalate to the reviewer** (GPT-OSS-120B), rather than gating on tests alone:
- The change is destructive (deletes data, rewrites configs, touches anything in production).
- The change is security-relevant (auth, input handling, network exposure, credentials).
- It involves RF / link-budget / signal-processing math, where a confident wrong number is expensive.
- A worker has already failed its own tests twice and is about to be trusted anyway.
- The task is high-consequence and no automated test fully proves correctness.
Everything else is gated by tests. Reserve the reviewer for judgment calls; it's too slow and protocol-noisy to sit in the routine throughput path.
A reference flow: the orchestrator (Qwen) frames the task and its assumptions; a worker (North or Devstral) implements in a sandbox; tests and observable state verify the result; and the reviewer (GPT-OSS-120B) is invoked only if the change hits an escalation trigger.
---
## The experiments
Four rounds, each deliberately testing something the previous round couldn't, because early rounds saturated and stopped separating models.
### Round 1 — Broad knowledge
40 questions across physics, chemistry, biology, history, geography, literature, music, RF engineering, mathematics, and economics/law. Every answer manually verified rather than trusted at face value.
This measured factual breadth but **did not separate the leaders**: most models clustered between 38 and 40 out of 40. Gemma4-31B was the only perfect 40/40; GPT-OSS-120B, Nemotron3-Super, North-Mini, and Qwen3-Next landed at 39.5. Granite4-Micro scored a striking 35.5 for a roughly 2GB file, though it proved too unreliable for unverified engineering work in later rounds. Aya was notably weak on RF and math. The round exposed common *fluent errors*, models confidently getting the Chandrasekhar-limit consequence, RF path-loss arithmetic, GPS modulation, a music production credit, or the number of groups of order 8 slightly wrong. The lesson: raw knowledge is nearly free among competent modern models and a poor basis for choosing between them. [SCORES.md]
### Round 2 — Hard recall, reasoning, and false-premise traps
24 questions split three ways: deep/obscure recall, multi-step numerical reasoning, and traps (false premises the model should reject rather than politely accept).
This produced real behavioral signal. Eight models tied at 24/24, but the reasoning and trap items exposed differences the totals hid. Qwen3.6-27B caught an error in the benchmark's *own* expected answer on a 5G NR question, correctly giving 51 resource blocks / 612 subcarriers where the answer key was wrong. Several otherwise-capable models made the *same* revealing mistake on a satellite problem, using geostationary orbital radius where the question stated altitude, a category error that matters for RF/systems work. The conclusion that shaped everything after: **premise-resistance and clean reasoning matter far more than small differences in factual recall.** [SCORES_HARD.md]
### Round 3 — Role fitness
The first round to test deployment roles explicitly, with the worker section graded by *actually executing* the code the models wrote. Three sections: main (calibration, strict formats, concise helpfulness, honest uncertainty), escalation (detecting planted technical, software, security, and math errors), and worker (eight live-executed code/format tasks).
Qwen3.6-27B was the only model to score a perfect 8/8 on main, and it also went 6/6 on escalation. North-Mini and Devstral-Small both hit 27/27 on the executed worker tasks. The pivotal discovery: **single-shot worker strength can coexist with poor open-ended control.** Several models that aced bounded, concrete tasks ran away, thousands of tokens with no final answer, on warm or ambiguous open prompts. That split (great when bounded, dangerous when open-ended) is the single most important behavioral fact for cluster design, and it's invisible to any knowledge benchmark. [SCORES_ROLES.md]
### Round 4 — Multi-turn agentic
The decisive round for autonomous use. Six models were placed in real tool loops: they emitted JSON tool calls, the harness executed them in a sandbox (file read/write, Python execution), fed results back, and let the model iterate, retry after errors, and call `done` when finished, up to a turn cap. This is the only round that measures what actually determines cluster viability: termination and recovery across a whole trajectory, not per-shot quality.
| Model | Solved | Clean termination | Turns | Bad calls |
|---|---:|---:|---:|---:|
| Qwen3.5-9B | 5/5 | 5/5 | 18 | 0 |
| North-Mini-Code | 5/5 | 5/5 | 21 | 0 |
| Qwen3.6-27B | 5/5 | 5/5 | 19 | 1 |
| Devstral-Small2 | 5/5 | 5/5 | 20 | 1 |
| GPT-OSS-120B | 5/5 | 5/5 | 24 | 4 |
| GPT-OSS-20B | 0/5 | 0/5 | 20 | 20 |
The headline result: **North-Mini's worker strength survived multi-turn.** Its earlier ambiguity-driven runaway did not recur across 21 agent turns; it recovered from tool results correctly and terminated immediately after verified success. Qwen3.6-27B was confirmed as orchestrator (full success, clean, one instantly-corrected malformed write). GPT-OSS-120B solved everything but had the weakest JSON discipline of the successful models, cementing it as reviewer rather than bulk worker. GPT-OSS-20B emitted **zero parseable tool calls on any turn**, which looks like a tool-format/template incompatibility rather than a capability failure, it remains useful for plain chat/coding. [SCORES_AGENTIC.md]
One honest limitation of this round: the tasks did not force most models to repair a *semantic* Python failure after a genuinely failed execution, they mostly wrote correct code on the first try. Real iterative debugging (read a traceback, diagnose, patch, re-run, recognize a new failure) is still unmeasured and should be a future round using existing multi-file projects with planted failing tests.
---
## Deprioritized
- **MiniMax-M2-IQ2:** repeated thinking loops produce empty finals even with a large token budget. The problem is termination/control, not knowledge. Unfit for autonomous use at this quant.
- **Aya-23-8B:** too many confident specialist and numerical errors; fluent prose masking weak reasoning.
- **Granite4-Micro:** genuinely impressive for its ~2GB footprint, but insufficient premise-resistance and arithmetic reliability for unverified technical work.
- **GPT-OSS-20B in tool mode:** disabled for agents until its tool-call template is fixed and re-tested. This does not exclude it from general chat or coding, where it performs well.
---
## What we have NOT validated
Know the edges of the evidence before over-trusting a recommendation:
- **Gemma4-31B was never run through the multi-turn agentic loop.** It's a proven chat/knowledge/writing model and a strong single-shot worker, but its fitness as an autonomous orchestrator is unproven. Run it through the same tool-loop test before wiring it into automation.
- **Debug-from-failure is untested.** Every capable model wrote correct code on its first executable attempt, so none was forced to read a real error and patch a semantic bug across turns. Iterative debugging in a loop is unmeasured.
- **GPT-OSS-20B's tool failure is assumed fixable but unconfirmed.** It looks like a template/format mismatch, not incapacity; until someone fixes the template and re-runs, that's a hypothesis, not a fact.
- **Small-worker control under ambiguous input is a known, live risk.** Qwen3.5-9B and North-Mini are clean on bounded tasks but have documented runaways on open-ended prompts. This is exactly why the harness-side control contract (caps, timeouts, test-based success) is non-optional rather than a nice-to-have.
---
## Bottom line
- **Qwen3.6-27B** — the brain and general default; strong everywhere, the safe pick for an unknown task.
- **North-Mini-Code** — the primary pair of hands for bounded, verified implementation.
- **GPT-OSS-120B** — the skeptical independent reviewer, invoked on escalation triggers, not every task.
- **Devstral-Small2** — worker diversity and overflow capacity.
- **Gemma4-31B** — deep knowledge, writing, and synthesis (chat-proven, agent-unproven).
- **Qwen3.5-9B** — a cheap, tightly-bounded worker tier that must stay narrowly routed.
Knowledge is nearly free among modern models; the traits that actually separate them for real work are reasoning, premise-resistance, and clean termination. Route work by those, and put verification and control in the harness rather than trusting any single model's intelligence to be enough on its own.