The story of a competition entry

Teaching a model to teach

I picked Math & Scientific Reasoning because pragmatic education is something African classrooms need and rarely get enough of: too many students for one teacher, and too little time to slow down on the question any single student actually got wrong.

The competition evaluates an offline model on a $150–$500 CPU-only laptop. The model must therefore balance teaching accuracy, response speed, and memory use. This report follows the experiments that narrowed those three variables to two runtime-dependent finalists.

Where the story stands, for now

Fine-tuning improved both finalists

Qwen3.5 0.8B remains the scalar leader after a 15-point ARC-Easy gain. Qwen2.5 1.5B remains the vector leader after a smaller 3.4-point gain. These are the best measured GGUFs, not yet the packaged submission: both still need the tutor metadata and live-prompt checks, and the final choice still depends on the audit CPU configuration.

Scalar leader, n=500Qwen3.5 0.8B Q4_0

Scalar total: 80.3664
Vector total: 82.3962
Scalar generation: 13.60 tok/s
Est. scalar RSS: 691 MiB
ARC-Easy, n=500: 70.2%

Vector leader, n=500Qwen2.5 1.5B Q4_K_M

Vector total: 84.1387
Scalar total: 67.0475
Vector generation: 17.44 tok/s
Est. vector RSS: 1,706 MiB
ARC-Easy, n=500: 77.8%

Selection condition: use the tuned Qwen3.5 candidate for the supplied scalar configuration; use tuned Qwen2.5 only if the audit is confirmed to use the vector configuration. Before submission, apply the same embedded tutor policy to the chosen GGUF and repeat the live-prompt and physical-target checks.

Scalar candidate size: 513.0 MB
Quantization: Q4_0
Target: 8 GB, CPU-only
Selection basis: Matched n=500 accuracy, speed, and RSS

The rules we're playing by

A $150–$500 laptop is the whole target

The Africa Deep Tech Challenge wants language models running entirely on-device, on the standard laptops already sitting in African cities, with no cloud behind them. It lists seven tracks: Math & Scientific Reasoning, Healthcare, Agriculture, Creative Writing, Coding Assistants, Corporate/Enterprise, and Autonomous Agents. We took the first one, because a tutor that explains a wrong answer instead of just marking it is closer to what a scarce teacher actually does.

The hardware is exact: an Intel Core i5, 10th to 12th generation, integrated graphics only, Ubuntu 22.04, and a hard 7 GB RAM ceiling. Go over it and the run is disqualified, full stop. The rules are just as narrow on software: llama.cpp only, weights in GGUF. Any open base model is fair game, quantized or fine-tuned however we like, but nothing closed and nothing needing a different runtime.We disable AVX-512 in our own build and check the disassembly for it, because several eligible consumer CPUs don't support it: an illegal instruction there means an automatic zero.

S_{total} = 0.50 \cdot S_{acc} + 0.30 \cdot S_{perf} + 0.20 \cdot S_{eff} - P_{thermal}

Fig. 1 Fixed-15 profiler score. Below 15 tok/s, an increase of 1 tok/s adds 2.00 total points. One accuracy point adds 0.50. A reduction of 1 GB in peak RSS adds 2.86.

Half the score is accuracy, so a fast model that can't teach still loses. But performance caps at 15 tok/s and efficiency zeroes out at 7 GB, so past those points speed and memory stop paying at all.S_perf = min(TPS/15, 1) × 100 and S_eff = max(0, (7 GB − RSS)/7 GB) × 100. The public challenge page instead defines performance relative to the fastest submission, TPS_max, rather than a fixed 15. We follow the fixed formula, since it's what the executable profiler actually runs, and keep the cohort-relative reading as a separate sensitivity check throughout this page. That formula produces a decision rule we checked every optimisation against for the rest of the project:

Two CPU configurations run through this story, named once so we can stop repeating the acronyms. The scalar configuration is the supplied profiler build, with the wider vector extensions disabled for the affected kernels. The vector configuration is a portable SIMD build with AVX2, FMA, and F16C enabled, on the same two physical cores. From here on, this page just says scalar and vector.

Why we keep five separate evidence categories on this page

Our numbers come from different systems, CPU features, and performance formulas, so we never blend them into one ranking: official profiler results (the participant executable's own runs), profiler-parity estimates (a scalar-only reconstruction with a 45 MiB profiler-root allowance we estimated, not measured), controlled vector proxies (a paired scalar/vector benchmark on the same host), website sensitivity (the cohort-relative formula as a separate check, never averaged in), and plain development results (Mac, Docker, and custom-engine tests that guided decisions but never ranked a submission). A colour and a badge on every figure below show which lane it belongs to.

Before we picked a model

Bigger teaches better. Bigger also costs more.

Before touching quantization, we ran a plain accuracy-by-scale study across the Qwen3.5 family: 0.8B, 2B, and 4B, on four reasoning tasks. The result was the least surprising thing in this whole project, and also the thing every later decision had to argue against.

Separate accuracy evaluations

Candidate measurements

Reasoning studyAudit campaign

Fig. 2 The reasoning-study points use a four-task development mean; audit candidates use ARC-Easy proxies. The two fine-tuned finalists are marked separately from the historical candidates.

4B beat 2B beat 0.8B, cleanly, on every task. Under a formula that pays 0.50 total-score points per accuracy point and only 2.86 per gigabyte saved, that curve makes a strong case for going as big as the RAM ceiling allows. But RSS on Linux tracks file bytes almost one-to-one, so "as big as the ceiling allows" is a number in the hundreds of megabytes, not gigabytes, once quantization is priced in. That tension split our search down two roads at once, plus one number neither road was allowed to cross:

Road one

Go small and dense

Stay in the Qwen3/3.5 family, quantize as hard as the audit kernel would tolerate, and win back accuracy through parameter dedication and behaviour baked into the file rather than raw scale.

Road two

Go big and ternary

Bet that an 8B-parameter model at roughly 2 bits per weight, BitCPM4 in TQ2_0, could out-accuracy anything dense enough to fit, if we pruned and compressed it hard enough to land in the same weight class as the dense candidates.

Every 100 MB above the smallest working file was already worth 1.4 points of efficiency score we'd rather spend on accuracy, so neither road got a free pass on size: road two's ternary bet only earned its keep by landing at about 2.1 GB, not the 8 GB the parameter count alone would suggest. We ran both roads in parallel for weeks. This page tells road one's story in full, because it's the one that shipped; road two still shows up wherever its lessons apply, and its ending is in the vocabulary-pruning section below.

The toolbox

Ten optimization methods

We tested ten ways to change accuracy, file size, RAM, or speed. Each method either narrowed the candidate set or was retained as a measured rejection.

Quantization

Which quantization type the audit binary can actually run fast, not just how many bits per weight.

Parameter deduplication

Dropping the duplicated output head when embedding and head can share one matrix.

Vocabulary pruning

Removing tokens the tutor will never emit, and checking token by token that nothing else changed.

Weight streaming

Paging weights from disk instead of keeping the whole file resident, and finding the boundary the competition draws around it.

Runtime tuning

Threads, KV cache, checkpoints, and two kinds of speculative decoding, before any model comparison could be trusted.

Instruction-set targeting

Discovering that the audit binary's actual CPU features change which quantization type wins, and testing both readings.

Widening the model search

Testing nine more candidates and a specialist fine-tune once the first roster stopped improving.

Baking behaviour into the file

Persona, chat template, and sampling defaults, since judges talk to the bare GGUF with no app layer in front of it.

Fine-tuning

Updating the two finalists on verified math and science questions matched to the profiler's continuation format.

Runtime configuration we couldn't submit

Repacking, mmap policy, thread count: measured changes that live in Muta's runtime, not in the submitted GGUF.

Quantization, deduplication, embedded behaviour, and fine-tuning can change the submitted GGUF. Streaming and runtime tuning cannot. The model search identifies which architecture receives those changes; the scalar/vector comparison determines how each candidate is scored.

Method one

Quantization: the file format is the kernel choice

Quantization means storing each weight in fewer bits than the 16 or 32 it was trained in, trading a little precision for a much smaller file. GGUF supports a whole ladder of these formats, from plain 4-bit rounding (Q4_0) through vector-quantized "k-quants" (Q4_K_M, Q5_K_M) to sub-4-bit "i-quants" and ternary formats. We assumed, like most guides do, that a newer, smarter format like Q4_K_M would be the better choice. On this evaluator, it wasn't.

Scalar profiler configuration

Kernel dispatch by tensor type

Scalar audit build

Fig. 3 We show the current scalar-ledger rates beside the kernel paths. Model size also affects the measured range. The vector build enables SIMD implementations for the listed tensor types.

We read the audit binary's own kernel source rather than trust a guide written for a different CPU, and found only Q4_0 has a hand-written SIMD path on the scalar build the executable profiler runs. Everything else, including the "smarter" k-quants, falls back to generic C. The original control, on the Qwen3 1.7B pair, measured 9.99 tok/s in Q4_0 against 5.30 tok/s in Q4_K_M, purely from that one format choice; the diagram above shows the same pattern holding on the file we eventually shipped, at 12.63 tok/s. That single fact ruled out most of the usual advice and pointed every candidate toward pure Q4_0, at least until instruction-set targeting complicated the picture again, several sections down.

Method two

Cutting the duplicate head

Many transformer architectures give the embedding table and the output head separate weight matrices, even though both are the same shape and often learn near-identical representations. Tying them, letting the head reuse the embedding matrix instead of storing its own copy, is a free win if the architecture supports it: no retraining, no accuracy cost, just a smaller file.

We reproduced our recommended file from its source quant using metadata changes alone, then ran an isolated A/B on the tied-versus-untied question: removing the duplicate saved about 175 MB with identical ARC-Easy accuracy in both conditions, an unambiguous win we took on every candidate that supported it.

The tutor persona, chat template, and sampling defaults go in through a similar scripted pass — though that one only touches metadata, not tensors, so it costs no file size at all. It gets its own full treatment in the behaviour-baking section further down, once the mechanisms that shrink the file are out of the way.

Method three, and the end of road two

Pruning the vocabulary, and letting the ternary bet go

Vocabulary pruning removes tokens a model will never realistically emit from its embedding table, shrinking the largest single matrix in a small model without touching a single weight the tutor actually uses. We built this for road two, the 8B ternary BitCPM4 model, since an English tutor has no use for most of a 73,448-token vocabulary built to cover CJK scripts.

Vocabulary pruning

We cut the vocabulary from 73,448 to 44,416 tokens and padded it to a multiple of 64, which shrank the file by 164 MB. English tokenisation matched across 20,464 checked tokens; perplexity fell from 10.558 to 10.473.

Kept

TQ1_0 body test

The body shrank by 340 MB, but generic CPU throughput fell from 3.70 to 2.88 tok/s. The lower bit width bought us nothing on the evaluated kernel's instruction cost.

Rejected

Head and embedding quantization

Head and embedding requantization saved at most 48 MB. The largest estimated total-score gain we found was about 0.14 points, too small to matter.

Rejected

Factorisation and sparsity tests

The ternary matrices turned out to be full-rank: rank-2048 factorisation error was about 0.80 before quantization. Dense GGUF storage and kernels give us no credit for unstructured zeros either, so sparsity saved nothing.

Rejected

Only the vocabulary prune was worth keeping, and it was the last real win road two got. BitCPM4-8B-TQ2_0 still recorded the highest accuracy of any candidate we ever tested, 88% ARC-Easy, well above every Qwen quantization we ran. But it decoded at just 0.81 tok/s on the scalar audit kernel, since ternary formats fall on the same generic-C path as k-quants. Even after instruction-set targeting later gave it a vector kernel and lifted that to 7.49 tok/s, its total reached only 72.5121, still short of every Qwen variant. The most accurate model we built turned out to be the one we couldn't ship. Road one, dense and quantization-first, was the road that survived.

Method four

Weight streaming, and where the submission boundary sits

If a model's file is too big to hold entirely in RAM, one option is to page the weights in from disk as each layer needs them. We built a residency-window streaming engine for llama.cpp to test exactly that on the 2.2 GB BitCPM model, still mid-flight on road two at the time.

Development result

Throughput and RSS by resident weight budget

Custom engine

Fig. 4 A historical custom-runtime experiment, completed before the current model campaign. Each point pins more of the 2.2 GB BitCPM model in memory; the result is not a submission score.

It worked, as engineering: full streaming cut peak RSS to 279 MiB, a huge win on paper. It also missed the 15 tok/s threshold, generating only 10.47 tok/s. Decoding one token at batch size 1 touches nearly every weight in the model, so throughput reduces to one ratio, bandwidth over model size:Hitting 20 tok/s would take about 44 GB/s of effective bandwidth. Even the fully resident point in the chart above, the fastest this model gets on this host, only reaches 41 GB/s and 18.70 tok/s. On this hardware, 20 tok/s is out of reach no matter how much RAM the residency window is given; only faster memory would clear it.

tok/s \approx \frac{{BW}_{eff}}{M}

Fig. 5 Effective bandwidth blends fast reads from whatever's resident with slow reads from whatever still has to stream off the SSD. Reaching 15 tok/s on this 2.2 GB model needs about 33 GB/s of it.

Working that ratio backward from each point in the chart above shows the same climb in different units: Stream all backs out to about 23 GB/s of effective bandwidth, Pin 1,500 MB to about 34 GB/s, Resident to about 41 GB/s. The 33 GB/s the target needs falls right at the 1,500 MB pin, where the measured rate, 15.35 tok/s, is the only point in the sweep that clears the line.

The real problem was upstream of the bandwidth math: streaming needs a custom binary, and the competition evaluates the submitted GGUF through the organiser's own unmodified llama.cpp. No engine change we make can travel with the file. That single fact drew a line we kept running into for the rest of the project:

Fig. 6 We classify every evaluated change by submission eligibility here. Only properties encoded in the GGUF can affect the model-only evaluation.

Method five

Tuning the runtime before trusting any comparison

None of the model-versus-model numbers above mean anything if the runtime underneath them is inconsistent. Before comparing candidates at all, we fixed threads, KV cache, and checkpoints, and along the way tested two forms of speculative decoding that looked promising in early sweeps and fell apart under load.

Development result

Selected runtime configurations

Development

Fig. 7 Our runtime sequence moves from the Docker baseline through resource caps, native execution, thread tuning, and speculation. These experiments guide configuration; they never rank submission models.

Rejected

Draft-model speculation

Acceptance reached 98.4%, but generation still fell from 30.84 to 24.72 tok/s.

Rejected

N-gram speculation

Acceptance ran only 12–22%, so lookup and verification overhead outweighed the gain.

Adopted

Six-thread cap

Raising the thread count to ten cut decode to about 4.4 tok/s. We had no temperature reading to explain why.

Adopted

Unified KV, two checkpoints

This configuration cut retained state and sped up prefill without slowing decode.Unified KV shares capacity across active slots instead of reserving a fixed block per slot.

Six threads and unified KV got us to 31.09 tok/s at 3,137 MiB, 83% of the estimated weight-bandwidth ceiling, on the development host. None of this configuration travels with the submitted GGUF; it's the fifth method from our list, the runtime work that stays ours to keep but never enters the score. It bought us trustworthy numbers, and that finally let us run the first real model-versus-model scoreboard.

Six smaller questions

What we tried and didn't need

Six more levers, each tested once we had a trustworthy runtime to test them on. None of them changed the file we submit, but each one closed a question we'd otherwise still be asking.

GGUF-contained

Mixed tensor or layer quantization

Uniform quantization treats every tensor the same. Mixed precision keeps the parts most sensitive to rounding — embeddings, the output head, the final blocks — at a higher bit width while compressing the rest, and only pays off if the recovered accuracy is worth the extra bytes and the slower kernel those tensors fall back to. We tried it twice, on two different models: on the Qwen3 1.7B ladder, a Q3_K_M body with a Q6_K head against uniform Q3_K_M, and an IQ4_XS variant with the same higher-precision head against plain IQ4_XS; later, on Math-Expert, a Q4_0 body with a Q6_K or Q8_0 tied embedding, and separately a Q4_0 body with Q5_0 in the last four blocks.

None of it held up. The Qwen3 head swap fell to 66% ARC-Easy, worse than uniform Q3_K_M, and the IQ4_XS variant came out both slower and larger than plain IQ4_XS. Math-Expert's mixed layouts recovered no meaningful accuracy either. We rejected every mixed layout we tested — uniform quantization, chosen for the kernel it runs on rather than its nominal precision, kept winning.

GGUF-contained if supported

Structured and unstructured pruning

Structured pruning removes whole layers or low-rank factors a dense runtime can actually skip; unstructured pruning zeros individual weights, and only helps if the file format and kernel give credit for the zeros. We tested both: a Qwen control removed one layer and repeated throughput and ARC-Easy, while the BitCPM branch — covered in full in the vocabulary-pruning chapter above — measured singular-value reconstruction error on its large matrices and checked whether dense GGUF storage could benefit from sparse zeros.

Removing one Qwen layer gained about 3.7% decode speed but cost two ARC-Easy points, worth a full accuracy-score point under the scoring formula — a bad trade for a few percent of throughput. BitCPM's ternary matrices stayed full-rank, with rank-2048 factorisation error near 0.80 before quantization, and dense GGUF storage gives unstructured zeros no credit at all. We rejected every pruning branch we tested; the one pruning result we kept was vocabulary pruning, which prunes tokens rather than weights.

GGUF-contained

Smaller architectures

A smaller model moves fewer weight bytes per decoded token and usually costs less RSS — it only wins if the capability it gives up is smaller than what performance and efficiency pay back. That question isn't a side branch here; it's most of the story on this page. The four-task scale study in the second chapter, the specialist search, and the second widening further down are all, at bottom, this same question asked of a different candidate set, and all three point the same way: the 0.6B–1.7B region is where both roads, and every finalist this story ends with, actually live. See those sections for the full results.

GGUF-contained after training

Distillation and fine-tuning

Distillation transfers a larger teacher's behaviour into a smaller model; fine-tuning adapts a base model to a task — here, mathematical reasoning. Either can raise accuracy without growing the file, and Math-Expert, the specialist search's raw leader, is exactly this kind of result: a Qwen3-0.6B fine-tune on OpenMathReasoning-mini. We screened several more finished public derivatives in that same search (its provenance section records why each one that didn't make the cut fell short), but ran no new training of our own.

We kept Math-Expert as a finished, tested fine-tune, and deferred new training rather than submit something we couldn't fully validate: a GPU, a full dataset audit, and a reproducible conversion path were all missing pieces we couldn't fill in the time we had.

Stored layout plus engine-only policy

Tensor layout, alignment, and runtime repacking

GGUF fixes how tensors are stored on disk, but the engine can still repack them into a faster in-memory layout at load time. Repacking can speed up a kernel, but it costs the RSS budget to do it. A 4B development control compared the default runtime with repacking disabled; we also reviewed a custom alignment or packing scheme on top of that, and decided against building one.

Disabling repacking cut the tested 4B footprint from about 3,236 to 602 MiB, a real product win, without a clear speed loss. But an unsupported layout that fails to load doesn't get a partial score, it gets zero, so we kept the stock GGUF layout for compatibility rather than risk it. No-repack stays a product-only setting — it's method ten on our list, the boundary this whole story keeps running into.

Engine-only

Context size and KV-cache implications

Context length sets how many token states the KV cache holds. A shorter or quantized cache can save memory, but it doesn't touch the weight traffic a batch-one decode has to move every token regardless of context. We'd already replaced runtime defaults with explicit context and KV limits back in the runtime-tuning chapter, which stopped memory growth and made later comparisons trustworthy — and the participant profiler fixes its own workload anyway, at a 512-token prompt and 128 generated tokens.

No GGUF-level context metadata we could set would change what gets scored, so we rejected context metadata as a profiler-facing lever. We keep the bounded context and KV settings we already adopted, but only as product configuration.

Checkpoint

The first scoreboard: Qwen3 1.7B wins, narrowly

With quantization, deduplication, and the runtime settled, we ran six candidates through the actual participant profiler. Qwen3 1.7B Q4_0, pure and tied, won at 72.4653, ahead of Qwen3.5 0.8B by 0.92 points and comfortably ahead of both the BitCPM ternary bet and a larger Qwen3.5 4B candidate.

Official profiler result

Score by component

Direct

Fig. 8 Contributions use the executable profiler's fixed 15 tok/s reference. ARC-Easy supplies the accuracy proxy; we still don't have the judging-panel score.

Official profiler result

Speed against memory

Direct

Fig. 9 We plot generation throughput against peak RSS. Higher throughput and lower RSS both raise the total. Circle area represents the ARC-Easy proxy.

Scalar measurements

Model	tok/s	First token	Peak RSS	ARC-Easy	Total
Math-Expert 0.6B Q4_K_MScalar leader · 396.7 MB	12.72	23.61 s	540 MiB	68% 54.2–79.2	77.9324
Qwen3.5 0.8B Q4_0Risk-adjusted recommendation · 507.2 MB	12.63	16.62 s	670 MiB	64% 50.1–75.9	75.3895
Qwen3 1.7B Q4_0Previous scalar leader · 974.2 MB	9.79	35.37 s	1,116 MiB	72% 58.3–82.5	72.4653
Qwen3.5 0.8B Q4_K_M	9.74	28.26 s	695 MiB	68% 54.2–79.2	71.54
BitCPM4 8B TQ2_0	0.81	584.15 s	2,307 MiB	88% 76.2–94.4	59.18
Qwen3.5 4B IQ4_XS	1.13	395.19 s	2,627 MiB	76% 62.6–85.7	52.93

The final Qwen model has the shortest measured first-token latency, 16.62 seconds against Math-Expert's 23.61. Neither number enters the score.

The table above already shows Math-Expert and the final Qwen3.5 0.8B ahead of the 1.7B model that won this first round. That's the ledger as it stands today: two later rounds moved the winner twice more. At the time of this first scoreboard, though, 1.7B genuinely led, and we treated it as settled long enough to ask a harder question: would the public webpage's cohort-relative scoring formula, instead of the fixed 15, change anything?

We checked, and the full breakdown, at every cohort floor we tested, is in the operational appendix under Website-relative sensitivity. It's a separate check, never blended with the profiler result above.

Method six

Finding the real CPU, and watching the ranking flip

Everything so far assumed the audit binary runs scalar only, because that's how the executable profiler we could get our hands on was built. But the competition's own public page describes hardware that plausibly supports the vector configuration defined above. We couldn't be sure which one the final judging actually runs, so we built both and measured the same seven models on each.

AVXON AVX2ON FMAON F16CON NATIVEOFF AVX-512OFF

Matched models and scoring function

Total score by CPU method

ScalarVector

Fig. 10 Each model carries a scalar and a vector fixed-15 total, both using its ARC-Easy-50 proxy. The finalist scalar rows are participant-profiler results; the quantization-ladder rows are controlled scalar screens. Outlined bars mark the highest total in each CPU configuration.

Turning the vector configuration on reordered the field as well as sped it up. Q4_K_M, penalised by the generic-C fallback under quantization's first lesson, jumped ahead of pure Q4_0 once it had a real vector kernel to run on: 80.4484 against 80.2818, a lead earned entirely by 59.7 MiB less repacking RSS, not by anything about the model, and not by any change to its measured accuracy. Even BitCPM, road two's abandoned bet, sped up 9.235× and became operationally viable, though its total still couldn't catch the Qwen variants.

Scalar and vector measurements

Scalar and vector benchmark results for seven GGUF models across the quantization and expanded model searches.
Model	Scalar → vector pp512	Scalar → vector tg128	Decode gain	Est. profiler RSS	ARC-Easy proxy	Total
Math-Expert 0.6B Q4_K_MExpanded-search leader	21.7268 → 153.9351	12.6339 → 39.2320	3.105×	759.7 MiB+202.9 MiB	68%	81.8803
Qwen3.5 0.8B Q4_0Risk-adjusted recommendation; tensor-identical vector source	31.0781 → 98.0094	12.6955 → 27.1509	2.139×	928.1 MiB+240.8 MiB	64%	79.4104
Qwen3 1.7B Q4_0Previous scalar choice	14.7166 → 47.0716	9.9869 → 16.8927	1.691×	2,049.4 MiB+916.3 MiB	72%	80.2818
Qwen3 1.7B Q4_K_MQuantization-ladder leader	7.1859 → 55.4554	5.2954 → 15.6714	2.959×	1,989.7 MiB+806.2 MiB	72%	80.4484
Qwen3 1.7B Q5_K_M	6.5613 → 24.3231	4.7839 → 12.7191	2.659×	1,364.6 MiB+0.1 MiB	76%	79.6307
Qwen3 1.7B IQ4_XS	3.2063 → 23.9364	2.4961 → 14.0644	5.635×	1,082.3 MiB+0.5 MiB	70%	80.1089
BitCPM4 8B TQ2_0	0.8762 → 13.6569	0.8108 → 7.4876	9.235×	2,316.4 MiB+0.1 MiB	88%	72.5121

Method seven

Widening the search finds a specialist we hadn't considered

Once the Qwen roster stopped producing gains, we widened the net: nine more candidates, and eight quantization layouts of one promising fine-tune, Math-Expert, a Qwen3-0.6B model tuned on mathematical reasoning data. It won the raw scoreboard. It didn't win cleanly.

Profiler proxy

Math-Expert 0.6B Q4_K_M

Direct scalar total: 77.9324
Vector total, n=50: 81.8803
Vector generation: 39.23 tok/s
Est. vector RSS: 759.7 MiB

Use this file only if you treat the executable profiler's 50 ARC-Easy items as the selection objective.

Risk-adjusted submission

`Muta-Tutor-Qwen3.5-0.8B-Q4_0-final.gguf`

Direct scalar total: 75.3895
Vector total, n=500: 76.8104
Vector generation: 27.15 tok/s
Est. vector RSS: 928.1 MiB

This file wins the matched result outside the 50-item profiler slice and reaches its first token sooner. Its template supplies the tutor policy and disables hidden reasoning whenever the caller sends no setting of its own.

Latest finalists, fixed-15 score

Direct scalar and vector totals

Direct scalarVector

Fig. 11 Both vector totals use the profiler's 15 tok/s cap and ARC-Easy-50. Math-Expert scores 81.8803; Qwen scores 79.4104. Vector RSS adds a 45 MiB profiler-root estimate to the measured child tree.

Latest scalar and vector finalist results

ARC-Easy-50 supplies the raw fixed-15 comparison; the ARC-Easy-500 column is diagnostic. The final Qwen vector row transfers the source measurement, and only after we verified tensor identity.

Sample sensitivity

Profiler total and larger-sample diagnostic

Same TPS and RSS

Fig. 12 The solid bar uses the profiler's 50 ARC-Easy items. The paired bar substitutes the larger ARC-Easy estimate, while keeping the same measured TPS and RSS. That second bar is diagnostic; it is not an official score.

Quantization search

Speed and ARC-Easy trade-off

Scalar screen

Fig. 13 Uniform Q4_0 is the fastest layout, but it loses 16 ARC-Easy points. Q4_K_M holds the best measured score. Neither higher embedding precision nor four higher-precision final layers recovers the loss.

We ran Math-Expert through the same quantization-method question we'd already answered once, on a completely different model, and got the same answer: the format with a real kernel wins, and no amount of extra precision in the embedding or final layers buys back what a slow kernel costs. That's the second confirmation we promised back in the quantization section.

Finalist accuracy checks

Confidence intervals are Wilson 95% intervals. These task results are diagnostics; they are not the judging-panel tutoring score.

Models screened and rejection reasons

Everything else we tried in this round

Method	Test or constraint	Decision
Uniform quantization	Q4_0, Q5_0, Q4_K_S, Q4_K_M, and IQ4_XS from the same F16 source.	Keep Q4_K_M. Q4_0 is fast but loses too much accuracy; Q5_0 is slow on the scalar profiler kernel.
Mixed tensor precision	Q4_0 body with Q6_K or Q8_0 tied embedding; Q5_0 in the final four blocks.	Reject. None recovered enough ARC-Easy accuracy.
Architecture and specialist search	Qwen3/3.5, Gemma 3, Noema, VibeThinker, OpenMath-Nemotron, NuminaMath, and a reasoning distill.	Keep Qwen3.5-0.8B and Math-Expert as finalists. Other files lost on throughput, accuracy, licensing, or provenance.
Pruning and vocabulary reduction	Prior layer-pruning and BitCPM vocabulary results were retained; Qwen BPE vocabulary pruning cannot be reproduced safely with the current tool.	No new submission candidate.
Distillation and fine-tuning	Finished public fine-tunes were tested. A new training run was not promoted without a GPU, a complete dataset audit, and a reproducible conversion path.	Defer new training; do not submit a partially validated derivative.
MTP and speculative decoding	`llama-bench` in the profiler does not invoke the runtime flags required by MTP or draft decoding.	Exclude from a GGUF-only profiler claim.
Context and embedded template	The profiler fixes p512/tg128. A live four-prompt test showed that unrestricted hidden reasoning could consume the full response allowance. The final GGUF embeds the tutor policy and forces non-thinking ChatML.	Adopt for interactive reliability; assign no profiler throughput gain.

Research and provenance

We built the search on public model cards, documented licences, reproducible conversion procedures, and retained raw measurements. Math-Expert is a Qwen3-0.6B fine-tune on OpenMathReasoning-mini; its public card gives no complete training recipe or benchmark table. The Qwen3.5 model derives from the official Qwen checkpoint.

Primary references: Math-Expert model card, GGUF repository, OpenMathReasoning dataset, OpenMathReasoning paper, low-bit reasoning study, and llama.cpp speculative-decoding documentation.

Method seven, continued

A second widening moves the vector leader again

Math-Expert's win over the six-model ledger settled nothing about architecture, only about one specialist fine-tune. We hadn't tried Qwen's own general-purpose 1.5B and 3B line at all, or comparable dense models from outside the Qwen tree. So we paired eight GGUFs — Qwen2.5 at 1.5B and 3B, Qwen2 at 1.5B, Llama 3.2 at 1B and 3B, Gemma 2 at 2B, Phi-4 Mini, and Orca Mini at 3B — and ran every one of them through the same 512-token prompt and 128-token decode, under both the scalar and vector configurations, with each model's accuracy measured exactly once and reused for both totals.

Eight matched GGUFs

Total score by CPU configuration

ScalarVector

Fig. 14 Each model carries a scalar and a vector fixed-15 total, both using its own ARC-Easy-50 proxy, measured once and shared across both CPU configurations. Outlined bars mark the highest total in each configuration.

Qwen2.5 1.5B took first under vector, 82.8697, with Qwen2 1.5B close behind at 82.2176 — the two smallest, most conventionally quantized entries in the field. Llama 3.2 1B placed third at 78.0999 on the least memory of any candidate, 1,397.8 MiB. Gemma 2 2B, quantized to Q8_0 instead of Q4_K_M, repeated quantization's oldest lesson from a new angle: its decode gain under vector was only 1.826×, against roughly 3× for the Q4_K_M models, because Q8_0 doesn't reach the same SIMD path. 74% accuracy wasn't enough to make up the difference, and it finished at 61.9274. Orca Mini 3B saw the largest decode gain of any model in this story, 6.583×, but started so slow under scalar and scored so low on accuracy, 58%, that vector execution only lifted it to 52.4683.

Eight-model scalar and vector measurements

Scalar and vector benchmark results for the eight models in the second widening.
Model	Scalar → vector pp512	Scalar → vector tg128	Decode gain	Vector RSS	ARC-Easy, n=50	Total
1 · Qwen2.5 1.5B Q4_K_M	7.6824 → 59.9786	5.6919 → 17.2265	3.026×	1,838.7 MiB	76%	65.9176 → 82.8697
2 · Qwen2 1.5B Q4_K_M	7.7000 → 59.1846	5.6976 → 16.8156	2.951×	1,714.0 MiB	74%	65.2782 → 82.2176
3 · Llama 3.2 1B Q4_K_M	10.6776 → 82.0402	7.0795 → 21.7591	3.074×	1,397.8 MiB	64%	63.5027 → 78.0999
4 · Qwen2.5 3B Q4_K_M	3.6584 → 28.4679	2.9188 → 8.8583	3.035×	3,470.9 MiB	78%	58.6863 → 67.0322
5 · Llama 3.2 3B Q4_K_M	3.6258 → 28.1515	2.8118 → 8.6254	3.068×	3,454.6 MiB	72%	55.6109 → 63.6118
6 · Gemma 2 2B Q8_0	5.5196 → 28.7048	3.5435 → 6.4692	1.826×	2,871.1 MiB	74%	56.0770 → 61.9274
7 · Phi-4 Mini 3.8B Q4_K_M	3.1238 → 19.3956	2.3084 → 6.6995	2.902×	3,827.2 MiB	78%	56.3229 → 61.7203
8 · Orca Mini 3B Q4_K_M	0.8970 → 13.0483	0.8482 → 5.5836	6.583×	2,759.3 MiB	58%	42.9981 → 52.4683

Narrowing. Promote Qwen2.5 1.5B as the new vector candidate; eliminate the other seven from that shortlist. None of this transfers to the scalar profiler, where the same Q4_K_M kernels that helped under vector are the slow generic-C path quantization already taught us to avoid.

76% on 50 ARC-Easy items is a small-sample estimate, and Qwen2.5 1.5B was about to become our leading candidate on the strength of it, so we ran the same benchmark again at ten times the sample size. The result: 71.8%, with a 95% interval of 67.70–75.57%. The scoring function takes accuracy as one input alongside throughput and RSS, which we already had measured and held fixed, so both totals move by exactly the same 2.10 points the accuracy alone accounts for: the vector total falls to 80.7697, the scalar total to 63.8176. Nothing about the CPU kernel changed; only the sample size did.

Qwen2.5 1.5B Q4_K_M	n=50	n=500
ARC-Easy accuracy	76.0%	71.8% (67.70–75.57% CI)
Scalar total	65.9176	63.8176
Vector total	82.8697	80.7697

That gave us reason to hold Math-Expert and the final Qwen3.5 file to the same larger sample. Both already had a matched 500-item ARC-Easy run on file from the specialist search: Qwen3.5 at 58.8%, Math-Expert at 54.6%, well below either one's 50-item read. Lined up against Qwen2.5's own 500-item score, the picture splits cleanly by CPU configuration:

Matched ARC-Easy-500 comparison across the three finalists, scalar and vector totals.
Finalist, matched to n=500	ARC-Easy, n=500	Scalar total	Vector total
Qwen2.5 1.5B Q4_K_MVector leader	71.8% 67.70–75.57	63.8176	80.7697
Qwen3.5 0.8B Q4_0 finalScalar leader	58.8% 54.43–63.03	72.7895	76.8104
Math-Expert 0.6B Q4_K_M	54.6% 50.22–58.91	71.2324	75.1803

Under vector, Qwen2.5 1.5B leads all three finalists. Under scalar, the ranking reverses entirely: Qwen3.5 leads, Math-Expert follows, and Qwen2.5 falls to third, because the profiler kernel that runs Qwen2.5's Q4_K_M tensors is the same slow generic-C path that cost every k-quant in this story. Neither raw finalist we found first turns out to be the best answer in every regime; the honest one depends on which binary the final audit actually runs.

The combined figure retains the original 15-model search and adds the two tuned finalists, so the latest result remains visible beside the candidates that produced it:

Model search and tuned finalists

Every paired model, scalar versus vector

ScalarVector

Fig. 15 Seventeen GGUF results on one axis: the 15-model search and two fine-tuned finalists. The tuned Qwen2.5 has the highest vector total; the tuned Qwen3.5 has the highest scalar total.

Narrowing. Qwen2.5 1.5B becomes the vector-configuration recommendation, provisionally, pending the same physical-target run and embedded-behaviour check Qwen3.5 has already been through. Qwen3.5 0.8B remains the scalar-configuration recommendation, ahead of Math-Expert once both are read at the same sample size. The submission choice depends on which CPU configuration the supplied profiler actually runs, not on either finalist's small-sample score alone.

Method eight

Teaching the file to behave, since nothing else is in the room

Every method so far shrinks bytes or speeds up decode. This one changes what the model says: judges chat with the bare GGUF directly, no system prompt and no application layer standing between them and the model, so the tutoring persona has to travel inside the file or it doesn't exist during evaluation.

Fig. 16 The previous packaged Qwen3.5 model differed from its source quant only in GGUF metadata. The tuned finalist must receive and revalidate the same policy before submission.

We replace the chat template with a clean ChatML template that injects the Muta tutoring persona as the system turn whenever the client sends none, and always opens the assistant turn with an empty <think></think> block, forcing a direct answer instead of a long reasoning trace. That last part isn't cosmetic. A live four-prompt acceptance test with unrestricted thinking left both finalists spending their entire 256-token allowance on hidden reasoning and returning no usable answer at all, a minute of silence per prompt at audit-box speed. Disabling thinking fixed that. Sampling defaults (temperature 0.4, top-p 0.9, min-p 0.05, repeat penalty 1.05) are read straight from the file by llama-server, and the persona runs about 130 tokens, so a judge's first turn pays little extra prefill for it.

We verified this on llama.cpp's Jinja engine and llama-cpp-python's separate jinja2 path. The previous Qwen3.5 package injected the persona and stopped correctly, but the hardest proof prompt still failed. The fine-tuned finalists below have not yet repeated this battery; measured accuracy alone does not clear that requirement.

Method nine

Fine-tuning the two finalists

We trained 15 candidates across Qwen3.5 0.8B and Qwen2.5 1.5B, varying LoRA rank, BF16 LoRA versus QLoRA, data mixture, learning rate, and training length. Every survivor was exported to its deployment quantization and evaluated as a GGUF; training loss did not select a winner.

The first eight runs did not improve ARC-Easy-500. Most training examples were long solutions or tutor dialogue, while the profiler scores short answer continuations. The original data builder also omitted the leading space between Answer: and the candidate continuation, changing the BPE tokens used during training. We corrected both problems, filtered against held-out questions, and used only source training splits. The final Qwen2.5 mixture excludes OpenBookQA because its published licence is unclear.

Initial sweep

Eight runs · no promotion

Balanced and reasoning-heavy mixtures improved validation loss but did not improve the exact profiler task.

Metric-aligned sweep

Seven runs · two promotions

Raw multiple-choice continuations, corrected token boundaries, verified answers, and leakage checks improved both finalists.

Matched 500-item evaluation

Accuracy before and after tuning

ControlFine-tuned

Fig. 17 Qwen3.5 improved from 55.2% to 70.2%; Qwen2.5 improved from 74.4% to 77.8%. Both comparisons use matched exports and the same 500-item metric.

Fine-tuned GGUFs

Total score by CPU configuration

ScalarVector

Fig. 18 The scalar and vector rankings remain split: tuned Qwen3.5 leads scalar; tuned Qwen2.5 leads vector.

Matched control, performance, and memory results

Qwen3.5 gained 15.0 accuracy points and approximately 7.5 total points in either CPU configuration. Qwen2.5 gained 3.4 accuracy points and approximately 1.7 total points. Throughput and memory were effectively unchanged within each matched pair, so these score gains come from accuracy rather than a deployment trade-off.

Secondary held-out checks

Current decision

Fine-tuning raises both scores; the runtime split remains

The matched 500-item comparison still selects one model per CPU configuration. Fine-tuning raises both totals without materially changing speed or memory.

Scalar configuration, n=500

Fine-tuned Qwen3.5 0.8B · 80.3664

70.2% ARC-Easy, 13.60 tok/s, and an estimated 691 MiB profiler RSS.

Vector configuration, n=500

Fine-tuned Qwen2.5 1.5B · 84.1387

77.8% ARC-Easy, 17.44 tok/s, and an estimated 1,706 MiB profiler RSS.

Quantization decides the available CPU kernel; fine-tuning only changes accuracy on top of it. So the final choice still comes down to which CPU configuration the evaluator actually runs.

The receipts

Every experiment, adopted, rejected, and deferred

The measurements above produced the decisions below. Adopted changes appear first; use the filters to see rejected, neutral, and deferred tests.

Challenge requirements

Requirement status

Model selection also has to satisfy the published challenge requirements. Each entry pairs one requirement with what we have actually implemented.

Challenge requirements follow the Africa Deep Tech Challenge 2026 page.

Operational appendix

Profiler results and comparison tables

The underlying measurements, live profiling controls, stored results, and mixed-machine archive all stay available below.

Published snapshot. This read-only copy was rendered from the repository’s stored evidence. Profiling, promotion, and record deletion need the local dashboard server (./dashboard/start.sh).

Profiler metadata

Loading metadata…

Quick diagnostic run skip accuracy

Official profiler resultDirect GGUF campaignLoading campaign evidence…

Profiler-parity estimateCandidate screening with the scalar audit kernelLoading reconstructed profiler evidence…

Controlled vector proxyPaired scalar and vector measurements, seven-model ladderLoading the seven-model instruction-set comparison…

Controlled vector proxyPaired scalar and vector measurements, eight-model architecture screenLoading the eight-model architecture screen…

Website-relative sensitivityVector measurements with the cohort-relative formulaLoading sensitivity data…

Profiling…

0:00

Mixed-machine archiveHistorical profiler archiveDifferent machines and engine configurations; excluded from the campaign ranking.

Historical display only: S_total = 0.50·S_acc + 0.30·S_perf + 0.20·S_eff − 10·thermal · legacy S_perf = min(TPS / TPS_max, 1) × 100 · TPS_max = fastest stored run · S_eff = (7 GB − peak RAM) / 7 GB × 100 · crash or OOM ⇒ disqualified (0).

Stored model runs

Gate 2 · Chapter 1

Model research

Gate 1 qualified Muta for the next stage while exposing the limits of optimizing mainly for speed and memory. Gate 2 therefore began with an open compact-STEM search. The research below defined the candidate field; the next chapter records the completed comparison and the model selected for continued development.

Desk research · not target-laptop measurements

Small STEM models for an 8 GB laptop

Decision

The first balanced-STEM candidates to benchmark are MiniCPM5-2B, LFM2.5-2.6B, and LFM2.5-1.2B-Thinking. MiniCPM is the capability-first candidate; the 2.6B Liquid model is an efficiency-and-knowledge compromise; the 1.2B Thinking model is the low-memory, speed-first baseline. For mathematics-only experiments, add VibeThinker-1.5B and Falcon-H1-Tiny-R-0.6B. These are deployment priorities, not a claim that one checkpoint dominates every task. ^[1] ^[3] ^[7] ^[15] ^[17]

Scope and evidence

This assessment covers the publicly accessible information retained in the supplied research snapshot. Assumptions are CPU-only, text-only inference, one concurrent request, quantized GGUF weights, and an initial context of roughly 4,096 tokens. No exact laptop CPU, RAM bandwidth, power limit, operating-system memory use, or local model measurement was available. No model was executed on the target laptop.

An 8 GB capacity limit cannot establish a tokens-per-second result. Speed classes below are qualitative feasibility judgments, not measured throughput ranges. Published tests used specific hardware, runtimes, quantizations, context lengths, and workloads. Context, threading, and quantization must be held explicit when comparing results. ^[4] ^[8] ^[32]

Local AI Zone was inspected as a discovery resource. Its dynamic model table did not expose a complete machine-readable inventory in the accessible page; its presence is not evidence that every entry was audited. Rankings instead rely on developer model cards, actual GGUF repositories, research reports, and selected evaluation-provider pages. ^[33]

Ranked practical shortlist

Ratings summarize the strength of the available evidence within the compact-model class, not reliability guarantees. A 5 / 5 rating does not mean frontier-model performance. “Unproven” means insufficient broad-science evidence, not demonstrated inability. Ranking prioritizes combined STEM capability subject to a local throughput gate; a slower candidate must be demoted after measurement.

Rank / model	Size	Math	Science / STEM	Expected speed class	Assessment
1. MiniCPM5-2B, Q4_K_M	2.52B total	5 / 5	5 / 5	10–15 plausible, CPU-dependent	Capability-first candidate; depth makes measurement essential. ^[1]
2. LFM2.5-2.6B, QAD-Q4_0	2.6B class	4 / 5	4 / 5	Promising efficiency class; verify 10–15 locally	Strong balance of scientific evidence and edge inference. ^[3] ^[4] ^[5]
3. LFM2.5-1.2B-Thinking, Q4_0	1.17B	4 / 5	2 / 5	Strong speed-first candidate	Lowest-risk starting point for modest CPU and RAM; the knowledge limitation matters. ^[7] ^[8]
4. Qwen3.5-2B, Q4_K_M	2B language model	3 / 5	4 / 5	10–15 plausible, CPU-dependent	Useful broad-STEM reference, not the sole small-model option. ^[10]
5. MiniCPM5-1B, Q4_K_M	1.08B total	4 / 5	2 / 5	Strong small-model candidate	Attractive mathematics-to-size trade-off; weaker science evidence. ^[12] ^[13]
6. VibeThinker-1.5B, Q4_K_M	1.5B class	5 / 5	Unproven	Small decode workload; long-answer risk	Competition-math specialist, not a default science tutor. ^[15] ^[16]
7. Falcon-H1-Tiny-R-0.6B, Q4_K_M	0.6B	5 / 5	Unproven	Tiny weights; long-answer risk	Outstanding reported math per parameter; not necessarily low latency. ^[17] ^[18]
8. OpenReasoning-Nemotron-1.5B, Q4-class	1.5B	4 / 5	2 / 5	Small decode workload; long-answer risk	Secondary reasoning baseline. ^[19]
9. Qwen3-1.7B, Q4-class	1.7B	3 / 5	3 / 5	Useful small-model baseline	Retain as a control rather than assume newest is faster. ^[25]

Benchmark evidence ledger

This is not a controlled leaderboard. Different sources use different prompts, sampling, repeat counts, output budgets, and model precision. Scores are percentages. An average of single-attempt results across repeated runs is not the same as choosing the best of multiple answers.

Model	Reported mathematics result	Scientific result	Qualification
MiniCPM5-2B	MATH-500 94.6; AIME 2025 86.5; AIME 2026 86.5	GPQA Diamond 70.2	OpenBMB table; GPQA attributed there to Artificial Analysis. Not a Q4, short-budget result. ^[1]
LFM2.5-2.6B	AIME 2025 51.87 in Liquid’s card; 41.9 in OpenBMB’s reproduced comparison	GPQA Diamond 55.8 in OpenBMB’s cited comparison	The math discrepancy illustrates protocol sensitivity. ^[1] ^[3]
LFM2.5-1.2B-Thinking	MATH-500 87.96; AIME 2025 31.73	GPQA Diamond 37.86	Publisher evaluation; not guaranteed Q4_0 performance. ^[7]
Qwen3.5-2B	HMMT February 2025 22.9	GPQA 51.6 in Qwen’s table	Do not relabel HMMT as AIME. ^[10]
MiniCPM5-1B	MATH-500 91.60; AIME 2025 40.42	GPQA Diamond 26.26	OpenBMB’s published comparison image. ^[13]
VibeThinker-1.5B	AIME 2024 80.3; AIME 2025 74.4	Not established in the inspected evidence	Developer explicitly recommends competition math and algorithmic coding. ^[15]
Falcon-H1-Tiny-R-0.6B	MATH-500 94.0; AIME 2025 pass@1 67.3	Not established in the inspected evidence	These are single-sample results, not its separate pass@16 figures. ^[17]
OpenReasoning-Nemotron-1.5B	AIME 2024 55.5; AIME 2025 45.6	GPQA 31.6	NVIDIA model-card results. ^[19]

MiniCPM5-2B uses a standard Llama-compatible architecture. Its name refers approximately to non-embedding parameters: the complete model has 2,516,756,480 parameters and 42 layers. Its higher capability is therefore a reason to test it, not evidence that its speed matches an arbitrary 2B model. ^[1] ^[35]

A mathematics benchmark mainly establishes performance on those mathematical tasks. Selecting an everyday science tutor additionally requires testing factual accuracy, explanations, uncertainty, physical units, and multi-turn consistency. This is why the tiny mathematics specialists are not automatically ranked first overall.

Verified weight files

Sizes are rounded decimal file sizes from the listed repositories, not total inference RAM. A community conversion is not a new independently trained model. Different converters may choose different tensor precisions even under similar quantization labels.

Model / file selection	Download size	Repository provenance
MiniCPM5-2B Q4_K_M	1.56 GB	Official OpenBMB. ^[2]
LFM2.5-2.6B QAD-Q4_0	1.59 GB	Official Liquid AI. ^[6]
LFM2.5-2.6B Q4_K_M	1.67 GB	Official Liquid AI. ^[6]
LFM2.5-1.2B-Thinking Q4_0	696 MB	Official Liquid AI. ^[9]
LFM2.5-1.2B-Thinking Q4_K_M	731 MB	Official Liquid AI. ^[9]
Qwen3.5-2B Q4_K_M	1.28 GB	Unsloth conversion; text weights, not the optional vision component. ^[11]
MiniCPM5-1B Q4_K_M	688 MB	Official OpenBMB. ^[14]
VibeThinker-1.5B Q4_K_M	1.12 GB	DevQuasar community conversion. ^[16]
Falcon-H1-Tiny-R-0.6B Q4_K_M	374 MB	Official TII. ^[18]

Quantization choice

For LFM2.5-1.2B-Thinking, compare Q4_0 with Q4_K_M rather than assuming the latter is always preferable. The 35 MB saving is small, but quantization also affects kernel efficiency. Liquid’s published CPU performance for this Thinking model used Q4_0. ^[8] ^[9]

QAD-Q4_0 is a different consideration. Liquid’s quantization-aware distillation release covers the 2.6B model and selected smaller checkpoints, including 1.2B-Instruct; it does not list 1.2B-Thinking. Do not apply a QAD quality claim to an ordinary Q4_0 Thinking file. ^[5]

What published speed measurements establish

Model	Hardware / setup	Published decode throughput
LFM2.5-1.2B-Thinking	Ryzen AI Max 395 CPU; llama.cpp Q4_0; 1k prefill and 100-token decode workload	235 tok/s; 853 MB reported memory. ^[8]
LFM2.5-2.6B	Ryzen AI Max+ 395 CPU; publisher Q4_K_M, 4k-context comparison	113 tok/s; under 2.5 GB reported memory. ^[4]

These are high-end hardware results, not forecasts for a budget laptop. The different workloads also prevent treating the two numbers as a clean speed ratio. Conversely, parameter count alone cannot prove that every 3–4B model misses 10–15 tok/s.

Answer time = load or queue time + prompt processing time + (reasoning tokens + answer tokens) / decode rate

At 10–15 tok/s, 600 generated tokens require 40–60 seconds, 2,000 require 133–200 seconds, and 8,000 require about 9–13 minutes, before other overhead. These are arithmetic examples, not predicted average responses. Falcon’s training report describes mean generations falling toward 8k tokens in that training setting; this is a warning to measure output length, not a claim that every tutoring answer is 8k. ^[17]

Quality-first stretch candidates

These belong in the evaluation only after the smaller candidates are measured. Quantized weights can be feasible in 8 GB, but operational headroom and throughput remain CPU- and context-dependent.

Model	Size class	Why it is relevant	Reason not to make it the initial low-end default
VibeThinker-3B	3B	Reported AIME 2026 94.3 and GPQA 70.2. ^[20]	Potentially extensive reasoning; latency must be tested.
Qwen3.5-4B	4B language	GPQA Diamond 76.2; strong broad-STEM candidate. ^[21]	Larger inference workload than the main shortlist.
Granite-4.2-3B	3B	AIME 2025 78.33; GPQA 54.80; low-effort reasoning option. ^[22]	An option to test, not established fast on the target CPU.
Ministral-3-3B-Reasoning-2512	3B class	AIME 2025 72.1; GPQA Diamond 53.4. ^[23]	Same hardware uncertainty; distinguish reasoning from instruct checkpoint.
Phi-4-mini-reasoning	3.8B	MATH-500 94.6 and GPQA 52.0 in its model card. ^[24]	Useful math baseline; no longer an uncontested tiny-model winner.
SmolLM3-3B	3B	Thinking-mode AIME 2025 36.7; GPQA 41.7. ^[25]	Less compelling STEM evidence for its size than newer alternatives.

Other screened candidates

Qwen2.5-Math-1.5B-Instruct: still a useful tool-assisted mathematics baseline. The frequently quoted 79.7 score is on MATH with tool-integrated reasoning; it must not be compared directly with unassisted MATH-500 scores. It is not the evidence-based automatic “best tiny math model” in this wider shortlist. ^[28]

DeepScaleR-1.5B-Preview: an alternative to the original DeepSeek-R1-Distill-Qwen-1.5B, reporting AIME 2024 43.1 and MATH-500 87.8. Worth retaining as an experimental baseline rather than a first deployment choice. ^[29]

MobileLLM-R1.5-950M: mathematically interesting, but its model card labels a FAIR noncommercial license. That warrants a separate licensing decision before considering it for a commercial tutoring product. ^[26]

K2-Horizon-0.9B: competitive tiny mathematics results, but its reported GPQA 27.3 does not establish a stronger broad-science option. ^[27]

LFM2-350M-Math: a genuinely smaller English mathematics specialist. Its positioning does not establish general science coverage; keep it in a narrow-task experiment, not as the sole tutor. ^[34]

Phi-4-mini-flash-reasoning: the “flash” name and published GPU speedups are not CPU throughput evidence. The official implementation and performance discussion center on GPU-oriented inference; verify a functioning local CPU backend before including it operationally. ^[30]

Gemma-4-E2B-it: the effective-size label is not a statement that only 2B weights must be stored. Its deployment footprint needs to be checked against the actual conversion, not inferred from the name. Liquid’s comparison shows materially higher memory use than its 2.6B model. ^[4] ^[31]

Likewise, a mixture-of-experts model advertised as “A1B” or “A3B” still has a total-weight storage requirement. Do not equate active parameters with resident-memory needs when selecting a comfortable 8 GB configuration.

Laptop evaluation protocol

Begin with one loaded model, one request, text-only mode, a 4,096-token context, and the developer’s actual chat template. Use appropriate publisher sampling defaults instead of assuming one temperature or a universal /no_think command applies to every model. Compare a default-quality run with a deployment-budget run, and label truncated answers as failures rather than scoring incomplete reasoning as correct.

A current llama.cpp synthetic benchmark can vary CPU threads and existing context depth:

./llama-bench -m /path/to/model.gguf -ngl 0 -p 512 -n 128 \
  -d 0,2048 -t 2,4 -r 3

The path is a placeholder. Select thread counts supported by the laptop. Inspect the tg rows for decode rate and the pp rows for prompt processing. The benchmark excludes tokenization and sampling overhead; use real application requests for time-to-first-token and time-to-complete-answer. ^[32]

Suggested evaluation: 60 held-out questions, comprising 20 mathematics questions, 20 physics, chemistry, biology, or engineering questions, 10 unit or numerical-error traps, and 10 multi-turn tutoring interactions. Include both easy explanations and hard problems. Run stochastic cases multiple times. Record correctness, explanation quality, refusal or uncertainty quality, output tokens, decode speed, first-token latency, final-answer latency, peak process memory, and system swapping.

For a deployment gate, require the chosen throughput floor on sustained runs with representative context, not just an empty prompt. Rank the passing models by correct completed answers within the chosen latency budget. An explicit initial output cap can be 512–1,024 tokens for ordinary questions, with a separately labeled longer mode; this is an experimental policy and may lower accuracy on difficult reasoning tasks.

Arithmetic and symbolic tools should be judged on two separate outcomes: whether the model formulated the right problem, and whether the tool solved that formulation correctly. A calculator cannot repair an incorrect physical assumption or an incorrectly constructed equation. For science tutoring, a compact model plus curated local reference material is an architectural option to test, not a guarantee of factual accuracy.

What advances to measurement

Selection rule

Benchmark MiniCPM5-2B against LFM2.5-2.6B for a single balanced-STEM default. Keep LFM2.5-1.2B-Thinking Q4_0 as the speed-and-memory fallback. Add VibeThinker-1.5B or Falcon-H1-Tiny-R-0.6B only when a separate mathematics mode can tolerate longer reasoning. Re-rank after measuring actual CPU throughput and correct-answer latency. This recommendation deliberately does not claim that small weight files alone satisfy the interactive-tutoring requirement.

Sources

These are the sources retained with the supplied research snapshot. Dynamic model repositories and evaluation pages may change. The model, quantization, and software revision should therefore be pinned for reproducible local testing.

OpenBMB. MiniCPM5-2B model card.
OpenBMB. MiniCPM5-2B official GGUF files.
Liquid AI. LFM2.5-2.6B model card.
Liquid AI. LFM2.5-2.6B: Deploy Agents Everywhere.
Liquid AI. Quantization-Aware Distillation for Edge Deployment.
Liquid AI. LFM2.5-2.6B official GGUF files.
Liquid AI. LFM2.5-1.2B-Thinking model card.
Liquid AI. LFM2.5-1.2B-Thinking: On-Device Reasoning Under 1GB.
Liquid AI. LFM2.5-1.2B-Thinking official GGUF files.
Qwen. Qwen3.5-2B model card.
Unsloth. Qwen3.5-2B GGUF files.
OpenBMB. MiniCPM5-1B model card.
OpenBMB. MiniCPM5-1B published benchmark table.
OpenBMB. MiniCPM5-1B official GGUF files.
WeiboAI. VibeThinker-1.5B model card.
DevQuasar. VibeThinker-1.5B community GGUF files.
Technology Innovation Institute. Falcon-H1-Tiny research report; Tiny-R section.
Technology Innovation Institute. Falcon-H1-Tiny-R-0.6B official GGUF files.
NVIDIA. OpenReasoning-Nemotron-1.5B model card.
WeiboAI. VibeThinker-3B model card.
Qwen. Qwen3.5-4B model card.
IBM. Granite-4.2-3B model card.
Mistral AI. Ministral-3-3B-Reasoning-2512 model card.
Microsoft. Phi-4-mini-reasoning model card.
Hugging Face. SmolLM3-3B model card.
Meta. MobileLLM-R1.5-950M model card and license label.
IFM. K2-Horizon-0.9B model card.
Qwen. Qwen2.5-Math release report.
Agentica. DeepScaleR-1.5B-Preview model card.
Microsoft. Phi-4-mini-flash-reasoning model card.
Google. Gemma-4-E2B-it model card.
ggml-org. llama-bench documentation.
Local AI Zone. GGUF model discovery directory.
Liquid AI. LFM2-350M-Math model card.
Artificial Analysis. MiniCPM5-2B evaluation and release information.

Gate 2 · Chapter 2

Model evaluation and ranking

The desk research produced a broader set of small models, but published benchmark claims could not answer the question I actually had: which artifact could teach reliably within Muta’s CPU and memory constraints? I therefore tested the executable field on the same resource proxies, then replayed both general STEM work and the prompts that had exposed Gate 1 failures.

Completed controlled campaign

From benchmark promise to usable answers

Thirteen GGUF artifacts completed performance and ARC-Easy-500 measurement. Twelve also completed a matched 100-prompt STEM battery and both ten-prompt Gate 1 replays. A fourteenth candidate, Spark-X2.5 1.7B, could not load in the pinned runtime and was excluded before measurement. It is recorded as runtime-excluded, not assigned zeros.

The tests answer different questions. ARC-Easy estimates broad multiple-choice knowledge through option likelihood. The STEM battery checks the complete raw completion on a balanced set of mathematics and science prompts. The Gate 1 replay checks only the answer delivered to the user, using the exact failure cases from the earlier judging feedback. I kept these rankings separate because a model can know an answer, expose it only inside unfinished reasoning, or fail to present it in a usable tutoring response.

Candidate field

The field includes general-purpose compact models, reasoning specialists, two quantization controls, and the fine-tuned Gate 1 model as a control. File size below is the GGUF size, not inference memory.

Model	Role	Quantization	File	Licence	Runtime gate
Muta Tutor Qwen2.5 1.5B	Gate 1 control	Q4_K_M	986 MB	Apache-2.0	Measured
LFM2.5 1.2B Thinking	Challenger	Q4_0	696 MB	LFM Open License v1.0	Measured
MiniCPM5 1B	Challenger	pure Q4_0	613 MB	Apache-2.0	Measured
Qwen3.5 2B	Challenger	Q4_0	1,215 MB	Apache-2.0	Measured
Qwen3 1.7B	Challenger	Q4_0	1,057 MB	Apache-2.0	Measured
LFM2.5 2.6B	Challenger	Q4_0	1,594 MB	LFM Open License v1.0	Measured
Spark-X2.5 1.7B	Architecture watchlist	Q4_K_M	1,107 MB	Apache-2.0	Excluded: unsupported architecture
MiniCPM5 2B	Challenger	Q4_K_M	1,561 MB	Apache-2.0	Measured
LFM2.5 2.6B QAD	Challenger	QAD-Q4_0	1,594 MB	LFM Open License v1.0	Measured
MiniCPM5 1B	Quantization control	Q4_K_M	688 MB	Apache-2.0	Measured
Qwen3.5 2B	Quantization control	Q4_K_M	1,281 MB	Apache-2.0	Measured
VibeThinker 1.5B	Challenger	Q4_K_M	1,117 MB	MIT	Measured
Falcon-H1-Tiny-R 0.6B	Challenger	Q4_K_M	374 MB	Falcon LLM License	Measured
OpenReasoning Nemotron 1.5B	Challenger	Q4_K_M	986 MB	CC-BY-4.0	Measured

Protocol and evidence boundaries

Scalar uses the supplied profiler configuration already defined in Gate 1; this does not prove that every emitted CPU instruction is scalar. Vector uses the corresponding portable SIMD configuration on the same two physical GCP cores. Scalar is the primary resource protocol and vector is a secondary CPU proxy.

Performance and memory

Each artifact ran five llama-bench repetitions at pp512 and tg128 with two threads and no GPU. RSS is measured across the benchmark process tree. The reconstructed profiler estimate adds a fixed 45 MiB Python-root allowance.

ARC-Easy-500

One shared acc_norm result per artifact came from the official profiler’s separately compiled accuracy path. It is not a scalar/vector accuracy pair and does not execute chat answers.

100-prompt STEM battery

The matched Mac M4 Pro run used Metal offload with 99 GPU layers, 50 mathematics and 50 science prompts evenly split between multiple-choice and written responses, context 2,048, temperature 0, seed 42, two CPU threads, and 256/512 output tokens. It used raw completion without the embedded chat template.

Ten-prompt Gate 1 replay

Five automated and five human-judge prompts ran on GCP scalar with two CPU threads and no GPU. The paired Mac M4 Pro replay used Metal offload with 99 GPU layers. Both used context 4,096, a 256 MiB host prompt-cache cap, temperature 0, seed 3407, top-p 1, 1,024 output tokens, and the embedded chat template without an external system prompt.

The GCP host had four logical CPUs, two physical cores, and about 8 GiB RAM. It is a controlled cloud proxy, not the physical submission laptop. Package temperature was unavailable, so the reconstructed totals apply no thermal penalty. They are not end-to-end official-profiler scores.

Reconstructed total = 0.50 × ARC-Easy + 0.30 × min(tg128 / 15, 1) × 100 + 0.20 × (7 GiB − estimated RSS) / 7 GiB × 100.

ARC-Easy-500: the control retains the strongest knowledge proxy

Muta Tutor Qwen2.5 records 77.8%, with a 95% Wilson interval of 73.95–81.22%. MiniCPM5 2B is second at 67.6%; Qwen3 1.7B and Qwen3.5 Q4_K_M are tied at 65.2%. The result supports Qwen2.5 as the strongest candidate on this particular likelihood-based task, not as a general tutoring verdict.

ARC-Easy accuracy, n=500. All thirteen executable artifacts use the same official-profiler accuracy path. Bars show acc_norm; exact confidence intervals are in the table below.

Model	Accuracy	95% CI	Samples
Muta Tutor Qwen2.5 1.5B Q4_K_M	77.8%	73.95–81.22%	500
MiniCPM5 2B Q4_K_M	67.6%	63.38–71.55%	500
Qwen3 1.7B Q4_0	65.2%	60.92–69.25%	500
Qwen3.5 2B Q4_K_M	65.2%	60.92–69.25%	500
Qwen3.5 2B Q4_0	65.0%	60.72–69.05%	500
MiniCPM5 1B Q4_K_M	56.0%	51.62–60.29%	500
MiniCPM5 1B pure Q4_0	55.8%	51.42–60.09%	500
OpenReasoning Nemotron 1.5B Q4_K_M	53.6%	49.22–57.93%	500
LFM2.5 1.2B Thinking Q4_0	46.0%	41.68–50.38%	500
LFM2.5 2.6B QAD-Q4_0	45.4%	41.09–49.78%	500
LFM2.5 2.6B Q4_0	43.6%	39.32–47.98%	500
VibeThinker 1.5B Q4_K_M	39.2%	35.02–43.55%	500
Falcon-H1-Tiny-R 0.6B Q4_K_M	30.4%	26.53–34.57%	500

Per-question ARC answers are unavailable. The retained accuracy.jsonl has thirteen aggregate rows with sample counts and task metrics, but no item IDs, documents, options, targets, predictions, or log-likelihoods. The local archive and read-only VM audit found no separate sample file. Reconstructing 6,500 model choices would require a new instrumented run; this report does not invent them.

Resource score: two CPU configurations produce different leaders

MiniCPM5 1B pure Q4_0 leads the scalar reconstruction at 75.8713 because its 18.3038 tok/s decode reaches the performance cap while estimated RSS remains 727.1 MiB. Under vector execution, Muta Tutor Qwen2.5 leads at 84.1383. Its accuracy has not changed between the two bars: only measured throughput and RSS change.

ScalarVector

Reconstructed competition-score proxy. Scalar and vector use separately measured five-repeat tg128 and process-tree RSS, with the same ARC-Easy-500 accuracy for each artifact. Temperature is unavailable and no penalty is applied.

Model	Scalar pp512 / tg128	Scalar RSS measured / estimated	Scalar total	Vector pp512 / tg128	Vector RSS measured / estimated	Vector total
MiniCPM5 1B pure Q4_0	29.7586 / 18.3038	682.1 / 727.1 MiB	75.8713	95.6728 / 30.8010	1,043.0 / 1,088.0 MiB	74.8643
LFM2.5 1.2B Thinking Q4_0	21.0637 / 13.7819	759.3 / 804.3 MiB	68.3197	67.7881 / 23.7017	1,305.8 / 1,350.8 MiB	69.2310
Muta Tutor Qwen2.5 1.5B Q4_K_M	7.8080 / 5.6393	1,072.1 / 1,117.1 MiB	67.0617	61.6019 / 17.2784	1,661.6 / 1,706.6 MiB	84.1383
MiniCPM5 1B Q4_K_M	14.6566 / 9.5460	753.6 / 798.6 MiB	64.8637	112.7986 / 27.7206	1,065.9 / 1,110.9 MiB	74.9004
Qwen3 1.7B Q4_0	13.6418 / 7.8070	1,167.2 / 1,212.2 MiB	64.8317	44.7098 / 15.3858	1,896.0 / 1,941.0 MiB	77.1843
Falcon-H1-Tiny-R 0.6B Q4_K_M	17.2442 / 13.6919	452.5 / 497.5 MiB	61.1957	119.3438 / 35.7896	720.9 / 765.9 MiB	63.0630
Qwen3.5 2B Q4_0	12.6073 / 6.1956	1,322.9 / 1,367.9 MiB	61.0746	42.4621 / 13.0953	1,998.0 / 2,043.0 MiB	72.9902
Qwen3.5 2B Q4_K_M	7.0812 / 4.4225	1,385.5 / 1,430.5 MiB	57.4536	40.3176 / 11.8554	1,875.8 / 1,920.8 MiB	70.9515
MiniCPM5 2B Q4_K_M	5.0992 / 3.9663	1,606.7 / 1,651.7 MiB	57.1241	40.1444 / 12.3598	2,515.5 / 2,560.5 MiB	71.3754
OpenReasoning Nemotron 1.5B Q4_K_M	7.7721 / 5.7720	1,072.2 / 1,117.2 MiB	55.2269	60.4724 / 17.1991	1,661.2 / 1,706.2 MiB	72.0394
LFM2.5 2.6B QAD-Q4_0	8.8444 / 6.1789	1,664.1 / 1,709.1 MiB	50.2891	28.9718 / 10.7653	2,961.4 / 3,006.4 MiB	55.8421
LFM2.5 2.6B Q4_0	8.8536 / 6.2264	1,664.4 / 1,709.4 MiB	49.4834	28.8601 / 10.3592	2,961.5 / 3,006.5 MiB	54.1297
VibeThinker 1.5B Q4_K_M	7.8076 / 5.7652	1,197.4 / 1,242.4 MiB	47.6639	60.2514 / 16.7253	1,786.3 / 1,831.3 MiB	64.4903

The STEM battery changes the ranking

Qwen3.5 2B Q4_K_M leads the matched Mac screen with 77 strict passes and 95 centrally correct answers. Its Q4_0 sibling follows at 71/94. Muta Tutor Qwen2.5 places fifth on strict completion at 57 and fourth on central correctness at 80. The gap matters: it shows that many answers contain the right core result but fail the complete-response policy through contradictory working, formatting, unfinished planning, or missing requested explanation.

Strict complete passCore answer correct

Matched Mac STEM battery, 100 prompts. All returned raw-completion text is graded. Falcon has no STEM responses and is omitted rather than plotted as zero.

Rank	Model	Strict pass	Core correct
1	Qwen3.5 2B Q4_K_M	77	95
2	Qwen3.5 2B Q4_0	71	94
3	LFM2.5 2.6B QAD-Q4_0	60	75
4	Qwen3 1.7B Q4_0	59	81
5	Muta Tutor Qwen2.5 1.5B Q4_K_M	57	80
6	LFM2.5 2.6B Q4_0	52	60
7	LFM2.5 1.2B Thinking Q4_0	41	53
8	OpenReasoning Nemotron 1.5B Q4_K_M	35	60
9	MiniCPM5 2B Q4_K_M	17	35
10	VibeThinker 1.5B Q4_K_M	16	22
11	MiniCPM5 1B Q4_K_M	11	31
12	MiniCPM5 1B pure Q4_0	6	16
—	Falcon-H1-Tiny-R 0.6B Q4_K_M	No Mac STEM responses; unranked

The earlier GCP STEM captures do not enlarge this ranking. They comprise nine complete original scalar sets, a 35-prompt Qwen3.5 fragment, a separate overlapping 28-prompt restart, and complete vector sets only for Qwen3.5 Q4_K_M and VibeThinker. They remain accessible in the response explorer as distinct attempts.

The Gate 1 replay tests whether the answer reaches the student

The known ten-prompt replay is small and includes the Two Sigma prompt twice, so it is not an unseen or independent benchmark. It is still useful because the prompts reproduce the exact presentation failures that mattered in Gate 1. Only final text earns rubric points; separate reasoning and unclosed thinking do not.

Muta Tutor Qwen2.5 leads the GCP scalar replay at 47/100 and is the only model to deliver a finished final answer on all ten prompts. It is not uniformly strong: the model earns 8/10 on proportionality, 1/10 on the mastery differential equation, 0/10 on the underdetermined capacity problem, 2/10 on the chalk calculation, 10/10 on photosynthesis, then 3/10, 6/10, 0/10, 8/10, and 9/10 across the five human-judge prompts. The selection therefore retains visible weaknesses rather than treating 47 as a pass mark.

GCP scalar Gate 1 replay, local rubric points out of 100. Falcon returned five responses before a parser failure and remains unranked. Nemotron completed all ten, but unclosed thinking left no creditable final output and therefore produces a reviewed 0/100.

Finished final answers out of ten. This chart separates response delivery from rubric quality. A finished answer can still be wrong; an absent final answer earns no Gate 1 rubric credit.

Rank	Model	Captured	Finished finals	Rubric score
1	Muta Tutor Qwen2.5 1.5B Q4_K_M	10/10	10	47
2	Qwen3 1.7B Q4_0	10/10	3	28
2	VibeThinker 1.5B Q4_K_M	10/10	2	28
4	LFM2.5 1.2B Thinking Q4_0	10/10	2	25
5	MiniCPM5 1B pure Q4_0	10/10	3	24
6	LFM2.5 2.6B Q4_0	10/10	2	20
6	LFM2.5 2.6B QAD-Q4_0	10/10	2	20
6	MiniCPM5 2B Q4_K_M	10/10	2	20
6	Qwen3.5 2B Q4_K_M	10/10	2	20
10	Qwen3.5 2B Q4_0	10/10	1	10
11	MiniCPM5 1B Q4_K_M	10/10	1	6
12	OpenReasoning Nemotron 1.5B Q4_K_M	10/10	0	0
—	Falcon-H1-Tiny-R 0.6B Q4_K_M	5/10	0	Failed; unranked

The paired Mac replay is supplementary development evidence. It completes for the same twelve models, while Falcon stops after three responses. Its ranking is not substituted into the GCP scalar result because backend behavior differs.

Why the model with the best single proxy is not automatically selected

Three leaders emerge. MiniCPM5 1B pure Q4_0 has the best scalar resource reconstruction, but only 6 strict STEM passes and 24 Gate 1 rubric points. Qwen3.5 2B Q4_K_M has the strongest raw-completion STEM result, but it finishes only two of ten GCP chat answers and scores 20/100. Muta Tutor Qwen2.5 combines the strongest ARC result, the highest GCP Gate 1 score, complete final-answer delivery, and the best vector reconstruction.

That balance is enough to promote one development path, not enough to certify a submission. Qwen2.5 still needs stronger written reasoning, a verified multilingual tutoring check, repeated physical-target measurement including temperature, and final licence and eligibility review.

Development decision

Continue with fine-tuned Muta Tutor Qwen2.5 1.5B Q4_K_M

Qwen2.5 remains the Gate 2 control for fine-tuning and product validation. Qwen3.5 Q4_K_M is retained as the behavior challenger because of its STEM result. Neither conclusion is a guarantee of competition success, and the physical target can still change the runtime decision.

Read the retained responses

The explorer loads one model and one test at a time, so the full archive does not delay the report. It preserves prompt text, unedited answer text, separate reasoning where the runtime returned it, completion state, and the manual rubric decision. ARC-Easy is aggregate-only for the limitation described above.

Test Model Prompt

Loading the evidence index…

Raw campaign files

Browse the published source files

Loading the published file index…

Gate 2 · Chapter 3

Improving Accuracy: Model fine-tuning

Fine-tuning was treated as a sequence of controlled interventions: freeze the data and lineage, train small pilots, expand only guarded candidates, export identical GGUFs, and compare them against unchanged controls.

What the campaign was designed to answer

The Gate 1 judges exposed failures that a narrow answer benchmark could miss: unfinished reasoning, wrong units, arithmetic drift, weak explanations, unsafe analogies, and poor adaptation to a learner. The Gate 2 campaign therefore measured both answer correctness and tutoring behaviour. The target was a small science-and-mathematics tutor, not a past-question lookup engine.

Method

BF16 LoRA was the executed method. QLoRA was retained as a preregistered memory-control branch, but no completed QLoRA result is in the evidence ledger.

Lineage

Runs were separated into fresh Qwen, recovered Muta, warm-pilot continuation, and a native DeepSeek control. A parent, adapter, optimizer state, schedule and host were recorded for every run.

Controls

Previous Muta, untouched Qwen2.5, historical warm, and untouched DeepSeek were kept beside the trained candidates. Controls were not silently replaced by a fine-tuned file.

Evidence boundary

Loss is a development signal. It is not an accuracy score, a tutoring grade, or an ADTC CPU result. Host and backend effects are reported separately from hyperparameters.

Data, verification and provenance

The larger private corpus contained 300,350 verified rows for the full-data runs. Its text remains private because source rights and competition-use terms were not equivalent to redistribution rights. The science-tutor release was a separate, reviewed mixture designed around conceptual understanding, quantitative reasoning, experiments, misconception repair, adaptive dialogue, scientific judgment and accessible explanations across mathematics, physics, chemistry, biology and Earth science.

Artifact	Rows	Use	Boundary
Private full-data artifact	300,350	Three BF16 LoRA runs	Private; not published
Science-tutor train	8,002	Four pilots; P2 refinement	Reviewed and frozen
Science-tutor development	1,577	Selection and regression guards	Separate from final holdout
Science-tutor final holdout	1,591	Held-out science comparison	Sealed before finalist testing

Candidate materials included source-screened science and dialogue resources, exam-style STEM material, and teacher-model drafts. Drafts were not treated as truth: calculations were recomputed; unsupported, contradictory, malformed, or unresolved examples were rejected. Distillation-style sources were quarantined unless a row passed the final manifest and verification rules. The separate 100-item STEM evaluation set (50 mathematics, 50 science) was published privately as Muta-STEM-100; it contains no model responses.

Twenty-thousand-row pilot sweep

Eight pilots used 20,000 training rows and 5,000 validation rows, 313 optimizer steps, effective batch 64, a 512-token maximum sequence, seed 3407, warmup/cosine scheduling, rank-specific α, dropout 0, and attention/MLP projection targets. Four started from fresh Qwen weights; four started from the recovered Muta parent.

Pilot	Init.	Rank	LR	Final dev loss
warm-r16-lr2e5	Muta	16	2 × 10⁻⁵	0.217060
warm-r32-lr1e5	Muta	32	1 × 10⁻⁵	0.339828
clean-r32-lr1e5	Qwen	32	1 × 10⁻⁵	0.346829
warm-r16-lr1e5	Muta	16	1 × 10⁻⁵	0.578009
clean-r16-lr1e5	Qwen	16	1 × 10⁻⁵	0.593257
warm-r32-lr5e6	Muta	32	5 × 10⁻⁶	0.685911
warm-r16-lr5e6	Muta	16	5 × 10⁻⁶	0.961763
warm-r8-lr5e6	Muta	8	5 × 10⁻⁶	1.151685

The warm-r16-lr5e6 lineage was selected for the full-data comparison as a practical lineage decision, not as proof that its lower pilot loss made it the best tutor. The complete pilot loss gallery is retained in the repository at provenance/results/all-training-loss-curves-20260919/eight-pilots.svg.

Full-data fine-tuning

Three full runs used the same verified 300,350-row private artifact. The selected checkpoint was step 4693. Each run was followed through adapter verification, merge, quantization and a single-copy Mac delivery; the resulting GGUFs were tested with the same request set.

Run	Initialization	Rank / LR	Final dev loss	Disposition
Clean	Fresh Qwen2.5-1.5B-Instruct	16 / 1 × 10⁻⁵	0.018999	Exported
Warm	Recovered Muta parent	16 / 5 × 10⁻⁶	0.022473	Exported
Pilot continuation	Exact warm-r16-lr5e6 adapter; fresh optimizer/schedule	16 / 5 × 10⁻⁶	0.022921	Exported

Loss fell sharply in the larger runs, but the values are not directly comparable with the 20K pilot losses because the data volume, stage, host and training history changed. The clean, warm and continuation curves remain available under provenance/training/full-best-*/loss-curve.svg.

Science-tutor pilots and guarded refinement

The science release tested whether the model could tutor rather than only match answers. Four candidates trained for 126 steps on the same 8,002 rows:

Pilot	Parent	Tokenizer / masking	Result
P1	Untouched Qwen	Fresh LoRA	Completed
P2	Original Muta	Fresh LoRA	Completed
P3	Historical warm pilot	Fresh optimizer/schedule	Completed
P4	DeepSeek-R1-Distill-Qwen-1.5B	Native tokenizer and masking	Completed

Development selection used 12 candidate checkpoints × 72 prompts: 64 mathematics/science multiple-choice items and 8 tutoring cases. P1-end had the highest unguarded point score but failed the preregistered regression guard. P2-half alone advanced:

Candidate	M / 64	S / 32	T / 32	Index	Decision
P2-half	52	17	24	72.66%	Advance
P1-end	51	18	26	74.22%	Guard failure
P4-half	45	15	25	66.41%	Reject
P3-half	45	6	15	51.56%	Reject

The full science-pilot curve panel is retained at provenance/science-tutor-20260919/results/pilot-loss-curves-v1/four-science-pilots.svg. Development ranking was not reused as the final test.

Matched evaluation and winner rule

The final roster was previous Muta, untouched Qwen, historical warm, untouched DeepSeek and P2-half. The sealed comparison combined held-out science, a 2,000-question practical battery, blinded tutoring review, the 100-prompt STEM screen and the known judges suite. The known judges score is a keyword screen; it is not a complete semantic grade.

Model	Held-out science	Practical / 2,000	Tutor / 64	Judges / 100	STEM MC / 50
Previous Muta	83.565%	41.325	22	57	32
P2-half	83.565%	39.850	12	50	32
Untouched Qwen	82.554%	38.425	17	61	35
Historical warm	83.375%	12.500	8	57	30
Untouched DeepSeek	Delivery failure	0.200	15	Failed gate	Failed gate

The preregistered conservative rule required a challenger to show a tutoring advantage without a material practical or known-suite regression. P2 tied the incumbent on held-out science but lost on the practical battery and tutor quality. The evidence therefore supports retaining the incumbent, not promoting P2. The full decision receipt is provenance/science-tutor-20260919/results/final-winner-decision-v1/decision.json.

Final Gate 2 decision

Retain Muta Tutor Qwen2.5-1.5B

No full-data challenger was promoted and the default deployment file was not swapped. The result is a practical evidence-based hold, not a claim that further verified data work cannot improve STEM tutoring.

Loss curves and reproducibility record

The repository preserves loss curves for the eight 20K pilots, the three full-data runs, the four science pilots, the five compared model histories and the original Muta training. Use the pilot table, full-run table and science-pilot table with the raw provenance paths rather than comparing loss across different datasets or hosts as though it were accuracy.

All histories

provenance/results/all-training-loss-curves-20260919/index.html and its five-model/original-Muta panels.

Eight pilots

provenance/results/all-training-loss-curves-20260919/eight-pilots.svg.

Science pilots

provenance/science-tutor-20260919/results/pilot-loss-curves-v1/four-science-pilots.svg.

Execution record

Training scripts, configs, row counts, adapter/base/GGUF hashes, failures and source/licence notes remain in provenance/.

Post-selection packaging

After the model decision, the Qwen3.5-style judge template in muta-iq/opt/templates/qwen35_judge_hybrid.jinja was embedded as GGUF metadata in a separate tensor-identical packaging variant. The original incumbent remained unchanged.

Artifact	Digest	Meaning
Original incumbent	`a750d00d…e1eb`	Unchanged deployment file
Template sibling	`ac512cc3…e3c`	Metadata-only packaging variant
Template bytes	`4844a94d…c3f6`	Embedded Qwen3.5-style hybrid template

The sibling passed single- and multi-turn smoke tests and was uploaded to the existing Hugging Face repository as a separate file. This was not a new fine-tune, and it was not silently made the winner.

What remains reproducible

The Gate 2 record keeps raw outputs, failed attempts, executed scripts, loss curves, exact row counts, adapter/base/GGUF hashes, model/config joins, and private-safe source/licence descriptions. The private training text is not copied into this public report. GPU timing is not called an ADTC CPU score; CPU qualification remains a separate deployment measurement.

For the concise experiment history, use the overview, pilot sweep, full-data runs, science-tutor pilots, matched decision, and loss record above.

Gate 2 · Chapter 4

Improving Accuracy while maximizing performance and efficiency

The previous chapter ended by retaining Muta Tutor Qwen2.5-1.5B: no further LoRA run made it a better tutor. It was also our slowest finalist on the machine that scores us. This chapter records everything we did after that decision to turn that tutor into the model we now submit: what we explored, what we trained, what failed, and what the delivered file can and cannot do.

Semi-final re-test

Where we started: an accurate tutor that was too slow

Round 1 results arrived for the Qwen3.5 0.8B model we had submitted: Accuracy 39.20, Performance 56.00, Efficiency 90.65, Total 54.53.

We did not want to choose the semi-final model from numbers produced by different tools. We therefore re-measured both of our published checkpoints under the organisers' own profiler image, with their audit recipe, on an otherwise idle cloud machine, and answered all ten Round 1 prompts with each model.

Model	tok/s	Peak RAM (MB)	ARC-Easy-50	Judge-prompt proxy	Complete answers
Qwen3.5 0.8B Q4_0, thinking on	13.42	670.8	70	22.0%	2 / 10
Qwen3.5 0.8B Q4_0, thinking off	13.42	670.8	70	34.0%	7 / 10
Muta Tutor Qwen2.5 1.5B Q4_K_M	5.77	1,099.5	84	38.0%	10 / 10

The Round 1 component scores imply that the organisers measured the 0.8B at about 8.4 tok/s and 655 MB. Our peak memory reproduces theirs within 2.5%, but their machine ran at roughly 0.63× the speed of our proxy. Every speed score in this chapter must therefore be read as an upper bound.

The 1.5B was the only model to finish all ten answers and the only one to get a full multi-step calculation right. It also answered our own chalk test prompt wrongly, produced no Yorùbá, and decoded at 5.77 tok/s, which earns only 38.5 of 100 performance points.

The score formula tells us what each improvement is worth. Below the 15 tok/s cap, 1 tok/s is worth 2.0 points, one accuracy point 0.5, and 100 MB 0.29. Our most accurate model was losing most of its points on speed.

The fine-tuning problem had become a compression problem. We no longer needed a tutor that knew more. We needed the same tutor in a shape the audit machine could run at 15 tok/s, and then training to win back what the reshaping cost.

Starting decision

Muta Tutor Qwen2.5-1.5B is the base. Qwen3.5-0.8B is discarded.

All work below targets one architecture: 28 decoder layers, hidden size 1,536, FFN width 8,960, 12 attention heads with 2 KV heads, tied embeddings, a 151,936-token vocabulary, and 1.54 B parameters, about 46.8 M per layer.

How we measured every step

We fixed the measurement before we changed the model. Every scored row in this chapter is one run of the organisers' profiler image, not of our own runtime.

S_total = 0.50 × S_acc + 0.30 × min(tok/s ÷ 15, 1) × 100 + 0.20 × (7 − peak GB) ÷ 7 × 100, minus 10 when throttled. No run in this chapter was throttled.

Audit build

The reference image adtc-profiler:latest, which builds llama.cpp in the scalar configuration defined in Gate 1, with four threads, on GCP n2 proxy machines with two cores and 8 GB. Decode tok/s, peak process RAM and the throttle flag come from that run. It is a cloud proxy, not the physical laptop, and it exposes no temperature sensor.

ARC-Easy-50

The profiler ranks four short answers by likelihood on the first 50 ARC-Easy questions. It uses no chat template and generates no text. At 50 questions the noise is about ±7 points, so we also ran a separate 500-question set for the finalists.

Judges' accuracy

The model answers tutoring prompts greedily through llama-server from the same image. Blind graders score each answer 0–10 against a rubric with worked reference answers. During exploration this used the ten Round 1 prompts. From the compression chain onward it used 40 prompts: 30 held-out dev prompts written for this work, which are the score of record, with the 10 official prompts reported beside them.

S_acc

S_acc is the mean of ARC-Easy-50 and judges' accuracy. We did this deliberately. ARC-Easy alone cannot see a model that knows the answer but never finishes saying it, and that was exactly our Gate 1 failure.

The graders are language models, so we measured their noise instead of assuming it. One fixed set of answers was regraded blind in four separate rounds and scored 28.0, 31.0, 32.0 and 30.7. That is a band of about 4 judges' points, or about 1 point of S_total. Differences smaller than that are ties, and we report them as ties.

One rule governed the whole campaign: never train on a test item or a judge prompt. We declared a perfect score unreachable at the start, so that nobody, including us, would be tempted to reach it the wrong way. A review after delivery found one place where the rule leaked, and we report it with the other limits.

Phase 1 · CPU only

Exploration: one audit per idea

We did not begin by assuming which technique would work. We had no GPU in this phase, so we gave each direction one cheap build and one full audit: depth pruning, a history-isolating chat template, importance-matrix quantization down to 2 bits, a training-free mixture of experts, vocabulary pruning, and a CPU-only distillation probe.

Judges' accuracy in this table is the ten Round 1 prompts, so these rows are comparable with each other but not with the results tables later in the chapter.

Candidate	ARC-Easy acc	Judges' acc	S_acc	tok/s	S_perf	Peak RAM (MB)	S_eff	S_total
Base, Q4_K_M	84	39	61.5	5.77	38.47	1,099.5	84.66	59.22
Vocabulary 151,936 → 48,064	82	41	61.5	6.29	41.93	948.0	86.77	60.68
History-isolating chat template	84	41	62.5	5.77	38.47	1,099.5	84.66	59.72
Q4_K_M + importance matrix	82	41	61.5	5.53	36.87	1,099.3	84.66	58.74
24 layers, LoRA-healed	80	14	47.0	6.38	42.53	1,011.6	85.89	53.44
21 layers, LoRA-healed	78	7	42.5	7.15	47.67	903.9	87.39	53.03
IQ4_XS + importance matrix	80	36	58.0	2.87	19.13	1,037.6	85.52	51.84
IQ3_M + importance matrix	80	35	57.5	1.89	12.60	926.2	87.08	49.95
21 layers, unhealed	64	2	33.0	7.10	47.33	912.3	87.27	48.15
Distillation 28 → 21 layers, CPU probe	64	1	32.5	7.00	46.67	906.0	87.36	47.72
MoE 8 × 1,120, top-4, untrained	54	9	31.5	6.72	44.80	1,288.0	82.03	45.60
IQ2_M + importance matrix	72	19	45.5	1.99	13.27	758.0	89.43	44.62
MoE 8 × 1,120, top-2, untrained	30	0	15.0	9.61	64.07	1,270.9	82.27	43.17
IQ2_XXS + importance matrix	72	2	37.0	1.98	13.20	672.9	90.61	40.58

Vocabulary pruning was the only idea that improved the total once judged answers counted. It gave +0.5 tok/s and −152 MB, held judged accuracy, produced no run-on answers, and needed no training. The cost is that the vocabulary becomes English-only, and tokens per character rise by 0.06% on our domain text.

Fewer bits were slower, not faster. Quality held down to 3 bits, but every IQ format decoded at 1.9–2.9 tok/s because the scalar build has no fast path for codebook formats. The memory saved was worth a fraction of the speed lost. An importance matrix on Q4_K_M itself changed nothing we could measure.

The chat template bought latency, not accuracy. In a multi-turn test under a 2,048-token context, the isolating template kept prompts at 107–212 tokens against 171–1,098 for the stock template and ran about twice as fast per turn. Speed, RAM and ARC-Easy do not depend on the template.

A mixture of experts without training is fast and broken. Splitting each FFN into eight experts gave +21% decode speed at top-4 and +73% at top-2, and quality collapsed. It also costs about 180 MB, because llama.cpp's qwen2moe architecture needs an untied output head.

Depth pruning looked like a win and was not. On ARC-Easy alone the healed 21-layer model edges out the base. On judged answers it collapses from 39 to 7. One thousand steps of LoRA healing restored the benchmark and did not restore generation. That result sent us to the grid below.

Phase 1 · eight parallel machines

The depth grid: which layers can go

Before paying for a GPU, we wanted to know which layers the model could lose. We scored 215 cuts by perplexity on held-out text: every contiguous window of 1 to 14 layers at every start position, six non-contiguous sets, and controls. The 77 even-start windows also got the profiler's ARC-Easy-50, and the best window at each depth got a full audit and the judge prompts. Nothing was healed; we wanted the raw damage.

Layers removed	Perplexity	Reading
None (base)	3.38	Choosing among about three plausible next tokens
Layer 0 alone	780,000	Fatal
Layer 1 alone	92	Fatal
Any other single layer	3.6–4.5	Cheap on its own
Best 2: layers 14–15	3.94	Nearly free
Best 4: layers 13–16	4.56	Cheap
Best 8: layers 9–16	7.93	Needs real healing
Best 14: layers 5–18	33.7	Badly damaged
Late window of 4: layers 21–24	103	Broken
Late window of 8: layers 17–24	3,984	Broken
Four lowest Block Influence layers: 15, 24, 25, 26	15.9	Far worse than the best contiguous cut of four (4.56)

The removable region is one basin around layers 9 to 17. Layers 0 and 1 are untouchable. Layers 21 to 26 are individually cheap but load-bearing as a group: they turn the hidden state into next-token predictions, and the remaining layers cannot stand in for them.

The published shortcut was wrong for our model. Block Influence ranks layers 24 to 26 as the most redundant in the network. Cutting the four lowest-ranked layers cost 16 ARC points more than cutting a contiguous window of the same size. Contiguous beat scattered at every size we tried.

Each removed layer buys about +0.14 tok/s and −27 MB, or +0.55 S_total, and speed tracked the file-size projection within 3%. The audits then showed what that price list leaves out:

Layers removed	ARC-Easy acc	Judges' acc	S_acc	tok/s	S_perf	Peak RAM (MB)	S_eff	S_total
None	84	44	64.0	5.56	37.07	1,110.0	84.51	60.02
8–9	84	26	55.0	5.85	39.00	1,068.3	85.10	56.22
14–15	80	18	49.0	5.90	39.33	1,045.3	85.42	53.38
10–15	72	11	41.5	6.74	44.93	938.8	86.90	51.61
14–19	74	5	39.5	6.75	45.00	937.4	86.92	50.63
10–19	60	3	31.5	7.88	52.53	832.2	88.39	49.19
6–19	50	1	25.5	9.40	62.67	726.1	89.87	49.52

The base stops on its own on all ten judge prompts, at a mean of 435 tokens. Removing just two layers already produced three to four run-on answers. Every cut of six or more layers, healed or not, ran to the token cap on eight to ten prompts with repetition loops.

ARC-Easy hides the damage. A pruned model can still rank four short options correctly while no longer being able to finish a sentence. Healing must be selected on generation quality, termination and the judge prompts, and never on ARC-Easy alone.

Round 0 of the chain · the finding that redirected the work

The kernel finding: only one format is fast on the audit build

In Gate 1 we learned that a quantization format is also a kernel decision. We had then let ourselves forget it, because Q4_K_M was the right choice under the vector configuration.

The audit build is the scalar configuration. In that build, pure Q4_0 is the only format with a SIMD (SSSE3) kernel. We requantized the same weights, changed nothing else, and measured:

Same weights, 28 layers	ARC-Easy acc	Judges' acc	S_acc	tok/s	S_perf	Peak RAM (MB)	S_eff	S_total
Published tutor, Q4_K_M	82	60.0	71.00	5.48	36.53	1,099.6	84.66	63.39
Pure Q4_0	70	39.0	54.50	10.88	72.53	992.1	86.16	66.24
Q4_0 with Q6_K embedding	72	53.0	62.50	8.92	59.47	1,049.8	85.35	66.16

Speed doubled, from 5.48 to 10.88 tok/s. The price was 12 ARC points and 21 judges' points, because Q4_0 is a blunter format than Q4_K_M. The total still rose, from 63.39 to 66.24. Keeping the embedding at Q6_K bought back 14 judges' points but gave up 2 tok/s, and the two layouts tied.

The rest of the work follows from this table. The format that scores is Q4_0, and Q4_0 hurts accuracy. So we would make a Q4_0 model small enough to reach 15 tok/s, and then train it to be accurate again under Q4_0's own rounding.

Phase 2 · rented A100-40GB

The five-step compression chain

With one rented GPU and a fixed deadline, we planned five steps in the order the exploration recommended: shape changes first, the most destructive ones healed before the next begins, and anything that depends on the final weights last. We always distilled from the original 28-layer tutor and never from the previous step, so that errors could not compound silently.

What we built first

LoRA had not healed a pruned model, so we built a full knowledge-distillation pipeline instead.

A teacher corpus of 104,475 rows. Training-split questions from GSM8K, ARC, QASC and OpenR1, each asked in four tutoring frames and answered by Qwen2.5-7B-Instruct, with a near-duplicate guard against every held-out prompt.
A teacher-logit cache. The top-32 teacher log-probabilities with a residual bucket for the rest: 59.4 M positions in 7.4 GB, so the 7B teacher never had to run during training.
A distillation trainer. KL divergence on the top-32 support, fp32 master weights under bf16 autocast, 8-bit AdamW, and packed 16k-token micro-batches at 262k tokens per step.
A termination gate. 130 prompts and a text-loop detector. A model passes only if at least 83.5% of its answers end cleanly. This is the check the depth grid taught us to make.
Bit-exact Q4_0 and Q8_0 fake quantization with straight-through gradients, verified against gguf-py and a C reference on 6.4 M elements, including tie cases.

Step 1 · Vocabulary pruning to 32,000 tokens

We kept the byte tokens, the special tokens and every token id seen in our corpora, then filled to 32,000 by merge order. The embedding fell from 233 M to 49 M parameters. It needed a patched GGUF converter and no training. The result was +0.99 tok/s and −108 MB with accuracy level. A 48,000-token variant tied with it; the promotion rule chose 32,000, and we rebuilt it from the full corpus and re-audited it before calling it the row of record.

Step 2 · Layer pruning healed by distillation

The perplexity map offered two cuts: layers 14–15, leaving 26, or layers 13–16, leaving 24. We healed each with 81.7 M distillation tokens.

The 26-layer model recovered from a validation KL of 0.363 to 0.179 and passed the termination gate at 0.854. The 24-layer model was 1 tok/s faster, failed the gate at 0.800, and looped or truncated on 21 of its 40 judged answers. We kept 26 layers.

We also tried one round of on-policy distillation on the 24-layer model, in which the teacher scores the student's own generations. It made the model worse: the gate fell from 0.80 to 0.68. Our untested hypothesis is that a teacher scoring looping text assigns high probability to continuing the loop. We did not retry it.

Step 3 · Mixture of experts trained by distillation: a negative result

We converted every FFN of the 26-layer model into a shared expert plus routed experts with top-2 routing, and distilled each variant for another 81.7 M tokens with a load-balancing loss.

Variant a (six routed experts, 779 M active parameters) reached 20.04 tok/s, well past the cap. It also looped on 27 of 40 answers and scored 2.3 on the dev prompts. Its speed could not buy back that much accuracy.

Variant b (four routed experts, 75% of the FFN active) passed its gate and was then slower than its own dense parent on the audit build: 12.42 against 13.15 tok/s. Under the vector configuration the same file had measured +10% faster. On the scalar build, routing each token to its experts costs more than the narrower matrix multiplications save.

Measure speed on the audit build before choosing an architecture. A speed-up measured on a different kernel path is a prediction about a different machine.

Steps 4 and 5 · Built, then cancelled

We had already built both remaining steps on variant b: seven quantization recipes, from pure Q4_0 to sensitivity-ranked Q8_0 promotions with an importance matrix, and a 26-minute quantization-aware run that recovered 57% of the quantization gap. Once the mixture-of-experts line lost to step 2, scoring them would have measured a model we were not going to ship. We cancelled both, and they stay in the record as unscored work.

Training run	Tokens	Validation KL, start → end	GPU hours	Clean-ending gate (pass ≥ 0.835)
Step 2 heal, 24 layers	81.7 M	0.507 → 0.225	1.76	0.800 · fails
Step 2 heal, 26 layers	81.7 M	0.363 → 0.179	1.91	0.854 · passes
24 layers + on-policy round	—	0.225 → 0.239	0.07	0.685 · worse
Step 3 MoE variant a	81.7 M	0.779 → 0.328	2.38	0.708 · fails
Step 3 MoE variant b	81.7 M	0.349 → 0.227	2.62	0.838 · passes

The chain therefore ended with step 2 as its practical best: a dense 26-layer, 32k-vocabulary Q4_0 tutor at 13.15 tok/s and 810 MB. It was still 1.85 tok/s short of the cap.

Phase 3 · A100-40GB

Refinement: width pruning, verified distillation and quantization-aware training

The mixture of experts had tried to buy FFN compute with routing. We decided to buy it the plain way: remove FFN neurons from the dense 26-layer model, then spend all the remaining GPU time healing it.

We wrote the targets down before training: ARC-Easy-50 of 75–80, judges' dev accuracy of 40–50, at most 3 looping answers in 40, at least 15 tok/s, about 800 MB, and a dev S_total of 72–75.

Choosing the width by measurement

We ranked every FFN neuron by E[a²]·‖W_down[:, j]‖², its mean squared activation times the norm of its output weights, on 200k calibration tokens. Speed depends only on tensor shapes, not on what the weights contain. So we exported three widths unhealed and timed them on the audit build before training anything.

FFN width	Parameters	Unhealed validation KL	tok/s	Peak RAM (MB)	ARC-Easy-50
8,960 (unpruned)	≈1.27 B	0.1789	13.15	809.9	70
7,680	1.113 B	0.2451	14.69	717.4	70
7,168 (chosen)	1.051 B	0.2791	15.45	681.1	68
6,656	0.990 B	0.3189	16.22	646.8	68

The rule was the widest model that clears 15.45 tok/s, which is the cap plus a 3% audit margin. Width 7,680 misses it. Width 6,656 clears it by more than the score can reward. We chose 7,168: 20% of the FFN width and about 17% of all parameters removed. Layers 1–2 and 24–25 lost almost nothing at any width; layers 6, 7 and 9 were the most prunable.

Verified data: checking the teacher, then checking the checker

In the chain we had trusted the 7B teacher's answers. For refinement we checked every teacher answer against the benchmark's gold answer and trained only on the ones that were right.

We then hand-labelled a sample of the verifier's verdicts, and found that the verifier was the one making mistakes. It was marking 33.5% of GSM8K answers wrong. The parser had been locking onto an intermediate number near a word like “total” instead of the final answer in the last paragraph. After the fix the rate was 6.8%. A second bug of the same family had discarded most of the orca-math questions; fixing it recovered usable questions from 4,377 to 18,118 on the same prompt ids.

Both “before” figures are our readings at the time of the fix and were not saved as files. Both “after” figures are backed by retained summaries.

Verify the answer-checker by hand before training on its output. Left alone, the two bugs would have thrown away more than a quarter of the GSM8K answers and three-quarters of the usable orca-math questions, all of them correct.

Component	Rows	Tokens	Why it is there
Verified single-turn corpus	54,896	22,292,307	Teacher answers that match gold; competition maths capped at 11% of tokens
Breadth top-up: orca-math and OpenBookQA	29,750	14,470,480	Gold-checked coverage beyond the four original sources
Regenerated GSM8K and ARC misses	1,508	682,425	Questions the teacher first got wrong, answered again and re-verified
Tutoring dialogues: MathDial and ConvoLearn	13,735	7,000,037	Unrolled human tutor turns, at weight 0.6
Style anchor	15,186	3,815,040	Scored by the original full-precision tutor, at weight 2, so the student keeps Muta's voice

Training

Distillation. The loss was KL on the teacher's top-32 plus 0.3 × cross-entropy, at a learning rate of 1 × 10⁻⁵ with cosine decay. We trained on 324 M tokens in 6.1 hours at 14.8k tokens per second. Validation KL fell from 0.279 to 0.197 and plateaued at step 1,100. The termination gate read 0.900, 117 of 130 clean endings and the best of any model in the campaign.

Quantization-aware training. We then trained with the forward pass rounded exactly as Q4_0 will round it, at a learning rate of 5 × 10⁻⁶, so that the weights learn to sit where the format can represent them. Validation KL measured under that rounding fell from 0.2466 to 0.2128 by step 100, about 68% of the quantization gap recovered.

Then we lost the GPU. The host became unreachable near step 160 of a planned 362. The step-100 checkpoint survived because the driver pushed checkpoints off the machine every hour. We exported it on a CPU-only cloud machine. Everything after step 100 is gone, and so is the gate's result file, which is why the 0.900 above is our own reading and not a retained file.

Even at 100 steps the effect is measurable: converting to Q4_0 costs 7.6% in perplexity without the quantization-aware pass and 1.6% with it.

Results

Every row is Q4_0 on the audit build unless named otherwise. The first table is the score of record.

KeptRejected

S_total on the 30 held-out dev prompts, step by step. The delivered model is outlined. Both mixture-of-experts variants score below the step they were built on.

Decode speed on the audit build

Decode tok/s on the scalar audit build. Performance points stop at the 15 tok/s gridline. Mixture-of-experts variant a is far past it and fails on accuracy; the delivered model clears it at 15.51.

Judges' accuracy from the 30 held-out dev prompts

Model	ARC-Easy acc	Judges' acc	S_acc	tok/s	S_perf	Peak RAM (MB)	S_eff	S_total
Published tutor, 28 layers, Q4_K_M	82	60.0	71.00	5.48	36.5	1,100	84.7	63.39
0 · Same weights, pure Q4_0	70	39.0	54.50	10.88	72.5	992	86.2	66.24
1 · Vocabulary → 32,000	70	39.7	54.83	11.87	79.1	884	87.7	68.69
2 · 26 layers + distillation heal	70	28.0	49.00	13.15	87.7	810	88.7	68.54
3 · MoE variant b + distillation (rejected)	68	20.7	44.33	12.42	82.8	816	88.6	64.73
3 · MoE variant a + distillation (rejected)	60	2.3	31.17	20.04	100.0	798	88.9	63.36
R · FFN 7,168 + verified distillation	70	24.0	47.00	15.42	100.0	683	90.5	71.59
R · + quantization-aware training (delivered)	72	31.3	51.67	15.51	100.0	706	90.2	73.86

Judges' accuracy from the 10 official prompts

Ten prompts is a noisy sample. We report it beside the score of record and never in place of it.

Model	ARC-Easy acc	Judges' acc	S_acc	tok/s	S_perf	Peak RAM (MB)	S_eff	S_total
Published tutor, 28 layers, Q4_K_M	82	45.0	63.50	5.48	36.5	1,100	84.7	59.64
0 · Same weights, pure Q4_0	70	27.0	48.50	10.88	72.5	992	86.2	63.24
1 · Vocabulary → 32,000	70	26.0	48.00	11.87	79.1	884	87.7	65.27
2 · 26 layers + distillation heal	70	45.0	57.50	13.15	87.7	810	88.7	72.79
3 · MoE variant b + distillation (rejected)	68	34.0	51.00	12.42	82.8	816	88.6	68.06
3 · MoE variant a + distillation (rejected)	60	14.0	37.00	20.04	100.0	798	88.9	66.27
R · FFN 7,168 + verified distillation	70	30.0	50.00	15.42	100.0	683	90.5	73.09
R · + quantization-aware training (delivered)	72	35.0	53.50	15.51	100.0	706	90.2	74.78

A second, independent blind grading of the delivered model gave 33.7 on the dev prompts and 32.0 on the official ten, for totals of 74.45 and 74.03. Its 26-layer parent, regraded in the same batch as a control, scored 69.21 and 72.79. An importance-matrix variant of the delivered model lost 1.5 points on the dev prompts and gained 2.8 on the official ten; that is inconclusive, and we did not adopt it.

Held-out capability

Model	ARC-Easy-500	GSM8K-100	Hit length cap	Looping answers / 40	Tutoring probe / 10
26-layer parent	70.0%	49%	6%	8	3.24
Delivered	73.4%	53%	3%	4	3.26

Where the points came from

Grading batch	From accuracy	From speed	From memory	S_total gain over the 26-layer parent
A	+0.34	+3.70	+0.29	+4.32 (69.54 → 73.86)
B	+1.25	+3.70	+0.29	+5.24 (69.21 → 74.45)

This is a speed result, and we say so plainly. Of the 4.3 to 5.2 points gained over the 26-layer parent, 3.70 come from reaching the 15 tok/s cap. We had estimated exactly that figure before training began, from the expected speed alone. The distillation and quantization-aware training contribute the remaining 0.6 to 1.5 points.

What the training did was hold the line. We removed 17% of the parameters and kept the quality. Judges' accuracy is level with the parent and inside the grader noise band. ARC-Easy-500, GSM8K-100 and ARC-Easy-50 all moved in the delivered model's favour, but none is significant on its own at these sample sizes: z is about 1.2 for ARC-Easy-500 and about 0.6 for GSM8K. Three consistent directions are not one proof.

Against the published tutor, the whole campaign gains +10.5 S_total. It does so by trading 29 judges' points and 10 ARC points for 2.8 times the speed and 36% less memory. The formula rewards that trade. A student may not, and that is why the next section exists.

One caution carries over from the re-test. The organisers' machine is slower than our proxy, so the speed term is an upper bound there. What we expect to carry over is the ratio between rows, not the absolute tok/s.

What did not work, and what we still owe

We set six targets before refinement. We met three, came close on one, and missed two.

Target	Aim	Outcome	Verdict
Decode speed	≥ 15 tok/s	15.51	Met
Peak RAM	≈ 800 MB	706 MB	Met
Dev S_total	72–75	73.86–74.45	Met
Looping answers	≤ 3 of 40	4 (parent: 8)	Close, not met
ARC-Easy-50	75–80	72	Not met
Judges' dev accuracy	40–50	31.3–33.7	Not met

Tutoring-dialogue mode does not work. On 50 held-out MathDial dialogues, the delivered model lectures instead of guiding. It tells the student the answer in 34 of 50 first turns, and 70% of its first turns contain a false statement; the parent's figure is 76%. It scores 3.26 of 10 against the parent's 3.24. Our hypothesis is that the dialogue rows were trained mostly toward the 7B teacher, whose own habit is to explain everything, and at only about 9% of tokens. They should have been trained on the human tutors' text alone, at a larger share.

Quantization-aware training is unfinished. It ran 100 of 362 planned steps and was still improving when the host was lost.

Our judges are language models. They are a measured proxy for the human panel, with a known noise band, and not a substitute for it.

The answer verifier still has holes. After delivery we ran an adversarial code review of the data pipeline. It found false-accept paths that had run on the training data. We re-ran the corrected verifier over the same corpus: 222 of 54,896 verdicts flip, and a hand-check puts genuinely wrong trained-on rows at about 18, or 0.03%. That is a lower bound, because the re-review found two more false-accept paths in the fixed code. The orca-math rows cannot be re-measured at all, since their raw generations were on the lost GPU host; the original hand-check bounds their error rate at about 10% and no tighter. The first version of those fixes passed 338 unit tests and still flipped 3,347 rows on real data, mostly wrongly.

Contamination is small, and it is not zero. A worst-case check found that 2 of the 100 GSM8K items and 1 of the 500 ARC-Easy items have near-duplicates in the training pools. Excluding them leaves the reported gaps unchanged. One GSM8K test question also appears verbatim inside MathDial's training dialogues, so the model was trained on it; it is not among the 100 items we evaluate. And the tutoring probe is held out by dialogue, not by problem: 34 of its 50 problems also occur in MathDial's training split. The model had seen most of those problems and still could not tutor them.

The same review found why we lost data. The hourly checkpoint push never made a final push when the session ended. The fix is in the code; the lost steps are not recoverable.

The model we deliver

Gate 2 model

refine-qat100-Q4_0.gguf

A dense Qwen2 tutor with 26 layers, FFN width 7,168, a 32,000-token vocabulary and 1.05 B parameters, in a 593 MB pure-Q4_0 file. On the audit build it decodes at 15.51 tok/s, peaks at 706 MB, and scores 73.86–74.45 on our held-out score of record, against 63.39 for the published tutor it came from.

The weights are public at timiiowolabi/muta-compress-20260920 under refine-final/gguf/, with digest 6e100450…04d3fc. The same repository holds the bf16 checkpoints before and after quantization-aware training, the step-2 and step-3 models, and the model card with these tables. The training corpus and the teacher caches are not published.

We are delivering this model because it is the best measured total we can defend, and not because it is the tutor we want. It fits the machine, it reaches the speed cap on our proxy, and it loops half as often as its parent. It does not yet guide a student the way a teacher does.

Next, with a GPU

Finish quantization-aware training from the published step-100 checkpoint, for the full 362 steps.
Retrain the dialogue rows on the human tutors' text only, with cross-entropy and at a larger share of tokens.
Regenerate the QASC questions the teacher missed, which we skipped for time.
Re-estimate the verifier's error rate with a fresh hand-labelled sample before it touches another corpus.
Repeat the audit on the physical target laptop, with temperature.

Candidate measurements

Go small and dense

Go big and ternary

Quantization

Parameter deduplication

Vocabulary pruning

Weight streaming

Runtime tuning

Instruction-set targeting

Widening the model search

Baking behaviour into the file

Fine-tuning

Runtime configuration we couldn't submit

Kernel dispatch by tensor type

Vocabulary pruning

TQ1_0 body test

Head and embedding quantization

Factorisation and sparsity tests

Throughput and RSS by resident weight budget

Selected runtime configurations

Draft-model speculation

N-gram speculation

Six-thread cap

Unified KV, two checkpoints

Mixed tensor or layer quantization

Structured and unstructured pruning

Smaller architectures

Distillation and fine-tuning

Tensor layout, alignment, and runtime repacking

Context size and KV-cache implications

Score by component

Speed against memory

Total score by CPU method

Math-Expert 0.6B Q4_K_M

Muta-Tutor-Qwen3.5-0.8B-Q4_0-final.gguf

Direct scalar and vector totals

Profiler total and larger-sample diagnostic

Speed and ARC-Easy trade-off

Everything else we tried in this round

Total score by CPU configuration

Every paired model, scalar versus vector

Eight runs · no promotion

Seven runs · two promotions

Accuracy before and after tuning

Total score by CPU configuration

Fine-tuned Qwen3.5 0.8B · 80.3664

Fine-tuned Qwen2.5 1.5B · 84.1387

Profiling…

Model research

Decision

Scope and evidence

Ranked practical shortlist

Benchmark evidence ledger

Verified weight files

Quantization choice

What published speed measurements establish

Quality-first stretch candidates

Other screened candidates

Laptop evaluation protocol

Selection rule

Sources

Model evaluation and ranking

From benchmark promise to usable answers

Candidate field

Protocol and evidence boundaries

Performance and memory

ARC-Easy-500

100-prompt STEM battery

Ten-prompt Gate 1 replay

ARC-Easy-500: the control retains the strongest knowledge proxy

Resource score: two CPU configurations produce different leaders

The STEM battery changes the ranking

The Gate 1 replay tests whether the answer reaches the student

Why the model with the best single proxy is not automatically selected

Continue with fine-tuned Muta Tutor Qwen2.5 1.5B Q4_K_M

Read the retained responses

Prompt

Answer delivered by the model

Separate reasoning

Assessment

`Muta-Tutor-Qwen3.5-0.8B-Q4_0-final.gguf`