What worked, what failed, on one laptop: the skycmd experiments as of 19 November 2026
19 November 2026
Updated on 20 November 2026: s6-ear v3 was trained and read late on 19 November, and failed its gate; its file is published as s6-ear3-nano, as a result. The failures, their count and the last paragraph say so, and its own article tells it.
Updated again on 19 November 2026: s6-ear v3's pre-registration was frozen that day, with amendments 1 and 2; its data build is running, and nothing has been trained or read. The last paragraph says so.
Updated on 19 November 2026 (evening): s6-sleep v4's second check, K1, failed, and v4 is closed with no model. The short version, the s6-sleep v4 item under "What stopped before training, or could not run", the paragraph on unknown causes and the last paragraph say so.
Updated on 19 November 2026 (later that day): three decisions taken that day change the status of four lines, and no number. The fine-tuning of LFM3-1.7B is closed with nothing trained; a frozen plan of four "flexible weights" experiments is closed without running, after a literature review; s6-sleep v4 goes on to its next check, and s6-ear v3's pre-registration is being frozen, both decided and not run yet. The short version, the section "What stopped before training, or could not run" and the last paragraph say so; the rest of the article is unchanged.
Over the autumn of 2026 I trained very small models on one laptop and wrote down, before each test, what would count as a pass. This article looks back at all of it at once. It does not repeat the tables: every claim and its evidence is on What we bring, and every number on Results. It tells what the lines of experiments did together, which passes held and with what limits, which failures came, and what the whole does and does not show.
The short version
Five models passed the gate written for them before their test, four at their first reading and one on a second look. At their first reading: a general decision model, a secret detector on its own synthetic bench, an end-of-turn detector and a tool router (on fresh slates, with a system defined after its first version's test was read). On a second look at the same test records: the general model's larger second version. Each pass comes with a limit stated next to it, and two of those limits are about real data that the gate did not read. Eleven rows of the Results status table are failures, each published with its file (eleven since s6-ear v3 was read, updated on 20 November 2026). Four more versions stopped at checks before their main training, a fifth (s6-sleep v4) failed both of its checks on training sequences and is closed, and one fine-tuning could not run at all on this machine and is closed. A planned series of four experiments was closed without running, after a literature review.
The context: skycmd, on one laptop
skycmd gathers very small "decision models": a model reads a state (a game frame, a text, a request with a list of tools, a few seconds of audio) and answers a typed question by picking one of the options given with it. Their weights are ternary, each one −1, 0 or +1, which keeps the s6 files between 142,912 bytes (s6-cast-pico) and 3.7 MB (s6-route, in two files). The code is Rust; the same file runs natively and in the browser through WebAssembly (WASM), so every published model can be tried in a browser, on this site or in its Hugging Face Space, without a server.
The models were trained on one MacBook with an Apple M5 chip and 16 GB of memory, the same machine its owner works on. The teacher that wrote the s6 labels, GLM-5.3-Flash, ran on its provider's servers, through its API.
The method, the same for every line
Every line below followed the same rules. They are the reason the failures are as informative as the passes, so they come first.
- Pre-registration. Before a model is trained, I commit to the repository the test it will be read on, the bar it has to clear and what a failure would mean. The bar is called the gate: a set of conditions that must all hold. Each pre-registration is a file in
docs/, for exampledocs/s6-gen-prereg.mdfor the general models ordocs/s6-route7-prereg.mdfor the last router. - Dated amendments, before the computation they concern. When a plan has to change, an amendment is appended with its date, what had been seen when it was written, and what it does not change. For example, the readings of s6-route v7's gate code were fixed by an amendment written while its training ran, before its fresh test was opened.
- One reading per test. A test file is opened once, at the end, and its result is the result. When a later version is designed after a test was read, its pass on that test is called a second look, on every page that reports it (s6-micro v2, s6-secrets v3), or it is judged on fresh data built afterwards (s6-route on fresh slates, s6-route v7 on a new tool family). For s5, the evaluation tool itself refused a test split unless the pre-registration listed its hash, and logged every pass over it.
- Blind readings and diagnostics from existing files. Two habits came up again and again. A blind reading hides the input the model is supposed to use: with the request replaced by
?, a tool router that reads the request should be at chance. A diagnosis explains a failure from files that already exist (training logs, validation files, the dumps of a test already read) without training or reading anything new. The diagnoses are indocs/(for exampledocs/s6-route4-diagnosis.mdordocs/s6-sleep4-diagnosis.md), and they change no number or verdict. - Failures are published like passes. Since the owner's decision of 15/11/2026, a model that fails its gate is published with its file and a card that opens with the failure, and since 17/11/2026 every published model has a Space, a demo page on Hugging Face.
- Synthetic data only from open-weight models. Every label, rewritten description or teacher answer that the s6 lines generated for training, validation or testing comes from GLM-5.3-Flash, a model with open weights under the MIT licence, called through its API (the rules are in
docs/s6-gen-prereg.md, amendments 1 and 3). For s6-nano and s6-micro it labelled 747,077 of the 747,198 training questions selected for them (BENCH.md). Public datasets are used under their own licences. Before s6, s4's teacher, Qwen3.5-4B in 4 bits, ran on the laptop itself with mlx-lm. - Training on one laptop, with an idle brake. A training at full speed makes a laptop hot and can freeze it. So a small daemon,
scripts/auto-brake.sh, writes one line to a file that the trainer reads between steps: work half of the time with a 15-minute rest each hour while the Mac is in use (0.5 45:15), 85% with a 10-minute rest when it is left alone (0.85 50:10), 95% at night (0.95 55:5), and a memory guard stops a training that holds too much memory. The rests are not counted, so a budget such as "120 working minutes" counts working time only, whatever the mode (docs/idle-training.md, and the Train while you work page).
What passed its gate, and the limit stated with each pass
Five models cleared every condition of their gate, four at their first reading and s6-micro v2 on a second look. The second column is the gated number against its bar, from Results; the third is the limit that the same page states with the pass.
| model | gated number against its bar | the limit stated with the pass |
|---|---|---|
| s6-nano, a 715 KB general decision model (games and questions in free text) | typed_decisions_bench 0.480 against 0.398; ECE 0.0479 against at most 0.08 | a constant answer per task passes the same gate: answering each question with its most common training answer scores 0.4815, and on 17 of the bench's 20 tasks s6-nano gives one answer to at least 95% of the items |
| s6-micro v2, its 5.6-million-parameter sibling | 0.553 against 0.45; ECE 0.0588 against at most 0.08 | a second look: it was designed after v1's calibration failure and judged on the same test records |
| s6-secrets-pico, a 250 KB secret detector | on its synthetic bench, secret 0.9990 against 0.977 and service 0.9795 against 0.787 for a rule scanner | the bench is synthetic; on 719 real files with no secret it flags 59.8% of them at p ≥ 0.9, so it is not a scanner |
| s6-ear2-nano, a 632 KB end-of-turn detector | accuracy 0.8993 against 0.85 (its teacher Smart Turn v3.2: 0.9260 on the same clips); false cut-off 0.0847 against 0.20; 2.39 ms per decision in WASM against 5 ms | on real conversations (eot-bench, which its design never read) it and its teacher are far weaker, and at its cautious threshold it answers "finished" on 0.5253 of the real ends of turn |
| s6-route, a 3.7 MB tool router (a failed model fused with a static embedding, nothing trained) | 0.8868 on fresh requests about unseen tools, above BM25 (0.8169; gain +0.0698 [0.0455, 0.0952]) | defined after its predecessor's test was read; 0.2497 with the tool names masked; without the test's three largest clusters, +0.0202 [−0.0062, +0.0481] over BM25 |
A few terms. ECE, the expected calibration error, measures how far a model's confidence is from its accuracy: 0 means that when it says 80% it is right 80% of the time. BM25 is a classic search formula that scores a tool by the words it shares with the request; it has no parameters to train. A number in brackets is a 95% bootstrap interval, the range where the true value most likely lies, estimated by resampling the test items.
Two of these passes later met harder data, and both readings were published. s6-secrets-pico was run on real files after publication, which gave the 59.8% above. s6-route was read, as a reported baseline, on 2,000 fresh requests about the tools of MCP servers, a family no training example came from: 0.5530, where BM25 reads 0.5675 (Results).
What failed, and was published as a result
The status table of Results has seventeen rows. Five are the passes above, one is a pilot that stopped the experiment (s6-sleep v1), and eleven are failures (the eleventh, s6-ear v3, added on 20 November 2026). The failures, line by line:
- s6-micro v1 failed on calibration only: ECE 0.0821 against at most 0.08, with a bench score of 0.547 that met its gate. Its second version is the pass above.
- s6-ear v1 (s6-ear-nano) failed three of its four gates: end of turn 0.5681 against 0.9263 for Smart Turn v3.2, and its own control, trained the same way without the speech-codec pretraining, did better on all four sets. Later, a bug in my harness turned out to have understated the reference model's score (it read 0.8518 instead of 0.9263); the end-of-turn article tells it.
- s6-cast v1 and v2, two ternary forecasters, failed on GIFT-Eval, a public benchmark of 97 forecasting configurations scored against a seasonal naive forecast (lower is better): v1 at nGMASE 0.9471 and nWQL 0.6646, v2 at 0.8847 and 0.6190, against bars of 0.838 and 0.587. v2 improved on v1 and stayed behind our port of TinyCast (0.7918 / 0.5622). By its pre-registration the line stops there, and the TinyCast port is the forecaster the project offers.
- s6-secrets on real files, then the verifier. After s6-secrets-pico's bench pass came the real-file measurement above. Its second version, s6-secrets-verify-pico, judges the candidates of a rule scanner on real repositories: line F1 0.8561 against a bar of 0.8832 (the best rule scanner's 0.7332 + 0.15), with six of its seven gates holding. The system made of the scanner's default rules and the verifier together (v3, a second look at the same test) failed again, 0.8477 (the secrets article).
- s6-route-nano (s6-route v2) tied BM25 instead of beating it (0.8395 against 0.8500) and took 38.9 ms per request in WASM where 2 ms was asked.
- s6-route v7 (s6-route7-nano), the router's model part retrained on rebuilt data, read once on the new MCP tool family: 0.5610 against 0.5675 for BM25 (−0.0065 [−0.0235, +0.0120]), an ECE of 0.1845, and with the request hidden the pair still picks the right tool on 0.1969, above its bar of 0.1929, while its trained part alone is at chance (0.1406, chance 1/7). By a decision announced before the run, a failure of v7 closed the line: there is no v8 (the router article).
- s6-ear v3 (s6-ear3-nano), v2's end-of-turn graph trained again on real two-speaker conversations and on the silences the demo asks at, read once on 19 November: on eot-bench, real speech addressed to an agent, 0.2 s into each pause, its AUC is 0.7197 against v2's 0.7448 (−0.0252 [−0.0336, −0.0167], where at least +0.02 was asked), while on held-out human-to-human conversations it reads 0.6878 against v2's 0.6031. By the reading written before the run, the human-to-human signal was learned and does not transfer to agents; v2 stays the line's file. v3 was designed after v2's eot-bench reading, and its label cross-check was amended after a measurement on five training sessions, both disclosed (its article).
- s6-sleep v1 and v3, a fast-weight memory for a decision model. v1's pilot stopped v1:
recall0.2460 against 0.2650 for the control without memory, where +0.10 was asked. v3's memory is used (+0.2452 [+0.2370, +0.2532] onrecall), but sleep passes add +0.0030 [−0.0010, +0.0068] and two games lose (Flappy 15.87 against 71.05 for the control) (the memory article). - The System 1 / System 2 router of LFM3-1.7B, which asks a 1.7-billion-parameter model to think only when a probe doubts its first answer, kept the accuracy of always thinking (0.786 against 0.796) but used more tokens (1,032 against 958), where at most half was asked.
- The earlier misses. s4-micro, distilled from a 4-billion-parameter teacher in one night, scored 0.343 on the typed-decisions bench against a gate of 0.55 (
BENCH.md). The three pre-registered bets of s5 each missed their criterion; the first, for example, on calibration: ECE 0.088 against 0.057 + 0.01 (Results).
Before the s6 models were trained, a pre-registered ablation on the smallest size tested eight additions to the recipe (a new optimizer, two attention variants, weight decay and distillation schemes, a weight average, a quantization gradient, a curriculum). None reached its required gain, so none was adopted, and the s6 models are the earlier architecture trained with AdamW (What we bring). That is a negative result too: no claim of the project rests on those additions.
What stopped before training, or could not run
Some versions never reached a trained model, because a check written into their pre-registration stopped them first. These cost CPU time or a short GPU check, not a full run.
- s6-sleep v2 ended at its CPU check C4: in all three allowed attempts of 300 CPU training steps, the loss on question frames stayed at chance (
docs/s6-sleep2-prereg.md). - s6-route v4, v5 and v6 stopped at their data checks, before any training. Each rebuilt the router's training data to remove a shortcut, then counted whether a rule that never reads the request could still pick the right answer above chance. v4 failed that count 17 times (for example, the shortest name was right on 0.4219 of the intents group's hard training slates, against a chance of 0.3721) (
runs/s6-route4/checks.md). v5 removed those cues and left smaller ones, +0.028 to +0.076 above chance, and a way to read none from how the options spread (right on 0.4410 against a share of 0.20) (runs/s6-route5/checks.md). v6 fixed the slates concerned and tripped its one stop rule: the same reading rose to 0.5792 against 0.20 (runs/s6-route6/checks.md). v7, above, is the version whose stop rule did not fire. - s6-sleep v4's first check, K3, failed on both seeds on 18/11/2026: with the base model frozen, the answer on
recallstayed at 0.2579 and 0.2546 over the last 100recallsteps, against a bar of 0.95 and a chance of 0.25 (docs/s6-sleep4-prereg.md, §15). That closes the route the pre-registration tried first. Updated later on 19/11/2026: after a literature review (docs/research-s6-sleep4.md), v4 goes on to its next check, K1, and to K1 only: the same recipe with more training steps, on both seeds, with no early stop within a run. If K1's bar fails on either seed, v4 stops there and that reading is its result; if it passes, nothing further runs until a further amendment (amendment 2 of the pre-registration). Updated in the evening of 19/11/2026: K1 failed, and v4 is closed. On each step's own training batch, read before its update (not a held-out test), the answer was right on 0.8442 of the questions over the last 100recallsteps at seed 0 and 1.0000 at seed 1, against 0.95 on both seeds; a pass on one seed only is a failure (§16). The curves are worth stating, as observations and not as a pass: seed 1 sat near 0.56 to 0.60 for about 150recallsteps, then jumped to 0.99 within the next 100; seed 0 did the same later and was still rising (0.88 over its last 19 steps) when its budget ended. It was not run further, because the budget was fixed before the run. With the memory switched off, both seeds stayed near chance (0.25), so what they learned goes through the memory. No model comes out of K1, and nothing is scheduled after it. - The fine-tuning of LFM3-1.7B could not run on this machine. Its smoke test, on 18/11/2026, tried the two LoRA methods its pre-registration allowed (LoRA trains a small add-on to a model instead of all its weights), peft on the Mac's GPU and the trainer of mlx-vlm, at batches of 8 and 4. Under a memory bound set by the owner before the deciding run (an attempt stops when the swap passes 3 GB or the one-minute load passes 8), no attempt finished a single optimizer step. By the rule written in advance, nothing was trained (amendment 7 of
docs/lfm3-ft-prereg.md). Updated later on 19/11/2026: the owner decided not to rent remote compute for it ("no LFM3 if it does not fit"), so the fine-tuning is closed, with no weights (amendment 8 of the same file). The System 1 / System 2 router of LFM3-1.7B, above, is a separate result and stays as published. - "Flexible weights": four experiments closed without running (added later on 19/11/2026). A plan frozen on 09/11/2026 (
docs/weights-prereg.md) asked whether a large language model (LLM), here GLM-5.3-Flash, can read and write the weights of our small models: edit a policy to fix a fault (7.1), write a starting point for training (7.2), or audit a model for a planted backdoor (7.4); a fourth experiment (7.3) would let a small add-on to s6-nano's output scores (a low-rank adapter, a few weights trained while the model itself stays frozen) learn online each time the games' coded safety reflex overrides the model's move. A literature review of 19/11/2026 (docs/research-flexible-weights.md) found that the source of the idea is a blog post, not a peer-reviewed paper, and that a third-party rerun of the post's own tasks, one run per task, measured GLM-5.3-Flash, the only teacher the project's rules allow: it was at or below random when writing small game policies (0.0 on Breakout) and returned no edit on the post's repair task. For the audit, statistics of the weights detect planted backdoors in small models without any LLM. 7.3 needs no LLM and is untested in the literature, but its pass asks for a gain on a game, and s6-nano already plays at the oracle's score in every game, so there is no gap for an adapter to close; the nearest work also warns of forgetting under online updates. All four were closed, 7.3 as untested, not as refuted, and nothing was computed (amendment 1 of the plan).
What recurs across the lines
Read together, the lines show a few patterns. Each comes from numbers above; none is a measured law.
- A trivial baseline was often the real bar. A constant answer per task cleared s6-nano's gate; BM25 matched or beat both trained routers (s6-route-nano and s6-route7-nano); the TinyCast port beat both forecasters. The lesson I took is that a gate should compare a model with such baselines, not only with a target number.
- Passing on the data a model was built around did not carry over to real data. The secret detector flagged most clean real files; the end-of-turn detector caught about half the real ends of turn on eot-bench; the retrained router fell from 0.8802 on validation to 0.5610 on a new tool family. In the last case, why is not known: no reading separates the new tools from the new source of requests.
- Stopping early was cheap and frequent. Four versions (s6-sleep v2, s6-route v4, v5 and v6) ended at checks on data or on training steps, and s6-sleep v1 at a pilot, before the GPU hours of a full run. The checks did what they were written to do. They did not lead to a passing version: the two lines that went on after them ended in failures (s6-sleep v3 and s6-route v7).
- Some causes are not known. Why s6-sleep's memory reaches 1.0000 at the pico size and about half of that at the nano size is not known (a diagnosis found that the nano runs ended halfway up their learning curve; with more steps, v4's K1 reached 1.0000 on one seed and 0.8442 on the other, on training sequences only, which narrows the question without settling it on any test); why the router loses a third of its accuracy on a new family is not measured. The pages say so instead of guessing.
What this shows, and what it does not
What it shows. Small ternary models can be trained from scratch on one 16 GB laptop, under a brake that keeps the machine usable, and run in a browser from the same file the numbers were measured on. Five of them passed gates written before their tests. The method itself held: tests were read once, every change of plan is dated, every failure is published with its file, and when a pass turned out to be weaker than it looked (s6-nano's constant-answer baseline, s6-route's masked names and shortcut, s6-secrets-pico's real files), the weakness was found and published next to the pass.
What it does not show. None of the five passes is a general-purpose tool: on most of its bench s6-nano does not read the state, s6-secrets-pico is not a scanner, s6-ear2-nano is weaker on real conversations, and s6-route needs distinctive tool names and does not beat BM25 on a new tool family. The results come from our own benchmarks and our own runs: one run per size for s6, no peer review, and a literature search that was not systematic. None of the building blocks is new, and the ablation adopted none of the additions it tested. Several readings were made after the gate (the studies and diagnoses), and they are labelled as such. Finally, s6-ear v3, the last line still open when this article was first written, was trained and read late on 19 November and failed its gate (docs/s6-ear3-prereg.md, §17; updated on 20 November 2026): it is one more failure above, not a pass. s6-sleep v4 ran its check K1 that day and is closed: its reading is on training sequences only and gives no model. The LFM3 fine-tuning and the flexible-weights experiments are closed, with nothing trained or computed, so they show nothing either way.
Check it
Every claim with its evidence: What we bring. Every number: Results, and BENCH.md for the full tables. Every model, with its verdict first and a demo in the browser: Models. The four lines told in detail: the tool router, the secrets verifier, the end-of-turn detector (and its third version, trained on real conversations) and the memory. The pre-registrations, amendments and diagnoses are in docs/, the result files in runs/, and the training on one laptop in docs/big-run-report.md and docs/idle-training.md.