Nuée documentation

What Nuée is

Someone launches a job, from the nuee command line or from the web page, and gets a link and a QR code. Everyone who opens the link joins the swarm: a browser tab lends a few cores, a nuee join lends a whole machine. The job is cut into tasks, the tasks are handed out, the results come back and are checked. The same links also carry encrypted files, exposed services and static sites.

The link is the cluster. It holds the coordinator’s public key, the relay to reach it, and a 16-byte secret. Opening it is the admission; there is no account, no sign-up and no server of ours in the middle.

It is meant for a group that already trusts itself: a talk or workshop where the audience joins from their phones, a class, a lab or a team with a few Macs and Linux boxes. Typical work is batch work that splits into independent tasks: fractals and simulations, evaluating a small model, sentence embeddings, sweeps, training small models, and handing files around.

What it is not:

• Not a market. There is no payment, no credit and no token. Nobody rents anything to anybody.
• Not a way to run large language models across peers. A model is never split between devices. The llm kind only scores answers with a small model that fits on one device.
• Not a way to run strangers’ native code in your browser. Browsers only run WebAssembly and WebGPU tasks in a sandbox. Native commands (exec) run only on command-line machines whose operator approved that exact command from one exact commit.
• Not anonymous and not for strangers. Relay operators see IP addresses, and anyone holding a link is a member. See Security and privacy.

Quick start

In the browser

Open the web page. It needs nothing installed.

1. Launch. Choose Launch, pick a computation (fractal on CPU or GPU, skycmd evaluation, embeddings, train together, or your own .wasm module), choose the verification mode and press Start. The page shows a link, its QR code and six symbols. Present full screen enlarges the QR code for a projector.
2. Join from a phone. Scan the QR code. The join card says who is asking (the six symbols), what will run and what a task can and cannot do, and offers how many cores to lend, the GPU when WebGPU is available, and an “only while plugged in” option. One tap on Contribute starts it; Pause and Stop are always there.
3. Compare the six symbols. They are derived from the coordinator’s key only. If the symbols on the phone differ from the ones on the presenter’s screen, the link was swapped on its way: close the page.
4. Watch. The presenter’s page fills a grid, one square per task, in each device’s ink. A result that was rejected is circled in red in the margin and stays circled once redone.

What runs in a tab: WebAssembly tasks in a pool of Web Workers, WGSL shaders through WebGPU when the browser has it, the embedding model, and the small LLM scorer on devices that do not report less than 4 GB of memory. What cannot run in a tab: native exec commands, pinning, offering storage, and downloads above 256 MiB. A mobile tab stops when it goes to the background, so keep the screen on while contributing.

With the command line

The nuee command runs on macOS and Linux. Windows machines join through the browser.

brew install maxgfr/tap/nuee

Or build it from a checkout of the repository (see Building from source):

cargo build --release -p nuee

Check the machine. selftest needs no key, no network and no files:

nuee selftest

On a source build without the embed-tasks feature it printed:

✓ wasm golden: 3 golden outputs match
✓ demo modules: embedded: mandelbrot, toy-trainer, optdigits; not embedded: skycmd-arena (build with --features embed-tasks)
✓ links: job, file and pin links round-trip
✓ manifest: manifest signatures and admission proofs hold
✓ encryption: age encryption round-trips and detects tampering
✓ swarm: in-memory swarm finished 6 tasks
selftest ok

Launch the fractal demo:

nuee run demo:mandelbrot

It prints a QR code, then the link and the fingerprint once the relay is reachable, and a progress line while peers work. A run with one nuee join worker, on an Apple M5 with a local relay, printed (QR code, path prefix and most progress lines left out):

https://maxgfr.github.io/nuee/#/J/AGG7I54LT3RYDIITQGUZBA6F…
fingerprint 🗻 🎃 💧 🎣 🍉 🥐  (compare with the screen of whoever joins)
256 tasks, verify Sample, 5 local slots, outputs in …/nuee-out/mandelbrot
waiting for 1 peer(s) to join
peer joined: dfbe5bccab04…c896ec (nuee-cli/0.0.0)
███████████████████████░░░░░░░ 197/256 tasks verified 145 · 1 peers · 1223.6/s
image: nuee-out/mandelbrot/mandelbrot.ppm
done: 256 tasks, results in nuee-out/mandelbrot/results.json

That run used --wait-peers 1 so the job waited for the worker; without it the coordinator starts computing at once with half its cores. On another machine, join with the link. Passing - reads it from standard input, so it does not land in your shell history:

nuee join -

coordinator fingerprint 🗻 🎃 💧 🎣 🍉 🥐  (compare with the coordinator's screen)
joined with 5 slots
job 0: mandelbrot
task 9 done in 43 ms (1 so far)
…
left the swarm after 84 tasks

Or open the link on a phone: the tab joins the same swarm. Every verb prints human text; add --json to get one JSON event per line on standard output instead (see NDJSON events).

How it works

The network: iroh

Every peer is an iroh endpoint whose identity is an ed25519 public key. Connections are QUIC with TLS 1.3, and both ends authenticate with their keys, so a worker only ever talks to the key in its link. Command-line peers try a direct UDP path first; when there is none (a strict firewall, a browser), traffic goes through a relay. Browsers always go through a relay over a WebSocket. The relay forwards encrypted packets and cannot read them.

Who sees which IP address: the relay operator sees the address of every peer connected to it, and when two command-line peers connect directly they see each other’s address. Other peers never see a browser’s address, except when the presenter turns on direct browser connections (WebRTC): then the presenter’s page and each participant who accepts see each other’s address. The default relays are run by n0 for development and hobby use; for anything else run your own with nuee relay or nuee serve --anchor.

The link is a capability

A link looks like https://maxgfr.github.io/nuee/#/J/AGG7…. Everything after # is the fragment, which browsers never send to the web server, so GitHub Pages never sees it. The letter says what the link is:

The payload is base32 of a version byte, the encoded body and a 4-byte BLAKE3 checksum. A link cut while copying is reported as truncated, a link from a newer version as unsupported, and a damaged one as corrupt, so the error says what to do. Anyone holding a link can use it: treat it like a password, show it in the room, do not post it publicly.

Admission

A joining peer proves it holds the secret without sending it: it sends an HMAC-SHA256 of both peers’ keys, keyed with the secret. The coordinator recomputes it and compares in constant time; because both keys are inside, the proof is useless on any other connection. A peer has 5 seconds to say hello, at most 64 handshakes run at once, and five wrong proofs in a minute shut a peer out for a minute. With --approve (or “Approve each device before it joins” on the page) every admitted peer also waits for a yes or no, and reads none of the swarm’s blobs until the yes. A relay address a peer announces is used only if it is the coordinator’s own relay or an https address outside private, loopback and link-local ranges, and so is any relay a peer names for a stored copy; only a link you open yourself may point at a local relay. A swarm takes at most 256 peers.

Jobs, tasks and leases

The coordinator signs a manifest for each job (kind, module hash, memory cap, number of tasks, verification policy, comparator, deadline) and sends it to every peer; workers refuse a manifest not signed by the key in their link. Workers say how many tasks they can take, and the coordinator hands out leases: one task, its input, a deadline. A worker runs it and returns the output with its signature. A task that fails (trap, out of memory, timeout) is handed to another peer, up to three attempts; a failure is never counted as cheating. A lease past its deadline is reassigned; the deadline is 60 seconds by default, then three times the peer’s median duration, between 5 seconds and 10 minutes. Outputs above 1 MiB travel as blobs that the coordinator fetches and checks against their hash. Workers apply their own limits on module size, memory, inputs and outputs whatever the manifest says, and keep at most 256 MiB of modules and assets between leases, only for the jobs they still hold.

Interrupted runs resume: outputs already in the output directory are kept, so the same command picks up where it stopped (--no-resume starts over).

Verification

Verification is recomputation. WebAssembly tasks give the same output bytes in wasmtime, Chromium, Firefox and Safari, so two honest runs are compared byte for byte. Kinds that use floats on a GPU or a model are compared with a measured tolerance instead (see What you can run).

When two results disagree, the coordinator runs the task itself and its result decides. Only then is the dishonest peer banned, and every result it produced that was not yet verified is computed again (retroactive redo). A result with a bad signature or a wrong hash bans its sender at once.

The coordinator keeps this role even when it takes no work (--local-slots 0, or “This page computes too” unchecked): checking runs on its own capacity, separate from contributing.

When the coordinator cannot run the kind (an exec job without --allow-exec, wgsl without a GPU, llm without --llm), there is nobody to arbitrate. Results are then only compared between two peers, a disagreement discards both, and nobody is banned. No canaries are sent, since a known answer is only one the coordinator computed itself; the peers in a disagreement have their other unverified results redone and need two agreeing executions from then on. nuee run says so before the link:

this machine cannot run these tasks, so it cannot check them: results are only compared between two peers, which one person holding the link can fake

A result is never accepted unchecked to keep such a job moving. If no second peer able to run the kind is connected, the tasks wait, and the coordinator says so once:

2 tasks wait for a second peer able to run them: this machine cannot check Exec results and no other admitted peer can run them. Join another device with --allow-exec for this command, or restart with --verify none to accept unchecked results

Browser keys are made fresh for each tab, so one person holding the link can open many tabs: “two different peers agree” proves little. Real assurance comes from the coordinator’s own recomputation, a higher sample rate, full, or --approve.

Corrections on the live board

The presenter’s page shows one square per task. A square fills with the ink of the device that computed it; a failed attempt is dashed; a rejected result is circled in red, and the circle stays after the task is redone and verified, with the honest device’s ink inside and a small red tick. The margin keeps a single red note with the number of corrections and, behind Show the corrections, the list: which task disagreed, which device was banned and how many tasks were redone. The device table shows each device’s fingerprint, type (browser or command line), tasks done and verified, state and average time.

What you can run

Each kind below has a command that was run for this page, and its limits. Timings marked “debug build” come from a development build of nuee on an Apple M5 with a local relay; a release build is faster.

• Verified CPU compute (WebAssembly)
• Your own module and job files
• GPU shaders (WGSL)
• Embeddings
• LLM option scoring
• Native commands (exec)
• Sweeps
• Files and durable storage
• Training
• Services and sites
• The anchor
• skycmd

Verified CPU compute (WebAssembly)

The default kind. A task module is plain WebAssembly with no imports at all, one memory with a declared maximum of at most 1 GiB, and two exports (nuee_alloc, nuee_run). Only a fixed list of WebAssembly features is accepted; relaxed SIMD, threads, memory64 and the other features that could make engines disagree are refused. A fresh instance runs each lease, with memory, time, stack and output caps. The same bytes give the same output in wasmtime, V8, SpiderMonkey and JavaScriptCore: the end-to-end tests check three golden outputs in all of them, and nuee selftest checks them in wasmtime.

nuee run demo:mandelbrot --verify full

Limits: modules up to 32 MiB, inline inputs and outputs up to 1 MiB (larger outputs travel as blobs), at most 32 leases per peer and 64 jobs per swarm.

Your own module and job files

Any module that follows the task ABI runs with a directory of inputs, one file per task, sorted by name:

nuee run my-task.wasm --inputs inputs/

2 tasks, verify Sample, 5 local slots, outputs in nuee-out/my-task
done: 2 tasks, results in nuee-out/my-task/results.json

A job TOML file describes the rest: the kind and its table ([wasm], [wgsl], [embed], [llm], [exec], [train]), verify, deadline, [inputs] (dir, or jsonl cut into shards of lines_per_task), [output] (dir, merge = "concat" or "jsonl") and [compare]. Unknown fields are errors. [wasm] assets hands files such as a model or a dataset to every task (at most 16, each at most 64 MiB); the module receives them in a small envelope before its input. The repository’s examples/skycmd/dagger.toml is one:

nuee run examples/skycmd/dagger.toml

It ran 16 tasks and merged 64,000 JSONL records into one file in 50 s (debug build, coordinator alone, 5 slots). The page’s “Your own module” entry takes a .wasm file and input files in the same way.

GPU shaders (WGSL)

A wgsl task is a compute shader run by wgpu natively (Metal, Vulkan, DX12) and by WebGPU in browsers. The shader is parsed and validated by naga before anything touches a device; buffers are capped at 64 MiB per binding and 128 MiB per task, and a task past its deadline loses its device.

nuee run demo:mandelbrot-gpu

The coordinator computes on its own GPU too (it printed recomputing samples on the GPU Apple M5); alone it finished in 5 s (debug build). Command-line workers offer their GPU only with --gpu:

nuee join - --gpu

offering the GPU Apple M5
joined with 5 slots
job 0: mandelbrot-gpu

GPUs do not agree bit for bit. Near the edge of the set the f32 iteration is chaotic: on the demo grid the worst tile has 89 of 4,096 pixels that differ between honest backends. The demo therefore accepts two outputs when at most 50 per mille of their values differ by more than 2 iterations. In plain words, up to 5 % of a tile’s pixels can be wrong without the recomputation noticing. Fine for a fractal; your own WGSL jobs set their own tolerance in [compare], and work that needs every value checked belongs in a wasm task.

WebGPU in browsers, as of October 2026:

A tab without WebGPU still joins and runs WebAssembly tasks; it just does not offer the GPU.

Embeddings

nuee embed turns text files into sentence vectors with all-MiniLM-L6-v2 (384 dimensions), on the CPU natively (candle, fp32) and with ONNX Runtime Web in browsers (q8). The model is pinned by revision, downloaded once from Hugging Face only, and every file is checked against its SHA-256.

nuee embed notes/

2 chunks in 1 tasks
checking the embedding model (downloaded once, about 90 MB)
embedding locally with all-MiniLM-L6-v2 on the cpu
1 tasks, verify Sample, 1 local slots, outputs in nuee-out/embed
2 vectors: nuee-out/embed/vectors.f32 (nuee-out/embed/meta.jsonl)
done: 1 tasks, results in nuee-out/embed/results.json

That first run took 16 s, model download included (debug build). vectors.f32 holds one row of 384 little-endian floats per chunk; meta.jsonl says where each chunk comes from. The two backends do not produce the same floats: the lowest cosine measured between them on the calibration corpus was 0.97173, so results agree when their cosine is at least τ = 0.966 and every row keeps an L2 norm of 1 (within 0.01). A forged vector that close to the true one would pass. On the page, “Embeddings” takes a .txt or .jsonl file.

LLM option scoring

An llm job scores decision records: each record becomes one prompt, the model is never sampled, and only the scores of the option letters are kept. On the command line a worker offers a local OpenAI-compatible endpoint that returns log-probabilities (llama.cpp’s llama-server, LM Studio, vLLM):

nuee join - --llm http://127.0.0.1:8080

A browser uses Qwen3.5-0.8B (ONNX, q4, pinned and checked by SHA-256), offered only on devices that do not report less than 4 GB of memory; the first use downloads about 650 MB. The endpoint address only ever comes from the worker’s own flag, never from a job. Scores are compared after centring: same best option, and no letter further apart than 1.5 across backends (0.25 is enough for one backend). This needs a running model server, which this page’s test machine did not have; the code path is covered by tests against a mock server. The guide is docs/GPU.md.

Native commands (exec)

An exec job runs a command from one exact Git commit, for work that needs native code (MLX, CUDA). It is off by default. A worker pins a repository and full 40-character commit, then approves each exact command line and environment before it runs:

nuee join - --allow-exec https://github.com/maxgfr/skycmd@781d4c176b2d6e301d0eeb82c69781b356505ec0

On a terminal, join shows each command of that commit in full (argv and environment values) and asks; another command at the same commit is asked about again. A worker without a terminal runs only the commands listed in --exec-approvals FILE, one JSON object per line ({"repo", "rev", "argv", "env"}), as the prompt and the refusal note print them. Without --allow-exec for the job’s commit, join never asks. Variables that change how programs load code or which program or configuration they start (LD_*, DYLD_*, *_OPTIONS, PYTHONPATH, NODE_OPTIONS, PATH, PIP_*, PAGER, SHELL…; the full list is in the threat model) are always refused, and the prompt shows every value.

The worker checks out that commit in a throwaway directory, checks the commit again, clears the environment, applies time limits (the checkout included) and output caps, kills the whole process group at the end, and uses sandbox-exec (macOS) or bwrap (Linux) when present, without network unless it also passed --exec-network. Jobs with gpu = true only go to workers that passed --gpu, one at a time. The command gets its input in $NUEE_IN and writes its output to $NUEE_OUT. nuee doctor reports whether a sandbox helper was found.

Approving a command means trusting the commit’s code with your user’s rights; the sandbox is best effort. Browsers never run exec. --exec-mirror URL=PATH fetches the allowed repository from a local clone instead of the network (air-gapped machines, tests); the commit is checked the same way. Accepted risk: an approved command can write ~/.cache and ~/Library/Caches even under the sandbox, because model and kernel caches need them (Hugging Face and MLX downloads), so it can leave files there that other programs later trust.

Sweeps

nuee sweep runs an exec job file with [[sweep]] entries, one task per entry, each entry’s args appended to the command, and ranks the entries by the metrics found in their output (loss, accuracy, acc and others). A sweep against a small local repository, run here on the coordinator itself, which approves the sweep file’s own commands for that commit:

nuee sweep sweep.toml --allow-exec https://example.org/lr-demo@23e3640dc26b718e57fa29ccf5e1d26d7d7e5d5b --exec-mirror https://example.org/lr-demo=../sweeprepo

name     status   duration  peer                acc
lr-2e-3  ok           0.2s  local            0.9000
lr-1e-3  ok           0.1s  local            0.8900
lr-3e-3  ok           0.2s  local            0.8900
3 sweep entries, outputs in nuee-out/lr-sweep

Each entry is a different experiment, so sweeps use verify = "none". Without --allow-exec only peers run the entries.

Files and durable storage

nuee share archives files and folders, encrypts the archive with age before it leaves the machine (file names included), and serves it until Ctrl-C:

nuee share report.txt photos/

https://maxgfr.github.io/nuee/#/F/AEAX7AQR4ZTK…
serving 304467 bytes (encrypted) until Ctrl-C

nuee get -o dl/ -

dl/report.txt
dl/photos/a.jpg
dl/photos/b.jpg
✓ verified (BLAKE3)

Every byte is checked against the BLAKE3 hash while it streams in, before decryption; a mismatch ends with exit code 6. get refuses to overwrite a file without --force and refuses archive entries that would land outside the output directory. The web page opens the same links, decrypts in the tab and says “Verified: the files match the link”; it shares and downloads up to 256 MiB.

nuee pin keeps a ciphertext-only copy (it drops the key before storing anything) and prints a pin link that names this node; nuee serve serves every pinned item, pin --list and pin --rm <HASH> manage them. A file link only names the providers it was created with, so to keep a file reachable after its sharer leaves, use durable storage.

Durable storage needs a running coordinator (a nuee run that waits, or an anchor) and members that offer disk space:

nuee run demo:mandelbrot --local-slots 0 --wait-peers 16

nuee join - --slots 1 --store 64MiB

nuee share archive.bin --replicas 2 --on -

https://maxgfr.github.io/nuee/#/F/AEA5ON4YA5NG…
waiting for the swarm to place the copies (Ctrl-C skips the wait)
https://maxgfr.github.io/nuee/#/F/AEBTOIQRMLDW…
serving 301925 bytes (encrypted) until Ctrl-C

Hand out the second link, the durable one: its providers are the holders. Members print offering 64 MiB of storage, 0 MiB in use and only ever receive ciphertext; the coordinator never gets the key either. With the sharer stopped and one holder killed, this still worked:

nuee get '<durable link>' --on '<swarm link>' -o dl3

The coordinator replaces a copy whose holder has been gone for 10 minutes, and challenges each holder every hour: it asks for a few 1 KiB ranges chosen from a fresh nonce (always including the last chunk) and checks them against the BLAKE3 root. A passed challenge proves the holder could produce those ranges at that moment, not that it stores the whole file; repeated challenges make a partial copy likely to be caught over time. Repair only happens while the coordinator runs. Limits: 8 replicas per blob, 1,024 blobs and 256 members per coordinator.

Only the peer that asked for a blob can release it, and each peer lists only its own; under --approve only peers you let in can ask, and banned peers never can. Each requester is limited to 128 tracked blobs and 64 GiB of copies, a blob that never got a first copy is dropped after 24 hours, and a member’s released or failed copies are removed from its disk (counted against its quota until then). Free room counts at most 64 GiB when picking members. A challenge proves possession of ranges, not independent copies: a member can relay ranges from another copy, and one link holder with several identities can hold every replica.

Instead of full copies, a Reed-Solomon set stores k data shards and m parity shards, one per member; any k rebuild the file. With 4+2, two members can disappear and storage costs 1.5 times the file:

nuee share big.bin --erasure 4+2 --on -

It prints a second link, to the set; nuee get on it fetched the header and enough shards, rebuilt and checked the ciphertext and decrypted it, with six members and the sharer stopped. The web page also opens set links, but only asks the providers printed in the link. Tabs do not pin and do not offer storage. Guide: docs/STORAGE.md.

Training

nuee train runs DiLoCo: data-parallel training where workers talk to the coordinator once per round instead of once per step. In ten lines:

1. The coordinator holds the parameters θ.
2. Each round, every worker receives θ.
3. It runs H inner steps of its own optimiser on its share of the data.
4. Its optimiser state stays on the worker.
5. It sends back Δ, its change over the round.
6. The coordinator drops deltas with non-finite values.
7. With three or more, it drops deltas pointing away from the median or much larger than it.
8. It averages the rest with a coordinate-wise trimmed mean (20 % at each end).
9. An outer step applies the average: SGD with Nesterov momentum, learning rate 0.7, momentum 0.9.
10. A checkpoint and train.json are written after every round, so a run resumes.

nuee train demo:train --wait-peers 2 --local-slots 0

round 1/10  loss 2.068  accepted 4  rejected 0
round 2/10  loss 1.402  accepted 4  rejected 0
…
round 10/10  loss 0.171  accepted 4  rejected 0
done: 10 rounds, model in nuee-out/toy-trainer/checkpoint-10.safetensors

With two nuee join workers of two slots each, the ten rounds took 7 s (debug build). The demo is a small digit recognizer on the UCI optdigits data set; the page’s “Train together” runs it with browser tabs and shows the loss curve and the final test accuracy.

Int8 with error feedback: per tensor, values are scaled to the int8 range, and what rounding loses is added to the worker’s next delta, so nothing is lost for good. Robust aggregation bounds what a minority can do in one round; it is weaker than recomputation, so use exec trainers with people you trust. Guide: docs/TRAINING.md.

Services and sites

nuee expose puts a local TCP service behind a link; it only ever connects to 127.0.0.1 on that port, so the tunnel cannot reach anything else on your network. It warns when nothing listens yet:

nuee expose 18000

https://maxgfr.github.io/nuee/#/X/AG7F53MK5YWD…
fingerprint 🐸 🍯 🧊 🌙 🏏 🌻  (compare with the screen of whoever joins)
nothing answers on 127.0.0.1:18000 yet (Connection refused (os error 61)); clients will be refused until it does
exposing 127.0.0.1:18000 until Ctrl-C

nuee connect --port 18001 -

open http://localhost:18001
tunnelling to the exposed service until Ctrl-C

Every client proves it holds the link’s secret; --approve asks before each new one, --no-http keeps raw TCP only. A console request starts with a small hello carrying the proof, checked before the request is read; connections that prove nothing are closed after 5 seconds. Opening an X link in the web page gives a request console: method, path, headers, JSON body. The console never renders what comes back: JSON is pretty-printed, text is shown as text, HTML as source, binary as a download.

Like ssh -L, nuee connect forwards every connection made to its local port, whoever makes it: a web page you visit while it runs can reach the remote service through DNS rebinding unless the service checks the Host header. Use a random port, close connect when you are done, and make exposed services check Host.

nuee connect passes the bytes untouched: a web app opened on http://localhost:<port> keeps its headers, cookies (Set-Cookie and Cookie), form posts and fetch() calls as if it ran locally. The anchor gateway, by contrast, strips cookies.

nuee site publish shares a folder with an index.html as an encrypted S link, and nuee site get downloads it:

nuee site publish ./public

nuee site get -o got -

got/public/index.html
✓ verified (BLAKE3)

The web page shows an isolated preview: the document is rebuilt without scripts, event handlers, frames or navigation, and shown in a sandboxed frame that has no script and no access to the page’s origin. A site with JavaScript previews as its static shell. Guide: docs/HOSTING.md.

The anchor

Nothing in Nuée needs an anchor. nuee serve --anchor is nuee on a machine that stays on: a persistent coordinator that keeps jobs across restarts, its own relay, a store member for durable copies, and an HTTP gateway for sites. Run locally with a development relay:

nuee serve --anchor --relay-dev --relay-bind 127.0.0.1:0 --gateway 127.0.0.1:18780

anchor link (keep it secret: it carries the admin secret):
https://maxgfr.github.io/nuee/#/X/AGGEAVKJMNKR…
swarm link: https://maxgfr.github.io/nuee/#/J/AGGEAVKJMNKR…
fingerprint ⛽ 🐍 🍳 🐧 🍼 🍅
gateway: http://127.0.0.1:18780
relay: http://127.0.0.1:55873

Hand it a job and get the results back; the job keeps running without the submitting machine:

nuee --relay http://127.0.0.1:55873 run demo:mandelbrot --anchor - --follow --out results

job 1 runs on the anchor 8c405549…a9b9; it keeps running without this
done: 257 files in results

A site published with --on <anchor link> is served by the gateway at /s/<id>/. That hands the anchor the decryption key, so its operator can read the site. Every gateway response carries Content-Security-Policy: sandbox, X-Content-Type-Options: nosniff, Referrer-Policy: no-referrer and Cross-Origin-Resource-Policy: cross-origin; --gateway-scripts allows scripts, still sandboxed and never same-origin. The gateway never relays a service’s Set-Cookie, replaces the visitor’s Forwarded, X-Forwarded-* and X-Real-IP headers with X-Forwarded-For set to the address it saw, and gives a request 10 seconds for its head and 30 more for its body. Serve the gateway on a host name of its own. The resource policy is cross-origin because a sandboxed page has an opaque origin: anything stricter would make browsers block the site’s own stylesheets, scripts and images.

In production the relay uses --relay-hostname and --acme-contact for a Let’s Encrypt certificate, which needs a real domain and ports 80 and 443; --relay-token-file, --relay-max-conns and --relay-rate restrict it, and --allow or --allow-file limit the swarm to listed peer ids. A browser passes a relay token in the URL, so it appears in the relay’s access logs. The repository ships a Docker Compose file and a systemd unit; setup, firewall ports, backups and upgrades are in docs/ANCHOR.md.

skycmd

skycmd trains small flight-decision models; it is the first real workload Nuée was built for.

• Evaluation (demo:skycmd): skycmd’s simulator compiled to WebAssembly plays four games with the s3-pico model, 4 games × 4 seeds × 10 episodes, decision threshold 0.6. The model is checked against its BLAKE3 hash before use, and results are the same bytes on every engine, so verification is exact. The page offers it as “skycmd evaluation”.
• DAgger records (demo:skycmd-dagger): the model plays, the game’s oracle labels every state it visits, and the 64,000 records are merged into one JSONL file for fine-tuning.
• Teacher labelling (examples/skycmd/label.toml): an exec job that scores 80,000 records in 40 shards on Macs with MLX, compared with a tolerance because MLX differs slightly between Macs.
• DiLoCo training of s4-micro (examples/skycmd/train-s4.toml): an exec trainer, 40 rounds × 500 inner steps on 4 workers.

nuee run demo:skycmd

game      runs  mean score  completion  crashes
dodge        4        8.65        0.72        0
doom         4        3.20        1.00        0
flappy       4      200.00        0.00        0
rescue       4        3.00        1.00        0
done: 16 tasks, results in nuee-out/skycmd/results.json

Measured and estimated, kept apart:

A source build without the embed-tasks feature runs the skycmd demos from a checkout of the repository; release binaries embed the module. Guide: docs/SKYCMD.md.

Security and privacy

Not protected, by design:

• A link holder is a member. Anyone who sees a link can use what it grants.
• The relay operator sees IP addresses and traffic volumes, never content. The n0 relays are for development and hobby use.
• WebRTC, when the presenter turns it on, shows the presenter and each participant who accepts each other’s IP address. It is off by default and both the switch and the join card say so.
• A coordinator chooses what its workers run. A task that stays inside its limits can still use all the CPU time it was granted or return nonsense.
• Sybil workers. Browser identities are free, so a ban costs a cheater a reconnect; in sample mode a wrong result that is neither sampled nor a canary is accepted, and in full mode so is a wrong result two of the cheater’s identities agree on, unless it is sampled. What stops them is the coordinator’s own recompute.
• Tolerances are room for a dishonest worker: 5 % of a GPU fractal tile, a cosine of 0.966 for embeddings, a logit gap for LLM scores.
• The fingerprint has 48 bits: it resists casual collisions, not a determined search against one fingerprint.

The full threat model, with the code that implements each mitigation, is docs/THREAT-MODEL.md; the wire format is docs/PROTOCOL.md.

Reporting a vulnerability

Use GitHub private vulnerability reporting on maxgfr/nuee, not a public issue. Include the affected version or commit, what an attacker needs, steps or a proof of concept, and the impact. Expect an acknowledgement within 72 hours. Only the latest release receives security fixes. Scope and details: SECURITY.md.

Command reference

Every verb prints or reads a link. nuee <verb> --help lists every flag; the key ones are below. A link argument accepts - to read it from standard input.

Verbs

Options of every verb

Global options go before the verb, for example nuee --relay http://127.0.0.1:55873 run demo:mandelbrot; every verb also accepts them after its name.

Exit codes

Environment variables

NDJSON events

With --json, each line on standard output is one JSON object whose event field names it, for example:

{"event":"link","fingerprint":["🗻","🎃","💧","🎣","🍉","🥐"],"kind":"job","link":"https://maxgfr.github.io/nuee/#/J/AGG7…"}
{"agent":"nuee-cli/0.0.0","event":"peer","id":"dfbe5bcc…c896ec","slots":5,"state":"joined"}
{"code":7,"event":"error","message":"interrupted; 0 of 256 outputs kept in … (run again to resume)"}

Troubleshooting and limits

error: relay unreachable: check your connection or pass --relay <url> (see `nuee relay`)

Exit code 3. No link is printed until the relay answers, within 10 seconds (NUEE_ONLINE_TIMEOUT_MS). Check the connection, or point at another relay with --relay. The page says “Cannot reach the relay. Check your connection and try again.”

error: the coordinator ended the swarm

Exit code 8 on nuee join: the coordinator stopped (its own run exits with 7 when interrupted, keeping finished outputs for a resume). The page says “The swarm has ended.” or, for a share whose source is gone, “This swarm or share is gone (or its host is offline).”

error: the link looks truncated; copy it again in full

Exit code 2. The link was cut while copying; the checksum at its end did not match. QR codes and - on standard input avoid this. A link from a newer version gives link version … is not supported; update nuee.

“This browser is too old for Nuée (WebAssembly SIMD is missing, as in Safari before 16.4).”

Tasks need WebAssembly SIMD. Update the browser or use another one.

“This browser does not run WebAssembly. On an iPhone or iPad, Lockdown Mode turns it off…”

Lockdown Mode disables WebAssembly. Allow the site in Lockdown Mode settings, or use another device.

… tasks wait for a second peer able to run them …

The coordinator cannot check this kind itself, and only one connected peer can run it. Join a second device that can (with --allow-exec, a GPU, --llm), or restart with --verify none to accept unchecked results. On the page: “1 task waits for a second device that can run this kind, to check it.”

No GPU tasks, or --gpu: no usable GPU adapter on this machine

A command-line worker takes wgsl tasks only with --gpu, and needs a usable adapter (nuee doctor shows it). A tab offers its GPU only when WebGPU is available; check chrome://gpu or about:support.

error: provider unreachable (…): is the sharer still online?

Exit code 3 on nuee get: nobody named in the link is serving it. Ask the sharer to run nuee share again, or use a durable link with --on <swarm link>.

error: dl/report.txt already exists (pass --force to overwrite)

nuee get never overwrites without --force.

native commands are disabled on this node (see --allow-exec)

The worker did not pin the job’s repo@rev; the join prompt shows both.

job N runs a native command this machine has not approved

The worker has no terminal to ask on and the command is not in its --exec-approvals file. The note prints the line to add; join again after adding it.

Limits, frankly

• Relay operators see browser IP addresses; WebRTC, when turned on, shows them to the presenter and participants.
• The n0 public relays are for development and hobby use.
• Browser keys are ephemeral, so agreement between two browser peers proves little.
• Mobile tabs stop in the background.
• The web page keeps files in memory: 256 MiB at most to share or download.
• Native trainers are not recomputed; GPU, embedding and LLM results are compared within tolerances.
• There is no Windows binary; Windows machines join through the browser.
• Nuée is for groups that trust each other enough to share a link. It is not an open market.

Licences and credits

• Nuée is free software under the MIT or Apache-2.0 licence, at your choice. The source is on GitHub.
• Fonts: Luciole by Laurent Bourcellier and Jonathan Fabreguettes (typographies.fr), under Creative Commons Attribution 4.0; JetBrains Mono, under the SIL Open Font License 1.1.
• Train together and demo:train use the UCI Optical Recognition of Handwritten Digits data set (Alpaydin and Kaynak, 1998), under Creative Commons Attribution 4.0.
• The skycmd evaluation uses the s3-pico model of the skycmd project, under the Apache-2.0 licence.
• Browsers run the embedding and LLM models with a self-hosted copy of Transformers.js 4.3.1 (Apache-2.0) and ONNX Runtime Web (MIT). The models are sentence-transformers/all-MiniLM-L6-v2 and Qwen3.5-0.8B, downloaded from Hugging Face at pinned revisions.
• Networking is iroh by n0; files are encrypted with age.