Method · a research project, built by agents

How a language for agents is being built by agents.

Aleph is two experiments at once. The first asks whether a language designed for machines lets agents write correct, contained programs with fewer tokens and fewer attempts. The second is how it is made: the specification is written by people, the compiler by an autonomous agent in a loop, with every step recorded in the repository. This page explains the method, what has been measured and what has not.

Status: v0.1, experimentalUpdated: 16 Nov 2026Main hypothesis: not yet tested

1 · Hypotheses

What the project is trying to prove.

  • H1 · Fewer tokensAn agent finishes the same task in fewer tokens in Aleph than in Python, counting the spec it has to load and every retry.
  • H2 · Fewer attemptsDiagnostics in JSON, with the exact node and the expected value, let an agent reach a passing program in fewer rounds.
  • H3 · Contained by constructionDeclared capabilities, checked at compile time and enforced at run time, stop a program from touching anything it did not declare.
  • H4 · Native speed for freeThe same source runs in the interpreter and as native code with byte-identical output, with no tuning by the agent.

2 · The setup

People write the rules. Agents write the code.

The work is split by who is trusted with what. People wrote the design document and the specification and decide what is accepted. The agent implements, tests and reviews, one task at a time, and can never change the rules it works under.

  • Design document · peoplev1.0, 10 November 2026: vision, non-goals, the language, the runtime Tlön, the registry Uqbar, the corpus Orbis.
  • Specification · peoplespec/SPEC.md and grammar.ebnf: the source of truth. When it is silent the agent may not guess silently: it writes a Proposed ADR and continues.
  • Rules · peopleAGENTS.md: tests first, one task per run, no network, no push, never weaken a test, deterministic output, a fixed commit format.
  • Implementation · agentDeepSeek v4.1 flash through opencode, driven by an orchestrating coding agent, with separate plan, implementation, test and review passes.

3 · The loop

An agent with no memory, and a repository that remembers.

Every iteration starts a fresh agent that knows nothing of the previous one. Its only memory is the repository: the task list, the progress log, the decisions and the git history. That makes every step inspectable and every run repeatable.

$ loop/run.sh build
timeout 1h per iteration · 30 s pause
stop: STOP file · no open tasks
stop: 4 iterations without a commit
  1. 01Read the stateAGENTS.md, the last 60 lines of PROGRESS.md, TASKS.md.
  2. 02Make the tree greenRun cargo fmt, clippy -D warnings and every test. If anything is red, that is the task.
  3. 03Take one taskThe first unchecked one that is not blocked, plus the spec sections and ADRs it cites.
  4. 04Tests firstGolden programs with expected output or diagnostics, then unit tests, then the code.
  5. 05Review until it holdsA review pass scores the change and lists problems; the agent fixes them and reviews again. The language card went 7 → 9 → 9 in three cycles.
  6. 06Record and commitTick the task, append to the log, write an ADR for any decision, one commit with intent, files, capabilities and verification.
  7. 07Know when to stopTwo failed runs on the same task mark it BLOCKED with the reason, and the loop moves on.

4 · How correctness is checked

Nothing counts until a test says so.

  • Golden programs90 .aph files with the exact stdout, stderr, exit code or JSON diagnostics they must produce.
  • Differential testingEvery runnable golden program runs in the interpreter and as native code; output and traps must match byte for byte.
  • DeterminismSame input, same bytes: diagnostics, formatter, AST and generated C. One canonical form per program.
  • Generated documentationllms.txt is produced by the compiler from its own tables and tested against its token budget, so it cannot drift.

5 · When people step in

The agent misses what its tests cannot see.

On 16 November an external review of the agent’s work found two problems the agent’s own test runs had not shown. Native code was about 75 times slower than C, because every expression was boxed and counted. And sorting floats gave a different order on x86-64 than on arm64, the only machine the agent had tested on.

Both were fixed by hand, in public, as a pull request on top of the agent’s branch: an unboxed fast path with an exact fallback, a NaN order that no longer depends on the processor, three new tests at the depth limit and a new decision record. The lesson goes back into the method: benchmarks and a second architecture now belong in the check, not only in a later review.

fib(35): 1.71 s → 0.115 sADR-0027 · PR #1

6 · Results so far

What has been measured, and where it comes from.

MeasureValueSource
Tasks completed24 of 33, including PR #1TASKS.md
Time for core compiler and native backend≈ 13 hoursgit log, 15 Nov 10:21 → 23:18
Agent commits18branch agent/aleph-m3
Decision records27 (14 still Proposed)adr/
Tests passing288 of 288cargo test
fib(35) native vs CPython0.115 s vs 1.12 sbench/heavy, x86-64
fib(35) re-run: native vs C vs CPython0.091 s vs 0.015 s vs 0.78 sbench/heavy + bench/ref, second x86-64 machine, median of 3, 16 Nov
Tokens per task vs Python[not measured]A/B benchmark, planned
Attempts to a passing program[not measured]Orbis corpus, planned
Spot check: new programs written from llms.txt alone2 of 2 pass on attempt 2daily review, aleph check/run/build, 16 Nov (n = 2, not the planned measure)

This table is updated by a daily review of the repository when something important changes.

7 · Limits

Why these results do not prove the hypothesis yet.

  • H1 and H2 are untestedNo token or attempt counts exist yet. Until the A/B benchmark runs, Aleph’s main claim is a design intention, not a finding.
  • One model familyAlmost all code comes from a single worker model. Its habits and blind spots shape the compiler.
  • Self-reviewThe same system writes and reviews. Human review found problems the agent’s reviews scored 9 out of 10.
  • One machine, small benchmarksSpeed numbers come from one x86-64 machine and short programs like fib. They show the backend works, not that real workloads are fast.

8 · Next experiments

What will decide whether Aleph is worth it.

  • A/B against Python20 tasks, same model, same budget. Measure success rate, total tokens including llms.txt, and rounds to a passing program.
  • Orbis: the corpusAgents write verified programs and keep every repair pair: broken source, diagnostics, fixed source. A public dataset of how agents learn a new language.
  • Real tokenizersaleph tokens counts with the tokenizers models actually use, not characters.
  • A second architecture in CIRun every test on x86-64 and arm64, so portability bugs surface where they are made.