Every number, reproducible.

Astra against C, Rust, Go, Java, Node.js and Python

Test: seven workloads, each written the normal way in every language, with the input size N passed on the command line (as the Computer Language Benchmarks Game does), so no compiler can precompute a result. benchmarks/run.py times each program as one process run (one warm-up, then the median of 7 runs; 2 for Python and the Astra interpreter) and fails if any output differs. Spread between runs was typically a few percent; the machine was not idle (load average 2–3.5 of 12 cores). Measured 2026-11-11 on v0.13.5-alpha. Apple clang 17 -O2 · rustc 1.94 -O · Go 1.26 · OpenJDK 25 · Node 25 · Python 3.11 · Astra via LLVM -O2. Median milliseconds, lower is better.

"Rust -O, checked" is Rust with overflow-checks=on. Astra checks every integer operation for overflow and stops with a runtime error, so this is the like-for-like comparison. Java and Node.js include their VM start-up (about 40 ms) in every run.

What the numbers say

• Floating point: in mandelbrot (from the Benchmarks Game) Astra is faster than C and plain Rust and ties overflow-checked Rust: 72 ms against 85 ms (C, Rust) and 72 ms (checked Rust).
• Plain computation: Astra is at parity with overflow-checked Rust on fib, loop and collatz (180 / 149 / 224 ms against 180 / 141 / 222). Against unchecked Rust and C it is 1.3–2.6× slower, the price of always-on overflow checking (unchecked compilers can also vectorize the plain loop), not of the code generator.
• Structs and arrays: structs match checked Rust; arrays are within about 15%. A struct that doesn't outlive its scope now lives on the stack, and array reads and bounds checks are compiled inline (each workload was 5–18× slower than Rust before).
• Strings: faster than C and Rust in this workload (149 ms against 260–287): "item-" + i.to_string() formats the number straight into the result, with no temporary string and no printf. Go is faster still (108 ms).
• Against Python: native Astra is about 5–200× faster.
• The interpreter (astra run, no LLVM needed) beats CPython on the plain loop (9.1 s vs 25.5), arrays and collatz, and trails on recursive calls (fib), mandelbrot, structs and strings (by 10–30%; in the 2026-10-30 run it was ahead on mandelbrot). A redesign landed this round: each function body is lowered once to slot-resolved form, no name lookups, no AST-walk dispatch, then extended to structs, arrays, tuples, templates, for-each and match.

Native speed, including compile time

Claim: for compute-shaped work, Astra's compile-then-run beats Python's run alone.

Test: scenario 3 runs the same 2,000,000-iteration arithmetic loop in both languages; each command is timed as one process run.

Result: ~2× faster, compile included, 93 ms to compile (parse, type-check, LLVM IR, llc, link) plus 4 ms to run, against 191 ms for Python to run the loop. Run alone, the binary is about 45× faster.

Verify after every edit

Claim: verify is cheap enough that an agent can call it after every edit, like a linter, with compiler guarantees.

Test: time astra verify (full parse + typecheck, no codegen) on a representative program.

Result: about 6 ms wall-clock. At that latency verification costs far less than a single LLM round-trip, there is no reason for an agent not to verify after every edit.

The repair loop closes

Claim: an agent can take broken source, get a structured diagnosis, repair it, and reach a native binary, fully mechanically.

Test: scenario 1 feeds a program with an undefined variable to astra verify --error-format=json, asserts the JSON carries the exact line/column/error_code, applies the fix, and asserts the binary compiles and prints the expected output.

Result: pass, broken → JSON(line 2, col 13, E0002) → repaired → native, in one scripted pass.

Agent mistakes rejected before execution

Claim: the classes of error LLMs most commonly generate never reach runtime.

Test: scenario 2 submits four canonical mistake programs; each must be rejected by verify with the right error class, and none may produce a binary.

Result: 4/4 rejected, in Python, all four would execute until the faulty line is reached (or silently succeed).

Deterministic dependency resolution

Claim: the same manifest always resolves to the same lockfile, byte for byte.

Test: resolve a workspace, hash astra.lock, wipe, resolve again, compare sha256.

Result: sha256-identical across repeated runs. A build an agent verified once stays verified, no drifting transitive dependencies.

Correctness metrics

The suite passes with nothing skipped, and every release is checked from the outside before it ships.

Real programs, checked against Python

Eight everyday programs give exactly the same answers as a Python reference, on both engines.

Each program solves a task developers actually have: summarising a server log, a CSV sales report, validating a config file, restocking inventory, grading, a password policy, text search and a Mandelbrot plot. Each has an independent Python reference, and Astra's output must match it byte for byte on the interpreter and natively. Writing them found three native wrong answers, fixed in v0.12.2 and logged with the tests that guard them; 74 more probe programs found eight bugs, fixed in v0.13.5. Since v0.13.2 none of the eight needs a workaround: they count with Map and rank with sort_by_key as a developer would. Measured on v0.13.5-alpha, 2026-11-11.

The benchmark is benchmarks/adoption/run.py; it will be published with the source.

Run it yourself

Once public downloads open (coming soon), the compiled-binary side reproduces in a minute: write the loop and time it —

$ astra verify loop.astra               # type-check: a few ms
$ time astra compile loop.astra --output loop && ./loop
$ time python3 loop.py                  # the comparison

The full harnesses (scripts/validate_metrics.sh, benchmarks/run.py with the seven-language programs, and the test suite) ship with the compiler source, which opens at v1.0-beta. If a number on this page doesn't reproduce on your machine, that's a bug, tell us.