Every number on this page
comes from a script you can run.

D1The repair loop closes

Claim: an agent can take broken source, get a structured diagnosis, repair it, and reach a native binary — fully mechanically.

Test: scenario 1 feeds a program with an undefined variable to astra verify --error-format=json, asserts the JSON carries the exact line/column/error_code, applies the fix, and asserts the binary compiles and prints the expected output.

Result: pass — broken → JSON(line 2, col 13, E0002) → repaired → native, in one scripted pass.

D2Agent mistakes rejected before execution

Claim: the classes of error LLMs most commonly generate never reach runtime.

Test: scenario 2 submits four canonical mistake programs; each must be rejected by verify with the right error class, and none may produce a binary.

Mistake class	Example	Caught pre-run
Undefined variable	print(totl) (typo)	yes — E0002
Type mismatch	let n: int = "hello"	yes — E0004
Wrong argument count	add(1) for add(a, b)	yes — E0005
Invalid return type	returning str from -> int	yes — E0004

Result: 4/4 rejected — in Python, all four would execute until the faulty line is reached (or silently succeed).

D3Native speed — including compile time

Claim: for compute-shaped work, Astra's compile-then-run beats Python's run alone.

Test: scenario 3 runs an identical 5,000,000-iteration arithmetic loop in both languages, wall-clock timed.

Result: ~8.3× faster — and the Astra figure includes full compilation (parse, typecheck, LLVM IR, llc, link). Run-only, the gap widens further.

D4Verification is fast enough for every loop iteration

Claim: verify is cheap enough that an agent can call it after every edit — like a linter, with compiler guarantees.

Test: time astra verify (full parse + typecheck, no codegen) on a representative program.

Result: 113 ms wall-clock. At that latency, verification costs less than a fraction of a single LLM token round-trip — there is no reason for an agent not to verify.

D5Deterministic dependency resolution

Claim: the same manifest always resolves to the same lockfile, byte for byte.

Test: resolve a workspace, hash astra.lock, wipe, resolve again, compare sha256.

Result: sha256-identical across repeated runs. A build an agent verified once stays verified — no drifting transitive dependencies.

Cross-languageAstra vs C vs Rust vs Go vs Java vs Node.js vs Python 3

Claim: Astra binaries run CPU-bound work in the same class as C, Rust, and Go — ahead of the JVM and a warmed-up JIT, far ahead of interpreters — with no VM on the target machine.

Test: scripts/run_benchmarks.sh runs the identical algorithm in all seven languages and fails if the outputs don't match, so no benchmark can quietly compute something different. Best-of-3 wall-clock, macOS ARM64: Astra v0.8.0-alpha (default LLVM-IR pipeline) · Apple clang 17 (-O2) · rustc 1.94 (-O) · go 1.26 · OpenJDK 25 LTS · Node v22 · Python 3.9. Run-only unless noted.

Benchmark	Astra	C -O2	Rust -O	Go	Java	Node.js	Python 3	Astra compile+run
Recursive fib(30)	101 ms	110 ms	103 ms	104 ms	135 ms	138 ms	449 ms	531 ms
10M-iteration arithmetic loop	117 ms	108 ms	111 ms	110 ms	146 ms	146 ms	2,088 ms	295 ms
Collatz steps for 1..100,000	117 ms	114 ms	117 ms	120 ms	152 ms	178 ms	1,812 ms	567 ms

10M-loop benchmark, run time. The full data and the seven-language benchmark sources ship with the compiler source (opens at v1.0-beta).

Result: C/Rust/Go-class native speed — statistically indistinguishable from C, Rust, and Go across all three workloads (±10 ms on a ~110 ms measurement is scheduler noise). 4–18× faster than Python run-only, with even compile + run beating Python's run-only on two of three workloads. Ahead of Java by ~30 ms of JVM startup per process, and a modest but consistent win over Node — with no JIT warm-up and no 40+ MB runtime on the target machine.

SuiteCorrectness metrics

Metric	Value	Source
Test suite	449 passed · 0 failed · 0 ignored	cargo test --quiet (v0.9.1-alpha)
Build warnings	0 — CI enforces clippy -D warnings	cargo build --quiet
Ignored-test burn-down	10 → 0 (complete)	Release KPI: tests un-ignore only when the feature runs natively end-to-end
End-to-end regression suite	108 / 108 — compile → IR → native run → stdout/exit-code asserted	tests/regression_tests.rs
Showcase examples runtime-verified	compile + execute + stdout asserted	tests/showcase_examples_tests.rs
IR validity gate	every corpus program's IR must pass llc	tests/ir_validity_tests.rs (in CI)
Validation checks	12 / 12	scripts/validate_metrics.sh

ReproduceRun it yourself

The compiled-binary side reproduces today: download a release, write the loop, and time it —

$ astra verify loop.astra               # the ~100 ms typecheck
$ time astra compile loop.astra --output loop && ./loop
$ time python3 loop.py                  # the comparison

The full scripted harness (validate_metrics.sh, run_benchmarks.sh, the output-verified seven-language scenario programs, and the 449-test suite) ships with the compiler source, which opens at v1.0-beta. If a number on this page doesn't reproduce on your machine, that's a bug — file it.