compiler optimization output

docs/testing.md

@@ -0,0 +1,170 @@
+---
+title: "Testing"
+description: "Writing and running tests in ƿit"
+weight: 45
+type: "docs"
+---
+
+ƿit has built-in support for writing and running tests. Tests live in the `tests/` directory of a package and are `.cm` modules that return a record of test functions.
+
+## Writing Tests
+
+A test file returns a record where each key starting with `test_` is a test function. A test passes if it returns `null` (or nothing). It fails if it returns a text string describing the failure.
+
+```javascript
+// tests/math.cm
+return {
+    test_addition: function() {
+        if (1 + 2 != 3) return "expected 3"
+    },
+
+    test_division: function() {
+        if (10 / 3 != 3.333333333333333333) return "unexpected result"
+    }
+}
+```
+
+Test functions take no arguments. Use early returns with a failure message to report errors:
+
+```javascript
+test_array_push: function() {
+    var a = [1, 2]
+    a[] = 3
+    if (length(a) != 3) return "expected length 3, got " + text(length(a))
+    if (a[2] != 3) return "expected a[2] to be 3"
+}
+```
+
+## Running Tests
+
+```bash
+pit test                          # run all tests in current package
+pit test suite                    # run a specific test file (tests/suite.cm)
+pit test tests/math               # same, with explicit path
+pit test all                      # run all tests in current package
+pit test package <name>           # run all tests in a named package
+pit test package <name> <test>    # run a specific test in a named package
+pit test package all              # run tests from all installed packages
+```
+
+### Flags
+
+```bash
+pit test suite -g          # run GC after each test (useful for detecting leaks)
+pit test suite --verify    # enable IR verification during compilation
+pit test suite --diff      # run each test optimized and unoptimized, compare results
+```
+
+`--verify` and `--diff` can be combined:
+
+```bash
+pit test suite --verify --diff
+```
+
+## IR Verification
+
+The `--verify` flag enables structural validation of the compiler's intermediate representation after each optimizer pass. This catches bugs like invalid slot references, broken jump targets, and malformed instructions.
+
+When verification fails, errors are printed with the pass name that introduced them:
+
+```
+[verify_ir] slot_bounds: slot 12 out of range 0..9 in instruction add_int
+[verify_ir] 1 errors after dead_code_elimination
+```
+
+IR verification adds overhead and is intended for development, not production use.
+
+## Differential Testing
+
+Differential testing runs each test through two paths — with the optimizer enabled and with it disabled — and compares results. Any mismatch between the two indicates an optimizer bug.
+
+### Inline Mode
+
+The `--diff` flag on `pit test` runs each test module through both paths during a normal test run:
+
+```bash
+pit test suite --diff
+```
+
+Output includes a mismatch count at the end:
+
+```
+Tests: 493 passed, 0 failed, 493 total
+Diff mismatches: 0
+```
+
+### Standalone Mode
+
+`pit diff` is a dedicated differential testing tool with detailed mismatch reporting:
+
+```bash
+pit diff                   # diff all test files in current package
+pit diff suite             # diff a specific test file
+pit diff tests/math        # same, with explicit path
+```
+
+For each test function, it reports whether the optimized and unoptimized results match:
+
+```
+  tests/suite.cm: 493 passed, 0 failed
+----------------------------------------
+Diff: 493 passed, 0 failed, 493 total
+```
+
+When a mismatch is found:
+
+```
+  tests/suite.cm: 492 passed, 1 failed
+    MISMATCH: test_foo: result mismatch opt=42 noopt=43
+```
+
+## Fuzz Testing
+
+The fuzzer generates random self-checking programs, compiles them, and runs them through both optimized and unoptimized paths. Each generated program contains test functions that validate their own expected results, so failures catch both correctness bugs and optimizer mismatches.
+
+```bash
+pit fuzz                   # 100 iterations, random seed
+pit fuzz 500               # 500 iterations, random seed
+pit fuzz --seed 42         # 100 iterations, deterministic seed
+pit fuzz 1000 --seed 42   # 1000 iterations, deterministic seed
+```
+
+The fuzzer generates programs that exercise:
+
+- Integer and float arithmetic with known expected results
+- Control flow (if/else, while loops)
+- Closures and captured variable mutation
+- Records and property access
+- Arrays and iteration
+- Higher-order functions
+- Disruption handling
+- Text concatenation
+
+On failure, the generated source is saved to `tests/fuzz_failures/` for reproduction:
+
+```
+Fuzzing: 1000 iterations, starting seed=42
+  FAIL seed=57: diff fuzz_3: opt=10 noopt=11
+    saved to tests/fuzz_failures/seed_57.cm
+----------------------------------------
+Fuzz: 999 passed, 1 failed, 1000 total
+Failures saved to tests/fuzz_failures/
+```
+
+Saved failure files are valid `.cm` modules that can be run directly or added to the test suite.
+
+## Test File Organization
+
+Tests live in the `tests/` directory of a package:
+
+```
+mypackage/
+├── pit.toml
+├── math.cm
+└── tests/
+    ├── suite.cm        # main test suite
+    ├── math.cm         # math-specific tests
+    └── disrupt.cm      # disruption tests
+```
+
+All `.cm` files under `tests/` are discovered automatically by `pit test`.