rust_browser/docs/test262_roadmap.md

# Test262 Roadmap

What needs to be built to run real [Test262](https://github.com/tc39/test262) conformance tests.

## Current State

We have a custom JS262 harness (`tests/js262_harness.rs`) with 214 hand-written tests plus
550 real Test262 tests vendored from `tc39/test262`. The real Test262 suite has 50,000+ tests.
Phase A vendored 50 tests; Phase B expanded to 550 with automated triage.

## Tier 1: Test262 Harness Requirements

These features are used by `assert.js` and `sta.js` — the support files that every Test262
test includes. **All of these must work before any real Test262 test can run.**

| Feature | Used For | Parser | Interpreter | Effort |
|---------|----------|--------|-------------|--------|
| `try` / `catch` / `finally` | `assert.throws()` wraps test code in try/catch | **Done** | **Done** | Medium |
| `throw` statement | `sta.js` throws `Test262Error` on failure | **Done** | **Done** | Small |
| Logical `&&` / `\|\|` | Short-circuit guards in assertions | **Done** | **Done** | Small |
| Object literals `{ k: v }` | Building error message objects | **Done** | **Done** | Medium |
| Property assignment `obj.prop = v` | `assert.sameValue = function ...` | **Done** (dot only) | **Done** (host + plain objects) | — |
| Prototype chain | `Test262Error.prototype.toString = ...` | N/A | **Done** | Large |
| `this` keyword | Constructor bodies: `this.message = msg` | **Done** | **Done** | — |
| `switch` / `case` | `assert._toString` type formatting | **Done** | **Done** | Medium |
| Template literals `` `${x}` `` | Assertion error messages | **Done** | **Done** | — |
| `new` + constructor functions | `new Test262Error(msg)` | **Done** | **Done** | Prototype-linked instances |
| String methods (`.name`, `.call`) | `Object.prototype.toString.call()` | **Done** | **Done** | Medium |

### Minimum viable slice

To run the simplest Test262 tests, implement in this order:

1. ~~**`throw`** — trivial addition to parser + interpreter~~ **Done**
2. ~~**`try` / `catch`** — parser + interpreter control flow~~ **Done**
3. ~~**Logical operators `&&` / `||`** — parser + short-circuit eval~~ **Done**
4. ~~**Object literals** — parser + runtime plain-object type~~ **Done**
5. ~~**`this` keyword** — parser + interpreter binding~~ **Done**
6. ~~**`switch` / `case`** — parser + interpreter control flow~~ **Done**
7. ~~**Template literals** — tokenizer + parser + string interpolation eval~~ **Done**

After these 7 features, we can shim `assert.js` and `sta.js` and run Test262 tests that
only use basic syntax (variable declarations, functions, if/else, comparisons).

## Tier 2: Common Test File Features

Most Test262 tests go beyond basic syntax. These features unlock large swaths of the suite.

| Feature | % of Test262 tests using it | Parser | Interpreter | Effort |
|---------|----------------------------|--------|-------------|--------|
| `for` loops | ~60% | **Done** | **Done** | Medium |
| `for...in` / `for...of` | ~25% | **Done** | **Done** | Medium |
| `while` / `do...while` | ~15% | **Done** | **Done** | Small |
| Array literals `[1,2,3]` | ~50% | **Done** | **Done** | Medium |
| Computed member `obj[key]` | ~40% | **Done** | **Done** | Small |
| Arrow functions `=>` | ~30% | **Done** | **Done** | Medium |
| Ternary `? :` | ~20% | **Done** | **Done** | Small |
| Compound assignment `+=` etc. | ~20% | **Done** | **Done** | Small |
| Increment/decrement `++`/`--` | ~15% | **Done** | **Done** | Small |
| Destructuring | ~10% | **Done** | **Done** | Large |
| Spread / rest `...` | ~10% | **Done** | **Done** | Medium |
| `class` declarations | ~10% | **Done** | **Done** | Large |

### Recommended implementation order

After Tier 1 is complete:

1. ~~**`for` loops** — unlocks the most tests~~ **Done**
2. ~~**Array literals + computed member** — tightly coupled, unlock array tests~~ **Done**
3. ~~**Arrow functions** — widely used in modern test262 tests~~ **Done**
4. ~~**`while` / `do...while`** — small effort, moderate test coverage~~ **Done**
5. ~~**Ternary `? :`** — trivial parser change~~ **Done**
6. ~~**Compound assignment / increment-decrement** — small parser + desugar~~ **Done**
7. ~~**Spread / rest `...`** — arrays, strings, function args/params, object spread~~ **Done**
8. ~~**Destructuring** — array/object destructuring in declarations, assignments, parameters, for-of/in~~ **Done**

## Tier 3: Richer Runtime

These require more substantial runtime work beyond parsing and basic interpretation.

| Feature | What it unlocks | Effort | Parser | Interpreter |
|---------|----------------|--------|--------|-------------|
| Plain JS objects | Object literals, enumeration, `Object.keys/values/entries`, `Object.create/assign/getPrototypeOf`, `hasOwnProperty`, `propertyIsEnumerable`, non-enumerable properties | Medium | **Done** | **Done** |
| Prototype chain / `Object.create` | Inheritance, method resolution, `for...in` inherited enumeration | Large | **Done** | **Done** |
| `JSON.stringify` / `JSON.parse` | Harness error messages, JSON tests | Medium | N/A | **Done** |
| `String.prototype` methods | `.length`, `.charAt()`, `.slice()`, `.indexOf()` | Medium | N/A | **Done** |
| `Array.prototype` methods | `.push()`, `.pop()`, `.map()`, `.forEach()` | Large | N/A | **Done** (push/pop/join/indexOf/slice/concat/toString/map/forEach/filter/find/findIndex/some/every/reduce/reduceRight/sort/includes/lastIndexOf/reverse/shift/unshift; splice/flat/fill deferred) |
| `RegExp` | Regex literal tests, string matching | Very Large | **Done** | **Done** (literals, constructor, `test`, `exec`, `toString`, flags, `lastIndex`, String integration; no function replacers, no `matchAll`/`replaceAll`) |
| Closures over mutable state | Counter patterns, module-like scoping | Medium | **Done** | **Partial** (no heap-allocated closure env for returned functions) |
| `Symbol`, `WeakMap`, `Proxy` | Advanced ES6+ tests | Very Large | N/A | **Done** (Symbol with well-known symbols, WeakMap, Proxy with get/set/has/deleteProperty/apply traps) |
| `eval()` | Direct eval with scope access, indirect eval, strict mode isolation | Medium | N/A | **Done** |

## Integration Plan

### Phase A: Shim the Harness (after Tier 1) — **Done**

1. ~~Add `mode = "test262"` to the JS262 manifest schema~~ **Done**
2. ~~Before running each test, prepend the contents of `sta.js` and `assert.js`~~ **Done**
3. ~~Map `Test262Error` exceptions to test failure outcomes~~ **Done**
4. ~~Vendor a curated subset of ~50 basic Test262 tests into `tests/external/js262/test262/`~~ **Done**
5. ~~Add manifest entries with `mode = "test262"` and feature tags from Test262 metadata~~ **Done**

#### Phase A Results

- **50** real Test262 tests vendored from `tc39/test262`
- **35 pass** (70%), **15 known_fail** (30%)
- Feature areas covered: ASI (5), block-scope (20), comments (5), directive-prologue (5), expressions (15)
- Known-fail breakdown: ASI gaps (2), label syntax (2), function declarations in blocks (2), strict mode (5), other parser gaps (4)
- New JS feature added during Phase A: function property support (`fn.prop = val`, `fn.prop`, `fn.method()`)

### Phase B: Expand Coverage (after Tiers 1+2) — **Done**

1. ~~Vendor ~500 tests covering variables, expressions, functions, control flow~~ **Done** (500 new tests)
2. ~~Use Test262's `features` metadata to filter to supported features only~~ **Done**
3. ~~Script the manifest generation: parse Test262 YAML frontmatter → TOML entries~~ **Done**
4. ~~Track pass rate as a project metric~~ **Done** (in `scripts/metrics.sh`)

#### Phase B Results

- **550** total real Test262 tests (50 from Phase A + 500 new)
- **263 pass** (47.8%), **287 known_fail** (52.2%)
- Feature areas covered: ASI (25), block-scope (39), comments (15), computed-property-names (18), destructuring (14), directive-prologue (25), expressions (414)
- Automated triage promoted 155 known_fail tests to pass on first run
- Updated `assert.throws` to use `instanceof` for error type checking
- ASI implementation promoted 39 tests (16 ASI, 4 comments, 19 expressions)
- Error constructors (Error, TypeError, ReferenceError, SyntaxError, RangeError, EvalError, URIError) promoted 72 tests
- New scripts: `scripts/curate_test262.py` (updated), `scripts/triage_test262.py` (new)
- New `just` commands: `curate-test262`, `triage-test262`

### Phase C: Full Test262 Integration — **Done**

1. ~~Clone tc39/test262 upstream to gitignored `tests/external/js262/upstream/`~~ **Done**
2. ~~Generate full-suite manifest (`js262_full_manifest.toml`) from upstream clone~~ **Done**
3. ~~Dynamic harness include loading (30+ harness files, not just `sta.js`/`assert.js`)~~ **Done**
4. ~~Two-tier system: Tier 1 (vendored, PR gate) + Tier 2 (full suite, `just test262-full`)~~ **Done**
5. ~~Automated triage script (`just triage-test262-full`)~~ **Done**
6. ~~ES5.1 subset tracking in metrics~~ **Done**

#### Phase C Infrastructure

- `scripts/clone_test262.py` — shallow-clones tc39/test262 to `upstream/`
- `scripts/scan_test262.py` — scans upstream, generates full manifest with all qualifying tests
- `scripts/triage_test262_full.py` — promotes passing known_fail tests, updates `js262_full_status.toml`
- Status overrides: `tests/external/js262/js262_full_status.toml` (checked in, simple `id = "pass"` format)
- New `just` commands: `clone-test262`, `test262-scan`, `test262-full`, `triage-test262-full`
- Full suite is NOT part of `just ci` / `just test` — runs separately via `just test262-full`
- Target: 90%+ pass rate on ES5.1 subset

## Quick Reference: Feature Gap Summary

**Blocking Test262 harness (must fix first):**
~~`this`~~, ~~`switch`~~, ~~template literals~~, ~~string methods / `.name` / `.call()`~~, ~~prototype chain~~ — All done.

**Blocking large test coverage:**
*(none — arrow functions now implemented)*

**Already implemented:**
`var`/`let`/`const`, `if`/`else`, functions (declaration + expression + `new`),
`typeof`, `===`/`!==`/`==`/`!=`, arithmetic, string concat, `!` (not),
object literals, `try`/`catch`/`finally`, `throw`, `&&`/`||`,
string methods (`.length`, `.charAt()`, `.indexOf()`, `.slice()`, etc.),
`Function.name`, `Function.prototype.call()`, `Object.prototype.toString.call()`,
`for`/`for...in`/`for...of`/`while`/`do-while` loops, `break`/`continue`, `++`/`--`, `+=`/`-=`/`*=`/`&=`/`|=`/`^=`/`<<=`/`>>=`/`>>>=`,
bitwise operators (`&`, `|`, `^`, `~`, `<<`, `>>`, `>>>`),
exponentiation (`**`, `**=`), nullish coalescing (`??`), logical assignment (`&&=`, `||=`, `??=`),
`delete`, `void`, `in` operator,
function properties (`fn.prop = val`, `fn.method()`),
array literals (`[1,2,3]`), computed member access (`obj[key]`),
array methods (`.push()`, `.pop()`, `.join()`, `.indexOf()`, `.slice()`, `.concat()`, `.map()`, `.forEach()`, `.filter()`, `.find()`, `.findIndex()`, `.some()`, `.every()`, `.reduce()`, `.reduceRight()`, `.sort()`, `.includes()`, `.lastIndexOf()`, `.reverse()`, `.shift()`, `.unshift()`),
`Array.isArray()`,
prototype chain, `class` declarations (`extends`, `super()`, `super.method()`, `static`), `instanceof`,
spread/rest `...` (arrays, strings, function args/params, object spread),
destructuring (array, object, nested, defaults, rest, function params, assignment, for-of/in),
ToPrimitive (`valueOf`/`toString` coercion with hint), wrapper objects (`new Number/String/Boolean`),
`Object.keys/values/entries`, `Object.create/assign/getPrototypeOf`, `hasOwnProperty`, `propertyIsEnumerable`,
non-enumerable properties, `for...in` inherited enumeration,
Promises, setTimeout, queueMicrotask, DOM bindings, event dispatch,
`eval()` (direct/indirect, scope access, strict mode isolation),
`Symbol` (unique identity, description, well-known symbols: iterator/toPrimitive/hasInstance/toStringTag/species/isConcatSpreadable, `Symbol.for()`/`Symbol.keyFor()`, symbol-keyed properties, `Object.getOwnPropertySymbols()`),
`WeakMap` (get/set/has/delete with object keys),
`Proxy` (get/set/has/deleteProperty/apply traps, `Proxy.revocable()`)

**JS262 conformance:** 738 tests, 572 pass, 165 known_fail, 1 skip. **Test262:** 550 vendored, 385 pass (70%) — Phase B complete. See `docs/js_conformance_report.md`.

See `docs/js_feature_matrix.md` for the full current feature status.