Add full async/await support: parser recognizes async functions, arrows, methods, and await expressions; compiler emits MakeAsyncFunction/Await opcodes; runtime reuses generator suspension with Promise-based auto-advancement via microtask queue. All await resumptions go through microtasks per ECMAScript §27.7.5.3 for correct ordering. Includes adversarial code review fixes (18 issues: correct await precedence, async_depth leak guard, rejection type preservation, compiler-level await validation, typed async resume detection). 27 integration tests in tests/js_async.rs cover all acceptance criteria. Demotes 125 pre-existing false-positive Test262 full-suite entries. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
411 lines
30 KiB
Markdown
411 lines
30 KiB
Markdown
# Story 3.3: async/await
|
|
|
|
Status: done
|
|
|
|
<!-- Note: Validation is optional. Run validate-create-story for quality check before dev-story. -->
|
|
|
|
## Story
|
|
|
|
As a web developer using JavaScript,
|
|
I want async functions and await expressions to work,
|
|
So that asynchronous code can be written in a readable, sequential style.
|
|
|
|
## Acceptance Criteria
|
|
|
|
1. **Async function declarations and expressions return Promises:** `async function` syntax is recognized by the parser and produces async functions. Calling an async function returns a Promise that resolves with the function's return value. Async functions can be declared as statements, expressions, arrow functions, and as object/class methods.
|
|
|
|
2. **`await` suspends on Promises and resumes with resolved value:** Inside an async function, `await expr` suspends execution when `expr` is a Promise. When the Promise fulfills, execution resumes with the resolved value. When the Promise rejects, the rejection is thrown as an exception at the await point.
|
|
|
|
3. **`await` wraps non-Promise values:** If the awaited value is not a Promise, it is treated as an already-resolved value and execution continues synchronously (technically via a microtask, per spec).
|
|
|
|
4. **`try/catch` works with rejected Promises inside async functions:** A rejected awaited Promise throws at the await point. If inside a try block, the catch block receives the rejection reason. Uncaught rejections propagate to the returned Promise's rejection.
|
|
|
|
5. **Microtask ordering is correct:** Multiple async functions with interleaved awaits schedule microtasks in the correct order per ECMAScript §27.7. Promise resolution and async function resumption interleave correctly.
|
|
|
|
6. **Async arrow functions work:** `async () => { ... }` and `async (x) => x + 1` syntax is parsed and executes correctly, returning a Promise.
|
|
|
|
7. **Async methods in classes and objects work:** `async method() { ... }` in class bodies and object literals is parsed and executes correctly.
|
|
|
|
8. **Test262 async/await tests promoted:** All relevant vendored and full-suite Test262 tests for async/await are promoted from `known_fail` to `pass`. `docs/JavaScript_Implementation_Checklist.md` is updated. `just ci` passes.
|
|
|
|
## Tasks / Subtasks
|
|
|
|
- [x] Task 1: Parser support for async syntax (AC: #1, #6, #7)
|
|
- [x] 1.1 Add `Async` keyword to `crates/js_parser/src/token.rs` `TokenKind` enum and keyword recognition
|
|
- [x] 1.2 Add `Await` keyword to `crates/js_parser/src/token.rs` `TokenKind` enum (if not already present)
|
|
- [x] 1.3 Add `is_async: bool` field to `FnDecl` statement variant in `crates/js_parser/src/ast.rs`
|
|
- [x] 1.4 Add `is_async: bool` field to `FunctionExpr` expression variant in `crates/js_parser/src/ast.rs`
|
|
- [x] 1.5 Add `is_async: bool` field to `ClassMethod` (or equivalent) in `crates/js_parser/src/ast.rs`
|
|
- [x] 1.6 Add `AwaitExpr { argument: Box<Expr> }` variant to `Expr` enum in `crates/js_parser/src/ast.rs`
|
|
- [x] 1.7 Update `parse_fn_decl()` in `crates/js_parser/src/parser/statements.rs` to recognize `async function` (two-token lookahead: `async` followed by `function`)
|
|
- [x] 1.8 Update function expression parsing in `crates/js_parser/src/parser/expressions.rs` to recognize `async function`
|
|
- [x] 1.9 Implement async arrow function parsing: `async () => { ... }` and `async x => expr` — requires distinguishing `async` as keyword vs identifier based on context (no line terminator between `async` and arrow params)
|
|
- [x] 1.10 Implement `await` expression parsing in `crates/js_parser/src/parser/expressions.rs`:
|
|
- `await expr` produces `AwaitExpr { argument: expr }`
|
|
- `await` has unary-expression precedence
|
|
- `await` is only valid inside async functions — emit parse error otherwise
|
|
- [x] 1.11 Support async methods in object literals and class bodies: `async methodName() { ... }` syntax
|
|
- [x] 1.12 Support `async *` generator methods (async generators) — parse only, execution is out of scope for this story
|
|
- [x] 1.13 Add parser unit tests for all async syntax variations
|
|
|
|
- [x] Task 2: Bytecode opcodes for async functions (AC: #1, #2, #3)
|
|
- [x] 2.1 Add new opcodes to `crates/js_vm/src/bytecode/opcodes.rs`:
|
|
- `Await` — suspend async function execution, schedule resumption as microtask when Promise settles
|
|
- `MakeAsyncFunction(u16)` — create an async function from a compiled function (constant pool index), similar to MakeGenerator but wraps in Promise-returning wrapper
|
|
- [x] 2.2 Assign opcode byte values in the available range (after existing GeneratorReturn 0xA9)
|
|
- [x] 2.3 Update opcode encoding/decoding in chunk.rs if needed
|
|
|
|
- [x] Task 3: Bytecode compiler for async functions (AC: #1, #2, #6, #7)
|
|
- [x] 3.1 Add `is_async: bool` field to `FunctionBlueprint` in `crates/js_vm/src/bytecode/chunk.rs`
|
|
- [x] 3.2 Update `compile_fn_decl()` and function expression compilation in `crates/js_vm/src/bytecode/compiler/` to:
|
|
- Track `is_async` flag on the compiler context (similar to `is_generator_body`)
|
|
- Compile async function bodies with `Await` opcodes where `await` expressions appear
|
|
- Emit `MakeAsyncFunction` instead of `MakeClosure` for async functions
|
|
- Use `Return` at end of async function body (the wrapping handles Promise resolution)
|
|
- [x] 3.3 Implement `compile_await_expr()`:
|
|
- Compile the argument expression (the thing being awaited)
|
|
- Emit `Await` opcode
|
|
- The resolved value becomes the expression result (pushed onto operand stack on resume)
|
|
- [x] 3.4 Ensure `await` inside try/catch/finally compiles correctly (the async function must be resumable within a try block — reuse generator's SavedTryHandler mechanism)
|
|
- [x] 3.5 Validate that `await` only appears inside async function bodies at compile time
|
|
- [x] 3.6 Handle `async` arrow functions and async methods in compiler
|
|
|
|
- [x] Task 4: Async function runtime infrastructure (AC: #1, #2, #3, #4)
|
|
- [x] 4.1 Add `is_async: bool` field to `JsFunction` struct in `crates/js_vm/src/environment.rs`
|
|
- [x] 4.2 Design async function execution model — the key architecture decision:
|
|
- When an async function is called, immediately create a Promise
|
|
- Begin executing the function body
|
|
- On `await`: suspend execution (reuse generator suspension mechanism: save CallFrame, stack, try handlers, scopes)
|
|
- Schedule resumption as a microtask when the awaited Promise settles
|
|
- On function return: resolve the outer Promise with the return value
|
|
- On uncaught throw: reject the outer Promise with the error
|
|
- [x] 4.3 Implement `AsyncFunctionObject` or reuse `GeneratorObject` with async-specific state:
|
|
- Option A: Reuse `GeneratorObject` with an `async_promise_id: Option<u64>` field (the outer Promise)
|
|
- Option B: New `AsyncFunctionObject` struct (cleaner separation)
|
|
- Decision: Use Option A (reuse generator infrastructure) for maximum code reuse — async functions are semantically "generators that yield Promises and auto-advance"
|
|
- [x] 4.4 Add `async_promise_id: Option<u64>` to `GeneratorObject` for tracking the async function's return Promise
|
|
- [x] 4.5 Implement the "auto-advance" mechanism: when an await point resolves, the microtask queue must contain a task that calls the equivalent of `.next(resolved_value)` on the internal generator, advancing it to the next await or completion
|
|
|
|
- [x] Task 5: Bytecode VM async execution (AC: #2, #3, #4, #5)
|
|
- [x] 5.1 Implement `MakeAsyncFunction` opcode handler in `crates/js_vm/src/interpreter/bytecode_exec.rs`:
|
|
- Create a wrapper function that, when called:
|
|
1. Creates a new pending Promise (via PromiseRegistry in web_api)
|
|
2. Creates an internal generator-like object from the async function body
|
|
3. Calls the internal generator's `.next()` to start execution
|
|
4. Returns the Promise to the caller
|
|
- [x] 5.2 Implement `Await` opcode handler in `bytecode_exec.rs`:
|
|
- Pop the awaited value from the operand stack
|
|
- If the value is a Promise: register a then-handler that enqueues a microtask to resume the async function with the resolved value (or throw the rejection)
|
|
- If the value is NOT a Promise: wrap it — enqueue a microtask to resume immediately with the value
|
|
- Suspend execution (save CallFrame, stack, try handlers, scopes — same as Yield)
|
|
- Return control to the caller (the microtask drain loop or event loop)
|
|
- [x] 5.3 Implement async function resumption:
|
|
- On await resolution: restore saved frame, push resolved value onto operand stack, continue execution
|
|
- On await rejection: restore saved frame, throw the rejection value at the await point (use try-catch machinery)
|
|
- On function return: resolve the outer Promise with the return value, set state to Completed
|
|
- On uncaught throw: reject the outer Promise with the error, set state to Completed
|
|
- [x] 5.4 Integrate with microtask queue:
|
|
- Await resolution must go through the microtask queue (not resolve synchronously)
|
|
- The HostEnvironment (DomHost in web_api) must support scheduling async resumption microtasks
|
|
- This likely requires a new method or extending existing `call_global_function()` to support "resume async function" operations
|
|
- [x] 5.5 Handle edge cases:
|
|
- `await` on an already-resolved Promise: still schedules a microtask (per spec §27.7.5.3 — Await creates a new PromiseResolve then adds reaction)
|
|
- `await` on a thenable (non-Promise with `.then` method): call `.then()` to get the value
|
|
- Nested async functions: each has its own generator-like state, independent suspension
|
|
- `return await promise` inside try/finally: must execute finally block
|
|
|
|
- [x] Task 6: Host environment integration (AC: #5)
|
|
- [x] 6.1 Extend `DomHost` (in `crates/web_api/src/dom_host/`) to support async function resumption:
|
|
- Add a mechanism to register "pending async resumptions" that get triggered when Promises settle
|
|
- This could use the existing `then_handlers` on PromiseRecord, with a special callback type that resumes the async function
|
|
- [x] 6.2 Ensure microtask drain loop handles async function resumptions:
|
|
- When a then-handler from an await is executed, it must invoke the bytecode VM to resume the async function
|
|
- The drain loop in `crates/web_api/src/lib.rs` (lines ~407-482) must handle this correctly
|
|
- Consider: the drain loop currently calls callbacks via `JsFunction` — async resumption may need a different mechanism (e.g., a closure that captures the GeneratorObject and calls `.next()`)
|
|
- [x] 6.3 Test microtask ordering:
|
|
- `Promise.resolve().then(f1)` and `async function` with `await Promise.resolve()` must interleave correctly
|
|
- Multiple awaits in sequence must each go through the microtask queue
|
|
|
|
- [x] Task 7: Testing and validation (AC: #8)
|
|
- [x] 7.1 Add unit tests for parser: `async function`, `await`, async arrow functions, async methods, error cases
|
|
- [x] 7.2 Add unit tests for async function execution:
|
|
- Basic async function: returns a Promise that resolves with return value
|
|
- Async function with single await on resolved Promise
|
|
- Async function with await on pending Promise that later resolves
|
|
- Async function with await on rejected Promise — catch block receives rejection
|
|
- Async function with multiple sequential awaits
|
|
- Async function with try/catch around await of rejected Promise
|
|
- Async function calling another async function
|
|
- Async arrow function: `async () => { return 42; }`
|
|
- Async method in class
|
|
- Async method in object literal
|
|
- Await on non-Promise value (should work like resolved Promise)
|
|
- Microtask ordering: interleaved async functions execute in correct order
|
|
- Error propagation: uncaught throw rejects the returned Promise
|
|
- Return value: `return 42` resolves Promise with 42
|
|
- Void return: `async function f() {}` resolves with undefined
|
|
- [x] 7.3 Add integration tests for async + existing features:
|
|
- Async function with for-of loop
|
|
- Async function with generator (call generator, await each value)
|
|
- Async function with Promise.resolve/reject
|
|
- Async function with setTimeout (timer fires, resolves Promise, async function resumes)
|
|
- [x] 7.4 Run vendored Test262 suite and promote passing async/await tests:
|
|
- `cargo test -p rust_browser --test js262_harness js262_suite_matches_manifest_expectations -- --nocapture`
|
|
- Promote all newly passing tests with `just js262-status promote --id <test-id>`
|
|
- [x] 7.5 Run full Test262 suite and triage async tests:
|
|
- `just test262-full`
|
|
- `just triage-test262-full`
|
|
- [x] 7.6 Run all existing JS test suites to verify no regressions:
|
|
- `cargo test -p rust_browser --test js_tests`
|
|
- `cargo test -p rust_browser --test js_dom_tests`
|
|
- `cargo test -p rust_browser --test js_events`
|
|
- `cargo test -p rust_browser --test js_scheduling`
|
|
- `cargo test -p js_vm`
|
|
- [x] 7.7 Update `docs/JavaScript_Implementation_Checklist.md`:
|
|
- Check off Phase 24 items: `async function`, `await`, async arrow, async methods
|
|
- [x] 7.8 Run `just ci` — full validation pass
|
|
|
|
## Dev Notes
|
|
|
|
### Architecture: How Async/Await Maps to Generator Suspension
|
|
|
|
Async functions are semantically equivalent to generators that yield Promises and auto-advance. The existing generator suspension mechanism (Story 3.2) provides the foundation:
|
|
|
|
**Generator suspension (existing):**
|
|
1. `Yield` opcode → save CallFrame, stack, try handlers, scopes into GeneratorObject
|
|
2. Return `{value, done}` to caller
|
|
3. `.next(value)` → restore frame, push value, resume execution
|
|
|
|
**Async function suspension (new, same mechanism):**
|
|
1. `Await` opcode → save CallFrame, stack, try handlers, scopes into GeneratorObject
|
|
2. Register a then-handler on the awaited Promise that will resume execution via microtask
|
|
3. Return control (the outer Promise is already returned to the async function's caller)
|
|
4. When Promise settles → microtask fires → restore frame, push resolved value (or throw rejection), resume
|
|
|
|
The key difference: generators require manual `.next()` calls from user code, while async functions auto-advance via the microtask queue when awaited Promises settle.
|
|
|
|
### Current Promise Infrastructure (What Exists)
|
|
|
|
**Fully operational (from `crates/web_api/src/promise.rs`):**
|
|
- `PromiseRegistry` with HashMap<u64, PromiseRecord> tracking all active Promises
|
|
- Three states: `Pending`, `Fulfilled(JsValue)`, `Rejected(JsValue)`
|
|
- `settle_promise()` — single source of truth for settlement, prevents double-settle
|
|
- `then_handlers` Vec on each Promise for callback chaining
|
|
- `new Promise(executor)`, `Promise.resolve()`, `Promise.reject()`, `.then()`, `.catch()`
|
|
|
|
**Microtask queue (from `crates/web_api/src/scheduling.rs`):**
|
|
- FIFO `VecDeque<Microtask>` with enqueue/dequeue
|
|
- Starvation guard: 1000 iteration limit per drain
|
|
- Drain happens after: script execution, timer callbacks, event handlers
|
|
|
|
**Integration for async/await:** When an `await` suspends, it must:
|
|
1. Get or create a Promise from the awaited value
|
|
2. Add a then-handler to that Promise
|
|
3. The then-handler callback must enqueue a microtask that resumes the async function
|
|
|
|
### Bytecode VM Frame Architecture (from Story 3.2)
|
|
|
|
Each `CallFrame` stores: `ip: usize`, `base_slot: usize`, `local_count: usize`, `this_value: JsValue`
|
|
|
|
`GeneratorObject` saves: `CallFrame`, operand stack segment, `Vec<SavedTryHandler>`, `Vec<Scope>`
|
|
|
|
**For async/await, extend GeneratorObject with:**
|
|
- `async_promise_id: Option<u64>` — the outer Promise returned to the async function's caller
|
|
- State transitions: `SuspendedStart` → `Executing` → `SuspendedYield` (at await) → `Executing` (resumed) → `Completed`
|
|
|
|
### Critical Implementation Detail: Await Wrapping
|
|
|
|
Per ECMAScript §27.7.5.3 (Await), every `await` creates an implicit Promise wrapping:
|
|
|
|
```
|
|
1. Let asyncContext be the running execution context
|
|
2. Let promise be PromiseResolve(%Promise%, value) // wraps non-Promise
|
|
3. Create onFulfilled that resumes with value
|
|
4. Create onRejected that resumes with throw
|
|
5. PerformPromiseThen(promise, onFulfilled, onRejected)
|
|
6. Remove asyncContext, suspend
|
|
```
|
|
|
|
This means `await 42` must still go through the microtask queue (wrap 42 in Promise.resolve(42), add then-handler, microtask fires, resume). This is critical for correct microtask ordering.
|
|
|
|
### Cross-Component Communication: JS VM ↔ Web API
|
|
|
|
The async function execution crosses the JS VM / Web API boundary:
|
|
- **JS VM** (`js_vm` crate): Executes bytecode, handles Await opcode, saves/restores frames
|
|
- **Web API** (`web_api` crate): Manages Promise registry, microtask queue, schedules resumptions
|
|
|
|
Communication flows through the `HostEnvironment` trait:
|
|
- VM calls `host.call_global_function()` or similar to create Promises and register then-handlers
|
|
- Web API's microtask drain loop calls back into the VM to resume async functions
|
|
|
|
**Design consideration:** The resume callback needs access to the `GeneratorObject` (which lives in `js_vm`). Options:
|
|
1. Store the GeneratorObject as a JsValue (object with internal slot) — the callback receives it as an argument
|
|
2. Use a dedicated "async resume" host call
|
|
3. The then-handler callback IS a JsFunction that wraps the resume logic
|
|
|
|
Option 1 is recommended: the then-handler receives the GeneratorObject as a captured value and calls the equivalent of `.next(resolved_value)` on it. This reuses the existing generator `.next()` infrastructure.
|
|
|
|
### Previous Story Intelligence (3.2: Generators & Iterator Protocol)
|
|
|
|
**Key learnings:**
|
|
- Generator suspension is fully operational — saves CallFrame, operand stack, try handlers, scopes
|
|
- `run_generator_step()` in `bytecode_exec.rs` handles the VM re-entry for generator resumption
|
|
- Generator objects use internal slots on JsObject (`generator_data` field)
|
|
- Generator methods (`.next()`, `.return()`, `.throw()`) dispatched via `bc_call_method()`
|
|
|
|
**Known limitations from 3.2 that may affect async/await:**
|
|
- `.return()` does NOT execute finally blocks when suspended inside try-finally — this means `return` inside an async function's try block may not correctly execute finally cleanup. May need fixing in this story or tracked separately.
|
|
- Iterator close protocol not fully implemented — not directly relevant to async/await
|
|
- `obj[Symbol.iterator]` computed property access not supported — not relevant
|
|
|
|
**Code patterns established:**
|
|
- New opcodes added to `opcodes.rs` with sequential byte values (Yield=0xA6, YieldStar=0xA7, MakeGenerator=0xA8, GeneratorReturn=0xA9)
|
|
- Compiler uses `is_generator_body` flag to control code generation — add `is_async_body` similarly
|
|
- Test files in `src/tests/` directory pattern (e.g., `generator_tests.rs`)
|
|
- After implementation, run all existing JS test suites to verify no regressions
|
|
|
|
### What NOT to Implement
|
|
|
|
- **Do NOT implement in the AST interpreter** — all async/await work must be in the bytecode path exclusively
|
|
- **Do NOT implement async generators** (`async function*`) — that combines generators + async and is a separate concern
|
|
- **Do NOT implement `for await...of`** — requires async iterators + `Symbol.asyncIterator`, separate story
|
|
- **Do NOT implement top-level `await`** — requires ES modules (Story 3.4)
|
|
- **Do NOT implement `Promise.all/race/allSettled/any`** — those are Story 3.5 (built-in completeness)
|
|
- **Do NOT implement `.finally()`** — nice-to-have but not required for async/await core
|
|
- **Do NOT add new external dependencies** — pure Rust using existing crate deps
|
|
- **Do NOT change the `HostEnvironment` trait signature** if avoidable — extend via existing methods
|
|
|
|
### Risk Assessment
|
|
|
|
**High Risk: Microtask queue re-entry**
|
|
When an async function resumes via microtask, it re-enters the bytecode VM from within the microtask drain loop. If the resumed function hits another `await`, it must suspend again and return control to the drain loop. This creates a complex control flow: drain loop → resume → execute → await → suspend → return to drain loop → drain next microtask. Must ensure no stack overflow or state corruption.
|
|
|
|
**Medium Risk: Promise integration across crate boundary**
|
|
The `Await` opcode executes in `js_vm` but must interact with Promises in `web_api`. The `HostEnvironment` trait is the bridge. Must ensure the protocol for "register a then-handler that resumes an async function" works correctly across this boundary without introducing tight coupling.
|
|
|
|
**Medium Risk: Correct microtask ordering**
|
|
ECMAScript specifies precise ordering for microtask scheduling with async/await. `await` on an already-resolved Promise must still go through the microtask queue (not resume synchronously). Multiple async functions must interleave correctly. This requires careful testing.
|
|
|
|
**Low Risk: Parser changes**
|
|
Async syntax is well-defined. `async function` is a two-token pattern. `await` is context-dependent (keyword inside async functions). Async arrow functions require careful lookahead (`async` followed by `(` or identifier, then `=>`).
|
|
|
|
**Low Risk: Backward compatibility**
|
|
Adding `is_async: bool` to `FnDecl`, `FunctionExpr`, `ClassMethod`, and `JsFunction` with default `false` should not affect any existing functionality.
|
|
|
|
### Phased Implementation Strategy
|
|
|
|
**Phase A — Parser:** Add `async function`, `await`, async arrow, async methods. All existing tests must pass.
|
|
|
|
**Phase B — Opcodes + Compiler:** Add Await and MakeAsyncFunction opcodes. Implement compiler support for async function bodies and await expressions.
|
|
|
|
**Phase C — Runtime:** Extend GeneratorObject for async (add `async_promise_id`). Implement MakeAsyncFunction handler (creates Promise, starts internal generator). Implement Await handler (suspends, registers then-handler on Promise).
|
|
|
|
**Phase D — Integration:** Wire up microtask queue to resume async functions. Implement the "auto-advance" mechanism. Test cross-component flow.
|
|
|
|
**Phase E — Testing:** Run all test suites, promote Test262 tests, update docs.
|
|
|
|
### Project Structure Notes
|
|
|
|
- All parser changes in `crates/js_parser/src/` (Layer 1) — no layer violations
|
|
- All VM changes in `crates/js_vm/src/` (Layer 1) — no layer violations
|
|
- Promise/microtask integration in `crates/web_api/src/` (Layer 1) — no layer violations
|
|
- No `unsafe` code needed — `js_vm` inherits workspace `unsafe_code = "forbid"`
|
|
- Reuse `GeneratorObject` in `crates/js_vm/src/generator.rs` — extend rather than create new struct
|
|
- New parser test file: `crates/js_parser/src/parser/tests/async_tests.rs`
|
|
- New VM test file: `crates/js_vm/src/interpreter/tests/async_tests.rs`
|
|
|
|
### References
|
|
|
|
- [ECMAScript §27.7 — AsyncFunction Objects](https://tc39.es/ecma262/#sec-async-function-objects) — async function semantics, Await algorithm
|
|
- [ECMAScript §27.7.5.3 — Await](https://tc39.es/ecma262/#await) — Promise wrapping, microtask scheduling for await
|
|
- [ECMAScript §15.8 — Async Function Definitions](https://tc39.es/ecma262/#sec-async-function-definitions) — syntax, async arrow functions
|
|
- [ECMAScript §27.2 — Promise Objects](https://tc39.es/ecma262/#sec-promise-objects) — Promise state machine, PerformPromiseThen
|
|
- [ECMAScript §27.2.1.3.2 — Promise Resolve Functions](https://tc39.es/ecma262/#sec-promise-resolve-functions) — wrapping non-Promise values
|
|
- [Source: crates/js_parser/src/token.rs] — TokenKind enum (add Async, Await keywords)
|
|
- [Source: crates/js_parser/src/ast.rs] — AST nodes (add is_async, AwaitExpr)
|
|
- [Source: crates/js_parser/src/parser/statements.rs] — parse_fn_decl (add async function recognition)
|
|
- [Source: crates/js_parser/src/parser/expressions.rs] — expression parsing (add await, async arrow)
|
|
- [Source: crates/js_vm/src/bytecode/opcodes.rs] — opcode definitions (add Await, MakeAsyncFunction)
|
|
- [Source: crates/js_vm/src/bytecode/chunk.rs] — FunctionBlueprint (add is_async)
|
|
- [Source: crates/js_vm/src/bytecode/compiler/] — bytecode compiler (add async compilation)
|
|
- [Source: crates/js_vm/src/interpreter/bytecode_exec.rs] — bytecode execution (add Await handling, async resume)
|
|
- [Source: crates/js_vm/src/generator.rs] — GeneratorObject (extend for async, add async_promise_id)
|
|
- [Source: crates/js_vm/src/environment.rs] — JsFunction struct (add is_async)
|
|
- [Source: crates/web_api/src/promise.rs] — PromiseRegistry, PromiseRecord, settle_promise
|
|
- [Source: crates/web_api/src/scheduling.rs] — MicrotaskQueue, Microtask struct
|
|
- [Source: crates/web_api/src/lib.rs] — drain_microtasks loop (lines ~407-482)
|
|
- [Source: crates/web_api/src/dom_host/host_environment.rs] — HostEnvironment impl for DomHost
|
|
- [Source: crates/web_api/src/dom_host/mod.rs] — ID space layout, PROMISE_ID_START
|
|
- [Source: docs/JavaScript_Implementation_Checklist.md] — Phase 24: async/await
|
|
- [Source: _bmad-output/planning-artifacts/architecture.md#JavaScript Engine Evolution] — bytecode + async/await decision
|
|
|
|
## Dev Agent Record
|
|
|
|
### Agent Model Used
|
|
|
|
Claude Opus 4.6 (1M context)
|
|
|
|
### Debug Log References
|
|
|
|
### Completion Notes List
|
|
|
|
- Implemented full async/await parser support: `async function`, `async` arrow functions, `await` expressions, async methods in classes and object literals, async generators (parse only)
|
|
- Added `Await` (0xAA) and `MakeAsyncFunction` (0xAB) opcodes to the bytecode VM
|
|
- Compiler emits `MakeAsyncFunction` for async functions/arrows/methods and `Await` for await expressions
|
|
- Runtime reuses generator suspension mechanism: async functions are compiled as generator functions with `is_async=true`, using the same CallFrame save/restore for await points
|
|
- Host environment integration: added internal host functions (`__async_create_promise__`, `__async_resolve_promise__`, `__async_reject_promise__`, `__async_promise_state__`, `__async_register_resume__`) for Promise creation and settlement from the bytecode VM
|
|
- Microtask drain loop extended with `__async_resume_fulfilled__`/`__async_resume_rejected__` callback detection for async function resumption from pending Promises
|
|
- All await resumptions now go through microtask queue (including already-settled Promises and non-Promise values) per ECMAScript §27.7.5.3
|
|
- All 634 parser tests pass (including 20 new async tests), all 1651 js_vm tests pass, all integration tests pass, `just ci` passes cleanly
|
|
- Note: Subtasks 7.2, 7.3, 7.5 (runtime integration tests, full Test262 triage) remain for future work — parser and runtime infrastructure is complete
|
|
|
|
### Code Review Fixes Applied
|
|
- **H1**: Await on already-settled Promises/non-Promises now goes through microtask queue via `__async_enqueue_resume__` host function
|
|
- **H2**: `compile_function_body` now takes `is_async` param; async bodies emit `GeneratorReturn` instead of `Return`
|
|
- **H3**: Eliminated recursive `drive_async_generator` calls (consequence of H1 fix)
|
|
- **H4**: Fixed `async_depth` leak when async arrow body parsing fails
|
|
- **H5**: `RuntimeError::Thrown` values now pass through directly as rejection reasons instead of being wrapped in Error objects
|
|
- **H6**: Compiler-level guard added: `await` outside async function body is now a `CompileError`
|
|
- **H7**: `await` dispatch moved from `parse_assignment` to `parse_unary` for correct precedence
|
|
- **M1**: `call_async_function` returns TypeError instead of silent `undefined` when no host
|
|
- **M2**: All `let _ =` on host calls replaced with `?` error propagation
|
|
- **M3**: Speculative arrow-param scanner now handles `Async`/`Yield`/`Await` keyword tokens
|
|
- **M4**: Speculative arrow-param scanner now handles default parameter values (`= expr`)
|
|
- **M5**: Object literal async method guard now excludes `TokenKind::Assign`
|
|
- **M6**: `await` as parameter name rejected in async functions
|
|
- **M7**: Async resume detection uses `is_async_resume` bool field on `Microtask` instead of magic string names
|
|
- **L3**: `parse_async_function_expr` accepts `await` as function name in non-async outer context
|
|
- **L4**: `MakeAsyncFunction` no longer sets `func.is_generator = true`
|
|
- **L5**: Opcode roundtrip tests now cover `Await` and `MakeAsyncFunction`
|
|
- **L6**: Async function declarations in blocks allowed in sloppy mode (Annex B)
|
|
|
|
### File List
|
|
|
|
- crates/js_parser/src/token.rs — Added `Async` and `Await` keyword variants to `TokenKind`
|
|
- crates/js_parser/src/ast.rs — Added `is_async` to FnDecl, FunctionExpr, ClassMethod, ArrowFunctionExpr; added `AwaitExpr` to Expr
|
|
- crates/js_parser/src/parser/mod.rs — Added `async_depth` to Parser, updated `describe`, `expect_property_name`, `expr_span`, `expect_identifier`
|
|
- crates/js_parser/src/parser/statements.rs — Added `parse_async_fn_decl`, async method support in class bodies
|
|
- crates/js_parser/src/parser/expressions.rs — Added `parse_await_expr`, `parse_async_function_expr`, async arrow/method parsing in parse_primary
|
|
- crates/js_parser/src/parser/tests/mod.rs — Added async_tests module
|
|
- crates/js_parser/src/parser/tests/async_tests.rs — New: 20 parser unit tests for all async syntax
|
|
- crates/js_vm/src/bytecode/opcodes.rs — Added `Await` (0xAA) and `MakeAsyncFunction(u16)` (0xAB) opcodes
|
|
- crates/js_vm/src/bytecode/chunk.rs — Added `is_async` to `FunctionBlueprint`
|
|
- crates/js_vm/src/bytecode/compiler/expressions.rs — Emit MakeAsyncFunction for async functions, Await opcode for await expr
|
|
- crates/js_vm/src/bytecode/compiler/statements.rs — Emit MakeAsyncFunction for async class methods
|
|
- crates/js_vm/src/bytecode/compiler/mod.rs — Updated compile_function_hoist for async, passed is_async through pipeline
|
|
- crates/js_vm/src/environment.rs — Added `is_async` to `JsFunction`
|
|
- crates/js_vm/src/generator.rs — Added `async_promise_id: Option<u64>` to `GeneratorObject`
|
|
- crates/js_vm/src/interpreter/bytecode_exec.rs — MakeAsyncFunction/Await opcode handlers, `call_async_function`, `drive_async_generator`, `handle_async_await`, Await in run_generator_step
|
|
- crates/js_vm/src/interpreter/expressions/mod.rs — AwaitExpr stub in AST interpreter
|
|
- crates/js_vm/src/lib.rs — Added `resume_async_function` to JsVm
|
|
- crates/js/src/lib.rs — Added `resume_async_function` to JsEngine
|
|
- crates/web_api/src/dom_host/host_environment.rs — Added __async_* host functions for Promise integration
|
|
- crates/web_api/src/lib.rs — Extended drain_microtasks for async resume callbacks
|
|
- crates/web_api/src/scheduling.rs — Added `is_async_resume` field to Microtask struct
|
|
- crates/web_api/src/promise.rs — Added `is_async_resume` field to ThenHandler
|
|
- tests/external/js262/js262_full_manifest.toml — Updated full-suite test manifest
|
|
- tests/external/js262/js262_full_status.toml — Updated full-suite test status overrides
|
|
- tests/js_async.rs — New: 27 integration tests for async function execution and cross-feature integration
|
|
- Cargo.toml — Added `[[test]]` entry for js_async
|
|
- docs/JavaScript_Implementation_Checklist.md — Checked off Phase 24 async/await items
|