104 lines
3.3 KiB
Markdown
104 lines
3.3 KiB
Markdown
Manage WPT (Web Platform Tests) for the browser engine.
|
|
|
|
Command: $ARGUMENTS
|
|
|
|
## Available commands
|
|
|
|
Parse the command from $ARGUMENTS and execute the appropriate action below. If the command is empty or "help", show the available commands summary.
|
|
|
|
### `run` — Run the WPT test suite
|
|
|
|
1. Run the full WPT suite:
|
|
```
|
|
cargo test -p rust_browser --test wpt_harness wpt_suite_matches_manifest_expectations -- --nocapture 2>&1
|
|
```
|
|
2. Parse the output to extract:
|
|
- Total tests, pass count, known_fail count, pixel_promoted count
|
|
- Any failures (lines after "WPT harness failures")
|
|
3. If there are **pixel_promoted > 0** tests (known_fail cases that now pass), report them and suggest promoting with `/wpt promote`
|
|
4. If there are **failures**, report each one and categorize:
|
|
- **Unexpected failure**: a `pass` case that failed — suggest demoting with `/wpt demote`
|
|
- **Unexpected pass**: a `known_fail` case that passed — suggest promoting with `/wpt promote`
|
|
5. Summarize results in a table
|
|
|
|
### `promote <id> [<id>...]` — Promote known_fail tests to pass
|
|
|
|
Promote tests that are now passing. For each ID:
|
|
|
|
1. First verify the test actually passes by running it:
|
|
```
|
|
cargo test -p rust_browser --test wpt_harness wpt_suite_matches_manifest_expectations -- --nocapture 2>&1
|
|
```
|
|
(You can skip this if you just ran `/wpt run` and confirmed the test passes)
|
|
|
|
2. For `single` mode tests, regenerate expected outputs:
|
|
```
|
|
just regen-wpt
|
|
```
|
|
|
|
3. Update the manifest:
|
|
```
|
|
just wpt-status promote --id <id1> --id <id2> ...
|
|
```
|
|
|
|
4. Run the suite again to confirm everything passes:
|
|
```
|
|
cargo test -p rust_browser --test wpt_harness wpt_suite_matches_manifest_expectations -- --nocapture 2>&1
|
|
```
|
|
|
|
5. Report the results
|
|
|
|
### `demote <id> [<id>...] <reason>` — Demote pass tests to known_fail
|
|
|
|
Demote tests that are now failing. Parse the arguments: all tokens that look like test IDs (contain "wpt-") are IDs, and the remaining text is the reason. If no reason is provided, ask the user for one.
|
|
|
|
1. Update the manifest:
|
|
```
|
|
just wpt-status demote --id <id1> --id <id2> ... --reason "<reason>"
|
|
```
|
|
|
|
2. Run the suite to confirm the manifest is now consistent:
|
|
```
|
|
cargo test -p rust_browser --test wpt_harness wpt_suite_matches_manifest_expectations -- --nocapture 2>&1
|
|
```
|
|
|
|
3. Report the results
|
|
|
|
### `status [<id>...]` — Show status of specific tests
|
|
|
|
If IDs are provided:
|
|
```
|
|
just wpt-status show --id <id1> --id <id2> ...
|
|
```
|
|
|
|
If no IDs are provided, show a summary:
|
|
```
|
|
just wpt-status list --status pass 2>&1
|
|
just wpt-status list --status known_fail 2>&1
|
|
just wpt-status list --status skip 2>&1
|
|
```
|
|
|
|
Report a summary table with counts for each status.
|
|
|
|
### `list <pass|known_fail|skip>` — List tests by status
|
|
|
|
```
|
|
just wpt-status list --status <status>
|
|
```
|
|
|
|
### `regen` — Regenerate expected outputs for all single-mode pass tests
|
|
|
|
```
|
|
just regen-wpt
|
|
```
|
|
|
|
Report which files were regenerated.
|
|
|
|
## Notes
|
|
|
|
- The WPT manifest is at `tests/external/wpt/wpt_manifest.toml`
|
|
- Test IDs typically look like `wpt-css-*` or `wpt-reftest-*` or `wpt-known-fail-*`
|
|
- The `just wpt-status` command wraps `python3 scripts/wpt_status.py`
|
|
- Tests run in parallel using `std::thread::scope`
|
|
- `pixel_promoted` in output means a known_fail reftest passed pixel comparison — these are candidates for promotion
|