rust_browser/docs/external_conformance_wpt.md

External Conformance: WPT Harness (HTML/CSS/Layout)

This project includes a curated Web Platform Tests (WPT)-style subset under tests/external/wpt.

Test source

Canonical upstream source:

• WPT repository: https://github.com/web-platform-tests/wpt
• WPT project site: https://web-platform-tests.org/

The files in tests/external/wpt/fixtures are local, pinned copies adapted from that upstream corpus for this engine's current no-JS scope. When adding tests, record the originating WPT test path (or URL) in the PR description so updates remain traceable.

Scope

Current scope intentionally excludes JavaScript-dependent tests. The harness focuses on deterministic HTML/CSS/layout behavior using the existing structural oracles:

• layout tree dump
• display list dump

Files

• Manifest: tests/external/wpt/wpt_manifest.toml
• Fixtures: tests/external/wpt/fixtures
• Expected outputs: tests/external/wpt/expected
• Harness test target: tests/wpt_harness.rs

Status policy

Each case is marked with one of:

• pass: must pass in CI
• known_fail: expected to fail for documented reasons
• skip: not executed in current phase (for example, JS-dependent)

known_fail and skip entries require non-empty reasons.

Running

Run the suite:

cargo test -p rust_browser --test wpt_harness

Failure artifacts are written to target/wpt_artifacts.

Regenerate expected outputs for pass + single cases:

just regen-wpt

Bulk-importing upstream WPT reftests

The import-wpt command sparse-clones the upstream WPT repo and imports qualifying CSS reftests as known_fail:

# See what's available without importing
just import-wpt --scan-only

# Import all qualifying reftests (idempotent)
just import-wpt

# Import from specific CSS modules only
just import-wpt --modules css-box,css-text

# Limit the number of tests imported
just import-wpt --max-tests 50

# Preview what would be imported
just import-wpt --dry-run

A test qualifies for import if it has <link rel="match">, no <script>, no external stylesheets, no <img>/<video>/<canvas>/<iframe>, and no @import/url() in CSS (except data:). The reference file must also pass the same checks.

The upstream repo is cloned to tests/external/wpt/upstream/ (gitignored).

Investigating and promoting known_fail tests

1. Pick a test

Browse the manifest for known_fail entries. The reason field records the upstream WPT path and detected CSS features:

[[case]]
id = "wpt-css-css-box-margin-trim-block-container-block-001"
status = "known_fail"
reason = "upstream: css/css-box/margin-trim/block-container-block-001.html"
flags = ["css-box"]

2. Read the test and reference fixtures

Open them side by side to understand what the test expects:

tests/external/wpt/fixtures/<id>-test.html
tests/external/wpt/fixtures/<id>-ref.html

The test passes when both files produce identical layout tree and display list output.

3. Check failure artifacts

Run the suite and inspect the diff:

cargo test -p rust_browser --test wpt_harness wpt_suite -- --nocapture

The harness writes artifacts to target/wpt_artifacts/:

• <id>.actual.layout.txt — layout tree for the test file
• <id>.actual.dl.txt — display list for the test file

Compare these against the reference file's output to identify the rendering gap.

4. Fix the engine

Fix the relevant crate (css, layout, style, etc.) and re-run:

cargo test -p rust_browser --test wpt_harness wpt_suite

If the test now passes, CI will fail with:

case '...' is marked known_fail but now passes; promote it to pass in manifest

5. Promote in the manifest

In wpt_manifest.toml, change the entry:

# Before
status = "known_fail"
reason = "upstream: css/css-box/margin-trim/block-container-block-001.html"

# After
status = "pass"
# (remove the reason line)

6. Verify

just ci

Adding a case manually

1. Add fixture HTML under tests/external/wpt/fixtures.
2. Add a [[case]] entry in tests/external/wpt/wpt_manifest.toml.
3. If mode = "single", regenerate expected outputs.
4. If mode = "reftest", provide reference fixture and ensure deterministic equivalence.
5. Use known_fail only with a concrete reason.