Repository navigation
Submitting WPT results to wpt.fyi for comparison #46443
Description
Activity
This is pretty interesting. I wonder what we are missing...
Just the extract and transform addition to
make test-wptto get results in a format the wpt.fyi API expects.I'm happy to take this effort on if there's interest.
Reacted by Yagiz Nizipli- figure out the expected result format we have to emulate, since we're not using the official WPT runner we need to write this part ourselves. We'll have to update our runner to spit out something that can be parsed and transformed into the expected format.
- the actual Github CI job
- request credentials so that the Node.js project can be added as a runner that can submit to wpt.fyi's API.
- start publishing daily
I'm currently at step 1. I found a suggested JSON schema for the report format. I plan to look at what Deno does in this regard.
It would be immensely helpful if we had a report at hand.
, I can reach out to the Deno folk for getting one that they submit.Generated one from Deno https://github.com/proxy/gist.github.com/panva/f00124fb416dd8572b5008c4ab6e9dcc- Reacted by Yagiz Nizipli and Trivikram Kamat
I've merged the job. Starting tonight, latest tomorrow night, there will be nightly results uploaded to staging.wpt.fyi
I will request production credentials next week.
Hi Filip,
The production service now has the same username/password for your project. We haven't deployed the code there just yet, but it should be OK for you to start submitting data.
Thanks,
Panos@targos can you please reconfigure the WPT_FYI_ENDPOINT variable on the
nodejs/noderepository tohttps://wpt.fyi/api/results/upload?We should leave the
nodejs/node-auto-testconfiguration as-is, I'll be extending our report with fetch-related WPTs from undici and will use that to test.Variable updated.
Thank you. This is now completed. I will followup with a PR to nodejs/undici and nodejs/node to include Fetch API related WPTs to the submitted report.
🎉 Node.js is now selectable in the wpt.fyi production Dashboard as a "Product".
We are submitting nightly and current builds daily already.
In progress is
- adding lts/* ([v18.x backport] test: add WPTRunner support for variants and generating WPT reports #46767)
- adding lts/-1 ([v16.x backport] test: add WPTRunner support for variants and generating WPT reports #46768)
- adding fetch related WPTs (tools: add undici WPTs to daily WPT Report #46763)
Is there interest in adding Node.js to the list of products that wpt.fyi supports displaying results and comparisons for?
If I understood the task right, it would require a daily CI job which checks out the WPT tests using a branch name that the wpt repo pushes to on a daily schedule, called
epochs/daily, executing our WPT runner, collecting its results, transforming and submitting them to the wpt.fyi together with metadata about the used Node.js version.This would be done in a matrix for all maintained release lines which wpt.fyi calls stable channels, plus one for the experimental channel which would be our
mainbranch checkout. So the job would e.g. checkout node's v18.x branch HEAD~1, updated the checked out WPTs toepochs/daily, ran the WPTs, transformed the results and submitted them.To submit the results we'd request an identifier/username to be created in the wpt.fyi API, an API key for that username would be given to us to be stored as a Github Actions Repository Secret.
We would still only be running the WPT suites we have an existing checkout for.
Example how Deno has their results submitted