[rascani]

Projects that need the component before the next release can now pin a
dev version from the registry instead of generating the component from
an ExecuTorch checkout. A manual run of publish-esp-idf.yml on the
nightly branch with version_suffix -dev publishes X.Y.Z-dev.YYYYMMDD of
that branch's commit, through the same packing, testing, review and
registry checks as a release. The generator's new --version-suffix -dev
stamps HEAD's commit date and checks the checkout as it does for a
release; --allow-dev-stamp stays the unchecked stamp for local builds.

A dev version's kernel bundles need executorch from the component's
exact commit. ExecuTorch's nightly wheels are built from the nightly
branch's commits, so the plan refuses -dev on any other ref, and before
the build the new wheel mode of test_espressif_idf_nightly.sh checks
that the nightly wheels of the version's date, for each of the four
platforms they are built for, record that commit. It also refuses a
version dated today, since a second nightly the same day, which has
happened, replaces that day's wheels; the runbook has the workflow run
between 00:00 UTC and the next nightly. Pull requests that touch the
workflow or that script rehearse the check on the latest finished days'
nightlies.

ExecuTorch's wheels declare no torch, so the component's VERSION now
records the torch requirement install_requirements.py builds the wheels
against, and the index that keeps it, and the new torch mode checks
before upload that it resolves there on each platform. The component's
README gives the pip commands for releases and dev versions. The
nightly index keeps about two months of wheels, so a project moves to a
newer dev version, or to the release, before its pin ages out; firmware
builds from a committed kernel bundle are unaffected.

Dev versions are semver pre-releases, which a version range picks up
only if it opts in, so they go to production without reaching projects
on releases. They read 1.6.0-dev.YYYYMMDD, as the nightly wheels do,
until main's version.txt moves past 1.6.0, and sort below 1.6.0
meanwhile; exact pins are unaffected. Dev runs queue in the nightly
branch's concurrency group, so one cannot replace a pending release run.

Before the first dev upload, a repo admin adds nightly to the deployment
branches of the esp-component-registry environment. The workflow accepts
-dev once this change reaches the nightly branch.

Authored with Claude Code.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24748

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 1 Unrelated Failure, 12 Unclassified Failures

As of commit 19dd234 with merge base 60a8b5f ():

NEW FAILURES - The following jobs have failed:

• trunk / test-models-linux-aarch64 (emformer_transcribe, xnnpack-quantization-delegation, mt-l-arm64g4-16-62) / linux-job (gh)
  executorch
• trunk / test-qnn-model (fp32, mb) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, efficientnet) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, mobilevit_v2) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• trunk / test-qnn-optimum-model (fp32, roberta) / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• trunk / test-torchao-huggingface-checkpoints (lfm2_5_1_2b, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gc... / linux-job (gh)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• Nightly ESP-IDF component / ESP-IDF master (1/4) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]At least one of the pre-conditions you specified did not hold
• Nightly ESP-IDF component / ESP-IDF master (2/4) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]At least one of the pre-conditions you specified did not hold
• Nightly ESP-IDF component / ESP-IDF master (3/4) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]At least one of the pre-conditions you specified did not hold
• Nightly ESP-IDF component / ESP-IDF master (4/4) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]At least one of the pre-conditions you specified did not hold
• Nightly ESP-IDF component / ESP-IDF v6.0.1 (2/4) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]At least one of the pre-conditions you specified did not hold
• Nightly ESP-IDF component / ESP-IDF v6.0.1 (3/4) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]At least one of the pre-conditions you specified did not hold
• Nightly ESP-IDF component / ESP-IDF v6.0.1 (4/4) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]At least one of the pre-conditions you specified did not hold
• Nightly ESP-IDF component / ESP-IDF v6.1 (1/4) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]At least one of the pre-conditions you specified did not hold
• Nightly ESP-IDF component / ESP-IDF v6.1 (2/4) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]Error uploading build-esp32-default.log
• Nightly ESP-IDF component / ESP-IDF v6.1 (3/4) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]At least one of the pre-conditions you specified did not hold
• Nightly ESP-IDF component / ESP-IDF v6.1 (4/4) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]At least one of the pre-conditions you specified did not hold
• Publish ESP-IDF component / Registry validation on https://components-staging.espressif.com (upload --dry-run) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 2

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• trunk / test-arm-backend-vkml (test_pytest_models_vkml) / linux-job (gh) (trunk failure)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[rascani]

@claude review effort=max

[claude]

Claude finished @rascani's task in 36m 34s —— View job

---

Request changes: the nightly wheel that the check downloads is uploaded with the esp-idf-release GitHub artifact even though nothing reads it, and torch_version is read from torch_pin.py instead of from what the nightly wheel build installs, so the README's torch install breaks once main builds against PyTorch nightlies again.

Detailed review

PR Review: #24748

Summary

The PR adds a -dev publish path. It stamps the nightly branch tip as X.Y.Z-dev.YYYYMMDD, checks that the nightly wheel for that day was built from the same commit, and records torch_version in VERSION for users.

Two problems need fixing:

• The downloaded wheel ends up in the uploaded artifact.
• torch_version comes from a file the wheel build doesn't use.

Smaller problems:

• Dev runs share the release concurrency group, so they can cancel a pending release run.
• The dirty-tree check parses free text, and the dev checks run only at the end of the job.
• The wheel check covers only one platform.
• Nothing tests the new path before its first real dispatch.

API Design

• torch_version comes from a file the nightly wheel build never reads (examples/espressif/build_idf_component.sh:619-630, idf_component_template/README.md:50-59).

  • The nightly wheels get torch from install_requirements.sh --example (.ci/scripts/wheel/pre_build_script.sh:165).
  • install_requirements.py installs its own literal torch==2.14.0 (line 124) from TORCH_URL_BASE = whl/test (line 18), and doesn't read torch_pin.py. The two are separate literals that happen to agree today.
  • Between torch releases, main normally builds against PyTorch nightlies (comments at install_requirements.py:16-17 and :74-79). The cu134 path already pins torch==2.15.0.dev20260921 (line 30).
  • Once main is back on nightlies, TORCH_VERSION alone will name a release that doesn't exist yet. Once that release exists, it will be a different build from the one the wheel was compiled against. Either way, the README's pip install … "torch==<torch_version>" fails or installs the wrong torch.
  • The registry archive can't be changed after upload, and wheel() doesn't check this.
  • In that mode, the torch to install is itself a nightly pinned by date, so it is older than the commit. install_requirements.py:77-79 ties it to NIGHTLY_VERSION, which is dev20261004, 33 days before HEAD. It therefore leaves the nightly index before the executorch wheel does, short of the "about two months" the README promises.
  • Suggested fix:
    • Move the torch pin at install_requirements.py:124 into a module-level constant.
    • Have the generator load that file the way setup.py:1118-1144 (_torchao_requirement) does, and record the exact requirement (including any .devYYYYMMDD) and its index in VERSION.
    • Use that index in the README.
    • Have wheel() resolve the README's exact command (e.g. pip download --no-deps "torch==…" with the same indexes) before upload.

  Fix this →

Infrastructure

• The nightly wheel is stored on GitHub, and nothing uses it (.ci/scripts/test_espressif_idf_nightly.sh:301,314-318).

  • wheels="${dir}/nightly-wheel", and the package job passes ${RUNNER_ARTIFACT_DIR} as dir (publish-esp-idf.yml:155).
  • That directory is uploaded as esp-idf-release (line 136, linux_job_v3, no S3).
  • No job reads nightly-wheel. Dry-run, build, registry and upload use only version, dist/, mirror/ and models/. A grep finds nightly-wheel only in wheel().
  • Footprint: one executorch wheel per -dev run (assumed tens of MB, not measured). Up to 13 jobs download it again (dry-run, 8 build shards, 2 uploads, 2 verifies). It is kept for the repository's default retention. The runbook allows up to one run a day.
  • S3 isn't an option, because upload-staging/upload-production read the artifact with actions/download-artifact on ubuntu-22.04. Keep the wheel out of the artifact directory instead: wheels=$(mktemp -d), or rm -rf it after the check.

  Fix this →

• Dev runs share the publish concurrency group with tag pushes and respins (publish-esp-idf.yml:37-40).

  • A dev run holds the group during its 8-shard build, and while upload-staging waits for an environment reviewer (examples/espressif/README.md:85).
  • cancel-in-progress is off for these runs, so GitHub keeps one pending run per group and cancels it when another run queues. If a release tag is pushed while a dev run is in progress, the next dev dispatch cancels the release run.
  • Dev versions can't collide with release versions, so -dev can have its own group.

  Fix this →

Testing

• Nothing runs wheel() or the -dev path before the first real dispatch on nightly.

  • The PR rehearsal always uses the release stamp (publish-esp-idf.yml:143-148).
  • nightly-espressif-idf.yml calls only prepare and build.
  • pull_request.paths (publish-esp-idf.yml:32-35) doesn't include test_espressif_idf_nightly.sh or build_idf_component.sh, so later edits to wheel() don't trigger a rehearsal.
  • Two assumptions are untested:
    • The version.py regex (test_espressif_idf_nightly.sh:326) depends on the format at setup.py:1317.
    • A nightly wheel's .devYYYYMMDD (the build-time BUILD_VERSION, setup.py:1296) is assumed to equal its nightly commit's UTC committer date (build_idf_component.sh:119). That depends on when the nightly is cut and when test-infra stamps the wheel, which I couldn't confirm from this repo.
  • A rehearsal step would cover both: download the newest nightly wheel, read its git_version, and check that the commit is on origin/nightly (already fetched, fetch-depth: 0) with that UTC date. Add the two scripts to the trigger paths too.
  • Was wheel() run by hand against a real nightly?

  Fix this →

Code Quality

• The dev publish skips the generator's checks and parses free text instead.

  • build_idf_component.sh:181-193 refuses uncommitted tracked changes, and untracked files where it globs sources, for every stamp except -dev.*. This PR makes --allow-dev-stamp publishable without extending those checks to it.
  • wheel() instead treats a space in executorch_commit as a dirty tree (test_espressif_idf_nightly.sh:304). That ties it to the wording and placement of the (tree had uncommitted changes) note (build_idf_component.sh:629).
  • The note follows a different rule from the release check. git status --porcelain at line 618 runs without --untracked-files=no, so any non-ignored untracked file anywhere in the checkout marks the tree dirty. A dev publish could therefore be refused over a file that a release would accept, for example one left behind by install_executorch. This part is suspected; I didn't check what CI leaves behind.
  • Suggested fix: a publishable dev stamp in the generator that runs the 181-193 checks (e.g. a bare --version-suffix -dev that resolves to HEAD's commit date).
    • It would replace this parse.
    • It would also replace the package job's stamp= special case (publish-esp-idf.yml:147-151). There, -dev travels through the suffix output even though it isn't a suffix.

  Fix this →

• The dev checks run last (publish-esp-idf.yml:152-155).

  • The date check (test_espressif_idf_nightly.sh:309) and the wheel check run only after install_executorch and the whole of prepare (ESP-IDF setup, four model exports, generation, pack, five gen_kernels runs).
  • A dispatch after the next nightly, which the runbook warns against, therefore fails only at the end. So does a wheel mismatch.
  • prepare already checks the stamp before the exports (lines 60-61). These two checks need only build_idf_component.sh --version --allow-dev-stamp and git rev-parse HEAD, so they can run first.

  Fix this →

• The wheel check covers only one of the day's wheels (test_espressif_idf_nightly.sh:316-318).

  • Without --platform/--python-version, pip download checks only the Linux x86_64 wheel for the job's Python.
  • The macOS, Windows and aarch64 wheels come from separate workflows triggered by the same push (build-wheels-macos.yml:27-28, build-wheels-windows.yml:21-22, build-wheels-aarch64-linux.yml:22-23). Each builds for 3.11–3.15 and can fail on its own.
  • If that nightly's macOS build fails, macOS users have no executorch==X.Y.Z.devYYYYMMDD to install, and the check still passes.
  • If a second nightly that day builds on Linux but fails on macOS, the macOS wheel is still the first nightly's, and gen_kernels.py rejects it.
  • Either check the other hosts' wheels, or say in the README which hosts were verified.

• The runbook doesn't say which registry dev versions go to (examples/espressif/README.md:74-80).

  • The dispatch defaults to staging, but projects resolve from production.
  • The plan bars -rc.N from production but not -dev (publish-esp-idf.yml:91-99). The PR's argument that ranges pick up pre-releases only on opt-in applies equally to RCs.
  • Each dev version stops being usable for new kernel bundles once its nightly wheel expires.
  • Please state the intended registry in the runbook. Also record the one-time change of adding nightly to esp-component-registry's deployment branches there, not only in the PR description.

• Low confidence (one reviewer, not reproduced): the build-from-source fallback probably expires as well (idf_component_template/README.md:61-63).

  • If "build from source" means ./install_executorch.sh, that commit's install_requirements.py pins exactly torchao==0.19.0.dev20261103 (lines 21, 125).
  • Its own comment says such nightly pins expire after about 60 days (line 27).
  • Name the override (EXECUTORCH_USE_RELEASE_CPU_DEPS=1, lines 43-50) or drop the fallback.

Specific Comments

• examples/espressif/build_idf_component.sh:24-26:
  • "never published" was dropped from --version-suffix -dev.YYYYMMDD, but the workflow still never publishes that stamp (the plan rejects it).
  • --allow-dev-stamp, now the published stamp, still reads "for builds from any commit".
• .ci/scripts/test_espressif_idf_nightly.sh:367:
  • wheel) wheel ;; drops "$@", unlike the other modes.
  • wheel() doesn't check that <dir> holds a -dev. version. On a release directory it fails with an unrelated pip "no matching distribution" error.
• examples/espressif/idf_component_template/README.md:52-53: "Python 3.11 or newer" should be 3.11–3.15 (build-wheels-linux.yml:47; requires-python is <3.16, pyproject.toml:60).
• examples/espressif/idf_component_template/README.md:50-51:
  • Release wheels don't declare torch either (setup.py:1223), and torch_version is now recorded for every version, but only the dev instructions mention it.
  • That a ~N respin uses executorch==X.Y.Z (gen_kernels.py:78) is only implied.
• examples/espressif/idf_component_template/README.md:19-20: the description of VERSION doesn't mention torch_version.
• examples/espressif/idf_component_template/tools/gen_kernels.py:98: "(see its README)" can be read as executorch's README. Say "the component's README.md".
• examples/espressif/README.md:42-43: "the executorch Python package of the same release" is out of date now that dev versions need a nightly wheel.
• examples/espressif/README.md:88-89: "once the nightly matrix is green on the release commit" has no dev equivalent. nightly-espressif-idf.yml runs on main at 07:00 UTC, not on the nightly commit.

Recommendation

Request Changes

• The wheel in esp-idf-release is confirmed GitHub storage waste, with a one-line fix.
• The torch_version source needs fixing before the first dev upload, because published archives can't be corrected later.
• The rest can go in this PR or a follow-up.