[rascani]

The ESP-IDF component will compile only the kernels a project's models need,
chosen at export time instead of from op profiles baked into the component.
gen_kernels.py is that export-side step. It calls ExecuTorch's selective-build
tools in-process from the installed executorch package (gen_oplist per model,
for --ops and per kernel library; gen_all_oplist for the union with any
--oplist files; codegen.gen's generate() per kernel library;
gen_selected_prim_ops and gen_max_kernel_num), the same pipeline
tools/cmake/Codegen.cmake runs, and writes a bundle of registration sources
plus kernels.json. generate() finds native_functions.yaml and tags.yaml in the
torchgen package that comes with torch, so gen_kernels.py names no torchgen
paths. kernels.json records the component version and commit, the executorch
version and commit, each model's file name and SHA-256, the ops per library,
the selected prim ops, the delegate IDs the models use, and the operators
requested with --ops or --oplist, which a regenerate has to pass again. Nothing
in it depends on where the bundle or the models live, so the same inputs always
give the same bundle. The component's CMake consumes the bundle in a follow-up.

An op that several kernel libraries declare must be declared identically by
each, or gen_kernels.py stops with an error naming the libraries. Today that
covers the 36 aten:: entries the Cadence generic yaml repeats verbatim from
portable's; such an op is generated once, from the first library in the
component's kernel_libraries.json order (portable, quantized, Cadence generic,
ESP-NN). Codegen runs once per library, because a single run over several yamls
can bind an op to another library's kernel. An op that no library provides, a
bundled program passed instead of a .pte, and an executorch install that does
not match the component are errors. A matching install is one built from the
component's commit or, for a release or its respins, that release's wheel. The output directory is replaced only when
every file in it is one a previous bundle wrote, or the .DS_Store, Thumbs.db
and desktop.ini that file browsers leave, so a model, op list or other file
kept anywhere inside it is never deleted.

export_espnn_test_programs.py also stops writing a trailing "." for operators
whose overload name is empty, since its operators.txt is meant as
gen_kernels.py --ops input and "aten::copy_." matches no kernel library.

Authored with Claude Code.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24705

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 39a872b with merge base 60a8b5f ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[JakeStevens]

consider letting a small fixed set of OS metadata names (.DS_Store, Thumbs.db, desktop.ini) be deleted along with the bundle, and keep refusing everything else

otherwise people can get blocked if they open a file thru finder, etc