Repository navigation
Document how to add a bytecode specialization #1286
Description
How to add a new bytecode specialization is knowledge only a few of us remember right now. We should document this somewhere (devguide or CPython, I don't mind). Hopefully a new contributor can pick this up!
There reference is this PR: https://github.com/python/cpython/pull/116385/files
Assuming you found an instruction that serves as a good specialization candidate. Let's use the example of CONTAINS_OP:
- Set the original
CONTAINS_OPto a uop by changing it fromCONTAINS_OPto_CONTAINS_OPinPython/bytecodes.c. The instruction definition should also change frominsttoop. - Add the uop that calls a specializing function. In this case
_SPECIALIZE_CONTAINS_OP. - The original
CONTAINS_OPis now a new macro consisting of_SPECIALIZE_CONTAINS_OP + _CONTAINS_OP. - Define the cache structure in
Include/internal/pycore_code.h. At the very least, a 16-bit counter is needed. - Write the specializing function itself in
Python/specialize.c. Refer to any other function in that file for the format. - Remember to update operation stats by calling
add_stat_dictinPython/specialize.c. - Add the cache layout in
Lib/opcode.pyso that Python'sdismodule will know how to represent it properly. - Bump magic number in
Include/internal/pycore_magic_number.h. - Run
make regen-allon *nix orbuild.bat --regenon PC.
Activity
AbudiMutamba commentedon Oct 10, 2024
Hey I would love to work on this issue if you don't mind, thanks,
willingc commentedon Oct 10, 2024
Hi @AbudiMutamba. Welcome 👋 Before you get too far along on this. Let's check with @Fidget-Spinner if this is still something that is needed?
Fidget-Spinner commentedon Oct 10, 2024
Yes we still need this.
willingc commentedon Oct 10, 2024
Thanks @Fidget-Spinner. If you want to give this a try @AbudiMutamba, please do so.
faizanoor3001 commentedon Feb 20, 2025
Hello, @willingc is it something still open ? Would like to contribute. :)
Fidget-Spinner commentedon Feb 20, 2025
@faizanoor3001 this is still open
faizanoor3001 commentedon Feb 24, 2025
Hello, @Fidget-Spinner, I’ve opened a PR adding “How to Add a New Bytecode Specialization” to the How-To section. We can verify the text changes or if any updates are required, but I’m unsure about the best place for this new guide. Would it fit better in:
Interpreter.md section - appended under the existing InternalDocs?
CPython Internals — as its own subsection?
Current:
Potential Alternative:
PS: This is my first contribution, so I appreciate your guidance. I’m also studying the Crafting Interpreters book, and your original PR helped me delve deeper into the interpreter’s workings. Thank you!
Fidget-Spinner commentedon Feb 24, 2025
@faizanoor3001 thanks for taking this up! I will review it soon.
faizanoor3001 commentedon Mar 4, 2025
@Fidget-Spinner Raised a PR, but since the issue is in devguide project and the PR I have raised in the cpython project, the bot is unable to recognize the gh-NNNN issue number in this case gh-1286 as correct issue number, could you please help here, as I guess we would need a new issue number ? Or how to work around on this ?
StanFromIreland commentedon Feb 21, 2026
In general we've removed the internals sections here, maybe this is better suited for InternalDocs?
nayanraj864-cmyk commentedon Sep 8, 2026
pushkarkurkure172-crypto commentedon Oct 29, 2026
Hello @Fidget-Spinner,
I'm interested in contributing to this issue as a new open-source contributor.
I noticed that previous contributors have already submitted pull requests related to this documentation. Is there still any work remaining that I could help with?
I would be happy to work on any improvements or updates needed.
Thank you!