Repository navigation
[C API] Deprecate _Py_Identifier API #142217
Description
I propose to deprecate the private _Py_Identifier API.
- Deprecate private
_PyObject_CallMethodId(),_PyObject_GetAttrId(),_PyUnicode_FromId(). - Remove immediately internal functions using
_Py_Identifier.
See also the discussions:
- Deprecate private C API functions #141049
- https://discuss.python.org/t/c-api-how-much-private-is-the-private-py-identifier-api/29190 (2023).
- https://mail.python.org/archives/list/python-dev@python.org/thread/DNMZAMB4M6RVR76RDZMUK2WRLI6KAAYS/ (2022)
Since 2022, CPython has been modified to replace _Py_IDENTIFIER() with _Py_STR(). Python no longer uses the _Py_IDENTIFIER() API anymore.
See also issue #141049 "Deprecate private C API functions".
Activity
pythongh-142217: Remove internal _Py_Identifier functions
gh-142217: Remove internal _Py_Identifier functions (#142219)
pythongh-142217: Deprecate the private _Py_Identifier C API
pythongh-142217: Remove internal _Py_Identifier functions (python#142219
gh-142217: Deprecate the private _Py_Identifier C API (#142221)
cdce8p commentedon Dec 14, 2025
I just tested the latest cpython build against mypy. #142221 is causing a lot of errors with
-Werror,-Wdeprecated-declarations. Yes it's possible to replace most_Py_IDENTIFIERcalls withPyUnicode_FromString, see python/mypy@f287f7d, but I'm not sure this is the right call here. The discuss thread mentions the need / desire for an API to get/create immortal strings, has there been any progress for it?cpython itself uses
&_Py_IDin a lot of places. These could actually also be used by mypy in most cases though I don't think it's intended that way. python/mypy@dbbc992encukou commentedon Dec 15, 2025
Not really.
Note that CPython's immortal strings (
_Py_STR) are statically allocated, in a way that requires code generation with lots of knowledge about the internals, and taking advantage of the fact that there are no conflicts.We unfortunately can't provide that exact API to extensions -- if two of those intern the same string, only one can “win”.
AFAIK, we can't really make
_Py_IDENTIFIERsubstantially better than users callingif(!x) {x = PyUnicode_InternFromString("...");}on init, andPy_CLEAR(x)on teardown.The advantage of
_Py_Identifieris the ease of use -- in particular, the teardown is automatic. But it comes at the cost of an elaborate (and largely untested) mechanism to make everything work correctly.Don't do that, this uses internal ABI that will break in patch releases. As in: it does break in practice in patch releases, it's not a CYA theoretical possibility.
@vstinner: would
PyUnicode_InternFromStringbe a better suggested replacement thanPyUnicode_FromString?pythongh-142217: Recommend PyUnicode_InternFromString() to replace _P…
vstinner commentedon Dec 15, 2025
Oh, good idea: I wrote #142746 to update the documentation.
gh-142217: Recommend PyUnicode_InternFromString() to replace _PyUnico…
pythongh-142217: Recommend PyUnicode_InternFromString() to replace _P…
vstinner commentedon Dec 16, 2025
Yes, it's a safe replacement. To avoid negative impact on performance, you might try to only decode bytes strings (
char*) from UTF-8 once by storing the object (PyObject*) in a module state, if possible.I close the issue since the APIs are now deprecated.
cdce8p commentedon Dec 22, 2025
This is what I got working for mypy: python/mypy#20460
It's heavily inspired by the numpy implementation.
pythongh-142217: Add tests on the deprecated _Py_Identifier C API
pythongh-142217: Add tests on the deprecated _Py_Identifier C API
gh-142217: Add tests on the deprecated _Py_Identifier C API (#157638)