Repository navigation
Build directory for editable installations #370
Description
The build directory for editable build is currently .mesonpy/editable/build in the source directory. I don't see a reason why the build directory is nested two levels deep into otherwise empty directories . A flatter directory structure could be used.
While doing so, attaching a Python API tag to the build directory name would allow to have multiple parallel installations of the same package for different interpreters.
Isn't there prior art for choosing the name of the build directory? I don't like something specific to meson-python much. Python uses __pycache__ for its cache. Is it allowed to add tool specific stuff in there? distutils uses a build folder, IIRC. What about other build backends?
It would be nice to have this decided before we release a version with the editable support.
EDIT: reworded for politeness
Activity
rgommers commentedon Mar 29, 2023
I'd say
build/in the root dir is the one more or less standard name - not just distutils/setuptools, but also Sphinx (also that also uses_buildsometimes). Not sure about other backends, never had a need to actually use them.The downside of
buildis that it may clash with a standard choice for using Meson directly (meson setup build,ninja -C buildis already mostly muscle memory for me).Reusing
__pycache__is a no no I'd say.Hmm, that is an interesting point.
rgommers commentedon Mar 29, 2023
I'd say that whatever we choose here is a private implementation detail though - the user has no business doing anything with it beyond adding it to
.gitignore, so we should be fine tweaking it later.rgommers commentedon Mar 30, 2023
Thinking about this some more:
buildis the standard namebuilddiroption (I tested it withpip install -e . --no-build-isolation --config-settings=builddir=build-py310and it works)buildclashing with thedistutilsdefault may actually be a good thing; having in-tree remnants from a distutils build can in some cases cause problemstl;dr let's use
build?dnicolodi commentedon Mar 30, 2023
I like
buildtoo. I like supporting multiple interpreters out of the box. Having a two level directory structure solves this:build/cp311orbuild/cpython-3.11/but I don't know if this is a common enough use case that is worth supporting.rgommers commentedon Mar 30, 2023
I'd prefer to not special-case interpreters I think. Different builds can also happen with different compilers (e.g., using GCC and Clang in parallel), different sets of build dependencies (on SciPy testing with 2 numpy versions is much more interesting than testing with 2 cpython versions), and so on.
Plus it does bring up questions like "what if we have regular and a debug 3.11 interpreters".
Matrix'ed environment testing is for tools like tox/nox or custom scripts I'd say, and they can easily pass a
builddirfor anything like this.FFY00 commentedon Mar 31, 2023
FFY00 commentedon Mar 31, 2023
Other possible option would be putting the build directory on the installation path, instead of the project source. This would also allow installations to different interpreters without any conflicts.
dnicolodi commentedon Mar 31, 2023
rgommers commentedon Mar 31, 2023
rgommers commentedon Mar 31, 2023
Okay, you both agree on this, so let's go that way.
For the other choices, how about I make an editorial decision here? Seems preferred over lots more discussion.
dnicolodi commentedon Mar 31, 2023
I proposed tagging the build directory with the python interpreter API tag in some form because the editable wheel documentation page prominently warns (or at least I remember it this way) to the fact that being able to have editable wheels for multiple interpreters as a limitation. I thought that, if the limitation was felt so important, adding a tiny amount of complexity to allow it could have been a good compromise.
However, I'm a big fan of keeping things simple unless necessary, thus I'm happy without this feature and a paragraph in the documentation that mentions how to achieve the same using the
builddirconfig setting.I'm fine with @rgommers picking the build directory name that he prefers. I opened this issue only because the nested directories are not required to exist anymore and I thought we could simplify what we present to the users before releasing editable wheel support in the next release.
FFY00 commentedon Apr 2, 2023
FFY00 commentedon Apr 2, 2023
dnicolodi commentedon Apr 3, 2023
dnicolodi commentedon Apr 3, 2023
FFY00 commentedon Apr 4, 2023
dnicolodi commentedon Apr 4, 2023
We can use the same trick that Meson uses and place a
.gitignorecontaining*in the build directory. Meson does not do that when the build directory is not empty, and we add the native file to the build directory before invokingmeson setup, thus Meson does not do that for us.dnicolodi commentedon Apr 7, 2023
setuptools uses a
build/__editable__.*directory for editable installs in strict mode https://setuptools.pypa.io/en/latest/userguide/development_mode.html#strict-editable-installs I haven't checked yet what.*should be replaced withENH: make the editable install build directory more discoverable
ENH: make the editable install build directory more discoverable