Poetry: build.py example
All notes in this series:
- Poetry: Fixing dubious ownership error
- Poetry: build.py example
- Poetry: Automatically generate package version from git commit
- Poetry: Fix warning about sources
- Poetry: Running Black and isort with pre-commit hooks
- Poetry: Fixing permission error when upgrading dulwich
- NiceGUI with Click, Poetry, auto-reload and classes
- Poetry: Offline installation of packages
- Run Poetry command as systemd service
- GitLab CI and poetry-dynamic-versioning
- Poetry: install alpha builds
- Upgrade version of pkginfo used by Poetry
Poetry contains an undocumented feature which allows running custom code when poetry build
is executed. This can be particularly useful to, for example, compile a C extension, or perform some sort of pre-processing. This is done via a build.py
and some configuration.
As can be seen by reading this thread, there has been a lot of churn over how to use this feature1. Given that it’s undocumented, this comes with all the usual caveats regarding stability. But for now it appears to be the only way to call custom code during poetry build
, which for some cases is a necessity.
So, without further delay, following is an example of using this feature, as it works today.
Example §
- Add a
build.py
in the same directory as yourpyproject.toml
. Inside, put adef build(setup_kwargs):
function, which is where your custom code will go:
- Delete your
setup.py
(assuming you already just have a shim). - Update your
pyproject.toml
to enable use of thebuild.py
. Thegenerate-setup-file = true
tells Poetry to generate asetup.py
whenpoetry build
is run (hence the need to remove your own one), and thescript = "build.py"
causes the generatedsetup.py
to have a line that executes yourbuild.py
:2
- Update your
pyproject.toml
to addsetuptools
as abuild-system
dependency:3
- Test your build:
- That’s it!
Caveat §
Beware that one caveat of this approach is that any use of a build.py
script causes Poetry to assume build-specific tags are necessary, and it uses the most specific ones. In other words, if you are building under Python 3.10 using Linux, your wheel will now have a name such as:
Rather than:
Since the use of build.py
is undocumented, it seems that features related to this are low priority.4
Some workarounds have been suggested by the community:
- One fix suggested by tedivm is to use this script to convert the wheel into a pure Python wheel after the Poetry build has finished.
- A similar fix suggested by domvwt is to use this script.
- A different workaround suggested by Kaiser1989 is to call setuptools within Poetry to explicitly generate a wheel, though I was not able to get this to work.
- A suggestion by gitonthescene is to perform the build in two passes: first with your
build.py
enabled, and second with it disabled.
Two pass build §
I successfully used the two pass build suggestion. A simplified example implementation follows:
Performing the build:
This is not ideal, but if you are using tox/nox to perform the build, then these steps can be wrapped up in a single command.
Conclusion §
Ultimately, I decided not to use build.py
. Since I am already wrapping by call to poetry build
with Tox, I decided to put the custom build command in the Tox script instead, and avoid the complexity and potential instability of using this undocumented feature.
To do so, I created my_build_script.py
: a simple standard library only Python script which performs the custom build command:
Then in tox.ini
called the script like so:
Then I can run tox -e py310-build
to perform my custom build.
Troubleshooting §
The build.py
is not being called §
Check the output of poetry build
to see whether it contains the following:
If so, the issue is likely that you have generate-setup-file = true
, but because a setup.py
already exists, poetry is not generating one. The generated setup.py
has a block of code that calls build.py
. A fix is to delete your setup.py
.
If that is not a suitable solution, you will probably need to modify your setup.py
to call your custom build code.
ModuleNotFoundError: No module named ‘setuptools’ §
This should be fixed by adding setuptools to the build-system
.
Backlinks §
- I’ve linked to this page from this GitHub comment.
This fairly prominent answer when searching online also no longer seems to work. ↩︎
Thank you to radoering for pointing this out, and linking to this official poetry example. ↩︎
Thank you to albireox for pointing this out. ↩︎
Though some related work was begun in this ticket, which was subsequently split into these two, the first of which has been merged. ↩︎
All notes in this series:
- Poetry: Fixing dubious ownership error
- Poetry: build.py example
- Poetry: Automatically generate package version from git commit
- Poetry: Fix warning about sources
- Poetry: Running Black and isort with pre-commit hooks
- Poetry: Fixing permission error when upgrading dulwich
- NiceGUI with Click, Poetry, auto-reload and classes
- Poetry: Offline installation of packages
- Run Poetry command as systemd service
- GitLab CI and poetry-dynamic-versioning
- Poetry: install alpha builds
- Upgrade version of pkginfo used by Poetry