Convert a Poetry package to the src layout

The src layout is commonly used in Python ecosystem nowadays (even the pytest docs recommend it), but I’m not really using it on my projects. I think I’ve never hit the pain points it solves, and last time I tried, it made my development workflow more complicated.

I’m not going into the whys, they are detailed in this excellent blog explaining its rationale in detail, where I read about it the first time.

In the recent years, as it became more popular, the tooling has caught up, and I wanted to try it again to see if the situation improved.

Starting a new project

If you start a new project, Poetry supports it out of the box in their poetry new command:

poetry new --src my-package

Although you might want to use a full blown template with other tools also configured (e.g. pytest).

Converting a project

I found it a bit less well documented how to convert a project, so here you go. Assuming the project uses Poetry, pytest and a layout as follows:

my-package
├── poetry.lock
├── pyproject.toml
├── my_package
│   ├── __init__.py
│   └── main.py
└── tests
    ├── __init__.py
    └── test_main.py

Here are the steps I followed to convert to src/ layout, without changing the imports in my tests:

1. Create a src/ folder. And I really mean folder, not package: do not create a __init__.py in that folder.

2. Move my_package into src/.

3. Change you packages section in your pyproject.toml:

     packages = [
   -     { include = "my_package" },
   +     { include = "my_package", from = "src" },
     ]

4. Add pythonpath to your pytest option as follows:

     [tool.pytest.ini_options]
   + pythonpath = ["src"]

   If using pytest <7, you’ll also need to install pytest-srcpaths:

   poetry add -D pytest-srcpaths

5. If your pyproject.toml references some paths, make sure to update them, e.g python-semantic-release:

     [tool.semantic_release]
     branch = "main"
   - version_variable = "my_package/__init__.py:__version__"
   + version_variable = "src/my_package/__init__.py:__version__"

If everything went well, you should have the following layount and running pytest should work:

my-package
├── poetry.lock
├── pyproject.toml
├── src
│   └── my_package
│       ├── __init__.py
│       └── main.py
└── tests
    ├── __init__.py
    └── test_main.py

Conclusion

This was much easier than I remembered! I think I’ll change all my projects to this layout, having this guide will help me reprodude it. Hopefulluy it’ll help you as well!