Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
P
pytreat-practicals-2020
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Tom Nichols
pytreat-practicals-2020
Commits
46f3ea65
Commit
46f3ea65
authored
5 years ago
by
Paul McCarthy
Browse files
Options
Downloads
Patches
Plain Diff
MNT: cosmetic tweaks
parent
cbbff0df
No related branches found
No related tags found
No related merge requests found
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
advanced_topics/09_structuring_projects.ipynb
+35
-28
35 additions, 28 deletions
advanced_topics/09_structuring_projects.ipynb
advanced_topics/09_structuring_projects.md
+35
-28
35 additions, 28 deletions
advanced_topics/09_structuring_projects.md
with
70 additions
and
56 deletions
advanced_topics/09_structuring_projects.ipynb
+
35
−
28
View file @
46f3ea65
...
@@ -19,15 +19,15 @@
...
@@ -19,15 +19,15 @@
"\n",
"\n",
"\n",
"\n",
"* [Recommended project structure](#recommended-project-structure)\n",
"* [Recommended project structure](#recommended-project-structure)\n",
" * [The `mypackage/` directory](#the-mypackage-directory)\n",
"
* [The `mypackage/` directory](#the-mypackage-directory)\n",
" * [`README`](#readme)\n",
"
* [`README`](#readme)\n",
" * [`LICENSE`](#license)\n",
"
* [`LICENSE`](#license)\n",
" * [`requirements.txt`](#requirements-txt)\n",
"
* [`requirements.txt`](#requirements-txt)\n",
" * [`setup.py`](#setup-py)\n",
"
* [`setup.py`](#setup-py)\n",
"* [Appendix: Tests](#appendix-tests)\n",
"* [Appendix: Tests](#appendix-tests)\n",
"* [Appendix: Versioning](#appendix-versioning)\n",
"* [Appendix: Versioning](#appendix-versioning)\n",
" * [Include the version in your code](#include-the-version-in-your-code)\n",
"
* [Include the version in your code](#include-the-version-in-your-code)\n",
" * [Deprecate, don't remove!](#deprecate-dont-remove)\n",
"
* [Deprecate, don't remove!](#deprecate-dont-remove)\n",
"* [Appendix: Cookiecutter](#appendix-cookiecutter)\n",
"* [Appendix: Cookiecutter](#appendix-cookiecutter)\n",
"\n",
"\n",
"\n",
"\n",
...
@@ -65,7 +65,7 @@
...
@@ -65,7 +65,7 @@
"\n",
"\n",
"The first thing you should do is make sure that all of your python code is\n",
"The first thing you should do is make sure that all of your python code is\n",
"organised into a sensibly-named\n",
"organised into a sensibly-named\n",
"[
_
package
_
](https://docs.python.org/3
.5
/tutorial/modules.html#packages). This\n",
"[
*
package
*
](https://docs.python.org/3/tutorial/modules.html#packages). This\n",
"is important, because it greatly reduces the possibility of naming collisions\n",
"is important, because it greatly reduces the possibility of naming collisions\n",
"when people install your library alongside other libraries. Hands up those of\n",
"when people install your library alongside other libraries. Hands up those of\n",
"you who have ever written a file called `utils.[py|m|c|cpp]`!\n",
"you who have ever written a file called `utils.[py|m|c|cpp]`!\n",
...
@@ -106,10 +106,12 @@
...
@@ -106,10 +106,12 @@
"that your project requires. For example:\n",
"that your project requires. For example:\n",
"\n",
"\n",
"\n",
"\n",
"> six==1.*\n",
"> ```\n",
"> numpy==1.*\n",
"> six==1.*\n",
"> scipy>=0.18,<2\n",
"> numpy==1.*\n",
"> nibabel==2.*\n",
"> scipy>=0.18\n",
"> nibabel==2.*\n",
"> ```\n",
"\n",
"\n",
"\n",
"\n",
"If your project has optional dependencies, i.e. libraries which are not\n",
"If your project has optional dependencies, i.e. libraries which are not\n",
...
@@ -122,7 +124,9 @@
...
@@ -122,7 +124,9 @@
"others to install the dependencies needed by your project, simply by running:\n",
"others to install the dependencies needed by your project, simply by running:\n",
"\n",
"\n",
"\n",
"\n",
"> pip install -r requirements.txt\n",
"> ```\n",
"> pip install -r requirements.txt\n",
"> ```\n",
"\n",
"\n",
"\n",
"\n",
"<a class=\"anchor\" id=\"setup-py\"></a>\n",
"<a class=\"anchor\" id=\"setup-py\"></a>\n",
...
@@ -174,7 +178,7 @@
...
@@ -174,7 +178,7 @@
">\n",
">\n",
"> name='Example project',\n",
"> name='Example project',\n",
"> description='Example Python project for PyTreat',\n",
"> description='Example Python project for PyTreat',\n",
"> url='https://git.fmrib.ox.ac.uk/fsl/pytreat-
2018-
practicals/',\n",
"> url='https://git.fmrib.ox.ac.uk/fsl/pytreat-practicals
-2020
/',\n",
"> author='Paul McCarthy',\n",
"> author='Paul McCarthy',\n",
"> author_email='pauldmccarthy@gmail.com',\n",
"> author_email='pauldmccarthy@gmail.com',\n",
"> license='Apache License Version 2.0',\n",
"> license='Apache License Version 2.0',\n",
...
@@ -220,7 +224,7 @@
...
@@ -220,7 +224,7 @@
"right?). There are two main conventions:\n",
"right?). There are two main conventions:\n",
"\n",
"\n",
"\n",
"\n",
"You can store your test files
_
inside
_
your package directory:\n",
"You can store your test files
*
inside
*
your package directory:\n",
"\n",
"\n",
"\n",
"\n",
"> ```\n",
"> ```\n",
...
@@ -234,8 +238,7 @@
...
@@ -234,8 +238,7 @@
"> ```\n",
"> ```\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"Or, you can store your test files *alongside* your package directory:\n",
"Or, you can store your test files _alongside_ your package directory:\n",
"\n",
"\n",
"\n",
"\n",
"> ```\n",
"> ```\n",
...
@@ -267,7 +270,7 @@
...
@@ -267,7 +270,7 @@
"\n",
"\n",
"If you are intending to make your project available for public use (e.g. on\n",
"If you are intending to make your project available for public use (e.g. on\n",
"[PyPI](https://pypi.python.org/pypi) and/or\n",
"[PyPI](https://pypi.python.org/pypi) and/or\n",
"[conda](https://anaconda.org/anaconda/repo)), it is
__
very important
__
to\n",
"[conda](https://anaconda.org/anaconda/repo)), it is
**
very important
**
to\n",
"manage the version number of your project. If somebody decides to build their\n",
"manage the version number of your project. If somebody decides to build their\n",
"software on top of your project, they are not going to be very happy with you\n",
"software on top of your project, they are not going to be very happy with you\n",
"if you make substantial, API-breaking changes without changing your version\n",
"if you make substantial, API-breaking changes without changing your version\n",
...
@@ -277,11 +280,13 @@
...
@@ -277,11 +280,13 @@
"Python has [official standards](https://www.python.org/dev/peps/pep-0440/) on\n",
"Python has [official standards](https://www.python.org/dev/peps/pep-0440/) on\n",
"what constitutes a valid version number. These standards can be quite\n",
"what constitutes a valid version number. These standards can be quite\n",
"complicated but, in the vast majority of cases, a simple three-number\n",
"complicated but, in the vast majority of cases, a simple three-number\n",
"versioning scheme comprising
_
major
_
,
_
minor
_
, and
_
patch
_
release\n",
"versioning scheme comprising
*
major
*
,
*
minor
*
, and
*
patch
*
release\n",
"numbers should suffice. Such a version number has the form:\n",
"numbers should suffice. Such a version number has the form:\n",
"\n",
"\n",
"\n",
"\n",
"> major.minor.patch\n",
"> ```\n",
"> major.minor.patch\n",
"> ```\n",
"\n",
"\n",
"\n",
"\n",
"For example, a version number of `1.3.2` has a _major_ release of 1, _minor_\n",
"For example, a version number of `1.3.2` has a _major_ release of 1, _minor_\n",
...
@@ -291,25 +296,25 @@
...
@@ -291,25 +296,25 @@
"If you follow some simple and rational guidelines for versioning\n",
"If you follow some simple and rational guidelines for versioning\n",
"`your_project`, then people who use your project can, for instance, specify\n",
"`your_project`, then people who use your project can, for instance, specify\n",
"that they depend on `your_project==1.*`, and be sure that their code will work\n",
"that they depend on `your_project==1.*`, and be sure that their code will work\n",
"for
_
any
_
version of `your_project` with a major release of 1. Following these\n",
"for
*
any
*
version of `your_project` with a major release of 1. Following these\n",
"simple guidelines greatly improves software interoperability, and makes\n",
"simple guidelines greatly improves software interoperability, and makes\n",
"everybody (i.e. developers of other projects, and end users) much happier!\n",
"everybody (i.e. developers of other projects, and end users) much happier!\n",
"\n",
"\n",
"\n",
"\n",
"Many modern Python projects use some form of [
_
semantic\n",
"Many modern Python projects use some form of [
*
semantic\n",
"versioning
_
](https://semver.org/). Semantic versioning is simply a set of\n",
"versioning
*
](https://semver.org/). Semantic versioning is simply a set of\n",
"guidelines on how to manage your version number:\n",
"guidelines on how to manage your version number:\n",
"\n",
"\n",
"\n",
"\n",
" - The
_
major
_
release number should be incremented whenever you introduce any\n",
" - The
*
major
*
release number should be incremented whenever you introduce any\n",
" backwards-incompatible changes. In other words, if you change your code\n",
" backwards-incompatible changes. In other words, if you change your code\n",
" such that some other code which uses your code would break, you should\n",
" such that some other code which uses your code would break, you should\n",
" increment the major release number.\n",
" increment the major release number.\n",
"\n",
"\n",
" - The
_
minor
_
release number should be incremented whenever you add any new\n",
" - The
*
minor
*
release number should be incremented whenever you add any new\n",
" (backwards-compatible) features to your project.\n",
" (backwards-compatible) features to your project.\n",
"\n",
"\n",
" - The
_
patch
_
release number should be incremented for backwards-compatible\n",
" - The
*
patch
*
release number should be incremented for backwards-compatible\n",
" bug-fixes and other minor changes.\n",
" bug-fixes and other minor changes.\n",
"\n",
"\n",
"\n",
"\n",
...
@@ -328,7 +333,9 @@
...
@@ -328,7 +333,9 @@
"our `example_project/mypackage/__init__.py` file contains this line:\n",
"our `example_project/mypackage/__init__.py` file contains this line:\n",
"\n",
"\n",
"\n",
"\n",
"> __version__ = '0.1.0'\n",
"> ```\n",
"> __version__ = '0.1.0'\n",
"> ```\n",
"\n",
"\n",
"\n",
"\n",
"This makes a library's version number programmatically accessible and\n",
"This makes a library's version number programmatically accessible and\n",
...
@@ -341,7 +348,7 @@
...
@@ -341,7 +348,7 @@
"\n",
"\n",
"If you really want to change your API, but can't bring yourself to increment\n",
"If you really want to change your API, but can't bring yourself to increment\n",
"your major release number, consider\n",
"your major release number, consider\n",
"[
_
deprecating
_
](https://en.wikipedia.org/wiki/Deprecation#Software_deprecation)\n",
"[
*
deprecating
*
](https://en.wikipedia.org/wiki/Deprecation#Software_deprecation)\n",
"the old API, and postponing its removal until you are ready for a major\n",
"the old API, and postponing its removal until you are ready for a major\n",
"release. This will allow you to change your API, but retain\n",
"release. This will allow you to change your API, but retain\n",
"backwards-compatilbiity with the old API until it can safely be removed at the\n",
"backwards-compatilbiity with the old API until it can safely be removed at the\n",
...
...
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
# Structuring a Python project
# Structuring a Python project
If you are writing code that you are sure will never be seen or used by
If you are writing code that you are sure will never be seen or used by
anybody else, then you can structure your project however you want, and you
anybody else, then you can structure your project however you want, and you
can stop reading now.
can stop reading now.
However, if you are intending to make your code available for others to use,
However, if you are intending to make your code available for others to use,
either as end users, or as a dependency of their own code, you will make their
either as end users, or as a dependency of their own code, you will make their
lives much easier if you spend a little time organising your project
lives much easier if you spend a little time organising your project
directory.
directory.
*
[
Recommended project structure
](
#recommended-project-structure
)
*
[
Recommended project structure
](
#recommended-project-structure
)
*
[
The `mypackage/` directory
](
#the-mypackage-directory
)
*
[
The `mypackage/` directory
](
#the-mypackage-directory
)
*
[
`README`
](
#readme
)
*
[
`README`
](
#readme
)
*
[
`LICENSE`
](
#license
)
*
[
`LICENSE`
](
#license
)
*
[
`requirements.txt`
](
#requirements-txt
)
*
[
`requirements.txt`
](
#requirements-txt
)
*
[
`setup.py`
](
#setup-py
)
*
[
`setup.py`
](
#setup-py
)
*
[
Appendix: Tests
](
#appendix-tests
)
*
[
Appendix: Tests
](
#appendix-tests
)
*
[
Appendix: Versioning
](
#appendix-versioning
)
*
[
Appendix: Versioning
](
#appendix-versioning
)
*
[
Include the version in your code
](
#include-the-version-in-your-code
)
*
[
Include the version in your code
](
#include-the-version-in-your-code
)
*
[
Deprecate, don't remove!
](
#deprecate-dont-remove
)
*
[
Deprecate, don't remove!
](
#deprecate-dont-remove
)
*
[
Appendix: Cookiecutter
](
#appendix-cookiecutter
)
*
[
Appendix: Cookiecutter
](
#appendix-cookiecutter
)
Official documentation:
Official documentation:
https://packaging.python.org/tutorials/distributing-packages/
https://packaging.python.org/tutorials/distributing-packages/
<a
class=
"anchor"
id=
"recommended-project-structure"
></a>
<a
class=
"anchor"
id=
"recommended-project-structure"
></a>
## Recommended project structure
## Recommended project structure
A Python project directory should, at the very least, have a structure that
A Python project directory should, at the very least, have a structure that
resembles the following:
resembles the following:
> ```
> ```
> myproject/
> myproject/
> mypackage/
> mypackage/
> __init__.py
> __init__.py
> mymodule.py
> mymodule.py
> README
> README
> LICENSE
> LICENSE
> requirements.txt
> requirements.txt
> setup.py
> setup.py
> ```
> ```
This example structure is in the
`example_project/`
sub-directory - have a
This example structure is in the
`example_project/`
sub-directory - have a
look through it if you like.
look through it if you like.
<a
class=
"anchor"
id=
"the-mypackage-directory"
></a>
<a
class=
"anchor"
id=
"the-mypackage-directory"
></a>
### The `mypackage/` directory
### The `mypackage/` directory
The first thing you should do is make sure that all of your python code is
The first thing you should do is make sure that all of your python code is
organised into a sensibly-named
organised into a sensibly-named
[
_
package
_
](
https://docs.python.org/3
.5
/tutorial/modules.html#packages
)
. This
[
*
package
*
](
https://docs.python.org/3/tutorial/modules.html#packages
)
. This
is important, because it greatly reduces the possibility of naming collisions
is important, because it greatly reduces the possibility of naming collisions
when people install your library alongside other libraries. Hands up those of
when people install your library alongside other libraries. Hands up those of
you who have ever written a file called
`utils.[py|m|c|cpp]`
!
you who have ever written a file called
`utils.[py|m|c|cpp]`
!
Check out the
`advanced_topics/02_modules_and_packages.ipynb`
practical for
Check out the
`advanced_topics/02_modules_and_packages.ipynb`
practical for
more details on packages in Python.
more details on packages in Python.
<a
class=
"anchor"
id=
"readme"
></a>
<a
class=
"anchor"
id=
"readme"
></a>
### `README`
### `README`
Every project should have a README file. This is simply a plain text file
Every project should have a README file. This is simply a plain text file
which describes your project and how to use it. It is common and acceptable
which describes your project and how to use it. It is common and acceptable
for a README file to be written in plain text,
for a README file to be written in plain text,
[
reStructuredText
](
http://www.sphinx-doc.org/en/stable/rest.html
)
[
reStructuredText
](
http://www.sphinx-doc.org/en/stable/rest.html
)
(
`README.rst`
), or
(
`README.rst`
), or
[
markdown
](
https://guides.github.com/features/mastering-markdown/
)
[
markdown
](
https://guides.github.com/features/mastering-markdown/
)
(
`README.md`
).
(
`README.md`
).
<a
class=
"anchor"
id=
"license"
></a>
<a
class=
"anchor"
id=
"license"
></a>
### `LICENSE`
### `LICENSE`
Having a LICENSE file makes it easy for people to understand the constraints
Having a LICENSE file makes it easy for people to understand the constraints
under which your code can be used.
under which your code can be used.
<a
class=
"anchor"
id=
"requirements-txt"
></a>
<a
class=
"anchor"
id=
"requirements-txt"
></a>
### `requirements.txt`
### `requirements.txt`
This file is not strictly necessary, but is very common in Python projects.
This file is not strictly necessary, but is very common in Python projects.
It contains a list of the Python-based dependencies of your project, in a
It contains a list of the Python-based dependencies of your project, in a
standardised syntax. You can specify the exact version, or range of versions,
standardised syntax. You can specify the exact version, or range of versions,
that your project requires. For example:
that your project requires. For example:
> six==1.*
> ```
> numpy==1.*
> six==1.*
> scipy>=0.18,<2
> numpy==1.*
> nibabel==2.*
> scipy>=0.18
> nibabel==2.*
> ```
If your project has optional dependencies, i.e. libraries which are not
If your project has optional dependencies, i.e. libraries which are not
critical but, if present, will allow your project to offer some extra
critical but, if present, will allow your project to offer some extra
features, you can list them in a separate requirements file called, for
features, you can list them in a separate requirements file called, for
example,
`requirements-extra.txt`
.
example,
`requirements-extra.txt`
.
Having all your dependencies listed in a file in this way makes it easy for
Having all your dependencies listed in a file in this way makes it easy for
others to install the dependencies needed by your project, simply by running:
others to install the dependencies needed by your project, simply by running:
> pip install -r requirements.txt
> ```
> pip install -r requirements.txt
> ```
<a
class=
"anchor"
id=
"setup-py"
></a>
<a
class=
"anchor"
id=
"setup-py"
></a>
### `setup.py`
### `setup.py`
This is the most important file (apart from your code, of course). Python
This is the most important file (apart from your code, of course). Python
projects are installed using
projects are installed using
[
`setuptools`
](
https://setuptools.readthedocs.io/en/latest/
)
, which is used
[
`setuptools`
](
https://setuptools.readthedocs.io/en/latest/
)
, which is used
internally during both the creation of, and installation of Python libraries.
internally during both the creation of, and installation of Python libraries.
The
`setup.py`
file in a Python project is akin to a
`Makefile`
in a C/C++
The
`setup.py`
file in a Python project is akin to a
`Makefile`
in a C/C++
project. But
`setup.py`
is also the location where you can define project
project. But
`setup.py`
is also the location where you can define project
metadata (e.g. name, author, URL, etc) in a standardised format and, if
metadata (e.g. name, author, URL, etc) in a standardised format and, if
necessary, customise aspects of the build process for your library.
necessary, customise aspects of the build process for your library.
You generally don't need to worry about, or interact with
`setuptools`
at all.
You generally don't need to worry about, or interact with
`setuptools`
at all.
With one exception -
`setup.py`
is a Python script, and its main job is to
With one exception -
`setup.py`
is a Python script, and its main job is to
call the
`setuptools.setup`
function, passing it information about your
call the
`setuptools.setup`
function, passing it information about your
project.
project.
The
`setup.py`
for our example project might look like this:
The
`setup.py`
for our example project might look like this:
> ```
> ```
> #!/usr/bin/env python
> #!/usr/bin/env python
>
>
> from setuptools import setup
> from setuptools import setup
> from setuptools import find_packages
> from setuptools import find_packages
>
>
> # Import version number from
> # Import version number from
> # the project package (see
> # the project package (see
> # the section on versioning).
> # the section on versioning).
> from mypackage import __version__
> from mypackage import __version__
>
>
> # Read in requirements from
> # Read in requirements from
> # the requirements.txt file.
> # the requirements.txt file.
> with open('requirements.txt', 'rt') as f:
> with open('requirements.txt', 'rt') as f:
> requirements = [l.strip() for l in f.readlines()]
> requirements = [l.strip() for l in f.readlines()]
>
>
> # Generate a list of all of the
> # Generate a list of all of the
> # packages that are in your project.
> # packages that are in your project.
> packages = find_packages()
> packages = find_packages()
>
>
> setup(
> setup(
>
>
> name='Example project',
> name='Example project',
> description='Example Python project for PyTreat',
> description='Example Python project for PyTreat',
> url='https://git.fmrib.ox.ac.uk/fsl/pytreat-
2018-
practicals/',
> url='https://git.fmrib.ox.ac.uk/fsl/pytreat-practicals
-2020
/',
> author='Paul McCarthy',
> author='Paul McCarthy',
> author_email='pauldmccarthy@gmail.com',
> author_email='pauldmccarthy@gmail.com',
> license='Apache License Version 2.0',
> license='Apache License Version 2.0',
>
>
> packages=packages,
> packages=packages,
>
>
> version=__version__,
> version=__version__,
>
>
> install_requires=requirements,
> install_requires=requirements,
>
>
> classifiers=[
> classifiers=[
> 'Development Status :: 3 - Alpha',
> 'Development Status :: 3 - Alpha',
> 'Intended Audience :: Developers',
> 'Intended Audience :: Developers',
> 'License :: OSI Approved :: Apache Software License',
> 'License :: OSI Approved :: Apache Software License',
> 'Programming Language :: Python :: 2.7',
> 'Programming Language :: Python :: 2.7',
> 'Programming Language :: Python :: 3.4',
> 'Programming Language :: Python :: 3.4',
> 'Programming Language :: Python :: 3.5',
> 'Programming Language :: Python :: 3.5',
> 'Programming Language :: Python :: 3.6',
> 'Programming Language :: Python :: 3.6',
> 'Topic :: Software Development :: Libraries :: Python Modules'],
> 'Topic :: Software Development :: Libraries :: Python Modules'],
> )
> )
> ```
> ```
The
`setup`
function gets passed all of your project's metadata, including its
The
`setup`
function gets passed all of your project's metadata, including its
version number, depedencies, and licensing information. The
`classifiers`
version number, depedencies, and licensing information. The
`classifiers`
argument should contain a list of
argument should contain a list of
[
classifiers
](
https://pypi.python.org/pypi?%3Aaction=list_classifiers
)
which
[
classifiers
](
https://pypi.python.org/pypi?%3Aaction=list_classifiers
)
which
are applicable to your project. Classifiers are purely for descriptive
are applicable to your project. Classifiers are purely for descriptive
purposes - they can be used to aid people in finding your project on
purposes - they can be used to aid people in finding your project on
[
`PyPI`
](
https://pypi.python.org/pypi
)
, if you release it there.
[
`PyPI`
](
https://pypi.python.org/pypi
)
, if you release it there.
See
See
[
here
](
https://packaging.python.org/tutorials/distributing-packages/#setup-args
)
[
here
](
https://packaging.python.org/tutorials/distributing-packages/#setup-args
)
for more details on
`setup.py`
and the
`setup`
function.
for more details on
`setup.py`
and the
`setup`
function.
<a
class=
"anchor"
id=
"appendix-tests"
></a>
<a
class=
"anchor"
id=
"appendix-tests"
></a>
## Appendix: Tests
## Appendix: Tests
There are no strict rules for where to put your tests (you have tests,
There are no strict rules for where to put your tests (you have tests,
right?). There are two main conventions:
right?). There are two main conventions:
You can store your test files
_
inside
_
your package directory:
You can store your test files
*
inside
*
your package directory:
> ```
> ```
> myproject/
> myproject/
> mypackage/
> mypackage/
> __init__.py
> __init__.py
> mymodule.py
> mymodule.py
> tests/
> tests/
> __init__.py
> __init__.py
> test_mymodule.py
> test_mymodule.py
> ```
> ```
Or, you can store your test files
*alongside*
your package directory:
Or, you can store your test files _alongside_ your package directory:
> ```
> ```
> myproject/
> myproject/
> mypackage/
> mypackage/
> __init__.py
> __init__.py
> mymodule.py
> mymodule.py
> tests/
> tests/
> test_mymodule.py
> test_mymodule.py
> ```
> ```
If you want your test code to be completely independent of your project's
If you want your test code to be completely independent of your project's
code, then go with the second option. However, if you would like your test
code, then go with the second option. However, if you would like your test
code to be distributed as part of your project (e.g. so that end users can run
code to be distributed as part of your project (e.g. so that end users can run
them), then the first option is probably the best.
them), then the first option is probably the best.
But in the end, the standard Python unit testing frameworks
But in the end, the standard Python unit testing frameworks
(
[
`pytest`
](
https://docs.pytest.org/en/latest/
)
and
(
[
`pytest`
](
https://docs.pytest.org/en/latest/
)
and
[
`nose`
](
http://nose2.readthedocs.io/en/latest/
)
) are pretty good at finding
[
`nose`
](
http://nose2.readthedocs.io/en/latest/
)
) are pretty good at finding
your test functions no matter where you've hidden them, so the choice is
your test functions no matter where you've hidden them, so the choice is
really up to you.
really up to you.
<a
class=
"anchor"
id=
"appendix-versioning"
></a>
<a
class=
"anchor"
id=
"appendix-versioning"
></a>
## Appendix: Versioning
## Appendix: Versioning
If you are intending to make your project available for public use (e.g. on
If you are intending to make your project available for public use (e.g. on
[
PyPI
](
https://pypi.python.org/pypi
)
and/or
[
PyPI
](
https://pypi.python.org/pypi
)
and/or
[
conda
](
https://anaconda.org/anaconda/repo
)
), it is
__
very important
__
to
[
conda
](
https://anaconda.org/anaconda/repo
)
), it is
**
very important
**
to
manage the version number of your project. If somebody decides to build their
manage the version number of your project. If somebody decides to build their
software on top of your project, they are not going to be very happy with you
software on top of your project, they are not going to be very happy with you
if you make substantial, API-breaking changes without changing your version
if you make substantial, API-breaking changes without changing your version
number in an appropriate manner.
number in an appropriate manner.
Python has
[
official standards
](
https://www.python.org/dev/peps/pep-0440/
)
on
Python has
[
official standards
](
https://www.python.org/dev/peps/pep-0440/
)
on
what constitutes a valid version number. These standards can be quite
what constitutes a valid version number. These standards can be quite
complicated but, in the vast majority of cases, a simple three-number
complicated but, in the vast majority of cases, a simple three-number
versioning scheme comprising
_
major
_
,
_
minor
_
, and
_
patch
_
release
versioning scheme comprising
*
major
*
,
*
minor
*
, and
*
patch
*
release
numbers should suffice. Such a version number has the form:
numbers should suffice. Such a version number has the form:
> major.minor.patch
> ```
> major.minor.patch
> ```
For example, a version number of
`1.3.2`
has a _major_ release of 1, _minor_
For example, a version number of
`1.3.2`
has a _major_ release of 1, _minor_
release of 3, and a _patch_ release of 2.
release of 3, and a _patch_ release of 2.
If you follow some simple and rational guidelines for versioning
If you follow some simple and rational guidelines for versioning
`your_project`
, then people who use your project can, for instance, specify
`your_project`
, then people who use your project can, for instance, specify
that they depend on
`your_project==1.*`
, and be sure that their code will work
that they depend on
`your_project==1.*`
, and be sure that their code will work
for
_
any
_
version of
`your_project`
with a major release of 1. Following these
for
*
any
*
version of
`your_project`
with a major release of 1. Following these
simple guidelines greatly improves software interoperability, and makes
simple guidelines greatly improves software interoperability, and makes
everybody (i.e. developers of other projects, and end users) much happier!
everybody (i.e. developers of other projects, and end users) much happier!
Many modern Python projects use some form of
[
_
semantic
Many modern Python projects use some form of
[
*
semantic
versioning
_
](
https://semver.org/
)
. Semantic versioning is simply a set of
versioning
*
](
https://semver.org/
)
. Semantic versioning is simply a set of
guidelines on how to manage your version number:
guidelines on how to manage your version number:
-
The
_
major
_
release number should be incremented whenever you introduce any
-
The
*
major
*
release number should be incremented whenever you introduce any
backwards-incompatible changes. In other words, if you change your code
backwards-incompatible changes. In other words, if you change your code
such that some other code which uses your code would break, you should
such that some other code which uses your code would break, you should
increment the major release number.
increment the major release number.
-
The
_
minor
_
release number should be incremented whenever you add any new
-
The
*
minor
*
release number should be incremented whenever you add any new
(backwards-compatible) features to your project.
(backwards-compatible) features to your project.
-
The
_
patch
_
release number should be incremented for backwards-compatible
-
The
*
patch
*
release number should be incremented for backwards-compatible
bug-fixes and other minor changes.
bug-fixes and other minor changes.
If you like to automate things,
If you like to automate things,
[
`bumpversion`
](
https://github.com/peritus/bumpversion
)
is a simple tool that
[
`bumpversion`
](
https://github.com/peritus/bumpversion
)
is a simple tool that
you can use to help manage your version number.
you can use to help manage your version number.
<a
class=
"anchor"
id=
"include-the-version-in-your-code"
></a>
<a
class=
"anchor"
id=
"include-the-version-in-your-code"
></a>
### Include the version in your code
### Include the version in your code
While the version of a library is ultimately defined in
`setup.py`
, it is
While the version of a library is ultimately defined in
`setup.py`
, it is
standard practice for a Python library to contain a version string called
standard practice for a Python library to contain a version string called
`__version__`
in the
`__init__.py`
file of the top-level package. For example,
`__version__`
in the
`__init__.py`
file of the top-level package. For example,
our
`example_project/mypackage/__init__.py`
file contains this line:
our
`example_project/mypackage/__init__.py`
file contains this line:
> __version__ = '0.1.0'
> ```
> __version__ = '0.1.0'
> ```
This makes a library's version number programmatically accessible and
This makes a library's version number programmatically accessible and
queryable.
queryable.
<a
class=
"anchor"
id=
"deprecate-dont-remove"
></a>
<a
class=
"anchor"
id=
"deprecate-dont-remove"
></a>
### Deprecate, don't remove!
### Deprecate, don't remove!
If you really want to change your API, but can't bring yourself to increment
If you really want to change your API, but can't bring yourself to increment
your major release number, consider
your major release number, consider
[
_
deprecating
_
](
https://en.wikipedia.org/wiki/Deprecation#Software_deprecation
)
[
*
deprecating
*
](
https://en.wikipedia.org/wiki/Deprecation#Software_deprecation
)
the old API, and postponing its removal until you are ready for a major
the old API, and postponing its removal until you are ready for a major
release. This will allow you to change your API, but retain
release. This will allow you to change your API, but retain
backwards-compatilbiity with the old API until it can safely be removed at the
backwards-compatilbiity with the old API until it can safely be removed at the
next major release.
next major release.
You can use the built-in
You can use the built-in
[
`warnings`
](
https://docs.python.org/3.5/library/exceptions.html#DeprecationWarning
)
[
`warnings`
](
https://docs.python.org/3.5/library/exceptions.html#DeprecationWarning
)
module to warn about uses of deprecated items. There are also some
module to warn about uses of deprecated items. There are also some
[
third-party libraries
](
https://github.com/briancurtin/deprecation
)
which make
[
third-party libraries
](
https://github.com/briancurtin/deprecation
)
which make
it easy to mark a function, method or class as being deprecated.
it easy to mark a function, method or class as being deprecated.
<a
class=
"anchor"
id=
"appendix-cookiecutter"
></a>
<a
class=
"anchor"
id=
"appendix-cookiecutter"
></a>
## Appendix: Cookiecutter
## Appendix: Cookiecutter
It is worth mentioning
It is worth mentioning
[
Cookiecutter
](
https://github.com/audreyr/cookiecutter
)
, a little utility
[
Cookiecutter
](
https://github.com/audreyr/cookiecutter
)
, a little utility
program which you can use to generate a skeleton file/directory structure for
program which you can use to generate a skeleton file/directory structure for
a new Python project.
a new Python project.
You need to give it a template (there are many available templates, including
You need to give it a template (there are many available templates, including
for projects in languages other than Python) - a couple of useful templates
for projects in languages other than Python) - a couple of useful templates
are the
[
minimal Python package
are the
[
minimal Python package
template
](
https://github.com/kragniz/cookiecutter-pypackage-minimal
)
, and the
template
](
https://github.com/kragniz/cookiecutter-pypackage-minimal
)
, and the
[
full Python package
[
full Python package
template
](
https://github.com/audreyr/cookiecutter-pypackage
)
(
although
the
template
](
https://github.com/audreyr/cookiecutter-pypackage
)
(
although
the
latter is probably overkill for most).
latter is probably overkill for most).
Here is how to create a skeleton project directory based off the minimal
Here is how to create a skeleton project directory based off the minimal
Python packagetemplate:
Python packagetemplate:
> ```
> ```
> pip install cookiecutter
> pip install cookiecutter
>
>
> # tell cookiecutter to create a directory
> # tell cookiecutter to create a directory
> # from the pypackage-minimal template
> # from the pypackage-minimal template
> cookiecutter https://github.com/kragniz/cookiecutter-pypackage-minimal.git
> cookiecutter https://github.com/kragniz/cookiecutter-pypackage-minimal.git
>
>
> # cookiecutter will then prompt you for
> # cookiecutter will then prompt you for
> # basic information (e.g. projectname,
> # basic information (e.g. projectname,
> # author name/email), and then create a
> # author name/email), and then create a
> # new directory containing the project
> # new directory containing the project
> # skeleton.
> # skeleton.
> ```
> ```
...
...
This diff is collapsed.
Click to expand it.
advanced_topics/09_structuring_projects.md
+
35
−
28
View file @
46f3ea65
...
@@ -13,15 +13,15 @@ directory.
...
@@ -13,15 +13,15 @@ directory.
*
[
Recommended project structure
](
#recommended-project-structure
)
*
[
Recommended project structure
](
#recommended-project-structure
)
*
[
The `mypackage/` directory
](
#the-mypackage-directory
)
*
[
The `mypackage/` directory
](
#the-mypackage-directory
)
*
[
`README`
](
#readme
)
*
[
`README`
](
#readme
)
*
[
`LICENSE`
](
#license
)
*
[
`LICENSE`
](
#license
)
*
[
`requirements.txt`
](
#requirements-txt
)
*
[
`requirements.txt`
](
#requirements-txt
)
*
[
`setup.py`
](
#setup-py
)
*
[
`setup.py`
](
#setup-py
)
*
[
Appendix: Tests
](
#appendix-tests
)
*
[
Appendix: Tests
](
#appendix-tests
)
*
[
Appendix: Versioning
](
#appendix-versioning
)
*
[
Appendix: Versioning
](
#appendix-versioning
)
*
[
Include the version in your code
](
#include-the-version-in-your-code
)
*
[
Include the version in your code
](
#include-the-version-in-your-code
)
*
[
Deprecate, don't remove!
](
#deprecate-dont-remove
)
*
[
Deprecate, don't remove!
](
#deprecate-dont-remove
)
*
[
Appendix: Cookiecutter
](
#appendix-cookiecutter
)
*
[
Appendix: Cookiecutter
](
#appendix-cookiecutter
)
...
@@ -59,7 +59,7 @@ look through it if you like.
...
@@ -59,7 +59,7 @@ look through it if you like.
The first thing you should do is make sure that all of your python code is
The first thing you should do is make sure that all of your python code is
organised into a sensibly-named
organised into a sensibly-named
[
_
package
_
](
https://docs.python.org/3
.5
/tutorial/modules.html#packages
)
. This
[
*
package
*
](
https://docs.python.org/3/tutorial/modules.html#packages
)
. This
is important, because it greatly reduces the possibility of naming collisions
is important, because it greatly reduces the possibility of naming collisions
when people install your library alongside other libraries. Hands up those of
when people install your library alongside other libraries. Hands up those of
you who have ever written a file called
`utils.[py|m|c|cpp]`
!
you who have ever written a file called
`utils.[py|m|c|cpp]`
!
...
@@ -100,10 +100,12 @@ standardised syntax. You can specify the exact version, or range of versions,
...
@@ -100,10 +100,12 @@ standardised syntax. You can specify the exact version, or range of versions,
that your project requires. For example:
that your project requires. For example:
> six==1.*
> ```
> numpy==1.*
> six==1.*
> scipy>=0.18,<2
> numpy==1.*
> nibabel==2.*
> scipy>=0.18
> nibabel==2.*
> ```
If your project has optional dependencies, i.e. libraries which are not
If your project has optional dependencies, i.e. libraries which are not
...
@@ -116,7 +118,9 @@ Having all your dependencies listed in a file in this way makes it easy for
...
@@ -116,7 +118,9 @@ Having all your dependencies listed in a file in this way makes it easy for
others to install the dependencies needed by your project, simply by running:
others to install the dependencies needed by your project, simply by running:
> pip install -r requirements.txt
> ```
> pip install -r requirements.txt
> ```
<a
class=
"anchor"
id=
"setup-py"
></a>
<a
class=
"anchor"
id=
"setup-py"
></a>
...
@@ -168,7 +172,7 @@ The `setup.py` for our example project might look like this:
...
@@ -168,7 +172,7 @@ The `setup.py` for our example project might look like this:
>
>
> name='Example project',
> name='Example project',
> description='Example Python project for PyTreat',
> description='Example Python project for PyTreat',
> url='https://git.fmrib.ox.ac.uk/fsl/pytreat-
2018-
practicals/',
> url='https://git.fmrib.ox.ac.uk/fsl/pytreat-practicals
-2020
/',
> author='Paul McCarthy',
> author='Paul McCarthy',
> author_email='pauldmccarthy@gmail.com',
> author_email='pauldmccarthy@gmail.com',
> license='Apache License Version 2.0',
> license='Apache License Version 2.0',
...
@@ -214,7 +218,7 @@ There are no strict rules for where to put your tests (you have tests,
...
@@ -214,7 +218,7 @@ There are no strict rules for where to put your tests (you have tests,
right?). There are two main conventions:
right?). There are two main conventions:
You can store your test files
_
inside
_
your package directory:
You can store your test files
*
inside
*
your package directory:
> ```
> ```
...
@@ -228,8 +232,7 @@ You can store your test files _inside_ your package directory:
...
@@ -228,8 +232,7 @@ You can store your test files _inside_ your package directory:
> ```
> ```
Or, you can store your test files
*alongside*
your package directory:
Or, you can store your test files _alongside_ your package directory:
> ```
> ```
...
@@ -261,7 +264,7 @@ really up to you.
...
@@ -261,7 +264,7 @@ really up to you.
If you are intending to make your project available for public use (e.g. on
If you are intending to make your project available for public use (e.g. on
[
PyPI
](
https://pypi.python.org/pypi
)
and/or
[
PyPI
](
https://pypi.python.org/pypi
)
and/or
[
conda
](
https://anaconda.org/anaconda/repo
)
), it is
__
very important
__
to
[
conda
](
https://anaconda.org/anaconda/repo
)
), it is
**
very important
**
to
manage the version number of your project. If somebody decides to build their
manage the version number of your project. If somebody decides to build their
software on top of your project, they are not going to be very happy with you
software on top of your project, they are not going to be very happy with you
if you make substantial, API-breaking changes without changing your version
if you make substantial, API-breaking changes without changing your version
...
@@ -271,11 +274,13 @@ number in an appropriate manner.
...
@@ -271,11 +274,13 @@ number in an appropriate manner.
Python has
[
official standards
](
https://www.python.org/dev/peps/pep-0440/
)
on
Python has
[
official standards
](
https://www.python.org/dev/peps/pep-0440/
)
on
what constitutes a valid version number. These standards can be quite
what constitutes a valid version number. These standards can be quite
complicated but, in the vast majority of cases, a simple three-number
complicated but, in the vast majority of cases, a simple three-number
versioning scheme comprising
_
major
_
,
_
minor
_
, and
_
patch
_
release
versioning scheme comprising
*
major
*
,
*
minor
*
, and
*
patch
*
release
numbers should suffice. Such a version number has the form:
numbers should suffice. Such a version number has the form:
> major.minor.patch
> ```
> major.minor.patch
> ```
For example, a version number of
`1.3.2`
has a _major_ release of 1, _minor_
For example, a version number of
`1.3.2`
has a _major_ release of 1, _minor_
...
@@ -285,25 +290,25 @@ release of 3, and a _patch_ release of 2.
...
@@ -285,25 +290,25 @@ release of 3, and a _patch_ release of 2.
If you follow some simple and rational guidelines for versioning
If you follow some simple and rational guidelines for versioning
`your_project`
, then people who use your project can, for instance, specify
`your_project`
, then people who use your project can, for instance, specify
that they depend on
`your_project==1.*`
, and be sure that their code will work
that they depend on
`your_project==1.*`
, and be sure that their code will work
for
_
any
_
version of
`your_project`
with a major release of 1. Following these
for
*
any
*
version of
`your_project`
with a major release of 1. Following these
simple guidelines greatly improves software interoperability, and makes
simple guidelines greatly improves software interoperability, and makes
everybody (i.e. developers of other projects, and end users) much happier!
everybody (i.e. developers of other projects, and end users) much happier!
Many modern Python projects use some form of
[
_
semantic
Many modern Python projects use some form of
[
*
semantic
versioning
_
](
https://semver.org/
)
. Semantic versioning is simply a set of
versioning
*
](
https://semver.org/
)
. Semantic versioning is simply a set of
guidelines on how to manage your version number:
guidelines on how to manage your version number:
-
The
_
major
_
release number should be incremented whenever you introduce any
-
The
*
major
*
release number should be incremented whenever you introduce any
backwards-incompatible changes. In other words, if you change your code
backwards-incompatible changes. In other words, if you change your code
such that some other code which uses your code would break, you should
such that some other code which uses your code would break, you should
increment the major release number.
increment the major release number.
-
The
_
minor
_
release number should be incremented whenever you add any new
-
The
*
minor
*
release number should be incremented whenever you add any new
(backwards-compatible) features to your project.
(backwards-compatible) features to your project.
-
The
_
patch
_
release number should be incremented for backwards-compatible
-
The
*
patch
*
release number should be incremented for backwards-compatible
bug-fixes and other minor changes.
bug-fixes and other minor changes.
...
@@ -322,7 +327,9 @@ standard practice for a Python library to contain a version string called
...
@@ -322,7 +327,9 @@ standard practice for a Python library to contain a version string called
our
`example_project/mypackage/__init__.py`
file contains this line:
our
`example_project/mypackage/__init__.py`
file contains this line:
> __version__ = '0.1.0'
> ```
> __version__ = '0.1.0'
> ```
This makes a library's version number programmatically accessible and
This makes a library's version number programmatically accessible and
...
@@ -335,7 +342,7 @@ queryable.
...
@@ -335,7 +342,7 @@ queryable.
If you really want to change your API, but can't bring yourself to increment
If you really want to change your API, but can't bring yourself to increment
your major release number, consider
your major release number, consider
[
_
deprecating
_
](
https://en.wikipedia.org/wiki/Deprecation#Software_deprecation
)
[
*
deprecating
*
](
https://en.wikipedia.org/wiki/Deprecation#Software_deprecation
)
the old API, and postponing its removal until you are ready for a major
the old API, and postponing its removal until you are ready for a major
release. This will allow you to change your API, but retain
release. This will allow you to change your API, but retain
backwards-compatilbiity with the old API until it can safely be removed at the
backwards-compatilbiity with the old API until it can safely be removed at the
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment