Skip to content
Snippets Groups Projects
Commit 76fc03da authored by Paul McCarthy's avatar Paul McCarthy :mountain_bicyclist:
Browse files

Merge branch 'master' into 'master' 

New practical on context managers, little fixes elsewhere

See merge request fsl/pytreat-2018-practicals!20
parents 09a1c0de a81b2753
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 19 additions and 19 deletions
README.md
+ 1
− 1
View file @ 76fc03da
...@@ -32,7 +32,7 @@ jupyter: ...@@ -32,7 +32,7 @@ jupyter:
# location! # location!
# #
# One further complication - once you have become root, $FSLDIR may not be set, # One further complication - once you have become root, $FSLDIR may not be set,
# so either set it as we have ione below, or make sure that it is set, before # so either set it as we have done below, or make sure that it is set, before
# proceeding. # proceeding.
sudo su sudo su
export FSLDIR=/usr/local/fsl export FSLDIR=/usr/local/fsl
......
advanced_topics/modules_and_packages.ipynb → advanced_topics/02_modules_and_packages.ipynb
+ 9
− 9
View file @ 76fc03da
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Modules and packages # Modules and packages
Python gives you a lot of flexibility in how you organise your code. If you Python gives you a lot of flexibility in how you organise your code. If you
want, you can write a Python program just as you would write a Bash script. want, you can write a Python program just as you would write a Bash script.
You don't _have_ to use functions, classes, modules or packages if you don't You don't _have_ to use functions, classes, modules or packages if you don't
want to, or if the script's task does not require them. want to, or if the script's task does not require them.
But when your code starts to grow beyond what can reasonably be defined in a But when your code starts to grow beyond what can reasonably be defined in a
single file, you will (hopefully) want to start arranging it in a more single file, you will (hopefully) want to start arranging it in a more
understandable manner. understandable manner.
For this practical we have prepared a handful of example files - you can find For this practical we have prepared a handful of example files - you can find
them alongside this notebook file, in a directory called them alongside this notebook file, in a directory called
`modules_and_packages/`. `02_modules_and_packages/`.
## Contents ## Contents
* [What is a module?](#what-is-a-module) * [What is a module?](#what-is-a-module)
* [Importing modules](#importing-modules) * [Importing modules](#importing-modules)
* [Importing specific items from a module](#importing-specific-items-from-a-module) * [Importing specific items from a module](#importing-specific-items-from-a-module)
* [Importing everything from a module](#importing-everything-from-a-module) * [Importing everything from a module](#importing-everything-from-a-module)
* [Module aliases](#module-aliases) * [Module aliases](#module-aliases)
* [What happens when I import a module?](#what-happens-when-i-import-a-module) * [What happens when I import a module?](#what-happens-when-i-import-a-module)
* [How can I make my own modules importable?](#how-can-i-make-my-own-modules-importable) * [How can I make my own modules importable?](#how-can-i-make-my-own-modules-importable)
* [Modules versus scripts](#modules-versus-scripts) * [Modules versus scripts](#modules-versus-scripts)
* [What is a package?](#what-is-a-package) * [What is a package?](#what-is-a-package)
* [`__init__.py`](#init-py) * [`__init__.py`](#init-py)
* [Useful references](#useful-references) * [Useful references](#useful-references)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import os import os
os.chdir('modules_and_packages') os.chdir('02_modules_and_packages')
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="what-is-a-module"></a> <a class="anchor" id="what-is-a-module"></a>
## What is a module? ## What is a module?
Any file ending with `.py` is considered to be a module in Python. Take a look Any file ending with `.py` is considered to be a module in Python. Take a look
at `modules_and_packages/numfuncs.py` - either open it in your editor, or run at `02_modules_and_packages/numfuncs.py` - either open it in your editor, or
this code block: run this code block:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
with open('numfuncs.py', 'rt') as f: with open('numfuncs.py', 'rt') as f:
for line in f: for line in f:
print(line.rstrip()) print(line.rstrip())
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
This is a perfectly valid Python module, although not a particularly useful This is a perfectly valid Python module, although not a particularly useful
one. It contains an attribute called `PI`, and a function `add`. one. It contains an attribute called `PI`, and a function `add`.
<a class="anchor" id="importing-modules"></a> <a class="anchor" id="importing-modules"></a>
## Importing modules ## Importing modules
Before we can use our module, we must `import` it. Importing a module in Before we can use our module, we must `import` it. Importing a module in
Python will make its contents available in the local scope. We can import the Python will make its contents available in the local scope. We can import the
contents of `numfuncs` like so: contents of `numfuncs` like so:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import numfuncs import numfuncs
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
This imports `numfuncs` into the local scope - everything defined in the This imports `numfuncs` into the local scope - everything defined in the
`numfuncs` module can be accessed by prefixing it with `numfuncs.`: `numfuncs` module can be accessed by prefixing it with `numfuncs.`:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
print('PI:', numfuncs.PI) print('PI:', numfuncs.PI)
print(numfuncs.add(1, 50)) print(numfuncs.add(1, 50))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
There are a couple of other ways to import items from a module... There are a couple of other ways to import items from a module...
<a class="anchor" id="importing-specific-items-from-a-module"></a> <a class="anchor" id="importing-specific-items-from-a-module"></a>
### Importing specific items from a module ### Importing specific items from a module
If you only want to use one, or a few items from a module, you can import just If you only want to use one, or a few items from a module, you can import just
those items - a reference to just those items will be created in the local those items - a reference to just those items will be created in the local
scope: scope:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
from numfuncs import add from numfuncs import add
print(add(1, 3)) print(add(1, 3))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="importing-everything-from-a-module"></a> <a class="anchor" id="importing-everything-from-a-module"></a>
### Importing everything from a module ### Importing everything from a module
It is possible to import _everything_ that is defined in a module like so: It is possible to import _everything_ that is defined in a module like so:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
from numfuncs import * from numfuncs import *
print('PI: ', PI) print('PI: ', PI)
print(add(1, 5)) print(add(1, 5))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
__PLEASE DON'T DO THIS!__ Because every time you do, somewhere in the world, a __PLEASE DON'T DO THIS!__ Because every time you do, somewhere in the world, a
software developer will will spontaneously stub his/her toe, and start crying. software developer will will spontaneously stub his/her toe, and start crying.
Using this approach can make more complicated programs very difficult to read, Using this approach can make more complicated programs very difficult to read,
because it is not possible to determine the origin of the functions and because it is not possible to determine the origin of the functions and
attributes that are being used. attributes that are being used.
And naming collisions are inevitable when importing multiple modules in this And naming collisions are inevitable when importing multiple modules in this
way, making it very difficult for somebody else to figure out what your code way, making it very difficult for somebody else to figure out what your code
is doing: is doing:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
from numfuncs import * from numfuncs import *
from strfuncs import * from strfuncs import *
print(add(1, 5)) print(add(1, 5))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Instead, it is better to give modules a name when you import them. While this Instead, it is better to give modules a name when you import them. While this
requires you to type more code, the benefits of doing this far outweigh the requires you to type more code, the benefits of doing this far outweigh the
hassle of typing a few extra characters - it becomes much easier to read and hassle of typing a few extra characters - it becomes much easier to read and
trace through code when the functions you use are accessed through a namespace trace through code when the functions you use are accessed through a namespace
for each module: for each module:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import numfuncs import numfuncs
import strfuncs import strfuncs
print('number add: ', numfuncs.add(1, 2)) print('number add: ', numfuncs.add(1, 2))
print('string add: ', strfuncs.add(1, 2)) print('string add: ', strfuncs.add(1, 2))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="module-aliases"></a> <a class="anchor" id="module-aliases"></a>
### Module aliases ### Module aliases
And Python allows you to define an _alias_ for a module when you import it, And Python allows you to define an _alias_ for a module when you import it,
so you don't necessarily need to type out the full module name each time so you don't necessarily need to type out the full module name each time
you want to access something inside: you want to access something inside:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import numfuncs as nf import numfuncs as nf
import strfuncs as sf import strfuncs as sf
print('number add: ', nf.add(1, 2)) print('number add: ', nf.add(1, 2))
print('string add: ', sf.add(1, 2)) print('string add: ', sf.add(1, 2))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
You have already seen this in the earlier practicals - here are a few You have already seen this in the earlier practicals - here are a few
aliases which have become a de-facto standard for commonly used Python aliases which have become a de-facto standard for commonly used Python
modules: modules:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import os.path as op import os.path as op
import numpy as np import numpy as np
import nibabel as nib import nibabel as nib
import matplotlib as mpl import matplotlib as mpl
import matplotlib.pyplot as plt import matplotlib.pyplot as plt
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="what-happens-when-i-import-a-module"></a> <a class="anchor" id="what-happens-when-i-import-a-module"></a>
### What happens when I import a module? ### What happens when I import a module?
When you `import` a module, the contents of the module file are literally When you `import` a module, the contents of the module file are literally
executed by the Python runtime, exactly the same as if you had typed its executed by the Python runtime, exactly the same as if you had typed its
contents into `ipython`. Any attributes, functions, or classes which are contents into `ipython`. Any attributes, functions, or classes which are
defined in the module will be bundled up into an object that represents the defined in the module will be bundled up into an object that represents the
module, and through which you can access the module's contents. module, and through which you can access the module's contents.
When we typed `import numfuncs` in the examples above, the following events When we typed `import numfuncs` in the examples above, the following events
occurred: occurred:
1. Python created a `module` object to represent the module. 1. Python created a `module` object to represent the module.
2. The `numfuncs.py` file was read and executed, and all of the items defined 2. The `numfuncs.py` file was read and executed, and all of the items defined
inside `numfuncs.py` (i.e. the `PI` attribute and the `add` function) were inside `numfuncs.py` (i.e. the `PI` attribute and the `add` function) were
added to the `module` object. added to the `module` object.
3. A local variable called `numfuncs`, pointing to the `module` object, 3. A local variable called `numfuncs`, pointing to the `module` object,
was added to the local scope. was added to the local scope.
Because module files are literally executed on import, any statements in the Because module files are literally executed on import, any statements in the
module file which are not encapsulated inside a class or function will be module file which are not encapsulated inside a class or function will be
executed. As an example, take a look at the file `sideeffects.py`. Let's executed. As an example, take a look at the file `sideeffects.py`. Let's
import it and see what happens: import it and see what happens:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import sideeffects import sideeffects
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Ok, hopefully that wasn't too much of a surprise. Something which may be less Ok, hopefully that wasn't too much of a surprise. Something which may be less
intuitive, however, is that a module's contents will only be executed on the intuitive, however, is that a module's contents will only be executed on the
_first_ time that it is imported. After the first import, Python caches the _first_ time that it is imported. After the first import, Python caches the
module's contents (all loaded modules are accessible through module's contents (all loaded modules are accessible through
[`sys.modules`](https://docs.python.org/3.5/library/sys.html#sys.modules)). On [`sys.modules`](https://docs.python.org/3.5/library/sys.html#sys.modules)). On
subsequent imports, the cached version of the module is returned: subsequent imports, the cached version of the module is returned:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import sideeffects import sideeffects
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="how-can-i-make-my-own-modules-importable"></a> <a class="anchor" id="how-can-i-make-my-own-modules-importable"></a>
### How can I make my own modules importable? ### How can I make my own modules importable?
When you `import` a module, Python searches for it in the following locations, When you `import` a module, Python searches for it in the following locations,
in the following order: in the following order:
1. Built-in modules (e.g. `os`, `sys`, etc.). 1. Built-in modules (e.g. `os`, `sys`, etc.).
2. In the current directory or, if a script has been executed, in the directory 2. In the current directory or, if a script has been executed, in the directory
containing that script. containing that script.
3. In directories listed in the `$PYTHONPATH` environment variable. 3. In directories listed in the `$PYTHONPATH` environment variable.
4. In installed third-party libraries (e.g. `numpy`). 4. In installed third-party libraries (e.g. `numpy`).
If you are experimenting or developing your program, the quickest and easiest If you are experimenting or developing your program, the quickest and easiest
way to make your module(s) importable is to add their containing directory to way to make your module(s) importable is to add their containing directory to
the `PYTHONPATH`. But if you are developing a larger piece of software, you the `PYTHONPATH`. But if you are developing a larger piece of software, you
should probably organise your modules into _packages_, which are [described should probably organise your modules into _packages_, which are [described
below](#what-is-a-package). below](#what-is-a-package).
<a class="anchor" id="modules-versus-scripts"></a> <a class="anchor" id="modules-versus-scripts"></a>
## Modules versus scripts ## Modules versus scripts
You now know that Python treats all files ending in `.py` as importable You now know that Python treats all files ending in `.py` as importable
modules. But all files ending in `.py` can also be treated as scripts. In modules. But all files ending in `.py` can also be treated as scripts. In
fact, there no difference between a _module_ and a _script_ - any `.py` file fact, there no difference between a _module_ and a _script_ - any `.py` file
can be executed as a script, or imported as a module, or both. can be executed as a script, or imported as a module, or both.
Have a look at the file `modules_and_packages/module_and_script.py`: Have a look at the file `02_modules_and_packages/module_and_script.py`:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
with open('module_and_script.py', 'rt') as f: with open('module_and_script.py', 'rt') as f:
for line in f: for line in f:
print(line.rstrip()) print(line.rstrip())
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
This file contains two functions `mul` and `main`. The This file contains two functions `mul` and `main`. The
`if __name__ == '__main__':` clause at the bottom is a standard trick in Python `if __name__ == '__main__':` clause at the bottom is a standard trick in Python
that allows you to add code to a file that is _only executed when the module is that allows you to add code to a file that is _only executed when the module is
called as a script_. Try it in a terminal now: called as a script_. Try it in a terminal now:
> `python modules_and_packages/module_and_script.py` > `python 02_modules_and_packages/module_and_script.py`
But if we `import` this module from another file, or from an interactive But if we `import` this module from another file, or from an interactive
session, the code within the `if __name__ == '__main__':` clause will not be session, the code within the `if __name__ == '__main__':` clause will not be
executed, and we can access its functions just like any other module that we executed, and we can access its functions just like any other module that we
import. import.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import module_and_script as mas import module_and_script as mas
a = 1.5 a = 1.5
b = 3 b = 3
print('mul({}, {}): {}'.format(a, b, mas.mul(a, b))) print('mul({}, {}): {}'.format(a, b, mas.mul(a, b)))
print('calling main...') print('calling main...')
mas.main([str(a), str(b)]) mas.main([str(a), str(b)])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="what-is-a-package"></a> <a class="anchor" id="what-is-a-package"></a>
## What is a package? ## What is a package?
You now know how to split your Python code up into separate files You now know how to split your Python code up into separate files
(a.k.a. _modules_). When your code grows beyond a handful of files, you may (a.k.a. _modules_). When your code grows beyond a handful of files, you may
wish for more fine-grained control over the namespaces in which your modules wish for more fine-grained control over the namespaces in which your modules
live. Python has another feature which allows you to organise your modules live. Python has another feature which allows you to organise your modules
into _packages_. into _packages_.
A package in Python is simply a directory which: A package in Python is simply a directory which:
* Contains a special file called `__init__.py` * Contains a special file called `__init__.py`
* May contain one or more module files (any other files ending in `*.py`) * May contain one or more module files (any other files ending in `*.py`)
* May contain other package directories. * May contain other package directories.
For example, the [FSLeyes](https://git.fmrib.ox.ac.uk/fsl/fsleyes/fsleyes) For example, the [FSLeyes](https://git.fmrib.ox.ac.uk/fsl/fsleyes/fsleyes)
code is organised into packages and sub-packages as follows (abridged): code is organised into packages and sub-packages as follows (abridged):
> ``` > ```
> fsleyes/ > fsleyes/
> __init__.py > __init__.py
> main.py > main.py
> frame.py > frame.py
> views/ > views/
> __init__.py > __init__.py
> orthopanel.py > orthopanel.py
> lightboxpanel.py > lightboxpanel.py
> controls/ > controls/
> __init__.py > __init__.py
> locationpanel.py > locationpanel.py
> overlaylistpanel.py > overlaylistpanel.py
> ``` > ```
Within a package structure, we will typically still import modules directly, Within a package structure, we will typically still import modules directly,
via their full path within the package: via their full path within the package:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import fsleyes.main as fmain import fsleyes.main as fmain
fmain.fsleyes_main() fmain.fsleyes_main()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="init-py"></a> <a class="anchor" id="init-py"></a>
### `__init__.py` ### `__init__.py`
Every Python package must have an `__init__.py` file. In many cases, this will Every Python package must have an `__init__.py` file. In many cases, this will
actually be an empty file, and you don't need to worry about it any more, apart actually be an empty file, and you don't need to worry about it any more, apart
from knowing that it is needed. But you can use `__init__.py` to perform some from knowing that it is needed. But you can use `__init__.py` to perform some
package-specific initialisation, and/or to customise the package's namespace. package-specific initialisation, and/or to customise the package's namespace.
As an example, take a look the `modules_and_packages/fsleyes/__init__.py` file As an example, take a look the `02_modules_and_packages/fsleyes/__init__.py`
in our mock FSLeyes package. We have imported the `fsleyes_main` function from file in our mock FSLeyes package. We have imported the `fsleyes_main` function
the `fsleyes.main` module, making it available at the package level. So from the `fsleyes.main` module, making it available at the package level. So
instead of importing the `fsleyes.main` module, we could instead just import instead of importing the `fsleyes.main` module, we could instead just import
the `fsleyes` package: the `fsleyes` package:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import fsleyes import fsleyes
fsleyes.fsleyes_main() fsleyes.fsleyes_main()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="useful-references"></a> <a class="anchor" id="useful-references"></a>
## Useful references ## Useful references
* [Modules and packages in Python](https://docs.python.org/3.5/tutorial/modules.html) * [Modules and packages in Python](https://docs.python.org/3.5/tutorial/modules.html)
* [Using `__init__.py`](http://mikegrouchy.com/blog/2012/05/be-pythonic-__init__py.html) * [Using `__init__.py`](http://mikegrouchy.com/blog/2012/05/be-pythonic-__init__py.html)
......
advanced_topics/modules_and_packages.md → advanced_topics/02_modules_and_packages.md
+ 9
− 9
View file @ 76fc03da
...@@ -14,7 +14,7 @@ understandable manner. ...@@ -14,7 +14,7 @@ understandable manner.
For this practical we have prepared a handful of example files - you can find For this practical we have prepared a handful of example files - you can find
them alongside this notebook file, in a directory called them alongside this notebook file, in a directory called
`modules_and_packages/`. `02_modules_and_packages/`.
## Contents ## Contents
...@@ -33,7 +33,7 @@ them alongside this notebook file, in a directory called ...@@ -33,7 +33,7 @@ them alongside this notebook file, in a directory called
``` ```
import os import os
os.chdir('modules_and_packages') os.chdir('02_modules_and_packages')
``` ```
<a class="anchor" id="what-is-a-module"></a> <a class="anchor" id="what-is-a-module"></a>
...@@ -41,8 +41,8 @@ os.chdir('modules_and_packages') ...@@ -41,8 +41,8 @@ os.chdir('modules_and_packages')
Any file ending with `.py` is considered to be a module in Python. Take a look Any file ending with `.py` is considered to be a module in Python. Take a look
at `modules_and_packages/numfuncs.py` - either open it in your editor, or run at `02_modules_and_packages/numfuncs.py` - either open it in your editor, or
this code block: run this code block:
``` ```
...@@ -256,7 +256,7 @@ fact, there no difference between a _module_ and a _script_ - any `.py` file ...@@ -256,7 +256,7 @@ fact, there no difference between a _module_ and a _script_ - any `.py` file
can be executed as a script, or imported as a module, or both. can be executed as a script, or imported as a module, or both.
Have a look at the file `modules_and_packages/module_and_script.py`: Have a look at the file `02_modules_and_packages/module_and_script.py`:
``` ```
...@@ -272,7 +272,7 @@ that allows you to add code to a file that is _only executed when the module is ...@@ -272,7 +272,7 @@ that allows you to add code to a file that is _only executed when the module is
called as a script_. Try it in a terminal now: called as a script_. Try it in a terminal now:
> `python modules_and_packages/module_and_script.py` > `python 02_modules_and_packages/module_and_script.py`
But if we `import` this module from another file, or from an interactive But if we `import` this module from another file, or from an interactive
...@@ -351,9 +351,9 @@ from knowing that it is needed. But you can use `__init__.py` to perform some ...@@ -351,9 +351,9 @@ from knowing that it is needed. But you can use `__init__.py` to perform some
package-specific initialisation, and/or to customise the package's namespace. package-specific initialisation, and/or to customise the package's namespace.
As an example, take a look the `modules_and_packages/fsleyes/__init__.py` file As an example, take a look the `02_modules_and_packages/fsleyes/__init__.py`
in our mock FSLeyes package. We have imported the `fsleyes_main` function from file in our mock FSLeyes package. We have imported the `fsleyes_main` function
the `fsleyes.main` module, making it available at the package level. So from the `fsleyes.main` module, making it available at the package level. So
instead of importing the `fsleyes.main` module, we could instead just import instead of importing the `fsleyes.main` module, we could instead just import
the `fsleyes` package: the `fsleyes` package:
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment