diff --git a/README.md b/README.md
index c3ca8ace3c12aaf0c3598b2f4c18cf3d3999be3b..3c7872620930c4ea5849149f70a0b7ef4fd9370c 100644
--- a/README.md
+++ b/README.md
@@ -32,7 +32,7 @@ jupyter:
# location!
#
# One further complication - once you have become root, $FSLDIR may not be set,
-# so either set it as we have ione below, or make sure that it is set, before
+# so either set it as we have done below, or make sure that it is set, before
# proceeding.
sudo su
export FSLDIR=/usr/local/fsl
diff --git a/advanced_topics/function_inputs_and_outputs.ipynb b/advanced_topics/01_function_inputs_and_outputs.ipynb
similarity index 100%
rename from advanced_topics/function_inputs_and_outputs.ipynb
rename to advanced_topics/01_function_inputs_and_outputs.ipynb
diff --git a/advanced_topics/function_inputs_and_outputs.md b/advanced_topics/01_function_inputs_and_outputs.md
similarity index 100%
rename from advanced_topics/function_inputs_and_outputs.md
rename to advanced_topics/01_function_inputs_and_outputs.md
diff --git a/advanced_topics/modules_and_packages.ipynb b/advanced_topics/02_modules_and_packages.ipynb
similarity index 96%
rename from advanced_topics/modules_and_packages.ipynb
rename to advanced_topics/02_modules_and_packages.ipynb
index 94994e265b660d37af3ddc9074bf3ebf5aab40ab..b49f0e5a1d7ce99ed66f00bc1b21c62fd5af88ed 100644
--- a/advanced_topics/modules_and_packages.ipynb
+++ b/advanced_topics/02_modules_and_packages.ipynb
@@ -20,7 +20,7 @@
"\n",
"For this practical we have prepared a handful of example files - you can find\n",
"them alongside this notebook file, in a directory called\n",
- "`modules_and_packages/`.\n",
+ "`02_modules_and_packages/`.\n",
"\n",
"\n",
"## Contents\n",
@@ -45,7 +45,7 @@
"outputs": [],
"source": [
"import os\n",
- "os.chdir('modules_and_packages')"
+ "os.chdir('02_modules_and_packages')"
]
},
{
@@ -57,8 +57,8 @@
"\n",
"\n",
"Any file ending with `.py` is considered to be a module in Python. Take a look\n",
- "at `modules_and_packages/numfuncs.py` - either open it in your editor, or run\n",
- "this code block:"
+ "at `02_modules_and_packages/numfuncs.py` - either open it in your editor, or\n",
+ "run this code block:"
]
},
{
@@ -363,7 +363,7 @@
"can be executed as a script, or imported as a module, or both.\n",
"\n",
"\n",
- "Have a look at the file `modules_and_packages/module_and_script.py`:"
+ "Have a look at the file `02_modules_and_packages/module_and_script.py`:"
]
},
{
@@ -387,7 +387,7 @@
"called as a script_. Try it in a terminal now:\n",
"\n",
"\n",
- "> `python modules_and_packages/module_and_script.py`\n",
+ "> `python 02_modules_and_packages/module_and_script.py`\n",
"\n",
"\n",
"But if we `import` this module from another file, or from an interactive\n",
@@ -483,9 +483,9 @@
"package-specific initialisation, and/or to customise the package's namespace.\n",
"\n",
"\n",
- "As an example, take a look the `modules_and_packages/fsleyes/__init__.py` file\n",
- "in our mock FSLeyes package. We have imported the `fsleyes_main` function from\n",
- "the `fsleyes.main` module, making it available at the package level. So\n",
+ "As an example, take a look the `02_modules_and_packages/fsleyes/__init__.py`\n",
+ "file in our mock FSLeyes package. We have imported the `fsleyes_main` function\n",
+ "from the `fsleyes.main` module, making it available at the package level. So\n",
"instead of importing the `fsleyes.main` module, we could instead just import\n",
"the `fsleyes` package:"
]
diff --git a/advanced_topics/modules_and_packages.md b/advanced_topics/02_modules_and_packages.md
similarity index 95%
rename from advanced_topics/modules_and_packages.md
rename to advanced_topics/02_modules_and_packages.md
index 16dfca6a785cd358333f5567856b7da5b2161f99..1fdccb84963ead3deb0305324be0bb793a3b10ff 100644
--- a/advanced_topics/modules_and_packages.md
+++ b/advanced_topics/02_modules_and_packages.md
@@ -14,7 +14,7 @@ understandable manner.
For this practical we have prepared a handful of example files - you can find
them alongside this notebook file, in a directory called
-`modules_and_packages/`.
+`02_modules_and_packages/`.
## Contents
@@ -33,7 +33,7 @@ them alongside this notebook file, in a directory called
```
import os
-os.chdir('modules_and_packages')
+os.chdir('02_modules_and_packages')
```
<a class="anchor" id="what-is-a-module"></a>
@@ -41,8 +41,8 @@ os.chdir('modules_and_packages')
Any file ending with `.py` is considered to be a module in Python. Take a look
-at `modules_and_packages/numfuncs.py` - either open it in your editor, or run
-this code block:
+at `02_modules_and_packages/numfuncs.py` - either open it in your editor, or
+run this code block:
```
@@ -256,7 +256,7 @@ fact, there no difference between a _module_ and a _script_ - any `.py` file
can be executed as a script, or imported as a module, or both.
-Have a look at the file `modules_and_packages/module_and_script.py`:
+Have a look at the file `02_modules_and_packages/module_and_script.py`:
```
@@ -272,7 +272,7 @@ that allows you to add code to a file that is _only executed when the module is
called as a script_. Try it in a terminal now:
-> `python modules_and_packages/module_and_script.py`
+> `python 02_modules_and_packages/module_and_script.py`
But if we `import` this module from another file, or from an interactive
@@ -351,9 +351,9 @@ from knowing that it is needed. But you can use `__init__.py` to perform some
package-specific initialisation, and/or to customise the package's namespace.
-As an example, take a look the `modules_and_packages/fsleyes/__init__.py` file
-in our mock FSLeyes package. We have imported the `fsleyes_main` function from
-the `fsleyes.main` module, making it available at the package level. So
+As an example, take a look the `02_modules_and_packages/fsleyes/__init__.py`
+file in our mock FSLeyes package. We have imported the `fsleyes_main` function
+from the `fsleyes.main` module, making it available at the package level. So
instead of importing the `fsleyes.main` module, we could instead just import
the `fsleyes` package:
diff --git a/advanced_topics/modules_and_packages/fsleyes/__init__.py b/advanced_topics/02_modules_and_packages/fsleyes/__init__.py
similarity index 100%
rename from advanced_topics/modules_and_packages/fsleyes/__init__.py
rename to advanced_topics/02_modules_and_packages/fsleyes/__init__.py
diff --git a/advanced_topics/modules_and_packages/fsleyes/controls/__init__.py b/advanced_topics/02_modules_and_packages/fsleyes/controls/__init__.py
similarity index 100%
rename from advanced_topics/modules_and_packages/fsleyes/controls/__init__.py
rename to advanced_topics/02_modules_and_packages/fsleyes/controls/__init__.py
diff --git a/advanced_topics/modules_and_packages/fsleyes/controls/locationpanel.py b/advanced_topics/02_modules_and_packages/fsleyes/controls/locationpanel.py
similarity index 100%
rename from advanced_topics/modules_and_packages/fsleyes/controls/locationpanel.py
rename to advanced_topics/02_modules_and_packages/fsleyes/controls/locationpanel.py
diff --git a/advanced_topics/modules_and_packages/fsleyes/controls/overlaylistpanel.py b/advanced_topics/02_modules_and_packages/fsleyes/controls/overlaylistpanel.py
similarity index 100%
rename from advanced_topics/modules_and_packages/fsleyes/controls/overlaylistpanel.py
rename to advanced_topics/02_modules_and_packages/fsleyes/controls/overlaylistpanel.py
diff --git a/advanced_topics/modules_and_packages/fsleyes/frame.py b/advanced_topics/02_modules_and_packages/fsleyes/frame.py
similarity index 100%
rename from advanced_topics/modules_and_packages/fsleyes/frame.py
rename to advanced_topics/02_modules_and_packages/fsleyes/frame.py
diff --git a/advanced_topics/modules_and_packages/fsleyes/main.py b/advanced_topics/02_modules_and_packages/fsleyes/main.py
similarity index 100%
rename from advanced_topics/modules_and_packages/fsleyes/main.py
rename to advanced_topics/02_modules_and_packages/fsleyes/main.py
diff --git a/advanced_topics/modules_and_packages/fsleyes/views/__init__.py b/advanced_topics/02_modules_and_packages/fsleyes/views/__init__.py
similarity index 100%
rename from advanced_topics/modules_and_packages/fsleyes/views/__init__.py
rename to advanced_topics/02_modules_and_packages/fsleyes/views/__init__.py
diff --git a/advanced_topics/modules_and_packages/fsleyes/views/lightboxpanel.py b/advanced_topics/02_modules_and_packages/fsleyes/views/lightboxpanel.py
similarity index 100%
rename from advanced_topics/modules_and_packages/fsleyes/views/lightboxpanel.py
rename to advanced_topics/02_modules_and_packages/fsleyes/views/lightboxpanel.py
diff --git a/advanced_topics/modules_and_packages/fsleyes/views/orthopanel.py b/advanced_topics/02_modules_and_packages/fsleyes/views/orthopanel.py
similarity index 100%
rename from advanced_topics/modules_and_packages/fsleyes/views/orthopanel.py
rename to advanced_topics/02_modules_and_packages/fsleyes/views/orthopanel.py
diff --git a/advanced_topics/modules_and_packages/module_and_script.py b/advanced_topics/02_modules_and_packages/module_and_script.py
similarity index 100%
rename from advanced_topics/modules_and_packages/module_and_script.py
rename to advanced_topics/02_modules_and_packages/module_and_script.py
diff --git a/advanced_topics/modules_and_packages/numfuncs.py b/advanced_topics/02_modules_and_packages/numfuncs.py
similarity index 100%
rename from advanced_topics/modules_and_packages/numfuncs.py
rename to advanced_topics/02_modules_and_packages/numfuncs.py
diff --git a/advanced_topics/modules_and_packages/sideeffects.py b/advanced_topics/02_modules_and_packages/sideeffects.py
similarity index 100%
rename from advanced_topics/modules_and_packages/sideeffects.py
rename to advanced_topics/02_modules_and_packages/sideeffects.py
diff --git a/advanced_topics/modules_and_packages/strfuncs.py b/advanced_topics/02_modules_and_packages/strfuncs.py
similarity index 100%
rename from advanced_topics/modules_and_packages/strfuncs.py
rename to advanced_topics/02_modules_and_packages/strfuncs.py
diff --git a/advanced_topics/object_oriented_programming.ipynb b/advanced_topics/03_object_oriented_programming.ipynb
similarity index 100%
rename from advanced_topics/object_oriented_programming.ipynb
rename to advanced_topics/03_object_oriented_programming.ipynb
diff --git a/advanced_topics/object_oriented_programming.md b/advanced_topics/03_object_oriented_programming.md
similarity index 100%
rename from advanced_topics/object_oriented_programming.md
rename to advanced_topics/03_object_oriented_programming.md
diff --git a/advanced_topics/operator_overloading.ipynb b/advanced_topics/04_operator_overloading.ipynb
similarity index 97%
rename from advanced_topics/operator_overloading.ipynb
rename to advanced_topics/04_operator_overloading.ipynb
index 1b9ab7f1830010834183417e0799b2e22d744273..567fd7e6ade801b930979254bc95954d723f34d9 100644
--- a/advanced_topics/operator_overloading.ipynb
+++ b/advanced_topics/04_operator_overloading.ipynb
@@ -238,10 +238,14 @@
"- Exclusive or (`^`): `__xor__`\n",
"\n",
"\n",
- "Take a look at the [official\n",
+ "When an operator is applied to operands of different types, a set of fall-back\n",
+ "rules are followed depending on the set of methods implemented on the\n",
+ "operands. For example, in the expression `a + b`, if `a.__add__` is not\n",
+ "implemented, but but `b.__radd__` is implemented, then the latter will be\n",
+ "called. Take a look at the [official\n",
"documentation](https://docs.python.org/3.5/reference/datamodel.html#emulating-numeric-types)\n",
- "for a full list of the arithmetic and logical operators that your classes can\n",
- "support.\n",
+ "for further details, including a full list of the arithmetic and logical\n",
+ "operators that your classes can support.\n",
"\n",
"\n",
"<a class=\"anchor\" id=\"equality-and-comparison-operators\"></a>\n",
@@ -332,7 +336,7 @@
"l3 = Label(3, 'Temporal', ( 0, 0, 255))\n",
"\n",
"print('{} > {}: {}'.format(l1, l2, l1 > l2))\n",
- "print('{} < {}: {}'.format(l1, l3, l1 < l3))\n",
+ "print('{} < {}: {}'.format(l1, l3, l1 <= l3))\n",
"print('{} != {}: {}'.format(l2, l3, l2 != l3))\n",
"print(sorted((l3, l1, l2)))"
]
@@ -715,7 +719,7 @@
"\n",
"print('v: ', v)\n",
"print('xyz: ', v.xyz)\n",
- "print('yz: ', v.zy)\n",
+ "print('zy: ', v.zy)\n",
"print('xx: ', v.xx)\n",
"\n",
"v.xz = 10, 30\n",
diff --git a/advanced_topics/operator_overloading.md b/advanced_topics/04_operator_overloading.md
similarity index 96%
rename from advanced_topics/operator_overloading.md
rename to advanced_topics/04_operator_overloading.md
index 142368fcee5807335115158025c9679362721ec1..3446401f9fa65815b69b2617ba76edcf85747d9c 100644
--- a/advanced_topics/operator_overloading.md
+++ b/advanced_topics/04_operator_overloading.md
@@ -168,10 +168,14 @@ appropriate method, for example:
- Exclusive or (`^`): `__xor__`
-Take a look at the [official
+When an operator is applied to operands of different types, a set of fall-back
+rules are followed depending on the set of methods implemented on the
+operands. For example, in the expression `a + b`, if `a.__add__` is not
+implemented, but but `b.__radd__` is implemented, then the latter will be
+called. Take a look at the [official
documentation](https://docs.python.org/3.5/reference/datamodel.html#emulating-numeric-types)
-for a full list of the arithmetic and logical operators that your classes can
-support.
+for further details, including a full list of the arithmetic and logical
+operators that your classes can support.
<a class="anchor" id="equality-and-comparison-operators"></a>
@@ -241,7 +245,7 @@ l2 = Label(2, 'Occipital', ( 0, 255, 0))
l3 = Label(3, 'Temporal', ( 0, 0, 255))
print('{} > {}: {}'.format(l1, l2, l1 > l2))
-print('{} < {}: {}'.format(l1, l3, l1 < l3))
+print('{} < {}: {}'.format(l1, l3, l1 <= l3))
print('{} != {}: {}'.format(l2, l3, l2 != l3))
print(sorted((l3, l1, l2)))
```
@@ -545,7 +549,7 @@ v = Vector((1, 2, 3))
print('v: ', v)
print('xyz: ', v.xyz)
-print('yz: ', v.zy)
+print('zy: ', v.zy)
print('xx: ', v.xx)
v.xz = 10, 30
diff --git a/advanced_topics/05_context_managers.ipynb b/advanced_topics/05_context_managers.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..ad3c231ca860ce526be06337f65cc74a1e09920d
--- /dev/null
+++ b/advanced_topics/05_context_managers.ipynb
@@ -0,0 +1,820 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Context managers\n",
+ "\n",
+ "\n",
+ "The recommended way to open a file in Python is via the `with` statement:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "with open('05_context_managers.md', 'rt') as f:\n",
+ " firstlines = f.readlines()[:4]\n",
+ " firstlines = [l.strip() for l in firstlines]\n",
+ " print('\\n'.join(firstlines))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This is because the `with` statement ensures that the file will be closed\n",
+ "automatically, even if an error occurs inside the `with` statement.\n",
+ "\n",
+ "\n",
+ "The `with` statement is obviously hiding some internal details from us. But\n",
+ "these internals are in fact quite straightforward, and are known as [_context\n",
+ "managers_](https://docs.python.org/3.5/reference/datamodel.html#context-managers).\n",
+ "\n",
+ "\n",
+ "* [Anatomy of a context manager](#anatomy-of-a-context-manager)\n",
+ " * [Why not just use `try ... finally`?](#why-not-just-use-try-finally)\n",
+ "* [Uses for context managers](#uses-for-context-managers)\n",
+ " * [Handling errors in `__exit__`](#handling-errors-in-exit)\n",
+ " * [Suppressing errors with `__exit__`](#suppressing-errors-with-exit)\n",
+ "* [Nesting context managers](#nesting-context-managers)\n",
+ "* [Functions as context managers](#functions-as-context-managers)\n",
+ "* [Methods as context managers](#methods-as-context-managers)\n",
+ "* [Useful references](#useful-references)\n",
+ "\n",
+ "\n",
+ "<a class=\"anchor\" id=\"anatomy-of-a-context-manager\"></a>\n",
+ "## Anatomy of a context manager\n",
+ "\n",
+ "\n",
+ "A _context manager_ is simply an object which has two specially named methods\n",
+ "`__enter__` and `__exit__`. Any object which has these methods can be used in\n",
+ "a `with` statement.\n",
+ "\n",
+ "\n",
+ "Let's define a context manager class that we can play with:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "class MyContextManager(object):\n",
+ " def __enter__(self):\n",
+ " print('In enter')\n",
+ " def __exit__(self, *args):\n",
+ " print('In exit')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now, what happens when we use `MyContextManager` in a `with` statement?"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "with MyContextManager():\n",
+ " print('In with block')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "So the `__enter__` method is called before the statements in the `with` block,\n",
+ "and the `__exit__` method is called afterwards.\n",
+ "\n",
+ "\n",
+ "Context managers are that simple. What makes them really useful though, is\n",
+ "that the `__exit__` method will be called even if the code in the `with` block\n",
+ "raises an error. The error will be held, and only raised after the `__exit__`\n",
+ "method has finished:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "with MyContextManager():\n",
+ " print('In with block')\n",
+ " assert 1 == 0"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This means that we can use context managers to perform any sort of clean up or\n",
+ "finalisation logic that we always want to have executed.\n",
+ "\n",
+ "\n",
+ "<a class=\"anchor\" id=\"why-not-just-use-try-finally\"></a>\n",
+ "### Why not just use `try ... finally`?\n",
+ "\n",
+ "\n",
+ "Context managers do not provide anything that cannot be accomplished in other\n",
+ "ways. For example, we could accomplish very similar behaviour using\n",
+ "[`try` - `finally` logic](https://docs.python.org/3.5/tutorial/errors.html#handling-exceptions) -\n",
+ "the statements in the `finally` clause will *always* be executed, whether an\n",
+ "error is raised or not:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "print('Before try block')\n",
+ "try:\n",
+ " print('In try block')\n",
+ " assert 1 == 0\n",
+ "finally:\n",
+ " print('In finally block')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "But context managers have the advantage that you can implement your clean-up\n",
+ "logic in one place, and re-use it as many times as you want.\n",
+ "\n",
+ "\n",
+ "<a class=\"anchor\" id=\"uses-for-context-managers\"></a>\n",
+ "## Uses for context managers\n",
+ "\n",
+ "\n",
+ "We have already talked about how context managers can be used to perform any\n",
+ "task which requires some initialistion and/or clean-up logic. As an example,\n",
+ "here is a context manager which creates a temporary directory, and then makes\n",
+ "sure that it is deleted afterwards."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "import shutil\n",
+ "import tempfile\n",
+ "\n",
+ "class TempDir(object):\n",
+ "\n",
+ " def __enter__(self):\n",
+ "\n",
+ " self.tempDir = tempfile.mkdtemp()\n",
+ " self.prevDir = os.getcwd()\n",
+ "\n",
+ " print('Changing to temp dir: {}'.format(self.tempDir))\n",
+ " print('Previous directory: {}'.format(self.prevDir))\n",
+ "\n",
+ " os.chdir(self.tempDir)\n",
+ "\n",
+ " def __exit__(self, *args):\n",
+ "\n",
+ " print('Changing back to: {}'.format(self.prevDir))\n",
+ " print('Removing temp dir: {}'.format(self.tempDir))\n",
+ "\n",
+ " os .chdir( self.prevDir)\n",
+ " shutil.rmtree(self.tempDir)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now imagine that we have a function which loads data from a file, and performs\n",
+ "some calculation on it:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "\n",
+ "def complexAlgorithm(infile):\n",
+ " data = np.loadtxt(infile)\n",
+ " return data.mean()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We could use the `TempDir` context manager to write a test case for this\n",
+ "function, and not have to worry about cleaning up the test data:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "with TempDir():\n",
+ " print('Testing complex algorithm')\n",
+ "\n",
+ " data = np.random.random((100, 100))\n",
+ " np.savetxt('data.txt', data)\n",
+ "\n",
+ " result = complexAlgorithm('data.txt')\n",
+ "\n",
+ " assert result > 0.1 and result < 0.9\n",
+ " print('Test passed (result: {})'.format(result))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a class=\"anchor\" id=\"handling-errors-in-exit\"></a>\n",
+ "### Handling errors in `__exit__`\n",
+ "\n",
+ "\n",
+ "By now you must be [panicking](https://youtu.be/cSU_5MgtDc8?t=9) about why I\n",
+ "haven't mentioned those conspicuous `*args` that get passed to the`__exit__`\n",
+ "method. It turns out that a context manager's [`__exit__`\n",
+ "method](https://docs.python.org/3.5/reference/datamodel.html#object.__exit__)\n",
+ "is always passed three arguments.\n",
+ "\n",
+ "\n",
+ "Let's adjust our `MyContextManager` class a little so we can see what these\n",
+ "arguments are for:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "class MyContextManager(object):\n",
+ " def __enter__(self):\n",
+ " print('In enter')\n",
+ "\n",
+ " def __exit__(self, arg1, arg2, arg3):\n",
+ " print('In exit')\n",
+ " print(' arg1: ', arg1)\n",
+ " print(' arg2: ', arg2)\n",
+ " print(' arg3: ', arg3)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "If the code inside the `with` statement does not raise an error, these three\n",
+ "arguments will all be `None`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "with MyContextManager():\n",
+ " print('In with block')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "However, if the code inside the `with` statement raises an error, things look\n",
+ "a little different:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "with MyContextManager():\n",
+ " print('In with block')\n",
+ " raise ValueError('Oh no!')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "So when an error occurs, the `__exit__` method is passed the following:\n",
+ "\n",
+ "- The [`Exception`](https://docs.python.org/3.5/tutorial/errors.html)\n",
+ " type that was raised.\n",
+ "- The `Exception` instance that was raised.\n",
+ "- A [`traceback`](https://docs.python.org/3.5/library/traceback.html) object\n",
+ " which can be used to get more information about the exception (e.g. line\n",
+ " number).\n",
+ "\n",
+ "\n",
+ "<a class=\"anchor\" id=\"suppressing-errors-with-exit\"></a>\n",
+ "### Suppressing errors with `__exit__`\n",
+ "\n",
+ "\n",
+ "The `__exit__` method is also capable of suppressing errors - if it returns a\n",
+ "value of `True`, then any error that was raised will be ignored. For example,\n",
+ "we could write a context manager which ignores any assertion errors, but\n",
+ "allows other errors to halt execution as normal:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "class MyContextManager(object):\n",
+ " def __enter__(self):\n",
+ " print('In enter')\n",
+ "\n",
+ " def __exit__(self, arg1, arg2, arg3):\n",
+ " print('In exit')\n",
+ " if issubclass(arg1, AssertionError):\n",
+ " return True\n",
+ " print(' arg1: ', arg1)\n",
+ " print(' arg2: ', arg2)\n",
+ " print(' arg3: ', arg3)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "> Note that if a function or method does not explicitly return a value, its\n",
+ "> return value is `None` (which would evaluate to `False` when converted to a\n",
+ "> `bool`). Also note that we are using the built-in\n",
+ "> [`issubclass`](https://docs.python.org/3.5/library/functions.html#issubclass)\n",
+ "> function, which allows us to test the type of a class.\n",
+ "\n",
+ "\n",
+ "Now, when we use `MyContextManager`, any assertion errors are suppressed,\n",
+ "whereas other errors will be raised as normal:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "with MyContextManager():\n",
+ " assert 1 == 0\n",
+ "\n",
+ "print('Continuing execution!')\n",
+ "\n",
+ "with MyContextManager():\n",
+ " raise ValueError('Oh no!')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a class=\"anchor\" id=\"nesting-context-managers\"></a>\n",
+ "## Nesting context managers\n",
+ "\n",
+ "\n",
+ "It is possible to nest `with` statements:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "with open('05_context_managers.md', 'rt') as inf:\n",
+ " with TempDir():\n",
+ " with open('05_context_managers.md.copy', 'wt') as outf:\n",
+ " outf.write(inf.read())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You can also use multiple context managers in a single `with` statement:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "with open('05_context_managers.md', 'rt') as inf, \\\n",
+ " TempDir(), \\\n",
+ " open('05_context_managers.md.copy', 'wt') as outf:\n",
+ " outf.write(inf.read())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a class=\"anchor\" id=\"functions-as-context-managers\"></a>\n",
+ "## Functions as context managers\n",
+ "\n",
+ "\n",
+ "In fact, there is another way to create context managers in Python. The\n",
+ "built-in [`contextlib`\n",
+ "module](https://docs.python.org/3.5/library/contextlib.html#contextlib.contextmanager)\n",
+ "has a decorator called `@contextmanager`, which allows us to turn __any\n",
+ "function__ into a context manager. The only requirement is that the function\n",
+ "must have a `yield` statement<sup>1</sup>. So we could rewrite our `TempDir`\n",
+ "class from above as a function:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "import shutil\n",
+ "import tempfile\n",
+ "import contextlib\n",
+ "\n",
+ "@contextlib.contextmanager\n",
+ "def tempdir():\n",
+ " tdir = tempfile.mkdtemp()\n",
+ " prevdir = os.getcwd()\n",
+ " try:\n",
+ "\n",
+ " os.chdir(tdir)\n",
+ " yield tdir\n",
+ "\n",
+ " finally:\n",
+ " os.chdir(prevdir)\n",
+ " shutil.rmtree(tdir)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This new `tempdir` function is used in exactly the same way as our `TempDir`\n",
+ "class:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "print('In directory: {}'.format(os.getcwd()))\n",
+ "\n",
+ "with tempdir() as tmp:\n",
+ " print('Now in directory: {}'.format(os.getcwd()))\n",
+ "\n",
+ "print('Back in directory: {}'.format(os.getcwd()))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The `yield tdir` statement in our `tempdir` function causes the `tdir` value\n",
+ "to be passed to the `with` statement, so in the line `with tempdir() as tmp`,\n",
+ "the variable `tmp` will be given the value `tdir`.\n",
+ "\n",
+ "\n",
+ "> <sup>1</sup> The `yield` keyword is used in _generator functions_.\n",
+ "> Functions which are used with the `@contextmanager` decorator must be\n",
+ "> generator functions which yield exactly one value.\n",
+ "> [Generators](https://www.python.org/dev/peps/pep-0289/) and [generator\n",
+ "> functions](https://docs.python.org/3.5/glossary.html#term-generator) are\n",
+ "> beyond the scope of this practical.\n",
+ "\n",
+ "\n",
+ "<a class=\"anchor\" id=\"methods-as-context-managers\"></a>\n",
+ "## Methods as context managers\n",
+ "\n",
+ "\n",
+ "Since it is possible to write a function which is a context manager, it is of\n",
+ "course also possible to write a _method_ which is a context manager. Let's\n",
+ "play with another example. We have a `Notifier` class which can be used to\n",
+ "notify interested listeners when an event occurs. Listeners can be registered\n",
+ "for notification via the `register` method:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from collections import OrderedDict\n",
+ "\n",
+ "class Notifier(object):\n",
+ " def __init__(self):\n",
+ " super().__init__()\n",
+ " self.listeners = OrderedDict()\n",
+ "\n",
+ " def register(self, name, func):\n",
+ " self.listeners[name] = func\n",
+ "\n",
+ " def notify(self):\n",
+ " for listener in self.listeners.values():\n",
+ " listener()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now, let's build a little plotting application. First of all, we have a `Line`\n",
+ "class, which represents a line plot. The `Line` class is a sub-class of\n",
+ "`Notifier`, so whenever its display properties (`colour`, `width`, or `name`)\n",
+ "change, it emits a notification, and whatever is drawing it can refresh the\n",
+ "display:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "\n",
+ "class Line(Notifier):\n",
+ "\n",
+ " def __init__(self, data):\n",
+ " super().__init__()\n",
+ " self.__data = data\n",
+ " self.__colour = '#000000'\n",
+ " self.__width = 1\n",
+ " self.__name = 'line'\n",
+ "\n",
+ " @property\n",
+ " def xdata(self):\n",
+ " return np.arange(len(self.__data))\n",
+ "\n",
+ " @property\n",
+ " def ydata(self):\n",
+ " return np.copy(self.__data)\n",
+ "\n",
+ " @property\n",
+ " def colour(self):\n",
+ " return self.__colour\n",
+ "\n",
+ " @colour.setter\n",
+ " def colour(self, newColour):\n",
+ " self.__colour = newColour\n",
+ " print('Line: colour changed: {}'.format(newColour))\n",
+ " self.notify()\n",
+ "\n",
+ " @property\n",
+ " def width(self):\n",
+ " return self.__width\n",
+ "\n",
+ " @width.setter\n",
+ " def width(self, newWidth):\n",
+ " self.__width = newWidth\n",
+ " print('Line: width changed: {}'.format(newWidth))\n",
+ " self.notify()\n",
+ "\n",
+ " @property\n",
+ " def name(self):\n",
+ " return self.__name\n",
+ "\n",
+ " @name.setter\n",
+ " def name(self, newName):\n",
+ " self.__name = newName\n",
+ " print('Line: name changed: {}'.format(newName))\n",
+ " self.notify()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now let's write a `Plotter` class, which can plot one or more `Line`\n",
+ "instances:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import matplotlib.pyplot as plt\n",
+ "\n",
+ "class Plotter(object):\n",
+ " def __init__(self, axis):\n",
+ " self.__axis = axis\n",
+ " self.__lines = []\n",
+ "\n",
+ " def addData(self, data):\n",
+ " line = Line(data)\n",
+ " self.__lines.append(line)\n",
+ " line.register('plot', self.lineChanged)\n",
+ " self.draw()\n",
+ " return line\n",
+ "\n",
+ " def lineChanged(self):\n",
+ " self.draw()\n",
+ "\n",
+ " def draw(self):\n",
+ " print('Plotter: redrawing plot')\n",
+ "\n",
+ " ax = self.__axis\n",
+ " ax.clear()\n",
+ " for line in self.__lines:\n",
+ " ax.plot(line.xdata,\n",
+ " line.ydata,\n",
+ " color=line.colour,\n",
+ " linewidth=line.width,\n",
+ " label=line.name)\n",
+ " ax.legend()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's create a `Plotter` object, and add a couple of lines to it (note that\n",
+ "the `matplotlib` plot will open in a separate window):"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# this line is only necessary when\n",
+ "# working in jupyer notebook/ipython\n",
+ "%matplotlib\n",
+ "\n",
+ "fig = plt.figure()\n",
+ "ax = fig.add_subplot(111)\n",
+ "plotter = Plotter(ax)\n",
+ "l1 = plotter.addData(np.sin(np.linspace(0, 6 * np.pi, 50)))\n",
+ "l2 = plotter.addData(np.cos(np.linspace(0, 6 * np.pi, 50)))\n",
+ "\n",
+ "fig.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now, when we change the properties of our `Line` instances, the plot will be\n",
+ "automatically updated:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "l1.colour = '#ff0000'\n",
+ "l2.colour = '#00ff00'\n",
+ "l1.width = 2\n",
+ "l2.width = 2\n",
+ "l1.name = 'sine'\n",
+ "l2.name = 'cosine'"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Pretty cool! However, this seems very inefficient - every time we change the\n",
+ "properties of a `Line`, the `Plotter` will refresh the plot. If we were\n",
+ "plotting large amounts of data, this would be unacceptable, as plotting would\n",
+ "simply take too long.\n",
+ "\n",
+ "\n",
+ "Wouldn't it be nice if we were able to perform batch-updates of `Line`\n",
+ "properties, and only refresh the plot when we are done? Let's add an extra\n",
+ "method to the `Plotter` class:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import contextlib\n",
+ "\n",
+ "class Plotter(object):\n",
+ " def __init__(self, axis):\n",
+ " self.__axis = axis\n",
+ " self.__lines = []\n",
+ " self.__holdUpdates = False\n",
+ "\n",
+ " def addData(self, data):\n",
+ " line = Line(data)\n",
+ " self.__lines.append(line)\n",
+ " line.register('plot', self.lineChanged)\n",
+ "\n",
+ " if not self.__holdUpdates:\n",
+ " self.draw()\n",
+ " return line\n",
+ "\n",
+ " def lineChanged(self):\n",
+ " if not self.__holdUpdates:\n",
+ " self.draw()\n",
+ "\n",
+ " def draw(self):\n",
+ " print('Plotter: redrawing plot')\n",
+ "\n",
+ " ax = self.__axis\n",
+ " ax.clear()\n",
+ " for line in self.__lines:\n",
+ " ax.plot(line.xdata,\n",
+ " line.ydata,\n",
+ " color=line.colour,\n",
+ " linewidth=line.width,\n",
+ " label=line.name)\n",
+ " ax.legend()\n",
+ "\n",
+ " @contextlib.contextmanager\n",
+ " def holdUpdates(self):\n",
+ " self.__holdUpdates = True\n",
+ " try:\n",
+ " yield\n",
+ " self.draw()\n",
+ " finally:\n",
+ " self.__holdUpdates = False"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This new `holdUpdates` method allows us to temporarily suppress notifications\n",
+ "from all `Line` instances. So now, we can update many `Line` properties\n",
+ "without performing any redundant redraws:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fig = plt.figure()\n",
+ "ax = fig.add_subplot(111)\n",
+ "plotter = Plotter(ax)\n",
+ "\n",
+ "plt.show()\n",
+ "\n",
+ "with plotter.holdUpdates():\n",
+ " l1 = plotter.addData(np.sin(np.linspace(0, 6 * np.pi, 50)))\n",
+ " l2 = plotter.addData(np.cos(np.linspace(0, 6 * np.pi, 50)))\n",
+ " l1.colour = '#0000ff'\n",
+ " l2.colour = '#ffff00'\n",
+ " l1.width = 1\n",
+ " l2.width = 1\n",
+ " l1.name = '$sin(x)$'\n",
+ " l2.name = '$cos(x)$'"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a class=\"anchor\" id=\"useful-references\"></a>\n",
+ "## Useful references\n",
+ "\n",
+ "\n",
+ "* [Context manager classes](https://docs.python.org/3.5/reference/datamodel.html#context-managers)\n",
+ "* The [`contextlib` module](https://docs.python.org/3.5/library/contextlib.html)"
+ ]
+ }
+ ],
+ "metadata": {},
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/advanced_topics/05_context_managers.md b/advanced_topics/05_context_managers.md
new file mode 100644
index 0000000000000000000000000000000000000000..2229471e09769cb25159ea203f4e598527b33b8a
--- /dev/null
+++ b/advanced_topics/05_context_managers.md
@@ -0,0 +1,613 @@
+# Context managers
+
+
+The recommended way to open a file in Python is via the `with` statement:
+
+
+```
+with open('05_context_managers.md', 'rt') as f:
+ firstlines = f.readlines()[:4]
+ firstlines = [l.strip() for l in firstlines]
+ print('\n'.join(firstlines))
+```
+
+
+This is because the `with` statement ensures that the file will be closed
+automatically, even if an error occurs inside the `with` statement.
+
+
+The `with` statement is obviously hiding some internal details from us. But
+these internals are in fact quite straightforward, and are known as [_context
+managers_](https://docs.python.org/3.5/reference/datamodel.html#context-managers).
+
+
+* [Anatomy of a context manager](#anatomy-of-a-context-manager)
+ * [Why not just use `try ... finally`?](#why-not-just-use-try-finally)
+* [Uses for context managers](#uses-for-context-managers)
+ * [Handling errors in `__exit__`](#handling-errors-in-exit)
+ * [Suppressing errors with `__exit__`](#suppressing-errors-with-exit)
+* [Nesting context managers](#nesting-context-managers)
+* [Functions as context managers](#functions-as-context-managers)
+* [Methods as context managers](#methods-as-context-managers)
+* [Useful references](#useful-references)
+
+
+<a class="anchor" id="anatomy-of-a-context-manager"></a>
+## Anatomy of a context manager
+
+
+A _context manager_ is simply an object which has two specially named methods
+`__enter__` and `__exit__`. Any object which has these methods can be used in
+a `with` statement.
+
+
+Let's define a context manager class that we can play with:
+
+
+```
+class MyContextManager(object):
+ def __enter__(self):
+ print('In enter')
+ def __exit__(self, *args):
+ print('In exit')
+```
+
+
+Now, what happens when we use `MyContextManager` in a `with` statement?
+
+
+```
+with MyContextManager():
+ print('In with block')
+```
+
+
+So the `__enter__` method is called before the statements in the `with` block,
+and the `__exit__` method is called afterwards.
+
+
+Context managers are that simple. What makes them really useful though, is
+that the `__exit__` method will be called even if the code in the `with` block
+raises an error. The error will be held, and only raised after the `__exit__`
+method has finished:
+
+
+```
+with MyContextManager():
+ print('In with block')
+ assert 1 == 0
+```
+
+
+This means that we can use context managers to perform any sort of clean up or
+finalisation logic that we always want to have executed.
+
+
+<a class="anchor" id="why-not-just-use-try-finally"></a>
+### Why not just use `try ... finally`?
+
+
+Context managers do not provide anything that cannot be accomplished in other
+ways. For example, we could accomplish very similar behaviour using
+[`try` - `finally` logic](https://docs.python.org/3.5/tutorial/errors.html#handling-exceptions) -
+the statements in the `finally` clause will *always* be executed, whether an
+error is raised or not:
+
+
+```
+print('Before try block')
+try:
+ print('In try block')
+ assert 1 == 0
+finally:
+ print('In finally block')
+```
+
+
+But context managers have the advantage that you can implement your clean-up
+logic in one place, and re-use it as many times as you want.
+
+
+<a class="anchor" id="uses-for-context-managers"></a>
+## Uses for context managers
+
+
+We have already talked about how context managers can be used to perform any
+task which requires some initialistion and/or clean-up logic. As an example,
+here is a context manager which creates a temporary directory, and then makes
+sure that it is deleted afterwards.
+
+
+```
+import os
+import shutil
+import tempfile
+
+class TempDir(object):
+
+ def __enter__(self):
+
+ self.tempDir = tempfile.mkdtemp()
+ self.prevDir = os.getcwd()
+
+ print('Changing to temp dir: {}'.format(self.tempDir))
+ print('Previous directory: {}'.format(self.prevDir))
+
+ os.chdir(self.tempDir)
+
+ def __exit__(self, *args):
+
+ print('Changing back to: {}'.format(self.prevDir))
+ print('Removing temp dir: {}'.format(self.tempDir))
+
+ os .chdir( self.prevDir)
+ shutil.rmtree(self.tempDir)
+```
+
+
+Now imagine that we have a function which loads data from a file, and performs
+some calculation on it:
+
+
+```
+import numpy as np
+
+def complexAlgorithm(infile):
+ data = np.loadtxt(infile)
+ return data.mean()
+```
+
+
+We could use the `TempDir` context manager to write a test case for this
+function, and not have to worry about cleaning up the test data:
+
+
+```
+with TempDir():
+ print('Testing complex algorithm')
+
+ data = np.random.random((100, 100))
+ np.savetxt('data.txt', data)
+
+ result = complexAlgorithm('data.txt')
+
+ assert result > 0.1 and result < 0.9
+ print('Test passed (result: {})'.format(result))
+```
+
+
+<a class="anchor" id="handling-errors-in-exit"></a>
+### Handling errors in `__exit__`
+
+
+By now you must be [panicking](https://youtu.be/cSU_5MgtDc8?t=9) about why I
+haven't mentioned those conspicuous `*args` that get passed to the`__exit__`
+method. It turns out that a context manager's [`__exit__`
+method](https://docs.python.org/3.5/reference/datamodel.html#object.__exit__)
+is always passed three arguments.
+
+
+Let's adjust our `MyContextManager` class a little so we can see what these
+arguments are for:
+
+
+```
+class MyContextManager(object):
+ def __enter__(self):
+ print('In enter')
+
+ def __exit__(self, arg1, arg2, arg3):
+ print('In exit')
+ print(' arg1: ', arg1)
+ print(' arg2: ', arg2)
+ print(' arg3: ', arg3)
+```
+
+
+If the code inside the `with` statement does not raise an error, these three
+arguments will all be `None`.
+
+
+```
+with MyContextManager():
+ print('In with block')
+```
+
+
+However, if the code inside the `with` statement raises an error, things look
+a little different:
+
+
+```
+with MyContextManager():
+ print('In with block')
+ raise ValueError('Oh no!')
+```
+
+
+So when an error occurs, the `__exit__` method is passed the following:
+
+- The [`Exception`](https://docs.python.org/3.5/tutorial/errors.html)
+ type that was raised.
+- The `Exception` instance that was raised.
+- A [`traceback`](https://docs.python.org/3.5/library/traceback.html) object
+ which can be used to get more information about the exception (e.g. line
+ number).
+
+
+<a class="anchor" id="suppressing-errors-with-exit"></a>
+### Suppressing errors with `__exit__`
+
+
+The `__exit__` method is also capable of suppressing errors - if it returns a
+value of `True`, then any error that was raised will be ignored. For example,
+we could write a context manager which ignores any assertion errors, but
+allows other errors to halt execution as normal:
+
+
+```
+class MyContextManager(object):
+ def __enter__(self):
+ print('In enter')
+
+ def __exit__(self, arg1, arg2, arg3):
+ print('In exit')
+ if issubclass(arg1, AssertionError):
+ return True
+ print(' arg1: ', arg1)
+ print(' arg2: ', arg2)
+ print(' arg3: ', arg3)
+```
+
+> Note that if a function or method does not explicitly return a value, its
+> return value is `None` (which would evaluate to `False` when converted to a
+> `bool`). Also note that we are using the built-in
+> [`issubclass`](https://docs.python.org/3.5/library/functions.html#issubclass)
+> function, which allows us to test the type of a class.
+
+
+Now, when we use `MyContextManager`, any assertion errors are suppressed,
+whereas other errors will be raised as normal:
+
+
+```
+with MyContextManager():
+ assert 1 == 0
+
+print('Continuing execution!')
+
+with MyContextManager():
+ raise ValueError('Oh no!')
+```
+
+
+<a class="anchor" id="nesting-context-managers"></a>
+## Nesting context managers
+
+
+It is possible to nest `with` statements:
+
+```
+with open('05_context_managers.md', 'rt') as inf:
+ with TempDir():
+ with open('05_context_managers.md.copy', 'wt') as outf:
+ outf.write(inf.read())
+```
+
+
+You can also use multiple context managers in a single `with` statement:
+
+
+```
+with open('05_context_managers.md', 'rt') as inf, \
+ TempDir(), \
+ open('05_context_managers.md.copy', 'wt') as outf:
+ outf.write(inf.read())
+```
+
+
+<a class="anchor" id="functions-as-context-managers"></a>
+## Functions as context managers
+
+
+In fact, there is another way to create context managers in Python. The
+built-in [`contextlib`
+module](https://docs.python.org/3.5/library/contextlib.html#contextlib.contextmanager)
+has a decorator called `@contextmanager`, which allows us to turn __any
+function__ into a context manager. The only requirement is that the function
+must have a `yield` statement<sup>1</sup>. So we could rewrite our `TempDir`
+class from above as a function:
+
+
+```
+import os
+import shutil
+import tempfile
+import contextlib
+
+@contextlib.contextmanager
+def tempdir():
+ tdir = tempfile.mkdtemp()
+ prevdir = os.getcwd()
+ try:
+
+ os.chdir(tdir)
+ yield tdir
+
+ finally:
+ os.chdir(prevdir)
+ shutil.rmtree(tdir)
+```
+
+
+This new `tempdir` function is used in exactly the same way as our `TempDir`
+class:
+
+
+```
+print('In directory: {}'.format(os.getcwd()))
+
+with tempdir() as tmp:
+ print('Now in directory: {}'.format(os.getcwd()))
+
+print('Back in directory: {}'.format(os.getcwd()))
+```
+
+
+The `yield tdir` statement in our `tempdir` function causes the `tdir` value
+to be passed to the `with` statement, so in the line `with tempdir() as tmp`,
+the variable `tmp` will be given the value `tdir`.
+
+
+> <sup>1</sup> The `yield` keyword is used in _generator functions_.
+> Functions which are used with the `@contextmanager` decorator must be
+> generator functions which yield exactly one value.
+> [Generators](https://www.python.org/dev/peps/pep-0289/) and [generator
+> functions](https://docs.python.org/3.5/glossary.html#term-generator) are
+> beyond the scope of this practical.
+
+
+<a class="anchor" id="methods-as-context-managers"></a>
+## Methods as context managers
+
+
+Since it is possible to write a function which is a context manager, it is of
+course also possible to write a _method_ which is a context manager. Let's
+play with another example. We have a `Notifier` class which can be used to
+notify interested listeners when an event occurs. Listeners can be registered
+for notification via the `register` method:
+
+
+```
+from collections import OrderedDict
+
+class Notifier(object):
+ def __init__(self):
+ super().__init__()
+ self.listeners = OrderedDict()
+
+ def register(self, name, func):
+ self.listeners[name] = func
+
+ def notify(self):
+ for listener in self.listeners.values():
+ listener()
+```
+
+
+Now, let's build a little plotting application. First of all, we have a `Line`
+class, which represents a line plot. The `Line` class is a sub-class of
+`Notifier`, so whenever its display properties (`colour`, `width`, or `name`)
+change, it emits a notification, and whatever is drawing it can refresh the
+display:
+
+
+```
+import numpy as np
+
+class Line(Notifier):
+
+ def __init__(self, data):
+ super().__init__()
+ self.__data = data
+ self.__colour = '#000000'
+ self.__width = 1
+ self.__name = 'line'
+
+ @property
+ def xdata(self):
+ return np.arange(len(self.__data))
+
+ @property
+ def ydata(self):
+ return np.copy(self.__data)
+
+ @property
+ def colour(self):
+ return self.__colour
+
+ @colour.setter
+ def colour(self, newColour):
+ self.__colour = newColour
+ print('Line: colour changed: {}'.format(newColour))
+ self.notify()
+
+ @property
+ def width(self):
+ return self.__width
+
+ @width.setter
+ def width(self, newWidth):
+ self.__width = newWidth
+ print('Line: width changed: {}'.format(newWidth))
+ self.notify()
+
+ @property
+ def name(self):
+ return self.__name
+
+ @name.setter
+ def name(self, newName):
+ self.__name = newName
+ print('Line: name changed: {}'.format(newName))
+ self.notify()
+```
+
+
+Now let's write a `Plotter` class, which can plot one or more `Line`
+instances:
+
+
+```
+import matplotlib.pyplot as plt
+
+class Plotter(object):
+ def __init__(self, axis):
+ self.__axis = axis
+ self.__lines = []
+
+ def addData(self, data):
+ line = Line(data)
+ self.__lines.append(line)
+ line.register('plot', self.lineChanged)
+ self.draw()
+ return line
+
+ def lineChanged(self):
+ self.draw()
+
+ def draw(self):
+ print('Plotter: redrawing plot')
+
+ ax = self.__axis
+ ax.clear()
+ for line in self.__lines:
+ ax.plot(line.xdata,
+ line.ydata,
+ color=line.colour,
+ linewidth=line.width,
+ label=line.name)
+ ax.legend()
+```
+
+
+Let's create a `Plotter` object, and add a couple of lines to it (note that
+the `matplotlib` plot will open in a separate window):
+
+
+```
+# this line is only necessary when
+# working in jupyer notebook/ipython
+%matplotlib
+
+fig = plt.figure()
+ax = fig.add_subplot(111)
+plotter = Plotter(ax)
+l1 = plotter.addData(np.sin(np.linspace(0, 6 * np.pi, 50)))
+l2 = plotter.addData(np.cos(np.linspace(0, 6 * np.pi, 50)))
+
+fig.show()
+```
+
+
+Now, when we change the properties of our `Line` instances, the plot will be
+automatically updated:
+
+
+```
+l1.colour = '#ff0000'
+l2.colour = '#00ff00'
+l1.width = 2
+l2.width = 2
+l1.name = 'sine'
+l2.name = 'cosine'
+```
+
+
+Pretty cool! However, this seems very inefficient - every time we change the
+properties of a `Line`, the `Plotter` will refresh the plot. If we were
+plotting large amounts of data, this would be unacceptable, as plotting would
+simply take too long.
+
+
+Wouldn't it be nice if we were able to perform batch-updates of `Line`
+properties, and only refresh the plot when we are done? Let's add an extra
+method to the `Plotter` class:
+
+
+```
+import contextlib
+
+class Plotter(object):
+ def __init__(self, axis):
+ self.__axis = axis
+ self.__lines = []
+ self.__holdUpdates = False
+
+ def addData(self, data):
+ line = Line(data)
+ self.__lines.append(line)
+ line.register('plot', self.lineChanged)
+
+ if not self.__holdUpdates:
+ self.draw()
+ return line
+
+ def lineChanged(self):
+ if not self.__holdUpdates:
+ self.draw()
+
+ def draw(self):
+ print('Plotter: redrawing plot')
+
+ ax = self.__axis
+ ax.clear()
+ for line in self.__lines:
+ ax.plot(line.xdata,
+ line.ydata,
+ color=line.colour,
+ linewidth=line.width,
+ label=line.name)
+ ax.legend()
+
+ @contextlib.contextmanager
+ def holdUpdates(self):
+ self.__holdUpdates = True
+ try:
+ yield
+ self.draw()
+ finally:
+ self.__holdUpdates = False
+```
+
+
+This new `holdUpdates` method allows us to temporarily suppress notifications
+from all `Line` instances. So now, we can update many `Line` properties
+without performing any redundant redraws:
+
+
+```
+fig = plt.figure()
+ax = fig.add_subplot(111)
+plotter = Plotter(ax)
+
+plt.show()
+
+with plotter.holdUpdates():
+ l1 = plotter.addData(np.sin(np.linspace(0, 6 * np.pi, 50)))
+ l2 = plotter.addData(np.cos(np.linspace(0, 6 * np.pi, 50)))
+ l1.colour = '#0000ff'
+ l2.colour = '#ffff00'
+ l1.width = 1
+ l2.width = 1
+ l1.name = '$sin(x)$'
+ l2.name = '$cos(x)$'
+```
+
+
+<a class="anchor" id="useful-references"></a>
+## Useful references
+
+
+* [Context manager classes](https://docs.python.org/3.5/reference/datamodel.html#context-managers)
+* The [`contextlib` module](https://docs.python.org/3.5/library/contextlib.html)
diff --git a/advanced_topics/decorators.ipynb b/advanced_topics/06_decorators.ipynb
similarity index 99%
rename from advanced_topics/decorators.ipynb
rename to advanced_topics/06_decorators.ipynb
index 41d2bd944bcb6d9f6400c1ce5ca7ef6566d5aa30..8601e8b2df6163bd508efb0ead93744a4c604a09 100644
--- a/advanced_topics/decorators.ipynb
+++ b/advanced_topics/06_decorators.ipynb
@@ -383,11 +383,11 @@
"\n",
" if n in (0, 1):\n",
" print('fib({}) = {}'.format(n, n))\n",
- " return 1\n",
+ " return n\n",
"\n",
" twoback = 1\n",
" oneback = 1\n",
- " val = 0\n",
+ " val = 1\n",
"\n",
" for _ in range(2, n):\n",
"\n",
@@ -541,13 +541,13 @@
"@limitedMemoize(5)\n",
"def fib(n):\n",
"\n",
- " if n in (1, 2):\n",
+ " if n in (0, 1):\n",
" print('fib({}) = 1'.format(n))\n",
- " return 1\n",
+ " return n\n",
"\n",
" twoback = 1\n",
" oneback = 1\n",
- " val = 0\n",
+ " val = 1\n",
"\n",
" for _ in range(2, n):\n",
"\n",
diff --git a/advanced_topics/decorators.md b/advanced_topics/06_decorators.md
similarity index 99%
rename from advanced_topics/decorators.md
rename to advanced_topics/06_decorators.md
index df8927bff086f29a4faf2efb3baeb50401f4d1ed..86b06335d131bad3441d8c65e03fd23cf2ba9272 100644
--- a/advanced_topics/decorators.md
+++ b/advanced_topics/06_decorators.md
@@ -292,11 +292,11 @@ def fib(n):
if n in (0, 1):
print('fib({}) = {}'.format(n, n))
- return 1
+ return n
twoback = 1
oneback = 1
- val = 0
+ val = 1
for _ in range(2, n):
@@ -417,13 +417,13 @@ earlier `memoize` decorator:
@limitedMemoize(5)
def fib(n):
- if n in (1, 2):
+ if n in (0, 1):
print('fib({}) = 1'.format(n))
- return 1
+ return n
twoback = 1
oneback = 1
- val = 0
+ val = 1
for _ in range(2, n):
diff --git a/advanced_topics/README.md b/advanced_topics/README.md
index 4a9efdab9c949ab0e9a8ff324f3cf15e27f14443..3cd803e8786886f2c4396012710f84591e4073c1 100644
--- a/advanced_topics/README.md
+++ b/advanced_topics/README.md
@@ -2,16 +2,17 @@ Advanced Python
===============
This directory contains a collection of practicals, each of which covers a
-particular feature of the Python Programming language. They are intended for
+particular feature of the Python programming language. They are intended for
people who are familiar with the basics of Python, and want to learn about
some of the more advanced features of the language.
-Practicals on the following topics are available:
+Practicals on the following topics are available. They can be viewed in any
+order, but we recommend going through them in this order:
-* Function inputs and outputs
-* Modules and packages
-* Object-oriented programming
-* Operator overloading
-* Decorators
-* Context managers
-* Testing
+1. Function inputs and outputs
+2. Modules and packages
+3. Object-oriented programming
+4. Operator overloading
+5. Context managers
+6. Decorators
+7. Testing
diff --git a/getting_started/03_file_management.ipynb b/getting_started/03_file_management.ipynb
index d1ee18a921d8729d2131562a5a39a7bac927d3f6..3aa9736d36d72f754dfeae3a7628d429afd2e55f 100644
--- a/getting_started/03_file_management.ipynb
+++ b/getting_started/03_file_management.ipynb
@@ -58,10 +58,6 @@
"* [Exercises](#exercises)\n",
" * [Re-name subject directories](#re-name-subject-directories)\n",
" * [Re-organise a data set](#re-organise-a-data-set)\n",
- " * [Re-name subject files](#re-name-subject-files)\n",
- " * [Compress all uncompressed images](#compress-all-uncompressed-images)\n",
- " * [Write your own `os.path.splitext`](#write-your-own-os-path-splitext)\n",
- " * [Write a function to return a specific image file](#write-a-function-to-return-a-specific-image-file)\n",
" * [Solutions](#solutions)\n",
"\n",
"\n",
@@ -161,13 +157,13 @@
"cwd = os.getcwd()\n",
"listing = os.listdir(cwd)\n",
"print('Directory listing: {}'.format(cwd))\n",
- "print('\\n'.join([p for p in listing]))\n",
+ "print('\\n'.join(listing))\n",
"print()\n",
"\n",
"datadir = 'raw_mri_data'\n",
"listing = os.listdir(datadir)\n",
"print('Directory listing: {}'.format(datadir))\n",
- "print('\\n'.join([p for p in listing]))"
+ "print('\\n'.join(listing))"
]
},
{
@@ -326,7 +322,7 @@
"source": [
"> Here we have explicitly named the `topdown` argument when passing it to the\n",
"> `os.walk` function. This is referred to as a a _keyword argument_ - unnamed\n",
- "> arguments aqe referred to as _positional arguments_. We'll give some more\n",
+ "> arguments are referred to as _positional arguments_. We'll give some more\n",
"> examples of positional and keyword arguments below.\n",
"\n",
"\n",
@@ -447,11 +443,6 @@
"metadata": {},
"outputs": [],
"source": [
- "# This function takes an optional keyword\n",
- "# argument \"existonly\", which controls\n",
- "# whether the path is only tested for\n",
- "# existence. We can call it either with\n",
- "# or without this argument.\n",
"def whatisit(path, existonly=False):\n",
"\n",
" print('Does {} exist? {}'.format(path, op.exists(path)))\n",
@@ -465,11 +456,53 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "Now let's use that function to test some paths.\n",
+ "> This is the first time in this series of practicals that we have defined our\n",
+ "> own function, [hooray!](https://www.youtube.com/watch?v=zQiibNVIvK4) All\n",
+ "> function definitions in Python begin with the `def` keyword:\n",
+ ">\n",
+ "> ```\n",
+ "> def myfunction():\n",
+ "> function_body\n",
+ "> ```\n",
+ ">\n",
+ "> Just like with other control flow tools, such as `if`, `for`, and `while`\n",
+ "> statements, the body of a function must be indented (with four spaces\n",
+ "> please!).\n",
+ ">\n",
+ "> Python functions can be written to accept any number of arguments:\n",
+ ">\n",
+ "> ```\n",
+ "> def myfunction(arg1, arg2, arg3):\n",
+ "> function_body\n",
+ "> ```\n",
+ ">\n",
+ "> Arguments can also be given default values:\n",
+ ">\n",
+ "> ```\n",
+ "> def myfunction(arg1, arg2, arg3=False):\n",
+ "> function_body\n",
+ "> ```\n",
+ ">\n",
+ "> In our `whatisit` function above, we gave the `existonly` argument (which\n",
+ "> controls whether the path is only tested for existence) a default value.\n",
+ "> This makes the `existonly` argument optional - we can call `whatisit` either\n",
+ "> with or without this argument.\n",
+ ">\n",
+ "> To return a value from a function, use the `return` keyword:\n",
+ ">\n",
+ "> ```\n",
+ "> def add(n1, n2):\n",
+ "> return n1 + n2\n",
+ "> ```\n",
+ ">\n",
+ "> Take a look at the [official Python\n",
+ "> tutorial](https://docs.python.org/3.5/tutorial/controlflow.html#defining-functions)\n",
+ "> for more details on defining your own functions.\n",
"\n",
"\n",
- "> Here we are using the `op.join` function to construct paths - it is [covered\n",
- "> below](#cross-platform-compatbility)."
+ "Now let's use that function to test some paths. Here we are using the\n",
+ "`op.join` function to construct paths - it is [covered\n",
+ "below](#cross-platform-compatbility):"
]
},
{
@@ -639,18 +672,18 @@
"metadata": {},
"outputs": [],
"source": [
- "import glob\n",
+ "from glob import glob\n",
"\n",
"root = 'raw_mri_data'\n",
"\n",
"# find all niftis for subject 1\n",
- "images = glob.glob(op.join(root, 'subj_1', '*.nii*'))\n",
+ "images = glob(op.join(root, 'subj_1', '*.nii*'))\n",
"\n",
"print('Subject #1 images:')\n",
"print('\\n'.join([' {}'.format(i) for i in images]))\n",
"\n",
"# find all subject directories\n",
- "subjdirs = glob.glob(op.join(root, 'subj_*'))\n",
+ "subjdirs = glob(op.join(root, 'subj_*'))\n",
"\n",
"print('Subject directories:')\n",
"print('\\n'.join([' {}'.format(d) for d in subjdirs]))"
@@ -661,7 +694,7 @@
"metadata": {},
"source": [
"As with [`os.walk`](walking-a-directory-tree), the order of the results\n",
- "returned by `glob.glob` is arbitrary. Unfortunately the undergraduate who\n",
+ "returned by `glob` is arbitrary. Unfortunately the undergraduate who\n",
"acquired this specific data set did not think to use zero-padded subject IDs\n",
"(you'll be pleased to know that this student was immediately kicked out of his\n",
"college and banned from ever returning), so we can't simply sort the paths\n",
@@ -734,7 +767,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "As of Python 3.5, `glob.glob` also supports recursive pattern matching via the\n",
+ "> Note that in Python, we can pass a function around just like any other\n",
+ "> variable - we passed the `get_subject_id` function as an argument to the\n",
+ "> `sorted` function. This is possible (and normal) because functions are\n",
+ "> [first class citizens](https://en.wikipedia.org/wiki/First-class_citizen) in\n",
+ "> Python!\n",
+ "\n",
+ "\n",
+ "As of Python 3.5, `glob` also supports recursive pattern matching via the\n",
"`recursive` flag. Let's say we want a list of all resting-state scans in our\n",
"data set:"
]
@@ -745,7 +785,7 @@
"metadata": {},
"outputs": [],
"source": [
- "rscans = glob.glob('raw_mri_data/**/rest.nii.gz', recursive=True)\n",
+ "rscans = glob('raw_mri_data/**/rest.nii.gz', recursive=True)\n",
"\n",
"print('Resting state scans:')\n",
"print('\\n'.join(rscans))"
@@ -765,6 +805,14 @@
" and `fnmatch.filter` functions to identify which paths match your pattern.\n",
"\n",
"\n",
+ "Note that the syntax used by `glob` and `fnmatch` is similar, but __not__\n",
+ "identical to the syntax that you are used to from `bash`. Refer to the\n",
+ "[`fnmatch` module](https://docs.python.org/3.5/library/fnmatch.html)\n",
+ "documentation for details. If you need more complicated pattern matching, you\n",
+ "can use regular expressions, available via the [`re`\n",
+ "module](https://docs.python.org/3.5/library/re.html).\n",
+ "\n",
+ "\n",
"For example, let's retrieve all images that are in our data set:"
]
},
@@ -774,7 +822,7 @@
"metadata": {},
"outputs": [],
"source": [
- "allimages = glob.glob(op.join('raw_mri_data', '**', '*.nii*'), recursive=True)\n",
+ "allimages = glob(op.join('raw_mri_data', '**', '*.nii*'), recursive=True)\n",
"print('All images in experiment:')\n",
"\n",
"# Let's just print the first and last few\n",
@@ -844,8 +892,8 @@
"\n",
"You have [already been\n",
"introduced](#querying-and-changing-the-current-directory) to the\n",
- "`op.expanduser` function. Another handy function is the `op.expandvars` function.\n",
- "which will expand expand any environment variables in a path:"
+ "`op.expanduser` function. Another handy function is the `op.expandvars`\n",
+ "function, which will expand expand any environment variables in a path:"
]
},
{
@@ -942,72 +990,8 @@
" - A list of lists, with each list containing the subject IDs for one group.\n",
"\n",
"\n",
- "<a class=\"anchor\" id=\"re-name-subject-files\"></a>\n",
- "### Re-name subject files\n",
- "\n",
- "\n",
- "Write a function which, given a subject directory, renames all of the image\n",
- "files for this subject so that they are prefixed with `[group]_subj_[id]`,\n",
- "where `[group]` is either `CON` or `PAT`, and `[id]` is the (zero-padded)\n",
- "subject ID.\n",
- "\n",
- "\n",
- "This function should accept the following parameters:\n",
- " - The subject directory\n",
- " - The subject group\n",
- "\n",
- "\n",
- "**Bonus 1** Make your function work with both `.nii` and `.nii.gz` files.\n",
- "\n",
- "**Bonus 2** If you completed [the previous exercise](#re-organise-a-data-set),\n",
- "write a second function which accepts the data set directory as a sole\n",
- "parameter, and then calls the first function for every subject.\n",
- "\n",
- "\n",
- "<a class=\"anchor\" id=\"compress-all-uncompressed-images\"></a>\n",
- "### Compress all uncompressed images\n",
- "\n",
- "\n",
- "Write a function which recursively scans a directory, and replaces all `.nii`\n",
- "files with `.nii.gz` files, using the built-in\n",
- "[`gzip`](https://docs.python.org/3.5/library/gzip.html) library to perform\n",
- "the compression.\n",
- "\n",
- "\n",
- "<a class=\"anchor\" id=\"write-your-own-os-path-splitext\"></a>\n",
- "### Write your own `os.path.splitext`\n",
- "\n",
- "\n",
- "Write an implementation of `os.path.splitext` which works with compressed or\n",
- "uncompressed NIFTI images.\n",
- "\n",
- "\n",
- "> Hint: you know what suffixes to expect!\n",
- "\n",
- "\n",
- "<a class=\"anchor\" id=\"write-a-function-to-return-a-specific-image-file\"></a>\n",
- "### Write a function to return a specific image file\n",
- "\n",
- "\n",
- "Assuming that you have completed the previous exercises, and re-organised\n",
- "`raw_mri_data` so that it has the structure:\n",
- "\n",
- " `raw_mri_data/[group]/subj_[id]/[group]_subj_[id]_[modality].nii.gz`\n",
- "\n",
- "write a function which is given:\n",
- "\n",
- " - the data set directory\n",
- " - a group label\n",
- " - integer ubject ID\n",
- " - modality (`'t1'`, `'t2'`, `'task'`, `'rest'`)\n",
- "\n",
- "and which returns the fully resolved path to the relevant image file.\n",
- "\n",
- " > Hint: Python has [regular\n",
- " expressions](https://docs.python.org/3.5/library/re.html) - you might want\n",
- " to use one to cope with zero-padding.\n",
- "\n",
- "**Bonus** Modify the function so the group label does not need to be passed in.\n",
+ "__Extra exercises:__ If you are looking for something more to do, you can find\n",
+ "some more exercises in the file `03_file_management_extra.md`.\n",
"\n",
"\n",
"<a class=\"anchor\" id=\"solutions\"></a>\n",
diff --git a/getting_started/03_file_management.md b/getting_started/03_file_management.md
index a3bfea0aecb50d0b225722c2bb74849447b658c0..614425ba204e7d8cee58d8b7b4acf39f55107d20 100644
--- a/getting_started/03_file_management.md
+++ b/getting_started/03_file_management.md
@@ -52,10 +52,6 @@ other sections as a reference. You might miss out on some neat tricks though.
* [Exercises](#exercises)
* [Re-name subject directories](#re-name-subject-directories)
* [Re-organise a data set](#re-organise-a-data-set)
- * [Re-name subject files](#re-name-subject-files)
- * [Compress all uncompressed images](#compress-all-uncompressed-images)
- * [Write your own `os.path.splitext`](#write-your-own-os-path-splitext)
- * [Write a function to return a specific image file](#write-a-function-to-return-a-specific-image-file)
* [Solutions](#solutions)
@@ -126,13 +122,13 @@ command):
cwd = os.getcwd()
listing = os.listdir(cwd)
print('Directory listing: {}'.format(cwd))
-print('\n'.join([p for p in listing]))
+print('\n'.join(listing))
print()
datadir = 'raw_mri_data'
listing = os.listdir(datadir)
print('Directory listing: {}'.format(datadir))
-print('\n'.join([p for p in listing]))
+print('\n'.join(listing))
```
@@ -248,7 +244,7 @@ for root, dirs, files in os.walk('raw_mri_data', topdown=False):
> Here we have explicitly named the `topdown` argument when passing it to the
> `os.walk` function. This is referred to as a a _keyword argument_ - unnamed
-> arguments aqe referred to as _positional arguments_. We'll give some more
+> arguments are referred to as _positional arguments_. We'll give some more
> examples of positional and keyword arguments below.
@@ -338,12 +334,8 @@ it exists at all, then the `os.path` module has got your back with its
`isfile`, `isdir`, and `exists` functions. Let's define a silly function which
will tell us what a path is:
+
```
-# This function takes an optional keyword
-# argument "existonly", which controls
-# whether the path is only tested for
-# existence. We can call it either with
-# or without this argument.
def whatisit(path, existonly=False):
print('Does {} exist? {}'.format(path, op.exists(path)))
@@ -354,11 +346,53 @@ def whatisit(path, existonly=False):
```
-Now let's use that function to test some paths.
+> This is the first time in this series of practicals that we have defined our
+> own function, [hooray!](https://www.youtube.com/watch?v=zQiibNVIvK4) All
+> function definitions in Python begin with the `def` keyword:
+>
+> ```
+> def myfunction():
+> function_body
+> ```
+>
+> Just like with other control flow tools, such as `if`, `for`, and `while`
+> statements, the body of a function must be indented (with four spaces
+> please!).
+>
+> Python functions can be written to accept any number of arguments:
+>
+> ```
+> def myfunction(arg1, arg2, arg3):
+> function_body
+> ```
+>
+> Arguments can also be given default values:
+>
+> ```
+> def myfunction(arg1, arg2, arg3=False):
+> function_body
+> ```
+>
+> In our `whatisit` function above, we gave the `existonly` argument (which
+> controls whether the path is only tested for existence) a default value.
+> This makes the `existonly` argument optional - we can call `whatisit` either
+> with or without this argument.
+>
+> To return a value from a function, use the `return` keyword:
+>
+> ```
+> def add(n1, n2):
+> return n1 + n2
+> ```
+>
+> Take a look at the [official Python
+> tutorial](https://docs.python.org/3.5/tutorial/controlflow.html#defining-functions)
+> for more details on defining your own functions.
-> Here we are using the `op.join` function to construct paths - it is [covered
-> below](#cross-platform-compatbility).
+Now let's use that function to test some paths. Here we are using the
+`op.join` function to construct paths - it is [covered
+below](#cross-platform-compatbility):
```
@@ -483,18 +517,18 @@ files, based on unix-style wildcard pattern matching.
```
-import glob
+from glob import glob
root = 'raw_mri_data'
# find all niftis for subject 1
-images = glob.glob(op.join(root, 'subj_1', '*.nii*'))
+images = glob(op.join(root, 'subj_1', '*.nii*'))
print('Subject #1 images:')
print('\n'.join([' {}'.format(i) for i in images]))
# find all subject directories
-subjdirs = glob.glob(op.join(root, 'subj_*'))
+subjdirs = glob(op.join(root, 'subj_*'))
print('Subject directories:')
print('\n'.join([' {}'.format(d) for d in subjdirs]))
@@ -502,7 +536,7 @@ print('\n'.join([' {}'.format(d) for d in subjdirs]))
As with [`os.walk`](walking-a-directory-tree), the order of the results
-returned by `glob.glob` is arbitrary. Unfortunately the undergraduate who
+returned by `glob` is arbitrary. Unfortunately the undergraduate who
acquired this specific data set did not think to use zero-padded subject IDs
(you'll be pleased to know that this student was immediately kicked out of his
college and banned from ever returning), so we can't simply sort the paths
@@ -550,13 +584,21 @@ print('Subject directories, sorted by ID:')
print('\n'.join([' {}'.format(d) for d in subjdirs]))
```
-As of Python 3.5, `glob.glob` also supports recursive pattern matching via the
+
+> Note that in Python, we can pass a function around just like any other
+> variable - we passed the `get_subject_id` function as an argument to the
+> `sorted` function. This is possible (and normal) because functions are
+> [first class citizens](https://en.wikipedia.org/wiki/First-class_citizen) in
+> Python!
+
+
+As of Python 3.5, `glob` also supports recursive pattern matching via the
`recursive` flag. Let's say we want a list of all resting-state scans in our
data set:
```
-rscans = glob.glob('raw_mri_data/**/rest.nii.gz', recursive=True)
+rscans = glob('raw_mri_data/**/rest.nii.gz', recursive=True)
print('Resting state scans:')
print('\n'.join(rscans))
@@ -573,10 +615,19 @@ pattern matching logic.
and `fnmatch.filter` functions to identify which paths match your pattern.
+Note that the syntax used by `glob` and `fnmatch` is similar, but __not__
+identical to the syntax that you are used to from `bash`. Refer to the
+[`fnmatch` module](https://docs.python.org/3.5/library/fnmatch.html)
+documentation for details. If you need more complicated pattern matching, you
+can use regular expressions, available via the [`re`
+module](https://docs.python.org/3.5/library/re.html).
+
+
For example, let's retrieve all images that are in our data set:
+
```
-allimages = glob.glob(op.join('raw_mri_data', '**', '*.nii*'), recursive=True)
+allimages = glob(op.join('raw_mri_data', '**', '*.nii*'), recursive=True)
print('All images in experiment:')
# Let's just print the first and last few
@@ -627,8 +678,8 @@ for filename in uncompressed:
You have [already been
introduced](#querying-and-changing-the-current-directory) to the
-`op.expanduser` function. Another handy function is the `op.expandvars` function.
-which will expand expand any environment variables in a path:
+`op.expanduser` function. Another handy function is the `op.expandvars`
+function, which will expand expand any environment variables in a path:
```
@@ -709,72 +760,8 @@ parameters:
- A list of lists, with each list containing the subject IDs for one group.
-<a class="anchor" id="re-name-subject-files"></a>
-### Re-name subject files
-
-
-Write a function which, given a subject directory, renames all of the image
-files for this subject so that they are prefixed with `[group]_subj_[id]`,
-where `[group]` is either `CON` or `PAT`, and `[id]` is the (zero-padded)
-subject ID.
-
-
-This function should accept the following parameters:
- - The subject directory
- - The subject group
-
-
-**Bonus 1** Make your function work with both `.nii` and `.nii.gz` files.
-
-**Bonus 2** If you completed [the previous exercise](#re-organise-a-data-set),
-write a second function which accepts the data set directory as a sole
-parameter, and then calls the first function for every subject.
-
-
-<a class="anchor" id="compress-all-uncompressed-images"></a>
-### Compress all uncompressed images
-
-
-Write a function which recursively scans a directory, and replaces all `.nii`
-files with `.nii.gz` files, using the built-in
-[`gzip`](https://docs.python.org/3.5/library/gzip.html) library to perform
-the compression.
-
-
-<a class="anchor" id="write-your-own-os-path-splitext"></a>
-### Write your own `os.path.splitext`
-
-
-Write an implementation of `os.path.splitext` which works with compressed or
-uncompressed NIFTI images.
-
-
-> Hint: you know what suffixes to expect!
-
-
-<a class="anchor" id="write-a-function-to-return-a-specific-image-file"></a>
-### Write a function to return a specific image file
-
-
-Assuming that you have completed the previous exercises, and re-organised
-`raw_mri_data` so that it has the structure:
-
- `raw_mri_data/[group]/subj_[id]/[group]_subj_[id]_[modality].nii.gz`
-
-write a function which is given:
-
- - the data set directory
- - a group label
- - integer ubject ID
- - modality (`'t1'`, `'t2'`, `'task'`, `'rest'`)
-
-and which returns the fully resolved path to the relevant image file.
-
- > Hint: Python has [regular
- expressions](https://docs.python.org/3.5/library/re.html) - you might want
- to use one to cope with zero-padding.
-
-**Bonus** Modify the function so the group label does not need to be passed in.
+__Extra exercises:__ If you are looking for something more to do, you can find
+some more exercises in the file `03_file_management_extra.md`.
<a class="anchor" id="solutions"></a>
diff --git a/getting_started/03_file_management_extra.md b/getting_started/03_file_management_extra.md
new file mode 100644
index 0000000000000000000000000000000000000000..a56a9fbf70d626bdd4a99068f829a159dbd019c8
--- /dev/null
+++ b/getting_started/03_file_management_extra.md
@@ -0,0 +1,62 @@
+### Re-name subject files
+
+
+Write a function which, given a subject directory, renames all of the image
+files for this subject so that they are prefixed with `[group]_subj_[id]`,
+where `[group]` is either `CON` or `PAT`, and `[id]` is the (zero-padded)
+subject ID.
+
+
+This function should accept the following parameters:
+ - The subject directory
+ - The subject group
+
+
+**Bonus 1** Make your function work with both `.nii` and `.nii.gz` files.
+
+**Bonus 2** If you completed [the previous exercise](#re-organise-a-data-set),
+write a second function which accepts the data set directory as a sole
+parameter, and then calls the first function for every subject.
+
+
+### Compress all uncompressed images
+
+
+Write a function which recursively scans a directory, and replaces all `.nii`
+files with `.nii.gz` files, using the built-in
+[`gzip`](https://docs.python.org/3.5/library/gzip.html) library to perform
+the compression.
+
+
+### Write your own `os.path.splitext`
+
+
+Write an implementation of `os.path.splitext` which works with compressed or
+uncompressed NIFTI images.
+
+
+> Hint: you know what suffixes to expect!
+
+
+### Write a function to return a specific image file
+
+
+Assuming that you have completed the previous exercises, and re-organised
+`raw_mri_data` so that it has the structure:
+
+ `raw_mri_data/[group]/subj_[id]/[group]_subj_[id]_[modality].nii.gz`
+
+write a function which is given:
+
+ - the data set directory
+ - a group label
+ - integer ubject ID
+ - modality (`'t1'`, `'t2'`, `'task'`, `'rest'`)
+
+and which returns the fully resolved path to the relevant image file.
+
+ > Hint: Python has [regular
+ expressions](https://docs.python.org/3.5/library/re.html) - you might want
+ to use one to cope with zero-padding.
+
+**Bonus** Modify the function so the group label does not need to be passed in.
diff --git a/getting_started/04_numpy.ipynb b/getting_started/04_numpy.ipynb
index 23b4131ecdd19d26f411a08610cdbc0c851138f5..bccf81a46347a23fed03a371d09d3f0096f1204b 100644
--- a/getting_started/04_numpy.ipynb
+++ b/getting_started/04_numpy.ipynb
@@ -1404,7 +1404,7 @@
"metadata": {},
"source": [
"But if you are writing a script or application using Numpy, I implore you to\n",
- "Numpy (and its commonly used sub-modules) like this instead:"
+ "import Numpy (and its commonly used sub-modules) like this instead:"
]
},
{
@@ -1497,12 +1497,12 @@
"By now you should be aware that a Numpy `array` does not behave in quite the\n",
"same way as a Matlab matrix. The primary difference between Numpy and Matlab\n",
"is that in Numpy, the `*` operator denotes element-wise multiplication,\n",
- "gwhereas in Matlab, `*` denotes matrix multiplication.\n",
+ "whereas in Matlab, `*` denotes matrix multiplication.\n",
"\n",
"\n",
"Numpy does support the `@` operator for matrix multiplication, but if this is\n",
- "a complete show-stopper for you - if you just can't bring yourself to write `A\n",
- "@ B` to denote the matrix product of `A` and `B` - if you _must_ have your\n",
+ "a complete show-stopper for you - if you just can't bring yourself to write\n",
+ "`A @ B` to denote the matrix product of `A` and `B` - if you _must_ have your\n",
"code looking as Matlab-like as possible, then you should look into the Numpy\n",
"[`matrix`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.matrix.html)\n",
"data type.\n",
diff --git a/getting_started/04_numpy.md b/getting_started/04_numpy.md
index 8ea10a8521108c14c0c4a37a39dca132e961da4d..3a28cc830442b08628633cd04372f9091a6683ee 100644
--- a/getting_started/04_numpy.md
+++ b/getting_started/04_numpy.md
@@ -1055,7 +1055,7 @@ print(d)
But if you are writing a script or application using Numpy, I implore you to
-Numpy (and its commonly used sub-modules) like this instead:
+import Numpy (and its commonly used sub-modules) like this instead:
```
@@ -1124,12 +1124,12 @@ print(np.atleast_2d(r).T)
By now you should be aware that a Numpy `array` does not behave in quite the
same way as a Matlab matrix. The primary difference between Numpy and Matlab
is that in Numpy, the `*` operator denotes element-wise multiplication,
-gwhereas in Matlab, `*` denotes matrix multiplication.
+whereas in Matlab, `*` denotes matrix multiplication.
Numpy does support the `@` operator for matrix multiplication, but if this is
-a complete show-stopper for you - if you just can't bring yourself to write `A
-@ B` to denote the matrix product of `A` and `B` - if you _must_ have your
+a complete show-stopper for you - if you just can't bring yourself to write
+`A @ B` to denote the matrix product of `A` and `B` - if you _must_ have your
code looking as Matlab-like as possible, then you should look into the Numpy
[`matrix`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.matrix.html)
data type.