Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
P
pytreat-practicals-2020
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
FSL
pytreat-practicals-2020
Commits
076a73f9
Commit
076a73f9
authored
7 years ago
by
Paul McCarthy
Browse files
Options
Downloads
Patches
Plain Diff
more updates to ctxman prac
parent
247c2e29
No related branches found
Branches containing commit
No related tags found
Tags containing commit
No related merge requests found
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
advanced_topics/05_context_managers.ipynb
+280
-4
280 additions, 4 deletions
advanced_topics/05_context_managers.ipynb
advanced_topics/05_context_managers.md
+232
-4
232 additions, 4 deletions
advanced_topics/05_context_managers.md
with
512 additions
and
8 deletions
advanced_topics/05_context_managers.ipynb
+
280
−
4
View file @
076a73f9
...
@@ -382,8 +382,8 @@
...
@@ -382,8 +382,8 @@
"In fact, there is another way to create context managers in Python. The\n",
"In fact, there is another way to create context managers in Python. The\n",
"built-in [`contextlib`\n",
"built-in [`contextlib`\n",
"module](https://docs.python.org/3.5/library/contextlib.html#contextlib.contextmanager)\n",
"module](https://docs.python.org/3.5/library/contextlib.html#contextlib.contextmanager)\n",
"has a decorator called `@contextmanager`, which allows us to turn __any
__
\n",
"has a decorator called `@contextmanager`, which allows us to turn __any\n",
"function into a context manager. The only requirement is that the function\n",
"function
__
into a context manager. The only requirement is that the function\n",
"must have a `yield` statement<sup>1</sup>. So we could rewrite our `TempDir`\n",
"must have a `yield` statement<sup>1</sup>. So we could rewrite our `TempDir`\n",
"class from above as a function:"
"class from above as a function:"
]
]
...
@@ -447,8 +447,270 @@
...
@@ -447,8 +447,270 @@
"> beyond the scope of this practical.\n",
"> beyond the scope of this practical.\n",
"\n",
"\n",
"\n",
"\n",
"## Methods as context managers\n",
"\n",
"\n",
"Since it is possible to write a function which is a context manager, it is of\n",
"Since it is possible to write a function which is a context manager, it is of\n",
"course also possible to write a _method_ which is a context manager."
"course also possible to write a _method_ which is a context manager. Let's\n",
"play with another example. We have a `Notifier` class which can be used to\n",
"notify interested listeners when an event occurs. Listeners can be registered\n",
"for notification via the `register` method:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from collections import OrderedDict\n",
"\n",
"class Notifier(object):\n",
" def __init__(self):\n",
" super().__init__()\n",
" self.listeners = OrderedDict()\n",
"\n",
" def register(self, name, func):\n",
" self.listeners[name] = func\n",
"\n",
" def notify(self):\n",
" for listener in self.listeners.values():\n",
" listener()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, let's build a little plotting application. First of all, we have a `Line`\n",
"class, which represents a line plot. The `Line` class is a sub-class of\n",
"`Notifier`, so whenever its display properties (`colour`, `width`, or `name`)\n",
"change, it emits a notification, and whatever is drawing it can refresh the\n",
"display:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"class Line(Notifier):\n",
"\n",
" def __init__(self, data):\n",
" super().__init__()\n",
" self.__data = data\n",
" self.__colour = '#000000'\n",
" self.__width = 1\n",
" self.__name = 'line'\n",
"\n",
" @property\n",
" def xdata(self):\n",
" return np.arange(len(self.__data))\n",
"\n",
" @property\n",
" def ydata(self):\n",
" return np.copy(self.__data)\n",
"\n",
" @property\n",
" def colour(self):\n",
" return self.__colour\n",
"\n",
" @colour.setter\n",
" def colour(self, newColour):\n",
" self.__colour = newColour\n",
" print('Line: colour changed: {}'.format(newColour))\n",
" self.notify()\n",
"\n",
" @property\n",
" def width(self):\n",
" return self.__width\n",
"\n",
" @width.setter\n",
" def width(self, newWidth):\n",
" self.__width = newWidth\n",
" print('Line: width changed: {}'.format(newWidth))\n",
" self.notify()\n",
"\n",
" @property\n",
" def name(self):\n",
" return self.__name\n",
"\n",
" @name.setter\n",
" def name(self, newName):\n",
" self.__name = newName\n",
" print('Line: name changed: {}'.format(newName))\n",
" self.notify()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let's write a `Plotter` class, which can plot one or more `Line`\n",
"instances:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# this line is only necessary when\n",
"# working in jupyer notebook/ipython\n",
"%matplotlib\n",
"\n",
"class Plotter(object):\n",
" def __init__(self, axis):\n",
" self.__axis = axis\n",
" self.__lines = []\n",
"\n",
" def addData(self, data):\n",
" line = Line(data)\n",
" self.__lines.append(line)\n",
" line.register('plot', self.lineChanged)\n",
" self.draw()\n",
" return line\n",
"\n",
" def lineChanged(self):\n",
" self.draw()\n",
"\n",
" def draw(self):\n",
" print('Plotter: redrawing plot')\n",
"\n",
" ax = self.__axis\n",
" ax.clear()\n",
" for line in self.__lines:\n",
" ax.plot(line.xdata,\n",
" line.ydata,\n",
" color=line.colour,\n",
" linewidth=line.width,\n",
" label=line.name)\n",
" ax.legend()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's create a `Plotter` object, and add a couple of lines to it (note that\n",
"the `matplotlib` plot will open in a separate window):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"fig = plt.figure()\n",
"ax = fig.add_subplot(111)\n",
"plotter = Plotter(ax)\n",
"l1 = plotter.addData(np.sin(np.linspace(0, 6 * np.pi, 50)))\n",
"l2 = plotter.addData(np.cos(np.linspace(0, 6 * np.pi, 50)))\n",
"\n",
"fig.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, when we change the properties of our `Line` instances, the plot will be\n",
"automatically updated:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"l1.colour = '#ff0000'\n",
"l2.colour = '#00ff00'\n",
"l1.width = 2\n",
"l2.width = 2\n",
"l1.name = 'sine'\n",
"l2.name = 'cosine'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Pretty cool! However, this seems very inefficient - every time we change the\n",
"properties of a `Line`, the `Plotter` will refresh the plot. If we were\n",
"plotting large amounts of data, this would be unacceptable, as plotting would\n",
"simply take too long.\n",
"\n",
"\n",
"Wouldn't it be nice if we were able to perform batch-updates of `Line`\n",
"properties, and only refresh the plot when we are done? Let's add an extra\n",
"method to the `Plotter` class:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import contextlib\n",
"\n",
"class Plotter(object):\n",
" def __init__(self, axis):\n",
" self.__axis = axis\n",
" self.__lines = []\n",
" self.__holdUpdates = False\n",
"\n",
" def addData(self, data):\n",
" line = Line(data)\n",
" self.__lines.append(line)\n",
" line.register('plot', self.lineChanged)\n",
"\n",
" if not self.__holdUpdates:\n",
" self.draw()\n",
" return line\n",
"\n",
" def lineChanged(self):\n",
" if not self.__holdUpdates:\n",
" self.draw()\n",
"\n",
" def draw(self):\n",
" print('Plotter: redrawing plot')\n",
"\n",
" ax = self.__axis\n",
" ax.clear()\n",
" for line in self.__lines:\n",
" ax.plot(line.xdata,\n",
" line.ydata,\n",
" color=line.colour,\n",
" linewidth=line.width,\n",
" label=line.name)\n",
" ax.legend()\n",
"\n",
" @contextlib.contextmanager\n",
" def holdUpdates(self):\n",
" self.__holdUpdates = True\n",
" try:\n",
" yield\n",
" self.draw()\n",
" finally:\n",
" self.__holdUpdates = False"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This new `holdUpdates` method allows us to temporarily suppress notifications\n",
"from all `Line` instances. So now, we can update many `Line` properties\n",
"without performing any redundant redraws:"
]
]
},
},
{
{
...
@@ -457,7 +719,21 @@
...
@@ -457,7 +719,21 @@
"metadata": {},
"metadata": {},
"outputs": [],
"outputs": [],
"source": [
"source": [
"TODO suppress notification example"
"fig = plt.figure()\n",
"ax = fig.add_subplot(111)\n",
"plotter = Plotter(ax)\n",
"\n",
"plt.show()\n",
"\n",
"with plotter.holdUpdates():\n",
" l1 = plotter.addData(np.sin(np.linspace(0, 6 * np.pi, 50)))\n",
" l2 = plotter.addData(np.cos(np.linspace(0, 6 * np.pi, 50)))\n",
" l1.colour = '#0000ff'\n",
" l2.colour = '#ffff00'\n",
" l1.width = 1\n",
" l2.width = 1\n",
" l1.name = '$sin(x)$'\n",
" l2.name = '$cos(x)$'"
]
]
},
},
{
{
...
...
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
# Context managers
# Context managers
The recommended way to open a file in Python is via the
`with`
statement:
The recommended way to open a file in Python is via the
`with`
statement:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
with open('05_context_managers.md', 'rt') as f:
with open('05_context_managers.md', 'rt') as f:
firstlines = f.readlines()[:4]
firstlines = f.readlines()[:4]
firstlines = [l.strip() for l in firstlines]
firstlines = [l.strip() for l in firstlines]
print('\n'.join(firstlines))
print('\n'.join(firstlines))
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
This is because the
`with`
statement ensures that the file will be closed
This is because the
`with`
statement ensures that the file will be closed
automatically, even if an error occurs inside the
`with`
statement.
automatically, even if an error occurs inside the
`with`
statement.
The
`with`
statement is obviously hiding some internal details from us. But
The
`with`
statement is obviously hiding some internal details from us. But
these internals are in fact quite straightforward, and are known as
[
_context
these internals are in fact quite straightforward, and are known as
[
_context
managers_
](
https://docs.python.org/3.5/reference/datamodel.html#context-managers
)
.
managers_
](
https://docs.python.org/3.5/reference/datamodel.html#context-managers
)
.
## Anatomy of a context manager
## Anatomy of a context manager
A _context manager_ is simply an object which has two specially named methods
A _context manager_ is simply an object which has two specially named methods
`__enter__`
and
`__exit__`
. Any object which has these methods can be used in
`__enter__`
and
`__exit__`
. Any object which has these methods can be used in
a
`with`
statement.
a
`with`
statement.
Let's define a context manager class that we can play with:
Let's define a context manager class that we can play with:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
class MyContextManager(object):
class MyContextManager(object):
def __enter__(self):
def __enter__(self):
print('In enter')
print('In enter')
def __exit__(self, *args):
def __exit__(self, *args):
print('In exit')
print('In exit')
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
Now, what happens when we use
`MyContextManager`
in a
`with`
statement?
Now, what happens when we use
`MyContextManager`
in a
`with`
statement?
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
with MyContextManager():
with MyContextManager():
print('In with block')
print('In with block')
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
So the
`__enter__`
method is called before the statements in the
`with`
block,
So the
`__enter__`
method is called before the statements in the
`with`
block,
and the
`__exit__`
method is called afterwards.
and the
`__exit__`
method is called afterwards.
Context managers are that simple. What makes them really useful though, is
Context managers are that simple. What makes them really useful though, is
that the
`__exit__`
method will be called even if the code in the
`with`
block
that the
`__exit__`
method will be called even if the code in the
`with`
block
raises an error. The error will be held, and only raised after the
`__exit__`
raises an error. The error will be held, and only raised after the
`__exit__`
method has finished:
method has finished:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
with MyContextManager():
with MyContextManager():
print('In with block')
print('In with block')
assert 1 == 0
assert 1 == 0
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
This means that we can use context managers to perform any sort of clean up or
This means that we can use context managers to perform any sort of clean up or
finalisation logic that we always want to have executed.
finalisation logic that we always want to have executed.
### Why not just use `try ... finally`?
### Why not just use `try ... finally`?
Context managers do not provide anything that cannot be accomplished in other
Context managers do not provide anything that cannot be accomplished in other
ways. For example, we could accomplish very similar behaviour using
ways. For example, we could accomplish very similar behaviour using
[
`try` - `finally` logic
](
https://docs.python.org/3.5/tutorial/errors.html#handling-exceptions
)
-
[
`try` - `finally` logic
](
https://docs.python.org/3.5/tutorial/errors.html#handling-exceptions
)
-
the statements in the
`finally`
clause will
*always*
be executed, whether an
the statements in the
`finally`
clause will
*always*
be executed, whether an
error is raised or not:
error is raised or not:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
print('Before try block')
print('Before try block')
try:
try:
print('In try block')
print('In try block')
assert 1 == 0
assert 1 == 0
finally:
finally:
print('In finally block')
print('In finally block')
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
But context managers have the advantage that you can implement your clean-up
But context managers have the advantage that you can implement your clean-up
logic in one place, and re-use it as many times as you want.
logic in one place, and re-use it as many times as you want.
## Uses for context managers
## Uses for context managers
We have already talked about how context managers can be used to perform any
We have already talked about how context managers can be used to perform any
task which requires some initialistion and/or clean-up logic. As an example,
task which requires some initialistion and/or clean-up logic. As an example,
here is a context manager which creates a temporary directory, and then makes
here is a context manager which creates a temporary directory, and then makes
sure that it is deleted afterwards.
sure that it is deleted afterwards.
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
import os
import os
import shutil
import shutil
import tempfile
import tempfile
class TempDir(object):
class TempDir(object):
def __enter__(self):
def __enter__(self):
self.tempDir = tempfile.mkdtemp()
self.tempDir = tempfile.mkdtemp()
self.prevDir = os.getcwd()
self.prevDir = os.getcwd()
print('Changing to temp dir: {}'.format(self.tempDir))
print('Changing to temp dir: {}'.format(self.tempDir))
print('Previous directory: {}'.format(self.prevDir))
print('Previous directory: {}'.format(self.prevDir))
os.chdir(self.tempDir)
os.chdir(self.tempDir)
def __exit__(self, *args):
def __exit__(self, *args):
print('Changing back to: {}'.format(self.prevDir))
print('Changing back to: {}'.format(self.prevDir))
print('Removing temp dir: {}'.format(self.tempDir))
print('Removing temp dir: {}'.format(self.tempDir))
os .chdir( self.prevDir)
os .chdir( self.prevDir)
shutil.rmtree(self.tempDir)
shutil.rmtree(self.tempDir)
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
Now imagine that we have a function which loads data from a file, and performs
Now imagine that we have a function which loads data from a file, and performs
some calculation on it:
some calculation on it:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
import numpy as np
import numpy as np
def complexAlgorithm(infile):
def complexAlgorithm(infile):
data = np.loadtxt(infile)
data = np.loadtxt(infile)
return data.mean()
return data.mean()
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
We could use the
`TempDir`
context manager to write a test case for this
We could use the
`TempDir`
context manager to write a test case for this
function, and not have to worry about cleaning up the test data:
function, and not have to worry about cleaning up the test data:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
with TempDir():
with TempDir():
print('Testing complex algorithm')
print('Testing complex algorithm')
data = np.random.random((100, 100))
data = np.random.random((100, 100))
np.savetxt('data.txt', data)
np.savetxt('data.txt', data)
result = complexAlgorithm('data.txt')
result = complexAlgorithm('data.txt')
assert result > 0.1 and result < 0.9
assert result > 0.1 and result < 0.9
print('Test passed (result: {})'.format(result))
print('Test passed (result: {})'.format(result))
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
### Handling errors in `__exit__`
### Handling errors in `__exit__`
By now you must be
[
panicking
](
https://youtu.be/cSU_5MgtDc8?t=9
)
about why I
By now you must be
[
panicking
](
https://youtu.be/cSU_5MgtDc8?t=9
)
about why I
haven't mentioned those conspicuous
`*args`
that get passed to the
`__exit__`
haven't mentioned those conspicuous
`*args`
that get passed to the
`__exit__`
method. It turns out that a context manager's
[
`__exit__`
method. It turns out that a context manager's
[
`__exit__`
method
](
https://docs.python.org/3.5/reference/datamodel.html#object.__exit__
)
method
](
https://docs.python.org/3.5/reference/datamodel.html#object.__exit__
)
is always passed three arguments.
is always passed three arguments.
Let's adjust our
`MyContextManager`
class a little so we can see what these
Let's adjust our
`MyContextManager`
class a little so we can see what these
arguments are for:
arguments are for:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
class MyContextManager(object):
class MyContextManager(object):
def __enter__(self):
def __enter__(self):
print('In enter')
print('In enter')
def __exit__(self, arg1, arg2, arg3):
def __exit__(self, arg1, arg2, arg3):
print('In exit')
print('In exit')
print(' arg1: ', arg1)
print(' arg1: ', arg1)
print(' arg2: ', arg2)
print(' arg2: ', arg2)
print(' arg3: ', arg3)
print(' arg3: ', arg3)
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
If the code inside the
`with`
statement does not raise an error, these three
If the code inside the
`with`
statement does not raise an error, these three
arguments will all be
`None`
.
arguments will all be
`None`
.
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
with MyContextManager():
with MyContextManager():
print('In with block')
print('In with block')
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
However, if the code inside the
`with`
statement raises an error, things look
However, if the code inside the
`with`
statement raises an error, things look
a little different:
a little different:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
with MyContextManager():
with MyContextManager():
print('In with block')
print('In with block')
raise ValueError('Oh no!')
raise ValueError('Oh no!')
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
So when an error occurs, the
`__exit__`
method is passed the following:
So when an error occurs, the
`__exit__`
method is passed the following:
-
The
[
`Exception`
](
https://docs.python.org/3.5/tutorial/errors.html
)
-
The
[
`Exception`
](
https://docs.python.org/3.5/tutorial/errors.html
)
type that was raised.
type that was raised.
-
The
`Exception`
instance that was raised.
-
The
`Exception`
instance that was raised.
-
A
[
`traceback`
](
https://docs.python.org/3.5/library/traceback.html
)
object
-
A
[
`traceback`
](
https://docs.python.org/3.5/library/traceback.html
)
object
which can be used to get more information about the exception (e.g. line
which can be used to get more information about the exception (e.g. line
number).
number).
### Suppressing errors with `__exit__`
### Suppressing errors with `__exit__`
The
`__exit__`
method is also capable of suppressing errors - if it returns a
The
`__exit__`
method is also capable of suppressing errors - if it returns a
value of
`True`
, then any error that was raised will be ignored. For example,
value of
`True`
, then any error that was raised will be ignored. For example,
we could write a context manager which ignores any assertion errors, but
we could write a context manager which ignores any assertion errors, but
allows other errors to halt execution as normal:
allows other errors to halt execution as normal:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
class MyContextManager(object):
class MyContextManager(object):
def __enter__(self):
def __enter__(self):
print('In enter')
print('In enter')
def __exit__(self, arg1, arg2, arg3):
def __exit__(self, arg1, arg2, arg3):
print('In exit')
print('In exit')
if issubclass(arg1, AssertionError):
if issubclass(arg1, AssertionError):
return True
return True
print(' arg1: ', arg1)
print(' arg1: ', arg1)
print(' arg2: ', arg2)
print(' arg2: ', arg2)
print(' arg3: ', arg3)
print(' arg3: ', arg3)
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
> Note that if a function or method does not explicitly return a value, its
> Note that if a function or method does not explicitly return a value, its
> return value is `None` (which would evaluate to `False` when converted to a
> return value is `None` (which would evaluate to `False` when converted to a
> `bool`). Also note that we are using the built-in
> `bool`). Also note that we are using the built-in
> [`issubclass`](https://docs.python.org/3.5/library/functions.html#issubclass)
> [`issubclass`](https://docs.python.org/3.5/library/functions.html#issubclass)
> function, which allows us to test the type of a class.
> function, which allows us to test the type of a class.
Now, when we use
`MyContextManager`
, any assertion errors are suppressed,
Now, when we use
`MyContextManager`
, any assertion errors are suppressed,
whereas other errors will be raised as normal:
whereas other errors will be raised as normal:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
with MyContextManager():
with MyContextManager():
assert 1 == 0
assert 1 == 0
print('Continuing execution!')
print('Continuing execution!')
with MyContextManager():
with MyContextManager():
raise ValueError('Oh no!')
raise ValueError('Oh no!')
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## Functions as context managers
## Functions as context managers
In fact, there is another way to create context managers in Python. The
In fact, there is another way to create context managers in Python. The
built-in
[
`contextlib`
built-in
[
`contextlib`
module
](
https://docs.python.org/3.5/library/contextlib.html#contextlib.contextmanager
)
module
](
https://docs.python.org/3.5/library/contextlib.html#contextlib.contextmanager
)
has a decorator called
`@contextmanager`
, which allows us to turn __any
__
has a decorator called
`@contextmanager`
, which allows us to turn __any
function into a context manager. The only requirement is that the function
function
__
into a context manager. The only requirement is that the function
must have a
`yield`
statement
<sup>
1
</sup>
. So we could rewrite our
`TempDir`
must have a
`yield`
statement
<sup>
1
</sup>
. So we could rewrite our
`TempDir`
class from above as a function:
class from above as a function:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
import os
import os
import shutil
import shutil
import tempfile
import tempfile
import contextlib
import contextlib
@contextlib.contextmanager
@contextlib.contextmanager
def tempdir():
def tempdir():
testdir = tempfile.mkdtemp()
testdir = tempfile.mkdtemp()
prevdir = os.getcwd()
prevdir = os.getcwd()
try:
try:
os.chdir(testdir)
os.chdir(testdir)
yield testdir
yield testdir
finally:
finally:
os.chdir(prevdir)
os.chdir(prevdir)
shutil.rmtree(testdir)
shutil.rmtree(testdir)
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
This new
`tempdir`
function is used in exactly the same way as our
`TempDir`
This new
`tempdir`
function is used in exactly the same way as our
`TempDir`
class:
class:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
print('In directory: {}'.format(os.getcwd()))
print('In directory: {}'.format(os.getcwd()))
with tempdir():
with tempdir():
print('Now in directory: {}'.format(os.getcwd()))
print('Now in directory: {}'.format(os.getcwd()))
print('Back in directory: {}'.format(os.getcwd()))
print('Back in directory: {}'.format(os.getcwd()))
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
> <sup>1</sup> The `yield` keyword is used in _generator functions_.
> <sup>1</sup> The `yield` keyword is used in _generator functions_.
> Functions which are used with the `@contextmanager` decorator must be
> Functions which are used with the `@contextmanager` decorator must be
> generator functions which yield exactly one value.
> generator functions which yield exactly one value.
> [Generators](https://www.python.org/dev/peps/pep-0289/) and [generator
> [Generators](https://www.python.org/dev/peps/pep-0289/) and [generator
> functions](https://docs.python.org/3.5/glossary.html#term-generator) are
> functions](https://docs.python.org/3.5/glossary.html#term-generator) are
> beyond the scope of this practical.
> beyond the scope of this practical.
## Methods as context managers
Since it is possible to write a function which is a context manager, it is of
Since it is possible to write a function which is a context manager, it is of
course also possible to write a _method_ which is a context manager.
course also possible to write a _method_ which is a context manager. Let's
play with another example. We have a
`Notifier`
class which can be used to
notify interested listeners when an event occurs. Listeners can be registered
for notification via the
`register`
method:
%% Cell type:code id: tags:
```
from collections import OrderedDict
class Notifier(object):
def __init__(self):
super().__init__()
self.listeners = OrderedDict()
def register(self, name, func):
self.listeners[name] = func
def notify(self):
for listener in self.listeners.values():
listener()
```
%% Cell type:markdown id: tags:
Now, let's build a little plotting application. First of all, we have a
`Line`
class, which represents a line plot. The
`Line`
class is a sub-class of
`Notifier`
, so whenever its display properties (
`colour`
,
`width`
, or
`name`
)
change, it emits a notification, and whatever is drawing it can refresh the
display:
%% Cell type:code id: tags:
```
import numpy as np
class Line(Notifier):
def __init__(self, data):
super().__init__()
self.__data = data
self.__colour = '#000000'
self.__width = 1
self.__name = 'line'
@property
def xdata(self):
return np.arange(len(self.__data))
@property
def ydata(self):
return np.copy(self.__data)
@property
def colour(self):
return self.__colour
@colour.setter
def colour(self, newColour):
self.__colour = newColour
print('Line: colour changed: {}'.format(newColour))
self.notify()
@property
def width(self):
return self.__width
@width.setter
def width(self, newWidth):
self.__width = newWidth
print('Line: width changed: {}'.format(newWidth))
self.notify()
@property
def name(self):
return self.__name
@name.setter
def name(self, newName):
self.__name = newName
print('Line: name changed: {}'.format(newName))
self.notify()
```
%% Cell type:markdown id: tags:
Now let's write a
`Plotter`
class, which can plot one or more
`Line`
instances:
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
```
TODO suppress notification example
import matplotlib.pyplot as plt
# this line is only necessary when
# working in jupyer notebook/ipython
%matplotlib
class Plotter(object):
def __init__(self, axis):
self.__axis = axis
self.__lines = []
def addData(self, data):
line = Line(data)
self.__lines.append(line)
line.register('plot', self.lineChanged)
self.draw()
return line
def lineChanged(self):
self.draw()
def draw(self):
print('Plotter: redrawing plot')
ax = self.__axis
ax.clear()
for line in self.__lines:
ax.plot(line.xdata,
line.ydata,
color=line.colour,
linewidth=line.width,
label=line.name)
ax.legend()
```
%% Cell type:markdown id: tags:
Let's create a
`Plotter`
object, and add a couple of lines to it (note that
the
`matplotlib`
plot will open in a separate window):
%% Cell type:code id: tags:
```
fig = plt.figure()
ax = fig.add_subplot(111)
plotter = Plotter(ax)
l1 = plotter.addData(np.sin(np.linspace(0, 6 * np.pi, 50)))
l2 = plotter.addData(np.cos(np.linspace(0, 6 * np.pi, 50)))
fig.show()
```
%% Cell type:markdown id: tags:
Now, when we change the properties of our
`Line`
instances, the plot will be
automatically updated:
%% Cell type:code id: tags:
```
l1.colour = '#ff0000'
l2.colour = '#00ff00'
l1.width = 2
l2.width = 2
l1.name = 'sine'
l2.name = 'cosine'
```
%% Cell type:markdown id: tags:
Pretty cool! However, this seems very inefficient - every time we change the
properties of a
`Line`
, the
`Plotter`
will refresh the plot. If we were
plotting large amounts of data, this would be unacceptable, as plotting would
simply take too long.
Wouldn't it be nice if we were able to perform batch-updates of
`Line`
properties, and only refresh the plot when we are done? Let's add an extra
method to the
`Plotter`
class:
%% Cell type:code id: tags:
```
import contextlib
class Plotter(object):
def __init__(self, axis):
self.__axis = axis
self.__lines = []
self.__holdUpdates = False
def addData(self, data):
line = Line(data)
self.__lines.append(line)
line.register('plot', self.lineChanged)
if not self.__holdUpdates:
self.draw()
return line
def lineChanged(self):
if not self.__holdUpdates:
self.draw()
def draw(self):
print('Plotter: redrawing plot')
ax = self.__axis
ax.clear()
for line in self.__lines:
ax.plot(line.xdata,
line.ydata,
color=line.colour,
linewidth=line.width,
label=line.name)
ax.legend()
@contextlib.contextmanager
def holdUpdates(self):
self.__holdUpdates = True
try:
yield
self.draw()
finally:
self.__holdUpdates = False
```
%% Cell type:markdown id: tags:
This new
`holdUpdates`
method allows us to temporarily suppress notifications
from all
`Line`
instances. So now, we can update many
`Line`
properties
without performing any redundant redraws:
%% Cell type:code id: tags:
```
fig = plt.figure()
ax = fig.add_subplot(111)
plotter = Plotter(ax)
plt.show()
with plotter.holdUpdates():
l1 = plotter.addData(np.sin(np.linspace(0, 6 * np.pi, 50)))
l2 = plotter.addData(np.cos(np.linspace(0, 6 * np.pi, 50)))
l1.colour = '#0000ff'
l2.colour = '#ffff00'
l1.width = 1
l2.width = 1
l1.name = '$sin(x)$'
l2.name = '$cos(x)$'
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## Nesting context managers
## Nesting context managers
## Useful references
## Useful references
*
[
Context manager classes
](
https://docs.python.org/3.5/reference/datamodel.html#context-managers
)
*
[
Context manager classes
](
https://docs.python.org/3.5/reference/datamodel.html#context-managers
)
*
The
[
`contextlib` module
](
https://docs.python.org/3.5/library/contextlib.html
)
*
The
[
`contextlib` module
](
https://docs.python.org/3.5/library/contextlib.html
)
...
...
This diff is collapsed.
Click to expand it.
advanced_topics/05_context_managers.md
+
232
−
4
View file @
076a73f9
...
@@ -271,8 +271,8 @@ with MyContextManager():
...
@@ -271,8 +271,8 @@ with MyContextManager():
In fact, there is another way to create context managers in Python. The
In fact, there is another way to create context managers in Python. The
built-in
[
`contextlib`
built-in
[
`contextlib`
module
](
https://docs.python.org/3.5/library/contextlib.html#contextlib.contextmanager
)
module
](
https://docs.python.org/3.5/library/contextlib.html#contextlib.contextmanager
)
has a decorator called
`@contextmanager`
, which allows us to turn __any
__
has a decorator called
`@contextmanager`
, which allows us to turn __any
function into a context manager. The only requirement is that the function
function
__
into a context manager. The only requirement is that the function
must have a
`yield`
statement
<sup>
1
</sup>
. So we could rewrite our
`TempDir`
must have a
`yield`
statement
<sup>
1
</sup>
. So we could rewrite our
`TempDir`
class from above as a function:
class from above as a function:
...
@@ -319,12 +319,240 @@ print('Back in directory: {}'.format(os.getcwd()))
...
@@ -319,12 +319,240 @@ print('Back in directory: {}'.format(os.getcwd()))
> beyond the scope of this practical.
> beyond the scope of this practical.
## Methods as context managers
Since it is possible to write a function which is a context manager, it is of
Since it is possible to write a function which is a context manager, it is of
course also possible to write a _method_ which is a context manager.
course also possible to write a _method_ which is a context manager. Let's
play with another example. We have a
`Notifier`
class which can be used to
notify interested listeners when an event occurs. Listeners can be registered
for notification via the
`register`
method:
```
from collections import OrderedDict
class Notifier(object):
def __init__(self):
super().__init__()
self.listeners = OrderedDict()
def register(self, name, func):
self.listeners[name] = func
def notify(self):
for listener in self.listeners.values():
listener()
```
```
TODO suppress notification example
Now, let's build a little plotting application. First of all, we have a
`Line`
class, which represents a line plot. The
`Line`
class is a sub-class of
`Notifier`
, so whenever its display properties (
`colour`
,
`width`
, or
`name`
)
change, it emits a notification, and whatever is drawing it can refresh the
display:
```
import numpy as np
class Line(Notifier):
def __init__(self, data):
super().__init__()
self.__data = data
self.__colour = '#000000'
self.__width = 1
self.__name = 'line'
@property
def xdata(self):
return np.arange(len(self.__data))
@property
def ydata(self):
return np.copy(self.__data)
@property
def colour(self):
return self.__colour
@colour.setter
def colour(self, newColour):
self.__colour = newColour
print('Line: colour changed: {}'.format(newColour))
self.notify()
@property
def width(self):
return self.__width
@width.setter
def width(self, newWidth):
self.__width = newWidth
print('Line: width changed: {}'.format(newWidth))
self.notify()
@property
def name(self):
return self.__name
@name.setter
def name(self, newName):
self.__name = newName
print('Line: name changed: {}'.format(newName))
self.notify()
```
Now let's write a
`Plotter`
class, which can plot one or more
`Line`
instances:
```
import matplotlib.pyplot as plt
# this line is only necessary when
# working in jupyer notebook/ipython
%matplotlib
class Plotter(object):
def __init__(self, axis):
self.__axis = axis
self.__lines = []
def addData(self, data):
line = Line(data)
self.__lines.append(line)
line.register('plot', self.lineChanged)
self.draw()
return line
def lineChanged(self):
self.draw()
def draw(self):
print('Plotter: redrawing plot')
ax = self.__axis
ax.clear()
for line in self.__lines:
ax.plot(line.xdata,
line.ydata,
color=line.colour,
linewidth=line.width,
label=line.name)
ax.legend()
```
Let's create a
`Plotter`
object, and add a couple of lines to it (note that
the
`matplotlib`
plot will open in a separate window):
```
fig = plt.figure()
ax = fig.add_subplot(111)
plotter = Plotter(ax)
l1 = plotter.addData(np.sin(np.linspace(0, 6 * np.pi, 50)))
l2 = plotter.addData(np.cos(np.linspace(0, 6 * np.pi, 50)))
fig.show()
```
Now, when we change the properties of our
`Line`
instances, the plot will be
automatically updated:
```
l1.colour = '#ff0000'
l2.colour = '#00ff00'
l1.width = 2
l2.width = 2
l1.name = 'sine'
l2.name = 'cosine'
```
Pretty cool! However, this seems very inefficient - every time we change the
properties of a
`Line`
, the
`Plotter`
will refresh the plot. If we were
plotting large amounts of data, this would be unacceptable, as plotting would
simply take too long.
Wouldn't it be nice if we were able to perform batch-updates of
`Line`
properties, and only refresh the plot when we are done? Let's add an extra
method to the
`Plotter`
class:
```
import contextlib
class Plotter(object):
def __init__(self, axis):
self.__axis = axis
self.__lines = []
self.__holdUpdates = False
def addData(self, data):
line = Line(data)
self.__lines.append(line)
line.register('plot', self.lineChanged)
if not self.__holdUpdates:
self.draw()
return line
def lineChanged(self):
if not self.__holdUpdates:
self.draw()
def draw(self):
print('Plotter: redrawing plot')
ax = self.__axis
ax.clear()
for line in self.__lines:
ax.plot(line.xdata,
line.ydata,
color=line.colour,
linewidth=line.width,
label=line.name)
ax.legend()
@contextlib.contextmanager
def holdUpdates(self):
self.__holdUpdates = True
try:
yield
self.draw()
finally:
self.__holdUpdates = False
```
This new
`holdUpdates`
method allows us to temporarily suppress notifications
from all
`Line`
instances. So now, we can update many
`Line`
properties
without performing any redundant redraws:
```
fig = plt.figure()
ax = fig.add_subplot(111)
plotter = Plotter(ax)
plt.show()
with plotter.holdUpdates():
l1 = plotter.addData(np.sin(np.linspace(0, 6 * np.pi, 50)))
l2 = plotter.addData(np.cos(np.linspace(0, 6 * np.pi, 50)))
l1.colour = '#0000ff'
l2.colour = '#ffff00'
l1.width = 1
l2.width = 1
l1.name = '$sin(x)$'
l2.name = '$cos(x)$'
```
```
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment