Skip to content
Snippets Groups Projects
Commit 1e6a4277 authored by Paul McCarthy's avatar Paul McCarthy :mountain_bicyclist:
Browse files

operator overloading practical finished

parent 9f3718f0
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
advanced_topics/operator_overloading.ipynb
+ 206
− 63
View file @ 1e6a4277
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Operator overloading # Operator overloading
> This practical assumes you are familiar with the basics of object-oriented > This practical assumes you are familiar with the basics of object-oriented
> programming in Python. > programming in Python.
Operator overloading, in an object-oriented programming language, is the Operator overloading, in an object-oriented programming language, is the
process of customising the behaviour of _operators_ (e.g. `+`, `*`, `/` and process of customising the behaviour of _operators_ (e.g. `+`, `*`, `/` and
`-`) on user-defined types. This practical aims to show you that operator `-`) on user-defined types. This practical aims to show you that operator
overloading is __very__ easy to do in Python. overloading is __very__ easy to do in Python.
This practical gives a brief overview of the operators which you may be most
interested in implementing. However, there are many operators (and other
special methods) which you can support in your own classes - the [official
documentation](https://docs.python.org/3.5/reference/datamodel.html#basic-customization)
is the best reference if you are interested in learning more.
* [Overview](#overview)
* [Arithmetic operators](#arithmetic-operators)
* [Equality and comparison operators](#equality-and-comparison-operators)
* [The indexing operator `[]`](#the-indexing-operator)
* [The call operator `()`](#the-call-operator)
* [The dot operator `.`](#the-dot-operator)
<a class="anchor" id="overview"></a>
## Overview ## Overview
In Python, when you add two numbers together: In Python, when you add two numbers together:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
a = 5 a = 5
b = 10 b = 10
r = a + b r = a + b
print(r) print(r)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
What actually goes on behind the scenes is this: What actually goes on behind the scenes is this:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
r = a.__add__(b) r = a.__add__(b)
print(r) print(r)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
In other words, whenever you use the `+` operator on two variables (the In other words, whenever you use the `+` operator on two variables (the
operands to the `+` operator), the Python interpreter calls the `__add__` operands to the `+` operator), the Python interpreter calls the `__add__`
method of the first operand (`a`), and passes the second operand (`b`) as an method of the first operand (`a`), and passes the second operand (`b`) as an
argument. argument.
So it is very easy to use the `+` operator with our own classes - all we have So it is very easy to use the `+` operator with our own classes - all we have
to do is implement a method called `__add__`. to do is implement a method called `__add__`.
<a class="anchor" id="arithmetic-operators"></a>
## Arithmetic operators ## Arithmetic operators
Let's play with an example - a class which represents a 2D vector: Let's play with an example - a class which represents a 2D vector:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
class Vector(object): class Vector2D(object):
def __init__(self, x, y): def __init__(self, x, y):
self.x = x self.x = x
self.y = y self.y = y
def __str__(self): def __str__(self):
return 'Vector({}, {})'.format(self.x, self.y) return 'Vector2D({}, {})'.format(self.x, self.y)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> Note that we have implemented the special `__str__` method, which allows our > Note that we have implemented the special `__str__` method, which allows our
> `Vector` instances to be converted into strings. > `Vector2D` instances to be converted into strings.
If we try to use the `+` operator on this class, we are bound to get an error: If we try to use the `+` operator on this class, we are bound to get an error:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
v1 = Vector(2, 3) v1 = Vector2D(2, 3)
v2 = Vector(4, 5) v2 = Vector2D(4, 5)
print(v1 + v2) print(v1 + v2)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
But all we need to do to support the `+` operator is to implement a method But all we need to do to support the `+` operator is to implement a method
called `__add__`: called `__add__`:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
class Vector(object): class Vector2D(object):
def __init__(self, x, y): def __init__(self, x, y):
self.x = x self.x = x
self.y = y self.y = y
def __str__(self): def __str__(self):
return 'Vector({}, {})'.format(self.x, self.y) return 'Vector2D({}, {})'.format(self.x, self.y)
def __add__(self, other): def __add__(self, other):
return Vector(self.x + other.x, return Vector2D(self.x + other.x,
self.y + other.y) self.y + other.y)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
And now we can use `+` on `Vector` objects - it's that easy: And now we can use `+` on `Vector2D` objects - it's that easy:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
v1 = Vector(2, 3) v1 = Vector2D(2, 3)
v2 = Vector(4, 5) v2 = Vector2D(4, 5)
print('{} + {} = {}'.format(v1, v2, v1 + v2)) print('{} + {} = {}'.format(v1, v2, v1 + v2))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Our `__add__` method creates and returns a new `Vector` which contains the sum Our `__add__` method creates and returns a new `Vector2D` which contains the
of the `x` and `y` components of the `Vector` on which it is called, and the sum of the `x` and `y` components of the `Vector2D` on which it is called, and
`Vector` which is passed in. We could also make the `__add__` method work the `Vector2D` which is passed in. We could also make the `__add__` method
with scalars, by extending its definition a bit: work with scalars, by extending its definition a bit:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
class Vector(object): class Vector2D(object):
def __init__(self, x, y): def __init__(self, x, y):
self.x = x self.x = x
self.y = y self.y = y
def __add__(self, other): def __add__(self, other):
if isinstance(other, Vector): if isinstance(other, Vector2D):
return Vector(self.x + other.x, return Vector2D(self.x + other.x,
self.y + other.y) self.y + other.y)
else: else:
return Vector(self.x + other, self.y + other) return Vector2D(self.x + other, self.y + other)
def __str__(self): def __str__(self):
return 'Vector({}, {})'.format(self.x, self.y) return 'Vector2D({}, {})'.format(self.x, self.y)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
So now we can add both `Vectors` and scalars numbers together: So now we can add both `Vector2D` instances and scalars numbers together:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
v1 = Vector(2, 3) v1 = Vector2D(2, 3)
v2 = Vector(4, 5) v2 = Vector2D(4, 5)
n = 6 n = 6
print('{} + {} = {}'.format(v1, v2, v1 + v2)) print('{} + {} = {}'.format(v1, v2, v1 + v2))
print('{} + {} = {}'.format(v1, n, v1 + n)) print('{} + {} = {}'.format(v1, n, v1 + n))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Other nuemric and logical operators can be supported by implementing the Other numeric and logical operators can be supported by implementing the
appropriate method, for example: appropriate method, for example:
- Multiplication (`*`): `__mul__` - Multiplication (`*`): `__mul__`
- Division (`/`): `__div__` - Division (`/`): `__div__`
- Negation (`-`): `__neg__` - Negation (`-`): `__neg__`
- In-place addition (`+=`): `__iadd__` - In-place addition (`+=`): `__iadd__`
- Exclusive or (`^`): `__xor__` - Exclusive or (`^`): `__xor__`
Take a look at the [official Take a look at the [official
documentation](https://docs.python.org/3.5/reference/datamodel.html#emulating-numeric-types) documentation](https://docs.python.org/3.5/reference/datamodel.html#emulating-numeric-types)
for a full list of the arithmetic and logical operators that your classes can for a full list of the arithmetic and logical operators that your classes can
support. support.
<a class="anchor" id="equality-and-comparison-operators"></a>
## Equality and comparison operators ## Equality and comparison operators
Adding support for equality (`==`, `!=`) and comparison (e.g. `>=`) operators Adding support for equality (`==`, `!=`) and comparison (e.g. `>=`) operators
is just as easy. Imagine that we have a class called `Label`, which represents is just as easy. Imagine that we have a class called `Label`, which represents
a label in a lookup table. Our `Label` has an integer label, a name, and an a label in a lookup table. Our `Label` has an integer label, a name, and an
RGB colour: RGB colour:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
class Label(object): class Label(object):
def __init__(self, label, name, colour): def __init__(self, label, name, colour):
self.label = label self.label = label
self.name = name self.name = name
self.colour = colour self.colour = colour
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
In order to ensure that a list of `Label` objects is ordered by their label In order to ensure that a list of `Label` objects is ordered by their label
values, we can implement a set of functions, so that `Label` classes can be values, we can implement a set of functions, so that `Label` classes can be
compared using the standard comparison operators: compared using the standard comparison operators:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import functools import functools
# Don't worry about this statement
# just yet - it is explained below
@functools.total_ordering @functools.total_ordering
class Label(object): class Label(object):
def __init__(self, label, name, colour): def __init__(self, label, name, colour):
self.label = label self.label = label
self.name = name self.name = name
self.colour = colour self.colour = colour
def __str__(self): def __str__(self):
rgb = ''.join(['{:02x}'.format(c) for c in self.colour]) rgb = ''.join(['{:02x}'.format(c) for c in self.colour])
return 'Label({}, {}, #{})'.format(self.label, self.name, rgb) return 'Label({}, {}, #{})'.format(self.label, self.name, rgb)
def __repr__(self): def __repr__(self):
return str(self) return str(self)
# implement Label == Label
def __eq__(self, other): def __eq__(self, other):
return self.label == other.label return self.label == other.label
# implement Label < Label
def __lt__(self, other): def __lt__(self, other):
return self.label < other.label return self.label < other.label
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> We also added `__str__` and `__repr__` methods to the `Label` class so that > We also added `__str__` and `__repr__` methods to the `Label` class so that
> `Label` instances will be printed nicely. > `Label` instances will be printed nicely.
Now we can compare and sort our `Label` instances: Now we can compare and sort our `Label` instances:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
l1 = Label(1, 'Parietal', (255, 0, 0)) l1 = Label(1, 'Parietal', (255, 0, 0))
l2 = Label(2, 'Occipital', ( 0, 255, 0)) l2 = Label(2, 'Occipital', ( 0, 255, 0))
l3 = Label(3, 'Temporal', ( 0, 0, 255)) l3 = Label(3, 'Temporal', ( 0, 0, 255))
print('{} > {}: {}'.format(l1, l2, l1 > l2)) print('{} > {}: {}'.format(l1, l2, l1 > l2))
print('{} < {}: {}'.format(l1, l3, l1 < l3)) print('{} < {}: {}'.format(l1, l3, l1 < l3))
print('{} != {}: {}'.format(l2, l3, l2 != l3)) print('{} != {}: {}'.format(l2, l3, l2 != l3))
print(sorted((l3, l1, l2))) print(sorted((l3, l1, l2)))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The `@functools.total_ordering` is a convenience decorator which, given a The
class that implements equality and a single comparison function (`__lt__` in [`@functools.total_ordering`](https://docs.python.org/3.5/library/functools.html#functools.total_ordering)
the above code), will "fill in" the remainder of the comparison operators. If is a convenience
you need very specific or complicated behaviour, then you can provide methods [decorator](https://docs.python.org/3.5/glossary.html#term-decorator) which,
for _all_ of the comparison operators, e.g. `__gt__` for `>`, `__ge__` for given a class that implements equality and a single comparison function
`>=`, etc.). (`__lt__` in the above code), will "fill in" the remainder of the comparison
operators. If you need very specific or complicated behaviour, then you can
provide methods for _all_ of the comparison operators, e.g. `__gt__` for `>`,
`__ge__` for `>=`, etc.).
> Decorators are introduced in another practical.
But if you just want the operators to work in the conventional manner, you can But if you just want the operators to work in the conventional manner, you can
just use the `@functools.total_ordering` decorator, and provide `__eq__`, and simply use the `@functools.total_ordering` decorator, and provide `__eq__`,
just one of `__lt__`, `__le__`, `__gt__` or `__ge__`. and just one of `__lt__`, `__le__`, `__gt__` or `__ge__`.
Refer to the [official Refer to the [official
documentation](https://docs.python.org/3.5/reference/datamodel.html#object.__lt__) documentation](https://docs.python.org/3.5/reference/datamodel.html#object.__lt__)
for all of the details on supporting comparison operators. for all of the details on supporting comparison operators.
> You may see the `__cmp__` method in older code bases - this provides a > You may see the `__cmp__` method in older code bases - this provides a
> C-style comparison function which returns `<0`, `0`, or `>0` based on > C-style comparison function which returns `<0`, `0`, or `>0` based on
> comparing two items. This has been superseded by the rich > comparing two items. This has been superseded by the rich comparison
> comparison operators and is no longer used in Python 3. > operators introduced here, and is no longer supported in Python 3.
<a class="anchor" id="the-indexing-operator"></a>
## The indexing operator `[]` ## The indexing operator `[]`
The indexing operator (`[]`) is generally used by "container" types, such as The indexing operator (`[]`) is generally used by "container" types, such as
the built-in `list` and `dict` classes. the built-in `list` and `dict` classes.
At its essence, there are only three types of behaviours that are possible At its essence, there are only three types of behaviours that are possible
with the `[]` operator. All that is needed to support them are to implement with the `[]` operator. All that is needed to support them are to implement
three special methods in your class, regardless of whether your class will be three special methods in your class, regardless of whether your class will be
indexed by sequential integers (like a `list`) or by hashable values (like a indexed by sequential integers (like a `list`) or by
`dict`): [hashable](https://docs.python.org/3.5/glossary.html#term-hashable) values
(like a `dict`):
- __Retrieval__ is performed by the `__getitem__` method - __Retrieval__ is performed by the `__getitem__` method
- __Assignment__ is performed by the `__setitem__` method - __Assignment__ is performed by the `__setitem__` method
- __Deletion__ is performed by the `__delitem__` method - __Deletion__ is performed by the `__delitem__` method
Note that, if you implement these methods in your own class, there is no Note that, if you implement these methods in your own class, there is no
requirement for them to actually provide any form. However if you don't, you requirement for them to actually provide any form of data storage or
will probably confuse users of your code - make sure you explain it all in retrieval. However if you don't, you will probably confuse users of your code
your comprehensive documentation! who are used to how the `list` and `dict` types work. Whenever you deviate
from conventional behaviour, make sure you explain it well in your
documentation!
The following contrived example demonstrates all three behaviours: The following contrived example demonstrates all three behaviours:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
class TwoTimes(object): class TwoTimes(object):
def __init__(self): def __init__(self):
self.__deleted = set() self.__deleted = set()
self.__assigned = {} self.__assigned = {}
def __getitem__(self, key): def __getitem__(self, key):
if key in self.__deleted: if key in self.__deleted:
raise KeyError('{} has been deleted!') raise KeyError('{} has been deleted!'.format(key))
elif key in self.__assigned: elif key in self.__assigned:
return self.__assigned[key] return self.__assigned[key]
else: else:
return key * 2 return key * 2
def __setitem__(self, key, value): def __setitem__(self, key, value):
self.__assigned[key] = value self.__assigned[key] = value
def __delitem__(self, key): def __delitem__(self, key):
self.__deleted.add(key) self.__deleted.add(key)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Guess what happens whenever we index a `TwoTimes` object: Guess what happens whenever we index a `TwoTimes` object:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
tt = TwoTimes() tt = TwoTimes()
print('TwoTimes[{}] = {}'.format(2, tt[2])) print('TwoTimes[{}] = {}'.format(2, tt[2]))
print('TwoTimes[{}] = {}'.format(6, tt[6])) print('TwoTimes[{}] = {}'.format(6, tt[6]))
print('TwoTimes[{}] = {}'.format('abc', tt['abc'])) print('TwoTimes[{}] = {}'.format('abc', tt['abc']))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
For some unknown reason, the `TwoTimes` class allows us to override the value The `TwoTimes` class allows us to override the value for a specific key:
for a specific key:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
print(tt[4]) print(tt[4])
tt[4] = 'this is not 4 * 4' tt[4] = 'this is not 4 * 4'
print(tt[4]) print(tt[4])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
And we can also "delete" keys: And we can also "delete" keys:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
print(tt['12345']) print(tt['12345'])
del tt['12345'] del tt['12345']
# this is going to raise an error # this is going to raise an error
print(tt['12345']) print(tt['12345'])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
If you wish to support the Python `start:stop:step` [slice If you wish to support the Python `start:stop:step` [slice
notation](https://www.pythoncentral.io/how-to-slice-listsarrays-and-tuples-in-python/), notation](https://docs.python.org/3.5/library/functions.html#slice), you
you simply need to write your `__getitem__` and `__setitem__` methods so that they simply need to write your `__getitem__` and `__setitem__` methods so that they
can detect `slice` objects: can detect `slice` objects:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
class TwoTimes(object): class TwoTimes(object):
def __init__(self, max): def __init__(self, max):
self.__max = max self.__max = max
def __getitem__(self, key): def __getitem__(self, key):
if isinstance(key, slice): if isinstance(key, slice):
start = key.start or 0 start = key.start or 0
stop = key.stop or self.__max stop = key.stop or self.__max
step = key.step or 1 step = key.step or 1
else: else:
start = key start = key
stop = key + 1 stop = key + 1
step = 1 step = 1
return [i * 2 for i in range(start, stop, step)] return [i * 2 for i in range(start, stop, step)]
``` ```
%% Cell type:markdown id: tags:
Now we can "slice" a `TwoTimes` instance:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
tt = TwoTimes(10) tt = TwoTimes(10)
print(tt[5]) print(tt[5])
print(tt[3:7]) print(tt[3:7])
print(tt[::2]) print(tt[::2])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> It is possible to sub-class the built-in `list` and `dict` classes if you > It is possible to sub-class the built-in `list` and `dict` classes if you
> wish to extend their functionality in some way. However, if you are writing > wish to extend their functionality in some way. However, if you are writing
> a class that should mimic the one of the `list` or `dict` classes, but work > a class that should mimic the one of the `list` or `dict` classes, but work
> in a different way internally (e.g. a `dict`-like object which uses a > in a different way internally (e.g. a `dict`-like object which uses a
> different hashing algorithm), the `Sequence` and `MutableMapping` classes > different hashing algorithm), the `Sequence` and `MutableMapping` classes
> are [a better choice](https://stackoverflow.com/a/7148602) - you can find > are [a better choice](https://stackoverflow.com/a/7148602) - you can find
> them in the > them in the
> [`collections.abc`](https://docs.python.org/3.5/library/collections.abc.html) > [`collections.abc`](https://docs.python.org/3.5/library/collections.abc.html)
> module. > module.
<a class="anchor" id="the-call-operator"></a>
## The call operator `()` ## The call operator `()`
Remember how everything in Python is an object, even functions? When you call Remember how everything in Python is an object, even functions? When you call
a function, a method called `__call__` is called on the function object. We can a function, a method called `__call__` is called on the function object. We can
implement the `__call__` method on our own class, which will allow us to "call" implement the `__call__` method on our own class, which will allow us to "call"
objects as if they are functions. objects as if they are functions.
For example, the `TimedFunction` class allows us to calculate the execution For example, the `TimedFunction` class allows us to calculate the execution
time of any function: time of any function:
%% Cell type:code id: tags:
```
import time
class TimedFunction(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print('Timing {}...'.format(self.func.__name__))
start = time.time()
retval = self.func(*args, **kwargs)
end = time.time()
print('Elapsed time: {:0.2f} seconds'.format(end - start))
return retval
```
%% Cell type:markdown id: tags:
Let's see how the `TimedFunction` behaves:
%% Cell type:code id: tags:
```
import numpy as np
import numpy.linalg as npla
def inverse(data):
return npla.inv(data)
tf = TimedFunction(inverse)
data = np.random.random((5000, 5000))
# Wait a few seconds after
# running this code block!
inv = tf(data)
```
%% Cell type:markdown id: tags:
> The `TimedFunction` class is conceptually very similar to a
> [decorator](https://docs.python.org/3.5/glossary.html#term-decorator) -
> decorators are covered in another practical.
<a class="anchor" id="the-dot-operator"></a>
## The dot operator `.` ## The dot operator `.`
Python allows us to override the `.` (dot) operator which is used to access Python allows us to override the `.` (dot) operator which is used to access
the attributes and methods of an object. attributes and methods of an the attributes and methods of an object. This is very powerful, but is also
object. This is a fairly niche feature, and you need to be careful that you quite a niche feature, and it is easy to trip yourself up, so if you wish to
don't unintentionally introduce recursive attribute lookups into your code. use this in your own project, make sure that you carefully read (and
understand) [the
documentation](https://docs.python.org/3.5/reference/datamodel.html#customizing-attribute-access),
and test your code comprehensively!
For this example, we need a little background information. OpenGL includes
the native data types `vec2`, `vec3`, and `vec4`, which can be used to
represent 2, 3, or 4 component vectors respectively. These data types have a
neat feature called [_swizzling_][glslref], which allows you to access any
component (`x`,`y`, `z`, `w` for vectors, or `r`, `g`, `b`, `a` for colours)
in any order, with a syntax similar to attribute access in Python.
[glslref]: https://www.khronos.org/opengl/wiki/Data_Type_(GLSL)#Swizzling
So here is an example which implements this swizzle-style attribute access on
a class called `Vector`, in which we have customised the behaviour of the `.`
operator:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
class Vector(object): class Vector(object):
def __init__(self, xyz): def __init__(self, xyz):
self.__xyz = list(xyz) self.__xyz = list(xyz)
def __str__(self): def __str__(self):
return 'Vector({})'.format(', '.join(self.__xyz)) return 'Vector({})'.format(self.__xyz)
def __getattr__(self, key): def __getattr__(self, key):
# Swizzling behaviour only occurs when
# the attribute name is entirely comprised
# of 'x', 'y', and 'z'.
if not all([c in 'xyz' for c in key]):
raise AttributeError(key)
key = ['xyz'.index(c) for c in key] key = ['xyz'.index(c) for c in key]
return [self.__xyz[c] for c in key] return [self.__xyz[c] for c in key]
def __setattr__(self, key, value): def __setattr__(self, key, value):
pass # key = ['xyz'.index(c) for c in key]
# Restrict swizzling behaviour as above
if not all([c in 'xyz' for c in key]):
return super().__setattr__(key, value)
if len(key) == 1:
value = (value,)
idxs = ['xyz'.index(c) for c in key]
for i, v in sorted(zip(idxs, value)):
self.__xyz[i] = v
``` ```
%% Cell type:markdown id: tags:
And here it is in action:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
v = Vector((4, 5, 6)) v = Vector((1, 2, 3))
print('v: ', v)
print('xyz: ', v.xyz) print('xyz: ', v.xyz)
print('yz: ', v.yz) print('yz: ', v.zy)
print('zxy: ', v.xzy) print('xx: ', v.xx)
print('y: ', v.y)
```
%% Cell type:markdown id: tags: v.xz = 10, 30
print(v)
## Other special methods v.y = 20
print(v)
```
......
advanced_topics/operator_overloading.md
+ 181
− 60
View file @ 1e6a4277
...@@ -11,6 +11,22 @@ process of customising the behaviour of _operators_ (e.g. `+`, `*`, `/` and ...@@ -11,6 +11,22 @@ process of customising the behaviour of _operators_ (e.g. `+`, `*`, `/` and
overloading is __very__ easy to do in Python. overloading is __very__ easy to do in Python.
This practical gives a brief overview of the operators which you may be most
interested in implementing. However, there are many operators (and other
special methods) which you can support in your own classes - the [official
documentation](https://docs.python.org/3.5/reference/datamodel.html#basic-customization)
is the best reference if you are interested in learning more.
* [Overview](#overview)
* [Arithmetic operators](#arithmetic-operators)
* [Equality and comparison operators](#equality-and-comparison-operators)
* [The indexing operator `[]`](#the-indexing-operator)
* [The call operator `()`](#the-call-operator)
* [The dot operator `.`](#the-dot-operator)
<a class="anchor" id="overview"></a>
## Overview ## Overview
...@@ -44,6 +60,7 @@ So it is very easy to use the `+` operator with our own classes - all we have ...@@ -44,6 +60,7 @@ So it is very easy to use the `+` operator with our own classes - all we have
to do is implement a method called `__add__`. to do is implement a method called `__add__`.
<a class="anchor" id="arithmetic-operators"></a>
## Arithmetic operators ## Arithmetic operators
...@@ -51,26 +68,26 @@ Let's play with an example - a class which represents a 2D vector: ...@@ -51,26 +68,26 @@ Let's play with an example - a class which represents a 2D vector:
``` ```
class Vector(object): class Vector2D(object):
def __init__(self, x, y): def __init__(self, x, y):
self.x = x self.x = x
self.y = y self.y = y
def __str__(self): def __str__(self):
return 'Vector({}, {})'.format(self.x, self.y) return 'Vector2D({}, {})'.format(self.x, self.y)
``` ```
> Note that we have implemented the special `__str__` method, which allows our > Note that we have implemented the special `__str__` method, which allows our
> `Vector` instances to be converted into strings. > `Vector2D` instances to be converted into strings.
If we try to use the `+` operator on this class, we are bound to get an error: If we try to use the `+` operator on this class, we are bound to get an error:
``` ```
v1 = Vector(2, 3) v1 = Vector2D(2, 3)
v2 = Vector(4, 5) v2 = Vector2D(4, 5)
print(v1 + v2) print(v1 + v2)
``` ```
...@@ -80,60 +97,60 @@ called `__add__`: ...@@ -80,60 +97,60 @@ called `__add__`:
``` ```
class Vector(object): class Vector2D(object):
def __init__(self, x, y): def __init__(self, x, y):
self.x = x self.x = x
self.y = y self.y = y
def __str__(self): def __str__(self):
return 'Vector({}, {})'.format(self.x, self.y) return 'Vector2D({}, {})'.format(self.x, self.y)
def __add__(self, other): def __add__(self, other):
return Vector(self.x + other.x, return Vector2D(self.x + other.x,
self.y + other.y) self.y + other.y)
``` ```
And now we can use `+` on `Vector` objects - it's that easy: And now we can use `+` on `Vector2D` objects - it's that easy:
``` ```
v1 = Vector(2, 3) v1 = Vector2D(2, 3)
v2 = Vector(4, 5) v2 = Vector2D(4, 5)
print('{} + {} = {}'.format(v1, v2, v1 + v2)) print('{} + {} = {}'.format(v1, v2, v1 + v2))
``` ```
Our `__add__` method creates and returns a new `Vector` which contains the sum Our `__add__` method creates and returns a new `Vector2D` which contains the
of the `x` and `y` components of the `Vector` on which it is called, and the sum of the `x` and `y` components of the `Vector2D` on which it is called, and
`Vector` which is passed in. We could also make the `__add__` method work the `Vector2D` which is passed in. We could also make the `__add__` method
with scalars, by extending its definition a bit: work with scalars, by extending its definition a bit:
``` ```
class Vector(object): class Vector2D(object):
def __init__(self, x, y): def __init__(self, x, y):
self.x = x self.x = x
self.y = y self.y = y
def __add__(self, other): def __add__(self, other):
if isinstance(other, Vector): if isinstance(other, Vector2D):
return Vector(self.x + other.x, return Vector2D(self.x + other.x,
self.y + other.y) self.y + other.y)
else: else:
return Vector(self.x + other, self.y + other) return Vector2D(self.x + other, self.y + other)
def __str__(self): def __str__(self):
return 'Vector({}, {})'.format(self.x, self.y) return 'Vector2D({}, {})'.format(self.x, self.y)
``` ```
So now we can add both `Vectors` and scalars numbers together: So now we can add both `Vector2D` instances and scalars numbers together:
``` ```
v1 = Vector(2, 3) v1 = Vector2D(2, 3)
v2 = Vector(4, 5) v2 = Vector2D(4, 5)
n = 6 n = 6
print('{} + {} = {}'.format(v1, v2, v1 + v2)) print('{} + {} = {}'.format(v1, v2, v1 + v2))
...@@ -141,7 +158,7 @@ print('{} + {} = {}'.format(v1, n, v1 + n)) ...@@ -141,7 +158,7 @@ print('{} + {} = {}'.format(v1, n, v1 + n))
``` ```
Other nuemric and logical operators can be supported by implementing the Other numeric and logical operators can be supported by implementing the
appropriate method, for example: appropriate method, for example:
- Multiplication (`*`): `__mul__` - Multiplication (`*`): `__mul__`
...@@ -157,6 +174,7 @@ for a full list of the arithmetic and logical operators that your classes can ...@@ -157,6 +174,7 @@ for a full list of the arithmetic and logical operators that your classes can
support. support.
<a class="anchor" id="equality-and-comparison-operators"></a>
## Equality and comparison operators ## Equality and comparison operators
...@@ -183,7 +201,10 @@ compared using the standard comparison operators: ...@@ -183,7 +201,10 @@ compared using the standard comparison operators:
``` ```
import functools import functools
# Don't worry about this statement
# just yet - it is explained below
@functools.total_ordering @functools.total_ordering
class Label(object): class Label(object):
def __init__(self, label, name, colour): def __init__(self, label, name, colour):
self.label = label self.label = label
...@@ -197,13 +218,16 @@ class Label(object): ...@@ -197,13 +218,16 @@ class Label(object):
def __repr__(self): def __repr__(self):
return str(self) return str(self)
# implement Label == Label
def __eq__(self, other): def __eq__(self, other):
return self.label == other.label return self.label == other.label
# implement Label < Label
def __lt__(self, other): def __lt__(self, other):
return self.label < other.label return self.label < other.label
``` ```
> We also added `__str__` and `__repr__` methods to the `Label` class so that > We also added `__str__` and `__repr__` methods to the `Label` class so that
> `Label` instances will be printed nicely. > `Label` instances will be printed nicely.
...@@ -223,17 +247,23 @@ print(sorted((l3, l1, l2))) ...@@ -223,17 +247,23 @@ print(sorted((l3, l1, l2)))
``` ```
The `@functools.total_ordering` is a convenience decorator which, given a The
class that implements equality and a single comparison function (`__lt__` in [`@functools.total_ordering`](https://docs.python.org/3.5/library/functools.html#functools.total_ordering)
the above code), will "fill in" the remainder of the comparison operators. If is a convenience
you need very specific or complicated behaviour, then you can provide methods [decorator](https://docs.python.org/3.5/glossary.html#term-decorator) which,
for _all_ of the comparison operators, e.g. `__gt__` for `>`, `__ge__` for given a class that implements equality and a single comparison function
`>=`, etc.). (`__lt__` in the above code), will "fill in" the remainder of the comparison
operators. If you need very specific or complicated behaviour, then you can
provide methods for _all_ of the comparison operators, e.g. `__gt__` for `>`,
`__ge__` for `>=`, etc.).
> Decorators are introduced in another practical.
But if you just want the operators to work in the conventional manner, you can But if you just want the operators to work in the conventional manner, you can
just use the `@functools.total_ordering` decorator, and provide `__eq__`, and simply use the `@functools.total_ordering` decorator, and provide `__eq__`,
just one of `__lt__`, `__le__`, `__gt__` or `__ge__`. and just one of `__lt__`, `__le__`, `__gt__` or `__ge__`.
Refer to the [official Refer to the [official
...@@ -243,10 +273,11 @@ for all of the details on supporting comparison operators. ...@@ -243,10 +273,11 @@ for all of the details on supporting comparison operators.
> You may see the `__cmp__` method in older code bases - this provides a > You may see the `__cmp__` method in older code bases - this provides a
> C-style comparison function which returns `<0`, `0`, or `>0` based on > C-style comparison function which returns `<0`, `0`, or `>0` based on
> comparing two items. This has been superseded by the rich > comparing two items. This has been superseded by the rich comparison
> comparison operators and is no longer used in Python 3. > operators introduced here, and is no longer supported in Python 3.
<a class="anchor" id="the-indexing-operator"></a>
## The indexing operator `[]` ## The indexing operator `[]`
...@@ -257,8 +288,9 @@ the built-in `list` and `dict` classes. ...@@ -257,8 +288,9 @@ the built-in `list` and `dict` classes.
At its essence, there are only three types of behaviours that are possible At its essence, there are only three types of behaviours that are possible
with the `[]` operator. All that is needed to support them are to implement with the `[]` operator. All that is needed to support them are to implement
three special methods in your class, regardless of whether your class will be three special methods in your class, regardless of whether your class will be
indexed by sequential integers (like a `list`) or by hashable values (like a indexed by sequential integers (like a `list`) or by
`dict`): [hashable](https://docs.python.org/3.5/glossary.html#term-hashable) values
(like a `dict`):
- __Retrieval__ is performed by the `__getitem__` method - __Retrieval__ is performed by the `__getitem__` method
...@@ -267,9 +299,11 @@ indexed by sequential integers (like a `list`) or by hashable values (like a ...@@ -267,9 +299,11 @@ indexed by sequential integers (like a `list`) or by hashable values (like a
Note that, if you implement these methods in your own class, there is no Note that, if you implement these methods in your own class, there is no
requirement for them to actually provide any form. However if you don't, you requirement for them to actually provide any form of data storage or
will probably confuse users of your code - make sure you explain it all in retrieval. However if you don't, you will probably confuse users of your code
your comprehensive documentation! who are used to how the `list` and `dict` types work. Whenever you deviate
from conventional behaviour, make sure you explain it well in your
documentation!
The following contrived example demonstrates all three behaviours: The following contrived example demonstrates all three behaviours:
...@@ -284,7 +318,7 @@ class TwoTimes(object): ...@@ -284,7 +318,7 @@ class TwoTimes(object):
def __getitem__(self, key): def __getitem__(self, key):
if key in self.__deleted: if key in self.__deleted:
raise KeyError('{} has been deleted!') raise KeyError('{} has been deleted!'.format(key))
elif key in self.__assigned: elif key in self.__assigned:
return self.__assigned[key] return self.__assigned[key]
else: else:
...@@ -309,8 +343,7 @@ print('TwoTimes[{}] = {}'.format('abc', tt['abc'])) ...@@ -309,8 +343,7 @@ print('TwoTimes[{}] = {}'.format('abc', tt['abc']))
``` ```
For some unknown reason, the `TwoTimes` class allows us to override the value The `TwoTimes` class allows us to override the value for a specific key:
for a specific key:
``` ```
...@@ -333,8 +366,8 @@ print(tt['12345']) ...@@ -333,8 +366,8 @@ print(tt['12345'])
If you wish to support the Python `start:stop:step` [slice If you wish to support the Python `start:stop:step` [slice
notation](https://www.pythoncentral.io/how-to-slice-listsarrays-and-tuples-in-python/), notation](https://docs.python.org/3.5/library/functions.html#slice), you
you simply need to write your `__getitem__` and `__setitem__` methods so that they simply need to write your `__getitem__` and `__setitem__` methods so that they
can detect `slice` objects: can detect `slice` objects:
...@@ -357,6 +390,10 @@ class TwoTimes(object): ...@@ -357,6 +390,10 @@ class TwoTimes(object):
return [i * 2 for i in range(start, stop, step)] return [i * 2 for i in range(start, stop, step)]
``` ```
Now we can "slice" a `TwoTimes` instance:
``` ```
tt = TwoTimes(10) tt = TwoTimes(10)
...@@ -366,8 +403,6 @@ print(tt[::2]) ...@@ -366,8 +403,6 @@ print(tt[::2])
``` ```
> It is possible to sub-class the built-in `list` and `dict` classes if you > It is possible to sub-class the built-in `list` and `dict` classes if you
> wish to extend their functionality in some way. However, if you are writing > wish to extend their functionality in some way. However, if you are writing
> a class that should mimic the one of the `list` or `dict` classes, but work > a class that should mimic the one of the `list` or `dict` classes, but work
...@@ -379,6 +414,7 @@ print(tt[::2]) ...@@ -379,6 +414,7 @@ print(tt[::2])
> module. > module.
<a class="anchor" id="the-call-operator"></a>
## The call operator `()` ## The call operator `()`
...@@ -392,16 +428,77 @@ For example, the `TimedFunction` class allows us to calculate the execution ...@@ -392,16 +428,77 @@ For example, the `TimedFunction` class allows us to calculate the execution
time of any function: time of any function:
```
import time
class TimedFunction(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print('Timing {}...'.format(self.func.__name__))
start = time.time()
retval = self.func(*args, **kwargs)
end = time.time()
print('Elapsed time: {:0.2f} seconds'.format(end - start))
return retval
```
Let's see how the `TimedFunction` behaves:
```
import numpy as np
import numpy.linalg as npla
def inverse(data):
return npla.inv(data)
tf = TimedFunction(inverse)
data = np.random.random((5000, 5000))
# Wait a few seconds after
# running this code block!
inv = tf(data)
```
> The `TimedFunction` class is conceptually very similar to a
> [decorator](https://docs.python.org/3.5/glossary.html#term-decorator) -
> decorators are covered in another practical.
<a class="anchor" id="the-dot-operator"></a>
## The dot operator `.` ## The dot operator `.`
Python allows us to override the `.` (dot) operator which is used to access Python allows us to override the `.` (dot) operator which is used to access
the attributes and methods of an object. attributes and methods of an the attributes and methods of an object. This is very powerful, but is also
object. This is a fairly niche feature, and you need to be careful that you quite a niche feature, and it is easy to trip yourself up, so if you wish to
don't unintentionally introduce recursive attribute lookups into your code. use this in your own project, make sure that you carefully read (and
understand) [the
documentation](https://docs.python.org/3.5/reference/datamodel.html#customizing-attribute-access),
and test your code comprehensively!
For this example, we need a little background information. OpenGL includes
the native data types `vec2`, `vec3`, and `vec4`, which can be used to
represent 2, 3, or 4 component vectors respectively. These data types have a
neat feature called [_swizzling_][glslref], which allows you to access any
component (`x`,`y`, `z`, `w` for vectors, or `r`, `g`, `b`, `a` for colours)
in any order, with a syntax similar to attribute access in Python.
[glslref]: https://www.khronos.org/opengl/wiki/Data_Type_(GLSL)#Swizzling
So here is an example which implements this swizzle-style attribute access on
a class called `Vector`, in which we have customised the behaviour of the `.`
operator:
``` ```
...@@ -410,25 +507,49 @@ class Vector(object): ...@@ -410,25 +507,49 @@ class Vector(object):
self.__xyz = list(xyz) self.__xyz = list(xyz)
def __str__(self): def __str__(self):
return 'Vector({})'.format(', '.join(self.__xyz)) return 'Vector({})'.format(self.__xyz)
def __getattr__(self, key): def __getattr__(self, key):
# Swizzling behaviour only occurs when
# the attribute name is entirely comprised
# of 'x', 'y', and 'z'.
if not all([c in 'xyz' for c in key]):
raise AttributeError(key)
key = ['xyz'.index(c) for c in key] key = ['xyz'.index(c) for c in key]
return [self.__xyz[c] for c in key] return [self.__xyz[c] for c in key]
def __setattr__(self, key, value): def __setattr__(self, key, value):
pass # key = ['xyz'.index(c) for c in key]
```
# Restrict swizzling behaviour as above
if not all([c in 'xyz' for c in key]):
return super().__setattr__(key, value)
if len(key) == 1:
value = (value,)
idxs = ['xyz'.index(c) for c in key]
for i, v in sorted(zip(idxs, value)):
self.__xyz[i] = v
``` ```
v = Vector((4, 5, 6))
print('xyz: ', v.xyz)
print('yz: ', v.yz) And here it is in action:
print('zxy: ', v.xzy)
print('y: ', v.y)
``` ```
v = Vector((1, 2, 3))
print('v: ', v)
print('xyz: ', v.xyz)
print('yz: ', v.zy)
print('xx: ', v.xx)
## Other special methods v.xz = 10, 30
print(v)
v.y = 20
print(v)
```
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment