{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Modules and packages\n",
"\n",
"\n",
"Python gives you a lot of flexibility in how you organise your code. If you\n",
"want, you can write a Python program just as you would write a Bash script.\n",
"You don't _have_ to use functions, classes, modules or packages if you don't\n",
"want to, or if the script's task does not require them.\n",
"\n",
"\n",
"But when your code starts to grow beyond what can reasonably be defined in a\n",
"single file, you will (hopefully) want to start arranging it in a more\n",
"understandable manner.\n",
"\n",
"\n",
"For this practical we have prepared a handful of example files - you can find\n",
"them alongside this notebook file, in a directory called\n",
"`02_modules_and_packages/`.\n",
"\n",
"\n",
"## Contents\n",
"\n",
"* [What is a module?](#what-is-a-module)\n",
"* [Importing modules](#importing-modules)\n",
" * [Importing specific items from a module](#importing-specific-items-from-a-module)\n",
" * [Importing everything from a module](#importing-everything-from-a-module)\n",
" * [Module aliases](#module-aliases)\n",
" * [What happens when I import a module?](#what-happens-when-i-import-a-module)\n",
" * [How can I make my own modules importable?](#how-can-i-make-my-own-modules-importable)\n",
"* [Modules versus scripts](#modules-versus-scripts)\n",
"* [What is a package?](#what-is-a-package)\n",
" * [`__init__.py`](#init-py)\n",
"* [Useful references](#useful-references)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"os.chdir('02_modules_and_packages')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a class=\"anchor\" id=\"what-is-a-module\"></a>\n",
"## What is a module?\n",
"\n",
"\n",
"Any file ending with `.py` is considered to be a module in Python. Take a look\n",
"at `02_modules_and_packages/numfuncs.py` - either open it in your editor, or\n",
"run this code block:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"with open('numfuncs.py', 'rt') as f:\n",
" for line in f:\n",
" print(line.rstrip())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This is a perfectly valid Python module, although not a particularly useful\n",
"one. It contains an attribute called `PI`, and a function `add`.\n",
"\n",
"\n",
"<a class=\"anchor\" id=\"importing-modules\"></a>\n",
"## Importing modules\n",
"\n",
"\n",
"Before we can use our module, we must `import` it. Importing a module in\n",
"Python will make its contents available in the local scope. We can import the\n",
"contents of `numfuncs` like so:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numfuncs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This imports `numfuncs` into the local scope - everything defined in the\n",
"`numfuncs` module can be accessed by prefixing it with `numfuncs.`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print('PI:', numfuncs.PI)\n",
"print(numfuncs.add(1, 50))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There are a couple of other ways to import items from a module...\n",
"\n",
"\n",
"<a class=\"anchor\" id=\"importing-specific-items-from-a-module\"></a>\n",
"### Importing specific items from a module\n",
"\n",
"\n",
"If you only want to use one, or a few items from a module, you can import just\n",
"those items - a reference to just those items will be created in the local\n",
"scope:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from numfuncs import add\n",
"print(add(1, 3))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a class=\"anchor\" id=\"importing-everything-from-a-module\"></a>\n",
"### Importing everything from a module\n",
"\n",
"\n",
"It is possible to import _everything_ that is defined in a module like so:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from numfuncs import *\n",
"print('PI: ', PI)\n",
"print(add(1, 5))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__PLEASE DON'T DO THIS!__ Because every time you do, somewhere in the world, a\n",
"software developer will will spontaneously stub his/her toe, and start crying.\n",
"Using this approach can make more complicated programs very difficult to read,\n",
"because it is not possible to determine the origin of the functions and\n",
"attributes that are being used.\n",
"\n",
"\n",
"And naming collisions are inevitable when importing multiple modules in this\n",
"way, making it very difficult for somebody else to figure out what your code\n",
"is doing:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from numfuncs import *\n",
"from strfuncs import *\n",
"\n",
"print(add(1, 5))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Instead, it is better to give modules a name when you import them. While this\n",
"requires you to type more code, the benefits of doing this far outweigh the\n",
"hassle of typing a few extra characters - it becomes much easier to read and\n",
"trace through code when the functions you use are accessed through a namespace\n",
"for each module:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numfuncs\n",
"import strfuncs\n",
"print('number add: ', numfuncs.add(1, 2))\n",
"print('string add: ', strfuncs.add(1, 2))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a class=\"anchor\" id=\"module-aliases\"></a>\n",
"### Module aliases\n",
"\n",
"\n",
"And Python allows you to define an _alias_ for a module when you import it,\n",
"so you don't necessarily need to type out the full module name each time\n",
"you want to access something inside:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numfuncs as nf\n",
"import strfuncs as sf\n",
"print('number add: ', nf.add(1, 2))\n",
"print('string add: ', sf.add(1, 2))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You have already seen this in the earlier practicals - here are a few\n",
"aliases which have become a de-facto standard for commonly used Python\n",
"modules:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os.path as op\n",
"import numpy as np\n",
"import nibabel as nib\n",
"import matplotlib as mpl\n",
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a class=\"anchor\" id=\"what-happens-when-i-import-a-module\"></a>\n",
"### What happens when I import a module?\n",
"\n",
"\n",
"When you `import` a module, the contents of the module file are literally\n",
"executed by the Python runtime, exactly the same as if you had typed its\n",
"contents into `ipython`. Any attributes, functions, or classes which are\n",
"defined in the module will be bundled up into an object that represents the\n",
"module, and through which you can access the module's contents.\n",
"\n",
"\n",
"When we typed `import numfuncs` in the examples above, the following events\n",
"occurred:\n",
"\n",
"\n",
"1. Python created a `module` object to represent the module.\n",
"\n",
"2. The `numfuncs.py` file was read and executed, and all of the items defined\n",
" inside `numfuncs.py` (i.e. the `PI` attribute and the `add` function) were\n",
" added to the `module` object.\n",
"\n",
"3. A local variable called `numfuncs`, pointing to the `module` object,\n",
" was added to the local scope.\n",
"\n",
"\n",
"Because module files are literally executed on import, any statements in the\n",
"module file which are not encapsulated inside a class or function will be\n",
"executed. As an example, take a look at the file `sideeffects.py`. Let's\n",
"import it and see what happens:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import sideeffects"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Ok, hopefully that wasn't too much of a surprise. Something which may be less\n",
"intuitive, however, is that a module's contents will only be executed on the\n",
"_first_ time that it is imported. After the first import, Python caches the\n",
"module's contents (all loaded modules are accessible through\n",
"[`sys.modules`](https://docs.python.org/3.5/library/sys.html#sys.modules)). On\n",
"subsequent imports, the cached version of the module is returned:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import sideeffects"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a class=\"anchor\" id=\"how-can-i-make-my-own-modules-importable\"></a>\n",
"### How can I make my own modules importable?\n",
"\n",
"\n",
"When you `import` a module, Python searches for it in the following locations,\n",
"in the following order:\n",
"\n",
"\n",
"1. Built-in modules (e.g. `os`, `sys`, etc.).\n",
"2. In the current directory or, if a script has been executed, in the directory\n",
" containing that script.\n",
"3. In directories listed in the `$PYTHONPATH` environment variable.\n",
"4. In installed third-party libraries (e.g. `numpy`).\n",
"\n",
"\n",
"If you are experimenting or developing your program, the quickest and easiest\n",
"way to make your module(s) importable is to add their containing directory to\n",
"the `PYTHONPATH`. But if you are developing a larger piece of software, you\n",
"should probably organise your modules into _packages_, which are [described\n",
"below](#what-is-a-package).\n",
"\n",
"\n",
"<a class=\"anchor\" id=\"modules-versus-scripts\"></a>\n",
"## Modules versus scripts\n",
"\n",
"\n",
"You now know that Python treats all files ending in `.py` as importable\n",
"modules. But all files ending in `.py` can also be treated as scripts. In\n",
"fact, there no difference between a _module_ and a _script_ - any `.py` file\n",
"can be executed as a script, or imported as a module, or both.\n",
"\n",
"\n",
"Have a look at the file `02_modules_and_packages/module_and_script.py`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"with open('module_and_script.py', 'rt') as f:\n",
" for line in f:\n",
" print(line.rstrip())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This file contains two functions `mul` and `main`. The\n",
"`if __name__ == '__main__':` clause at the bottom is a standard trick in Python\n",
"that allows you to add code to a file that is _only executed when the module is\n",
"called as a script_. Try it in a terminal now:\n",
"\n",
"\n",
"> `python 02_modules_and_packages/module_and_script.py`\n",
"\n",
"\n",
"But if we `import` this module from another file, or from an interactive\n",
"session, the code within the `if __name__ == '__main__':` clause will not be\n",
"executed, and we can access its functions just like any other module that we\n",
"import."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import module_and_script as mas\n",
"\n",
"a = 1.5\n",
"b = 3\n",
"\n",
"print('mul({}, {}): {}'.format(a, b, mas.mul(a, b)))\n",
"print('calling main...')\n",
"mas.main([str(a), str(b)])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a class=\"anchor\" id=\"what-is-a-package\"></a>\n",
"## What is a package?\n",
"\n",
"\n",
"You now know how to split your Python code up into separate files\n",
"(a.k.a. _modules_). When your code grows beyond a handful of files, you may\n",
"wish for more fine-grained control over the namespaces in which your modules\n",
"live. Python has another feature which allows you to organise your modules\n",
"into _packages_.\n",
"\n",
"\n",
"A package in Python is simply a directory which:\n",
"\n",
"\n",
"* Contains a special file called `__init__.py`\n",
"* May contain one or more module files (any other files ending in `*.py`)\n",
"* May contain other package directories.\n",
"\n",
"\n",
"For example, the [FSLeyes](https://git.fmrib.ox.ac.uk/fsl/fsleyes/fsleyes)\n",
"code is organised into packages and sub-packages as follows (abridged):\n",
"\n",
"\n",
"> ```\n",
"> fsleyes/\n",
"> __init__.py\n",
"> main.py\n",
"> frame.py\n",
"> views/\n",
"> __init__.py\n",
"> orthopanel.py\n",
"> lightboxpanel.py\n",
"> controls/\n",
"> __init__.py\n",
"> locationpanel.py\n",
"> overlaylistpanel.py\n",
"> ```\n",
"\n",
"\n",
"Within a package structure, we will typically still import modules directly,\n",
"via their full path within the package:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import fsleyes.main as fmain\n",
"fmain.fsleyes_main()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a class=\"anchor\" id=\"init-py\"></a>\n",
"### `__init__.py`\n",
"\n",
"\n",
"Every Python package must have an `__init__.py` file. In many cases, this will\n",
"actually be an empty file, and you don't need to worry about it any more, apart\n",
"from knowing that it is needed. But you can use `__init__.py` to perform some\n",
"package-specific initialisation, and/or to customise the package's namespace.\n",
"\n",
"\n",
"As an example, take a look the `02_modules_and_packages/fsleyes/__init__.py`\n",
"file in our mock FSLeyes package. We have imported the `fsleyes_main` function\n",
"from the `fsleyes.main` module, making it available at the package level. So\n",
"instead of importing the `fsleyes.main` module, we could instead just import\n",
"the `fsleyes` package:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import fsleyes\n",
"fsleyes.fsleyes_main()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a class=\"anchor\" id=\"useful-references\"></a>\n",
"## Useful references\n",
"\n",
"* [Modules and packages in Python](https://docs.python.org/3.5/tutorial/modules.html)\n",
"* [Using `__init__.py`](http://mikegrouchy.com/blog/2012/05/be-pythonic-__init__py.html)"
]
}
],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 2
}