Skip to content
Snippets Groups Projects
Commit f56044ca authored by Paul McCarthy's avatar Paul McCarthy :mountain_bicyclist:
Browse files

Merge branch 'master' into 'master' 

MJ's work is done ... he hopes

See merge request fsl/pytreat-2018-practicals!21
parents 2860e4e0 8fb77dc8
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 1354 additions and 27 deletions
getting_started/01_basics.ipynb
+ 272
− 11
View file @ f56044ca
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Basic python # Basic python
This tutorial is aimed at briefly introducing you to the main language This tutorial is aimed at briefly introducing you to the main language
features of python, with emphasis on some of the common difficulties features of python, with emphasis on some of the common difficulties
and pitfalls that are commonly encountered when moving to python. and pitfalls that are commonly encountered when moving to python.
When going through this make sure that you _run_ each code block When going through this make sure that you _run_ each code block and
and look at the output, as these are crucial for understanding the look at the output, as these are crucial for understanding the
explanations. You can run each block by using _shift + enter_ (including the text blocks, so you can just move down the document with shift + enter). explanations. You can run each block by using _shift + enter_
(including the text blocks, so you can just move down the document
with shift + enter).
It is also possible to _change_ the contents of each code block (these pages are completely interactive) so do experiment with the code you see and try some variations!
## Contents
* [Basic types](#Basic-types)
- [Strings](#Strings)
+ [Format](#Format)
+ [String manipulation](#String-manipulation)
- [Tuples and lists](#Tuples-and-lists)
+ [Adding to a list](#Adding-to-a-list)
+ [Indexing](#Indexing)
+ [Slicing](#Slicing)
- [List operations](#List-operations)
+ [Looping over elements in a list (or tuple)](#Looping)
+ [Getting help](#Getting-help)
- [Dictionaries](#Dictionaries)
+ [Adding to a dictionary](#Adding-to-a-dictionary)
+ [Removing elements from a dictionary](#Removing-elements-dictionary)
+ [Looping over everything in a dictionary](#Looping-dictionary)
- [Copying and references](#Copying-and-references)
* [Control flow](#Control-flow)
- [Boolean operators](#Boolean-operators)
- [If statements](#If-statements)
- [For loops](#For-loops)
- [While loops](#While-loops)
- [A quick intro to conditional expressions and list comprehensions](#quick-intro)
+ [Conditional expressions](#Conditional-expressions)
+ [List comprehensions](#List-comprehensions)
* [Functions](#functions)
* [Exercise](#exercise)
--- ---
<a class="anchor" id="Basic-types"></a>
# Basic types # Basic types
Python has many different types and variables are dynamic and can change types (like MATLAB). Some of the most commonly used in-built types are: Python has many different types and variables are dynamic and can change types (like MATLAB). Some of the most commonly used in-built types are:
* integer and floating point scalars * integer and floating point scalars
* strings * strings
* tuples * tuples
* lists * lists
* dictionary * dictionary
N-dimensional arrays and other types are supported through common modules (e.g., numpy, scipy, scikit-learn). These will be covered in a subsequent exercise. N-dimensional arrays and other types are supported through common modules (e.g., numpy, scipy, scikit-learn). These will be covered in a subsequent exercise.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
a = 4 a = 4
b = 3.6 b = 3.6
c = 'abc' c = 'abc'
d = [10, 20, 30] d = [10, 20, 30]
e = {'a' : 10, 'b': 20} e = {'a' : 10, 'b': 20}
print(a) print(a)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Any variable or combination of variables can be printed using the function `print()`: Any variable or combination of variables can be printed using the function `print()`:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
print(d) print(d)
print(e) print(e)
print(a, b, c) print(a, b, c)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> _*Python 3 versus python 2*_: > _*Python 3 versus python 2*_:
> >
> Print - for the print statement the brackets are compulsory for *python 3*, but are optional in python 2. So you will see plenty of code without the brackets but you should never use `print` without brackets, as this is incompatible with Python 3. > Print - for the print statement the brackets are compulsory for *python 3*, but are optional in python 2. So you will see plenty of code without the brackets but you should never use `print` without brackets, as this is incompatible with Python 3.
> >
> Division - in python 3 all division is floating point (like in MATLAB), even if the values are integers, but in python 2 integer division works like it does in C. > Division - in python 3 all division is floating point (like in MATLAB), even if the values are integers, but in python 2 integer division works like it does in C.
--- ---
<a class="anchor" id="Strings"></a>
## Strings ## Strings
Strings can be specified using single quotes *or* double quotes - as long as they are matched. Strings can be specified using single quotes *or* double quotes - as long as they are matched.
Strings can be dereferenced like lists (see later). Strings can be indexed like lists (see later).
For example: For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
s1 = "test string" s1 = "test string"
s2 = 'another test string' s2 = 'another test string'
print(s1, ' :: ', s2) print(s1, ' :: ', s2)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
You can also use triple quotes to capture multi-line strings. For example: You can also use triple quotes to capture multi-line strings. For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
s3 = '''This is s3 = '''This is
a string over a string over
multiple lines multiple lines
''' '''
print(s3) print(s3)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="Format"></a>
### Format ### Format
More interesting strings can be created using the `format` statement, which is very useful in print statements: More interesting strings can be created using the `format` statement, which is very useful in print statements:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
x = 1 x = 1
y = 'PyTreat' y = 'PyTreat'
s = 'The numerical value is {} and a name is {}'.format(x, y) s = 'The numerical value is {} and a name is {}'.format(x, y)
print(s) print(s)
print('A name is {} and a number is {}'.format(y, x)) print('A name is {} and a number is {}'.format(y, x))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
There are also other options along these lines, but this is the more modern version, although you will see plenty of the other alternatives in old code (i.e., code written before last week). There are also other options along these lines, but this is the more modern version, although you will see plenty of the other alternatives in "old" code (to python coders this means anything written before last week).
<a class="anchor" id="String-manipulation"></a>
### String manipulation ### String manipulation
The methods `lower()` and `upper()` are useful for strings. For example: The methods `lower()` and `upper()` are useful for strings. For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
s = 'This is a Test String' s = 'This is a Test String'
print(s.upper()) print(s.upper())
print(s.lower()) print(s.lower())
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Another useful method is: Another useful method is:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
s = 'This is a Test String' s = 'This is a Test String'
s2 = s.replace('Test', 'Better') s2 = s.replace('Test', 'Better')
print(s2) print(s2)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Strings can be concatenated just by using the `+` operator:
%% Cell type:code id: tags:
``` python
s3 = s + ' :: ' + s2
print(s3)
```
%% Cell type:markdown id: tags:
If you like regular expressions then you're in luck as these are well supported in python using the `re` module. To use this (like many other "extensions" - called _modules_ in Python - you need to `import` it). For example: If you like regular expressions then you're in luck as these are well supported in python using the `re` module. To use this (like many other "extensions" - called _modules_ in Python - you need to `import` it). For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
import re import re
s = 'This is a test of a Test String' s = 'This is a test of a Test String'
s1 = re.sub(r'a [Tt]est', "an example", s) s1 = re.sub(r'a [Tt]est', "an example", s)
print(s1) print(s1)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
where the `r` before the quote is used to force the regular expression specification to be a `raw string`. where the `r` before the quote is used to force the regular expression specification to be a `raw string`.
For more information on matching and substitutions, look up the regular expression module on the web. For more information on matching and substitutions, look up the regular expression module on the web.
Two common and convenient string methods are `strip()` and `split()`. The first will remove any whitespace at the beginning and end of a string:
You can also split, or tokenize, a string (to turn it into a list) like this: %% Cell type:code id: tags:
``` python
s2 = ' A very spacy string '
print('*' + s2 + '*')
print('*' + s2.strip() + '*')
```
%% Cell type:markdown id: tags:
With `split()` we can tokenize a string (to turn it into a list of strings) like this:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
print(s.split()) print(s.split())
print(s2.split())
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
By default it splits at whitespace, but it can also split at a specified delimiter:
%% Cell type:code id: tags:
``` python
s4 = ' This is, as you can see , a very weirdly spaced and punctuated string ... '
print(s4.split(','))
```
%% Cell type:markdown id: tags:
There are more powerful ways of dealing with this like csv files/strings, which are covered in later practicals, but even this can get you a long way.
> Note that strings in python 3 are _unicode_ so can represent Chinese characters, etc, and is therefore very flexible. However, in general you can just be blissfully ignorant of this fact. > Note that strings in python 3 are _unicode_ so can represent Chinese characters, etc, and is therefore very flexible. However, in general you can just be blissfully ignorant of this fact.
Strings can be converted to integer or floating-point values by using the `int()` and `float()` calls:
%% Cell type:code id: tags:
``` python
sint='23'
sfp='2.03'
print(sint + sfp)
print(int(sint) + float(sfp))
print(float(sint) + float(sfp))
```
%% Cell type:markdown id: tags:
> Note that calling `int()` on a non-integer (e.g., on `sfp` above) will raise an error.
--- ---
## Tuples and Lists <a class="anchor" id="Tuples-and-lists"></a>
## Tuples and lists
Both tuples and lists are builtin python types and are like vectors, Both tuples and lists are builtin python types and are like vectors,
but for numerical vectors and arrays it is much better to use _numpy_ but for numerical vectors and arrays it is much better to use _numpy_
arrays (or matrices), which are covered in a later tutorial. arrays (or matrices), which are covered in a later tutorial.
A tuple is like a list or a vector, but with less flexibility than a full list, however anything can be stored in either a list or tuple, without any consistency being required. For example: A tuple is like a list or a vector, but with less flexibility than a full list, however anything can be stored in either a list or tuple, without any consistency being required. For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
xtuple = (3, 7.6, 'str') xtuple = (3, 7.6, 'str')
xlist = [1, 'mj', -5.4] xlist = [1, 'mj', -5.4]
print(xtuple) print(xtuple)
print(xlist) print(xlist)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
They can also be nested: They can also be nested:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
x2 = (xtuple, xlist) x2 = (xtuple, xlist)
x3 = [xtuple, xlist] x3 = [xtuple, xlist]
print('x2 is: ', x2) print('x2 is: ', x2)
print('x3 is: ', x3) print('x3 is: ', x3)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="Adding-to-a-list"></a>
### Adding to a list ### Adding to a list
This is easy: This is easy:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
a = [10, 20, 30] a = [10, 20, 30]
a = a + [70] a = a + [70]
a += [80] a += [80]
print(a) print(a)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="Indexing"></a>
### Indexing ### Indexing
Square brackets are used to index tuples, lists, dictionaries, etc. For example: Square brackets are used to index tuples, lists, dictionaries, etc. For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
d = [10, 20, 30] d = [10, 20, 30]
print(d[1]) print(d[1])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> _*Pitfall:*_ > _*Pitfall:*_
> Python uses zero-based indexing, unlike MATLAB > Python uses zero-based indexing, unlike MATLAB
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
a = [10, 20, 30, 40, 50, 60] a = [10, 20, 30, 40, 50, 60]
print(a[0]) print(a[0])
print(a[2]) print(a[2])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Indices naturally run from 0 to N-1, _but_ negative numbers can be used to reference from the end (circular wrap-around). Indices naturally run from 0 to N-1, _but_ negative numbers can be used to reference from the end (circular wrap-around).
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
print(a[-1]) print(a[-1])
print(a[-6]) print(a[-6])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
However, this is only true for -1 to -N. Outside of -N to N-1 will generate an `index out of range` error. However, this is only true for -1 to -N. Outside of -N to N-1 will generate an `index out of range` error.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
print(a[-7]) print(a[-7])
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
print(a[6]) print(a[6])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Length of a tuple or list is given by the `len()` function: Length of a tuple or list is given by the `len()` function:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
print(len(a)) print(len(a))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Nested lists can have nested indexing: Nested lists can have nested indexing:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
b = [[10, 20, 30], [40, 50, 60]] b = [[10, 20, 30], [40, 50, 60]]
print(b[0][1]) print(b[0][1])
print(b[1][0]) print(b[1][0])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
but *not* an index like b[0, 1]. but *not* an index like b[0, 1].
> Note that `len` will only give the length of the top level. > Note that `len` will only give the length of the top level.
> In general, numpy arrays should be preferred to nested lists when the contents are numerical. > In general, numpy arrays should be preferred to nested lists when the contents are numerical.
<a class="anchor" id="Slicing"></a>
### Slicing ### Slicing
A range of values for the indices can be specified to extract values from a list. For example: A range of values for the indices can be specified to extract values from a list. For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
print(a[0:3]) print(a[0:3])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> _*Pitfall:*_ > _*Pitfall:*_
> >
> Slicing syntax is different from MATLAB in that second number is > Slicing syntax is different from MATLAB in that second number is
> exclusive (i.e., one plus final index) - this is in addition to the zero index difference. > exclusive (i.e., one plus final index) - this is in addition to the zero index difference.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
a = [10, 20, 30, 40, 50, 60] a = [10, 20, 30, 40, 50, 60]
print(a[0:3]) # same as a(1:3) in MATLAB print(a[0:3]) # same as a(1:3) in MATLAB
print(a[1:3]) # same as a(2:3) in MATLAB print(a[1:3]) # same as a(2:3) in MATLAB
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> _*Pitfall:*_ > _*Pitfall:*_
> >
> Unlike in MATLAB, you cannot use a list as indices instead of an > Unlike in MATLAB, you cannot use a list as indices instead of an
> integer or a slice (although these can be done in _numpy_). > integer or a slice (although these can be done in _numpy_).
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
b = [3, 4] b = [3, 4]
print(a[b]) print(a[b])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="List-operations"></a>
### List operations ### List operations
Multiplication can be used with lists, where multiplication implements replication. Multiplication can be used with lists, where multiplication implements replication.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
d = [10, 20, 30] d = [10, 20, 30]
print(d * 4) print(d * 4)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
There are also other operations such as: There are also other operations such as:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
d.append(40) d.append(40)
print(d) print(d)
d.remove(20) d.remove(20)
print(d) print(d)
d.pop(0) d.pop(0)
print(d) print(d)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="Looping"></a>
### Looping over elements in a list (or tuple) ### Looping over elements in a list (or tuple)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
d = [10, 20, 30] d = [10, 20, 30]
for x in d: for x in d:
print(x) print(x)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> Note that the indentation within the loop is _*crucial*_. All python control blocks are delineated purely by indentation. > Note that the indentation within the loop is _*crucial*_. All python control blocks are delineated purely by indentation.
<a class="anchor" id="Getting-help"></a>
### Getting help ### Getting help
The function `help()` can be used to get information about any variable/object/function in python. It lists the possible operations. In `ipython` you can also just type `?<blah>` or `<blah>?` instead: The function `help()` can be used to get information about any variable/object/function in python. It lists the possible operations. In `ipython` you can also just type `?<blah>` or `<blah>?` instead:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
help(d) help(d)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
There is also a `dir()` function that gives a basic listing of the operations: There is also a `dir()` function that gives a basic listing of the operations:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
dir(d) dir(d)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> Note that google is often more helpful! > Note that google is often more helpful!
--- ---
<a class="anchor" id="Dictionaries"></a>
## Dictionaries ## Dictionaries
These store key-value pairs. For example: These store key-value pairs. For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
e = {'a' : 10, 'b': 20} e = {'a' : 10, 'b': 20}
print(len(e)) print(len(e))
print(e.keys()) print(e.keys())
print(e.values()) print(e.values())
print(e['a']) print(e['a'])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The keys and values can take on almost any type, even dictionaries! The keys and values can take on almost any type, even dictionaries!
Python is nothing if not flexible. However, each key must be unique Python is nothing if not flexible. However, each key must be unique
and the dictionary must be "hashable". and the dictionary must be "hashable".
<a class="anchor" id="Adding-to-a-dictionary"></a>
### Adding to a dictionary ### Adding to a dictionary
This is very easy: This is very easy:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
e['c'] = 555 # just like in Biobank! ;) e['c'] = 555 # just like in Biobank! ;)
print(e) print(e)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="Removing-elements-dictionary"></a>
### Removing elements from a dictionary ### Removing elements from a dictionary
There are two main approaches - `pop` and `del`: There are two main approaches - `pop` and `del`:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
e.pop('b') e.pop('b')
print(e) print(e)
del e['c'] del e['c']
print(e) print(e)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="Looping-dictionary"></a>
### Looping over everything in a dictionary ### Looping over everything in a dictionary
Several variables can jointly work as loop variables in python, which is very convenient. For example: Several variables can jointly work as loop variables in python, which is very convenient. For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
e = {'a' : 10, 'b': 20, 'c':555} e = {'a' : 10, 'b': 20, 'c':555}
for k, v in e.items(): for k, v in e.items():
print((k, v)) print((k, v))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The print statement here constructs a tuple, which is often used in python. The print statement here constructs a tuple, which is often used in python.
Another option is: Another option is:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
for k in e: for k in e:
print((k, e[k])) print((k, e[k]))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> Note that in both cases the order is arbitrary. The `sorted` function can be used if you want keys in a sorted order; e.g. `for k in sorted(e):` ... > Note that in both cases the order is arbitrary. The `sorted` function can be used if you want keys in a sorted order; e.g. `for k in sorted(e):` ...
> >
> There are also other options if you want a dictionary with ordering. > There are also other options if you want a dictionary with ordering.
--- ---
<a class="anchor" id="Copying-and-references"></a>
## Copying and references ## Copying and references
In python there are immutable types (e.g. numbers) and mutable types (e.g. lists). The main thing to know is that assignment can sometimes create separate copies and sometimes create references (as in C++). In general, the more complicated types are assigned via references. For example: In python there are immutable types (e.g. numbers) and mutable types (e.g. lists). The main thing to know is that assignment can sometimes create separate copies and sometimes create references (as in C++). In general, the more complicated types are assigned via references. For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
a = 7 a = 7
b = a b = a
a = 2348 a = 2348
print(b) print(b)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
As opposed to: As opposed to:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
a = [7] a = [7]
b = a b = a
a[0] = 8888 a[0] = 8888
print(b) print(b)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
But if an operation is performed then a copy might be made: But if an operation is performed then a copy might be made:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
a = [7] a = [7]
b = a * 2 b = a * 2
a[0] = 8888 a[0] = 8888
print(b) print(b)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
If an explicit copy is necessary then this can be made using the `list()` constructor: If an explicit copy is necessary then this can be made using the `list()` constructor:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
a = [7] a = [7]
b = list(a) b = list(a)
a[0] = 8888 a[0] = 8888
print(b) print(b)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
There is a constructor for each type and this con be useful for converting between types: There is a constructor for each type and this con be useful for converting between types:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
xt = (2, 5, 7) xt = (2, 5, 7)
xl = list(xt) xl = list(xt)
print(xt) print(xt)
print(xl) print(xl)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> _*Pitfall:*_ > _*Pitfall:*_
> >
> When writing functions you need to be particularly careful about references and copies. > When writing functions you need to be particularly careful about references and copies.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
def foo1(x): def foo1(x):
x.append(10) x.append(10)
def foo2(x): def foo2(x):
x = x + [10] x = x + [10]
def foo3(x): def foo3(x):
return x + [10] return x + [10]
a = [5] a = [5]
print(a) print(a)
foo1(a) foo1(a)
print(a) print(a)
foo2(a) foo2(a)
print(a) print(a)
foo3(a) foo3(a)
print(a) print(a)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> Note that we have defined some functions here - and the syntax
> should be relatively intuitive. See <a href="#functions">below</a>
> for a bit more detail on function definitions.
--- ---
<a class="anchor" id="Control-flow"></a>
## Control flow ## Control flow
<a class="anchor" id="Boolean-operators"></a>
### Boolean operators ### Boolean operators
There is a boolean type in python that can be `True` or `False` (note the capitals). Other values can also be used for True or False (e.g., 1 for True; 0 or None or [] or {} or "") although they are not considered 'equal' in the sense that the operator `==` would consider them the same. There is a boolean type in python that can be `True` or `False` (note the capitals). Other values can also be used for True or False (e.g., 1 for True; 0 or None or [] or {} or "") although they are not considered 'equal' in the sense that the operator `==` would consider them the same.
Relevant boolean and comparison operators include: `not`, `and`, `or`, `==` and `!=` Relevant boolean and comparison operators include: `not`, `and`, `or`, `==` and `!=`
For example: For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
a = True a = True
print('Not a is:', not a) print('Not a is:', not a)
print('Not 1 is:', not 1) print('Not 1 is:', not 1)
print('Not 0 is:', not 0) print('Not 0 is:', not 0)
print('Not {} is:', not {}) print('Not {} is:', not {})
print('{}==0 is:', {}==0) print('{}==0 is:', {}==0)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
There is also the `in` test for strings, lists, etc: There is also the `in` test for strings, lists, etc:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
print('the' in 'a number of words') print('the' in 'a number of words')
print('of' in 'a number of words') print('of' in 'a number of words')
print(3 in [1, 2, 3, 4]) print(3 in [1, 2, 3, 4])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="If-statements"></a>
### If statements ### If statements
The basic syntax of `if` statements is fairly standard, though don't forget that you _*must*_ indent the statements within the conditional/loop block as this is the way of delineating blocks of code in python. For example: The basic syntax of `if` statements is fairly standard, though don't forget that you _*must*_ indent the statements within the conditional/loop block as this is the way of delineating blocks of code in python. For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
import random import random
a = random.uniform(-1, 1) a = random.uniform(-1, 1)
print(a) print(a)
if a>0: if a>0:
print('Positive') print('Positive')
elif a<0: elif a<0:
print('Negative') print('Negative')
else: else:
print('Zero') print('Zero')
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Or more generally: Or more generally:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
a = [] # just one of many examples a = [] # just one of many examples
if not a: if not a:
print('Variable is true, or at least not empty') print('Variable is true, or at least not empty')
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
This can be useful for functions where a variety of possible input types are being dealt with. This can be useful for functions where a variety of possible input types are being dealt with.
--- ---
<a class="anchor" id="For-loops"></a>
### For loops ### For loops
The `for` loop works like in bash: The `for` loop works like in bash:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
for x in [2, 'is', 'more', 'than', 1]: for x in [2, 'is', 'more', 'than', 1]:
print(x) print(x)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
where a list or any other sequence (e.g. tuple) can be used. where a list or any other sequence (e.g. tuple) can be used.
If you want a numerical range then use: If you want a numerical range then use:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
for x in range(2, 9): for x in range(2, 9):
print(x) print(x)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Note that, like slicing, the maximum value is one less than the value specified. Also, `range` actually returns an object that can be iterated over but is not just a list of numbers. If you want a list of numbers then `list(range(2, 9))` will give you this. Note that, like slicing, the maximum value is one less than the value specified. Also, `range` actually returns an object that can be iterated over but is not just a list of numbers. If you want a list of numbers then `list(range(2, 9))` will give you this.
A very nice feature of python is that multiple variables can be assigned from a tuple or list: A very nice feature of python is that multiple variables can be assigned from a tuple or list:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
x, y = [4, 7] x, y = [4, 7]
print(x) print(x)
print(y) print(y)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
and this can be combined with a function called `zip` to make very convenient dual variable loops: and this can be combined with a function called `zip` to make very convenient dual variable loops:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
alist = ['Some', 'set', 'of', 'items'] alist = ['Some', 'set', 'of', 'items']
blist = list(range(len(alist))) blist = list(range(len(alist)))
print(list(zip(alist, blist))) print(list(zip(alist, blist)))
for x, y in zip(alist, blist): for x, y in zip(alist, blist):
print(y, x) print(y, x)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
This type of loop can be used with any two lists (or similar) to iterate over them jointly. This type of loop can be used with any two lists (or similar) to iterate over them jointly.
<a class="anchor" id="While-loops"></a>
### While loops ### While loops
The syntax for this is pretty standard: The syntax for this is pretty standard:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
import random import random
n = 0 n = 0
x = 0 x = 0
while n<100: while n<100:
x += random.uniform(0, 1)**2 # where ** is a power operation x += random.uniform(0, 1)**2 # where ** is a power operation
if x>50: if x>50:
break break
n += 1 n += 1
print(x) print(x)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
You can also use `continue` as in other languages. You can also use `continue` as in other languages.
--- ---
<a class="anchor" id="quick-intro"></a>
### A quick intro to conditional expressions and list comprehensions ### A quick intro to conditional expressions and list comprehensions
These are more advanced bits of python but are really useful and common, so worth having a little familiarity with at this stage. These are more advanced bits of python but are really useful and common, so worth having a little familiarity with at this stage.
<a class="anchor" id="Conditional-expressions"></a>
#### Conditional expressions #### Conditional expressions
A general expression that can be used in python is: A `if` condition `else` B A general expression that can be used in python is: A `if` condition `else` B
For example: For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
import random import random
x = random.uniform(0, 1) x = random.uniform(0, 1)
y = x**2 if x<0.5 else (1 - x)**2 y = x**2 if x<0.5 else (1 - x)**2
print(x, y) print(x, y)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="List-comprehensions"></a>
#### List comprehensions #### List comprehensions
This is a shorthand syntax for building a list like a for loop but doing it in one line, and is very popular in python. It is quite similar to mathematical set notation. For example: This is a shorthand syntax for building a list like a for loop but doing it in one line, and is very popular in python. It is quite similar to mathematical set notation. For example:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
v1 = [ x**2 for x in range(10) ] v1 = [ x**2 for x in range(10) ]
print(v1) print(v1)
v2 = [ x**2 for x in range(10) if x!=7 ] v2 = [ x**2 for x in range(10) if x!=7 ]
print(v2) print(v2)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
You'll find that python programmers use this kind of construction _*a lot*_. You'll find that python programmers use this kind of construction _*a lot*_.
---
<a class="anchor" id="functions"></a>
## Functions
You will find functions pretty familiar in python to start with,
although they have a few options which are really handy and different
from C++ or matlab (to be covered in a later practical). To start
with we'll look at a simple function but note a few key points:
* you _must_ indent everything inside the function (it is a code
block and indentation is the only way of determining this - just
like for the guts of a loop)
* you can return _whatever you want_ from a python function, but only
a single object - it is usual to package up multiple things in a
tuple or list, which is easily unpacked by the calling invocation:
e.g., `a, b, c = myfunc(x)`
* parameters are passed by reference (see section on <a
href="#Copying-and-references">copying and references</a>)
%% Cell type:code id: tags:
``` python
def myfunc(x, y, z=0):
r2 = x*x + y*y + z*z
r = r2**0.5
return (r, r2)
rad = myfunc(10, 20)
print(rad)
rad, dummy = myfunc(10, 20, 30)
print(rad)
rad, _ = myfunc(10,20,30)
print(rad)
```
%% Cell type:markdown id: tags:
> Note that the `_` is used as shorthand here for a dummy variable
> that you want to throw away.
One nice feature of python functions is that you can name the
arguments when you call them, rather than only doing it by position.
For example:
%% Cell type:code id: tags:
``` python
def myfunc(x, y, z=0, flag=''):
if flag=='L1':
r = abs(x) + abs(y) + abs(z)
else:
r = (x*x + y*y + z*z)**0.5
return r
rA = myfunc(10, 20)
rB = myfunc(10, 20, flag='L1')
rC = myfunc(10, 20, flag='L1', z=30)
print(rA, rB, rC)
```
%% Cell type:markdown id: tags:
You will often see python functions called with these named arguments.
---
<a class="anchor" id="exercise"></a>
## Exercise
Let's say you are given a single string with comma separated elements
that represent filenames and ID codes: e.g., `/vols/Data/pytreat/AAC, 165873, /vols/Data/pytreat/AAG, 170285, ...`
Write some code to do the following:
* separate out the filenames and ID codes into separate lists (ID's
should be numerical values, not strings)
* loop over the two and generate a _string_ that could be used to
rename the directories (e.g., `mv /vols/Data/pytreat/AAC /vols/Data/pytreat/S165873`) - we will cover how to actually execute these in a later practical
* convert your dual lists into a dictionary, with ID as the key
* write a small function to determine if an ID is present in this
set of not, and also return the filename if it is
* write a for loop to create a list of all the odd-numbered IDs (you can use the `%` operator for modulus - i.e., `5 % 2` is 1)
* re-write the for loop as a list comprehension
* now generate a list of the filenames corresponding to these odd-numbered IDs
%% Cell type:code id: tags:
``` python
mstr = '/vols/Data/pytreat/AAC, 165873, /vols/Data/pytreat/AAG, 170285, /vols/Data/pytreat/AAH, 196792, /vols/Data/pytreat/AAK, 212577, /vols/Data/pytreat/AAQ, 385376, /vols/Data/pytreat/AB, 444600, /vols/Data/pytreat/AC6, 454578, /vols/Data/pytreat/V8, 501502, /vols/Data/pytreat/2YK, 667688, /vols/Data/pytreat/C3PO, 821971'
```
......
getting_started/01_basics.md
+ 180
− 10
View file @ f56044ca
...@@ -4,12 +4,46 @@ This tutorial is aimed at briefly introducing you to the main language ...@@ -4,12 +4,46 @@ This tutorial is aimed at briefly introducing you to the main language
features of python, with emphasis on some of the common difficulties features of python, with emphasis on some of the common difficulties
and pitfalls that are commonly encountered when moving to python. and pitfalls that are commonly encountered when moving to python.
When going through this make sure that you _run_ each code block When going through this make sure that you _run_ each code block and
and look at the output, as these are crucial for understanding the look at the output, as these are crucial for understanding the
explanations. You can run each block by using _shift + enter_ (including the text blocks, so you can just move down the document with shift + enter). explanations. You can run each block by using _shift + enter_
(including the text blocks, so you can just move down the document
with shift + enter).
It is also possible to _change_ the contents of each code block (these pages are completely interactive) so do experiment with the code you see and try some variations!
## Contents
* [Basic types](#Basic-types)
- [Strings](#Strings)
+ [Format](#Format)
+ [String manipulation](#String-manipulation)
- [Tuples and lists](#Tuples-and-lists)
+ [Adding to a list](#Adding-to-a-list)
+ [Indexing](#Indexing)
+ [Slicing](#Slicing)
- [List operations](#List-operations)
+ [Looping over elements in a list (or tuple)](#Looping)
+ [Getting help](#Getting-help)
- [Dictionaries](#Dictionaries)
+ [Adding to a dictionary](#Adding-to-a-dictionary)
+ [Removing elements from a dictionary](#Removing-elements-dictionary)
+ [Looping over everything in a dictionary](#Looping-dictionary)
- [Copying and references](#Copying-and-references)
* [Control flow](#Control-flow)
- [Boolean operators](#Boolean-operators)
- [If statements](#If-statements)
- [For loops](#For-loops)
- [While loops](#While-loops)
- [A quick intro to conditional expressions and list comprehensions](#quick-intro)
+ [Conditional expressions](#Conditional-expressions)
+ [List comprehensions](#List-comprehensions)
* [Functions](#functions)
* [Exercise](#exercise)
--- ---
<a class="anchor" id="Basic-types"></a>
# Basic types # Basic types
Python has many different types and variables are dynamic and can change types (like MATLAB). Some of the most commonly used in-built types are: Python has many different types and variables are dynamic and can change types (like MATLAB). Some of the most commonly used in-built types are:
...@@ -45,10 +79,11 @@ print(a, b, c) ...@@ -45,10 +79,11 @@ print(a, b, c)
--- ---
<a class="anchor" id="Strings"></a>
## Strings ## Strings
Strings can be specified using single quotes *or* double quotes - as long as they are matched. Strings can be specified using single quotes *or* double quotes - as long as they are matched.
Strings can be dereferenced like lists (see later). Strings can be indexed like lists (see later).
For example: For example:
``` ```
...@@ -66,6 +101,7 @@ multiple lines ...@@ -66,6 +101,7 @@ multiple lines
print(s3) print(s3)
``` ```
<a class="anchor" id="Format"></a>
### Format ### Format
More interesting strings can be created using the `format` statement, which is very useful in print statements: More interesting strings can be created using the `format` statement, which is very useful in print statements:
...@@ -77,8 +113,9 @@ print(s) ...@@ -77,8 +113,9 @@ print(s)
print('A name is {} and a number is {}'.format(y, x)) print('A name is {} and a number is {}'.format(y, x))
``` ```
There are also other options along these lines, but this is the more modern version, although you will see plenty of the other alternatives in old code (i.e., code written before last week). There are also other options along these lines, but this is the more modern version, although you will see plenty of the other alternatives in "old" code (to python coders this means anything written before last week).
<a class="anchor" id="String-manipulation"></a>
### String manipulation ### String manipulation
The methods `lower()` and `upper()` are useful for strings. For example: The methods `lower()` and `upper()` are useful for strings. For example:
...@@ -95,6 +132,13 @@ s2 = s.replace('Test', 'Better') ...@@ -95,6 +132,13 @@ s2 = s.replace('Test', 'Better')
print(s2) print(s2)
``` ```
Strings can be concatenated just by using the `+` operator:
```
s3 = s + ' :: ' + s2
print(s3)
```
If you like regular expressions then you're in luck as these are well supported in python using the `re` module. To use this (like many other "extensions" - called _modules_ in Python - you need to `import` it). For example: If you like regular expressions then you're in luck as these are well supported in python using the `re` module. To use this (like many other "extensions" - called _modules_ in Python - you need to `import` it). For example:
``` ```
import re import re
...@@ -106,17 +150,46 @@ where the `r` before the quote is used to force the regular expression specifica ...@@ -106,17 +150,46 @@ where the `r` before the quote is used to force the regular expression specifica
For more information on matching and substitutions, look up the regular expression module on the web. For more information on matching and substitutions, look up the regular expression module on the web.
Two common and convenient string methods are `strip()` and `split()`. The first will remove any whitespace at the beginning and end of a string:
You can also split, or tokenize, a string (to turn it into a list) like this: ```
s2 = ' A very spacy string '
print('*' + s2 + '*')
print('*' + s2.strip() + '*')
```
With `split()` we can tokenize a string (to turn it into a list of strings) like this:
``` ```
print(s.split()) print(s.split())
print(s2.split())
```
By default it splits at whitespace, but it can also split at a specified delimiter:
```
s4 = ' This is, as you can see , a very weirdly spaced and punctuated string ... '
print(s4.split(','))
``` ```
There are more powerful ways of dealing with this like csv files/strings, which are covered in later practicals, but even this can get you a long way.
> Note that strings in python 3 are _unicode_ so can represent Chinese characters, etc, and is therefore very flexible. However, in general you can just be blissfully ignorant of this fact. > Note that strings in python 3 are _unicode_ so can represent Chinese characters, etc, and is therefore very flexible. However, in general you can just be blissfully ignorant of this fact.
Strings can be converted to integer or floating-point values by using the `int()` and `float()` calls:
```
sint='23'
sfp='2.03'
print(sint + sfp)
print(int(sint) + float(sfp))
print(float(sint) + float(sfp))
```
> Note that calling `int()` on a non-integer (e.g., on `sfp` above) will raise an error.
--- ---
## Tuples and Lists <a class="anchor" id="Tuples-and-lists"></a>
## Tuples and lists
Both tuples and lists are builtin python types and are like vectors, Both tuples and lists are builtin python types and are like vectors,
but for numerical vectors and arrays it is much better to use _numpy_ but for numerical vectors and arrays it is much better to use _numpy_
...@@ -138,6 +211,7 @@ print('x2 is: ', x2) ...@@ -138,6 +211,7 @@ print('x2 is: ', x2)
print('x3 is: ', x3) print('x3 is: ', x3)
``` ```
<a class="anchor" id="Adding-to-a-list"></a>
### Adding to a list ### Adding to a list
This is easy: This is easy:
...@@ -148,6 +222,7 @@ a += [80] ...@@ -148,6 +222,7 @@ a += [80]
print(a) print(a)
``` ```
<a class="anchor" id="Indexing"></a>
### Indexing ### Indexing
Square brackets are used to index tuples, lists, dictionaries, etc. For example: Square brackets are used to index tuples, lists, dictionaries, etc. For example:
...@@ -195,6 +270,7 @@ but *not* an index like b[0, 1]. ...@@ -195,6 +270,7 @@ but *not* an index like b[0, 1].
> Note that `len` will only give the length of the top level. > Note that `len` will only give the length of the top level.
> In general, numpy arrays should be preferred to nested lists when the contents are numerical. > In general, numpy arrays should be preferred to nested lists when the contents are numerical.
<a class="anchor" id="Slicing"></a>
### Slicing ### Slicing
A range of values for the indices can be specified to extract values from a list. For example: A range of values for the indices can be specified to extract values from a list. For example:
...@@ -223,6 +299,7 @@ b = [3, 4] ...@@ -223,6 +299,7 @@ b = [3, 4]
print(a[b]) print(a[b])
``` ```
<a class="anchor" id="List-operations"></a>
### List operations ### List operations
Multiplication can be used with lists, where multiplication implements replication. Multiplication can be used with lists, where multiplication implements replication.
...@@ -242,6 +319,7 @@ d.pop(0) ...@@ -242,6 +319,7 @@ d.pop(0)
print(d) print(d)
``` ```
<a class="anchor" id="Looping"></a>
### Looping over elements in a list (or tuple) ### Looping over elements in a list (or tuple)
``` ```
...@@ -252,6 +330,7 @@ for x in d: ...@@ -252,6 +330,7 @@ for x in d:
> Note that the indentation within the loop is _*crucial*_. All python control blocks are delineated purely by indentation. > Note that the indentation within the loop is _*crucial*_. All python control blocks are delineated purely by indentation.
<a class="anchor" id="Getting-help"></a>
### Getting help ### Getting help
The function `help()` can be used to get information about any variable/object/function in python. It lists the possible operations. In `ipython` you can also just type `?<blah>` or `<blah>?` instead: The function `help()` can be used to get information about any variable/object/function in python. It lists the possible operations. In `ipython` you can also just type `?<blah>` or `<blah>?` instead:
...@@ -272,6 +351,7 @@ dir(d) ...@@ -272,6 +351,7 @@ dir(d)
--- ---
<a class="anchor" id="Dictionaries"></a>
## Dictionaries ## Dictionaries
These store key-value pairs. For example: These store key-value pairs. For example:
...@@ -287,6 +367,7 @@ The keys and values can take on almost any type, even dictionaries! ...@@ -287,6 +367,7 @@ The keys and values can take on almost any type, even dictionaries!
Python is nothing if not flexible. However, each key must be unique Python is nothing if not flexible. However, each key must be unique
and the dictionary must be "hashable". and the dictionary must be "hashable".
<a class="anchor" id="Adding-to-a-dictionary"></a>
### Adding to a dictionary ### Adding to a dictionary
This is very easy: This is very easy:
...@@ -295,7 +376,7 @@ e['c'] = 555 # just like in Biobank! ;) ...@@ -295,7 +376,7 @@ e['c'] = 555 # just like in Biobank! ;)
print(e) print(e)
``` ```
<a class="anchor" id="Removing-elements-dictionary"></a>
### Removing elements from a dictionary ### Removing elements from a dictionary
There are two main approaches - `pop` and `del`: There are two main approaches - `pop` and `del`:
...@@ -306,6 +387,7 @@ del e['c'] ...@@ -306,6 +387,7 @@ del e['c']
print(e) print(e)
``` ```
<a class="anchor" id="Looping-dictionary"></a>
### Looping over everything in a dictionary ### Looping over everything in a dictionary
Several variables can jointly work as loop variables in python, which is very convenient. For example: Several variables can jointly work as loop variables in python, which is very convenient. For example:
...@@ -329,6 +411,7 @@ for k in e: ...@@ -329,6 +411,7 @@ for k in e:
--- ---
<a class="anchor" id="Copying-and-references"></a>
## Copying and references ## Copying and references
In python there are immutable types (e.g. numbers) and mutable types (e.g. lists). The main thing to know is that assignment can sometimes create separate copies and sometimes create references (as in C++). In general, the more complicated types are assigned via references. For example: In python there are immutable types (e.g. numbers) and mutable types (e.g. lists). The main thing to know is that assignment can sometimes create separate copies and sometimes create references (as in C++). In general, the more complicated types are assigned via references. For example:
...@@ -393,10 +476,16 @@ foo3(a) ...@@ -393,10 +476,16 @@ foo3(a)
print(a) print(a)
``` ```
> Note that we have defined some functions here - and the syntax
> should be relatively intuitive. See <a href="#functions">below</a>
> for a bit more detail on function definitions.
--- ---
<a class="anchor" id="Control-flow"></a>
## Control flow ## Control flow
<a class="anchor" id="Boolean-operators"></a>
### Boolean operators ### Boolean operators
There is a boolean type in python that can be `True` or `False` (note the capitals). Other values can also be used for True or False (e.g., 1 for True; 0 or None or [] or {} or "") although they are not considered 'equal' in the sense that the operator `==` would consider them the same. There is a boolean type in python that can be `True` or `False` (note the capitals). Other values can also be used for True or False (e.g., 1 for True; 0 or None or [] or {} or "") although they are not considered 'equal' in the sense that the operator `==` would consider them the same.
...@@ -420,7 +509,7 @@ print('of' in 'a number of words') ...@@ -420,7 +509,7 @@ print('of' in 'a number of words')
print(3 in [1, 2, 3, 4]) print(3 in [1, 2, 3, 4])
``` ```
<a class="anchor" id="If-statements"></a>
### If statements ### If statements
The basic syntax of `if` statements is fairly standard, though don't forget that you _*must*_ indent the statements within the conditional/loop block as this is the way of delineating blocks of code in python. For example: The basic syntax of `if` statements is fairly standard, though don't forget that you _*must*_ indent the statements within the conditional/loop block as this is the way of delineating blocks of code in python. For example:
...@@ -446,6 +535,7 @@ This can be useful for functions where a variety of possible input types are bei ...@@ -446,6 +535,7 @@ This can be useful for functions where a variety of possible input types are bei
--- ---
<a class="anchor" id="For-loops"></a>
### For loops ### For loops
The `for` loop works like in bash: The `for` loop works like in bash:
...@@ -480,6 +570,7 @@ for x, y in zip(alist, blist): ...@@ -480,6 +570,7 @@ for x, y in zip(alist, blist):
This type of loop can be used with any two lists (or similar) to iterate over them jointly. This type of loop can be used with any two lists (or similar) to iterate over them jointly.
<a class="anchor" id="While-loops"></a>
### While loops ### While loops
The syntax for this is pretty standard: The syntax for this is pretty standard:
...@@ -499,10 +590,12 @@ You can also use `continue` as in other languages. ...@@ -499,10 +590,12 @@ You can also use `continue` as in other languages.
--- ---
<a class="anchor" id="quick-intro"></a>
### A quick intro to conditional expressions and list comprehensions ### A quick intro to conditional expressions and list comprehensions
These are more advanced bits of python but are really useful and common, so worth having a little familiarity with at this stage. These are more advanced bits of python but are really useful and common, so worth having a little familiarity with at this stage.
<a class="anchor" id="Conditional-expressions"></a>
#### Conditional expressions #### Conditional expressions
A general expression that can be used in python is: A `if` condition `else` B A general expression that can be used in python is: A `if` condition `else` B
...@@ -515,7 +608,7 @@ y = x**2 if x<0.5 else (1 - x)**2 ...@@ -515,7 +608,7 @@ y = x**2 if x<0.5 else (1 - x)**2
print(x, y) print(x, y)
``` ```
<a class="anchor" id="List-comprehensions"></a>
#### List comprehensions #### List comprehensions
This is a shorthand syntax for building a list like a for loop but doing it in one line, and is very popular in python. It is quite similar to mathematical set notation. For example: This is a shorthand syntax for building a list like a for loop but doing it in one line, and is very popular in python. It is quite similar to mathematical set notation. For example:
...@@ -529,6 +622,83 @@ print(v2) ...@@ -529,6 +622,83 @@ print(v2)
You'll find that python programmers use this kind of construction _*a lot*_. You'll find that python programmers use this kind of construction _*a lot*_.
---
<a class="anchor" id="functions"></a>
## Functions
You will find functions pretty familiar in python to start with,
although they have a few options which are really handy and different
from C++ or matlab (to be covered in a later practical). To start
with we'll look at a simple function but note a few key points:
* you _must_ indent everything inside the function (it is a code
block and indentation is the only way of determining this - just
like for the guts of a loop)
* you can return _whatever you want_ from a python function, but only
a single object - it is usual to package up multiple things in a
tuple or list, which is easily unpacked by the calling invocation:
e.g., `a, b, c = myfunc(x)`
* parameters are passed by reference (see section on <a
href="#Copying-and-references">copying and references</a>)
```
def myfunc(x, y, z=0):
r2 = x*x + y*y + z*z
r = r2**0.5
return (r, r2)
rad = myfunc(10, 20)
print(rad)
rad, dummy = myfunc(10, 20, 30)
print(rad)
rad, _ = myfunc(10,20,30)
print(rad)
```
> Note that the `_` is used as shorthand here for a dummy variable
> that you want to throw away.
One nice feature of python functions is that you can name the
arguments when you call them, rather than only doing it by position.
For example:
```
def myfunc(x, y, z=0, flag=''):
if flag=='L1':
r = abs(x) + abs(y) + abs(z)
else:
r = (x*x + y*y + z*z)**0.5
return r
rA = myfunc(10, 20)
rB = myfunc(10, 20, flag='L1')
rC = myfunc(10, 20, flag='L1', z=30)
print(rA, rB, rC)
```
You will often see python functions called with these named arguments.
---
<a class="anchor" id="exercise"></a>
## Exercise
Let's say you are given a single string with comma separated elements
that represent filenames and ID codes: e.g., `/vols/Data/pytreat/AAC, 165873, /vols/Data/pytreat/AAG, 170285, ...`
Write some code to do the following:
* separate out the filenames and ID codes into separate lists (ID's
should be numerical values, not strings)
* loop over the two and generate a _string_ that could be used to
rename the directories (e.g., `mv /vols/Data/pytreat/AAC /vols/Data/pytreat/S165873`) - we will cover how to actually execute these in a later practical
* convert your dual lists into a dictionary, with ID as the key
* write a small function to determine if an ID is present in this
set of not, and also return the filename if it is
* write a for loop to create a list of all the odd-numbered IDs (you can use the `%` operator for modulus - i.e., `5 % 2` is 1)
* re-write the for loop as a list comprehension
* now generate a list of the filenames corresponding to these odd-numbered IDs
```
mstr = '/vols/Data/pytreat/AAC, 165873, /vols/Data/pytreat/AAG, 170285, /vols/Data/pytreat/AAH, 196792, /vols/Data/pytreat/AAK, 212577, /vols/Data/pytreat/AAQ, 385376, /vols/Data/pytreat/AB, 444600, /vols/Data/pytreat/AC6, 454578, /vols/Data/pytreat/V8, 501502, /vols/Data/pytreat/2YK, 667688, /vols/Data/pytreat/C3PO, 821971'
```
getting_started/05_nifti.ipynb
+ 71
− 2
View file @ f56044ca
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# NIfTI images and python # NIfTI images and python
The [nibabel](http://nipy.org/nibabel/) module is used to read and write NIfTI images and also some other medical imaging formats (e.g., ANALYZE, GIFTI, MINC, MGH). This module is included within the FSL python environment. The [nibabel](http://nipy.org/nibabel/) module is used to read and write NIfTI images and also some other medical imaging formats (e.g., ANALYZE, GIFTI, MINC, MGH). This module is included within the FSL python environment.
## Contents
* [Reading images](#reading-images)
* [Header info](#header-info)
* [Voxel sizes](#voxel-sizes)
* [Coordinate orientations and mappings](#orientation-info)
* [Writing images](#writing-images)
* [Exercise](#exercise)
---
<a class="anchor" id="reading-images"></a>
## Reading images ## Reading images
It is easy to read an image: It is easy to read an image:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import numpy as np import numpy as np
import nibabel as nib import nibabel as nib
filename = '/usr/local/fsl/data/standard/MNI152_T1_1mm.nii.gz' import os.path as op
filename = op.expandvars('${FSLDIR}/data/standard/MNI152_T1_1mm.nii.gz')
imobj = nib.load(filename, mmap=False) imobj = nib.load(filename, mmap=False)
# display header object # display header object
imhdr = imobj.header imhdr = imobj.header
# extract data (as an numpy array) # extract data (as an numpy array)
imdat = imobj.get_data().astype(float) imdat = imobj.get_data().astype(float)
print(imdat.shape) print(imdat.shape)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> Make sure you use the full filename, including the .nii.gz extension. > Make sure you use the full filename, including the .nii.gz extension.
> We use the expandvars() function above to insert the FSLDIR
>environmental variable into our string. This function is
>discussed more fully in the file management practical.
Reading the data off the disk is not done until `get_data()` is called. Reading the data off the disk is not done until `get_data()` is called.
> Pitfall: > Pitfall:
> >
> The option `mmap=False`is necessary as turns off memory mapping, which otherwise would be invoked for uncompressed NIfTI files but not for compressed files. Since some functionality behaves differently on memory mapped objects, it is advisable to turn this off. > The option `mmap=False`is necessary as turns off memory mapping, which otherwise would be invoked for uncompressed NIfTI files but not for compressed files. Since some functionality behaves differently on memory mapped objects, it is advisable to turn this off.
Once the data is read into a numpy array then it is easily manipulated. Once the data is read into a numpy array then it is easily manipulated.
> We recommend converting it to float at the start to avoid problems with integer arithmetic and overflow, though this is not compulsory. > We recommend converting it to float at the start to avoid problems with integer arithmetic and overflow, though this is not compulsory.
---
<a class="anchor" id="header-info"></a>
## Header info ## Header info
There are many methods available on the header object - for example, look at `dir(imhdr)` or `help(imhdr)` or the [nibabel webpage about NIfTI images](http://nipy.org/nibabel/nifti_images.html) There are many methods available on the header object - for example, look at `dir(imhdr)` or `help(imhdr)` or the [nibabel webpage about NIfTI images](http://nipy.org/nibabel/nifti_images.html)
<a class="anchor" id="voxel-sizes"></a>
### Voxel sizes ### Voxel sizes
Dimensions of the voxels, in mm, can be found from: Dimensions of the voxels, in mm, can be found from:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
voxsize = imhdr.get_zooms() voxsize = imhdr.get_zooms()
print(voxsize) print(voxsize)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="orientation-info"></a>
### Coordinate orientations and mappings ### Coordinate orientations and mappings
Information about the NIfTI qform and sform matrices can be extracted like this: Information about the NIfTI qform and sform matrices can be extracted like this:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
sform = imhdr.get_sform() sform = imhdr.get_sform()
sformcode = imhdr['sform_code'] sformcode = imhdr['sform_code']
qform = imhdr.get_qform() qform = imhdr.get_qform()
qformcode = imhdr['qform_code'] qformcode = imhdr['qform_code']
print(qformcode) print(qformcode)
print(qform) print(qform)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
You can also get both code and matrix together like this:
%% Cell type:code id: tags:
```
affine, code = imhdr.get_qform(coded=True)
print(affine, code)
```
%% Cell type:markdown id: tags:
---
<a class="anchor" id="writing-images"></a>
## Writing images ## Writing images
If you have created a modified image by making or modifying a numpy array then you need to put this into a NIfTI image object in order to save it to a file. The easiest way to do this is to copy all the header info from an existing image like this: If you have created a modified image by making or modifying a numpy array then you need to put this into a NIfTI image object in order to save it to a file. The easiest way to do this is to copy all the header info from an existing image like this:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
newdata = imdat * imdat newdata = imdat * imdat
newhdr = imhdr.copy() newhdr = imhdr.copy()
newobj = nib.nifti1.Nifti1Image(newdata, None, header=newhdr) newobj = nib.nifti1.Nifti1Image(newdata, None, header=newhdr)
nib.save(newobj, "mynewname.nii.gz") nib.save(newobj, "mynewname.nii.gz")
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
where `newdata` is the numpy array (the above is a random example only) and `imhdr` is the existing image header (as above). where `newdata` is the numpy array (the above is a random example only) and `imhdr` is the existing image header (as above).
If the dimensions of the image are different, then extra modifications will be required. For this, or for making an image from scratch, see the [nibabel documentation](http://nipy.org/nibabel/nifti_images.html) on NIfTI images. > It is possible to also just pass in an affine matrix rather than a
> copied header, but we *strongly* recommend against this if you are
> processing an existing image as otherwise you run the risk of
> swapping the left-right orientation. Those that have used
> `save_avw` in matlab may well have been bitten in this way in the
> past. Therefore, copy a header from one of the input images
> whenever possible, and just use the affine matrix option if you are
> creating an entirely separate image, like a simulation.
If the voxel size of the image is different, then extra modifications will be required. For this, or for building an image from scratch, see the [nibabel documentation](http://nipy.org/nibabel/nifti_images.html) on NIfTI images.
---
<a class="anchor" id="exercise"></a>
## Exercise
Write some code to read in a 4D fMRI image (you can find one [here] if
you don't have one handy), calculate the tSNR and then save the 3D result.
%% Cell type:code id: tags:
```
# Calculate tSNR
```
......
getting_started/05_nifti.md
+ 53
− 2
View file @ f56044ca
...@@ -2,6 +2,18 @@ ...@@ -2,6 +2,18 @@
The [nibabel](http://nipy.org/nibabel/) module is used to read and write NIfTI images and also some other medical imaging formats (e.g., ANALYZE, GIFTI, MINC, MGH). This module is included within the FSL python environment. The [nibabel](http://nipy.org/nibabel/) module is used to read and write NIfTI images and also some other medical imaging formats (e.g., ANALYZE, GIFTI, MINC, MGH). This module is included within the FSL python environment.
## Contents
* [Reading images](#reading-images)
* [Header info](#header-info)
* [Voxel sizes](#voxel-sizes)
* [Coordinate orientations and mappings](#orientation-info)
* [Writing images](#writing-images)
* [Exercise](#exercise)
---
<a class="anchor" id="reading-images"></a>
## Reading images ## Reading images
It is easy to read an image: It is easy to read an image:
...@@ -9,7 +21,8 @@ It is easy to read an image: ...@@ -9,7 +21,8 @@ It is easy to read an image:
``` ```
import numpy as np import numpy as np
import nibabel as nib import nibabel as nib
filename = '/usr/local/fsl/data/standard/MNI152_T1_1mm.nii.gz' import os.path as op
filename = op.expandvars('${FSLDIR}/data/standard/MNI152_T1_1mm.nii.gz')
imobj = nib.load(filename, mmap=False) imobj = nib.load(filename, mmap=False)
# display header object # display header object
imhdr = imobj.header imhdr = imobj.header
...@@ -20,6 +33,11 @@ print(imdat.shape) ...@@ -20,6 +33,11 @@ print(imdat.shape)
> Make sure you use the full filename, including the .nii.gz extension. > Make sure you use the full filename, including the .nii.gz extension.
> We use the expandvars() function above to insert the FSLDIR
>environmental variable into our string. This function is
>discussed more fully in the file management practical.
Reading the data off the disk is not done until `get_data()` is called. Reading the data off the disk is not done until `get_data()` is called.
> Pitfall: > Pitfall:
...@@ -30,10 +48,14 @@ Once the data is read into a numpy array then it is easily manipulated. ...@@ -30,10 +48,14 @@ Once the data is read into a numpy array then it is easily manipulated.
> We recommend converting it to float at the start to avoid problems with integer arithmetic and overflow, though this is not compulsory. > We recommend converting it to float at the start to avoid problems with integer arithmetic and overflow, though this is not compulsory.
---
<a class="anchor" id="header-info"></a>
## Header info ## Header info
There are many methods available on the header object - for example, look at `dir(imhdr)` or `help(imhdr)` or the [nibabel webpage about NIfTI images](http://nipy.org/nibabel/nifti_images.html) There are many methods available on the header object - for example, look at `dir(imhdr)` or `help(imhdr)` or the [nibabel webpage about NIfTI images](http://nipy.org/nibabel/nifti_images.html)
<a class="anchor" id="voxel-sizes"></a>
### Voxel sizes ### Voxel sizes
Dimensions of the voxels, in mm, can be found from: Dimensions of the voxels, in mm, can be found from:
...@@ -43,6 +65,7 @@ voxsize = imhdr.get_zooms() ...@@ -43,6 +65,7 @@ voxsize = imhdr.get_zooms()
print(voxsize) print(voxsize)
``` ```
<a class="anchor" id="orientation-info"></a>
### Coordinate orientations and mappings ### Coordinate orientations and mappings
Information about the NIfTI qform and sform matrices can be extracted like this: Information about the NIfTI qform and sform matrices can be extracted like this:
...@@ -56,6 +79,16 @@ print(qformcode) ...@@ -56,6 +79,16 @@ print(qformcode)
print(qform) print(qform)
``` ```
You can also get both code and matrix together like this:
```
affine, code = imhdr.get_qform(coded=True)
print(affine, code)
```
---
<a class="anchor" id="writing-images"></a>
## Writing images ## Writing images
If you have created a modified image by making or modifying a numpy array then you need to put this into a NIfTI image object in order to save it to a file. The easiest way to do this is to copy all the header info from an existing image like this: If you have created a modified image by making or modifying a numpy array then you need to put this into a NIfTI image object in order to save it to a file. The easiest way to do this is to copy all the header info from an existing image like this:
...@@ -68,8 +101,26 @@ nib.save(newobj, "mynewname.nii.gz") ...@@ -68,8 +101,26 @@ nib.save(newobj, "mynewname.nii.gz")
``` ```
where `newdata` is the numpy array (the above is a random example only) and `imhdr` is the existing image header (as above). where `newdata` is the numpy array (the above is a random example only) and `imhdr` is the existing image header (as above).
If the dimensions of the image are different, then extra modifications will be required. For this, or for making an image from scratch, see the [nibabel documentation](http://nipy.org/nibabel/nifti_images.html) on NIfTI images. > It is possible to also just pass in an affine matrix rather than a
> copied header, but we *strongly* recommend against this if you are
> processing an existing image as otherwise you run the risk of
> swapping the left-right orientation. Those that have used
> `save_avw` in matlab may well have been bitten in this way in the
> past. Therefore, copy a header from one of the input images
> whenever possible, and just use the affine matrix option if you are
> creating an entirely separate image, like a simulation.
If the voxel size of the image is different, then extra modifications will be required. For this, or for building an image from scratch, see the [nibabel documentation](http://nipy.org/nibabel/nifti_images.html) on NIfTI images.
---
<a class="anchor" id="exercise"></a>
## Exercise
Write some code to read in a 4D fMRI image (you can find one [here] if
you don't have one handy), calculate the tSNR and then save the 3D result.
```
# Calculate tSNR
```
getting_started/06_plotting.ipynb 0 → 100644
+ 499
− 0
View file @ f56044ca
Source diff could not be displayed: it is too large. Options to address this: view the blob.
getting_started/06_plotting.md 0 → 100644
+ 212
− 0
View file @ f56044ca
# Plotting with python
The main plotting module in python is `matplotlib`. There is a lot
that can be done with it - see the [webpage](https://matplotlib.org/gallery/index.html)
## Contents
* [Running inside a notebook](#inside-notebook)
* [2D plots](#2D-plots)
* [Histograms and Bar Plots](#histograms)
* [Scatter plots](#scatter-plots)
* [Subplots](#subplots)
* [Displaying Images](#displaying-images)
* [3D plots](#3D-plots)
* [Running in a standalone script](#plotting-in-scripts)
* [Exercise](#exercise)
---
<a class="anchor" id="inside-notebook"></a>
## Inside a notebook
Inside a jupyter notebook you get access to this in a slightly
different way, compared to other modules:
```
%matplotlib inline
```
This only needs to be done once in a notebook, like for standard imports.
> There are also other alternatives, including interactive versions - see the practical on Jupyter notebooks for more information about this.
The library works very similarly to plotting in matlab. Let's start
with some simple examples.
<a class="anchor" id="2D-plots"></a>
### 2D plots
```
import matplotlib.pyplot as plt
import numpy as np
plt.style.use('bmh')
x = np.linspace(-np.pi, np.pi, 256)
cosx, sinx = np.cos(x), np.sin(x)
plt.plot(x, cosx)
plt.plot(x, sinx, color='red', linewidth=4, linestyle='-.')
plt.plot(x, sinx**2)
plt.xlim(-np.pi, np.pi)
plt.title('Our first plots')
```
> Note that the `plt.style.use('bmh')` command is not necessary, but it
> does make nicer looking plots in general. You can use `ggplot`
> instead of `bmh` if you want something resembling plots made by R.
> For a list of options run: `print(plt.style.available)`
You can also save the objects and interrogate/set their properties, as
well as those for the general axes:
```
hdl = plt.plot(x, cosx)
print(hdl[0].get_color())
hdl[0].set_color('#707010')
hdl[0].set_linewidth(0.5)
plt.grid('off')
```
Use `dir()` or `help()` or the online docs to get more info on what
you can do with these.
<a class="anchor" id="histograms"></a>
### Histograms and bar charts
For a simple histogram you can do this:
```
r = np.random.rand(1000)
n,bins,_ = plt.hist((r-0.5)**2, bins=30)
```
where it also returns the number of elements in each bin, as `n`, and
the bin centres, as `bins`.
> The `_` in the third part on the left
> hand side is a shorthand for just throwing away the corresponding part
> of the return structure.
There is also a call for doing bar plots:
```
samp1 = r[0:10]
samp2 = r[10:20]
bwidth = 0.3
xcoord = np.arange(10)
plt.bar(xcoord-bwidth, samp1, width=bwidth, color='red', label='Sample 1')
plt.bar(xcoord, samp2, width=bwidth, color='blue', label='Sample 2')
plt.legend(loc='upper left')
```
<a class="anchor" id="scatter-plots"></a>
### Scatter plots
It would be possible to use `plot()` to create a scatter plot, but
there is also an alternative: `scatter()`
```
fig, ax = plt.subplots()
# setup some sizes for each point (arbitrarily example here)
ssize = 100*abs(samp1-samp2) + 10
ax.scatter(samp1, samp2, s=ssize, alpha=0.5)
# now add the y=x line
allsamps = np.hstack((samp1,samp2))
ax.plot([min(allsamps),max(allsamps)],[min(allsamps),max(allsamps)], color='red', linestyle='--')
plt.xlim(min(allsamps),max(allsamps))
plt.ylim(min(allsamps),max(allsamps))
```
> Note that in this case we use the first line return to get a handle to
> the axis, `ax`, and the figure ,`fig`. The axis can be used instead of
> `plt` in most cases, although the `xlim()` and `ylim()` calls can only
> be done through `plt`.
> In general, figures and subplots can be created in matplotlib in a
> similar fashion to matlab, but they do not have to be explicitly
> invoked as you can see from the earlier examples.
<a class="anchor" id="subplots"></a>
### Subplots
These are very similar to matlab:
```
plt.subplot(2, 1, 1)
plt.plot(x,cosx, '.-')
plt.xlim(-np.pi, np.pi)
plt.ylabel('Full sampling')
plt.subplot(2, 1, 2)
plt.plot(x[::30], cosx[::30], '.-')
plt.xlim(-np.pi, np.pi)
plt.ylabel('Subsampled')
```
<a class="anchor" id="displaying-images"></a>
### Displaying images
The main command for displaying images is `imshow()`
```
import nibabel as nib
import os.path as op
nim = nib.load(op.expandvars('${FSLDIR}/data/standard/MNI152_T1_1mm.nii.gz'), mmap=False)
imdat = nim.get_data().astype(float)
plt.imshow(imdat[:,:,70], cmap=plt.cm.gray)
plt.colorbar()
plt.grid('off')
```
<a class="anchor" id="3D-plots"></a>
### 3D plots
```
# Taken from https://matplotlib.org/gallery/mplot3d/wire3d.html#sphx-glr-gallery-mplot3d-wire3d-py
from mpl_toolkits.mplot3d import axes3d
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
# Grab some test data.
X, Y, Z = axes3d.get_test_data(0.05)
# Plot a basic wireframe.
ax.plot_wireframe(X, Y, Z, rstride=10, cstride=10)
```
Surface renderings are many other plots are possible - see 3D examples on
the [matplotlib webpage](https://matplotlib.org/gallery/index.html#mplot3d-examples-index)
---
<a class="anchor" id="plotting-in-scripts"></a>
## Plotting from standalone scripts
When running from a standalone script, the same `matplotlib` import is required,
but the line `%matplotlib <backend>` should *not* be used.
In a script it is also necessary to _finish_ with `plt.show()` as
otherwise nothing is actually displayed. For example, the above
examples would setup a plot but the actual graphic would only appear
after the `plt.show()` command was executed. Furthermore, control is
not returned to the script immediately as the plot is interactive by default.
---
<a class="anchor" id="exercise"></a>
## Exercise
Find a different type of plot (e.g., boxplot, violin plot, quiver
plot, pie chart, etc.), look up
the documentation and then write _your own code that calls this_ to create a plot
from some data that you create yourself (i.e., don't just blindly copy
example code from the docs).
```
# Make up some data and do the funky plot
```
getting_started/08_scripts.ipynb
+ 39
− 1
View file @ f56044ca
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Callable scripts in python # Callable scripts in python
In this tutorial we will cover how to write simple stand-alone scripts in python that can be used as alternatives to bash scripts. In this tutorial we will cover how to write simple stand-alone scripts in python that can be used as alternatives to bash scripts.
There are some code blocks within this webpage, but we recommend that you write the code in an IDE or editor instead and then run the scripts from a terminal. There are some code blocks within this webpage, but for this practical we _**strongly
recommend that you write the code in an IDE or editor**_ instead and then run the scripts from a terminal.
## Contents
* [Basic script](#basic-script)
* [Calling other executables](#calling-other-executables)
* [Command line arguments](#command-line-arguments)
* [Example script](#example-script)
* [Exercise](#exercise)
---
<a class="anchor" id="basic-script"></a>
## Basic script ## Basic script
The first line of a python script is usually: The first line of a python script is usually:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
#!/usr/bin/env python #!/usr/bin/env python
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
which invokes whichever version of python can be found by `/usr/bin/env` since python can be located in many different places. which invokes whichever version of python can be found by `/usr/bin/env` since python can be located in many different places.
For FSL scripts we use an alternative, to ensure that we pick up the version of python (and associated packages) that we ship with FSL. To do this we use the line: For FSL scripts we use an alternative, to ensure that we pick up the version of python (and associated packages) that we ship with FSL. To do this we use the line:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
#!/usr/bin/env fslpython #!/usr/bin/env fslpython
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
After this line the rest of the file just uses regular python syntax, as in the other tutorials. Make sure you make the file executable - just like a bash script. After this line the rest of the file just uses regular python syntax, as in the other tutorials. Make sure you make the file executable - just like a bash script.
<a class="anchor" id="calling-other-executables"></a>
## Calling other executables ## Calling other executables
The most essential call that you need to use to replicate the way a bash script calls executables is `subprocess.run()`. A simple call looks like this: The most essential call that you need to use to replicate the way a bash script calls executables is `subprocess.run()`. A simple call looks like this:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import subprocess as sp import subprocess as sp
sp.run(['ls', '-la']) sp.run(['ls', '-la'])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
To suppress the output do this: To suppress the output do this:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
spobj = sp.run(['ls'], stdout = sp.PIPE) spobj = sp.run(['ls'], stdout = sp.PIPE)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
To store the output do this: To store the output do this:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
spobj = sp.run('ls -la'.split(), stdout = sp.PIPE) spobj = sp.run('ls -la'.split(), stdout = sp.PIPE)
sout = spobj.stdout.decode('utf-8') sout = spobj.stdout.decode('utf-8')
print(sout) print(sout)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
> Note that the `decode` call in the middle line converts the string from a byte string to a normal string. In Python 3 there is a distinction between strings (sequences of characters, possibly using multiple bytes to store each character) and bytes (sequences of bytes). The world has moved on from ASCII, so in this day and age, this distinction is absolutely necessary, and Python does a fairly good job of it. > Note that the `decode` call in the middle line converts the string from a byte string to a normal string. In Python 3 there is a distinction between strings (sequences of characters, possibly using multiple bytes to store each character) and bytes (sequences of bytes). The world has moved on from ASCII, so in this day and age, this distinction is absolutely necessary, and Python does a fairly good job of it.
If the output is numerical then this can be extracted like this: If the output is numerical then this can be extracted like this:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import os import os
fsldir = os.getenv('FSLDIR') fsldir = os.getenv('FSLDIR')
spobj = sp.run([fsldir+'/bin/fslstats', fsldir+'/data/standard/MNI152_T1_1mm_brain', '-V'], stdout = sp.PIPE) spobj = sp.run([fsldir+'/bin/fslstats', fsldir+'/data/standard/MNI152_T1_1mm_brain', '-V'], stdout = sp.PIPE)
sout = spobj.stdout.decode('utf-8') sout = spobj.stdout.decode('utf-8')
vol_vox = float(sout.split()[0]) vol_vox = float(sout.split()[0])
vol_mm = float(sout.split()[1]) vol_mm = float(sout.split()[1])
print('Volumes are: ', vol_vox, ' in voxels and ', vol_mm, ' in mm') print('Volumes are: ', vol_vox, ' in voxels and ', vol_mm, ' in mm')
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
An alternative way to run a set of commands would be like this: An alternative way to run a set of commands would be like this:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
commands = """ commands = """
{fsldir}/bin/fslmaths {t1} -bin {t1_mask} {fsldir}/bin/fslmaths {t1} -bin {t1_mask}
{fsldir}/bin/fslmaths {t2} -mas {t1_mask} {t2_masked} {fsldir}/bin/fslmaths {t2} -mas {t1_mask} {t2_masked}
""" """
fsldirpath = os.getenv('FSLDIR') fsldirpath = os.getenv('FSLDIR')
commands = commands.format(t1 = 't1.nii.gz', t1_mask = 't1_mask', t2 = 't2', t2_masked = 't2_masked', fsldir = fsldirpath) commands = commands.format(t1 = 't1.nii.gz', t1_mask = 't1_mask', t2 = 't2', t2_masked = 't2_masked', fsldir = fsldirpath)
sout=[] sout=[]
for cmd in commands.split('\n'): for cmd in commands.split('\n'):
if cmd: # avoids empty strings getting passed to sp.run() if cmd: # avoids empty strings getting passed to sp.run()
print('Running command: ', cmd) print('Running command: ', cmd)
spobj = sp.run(cmd.split(), stdout = sp.PIPE) spobj = sp.run(cmd.split(), stdout = sp.PIPE)
sout.append(spobj.stdout.decode('utf-8')) sout.append(spobj.stdout.decode('utf-8'))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
<a class="anchor" id="command-line-arguments"></a>
## Command line arguments ## Command line arguments
The simplest way of dealing with command line arguments is use the module `sys`, which gives access to an `argv` list: The simplest way of dealing with command line arguments is use the module `sys`, which gives access to an `argv` list:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
import sys import sys
print(len(sys.argv)) print(len(sys.argv))
print(sys.argv[0]) print(sys.argv[0])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
For more sophisticated argument parsing you can use `argparse` - good documentation and examples of this can be found on the web. For more sophisticated argument parsing you can use `argparse` - good documentation and examples of this can be found on the web.
---
<a class="anchor" id="example-script"></a>
## Example script ## Example script
Here is a simple bash script (it masks an image and calculates volumes - just as a random example). DO NOT execute the code blocks here within the notebook/webpage: Here is a simple bash script (it masks an image and calculates volumes - just as a random example). DO NOT execute the code blocks here within the notebook/webpage:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
#!/bin/bash #!/bin/bash
if [ $# -lt 2 ] ; then if [ $# -lt 2 ] ; then
echo "Usage: $0 <input image> <output image>" echo "Usage: $0 <input image> <output image>"
exit 1 exit 1
fi fi
infile=$1 infile=$1
outfile=$2 outfile=$2
# mask input image with MNI # mask input image with MNI
$FSLDIR/bin/fslmaths $infile -mas $FSLDIR/data/standard/MNI152_T1_1mm_brain $outfile $FSLDIR/bin/fslmaths $infile -mas $FSLDIR/data/standard/MNI152_T1_1mm_brain $outfile
# calculate volumes of masked image # calculate volumes of masked image
vv=`$FSLDIR/bin/fslstats $outfile -V` vv=`$FSLDIR/bin/fslstats $outfile -V`
vol_vox=`echo $vv | awk '{ print $1 }'` vol_vox=`echo $vv | awk '{ print $1 }'`
vol_mm=`echo $vv | awk '{ print $2 }'` vol_mm=`echo $vv | awk '{ print $2 }'`
echo "Volumes are: $vol_vox in voxels and $vol_mm in mm" echo "Volumes are: $vol_vox in voxels and $vol_mm in mm"
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
And an alternative in python: And an alternative in python:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ```
#!/usr/bin/env fslpython #!/usr/bin/env fslpython
import os, sys import os, sys
import subprocess as sp import subprocess as sp
fsldir=os.getenv('FSLDIR') fsldir=os.getenv('FSLDIR')
if len(sys.argv)<2: if len(sys.argv)<2:
print('Usage: ', sys.argv[0], ' <input image> <output image>') print('Usage: ', sys.argv[0], ' <input image> <output image>')
sys.exit(1) sys.exit(1)
infile = sys.argv[1] infile = sys.argv[1]
outfile = sys.argv[2] outfile = sys.argv[2]
# mask input image with MNI # mask input image with MNI
spobj = sp.run([fsldir+'/bin/fslmaths', infile, '-mas', fsldir+'/data/standard/MNI152_T1_1mm_brain', outfile], stdout = sp.PIPE) spobj = sp.run([fsldir+'/bin/fslmaths', infile, '-mas', fsldir+'/data/standard/MNI152_T1_1mm_brain', outfile], stdout = sp.PIPE)
# calculate volumes of masked image # calculate volumes of masked image
spobj = sp.run([fsldir+'/bin/fslstats', outfile, '-V'], stdout = sp.PIPE) spobj = sp.run([fsldir+'/bin/fslstats', outfile, '-V'], stdout = sp.PIPE)
sout = spobj.stdout.decode('utf-8') sout = spobj.stdout.decode('utf-8')
vol_vox = float(sout.split()[0]) vol_vox = float(sout.split()[0])
vol_mm = float(sout.split()[1]) vol_mm = float(sout.split()[1])
print('Volumes are: ', vol_vox, ' in voxels and ', vol_mm, ' in mm') print('Volumes are: ', vol_vox, ' in voxels and ', vol_mm, ' in mm')
``` ```
%% Cell type:markdown id: tags:
---
<a class="anchor" id="exercise"></a>
## Exercise
Write a simple version of fslstats that is able to calculate either a
mean or a _sum_ (and hence can do something that fslstats cannot!)
%% Cell type:code id: tags:
```
# Don't write anything here - do it in a standalone script!
```
......
getting_started/08_scripts.md
+ 28
− 1
View file @ f56044ca
...@@ -2,8 +2,20 @@ ...@@ -2,8 +2,20 @@
In this tutorial we will cover how to write simple stand-alone scripts in python that can be used as alternatives to bash scripts. In this tutorial we will cover how to write simple stand-alone scripts in python that can be used as alternatives to bash scripts.
There are some code blocks within this webpage, but we recommend that you write the code in an IDE or editor instead and then run the scripts from a terminal. There are some code blocks within this webpage, but for this practical we _**strongly
recommend that you write the code in an IDE or editor**_ instead and then run the scripts from a terminal.
## Contents
* [Basic script](#basic-script)
* [Calling other executables](#calling-other-executables)
* [Command line arguments](#command-line-arguments)
* [Example script](#example-script)
* [Exercise](#exercise)
---
<a class="anchor" id="basic-script"></a>
## Basic script ## Basic script
The first line of a python script is usually: The first line of a python script is usually:
...@@ -19,6 +31,7 @@ For FSL scripts we use an alternative, to ensure that we pick up the version of ...@@ -19,6 +31,7 @@ For FSL scripts we use an alternative, to ensure that we pick up the version of
After this line the rest of the file just uses regular python syntax, as in the other tutorials. Make sure you make the file executable - just like a bash script. After this line the rest of the file just uses regular python syntax, as in the other tutorials. Make sure you make the file executable - just like a bash script.
<a class="anchor" id="calling-other-executables"></a>
## Calling other executables ## Calling other executables
The most essential call that you need to use to replicate the way a bash script calls executables is `subprocess.run()`. A simple call looks like this: The most essential call that you need to use to replicate the way a bash script calls executables is `subprocess.run()`. A simple call looks like this:
...@@ -77,6 +90,7 @@ for cmd in commands.split('\n'): ...@@ -77,6 +90,7 @@ for cmd in commands.split('\n'):
``` ```
<a class="anchor" id="command-line-arguments"></a>
## Command line arguments ## Command line arguments
The simplest way of dealing with command line arguments is use the module `sys`, which gives access to an `argv` list: The simplest way of dealing with command line arguments is use the module `sys`, which gives access to an `argv` list:
...@@ -88,7 +102,9 @@ print(sys.argv[0]) ...@@ -88,7 +102,9 @@ print(sys.argv[0])
For more sophisticated argument parsing you can use `argparse` - good documentation and examples of this can be found on the web. For more sophisticated argument parsing you can use `argparse` - good documentation and examples of this can be found on the web.
---
<a class="anchor" id="example-script"></a>
## Example script ## Example script
Here is a simple bash script (it masks an image and calculates volumes - just as a random example). DO NOT execute the code blocks here within the notebook/webpage: Here is a simple bash script (it masks an image and calculates volumes - just as a random example). DO NOT execute the code blocks here within the notebook/webpage:
...@@ -133,4 +149,15 @@ vol_mm = float(sout.split()[1]) ...@@ -133,4 +149,15 @@ vol_mm = float(sout.split()[1])
print('Volumes are: ', vol_vox, ' in voxels and ', vol_mm, ' in mm') print('Volumes are: ', vol_vox, ' in voxels and ', vol_mm, ' in mm')
``` ```
---
<a class="anchor" id="exercise"></a>
## Exercise
Write a simple version of fslstats that is able to calculate either a
mean or a _sum_ (and hence can do something that fslstats cannot!)
```
# Don't write anything here - do it in a standalone script!
```
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment