Skip to content
Snippets Groups Projects
Commit d66a541c authored by Michiel Cottaar's avatar Michiel Cottaar
Browse files

mention many more packages

parent fee86d96
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
talks/packages/packages.ipynb
+ 206
− 38
View file @ d66a541c
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Main scientific python libraries # Main scientific python libraries
See https://scipy.org/ See https://scipy.org/
Most of these packages have or are in thr progress of dropping support for python2. Most of these packages have or are in thr progress of dropping support for python2.
So use python3! So use python3!
## [Numpy](http://www.numpy.org/): arrays ## [Numpy](http://www.numpy.org/): arrays
This is the main library underlying (nearly) all of the scientific python ecosystem. This is the main library underlying (nearly) all of the scientific python ecosystem.
See the tutorial in the beginner session or [the official numpy tutorial](https://docs.scipy.org/doc/numpy-dev/user/quickstart.html) for usage details. See the tutorial in the beginner session or [the official numpy tutorial](https://docs.scipy.org/doc/numpy-dev/user/quickstart.html) for usage details.
The usual nickname of numpy is np: The usual nickname of numpy is np:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
import numpy as np import numpy as np
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Numpy includes support for: Numpy includes support for:
- N-dimensional arrays with various datatypes - N-dimensional arrays with various datatypes
- masked arrays - masked arrays
- matrices - matrices
- structured/record array - structured/record array
- basic functions (e.g., sin, log, arctan, polynomials) - basic functions (e.g., sin, log, arctan, polynomials)
- basic linear algebra - basic linear algebra
- random number generation - random number generation
## [Scipy](https://scipy.org/scipylib/index.html): most general scientific tools ## [Scipy](https://scipy.org/scipylib/index.html): most general scientific tools
At the top level this module includes all of the basic functionality from numpy. At the top level this module includes all of the basic functionality from numpy.
You could import this as, but you might as well import numpy directly. You could import this as, but you might as well import numpy directly.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
import scipy as sp import scipy as sp
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The main strength in scipy lies in its sub-packages: The main strength in scipy lies in its sub-packages:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
from scipy import optimize from scipy import optimize
def costfunc(params): def costfunc(params):
return params[0] ** 2 * (params[1] - 3) ** 2 + (params[0] - 2) ** 2 return params[0] ** 2 * (params[1] - 3) ** 2 + (params[0] - 2) ** 2
optimize.minimize(costfunc, x0=[0, 0], method='l-bfgs-b') optimize.minimize(costfunc, x0=[0, 0], method='l-bfgs-b')
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Tutorials for all sub-packages can be found [here](https://docs.scipy.org/doc/scipy-1.0.0/reference/). Tutorials for all sub-packages can be found [here](https://docs.scipy.org/doc/scipy-1.0.0/reference/).
## [Matplotlib](https://matplotlib.org/): Main plotting library ## [Matplotlib](https://matplotlib.org/): Main plotting library
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
import matplotlib as mpl import matplotlib as mpl
mpl.use('nbagg') mpl.use('nbagg')
import matplotlib.pyplot as plt import matplotlib.pyplot as plt
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The matplotlib tutorials are [here](https://matplotlib.org/tutorials/index.html) The matplotlib tutorials are [here](https://matplotlib.org/tutorials/index.html)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
x = np.linspace(0, 2, 100) x = np.linspace(0, 2, 100)
plt.plot(x, x, label='linear') plt.plot(x, x, label='linear')
plt.plot(x, x**2, label='quadratic') plt.plot(x, x**2, label='quadratic')
plt.plot(x, x**3, label='cubic') plt.plot(x, x**3, label='cubic')
plt.xlabel('x label') plt.xlabel('x label')
plt.ylabel('y label') plt.ylabel('y label')
plt.title("Simple Plot") plt.title("Simple Plot")
plt.legend() plt.legend()
plt.show() plt.show()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Alternatives: Alternatives:
- [Mayavi](http://docs.enthought.com/mayavi/mayavi/): 3D plotting (hard to install) - [Mayavi](http://docs.enthought.com/mayavi/mayavi/): 3D plotting (hard to install)
- [Bokeh](https://bokeh.pydata.org/en/latest/) among many others: interactive plots in the browser (i.e., in javascript) - [Bokeh](https://bokeh.pydata.org/en/latest/) among many others: interactive plots in the browser (i.e., in javascript)
## [Ipython](http://ipython.org/)/[Jupyter](https://jupyter.org/) notebook: interactive python environments ## [Ipython](http://ipython.org/)/[Jupyter](https://jupyter.org/) notebook: interactive python environments
There are many [useful extensions available](https://github.com/ipython-contrib/jupyter_contrib_nbextensions). There are many [useful extensions available](https://github.com/ipython-contrib/jupyter_contrib_nbextensions).
## [Pandas](https://pandas.pydata.org/): Analyzing "clean" data ## [Pandas](https://pandas.pydata.org/): Analyzing "clean" data
Once your data is in tabular form (e.g. Biobank IDP's), you want to use pandas dataframes to analyze them. Once your data is in tabular form (e.g. Biobank IDP's), you want to use pandas dataframes to analyze them.
This brings most of the functionality of R into python. This brings most of the functionality of R into python.
Pandas has excellent support for: Pandas has excellent support for:
- fast IO to many tabular formats - fast IO to many tabular formats
- accurate handling of missing data - accurate handling of missing data
- Many, many routines to handle data - Many, many routines to handle data
- group by categorical data (i.e., male/female, or age groups) - group by categorical data (i.e., male/female, or age groups)
- joining/merging data - joining/merging data
- time series support - time series support
- statistical models through [statsmodels](http://www.statsmodels.org/stable/index.html) - statistical models through [statsmodels](http://www.statsmodels.org/stable/index.html)
- plotting though seaborn [seaborn](https://seaborn.pydata.org/) - plotting though seaborn [seaborn](https://seaborn.pydata.org/)
- Use [dask](https://dask.pydata.org/en/latest/) if your data is too big for memory (or if you want to run in parallel) - Use [dask](https://dask.pydata.org/en/latest/) if your data is too big for memory (or if you want to run in parallel)
You should also install `numexpr` and `bottleneck` for optimal performance. You should also install `numexpr` and `bottleneck` for optimal performance.
For the documentation check [here](http://pandas.pydata.org/pandas-docs/stable/index.html) For the documentation check [here](http://pandas.pydata.org/pandas-docs/stable/index.html)
### Adjusted example from statsmodels tutorial ### Adjusted example from statsmodels tutorial
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
import statsmodels.api as sm import statsmodels.api as sm
import statsmodels.formula.api as smf import statsmodels.formula.api as smf
import numpy as np import numpy as np
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
df = sm.datasets.get_rdataset("Guerry", "HistData").data df = sm.datasets.get_rdataset("Guerry", "HistData").data
df df
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
df.describe() df.describe()
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
df.groupby('Region').mean() df.groupby('Region').mean()
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
results = smf.ols('Lottery ~ Literacy + np.log(Pop1831)', data=df).fit() results = smf.ols('Lottery ~ Literacy + np.log(Pop1831)', data=df).fit()
results.summary() results.summary()
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
df['log_pop'] = np.log(df.Pop1831) df['log_pop'] = np.log(df.Pop1831)
df df
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
results = smf.ols('Lottery ~ Literacy + log_pop', data=df).fit() results = smf.ols('Lottery ~ Literacy + log_pop', data=df).fit()
results.summary() results.summary()
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
results = smf.ols('Lottery ~ Literacy + np.log(Pop1831) + Region', data=df).fit() results = smf.ols('Lottery ~ Literacy + np.log(Pop1831) + Region', data=df).fit()
results.summary() results.summary()
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
results = smf.ols('Lottery ~ Literacy + np.log(Pop1831) + Region + Region * Literacy', data=df).fit() results = smf.ols('Lottery ~ Literacy + np.log(Pop1831) + Region + Region * Literacy', data=df).fit()
results.summary() results.summary()
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
%matplotlib nbagg %matplotlib nbagg
import seaborn as sns import seaborn as sns
sns.pairplot(df, hue="Region", vars=('Lottery', 'Literacy', 'log_pop')) sns.pairplot(df, hue="Region", vars=('Lottery', 'Literacy', 'log_pop'))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## [Sympy](http://www.sympy.org/en/index.html): Symbolic programming ## [Sympy](http://www.sympy.org/en/index.html): Symbolic programming
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
import sympy as sym # no standard nickname import sympy as sym # no standard nickname
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
x, a, b, c = sym.symbols('x, a, b, c') x, a, b, c = sym.symbols('x, a, b, c')
sym.solve(a * x ** 2 + b * x + c, x) sym.solve(a * x ** 2 + b * x + c, x)
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
sym.integrate(x/(x**2+a*x+2), x) sym.integrate(x/(x**2+a*x+2), x)
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
f = sym.utilities.lambdify((x, a), sym.integrate((x**2+a*x+2), x)) f = sym.utilities.lambdify((x, a), sym.integrate((x**2+a*x+2), x))
f(np.random.rand(10), np.random.rand(10)) f(np.random.rand(10), np.random.rand(10))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Other topics # Other topics
## [Argparse](https://docs.python.org/3.6/howto/argparse.html): Command line arguments ## [Argparse](https://docs.python.org/3.6/howto/argparse.html): Command line arguments
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
%%writefile test_argparse.py %%writefile test_argparse.py
import argparse import argparse
def main(): def main():
parser = argparse.ArgumentParser(description="calculate X to the power of Y") parser = argparse.ArgumentParser(description="calculate X to the power of Y")
parser.add_argument("-v", "--verbose", action="store_true") parser.add_argument("-v", "--verbose", action="store_true")
parser.add_argument("x", type=int, help="the base") parser.add_argument("x", type=int, help="the base")
parser.add_argument("y", type=int, help="the exponent") parser.add_argument("y", type=int, help="the exponent")
args = parser.parse_args() args = parser.parse_args()
answer = args.x**args.y answer = args.x**args.y
if args.verbose: if args.verbose:
print("{} to the power {} equals {}".format(args.x, args.y, answer)) print("{} to the power {} equals {}".format(args.x, args.y, answer))
else: else:
print("{}^{} == {}".format(args.x, args.y, answer)) print("{}^{} == {}".format(args.x, args.y, answer))
if __name__ == '__main__': if __name__ == '__main__':
main() main()
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
%run test_argparse.py 3 8 -v %run test_argparse.py 3 8 -v
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
%run test_argparse.py -h %run test_argparse.py -h
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
%run test_argparse.py 3 8.5 -q %run test_argparse.py 3 8.5 -q
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Alternatives:
- [docopt](http://docopt.org/): You write a usage string, docopt will generate the parser
> ```
> # example from https://realpython.com/blog/python/comparing-python-command-line-parsing-libraries-argparse-docopt-click/
> """Greeter.
>
> Usage:
> commands.py hello
> commands.py goodbye
> commands.py -h | --help
>
> Options:
> -h --help Show this screen.
> """
> from docopt import docopt
>
> if __name__ == '__main__':
> arguments = docopt(__doc__)
> ```
- [clize](http://clize.readthedocs.io/en/stable/why.html): You write a function, clize will generate the parser
> ```
> from clize import run
>
> def echo(word):
> return word
>
> if __name__ == '__main__':
> run(echo)
> ```
### [Gooey](https://github.com/chriskiehl/Gooey): GUI from command line tool ### [Gooey](https://github.com/chriskiehl/Gooey): GUI from command line tool
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
%%writefile test_gooey.py %%writefile test_gooey.py
import argparse import argparse
from gooey import Gooey from gooey import Gooey
@Gooey @Gooey
def main(): def main():
parser = argparse.ArgumentParser(description="calculate X to the power of Y") parser = argparse.ArgumentParser(description="calculate X to the power of Y")
parser.add_argument("-v", "--verbose", action="store_true") parser.add_argument("-v", "--verbose", action="store_true")
parser.add_argument("x", type=int, help="the base") parser.add_argument("x", type=int, help="the base")
parser.add_argument("y", type=int, help="the exponent") parser.add_argument("y", type=int, help="the exponent")
args = parser.parse_args() args = parser.parse_args()
answer = args.x**args.y answer = args.x**args.y
if args.verbose: if args.verbose:
print("{} to the power {} equals {}".format(args.x, args.y, answer)) print("{} to the power {} equals {}".format(args.x, args.y, answer))
else: else:
print("{}^{} == {}".format(args.x, args.y, answer)) print("{}^{} == {}".format(args.x, args.y, answer))
if __name__ == '__main__': if __name__ == '__main__':
main() main()
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
%run test_gooey.py %run test_gooey.py
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
!gcoord_gui !gcoord_gui
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## [Jinja2](http://jinja.pocoo.org/docs/2.10/): HTML generation ## [Jinja2](http://jinja.pocoo.org/docs/2.10/): Templating language
Jinja2 allows to create templates of files with placeholders, where future content will go.
This allows for the creation of a large number of similar files.
This can for example be used to produce static HTML output in a highly flexible manner.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
%%writefile image_list.jinja2 %%writefile image_list.jinja2
<!DOCTYPE html> <!DOCTYPE html>
<html lang="en"> <html lang="en">
<head> <head>
{% block head %} {% block head %}
<title>{{ title }}</title> <title>{{ title }}</title>
{% endblock %} {% endblock %}
</head> </head>
<body> <body>
<div id="content"> <div id="content">
{% block content %} {% block content %}
{% for description, filenames in images %} {% for description, filenames in images %}
<p> <p>
{{ description }} {{ description }}
</p> </p>
{% for filename in filenames %} {% for filename in filenames %}
<a href="{{ filename }}"> <a href="{{ filename }}">
<img src="{{ filename }}"> <img src="{{ filename }}">
</a> </a>
{% endfor %} {% endfor %}
{% endfor %} {% endfor %}
{% endblock %} {% endblock %}
</div> </div>
<footer> <footer>
Created on {{ time }} Created on {{ time }}
</footer> </footer>
</body> </body>
</html> </html>
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
def plot_sine(amplitude, frequency): def plot_sine(amplitude, frequency):
x = np.linspace(0, 2 * np.pi, 100) x = np.linspace(0, 2 * np.pi, 100)
y = amplitude * np.sin(frequency * x) y = amplitude * np.sin(frequency * x)
plt.plot(x, y) plt.plot(x, y)
plt.xticks([0, np.pi, 2 * np.pi], ['0', '$\pi$', '$2 \pi$']) plt.xticks([0, np.pi, 2 * np.pi], ['0', '$\pi$', '$2 \pi$'])
plt.ylim(-1.1, 1.1) plt.ylim(-1.1, 1.1)
filename = 'plots/A{:.2f}_F{:.2f}.png'.format(amplitude, frequency) filename = 'plots/A{:.2f}_F{:.2f}.png'.format(amplitude, frequency)
plt.title('A={:.2f}, F={:.2f}'.format(amplitude, frequency)) plt.title('A={:.2f}, F={:.2f}'.format(amplitude, frequency))
plt.savefig(filename) plt.savefig(filename)
plt.close(plt.gcf()) plt.close(plt.gcf())
return filename return filename
!mkdir plots !mkdir plots
amplitudes = [plot_sine(A, 1.) for A in [0.1, 0.3, 0.7, 1.0]] amplitudes = [plot_sine(A, 1.) for A in [0.1, 0.3, 0.7, 1.0]]
frequencies = [plot_sine(1., F) for F in [1, 2, 3, 4, 5, 6]] frequencies = [plot_sine(1., F) for F in [1, 2, 3, 4, 5, 6]]
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
from jinja2 import Environment, FileSystemLoader from jinja2 import Environment, FileSystemLoader
from datetime import datetime from datetime import datetime
loader = FileSystemLoader('.') loader = FileSystemLoader('.')
env = Environment(loader=loader) env = Environment(loader=loader)
template = env.get_template('image_list.jinja2') template = env.get_template('image_list.jinja2')
images = [ images = [
('Varying the amplitude', amplitudes), ('Varying the amplitude', amplitudes),
('Varying the frequency', frequencies), ('Varying the frequency', frequencies),
] ]
with open('image_list.html', 'w') as f: with open('image_list.html', 'w') as f:
f.write(template.render(title='Lots of sines', f.write(template.render(title='Lots of sines',
images=images, time=datetime.now())) images=images, time=datetime.now()))
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
!open image_list.html !open image_list.html
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Neuroimage packages ## Neuroimage packages
The [nipy](http://nipy.org/) ecosystem covers most of these. The [nipy](http://nipy.org/) ecosystem covers most of these.
## [networkx](https://networkx.github.io/): graph theory ## [networkx](https://networkx.github.io/): graph theory
## GUI ## GUI
- [tkinter](https://docs.python.org/3.6/library/tkinter.html): thin wrapper around Tcl/Tk; included in python - [tkinter](https://docs.python.org/3.6/library/tkinter.html): thin wrapper around Tcl/Tk; included in python
- [wxpython](https://www.wxpython.org/): Wrapper around the C++ wxWidgets library - [wxpython](https://www.wxpython.org/): Wrapper around the C++ wxWidgets library
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
%%writefile wx_hello_world.py %%writefile wx_hello_world.py
#!/usr/bin/env python #!/usr/bin/env python
""" """
Hello World, but with more meat. Hello World, but with more meat.
""" """
import wx import wx
class HelloFrame(wx.Frame): class HelloFrame(wx.Frame):
""" """
A Frame that says Hello World A Frame that says Hello World
""" """
def __init__(self, *args, **kw): def __init__(self, *args, **kw):
# ensure the parent's __init__ is called # ensure the parent's __init__ is called
super(HelloFrame, self).__init__(*args, **kw) super(HelloFrame, self).__init__(*args, **kw)
# create a panel in the frame # create a panel in the frame
pnl = wx.Panel(self) pnl = wx.Panel(self)
# and put some text with a larger bold font on it # and put some text with a larger bold font on it
st = wx.StaticText(pnl, label="Hello World!", pos=(25,25)) st = wx.StaticText(pnl, label="Hello World!", pos=(25,25))
font = st.GetFont() font = st.GetFont()
font.PointSize += 10 font.PointSize += 10
font = font.Bold() font = font.Bold()
st.SetFont(font) st.SetFont(font)
# create a menu bar # create a menu bar
self.makeMenuBar() self.makeMenuBar()
# and a status bar # and a status bar
self.CreateStatusBar() self.CreateStatusBar()
self.SetStatusText("Welcome to wxPython!") self.SetStatusText("Welcome to wxPython!")
def makeMenuBar(self): def makeMenuBar(self):
""" """
A menu bar is composed of menus, which are composed of menu items. A menu bar is composed of menus, which are composed of menu items.
This method builds a set of menus and binds handlers to be called This method builds a set of menus and binds handlers to be called
when the menu item is selected. when the menu item is selected.
""" """
# Make a file menu with Hello and Exit items # Make a file menu with Hello and Exit items
fileMenu = wx.Menu() fileMenu = wx.Menu()
# The "\t..." syntax defines an accelerator key that also triggers # The "\t..." syntax defines an accelerator key that also triggers
# the same event # the same event
helloItem = fileMenu.Append(-1, "&Hello...\tCtrl-H", helloItem = fileMenu.Append(-1, "&Hello...\tCtrl-H",
"Help string shown in status bar for this menu item") "Help string shown in status bar for this menu item")
fileMenu.AppendSeparator() fileMenu.AppendSeparator()
# When using a stock ID we don't need to specify the menu item's # When using a stock ID we don't need to specify the menu item's
# label # label
exitItem = fileMenu.Append(wx.ID_EXIT) exitItem = fileMenu.Append(wx.ID_EXIT)
# Now a help menu for the about item # Now a help menu for the about item
helpMenu = wx.Menu() helpMenu = wx.Menu()
aboutItem = helpMenu.Append(wx.ID_ABOUT) aboutItem = helpMenu.Append(wx.ID_ABOUT)
# Make the menu bar and add the two menus to it. The '&' defines # Make the menu bar and add the two menus to it. The '&' defines
# that the next letter is the "mnemonic" for the menu item. On the # that the next letter is the "mnemonic" for the menu item. On the
# platforms that support it those letters are underlined and can be # platforms that support it those letters are underlined and can be
# triggered from the keyboard. # triggered from the keyboard.
menuBar = wx.MenuBar() menuBar = wx.MenuBar()
menuBar.Append(fileMenu, "&File") menuBar.Append(fileMenu, "&File")
menuBar.Append(helpMenu, "&Help") menuBar.Append(helpMenu, "&Help")
# Give the menu bar to the frame # Give the menu bar to the frame
self.SetMenuBar(menuBar) self.SetMenuBar(menuBar)
# Finally, associate a handler function with the EVT_MENU event for # Finally, associate a handler function with the EVT_MENU event for
# each of the menu items. That means that when that menu item is # each of the menu items. That means that when that menu item is
# activated then the associated handler function will be called. # activated then the associated handler function will be called.
self.Bind(wx.EVT_MENU, self.OnHello, helloItem) self.Bind(wx.EVT_MENU, self.OnHello, helloItem)
self.Bind(wx.EVT_MENU, self.OnExit, exitItem) self.Bind(wx.EVT_MENU, self.OnExit, exitItem)
self.Bind(wx.EVT_MENU, self.OnAbout, aboutItem) self.Bind(wx.EVT_MENU, self.OnAbout, aboutItem)
def OnExit(self, event): def OnExit(self, event):
"""Close the frame, terminating the application.""" """Close the frame, terminating the application."""
self.Close(True) self.Close(True)
def OnHello(self, event): def OnHello(self, event):
"""Say hello to the user.""" """Say hello to the user."""
wx.MessageBox("Hello again from wxPython") wx.MessageBox("Hello again from wxPython")
def OnAbout(self, event): def OnAbout(self, event):
"""Display an About Dialog""" """Display an About Dialog"""
wx.MessageBox("This is a wxPython Hello World sample", wx.MessageBox("This is a wxPython Hello World sample",
"About Hello World 2", "About Hello World 2",
wx.OK|wx.ICON_INFORMATION) wx.OK|wx.ICON_INFORMATION)
if __name__ == '__main__': if __name__ == '__main__':
# When this module is run (not imported) then create the app, the # When this module is run (not imported) then create the app, the
# frame, show it, and start the event loop. # frame, show it, and start the event loop.
app = wx.App() app = wx.App()
frm = HelloFrame(None, title='Hello World 2') frm = HelloFrame(None, title='Hello World 2')
frm.Show() frm.Show()
app.MainLoop() app.MainLoop()
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
%run wx_hello_world.py %run wx_hello_world.py
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Machine learning ## Machine learning
- scikit-learn - scikit-learn
- theano/tensorflow/pytorch - theano/tensorflow/pytorch
- keras - keras
## [Pycuda](https://documen.tician.de/pycuda/): Programming the GPU ## [Pycuda](https://documen.tician.de/pycuda/): Programming the GPU
Wrapper around [Cuda](https://developer.nvidia.com/cuda-zone).
The alternative [Pyopencl](https://documen.tician.de/pyopencl/) provides a very similar wrapper around [OpenCL](https://www.khronos.org/opencl/).
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` ``` python
import pycuda.autoinit import pycuda.autoinit
import pycuda.driver as drv import pycuda.driver as drv
from pycuda.compiler import SourceModule from pycuda.compiler import SourceModule
mod = SourceModule(""" mod = SourceModule("""
__global__ void multiply_them(double *dest, double *a, double *b) __global__ void multiply_them(double *dest, double *a, double *b)
{ {
const int i = threadIdx.x; const int i = threadIdx.x;
dest[i] = a[i] * b[i]; dest[i] = a[i] * b[i];
} }
""") """)
multiply_them = mod.get_function("multiply_them") multiply_them = mod.get_function("multiply_them")
a = np.random.randn(400) a = np.random.randn(400)
b = np.random.randn(400) b = np.random.randn(400)
dest = np.zeros_like(a) dest = np.zeros_like(a)
multiply_them( multiply_them(
drv.Out(dest), drv.In(a), drv.In(b), drv.Out(dest), drv.In(a), drv.In(b),
block=(400,1,1), grid=(1,1)) block=(400,1,1), grid=(1,1))
print(dest-a*b) print(dest-a*b)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Testing ## Testing
- [unittest](https://docs.python.org/3.6/library/unittest.html): python built-in testing - [unittest](https://docs.python.org/3.6/library/unittest.html): python built-in testing
> ``` > ```
> import unittest > import unittest
> >
> class TestStringMethods(unittest.TestCase): > class TestStringMethods(unittest.TestCase):
> >
> def test_upper(self): > def test_upper(self):
> self.assertEqual('foo'.upper(), 'FOO') > self.assertEqual('foo'.upper(), 'FOO')
> >
> def test_isupper(self): > def test_isupper(self):
> self.assertTrue('FOO'.isupper()) > self.assertTrue('FOO'.isupper())
> self.assertFalse('Foo'.isupper()) > self.assertFalse('Foo'.isupper())
> >
> def test_split(self): > def test_split(self):
> s = 'hello world' > s = 'hello world'
> self.assertEqual(s.split(), ['hello', 'world']) > self.assertEqual(s.split(), ['hello', 'world'])
> # check that s.split fails when the separator is not a string > # check that s.split fails when the separator is not a string
> with self.assertRaises(TypeError): > with self.assertRaises(TypeError):
> s.split(2) > s.split(2)
> >
> if __name__ == '__main__': > if __name__ == '__main__':
> unittest.main() > unittest.main()
> ``` > ```
- [doctest](https://docs.python.org/3.6/library/doctest.html): checks the example usage in the documentation - [doctest](https://docs.python.org/3.6/library/doctest.html): checks the example usage in the documentation
> ``` > ```
> def factorial(n): > def factorial(n):
> """Return the factorial of n, an exact integer >= 0. > """Return the factorial of n, an exact integer >= 0.
> >
> >>> [factorial(n) for n in range(6)] > >>> [factorial(n) for n in range(6)]
> [1, 1, 2, 6, 24, 120] > [1, 1, 2, 6, 24, 120]
> >>> factorial(30) > >>> factorial(30)
> 265252859812191058636308480000000 > 265252859812191058636308480000000
> >>> factorial(-1) > >>> factorial(-1)
> Traceback (most recent call last): > Traceback (most recent call last):
> ... > ...
> ValueError: n must be >= 0 > ValueError: n must be >= 0
> >
> Factorials of floats are OK, but the float must be an exact integer: > Factorials of floats are OK, but the float must be an exact integer:
> >>> factorial(30.1) > >>> factorial(30.1)
> Traceback (most recent call last): > Traceback (most recent call last):
> ... > ...
> ValueError: n must be exact integer > ValueError: n must be exact integer
> >>> factorial(30.0) > >>> factorial(30.0)
> 265252859812191058636308480000000 > 265252859812191058636308480000000
> >
> It must also not be ridiculously large: > It must also not be ridiculously large:
> >>> factorial(1e100) > >>> factorial(1e100)
> Traceback (most recent call last): > Traceback (most recent call last):
> ... > ...
> OverflowError: n too large > OverflowError: n too large
> """ > """
> >
> import math > import math
> if not n >= 0: > if not n >= 0:
> raise ValueError("n must be >= 0") > raise ValueError("n must be >= 0")
> if math.floor(n) != n: > if math.floor(n) != n:
> raise ValueError("n must be exact integer") > raise ValueError("n must be exact integer")
> if n+1 == n: # catch a value like 1e300 > if n+1 == n: # catch a value like 1e300
> raise OverflowError("n too large") > raise OverflowError("n too large")
> result = 1 > result = 1
> factor = 2 > factor = 2
> while factor <= n: > while factor <= n:
> result *= factor > result *= factor
> factor += 1 > factor += 1
> return result > return result
> >
> >
> if __name__ == "__main__": > if __name__ == "__main__":
> import doctest > import doctest
> doctest.testmod() > doctest.testmod()
> ``` > ```
Two external packages provide more convenient unit tests: Two external packages provide more convenient unit tests:
- [py.test](https://docs.pytest.org/en/latest/) - [py.test](https://docs.pytest.org/en/latest/)
- [nose2](http://nose2.readthedocs.io/en/latest/usage.html) - [nose2](http://nose2.readthedocs.io/en/latest/usage.html)
> ``` > ```
> # content of test_sample.py > # content of test_sample.py
> def inc(x): > def inc(x):
> return x + 1 > return x + 1
> >
> def test_answer(): > def test_answer():
> assert inc(3) == 5 > assert inc(3) == 5
> ``` > ```
- [coverage](https://coverage.readthedocs.io/en/coverage-4.5.1/): measures which part of the code is covered by the tests - [coverage](https://coverage.readthedocs.io/en/coverage-4.5.1/): measures which part of the code is covered by the tests
## Linters ## Linters
Linters check the code for any syntax errors, [style errors](https://www.python.org/dev/peps/pep-0008/), unused variables, unreachable code, etc. Linters check the code for any syntax errors, [style errors](https://www.python.org/dev/peps/pep-0008/), unused variables, unreachable code, etc.
- [pylint](https://pypi.python.org/pypi/pylint): most extensive linter - [pylint](https://pypi.python.org/pypi/pylint): most extensive linter
- [pyflake](https://pypi.python.org/pypi/pyflakes): if you think pylint is too strict - [pyflake](https://pypi.python.org/pypi/pyflakes): if you think pylint is too strict
- [pep8](https://pypi.python.org/pypi/pep8): just checks for style errors - [pep8](https://pypi.python.org/pypi/pep8): just checks for style errors
- [mypy](http://mypy-lang.org/): adding explicit typing to python - [mypy](http://mypy-lang.org/): adding explicit typing to python
## Web frameworks
- [Django2](https://www.djangoproject.com/): includes the most features, but also forces you to do things their way
- [Pyramid](https://trypyramid.com): Intermediate options
- [Flask](http://flask.pocoo.org/): Bare-bone web framework, but many extensions available
There are also many, many libraries to interact with databases, but you will have to google those yourself.
# Several honourable mentions
- [trimesh](https://github.com/mikedh/trimesh): Triangular mesh algorithms
- [Pillow](https://pillow.readthedocs.io/en/latest/): Read/write/manipulate a wide variety of images (png, jpg, tiff, etc.)
- [psychopy](http://www.psychopy.org/): equivalent of psychtoolbox (workshop coming up in April in Nottingham)
- [Buit-in libraries](https://docs.python.org/3/py-modindex.html)
- [collections](https://docs.python.org/3.6/library/collections.html): deque, OrderedDict, namedtuple, and more
- [datetime](https://docs.python.org/3/library/datetime.html): Basic date and time types
- [enum](https://docs.python.org/3/library/enum.html): Enumerators
- [fractions](https://docs.python.org/3/library/fractions.html): rational numbers
- [functools](https://docs.python.org/3/library/functools.html): caching, decorators, and support for functional programming
- [json](https://docs.python.org/3/library/json.html)/[ipaddress](https://docs.python.org/3/library/ipaddress.html)/[xml](https://docs.python.org/3/library/xml.html#module-xml): parsing/writing
- [itertools](https://docs.python.org/3/library/itertools.html): more tools to loop over sequences
- [logging](https://docs.python.org/3/library/logging.htm): log your output to stdout or a file (more flexible than print statements)
- [multiprocessing](https://docs.python.org/3/library/multiprocessing.html)
- [os](https://docs.python.org/3/library/os.html#module-os)/[sys](https://docs.python.org/3/library/sys.html): Miscellaneous operating system interfaces
- [os.path](https://docs.python.org/3/library/os.path.html)/[pathlib](https://docs.python.org/3/library/pathlib.html): utilities to deal with filesystem paths (latter provides an object-oriented interface)
- [pickle](https://docs.python.org/3/library/pickle.html): Store/load any python object
- [shutil](https://docs.python.org/3/library/shutil.html): copy/move files
- [subprocess](https://docs.python.org/3/library/subprocess.html): call shell commands
- [turtule](https://docs.python.org/3/library/turtle.html#module-turtle): teach python to your kids!
- [warnings](https://docs.python.org/3/library/warnings.html#module-warnings): tell people they are not using your code properly
......
talks/packages/packages.md
+ 69
− 2
View file @ d66a541c
...@@ -194,6 +194,36 @@ if __name__ == '__main__': ...@@ -194,6 +194,36 @@ if __name__ == '__main__':
%run test_argparse.py 3 8.5 -q %run test_argparse.py 3 8.5 -q
``` ```
Alternatives:
- [docopt](http://docopt.org/): You write a usage string, docopt will generate the parser
> ```
> # example from https://realpython.com/blog/python/comparing-python-command-line-parsing-libraries-argparse-docopt-click/
> """Greeter.
>
> Usage:
> commands.py hello
> commands.py goodbye
> commands.py -h | --help
>
> Options:
> -h --help Show this screen.
> """
> from docopt import docopt
>
> if __name__ == '__main__':
> arguments = docopt(__doc__)
> ```
- [clize](http://clize.readthedocs.io/en/stable/why.html): You write a function, clize will generate the parser
> ```
> from clize import run
>
> def echo(word):
> return word
>
> if __name__ == '__main__':
> run(echo)
> ```
### [Gooey](https://github.com/chriskiehl/Gooey): GUI from command line tool ### [Gooey](https://github.com/chriskiehl/Gooey): GUI from command line tool
``` ```
%%writefile test_gooey.py %%writefile test_gooey.py
...@@ -226,7 +256,11 @@ if __name__ == '__main__': ...@@ -226,7 +256,11 @@ if __name__ == '__main__':
!gcoord_gui !gcoord_gui
``` ```
## [Jinja2](http://jinja.pocoo.org/docs/2.10/): HTML generation ## [Jinja2](http://jinja.pocoo.org/docs/2.10/): Templating language
Jinja2 allows to create templates of files with placeholders, where future content will go.
This allows for the creation of a large number of similar files.
This can for example be used to produce static HTML output in a highly flexible manner.
``` ```
%%writefile image_list.jinja2 %%writefile image_list.jinja2
<!DOCTYPE html> <!DOCTYPE html>
...@@ -418,6 +452,8 @@ if __name__ == '__main__': ...@@ -418,6 +452,8 @@ if __name__ == '__main__':
- keras - keras
## [Pycuda](https://documen.tician.de/pycuda/): Programming the GPU ## [Pycuda](https://documen.tician.de/pycuda/): Programming the GPU
Wrapper around [Cuda](https://developer.nvidia.com/cuda-zone).
The alternative [Pyopencl](https://documen.tician.de/pyopencl/) provides a very similar wrapper around [OpenCL](https://www.khronos.org/opencl/).
``` ```
import pycuda.autoinit import pycuda.autoinit
import pycuda.driver as drv import pycuda.driver as drv
...@@ -444,6 +480,7 @@ multiply_them( ...@@ -444,6 +480,7 @@ multiply_them(
print(dest-a*b) print(dest-a*b)
``` ```
Also see [pyopenGL](http://pyopengl.sourceforge.net/): graphics programming in python (used in FSLeyes)
## Testing ## Testing
- [unittest](https://docs.python.org/3.6/library/unittest.html): python built-in testing - [unittest](https://docs.python.org/3.6/library/unittest.html): python built-in testing
> ``` > ```
...@@ -535,4 +572,34 @@ Linters check the code for any syntax errors, [style errors](https://www.python. ...@@ -535,4 +572,34 @@ Linters check the code for any syntax errors, [style errors](https://www.python.
- [pylint](https://pypi.python.org/pypi/pylint): most extensive linter - [pylint](https://pypi.python.org/pypi/pylint): most extensive linter
- [pyflake](https://pypi.python.org/pypi/pyflakes): if you think pylint is too strict - [pyflake](https://pypi.python.org/pypi/pyflakes): if you think pylint is too strict
- [pep8](https://pypi.python.org/pypi/pep8): just checks for style errors - [pep8](https://pypi.python.org/pypi/pep8): just checks for style errors
- [mypy](http://mypy-lang.org/): adding explicit typing to python - [mypy](http://mypy-lang.org/): adding explicit typing to python
\ No newline at end of file
## Web frameworks
- [Django2](https://www.djangoproject.com/): includes the most features, but also forces you to do things their way
- [Pyramid](https://trypyramid.com): Intermediate options
- [Flask](http://flask.pocoo.org/): Bare-bone web framework, but many extensions available
There are also many, many libraries to interact with databases, but you will have to google those yourself.
# Several honourable mentions
- [trimesh](https://github.com/mikedh/trimesh): Triangular mesh algorithms
- [Pillow](https://pillow.readthedocs.io/en/latest/): Read/write/manipulate a wide variety of images (png, jpg, tiff, etc.)
- [psychopy](http://www.psychopy.org/): equivalent of psychtoolbox (workshop coming up in April in Nottingham)
- [Buit-in libraries](https://docs.python.org/3/py-modindex.html)
- [collections](https://docs.python.org/3.6/library/collections.html): deque, OrderedDict, namedtuple, and more
- [datetime](https://docs.python.org/3/library/datetime.html): Basic date and time types
- [enum](https://docs.python.org/3/library/enum.html): Enumerators
- [fractions](https://docs.python.org/3/library/fractions.html): rational numbers
- [functools](https://docs.python.org/3/library/functools.html): caching, decorators, and support for functional programming
- [json](https://docs.python.org/3/library/json.html)/[ipaddress](https://docs.python.org/3/library/ipaddress.html)/[xml](https://docs.python.org/3/library/xml.html#module-xml): parsing/writing
- [itertools](https://docs.python.org/3/library/itertools.html): more tools to loop over sequences
- [logging](https://docs.python.org/3/library/logging.htm): log your output to stdout or a file (more flexible than print statements)
- [multiprocessing](https://docs.python.org/3/library/multiprocessing.html)
- [os](https://docs.python.org/3/library/os.html#module-os)/[sys](https://docs.python.org/3/library/sys.html): Miscellaneous operating system interfaces
- [os.path](https://docs.python.org/3/library/os.path.html)/[pathlib](https://docs.python.org/3/library/pathlib.html): utilities to deal with filesystem paths (latter provides an object-oriented interface)
- [pickle](https://docs.python.org/3/library/pickle.html): Store/load any python object
- [shutil](https://docs.python.org/3/library/shutil.html): copy/move files
- [subprocess](https://docs.python.org/3/library/subprocess.html): call shell commands
- [time](https://docs.python.org/3/library/time.html)/[timeit](https://docs.python.org/3/library/timeit.html): keeping track of it
- [turtule](https://docs.python.org/3/library/turtle.html#module-turtle): teach python to your kids!
- [warnings](https://docs.python.org/3/library/warnings.html#module-warnings): tell people they are not using your code properly
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment