README.md 4.66 KB
Newer Older
1
2
3
4
5
6
# FSL conda package infrastructure


_This documentation is a work in progress_


Paul McCarthy's avatar
Paul McCarthy committed
7
8
This repository hosts information and documentation regarding the FSL conda
package infrastructure.
9

10

Paul McCarthy's avatar
Paul McCarthy committed
11
12
This `README` file contains a general overview. More detailed information on
different topics can be found in the folowing files:
13

14

Paul McCarthy's avatar
Paul McCarthy committed
15
16
17
18
19
20
21
22
23
24
 - [`development_workflow.md`](development_workflow.md): Details on
   development and release workflows and processes.
 - [`official_fsl_installations.md`](official_fsl_installations.md): Details
   on official/full conda-based FSL installations.
 - [`local_development.md`](local_development.md): Setting up an FSL
   development environment on your own machine.
 - [`building_fsl_conda_packages.md`](building_fsl_conda_packages.md): Details
   on how to build a FSL conda package locally.
 - [`creating_fsl_conda_recipes.md`](creating_fsl_conda_recipes.md): Details
   on how to create a FSL conda recipe.
25
26
27
28


## Overview

Paul McCarthy's avatar
Paul McCarthy committed
29
30
31
FSL is installed using the [**conda**](https://conda.io/) package and
environment management system. Each FSL project is built as a conda package,
and published to an internally hosted conda channel.
32

33

Paul McCarthy's avatar
Paul McCarthy committed
34
35
36
37
38
Prior to FSL 6.1.0, FSL was released and installed as a monolithic archive
file, containing most FSL tools, and a separate conda environment called
`fslpython`, configured after the archive file was installed. Some of the
reasons for transitioning to a purely conda-based release model are as
follows:
39
40
41

 - Simplify FSL installation procedure
 - Reduce the size of `$FSLDIR`
Paul McCarthy's avatar
Paul McCarthy committed
42
43
44
 - Allow "minimal" FSL installations, with only the utilities that are
   required
 - Improve build reproducibility
45
 - Improve support for container-based FSL installations
Paul McCarthy's avatar
Paul McCarthy committed
46
47
 - Simplify cross-platform management of external dependencies (e.g. VTK,
   OpenBLAS)
48
49
50
51
52


## Official FSL installations


Paul McCarthy's avatar
Paul McCarthy committed
53
54
55
56
Each official FSL release is defined by a conda [`environment.yml`
specification](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually). This
specification contains a list of all of the packages, including their
versions, to be installed.
57

58

59
60
The official fsl/installer> script performs the following tasks:

61

62
1. Downloads a `miniconda` installer.
Paul McCarthy's avatar
Paul McCarthy committed
63
64
65
2. Installs a base `miniconda` environment into `$FSLDIR/../.fslconda/` (where
   `$FSLDIR` defaults to `/usr/local/fsl/`, or is explicitly specified by the
   user).
66
67
3. Creates a child conda environment at `$FSLDIR`.
4. Installs all FSL packages into the `$FSLDIR` conda environment.
Paul McCarthy's avatar
Paul McCarthy committed
68
69
5. Sets up the user shell profile so that FSL is configured in their shell
   environment by default.
70
71
72
73
74


## Partial FSL installations


Paul McCarthy's avatar
Paul McCarthy committed
75
76
77
Users may choose to install specific FSL packages by installing the desired
FSL packages directly from the FSL conda channel into their own conda
environment.
78
79


Paul McCarthy's avatar
Paul McCarthy committed
80
81
82
83
84
85
> **Note:** All FSL conda packages list the dependencies required for their
> execution, so when a particular package is installed, all of its run-time
> dependencies will also be installed. However, as we transition away from
> monolithic installations and towards conda-based installations, there are
> likely to be omissions and errors in the dependency hierarchy. These issues
> will be resolved on a case-by-case basis.
86
87


88
89
## FSL conda packages

90

Paul McCarthy's avatar
Paul McCarthy committed
91
92
93
All FSL projects and their dependencies are published as conda packages,
either in the internal FMRIB conda channel, or on https://anaconda.org, in one
of the `conda-forge` or `defaults` channels.
94

95

Paul McCarthy's avatar
Paul McCarthy committed
96
97
98
99
100
All FSL projects, and any dependencies which are not available on
https://anaconda.org, are built according to a conda *recipe*, which describes
how the project can be built as a conda package. All FSL conda recipes are
hosted as git repositories on the FMRIB gitlab server, under the `fsl/conda/`
group.
101

102

Paul McCarthy's avatar
Paul McCarthy committed
103
104
To clarify, each FSL project, and internally managed dependency, comprises
**two** git repositories<sup>*</sup>:
105

Paul McCarthy's avatar
Paul McCarthy committed
106
107
108
109
* The **project** repository contains the project source code/resources, and
  is typically hosted at `https://git.fmrib.ox.ac.uk/fsl/<project>`
* The **recipe** repository contains a conda recipe for the project, and is
  hosted at `https://git.fmrib.ox.ac.uk/fsl/conda/fsl-<project>`.
110
111


Paul McCarthy's avatar
Paul McCarthy committed
112
113
114
FSL conda packages are built automatically using Gitlab CI - the CI
infrastructure for automatically building conda packages is hosted in the
fsl/fsl-ci-rules> repository.
115
116


Paul McCarthy's avatar
Paul McCarthy committed
117
118
119
120
121
122
123
> <sup>*</sup>A small number of FSL projects have more than one recipe
> repository associated with them - for example, the
> [`fsl/fdt`](https://git.fmrib.ox.ac.uk/fsl/fdt) project has two recipes -
> the [`fsl/conda/fsl-fdt`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-fdt)
> recipe provides CPU executables, and the
> [`fsl/conda/fsl-fdt-gpu`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-fdt-gpu)
> recipe provides GPU/CUDA executables.