Commit 778c9151 authored by Paul McCarthy's avatar Paul McCarthy 🚵
Browse files

RF: line breaks

parent 1b4259f7
......@@ -4,74 +4,120 @@
_This documentation is a work in progress_
This repository hosts information and documentation regarding the FSL conda package infrastructure.
This repository hosts information and documentation regarding the FSL conda
package infrastructure.
This `README` file contains a general overview. More detailed information on different topics can be found in the folowing files:
This `README` file contains a general overview. More detailed information on
different topics can be found in the folowing files:
- [`development_workflow.md`](development_workflow.md): Details on development and release workflows and processes.
- [`official_fsl_installations.md`](official_fsl_installations.md): Details on official/full conda-based FSL installations.
- [`local_development.md`](local_development.md): Setting up an FSL development environment on your own machine.
- [`building_fsl_conda_packages.md`](building_fsl_conda_packages.md): Details on how to build a FSL conda package locally.
- [`creating_fsl_conda_recipes.md`](creating_fsl_conda_recipes.md): Details on how to create a FSL conda recipe.
- [`development_workflow.md`](development_workflow.md): Details on
development and release workflows and processes.
- [`official_fsl_installations.md`](official_fsl_installations.md): Details
on official/full conda-based FSL installations.
- [`local_development.md`](local_development.md): Setting up an FSL
development environment on your own machine.
- [`building_fsl_conda_packages.md`](building_fsl_conda_packages.md): Details
on how to build a FSL conda package locally.
- [`creating_fsl_conda_recipes.md`](creating_fsl_conda_recipes.md): Details
on how to create a FSL conda recipe.
## Overview
FSL is installed using the [**conda**](https://conda.io/) package and environment management system. Each FSL project is built as a conda package, and published to an internally hosted conda channel.
FSL is installed using the [**conda**](https://conda.io/) package and
environment management system. Each FSL project is built as a conda package,
and published to an internally hosted conda channel.
Prior to FSL 6.1.0, FSL was released and installed as a monolithic archive file, containing most FSL tools, and a separate conda environment called `fslpython`, configured after the archive file was installed. Some of the reasons for transitioning to a purely conda-based release model are as follows:
Prior to FSL 6.1.0, FSL was released and installed as a monolithic archive
file, containing most FSL tools, and a separate conda environment called
`fslpython`, configured after the archive file was installed. Some of the
reasons for transitioning to a purely conda-based release model are as
follows:
- Simplify FSL installation procedure
- Reduce the size of `$FSLDIR`
- Allow "minimal" FSL installations, with only the utilities that are required
- Allow "minimal" FSL installations, with only the utilities that are
required
- Improve build reproducibility
- Improve support for container-based FSL installations
- Simplify cross-platform management of external dependencies (e.g. VTK, OpenBLAS)
- Simplify cross-platform management of external dependencies (e.g. VTK,
OpenBLAS)
## Official FSL installations
Each official FSL release is defined by a conda [`environment.yml` specification](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually). This specification contains a list of all of the packages, including their versions, to be installed.
Each official FSL release is defined by a conda [`environment.yml`
specification](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually). This
specification contains a list of all of the packages, including their
versions, to be installed.
The official fsl/installer> script performs the following tasks:
1. Downloads a `miniconda` installer.
2. Installs a base `miniconda` environment into `$FSLDIR/../.fslconda/` (where `$FSLDIR` defaults to `/usr/local/fsl/`, or is explicitly specified by the user).
2. Installs a base `miniconda` environment into `$FSLDIR/../.fslconda/` (where
`$FSLDIR` defaults to `/usr/local/fsl/`, or is explicitly specified by the
user).
3. Creates a child conda environment at `$FSLDIR`.
4. Installs all FSL packages into the `$FSLDIR` conda environment.
5. Sets up the user shell profile so that FSL is configured in their shell environment by default.
5. Sets up the user shell profile so that FSL is configured in their shell
environment by default.
## Partial FSL installations
Users may choose to install specific FSL packages by installing the desired FSL packages directly from the FSL conda channel into their own conda environment.
Users may choose to install specific FSL packages by installing the desired
FSL packages directly from the FSL conda channel into their own conda
environment.
> **Note:** All FSL conda packages list the dependencies required for their execution, so when a particular package is installed, all of its run-time dependencies will also be installed. However, as we transition away from monolithic installations and towards conda-based installations, there are likely to be omissions and errors in the dependency hierarchy. These issues will be resolved on a case-by-case basis.
> **Note:** All FSL conda packages list the dependencies required for their
> execution, so when a particular package is installed, all of its run-time
> dependencies will also be installed. However, as we transition away from
> monolithic installations and towards conda-based installations, there are
> likely to be omissions and errors in the dependency hierarchy. These issues
> will be resolved on a case-by-case basis.
## FSL conda packages
All FSL projects and their dependencies are published as conda packages, either in the internal FMRIB conda channel, or on https://anaconda.org, in one of the `conda-forge` or `defaults` channels.
All FSL projects and their dependencies are published as conda packages,
either in the internal FMRIB conda channel, or on https://anaconda.org, in one
of the `conda-forge` or `defaults` channels.
All FSL projects, and any dependencies which are not available on https://anaconda.org, are built according to a conda *recipe*, which describes how the project can be built as a conda package. All FSL conda recipes are hosted as git repositories on the FMRIB gitlab server, under the `fsl/conda/` group.
All FSL projects, and any dependencies which are not available on
https://anaconda.org, are built according to a conda *recipe*, which describes
how the project can be built as a conda package. All FSL conda recipes are
hosted as git repositories on the FMRIB gitlab server, under the `fsl/conda/`
group.
To clarify, each FSL project, and internally managed dependency, comprises **two** git repositories<sup>*</sup>:
To clarify, each FSL project, and internally managed dependency, comprises
**two** git repositories<sup>*</sup>:
* The project repository contains the project source code/resources, and is typically hosted at `https://git.fmrib.ox.ac.uk/fsl/<project>`
* The recipe repository contains a conda recipe for the project, and is hosted at `https://git.fmrib.ox.ac.uk/fsl/conda/fsl-<project>`.
* The **project** repository contains the project source code/resources, and
is typically hosted at `https://git.fmrib.ox.ac.uk/fsl/<project>`
* The **recipe** repository contains a conda recipe for the project, and is
hosted at `https://git.fmrib.ox.ac.uk/fsl/conda/fsl-<project>`.
FSL conda packages are built automatically using Gitlab CI - the CI infrastructure for automatically building conda packages is hosted in the fsl/fsl-ci-rules> repository.
FSL conda packages are built automatically using Gitlab CI - the CI
infrastructure for automatically building conda packages is hosted in the
fsl/fsl-ci-rules> repository.
> <sup>*</sup>A small number of FSL projects have more than one recipe repository associated with them - for example, the [`fsl/fdt`](https://git.fmrib.ox.ac.uk/fsl/fdt) project has two recipes - the [`fsl/conda/fsl-fdt`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-fdt) recipe provides CPU executables, and the [`fsl/conda/fsl-fdt-gpu`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-fdt-gpu) recipe provides GPU/CUDA executables.
> <sup>*</sup>A small number of FSL projects have more than one recipe
> repository associated with them - for example, the
> [`fsl/fdt`](https://git.fmrib.ox.ac.uk/fsl/fdt) project has two recipes -
> the [`fsl/conda/fsl-fdt`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-fdt)
> recipe provides CPU executables, and the
> [`fsl/conda/fsl-fdt-gpu`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-fdt-gpu)
> recipe provides GPU/CUDA executables.
# Building FSL conda packages locally
This document describes how to build a conda package for an FSL project from the corresponding conda recipe.
This document describes how to build a conda package for an FSL project from
the corresponding conda recipe.
It does not describe how to *create* a conda recipe for a FSL project - that is described in [`fsl_conda_recipes.md`](fsl_conda_recipes.md).
It does not describe how to *create* a conda recipe for a FSL project - that
is described in [`fsl_conda_recipes.md`](fsl_conda_recipes.md).
Normally there should be no need to build FSL conda packages by hand, as stable and development packages are automatically built using Gitlab CI. However the need may arise for development, testing, or debugging purposes, and hence the process is described here.
Normally there should be no need to build FSL conda packages by hand, as
stable and development packages are automatically built using Gitlab
CI. However the need may arise for development, testing, or debugging
purposes, and hence the process is described here.
In order to build a FSL conda package locally, all you need is a (mini)conda environment with `conda-build` installed. You do not need to have a compiler, or even FSL, installed.
In order to build a FSL conda package locally, all you need is a (mini)conda
environment with `conda-build` installed. You do not need to have a compiler,
or even FSL, installed.
> **Note:** If you are building a CUDA package, you need to have `nvcc` installed - this cannot be installed through `conda`, so must be installed externally.
> **Note:** If you are building a CUDA package, you need to have `nvcc`
> installed - this cannot be installed through `conda`, so must be installed
> externally.
## Step 1: Clone the recipe repository
To start, you need to create a local clone of the recipe repository. For example, if you would like to build a package for `fsl/avwutils`:
To start, you need to create a local clone of the recipe repository. For
example, if you would like to build a package for `fsl/avwutils`:
```bash
......@@ -30,7 +40,11 @@ git clone https://git.fmrib.ox.ac.uk/fsl/conda/fsl-avwuitls
## Step 2: Clone the project repository (optional)
By default, the `fsl-avwutils` recipe will build a package from the `fsl/avwutils` gitlab repository. If you would like to build a package from a *local* clone, or a personal fork, of `fsl/avwutils`, you can use the `FSLCONDA_REPOSITORY` variable to override the default setting (which is to build from the gitlab `fsl/avwutils` project):
By default, the `fsl-avwutils` recipe will build a package from the
`fsl/avwutils` gitlab repository. If you would like to build a package from a
*local* clone, or a personal fork, of `fsl/avwutils`, you can use the
`FSLCONDA_REPOSITORY` variable to override the default setting (which is to
build from the gitlab `fsl/avwutils` project):
```bash
......@@ -39,23 +53,33 @@ export FSLCONDA_REPOSITORY=$(pwd)/avwutils
```
> **Note:** It is not currently possible to build a conda package from a local copy of a project which is *not* a git repository, nor from a dirty working tree (i.e. the changes you want to build must be committed). This functionality may be added in the future if it is deemed necessary.
> **Note:** It is not currently possible to build a conda package from a local
> copy of a project which is *not* a git repository, nor from a dirty working
> tree (i.e. the changes you want to build must be committed). This
> functionality may be added in the future if it is deemed necessary.
## Step 3: Choose the revision you want to build (optional)
By default, the `fsl-avwutils` conda recipe will build a package from the last released version (tag) on the `fsl/avwutils` gitlab repository - this is specified in in the recipe `meta.yaml`. If you would like to build a package for a different release, or for a specific branch, you can set the `FSLCONDA_REVISION` variable to override the default setting. For example, if you want to build a package for the `master` branch:
By default, the `fsl-avwutils` conda recipe will build a package from the last
released version (tag) on the `fsl/avwutils` gitlab repository - this is
specified in in the recipe `meta.yaml`. If you would like to build a package
for a different release, or for a specific branch, you can set the
`FSLCONDA_REVISION` variable to override the default setting. For example, if
you want to build a package for the `master` branch:
```bash
export FSLCONDA_REVISION=master
```
## Step 4: Run `conda build` to build the package
Now you can run `conda build` to build the conda package. The channel order is very important, and must be:
Now you can run `conda build` to build the conda package. The channel order is
very important, and must be:
1. `conda-forge` Packages from conda-forge have highest priority
2. `defaults` Packages from anaconda have higher priority than the FSL channel
......@@ -75,19 +99,32 @@ conda build \
.
```
If the build succeeds, the built package will be saved to the `dist` directory - binary packages will be saved in either the `osx-64` or `linux-64` sub-directories, and platform-independent (e.g. python) packages will be saved in the `noarch` sub-directory.
If the build succeeds, the built package will be saved to the `dist`
directory - binary packages will be saved in either the `osx-64` or `linux-64`
sub-directories, and platform-independent (e.g. python) packages will be saved
in the `noarch` sub-directory.
> **Note:** IF you are building a CUDA package, you must define an environment variable called `CUDA_VER`, which contains the `major.minor` CUDA version that you wish to build the package against, before calling `conda build`. For example, `export CUDA_VER="9.2"`.
> **Note:** IF you are building a CUDA package, you must define an environment
> variable called `CUDA_VER`, which contains the `major.minor` CUDA version
> that you wish to build the package against, before calling `conda
> build`. For example, `export CUDA_VER="9.2"`.
## Building multiple FSL conda packages locally
The need may arise to build conda packages for multiple FSL projects at once. For example, you might be making changes to one project which depends on changes you have made to another project (a dependency of the first project). In this case it makes sense to set up a local conda channel, and to build/install the dependencies into that channel.
The need may arise to build conda packages for multiple FSL projects at
once. For example, you might be making changes to one project which depends on
changes you have made to another project (a dependency of the first
project). In this case it makes sense to set up a local conda channel, and to
build/install the dependencies into that channel.
Let's say we are working on the `randomise` project, and are simultaneously making changes to the `newimage` project. We will need local clones of all the project and recipe repositories:
Let's say we are working on the `randomise` project, and are simultaneously
making changes to the `newimage` project. We will need local clones of all the
project and recipe repositories:
```bash
......@@ -98,7 +135,11 @@ git clone https://git.fmrib.ox.ac.uk/fsl/conda/fsl-newimage.git
```
Before we can build a conda package for `randomise`, we need to build a conda package from our local development version of `newimage`. We use `FSLCONDA_REPOSITORY` and `FSLCONDA_REVISION` to direct the build to use our local `newimage` repository, and direct the build to `my_local_conda_channel`, which we will use later.
Before we can build a conda package for `randomise`, we need to build a conda
package from our local development version of `newimage`. We use
`FSLCONDA_REPOSITORY` and `FSLCONDA_REVISION` to direct the build to use our
local `newimage` repository, and direct the build to `my_local_conda_channel`,
which we will use later.
```bash
......@@ -116,7 +157,10 @@ conda build \
```
Now we can build our development version of `randomise`, using the development conda package we just built for `newimage`, simply by adding our local channel directory as a conda channel, making sure to list it *before* the FMRIB conda channel URL, so that it takes precedence:
Now we can build our development version of `randomise`, using the development
conda package we just built for `newimage`, simply by adding our local channel
directory as a conda channel, making sure to list it *before* the FMRIB conda
channel URL, so that it takes precedence:
```bash
......
# FSL development and release workflows
This document describes development and release workflows for individual FSL projects - it does not describe the release process for full/official FSL installations.
This document describes development and release workflows for individual FSL
projects - it does not describe the release process for full/official FSL
installations.
All FSL projects are released and published as conda packages, either in the internal FMRIB conda channel, or on https://anaconda.org, via [`conda-forge`](https://conda-forge.org/).
All FSL projects are released and published as conda packages, either in the
internal FMRIB conda channel, or on https://anaconda.org, via
[`conda-forge`](https://conda-forge.org/).
## Project and recipe repositories
Every FSL project comprises two git repositories:
Every FSL project comprises two git repositories<sup>1</sup>:
- The project repository contains the project source code/resources, and is typically hosted at `https://git.fmrib.ox.ac.uk/fsl/<project>`<sup>1</sup>.
- The **project** repository contains the project source code/resources, and is
typically hosted at `https://git.fmrib.ox.ac.uk/fsl/<project>`<sup>2</sup>.
- The recipe repository contains a conda recipe for the project, and is hosted at `https://git.fmrib.ox.ac.uk/fsl/conda/fsl-<project>`<sup>2</sup>.
- The **recipe** repository contains a conda recipe for the project, and is
hosted at `https://git.fmrib.ox.ac.uk/fsl/conda/fsl-<project>`<sup>3</sup>.
Activity on the project repository proceeds independently of activity on the recipe repository. The recipe repository is only updated when the project developer wishes to release a new version.
Activity on the project repository proceeds independently of activity on the
recipe repository. The recipe repository is only updated when the project
developer wishes to release a new version.
> <sup>1</sup>Most, but not all, FSL projects are hosted in the FMRIB Gitlab [`fsl/` group](https://git.fmrib.ox.ac.uk/fsl). Some projects are hosted under the username of the developer (e.g. [`matteob/eddy_qc`](https://git.fmrib.ox.ac.uk/matteob/eddy_qc)). Some FSL projects and dependencies are hosted externally, such as [MSM](https://github.com/ecr05/MSM_HOCR) and [oxford_asl](https://github.com/ibme-qubic/oxford_asl/). The conda recipes for these projects are, however, hosted internally in the `fsl/conda/` Gitlab namespace.
> <sup>1</sup>A small number of FSL projects have more than one recipe
> repository associated with them - for example, the
> [`fsl/fdt`](https://git.fmrib.ox.ac.uk/fsl/fdt) project has two recipes -
> the [`fsl/conda/fsl-fdt`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-fdt)
> recipe provides CPU executables, and the
> [`fsl/conda/fsl-fdt-gpu`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-fdt-gpu)
> recipe provides GPU/CUDA executables.
> <sup>2</sup>**All** FSL conda recipes are hosted in the [`fsl/conda/`](https://git.fmrib.ox.ac.uk/fsl/conda) Gitlab namespace, with the sole exception of FSL projects which are published to `conda-forge` (e.g. [`fsleyes`](https://github.com/conda-forge/fsleyes-feedstock)).
> <sup>2</sup>Most, but not all, FSL projects are hosted in the FMRIB GitLab
> [`fsl/` group](https://git.fmrib.ox.ac.uk/fsl). Some projects are hosted
> under the username of the developer
> (e.g. [`matteob/eddy_qc`](https://git.fmrib.ox.ac.uk/matteob/eddy_qc)). Some
> FSL projects and dependencies are hosted externally, such as
> [MSM](https://github.com/ecr05/MSM_HOCR) and
> [oxford_asl](https://github.com/ibme-qubic/oxford_asl/). The conda recipes
> for these projects are, however, hosted internally in the `fsl/conda/`
> GitLab group.
## FSL project versioning scheme
> <sup>3</sup>**All** FSL conda recipes are hosted in the
> [`fsl/conda/`](https://git.fmrib.ox.ac.uk/fsl/conda) GitLab group, with
> the sole exception of FSL projects which are published to `conda-forge`
> (e.g. [`fsleyes`](https://github.com/conda-forge/fsleyes-feedstock)).
FSL projects (and hence their conda packages) must follow a versioning scheme whereby each version comprises sequentially increasing numbers followed by periods. There are no more requirements beyond this, but the use of [semantic versioning](https://semver.org) is encouraged.
## FSL project versioning scheme
If you are developing a project with a programming or command-line interface, a responsible strategy for managing releases is to use the version number to denote changes to the interface (for example, by using semantic versioning), and to use deprecation warnings to warn users of impending changes to the interface.
FSL projects (and hence their conda packages) must follow a versioning scheme
whereby each version comprises sequentially increasing numbers followed by
periods. There are no more requirements beyond this, but the use of [semantic
versioning](https://semver.org) is encouraged.
Having said this, many core FSL projects (e.g. `miscmaths`, `newimage`, etc) change very infrequently, so the decision was made to *not* follow semantic versioning principles, but instead to use a versioning scheme whereby each new release is given a version number of the form:
If you are developing a project with a programming or command-line interface,
a responsible strategy for managing releases is to use the version number to
denote changes to the interface (for example, by using semantic versioning),
and to use deprecation warnings to warn users of impending changes to the
interface.
Having said this, many core FSL projects (e.g. `miscmaths`, `newimage`, etc)
change very infrequently, so the decision was made to *not* follow semantic
versioning principles, but instead to use a versioning scheme whereby each new
release is given a version number of the form:
YYMM.X
where `YYMM` is the year and month of release (e.g. `2101` for January 2021), and `X` is an incremental release number, starting at `0`, which is used to avoid conflict in the event of multiple releases within the same month. For example, two versions of a core FSL project released in the month of January 2021 would be assigned versions `2021.0` and `2021.1`.
where `YYMM` is the year and month of release (e.g. `2101` for January 2021),
and `X` is an incremental release number, starting at `0`, which is used to
avoid conflict in the event of multiple releases within the same month. For
example, two versions of a core FSL project released in the month of January
2021 would be assigned versions `2021.0` and `2021.1`.
## FSL conda package naming conventions
FSL conda package names must follow the [conda package naming conventions](https://conda.io/projects/conda-build/en/latest/concepts/package-naming-conv.html#index-0), and be comprised solely of _lowercase alpha characters, numeric digits, underscores, hyphens, or dots_.
FSL conda package names must follow the [conda package naming
conventions](https://conda.io/projects/conda-build/en/latest/concepts/package-naming-conv.html#index-0),
and be comprised solely of _lowercase alpha characters, numeric digits,
underscores, hyphens, or dots_.
Furthermore, all FSL conda packages are prefixed with `fsl-`. An FSL project with name `<project>` will have a corresponding conda package name of `fsl-<project>`. For FSL projects with a name that begins with `fsl` (e.g. `fslvbm`, `fsl_deface`), the leading `fsl` will be dropped in the construction of the corresponding conda-package name. For example:
Furthermore, all FSL conda packages are prefixed with `fsl-`. An FSL project
with name `<project>` will have a corresponding conda package name of
`fsl-<project>`. For FSL projects with a name that begins with `fsl`
(e.g. `fslvbm`, `fsl_deface`), the leading `fsl` will be dropped in the
construction of the corresponding conda-package name. For example:
| **FSL project name** | **Conda package name** |
| -------------------- | ---------------------- |
......@@ -55,33 +106,49 @@ Furthermore, all FSL conda packages are prefixed with `fsl-`. An FSL project wit
| `fsl_deface` | `fsl-deface` |
| `fsl-mrs` | `fsl-mrs` |
| `NewNifti` | `fsl-newnifti` |
## Contributing to a FSL project
No restrictions are placed on the ways in which development of a particular FSL project takes place. Developers are free to manage their project in any way they wish.
No restrictions are placed on the ways in which development of a particular
FSL project takes place. Developers are free to manage their project in any
way they wish.
However, contribution to core FSL projects (`avwutils`, `newimage`, etc) follow a merge request and review process, and it is recommended that developers follow a similar process for their own projects.
However, contribution to core FSL projects (`avwutils`, `newimage`, etc)
follow a merge request and review process, and it is recommended that
developers follow a similar process for their own projects.
- The `master` branch of the project repository should be considered stable and ready to release at any time.
- All development should occur on a separate branch within the project repository, or personal fork of the project repository.
- All changes to the `master` branch should take place through Gitlab [merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/) (MRs).
- When a MR is opened, it is reviewed, approved, and merged into the `master` branch by the project maintainer, possibly after any requested changes to the MR have been made.
- The `master` branch of the project repository should be considered stable
and ready to release at any time.
- All development should occur on a separate branch within the project
repository, or personal fork of the project repository.
- All changes to the `master` branch should take place through Gitlab [merge
requests](https://docs.gitlab.com/ee/user/project/merge_requests/) (MRs).
- When a MR is opened, it is reviewed, approved, and merged into the `master`
branch by the project maintainer, possibly after any requested changes to
the MR have been made.
Whenever new commits are added to the `master` of a FSL project repository, a conda package is built from the state of the `master` branch, and released to an internal development channel.
Whenever new commits are added to the `master` of a FSL project repository, a
conda package is built from the state of the `master` branch, and released to
an internal development channel.
## Releasing a new version of a FSL project
## Releasing a new version of a FSL project
A new versions of a FSL project is denoted by a new tag being added to the project repository. Tag names must follow the **versioning scheme** outlined above.
A new versions of a FSL project is denoted by a new tag being added to the
project repository. Tag names must follow the **versioning scheme** outlined
above.
When a new tag is added to a FSL project repository, the corresponding recipe repository is updated, and a new conda package built and released.
When a new tag is added to a FSL project repository, the corresponding recipe
repository is updated, and a new conda package built and released.
For projects which are included in FSL, but developed externally (e.g. [MSM](https://github.com/ecr05/MSM_HOCR)), the recipe repository is manually updated on an as-needed basis.
For projects which are included in FSL, but developed externally
(e.g. [MSM](https://github.com/ecr05/MSM_HOCR)), the recipe repository is
manually updated on an as-needed basis.
# Setting up a local FSL development environment
*This document is written under the assumption that you have FSL 6.1.0 or newer, installed via the fslinstaller script. These instructions will not work for manually managed FSL installations.*
*This document is written under the assumption that you have FSL 6.1.0 or
newer, installed via the fslinstaller script. These instructions will not work
for manually managed FSL installations.*
This document describes how to set up a local development environment for compiling FSL C/C++ projects. It does **not** describe how to build conda packages for FSL projects - this is described in [`building_fsl_conda_packages.md`](building_fsl_conda_packages.md).
This document describes how to set up a local development environment for
compiling FSL C/C++ projects. It does **not** describe how to build conda
packages for FSL projects - this is described in
[`building_fsl_conda_packages.md`](building_fsl_conda_packages.md).
A standard FSL installation comes with the `Makefile` machinery required to compile FSL C/C++ projects. All that you need to add is a C/C++ compiler. You can use your operating system compiler if you wish, but be warned that doing so will potentially compile your code in a different way to how the official FSL conda packages have been compiled.
A standard FSL installation comes with the `Makefile` machinery required to
compile FSL C/C++ projects. All that you need to add is a C/C++ compiler. You
can use your operating system compiler if you wish, but be warned that doing
so will potentially compile your code in a different way to how the official
FSL conda packages have been compiled.
A better option is to use the [conda-forge](https://conda-forge.org/) compiler toolchain. You can install the conda-forge compilers directly into `$FSLDIR` like so:
A better option is to use the [conda-forge](https://conda-forge.org/) compiler
toolchain. You can install the conda-forge compilers directly into `$FSLDIR`
like so:
```bash
......@@ -21,18 +32,19 @@ A better option is to use the [conda-forge](https://conda-forge.org/) compiler t
source $FSLDIR/etc/fslconf/fsl-devel.sh
# Install the conda-forge C/C++ compiler toolchain
conda install -c conda-forge c-compiler cxx-compiler
conda install -c conda-forge c-compiler cxx-compiler
```
Now, when you want to work on a FSL project, you can follow these steps:
```bash
# Configure your environment for FSL development
source $FSLDIR/etc/fslconf/fsl-devel.sh
# Get a copy of the source
# Get a copy of the source
git clone https://git.fmrib.ox.ac.uk/fsl/randomise.git
cd randomise
cd randomise
# Or work with the version you have installed
cd $FSLDIR/src/randomise/
......
# Official / full FSL installations
An official FSL installation is a self-contained directory installed on an end user's system, containing the full collection of FSL commands. An official FSL installation is typically located at `/usr/local/fsl/` - the FSL installation directory is identified by the `$FSLDIR` environment variable. A [**miniconda**](https://docs.conda.io/en/latest/miniconda.html#) environment is used to create `$FSLDIR`, and install all FSL projects and their dependencies into `$FSLDIR`, which is itself a conda environment<sup>*</sup>.
An official FSL installation is a self-contained directory installed on an end
user's system, containing the full collection of FSL commands. An official FSL
installation is typically located at `/usr/local/fsl/` - the FSL installation
directory is identified by the `$FSLDIR` environment variable. A
[**miniconda**](https://docs.conda.io/en/latest/miniconda.html#) environment
is used to create `$FSLDIR`, and install all FSL projects and their
dependencies into `$FSLDIR`, which is itself a conda environment<sup>*</sup>.
> <sup>*</sup> The [conda documentation](https://docs.conda.io/projects/conda/en/latest/user-guide/getting-started.html#managing-envs) recommends against installing packages into a base conda environment, therefore `$FSLDIR` is created as a child environment.
> <sup>*</sup> The [conda
> documentation](https://docs.conda.io/projects/conda/en/latest/user-guide/getting-started.html#managing-envs)
> recommends against installing packages into a base conda environment,
> therefore `$FSLDIR` is created as a child environment.
An official (`fslinstaller`-installed) FSL installation can be distinguished from a manually installed/managed FSL installation by the presence of a file called `$FSLDIR/share/fsl/base_environment`. This file is created by the `fslinstaller` script, and is a plain-text file which contains the location of the base miniconda environment.
An official (`fslinstaller`-installed) FSL installation can be distinguished
from a manually installed/managed FSL installation by the presence of a file
called `$FSLDIR/share/fsl/base_environment`. This file is created by the
`fslinstaller` script, and is a plain-text file which contains the location of
the base miniconda environment.
By default, the base miniconda environment is installed into `$FSLDIR/../.fslconda/` (i.e. `/usr/local/.fslconda/`, if the default setting of `FSLDIR=/usr/local/fsl/` is used), but this can be overridden by the user<sup>*</sup>.
By default, the base miniconda environment is installed into
`$FSLDIR/../.fslconda/` (i.e. `/usr/local/.fslconda/`, if the default setting
of `FSLDIR=/usr/local/fsl/` is used), but this can be overridden by the
user<sup>*</sup>.
> <sup>*</sup> The only possible options for the base/child environment locations are either to have them in separate directories, or to have the child environment directory located within the base environment directory. The former option was chosen, as the latter option would require a system of links to be created from `$FSLDIR/*` into `$FSLDIR/<child-environment>/*`.
> <sup>*</sup> The only possible options for the base/child environment
> locations are either to have them in separate directories, or to have the
> child environment directory located within the base environment
> directory. The former option was chosen, as the latter option would require
> a system of links to be created from `$FSLDIR/*` into
> `$FSLDIR/<child-environment>/*`.
Because `$FSLDIR` is a conda environment into which all FSL projects and their dependencies are installed, `$FSLDIR/bin/` will contain executable files provided by both FSL, and by its dependencies. This can give rise to a problem whereby executables in `$FSLDIR/bin/` can mask other executables of the same name which are also installed in the user's environment, but come after `$FSLDIR/bin/` on the user's `$PATH` (e.g. `python`, `conda`, etc).
Because `$FSLDIR` is a conda environment into which all FSL projects and their
dependencies are installed, `$FSLDIR/bin/` will contain executable files
provided by both FSL, and by its dependencies. This can give rise to a problem
whereby executables in `$FSLDIR/bin/` can mask other executables of the same
name which are also installed in the user's environment, but come after
`$FSLDIR/bin/` on the user's `$PATH` (e.g. `python`, `conda`, etc).
To work around this problem, and to isolate all of the FSL executables from the other executables installed into the same environment, the conda packages for all FSL projects use [`post-link.sh` and `pre-unlink.sh`](https://docs.conda.io/projects/conda-build/en/latest/resources/link-scripts.html) scripts to create _wrapper scripts_ within `$FSLDIR/share/fsl/bin/`<sup>*</sup>. These wrapper scripts are created by the [`$FSLDIR/share/fsl/sbin/createFSLWrapper`](https://git.fmrib.ox.ac.uk/fsl/base/-/blob/master/share/fsl/sbin/createFSLWrapper) script, which is installed as part of the [`fsl-base`](https://git.fmrib.ox.ac.uk/fsl/base/) package.
To work around this problem, and to isolate all of the FSL executables from
the other executables installed into the same environment, the conda packages
for all FSL projects use [`post-link.sh` and
`pre-unlink.sh`](https://docs.conda.io/projects/conda-build/en/latest/resources/link-scripts.html)
scripts to create _wrapper scripts_ within
`$FSLDIR/share/fsl/bin/`<sup>*</sup>. These wrapper scripts are created by the
[`$FSLDIR/share/fsl/sbin/createFSLWrapper`](https://git.fmrib.ox.ac.uk/fsl/base/-/blob/master/share/fsl/sbin/createFSLWrapper)
script, which is installed as part of the
[`fsl-base`](https://git.fmrib.ox.ac.uk/fsl/base/) package.
> <sup>*</sup> More details on FSL project conda recipes are outlined in [`creating_fsl_conda_recipes.md`](creating_fsl_conda_recipes.md).
> <sup>*</sup> More details on FSL project conda recipes are outlined in
> [`creating_fsl_conda_recipes.md`](creating_fsl_conda_recipes.md).
Users can then add the `$FSLDIR/share/fsl/bin/` directory to their `$PATH`, with the effect that only FSL executables will be made available in their environment. This mechanism also has the advantage that all existing FSL and third-party scripts which refer to `$FSLDIR/bin/<command>` will continue to work without modification.
Users can then add the `$FSLDIR/share/fsl/bin/` directory to their `$PATH`,
with the effect that only FSL executables will be made available in their
environment. This mechanism also has the advantage that all existing FSL and
third-party scripts which refer to `$FSLDIR/bin/<command>` will continue to
work without modification.
Once an official FSL release has installed onto a user's system, they can configure their environment to use FSL as follows (for a `bash`-compatible shell):
Once an official FSL release has installed onto a user's system, they can
configure their environment to use FSL as follows (for a `bash`-compatible
shell):
```
export FSLDIR=/usr/local/fsl/
source $FSLDIR/etc/fslconf/fsl.sh
```
For a typical installation, this environment configuration is automatically added to the user's iniitalisation script by the `fslinstaller`.
For a typical installation, this environment configuration is automatically
added to the user's iniitalisation script by the `fslinstaller`.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment