# FSL conda package infrastructure _This documentation is a work in progress_ This repository hosts information and documentation regarding the FSL conda package infrastructure. This `README` file contains a general overview. More detailed information on different topics can be found in the folowing files: - [`development_workflow.md`](development_workflow.md): Details on development and release workflows and processes. - [`official_fsl_installations.md`](official_fsl_installations.md): Details on official/full conda-based FSL installations. - [`local_development.md`](local_development.md): Setting up an FSL development environment on your own machine. - [`building_fsl_conda_packages.md`](building_fsl_conda_packages.md): Details on how to build a FSL conda package locally. - [`creating_fsl_conda_recipes.md`](creating_fsl_conda_recipes.md): Details on how to create a FSL conda recipe. ## Quick summary FSL is organised into a number of repositories hosted at https://git.fmrib.ox.ac.uk. - fsl/conda/docs> (this repository): High level documentation - fsl/conda/manifest>: Metadata for managing public and internal FSL releases - fsl/conda/manifest-rules>: CI configuration used by the fsl/conda/manifest> repository - fsl/conda/installer>: Home of the `fslinstaller.py` script, for installing FSL - fsl/conda/fsl-ci-rules>: CI rules for automated building and publishing of FSL conda packages - `fsl/`: Source code repository for FSL project `` (e.g. fsl/avwutils>) - `fsl/conda/fsl-`: Conda recipe repository for FSL project `` (e.g. fsl/conda/fsl-avwutils>) ## Overview FSL is installed using the [**conda**](https://conda.io/) package and environment management system. Each FSL project is built as a conda package, and published to an internally hosted conda channel. Prior to FSL 6.0.6, FSL was released and installed as a monolithic archive file, containing most FSL tools, and a separate conda environment called `fslpython`, configured after the archive file was installed. Some of the reasons for transitioning to a purely conda-based release model are as follows: - Simplify FSL installation procedure - Reduce the size of `$FSLDIR` - Allow "minimal" FSL installations, with only the utilities that are required - Improve build reproducibility - Improve support for container-based FSL installations - Simplify cross-platform management of external dependencies (e.g. VTK, OpenBLAS) ## FSL releases Public and internal FSL release are defined by a set of conda [`environment.yml` files](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually), one for each supported platform and CUDA version. Each `environment.yml` file contains a list of the packages which are included in the FSL release. FSL releases are described by a `manifest.json` file, which is an index of all available `environment.yml` files, and which contains all of the information needed to install a particular FSL release. The fsl/conda/manifest> repository is where new FSL releases, both public and internal, are created. The `environment.yml` and `manifest.json` files for a new release are automatically generated from information stored in the `fsl-release.yml` file in that repository. The `manifest.json` and `environment.yml` files for public and internal release are currently published to https://fsl.fmrib.ox.ac.uk/fsldownloads/fslconda/releases/. ## The `fslinstaller.py` script FSL releases are installed via a script called `fslinstaller.py`, which is maintained at the fsl/conda/installer> repository. The `fslinstaller.py` script performs the following tasks: 1. Downloads the FSL release `manifest.json` file, and the appropriate `environment.yml` file for the platform. 2. Downloads a `miniconda` installer. 3. Installs a base `miniconda` environment into `$FSLDIR` (where `$FSLDIR` defaults to `/usr/local/fsl/`, or is explicitly specified by the user). 4. Installs all FSL packages into the `$FSLDIR` base conda environment. 5. Sets up the user shell profile so that FSL is configured in their shell environment by default. ## Partial FSL installations Users may choose to install specific FSL packages by installing the desired FSL packages directly from the FSL conda channel into their own conda environment. > **Note:** All FSL conda packages list the dependencies required for their > execution, so when a particular package is installed, all of its run-time > dependencies will also be installed. However, as we transition away from > monolithic installations and towards conda-based installations, there are > likely to be omissions and errors in the dependency hierarchy. These issues > will be resolved on a case-by-case basis. ## FSL conda packages All FSL projects and their dependencies are published as conda packages. Most FSL conda packages are published to an internally maintained channel, but some packages are published to the `conda-forge` channel at https://anaconda.org. All FSL projects, and any dependencies which are not available on https://anaconda.org, are built according to a conda *recipe*, which describes how the project can be built as a conda package. All FSL conda recipes are hosted as git repositories on the FMRIB gitlab server, under the `fsl/conda/` group. To clarify, each FSL project, and internally managed dependency, comprises **two** git repositories*: * The **project** repository contains the project source code/resources, and is typically hosted at `https://git.fmrib.ox.ac.uk/fsl/` * The **recipe** repository contains a conda recipe for the project, and is hosted at `https://git.fmrib.ox.ac.uk/fsl/conda/fsl-`. FSL conda packages are built automatically using Gitlab CI - the CI infrastructure for automatically building conda packages is hosted in the fsl/fsl-ci-rules> repository. > *A small number of FSL projects have more than one recipe > repository associated with them - for example, the > [`fsl/fdt`](https://git.fmrib.ox.ac.uk/fsl/fdt) project has two recipes - > the [`fsl/conda/fsl-fdt`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-fdt) > recipe provides CPU executables, and the > [`fsl/conda/fsl-fdt-cuda`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-fdt-cuda) > recipe provides GPU/CUDA executables. ## Public and internal channels FSL projects are published to one of two conda internally managed conda channels: - The **public** channel contains stable and development packages for most FSL projects. Every time a new tag is added to a project repository, a stable conda package is built and published to the public channel. Every time commits are merged into the master branch of a repository, a development conda package is built and published to the public channel. - The **internal** channel contains stable and development packages for FSL projects which are not part of official FSL releases, and are only part of internal release. These channels are currently hosted online at the following URLs: - https://fsl.fmrib.ox.ac.uk/fsldownloads/fslconda/public/ - https://fsl.fmrib.ox.ac.uk/fsldownloads/fslconda/internal/