Commit 49530d8c authored by Paul McCarthy's avatar Paul McCarthy 🚵
Browse files

ENH: Document describing development / release workflow.

parent e48dd83d
......@@ -6,18 +6,21 @@ _This documentation is a work in progress_
This repository hosts information and documentation regarding the FSL conda package infrastructure.
This `README` file contains a general overview. More detailed information on different topics can be found in the folowing files:
- [`official_fsl_installations.md`](official_fsl_installations.md): Details on "official"/full conda-based FSL installations.
- [`development_workflow.md`](development_workflow.md): Details on development and release workflows and processes.
- [`official_fsl_installations.md`](official_fsl_installations.md): Details on official/full conda-based FSL installations.
- [`local_development.md`](local_development.md): Setting up an FSL development environment on your own machine.
- [`creating_fsl_conda_recipes.md`](creating_fsl_conda_recipes.md): Details on how to create a FSL conda recipe.
- [`building_fsl_conda_packages.md`](building_fsl_conda_packages.md): Details on how to build a FSL conda package locally.
- [`creating_fsl_conda_recipes.md`](creating_fsl_conda_recipes.md): Details on how to create a FSL conda recipe.
## Overview
FSL is installed using the [**conda**](https://conda.io/) package and environment management system. Each FSL project is built as a conda package, and published to an internally hosted conda channel.
Prior to FSL 6.1.0, FSL was released and installed as a monolithic archive file, containing most FSL tools, and a separate conda environment called "`fslpython`", configured after the archive file was installed. Some of the reasons for transitioning to a purely conda-based release model are as follows:
- Simplify FSL installation procedure
......@@ -32,12 +35,15 @@ Prior to FSL 6.1.0, FSL was released and installed as a monolithic archive file,
Each official FSL release is defined by a conda [`environment.yml` specification](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually). This specification contains a list of all of the packages, including their versions, to be installed.
The official fsl/installer> script performs the following tasks:
1. Downloads a `miniconda` installer.
2. Installs a base `miniconda` environment into `$FSLDIR/../.fslconda/` (where `$FSLDIR` defaults to `/usr/local/fsl/`, or is explicitly specified by the user).
3. Creates a child conda environment at `$FSLDIR`.
4. Installs all FSL packages into the `$FSLDIR` conda environment.
5. Sets up the user shell profile so that FSL is configured in their shell environment by default.
## Partial FSL installations
......@@ -48,17 +54,17 @@ Users may choose to install specific FSL packages by installing the desired FSL
## FSL conda packages
All FSL projects and their dependencies are published as conda packages, either in the internal FMRIB conda channel, or on https://anaconda.org, in one of the `conda-forge` or `defaults` channels.
All FSL projects, and any dependencies which are not available on https://anaconda.org, are built according to a conda "recipe", which describes how the project can be built as a conda package. All FSL conda recipes are hosted as git repositories on the FMRIB gitlab server, under the `fsl/conda/` group.
To clarify, each FSL project, and internally managed dependency, comprises **two** git repositories:
* The project repository contains the project source code/resources, and is typically hosted at `https://git.fmrib.ox.ac.uk/fsl/<project>`<sup>*</sup>
* The project repository contains the project source code/resources, and is typically hosted at `https://git.fmrib.ox.ac.uk/fsl/<project>`
* The recipe repository contains a conda recipe for the project, and is hosted at `https://git.fmrib.ox.ac.uk/fsl/conda/fsl-<project>`.
> <sup>*</sup> Some FSL projects and dependencies are hosted externally, such as [MSM](https://github.com/ecr05/MSM_HOCR) and [oxford_asl](https://github.com/ibme-qubic/oxford_asl/). The conda recipes for these projects are, however, hosted internally in the `fsl/conda/` gitlab group.
FSL conda packages are built automatically using Gitlab CI - the CI infrastructure for automatically building conda packages is hosted in the fsl/fsl-ci-rules> repository.
# FSL development and release workflows
This document describes development and release workflows for individual FSL projects - it does not describe the release process for full/official FSL installations.
All FSL projects are released and published as conda packages, either in the internal FMRIB conda channel, or on https://anaconda.org, via [`conda-forge`](https://conda-forge.org/).
## Project and recipe repositories
Every FSL project comprises two git repositories:
- The project repository contains the project source code/resources, and is typically hosted at `https://git.fmrib.ox.ac.uk/fsl/<project>`<sup>1</sup>.
- The recipe repository contains a conda recipe for the project, and is hosted at `https://git.fmrib.ox.ac.uk/fsl/conda/fsl-<project>`<sup>2</sup>.
Activity on the project repository proceeds independently of activity on the recipe repository. The recipe repository is only updated when the project developer wishes to release a new version.
> <sup>1</sup>Most, but not all, FSL projects are hosted in the FMRIB Gitlab [`fsl/` group](https://git.fmrib.ox.ac.uk/fsl). Some projects are hosted under the username of the developer (e.g. [`matteob/eddy_qc`](https://git.fmrib.ox.ac.uk/matteob/eddy_qc)). Some FSL projects and dependencies are hosted externally, such as [MSM](https://github.com/ecr05/MSM_HOCR) and [oxford_asl](https://github.com/ibme-qubic/oxford_asl/). The conda recipes for these projects are, however, hosted internally in the `fsl/conda/` Gitlab namespace.
> <sup>2</sup>**All** FSL conda recipes are hosted in the [`fsl/conda/`](https://git.fmrib.ox.ac.uk/fsl/conda) Gitlab namespace, with the sole exception of FSL projects which are published to `conda-forge` (e.g. [`fsleyes`](https://github.com/conda-forge/fsleyes-feedstock)).
## FSL project versioning scheme
FSL projects (and hence their conda packages) must follow a versioning scheme whereby each version comprises sequentially increasing numbers followed by periods. There are no more requirements beyond this, but the use of [semantic versioning](https://semver.org) is encouraged.
If you are developing a project with a programming or command-line interface, a responsible strategy for managing releases is to use the version number to denote changes to the interface (for example, by using semantic versioning), and to use deprecation warnings to warn users of impending changes to the interface.
Having said this, many core FSL projects (e.g. `miscmaths`, `newimage`, etc) change very infrequently, so the decision was made to *not* follow semantic versioning principles, but instead to use a versioning scheme whereby each new release is given a version number of the form:
YYMM.X
where `YYMM` is the year and month of release (e.g. `2101` for January 2021), and `X` is an incremental release number, starting at `0`, which is used to avoid conflict in the event of multiple releases within the same month. For example, two versions of a core FSL project released in the month of January 2021 would be assigned versions `2021.0` and `2021.1`.
## FSL conda package naming conventions
FSL conda package names must follow the [conda package naming conventions](https://conda.io/projects/conda-build/en/latest/concepts/package-naming-conv.html#index-0), and be comprised solely of _lowercase alpha characters, numeric digits, underscores, hyphens, or dots_.
Furthermore, all FSL conda packages are prefixed with `fsl-`. An FSL project with name `<project>` will have a corresponding conda package name of `fsl-<project>`. For FSL projects with a name that begins with `fsl` (e.g. `fslvbm`, `fsl_deface`), the leading `fsl` will be dropped in the construction of the corresponding conda-package name. For example:
| **FSL project name** | **Conda package name** |
| -------------------- | ---------------------- |
| `avwutils` | `fsl-avwutils` |
| `fslvbm` | `fsl-vbm` |
| `fsl_deface` | `fsl-deface` |
| `fsl-mrs` | `fsl-mrs` |
| `NewNifti` | `fsl-newnifti` |
## Contributing to a FSL project
No restrictions are placed on the ways in which development of a particular FSL project takes place. Developers are free to manage their project in any way they wish.
However, contribution to core FSL projects (`avwutils`, `newimage`, etc) follow a merge request and review process, and it is recommended that developers follow a similar process for their own projects.
- The `master` branch of the project repository should be considered stable and ready to release at any time.
- All development should occur on a separate branch within the project repository, or personal fork of the project repository.
- All changes to the `master` branch should take place through Gitlab [merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/) (MRs).
- When a MR is opened, it is reviewed, approved, and merged into the `master` branch by the project maintainer, possibly after any requested changes to the MR have been made.
## Releasing a new version of a FSL project
A new versions of a FSL project is denoted by a new tag being added to the project repository. Tag names must follow the **versioning scheme** outlined above.
When a new tag is added to a FSL project repository, the corresponding recipe repository is updated, and a new conda package built and released.
For projects which are included in FSL, but developed externally (e.g. [MSM](https://github.com/ecr05/MSM_HOCR)), the recipe repository is manually updated on an as-needed basis.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment