# FSL release manifest This repository is used to manage FSL releases. The `fsl-release.yml` file defines the packages that are installed as part of a FSL release. FSL release `manifest.json` and `environment.yml` files are automatically generated from information stored in this repository, using CI rules and scripts in the fsl/conda/manifest-rules> repository. This repository contains two branches: - Public FSL releases are created from tags on the `external` branch. - Development releases are created from commits on the `external` branch. - Internal FSL releases are created from the `master` branch ## The `fslinstaller.py` script FSL is installed using a Python 2/3 script called [fslinstaller.py](https://git.fmrib.ox.ac.uk/fsl/conda/installer). The `fslinstaller` script starts by downloading a _manifest_ file (referred to as `manifest.json`), a JSON file which contains information about: - available FSL releases (past and present) - miniconda installer releases, upon which FSL installations are based - The latest available version of the fslinstaller script itself (used for self-updating) Once the appropriate FSL release has been determined, the `fslinstaller` downloads a FSL release file - A FSL release is defined by a conda environment specification file (referred to as `environment.yml`), which is a YAML file that contains a list of the individual packages and their respective versions that comprise a specific FSL release. Separate `environment.yml` files exist for each supported platform and CUDA version. ## Generating the `manifest.json` and `environment.yml` files The `manifest.json` and `environment.yml` files are automatically generated from information stored in this repository: - The `fsl-release.yml` file contains information about the `fslinstaller`, miniconda installers, and packages for the latest FSL release. - Information about past FSL releases is obtained from the most recently generated `manifest.json` file. The `fsl-release.yml` file is a YAML file with the following structure: ```yaml # URL to FSL release information. The fslinstaller.py # script, and previously generated manifest.json # file is assumed to be downloadable from this URL. # URLs for newly generated environment files are # constructed using this URL. release_url: https://fsl.fmrib.ox.ac.uk/fsldownloads/fslconda/releases/ # Latest fslinstaller version installer: 1.2.3 # URLs to miniconda installers to # use for all supported platforms miniconda: linux-64: https://github.com/conda-forge/miniforge/releases/download/4.10.3-1/Miniforge3-4.10.3-1-Linux-x86_64.sh macos-64: https://github.com/conda-forge/miniforge/releases/download/4.10.3-1/Miniforge3-4.10.3-1-MacOSX-x86_64.sh # Versions of CUDA for which FSL packages are # built (currently only available on Linux). cuda: - 9.2 - 10.2 - 11.0 # Conda channels to source packages from channels: - https://fsl.fmrib.ox.ac.uk/fsldownloads/fslconda/public/ - conda-forge - defaults # Private channel containing packages that are # only released internally. This is prepended # to the channels list for internal releases. internal_channel: http://${FSLCONDA_USERNAME}:${FSLCONDA_PASSWORD}@https://fsl.fmrib.ox.ac.uk/fsldownloads/fslconda/internal/ # The FSL installation is created in two stages. # First of all, the base_packages are installed. # Versions of these packages are specified in # the packages list below base_packages: - fsl-base - fslpy - fsl-sub - libopenblas ... # List of all packages that comprise a FSL # installation. All packages not installed # in the first stage (see base_packages # above) are installed in the second stage. packages: - ciftilib 1.5.3 - fsleyes 1.3.0 - fslpy 3.7.0 - fsl-avwutils 2007.0 - fsl-base 2109.1 - fsl-eddy-cpu 2106.0 - fsl-sub 2.5.8 - libopenblas 0.3.18 *openmp* ... ``` When changes are made to the `fsl-release.yml` file, new versions of the `manifest.json` and `environment.yml` files are generated and made available for download. Note that there are two types of manifest file: - The official release manifest, containing information about past and present public FSL releases - this is always called `manifest.json`, and may be re-generated on new public releases, and on updates to e.g. the fslinstaller script. - Development manifests, containing information about one internal or development FSL release. Development manifests are generated for every internal or development release. Development manifests are named according to a standard convention described in the **Internal/development FSL releases** section, below. Manifest and environment files can only be published when they are generated from a new tag, or when they are generated from specific branches (currently `master` or `external`). However, manifest and environment files can be downloaded as GitLab artifacts from any branch. ## New FSL releases _Summary:_ - New public FSL release are created by adding a tag to the `master` branch of this repository - The `deploy-release-manifest` and `deploy-environment-files` jobs, on the CI pipeline for the new tag, must be manually started in order to publish the new release. New public FSL releases are denoted by adding a tag to the `external` branch of this repository. The tag is used as the FSL version identifier, which must be of the form `X.Y.Z[.W]`, where X, Y, Z, and W are integers. When a new tag is added to this repository, the `manifest.json` and `environment.yml` files are generated from the contents of `fsl-release.yml`, and the contents of the previous `manifest.json` (used for obtaining information about past FSL releases). These files are named according to these conventions: - `manifest.json` - the release manifest - `fsl-_.yml` - FSL version `` for platform `` (e.g. `fsl-6.1.0_macos-64.yml`). - `fsl-__cuda.yml` - FSL version `` for platform `` and CUDA version `` (e.g. `fsl-6.1.0_linux-64_cuda11.0.yml`). After they have been generated, a test installation of the new release on each supported platform is performed using the `fslinstaller.py` script, and then the manifest and environment files are published to the FSL release web server. This publishing step must be manually triggered through the GitLab web interface, via the `deploy-environment-files` and `deploy-release-manifest` jobs. ## Internal/development FSL releases _Summary:_ - New internal FSL release may be created by adding commits to the `master` branch of this repository - New development releases may be created by adding commits to the `external` branch of this repository - In both cases, the `deploy-environment-files` and `deploy-development-manifest` jobs on the associated CI pipelines must be started manually. When commits are added to the `master` or `external` branch of this repository, separate `manifest.json` and `environment.yml` files are generated from the contents of `fsl-release.yml` and made available for download. These files are assigned a version identifier of the form `...`, where: where: - `` is the _most recent_ public FSL release version identifier (e.g `6.1.0`). - `` is the date of the commit that the files are generated from, in the form `YYYYMMDD`. - `` is the short hash of the git commit that the files are generated from. - `` is the name of the branch on this repository. The released files are then named as follows, where: - `manifest-....json` - the development release manifest file - `fsl-..._.yml` - FSL environment file for platform ``. - `fsl-...__cuda.yml` - FSL environment file for platform `` (e.g. `linux-64`) and CUDA version `` (e.g. `11.0`). After they have been generated, the manifest and environment files may be published. As with public releases, the process of publishing the manifest and environment files must be manually triggered through the GitLab web interface, via the `deploy-environment-files` and `deploy-development-manifest` jobs. ## Updates to `manifest.json` _Summary:_ - The public release `manifest.json` file can be updated by adding commits to the `master` or `external` branches of this repository - The `deploy-release-manifest` job on the associated CI pipeline must be started manually. Sometimes the contents of the official release `manifest.json` file may need to be changed independently of a public FSL release - for example, a new version of the `fslinstaller.py` script may be released, or the `miniconda` installer may need to be changed. Normally, `manifest.json` files that are not associated with a tag are named `manifest-....json` as outlined above. However, a new release `manifest.json` file is always re-generated, and can be deployed through the Gitlab web interface, via the `deploy-release-manifest` job (with the caveat that only files generated from the `master` or `external` branches may be published). Release `manifest.json` files which are generated from branches will only contain information about past official FSL releases. Release `manifest.json` files which are generated from new tags will contain information about past official FSL release, and also about the new FSL release.