development_workflow.md 8.36 KB
Newer Older
1
2
3
# FSL development and release workflows


Paul McCarthy's avatar
Paul McCarthy committed
4
5
6
This document describes development and release workflows for individual FSL
projects - it does not describe the release process for full/official FSL
installations.
7
8


Paul McCarthy's avatar
Paul McCarthy committed
9
10
11
All FSL projects and dependencies are released and published as conda
packages, either in the internally managed FMRIB conda channel, or on
https://anaconda.org, via [`conda-forge`](https://conda-forge.org/).
12
13
14
15
16


## Project and recipe repositories


Paul McCarthy's avatar
Paul McCarthy committed
17
Every FSL project comprises two git repositories<sup>1</sup>:
18

19
20
21
 - The **project** repository contains the project source code/resources, and
   is typically hosted at
   `https://git.fmrib.ox.ac.uk/fsl/<project>`<sup>2</sup>.
22

Paul McCarthy's avatar
Paul McCarthy committed
23
24
 - The **recipe** repository contains a conda recipe for the project, and is
   hosted at `https://git.fmrib.ox.ac.uk/fsl/conda/fsl-<project>`<sup>3</sup>.
25

Paul McCarthy's avatar
Paul McCarthy committed
26
27
28
Activity on the project repository proceeds independently of activity on the
recipe repository. The recipe repository is only updated when the project
developer wishes to release a new version.
29
30


Paul McCarthy's avatar
Paul McCarthy committed
31
32
33
34
35
> <sup>1</sup>A small number of FSL projects have more than one recipe
> repository associated with them - for example, the
> [`fsl/fdt`](https://git.fmrib.ox.ac.uk/fsl/fdt) project has two recipes -
> the [`fsl/conda/fsl-fdt`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-fdt)
> recipe provides CPU executables, and the
36
> [`fsl/conda/fsl-fdt-cuda`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-fdt-cuda)
Paul McCarthy's avatar
Paul McCarthy committed
37
> recipe provides GPU/CUDA executables.
38
39


Paul McCarthy's avatar
Paul McCarthy committed
40
41
42
43
44
45
46
> <sup>2</sup>Most, but not all, FSL projects are hosted in the FMRIB GitLab
> [`fsl/` group](https://git.fmrib.ox.ac.uk/fsl). Some projects are hosted
> under the username of the developer
> (e.g. [`matteob/eddy_qc`](https://git.fmrib.ox.ac.uk/matteob/eddy_qc)). Some
> FSL projects and dependencies are hosted externally, such as
> [MSM](https://github.com/ecr05/MSM_HOCR) and
> [oxford_asl](https://github.com/ibme-qubic/oxford_asl/). The conda recipes
47
> for these projects are, however, all hosted internally in the `fsl/conda/`
Paul McCarthy's avatar
Paul McCarthy committed
48
> GitLab group.
49
50


Paul McCarthy's avatar
Paul McCarthy committed
51
52
53
54
> <sup>3</sup>**All** FSL conda recipes are hosted in the
> [`fsl/conda/`](https://git.fmrib.ox.ac.uk/fsl/conda) GitLab group, with
> the sole exception of FSL projects which are published to `conda-forge`
> (e.g. [`fsleyes`](https://github.com/conda-forge/fsleyes-feedstock)).
55
56


Paul McCarthy's avatar
Paul McCarthy committed
57
## FSL project versioning scheme
58
59


Paul McCarthy's avatar
Paul McCarthy committed
60
61
62
63
FSL projects (and hence their conda packages) must follow a versioning scheme
whereby each version comprises sequentially increasing numbers followed by
periods. There are no more requirements beyond this, but the use of [semantic
versioning](https://semver.org) is encouraged.
64

Paul McCarthy's avatar
Paul McCarthy committed
65
66
67
68
69
70
71
72
73
74
75
76

If you are developing a project with a programming or command-line interface,
a responsible strategy for managing releases is to use the version number to
denote changes to the interface (for example, by using semantic versioning),
and to use deprecation warnings to warn users of impending changes to the
interface.


Having said this, many core FSL projects (e.g. `miscmaths`, `newimage`, etc)
change very infrequently, so the decision was made to *not* follow semantic
versioning principles, but instead to use a versioning scheme whereby each new
release is given a version number of the form:
77
78
79

    YYMM.X

Paul McCarthy's avatar
Paul McCarthy committed
80
81
82
83
84
where `YYMM` is the year and month of release (e.g. `2101` for January 2021),
and `X` is an incremental release number, starting at `0`, which is used to
avoid conflict in the event of multiple releases within the same month. For
example, two versions of a core FSL project released in the month of January
2021 would be assigned versions `2021.0` and `2021.1`.
85
86


87
88
89
> See also the section below on development/staging releases.


90
91
92
## FSL conda package naming conventions


Paul McCarthy's avatar
Paul McCarthy committed
93
94
95
96
FSL conda package names must follow the [conda package naming
conventions](https://conda.io/projects/conda-build/en/latest/concepts/package-naming-conv.html#index-0),
and be comprised solely of _lowercase alpha characters, numeric digits,
underscores, hyphens, or dots_.
97
98


Paul McCarthy's avatar
Paul McCarthy committed
99
100
101
102
103
104
Furthermore, all FSL conda packages are prefixed with `fsl-`. An FSL project
with name `<project>` will have a corresponding conda package name of
`fsl-<project>`. For FSL projects with a name that begins with `fsl`
(e.g. `fslvbm`, `fsl_deface`), the leading `fsl` will be dropped in the
construction of the corresponding conda-package name. For example:

105
106
107
108
109
110
111
112

| **FSL project name** | **Conda package name** |
| -------------------- | ---------------------- |
| `avwutils`           | `fsl-avwutils`         |
| `fslvbm`             | `fsl-vbm`              |
| `fsl_deface`         | `fsl-deface`           |
| `fsl-mrs`            | `fsl-mrs`              |
| `NewNifti`           | `fsl-newnifti`         |
Paul McCarthy's avatar
Paul McCarthy committed
113

114
115
116
117

## Contributing to a FSL project


Paul McCarthy's avatar
Paul McCarthy committed
118
119
120
No restrictions are placed on the ways in which development of a particular
FSL project takes place. Developers are free to manage their project in any
way they wish.
121
122


Paul McCarthy's avatar
Paul McCarthy committed
123
124
125
However, contribution to core FSL projects (`avwutils`, `newimage`, etc)
follow a merge request and review process, and it is recommended that
developers follow a similar process for their own projects.
126
127


Paul McCarthy's avatar
Paul McCarthy committed
128
129
130
131
 - The `master` branch of the project repository should be considered stable
   and ready to release at any time.
 - All development should occur on a separate branch within the project
   repository, or personal fork of the project repository.
132
 - All changes to the `master` branch should take place through GitLab [merge
Paul McCarthy's avatar
Paul McCarthy committed
133
134
135
136
   requests](https://docs.gitlab.com/ee/user/project/merge_requests/) (MRs).
 - When a MR is opened, it is reviewed, approved, and merged into the `master`
   branch by the project maintainer, possibly after any requested changes to
   the MR have been made.
137
138


Paul McCarthy's avatar
Paul McCarthy committed
139
## Releasing a new version of a FSL project
140
141


Paul McCarthy's avatar
Paul McCarthy committed
142
143
144
A new versions of a FSL project is denoted by a new tag being added to the
project repository. Tag names must follow the **versioning scheme** outlined
above.
145
146


Paul McCarthy's avatar
Paul McCarthy committed
147
When a new tag is added to a FSL project repository, the corresponding recipe
148
149
repository is updated, and a new conda package built and published to the
public FSL conda channel.
150
151


Paul McCarthy's avatar
Paul McCarthy committed
152
153
154
For projects which are included in FSL, but developed externally
(e.g. [MSM](https://github.com/ecr05/MSM_HOCR)), the recipe repository is
manually updated on an as-needed basis.
155
156


157
158
159
160
161
> **Note:** Some FSL projects are not included in public FSL releases, and are
> only intended to be installed internally. The conda recipe repository for
> such a project can be configured so that both stable and development
> packages are instead published to the internal FSL conda channel - see the
> fsl/fsl-ci-rules> documentation for more details.
162

163

164
## Development releases
165
166


167
168
169
170
171
As described above, every time a new tag is added to a project repository, a
*stable* package is built and uploaded to the public FSL conda
channel. Additionally, every time new commits are added to the master branch
of a project repository, a *development* package is built and uploaded to the
public FSL conda channel.
172
173


174
175
176
177
178
FSL projects are released as conda packages, and therefore must adhere to
conda package versioning conventions, described at
https://docs.conda.io/projects/conda/en/latest/user-guide/concepts/pkg-specs.html#version-ordering.


179
180
181
182
183
Stable FSL packages are versioned as described above in the **FSL project
versioning scheme** section. Development FSL packages are versioned slightly
differently, to ensure a sensible ordering of version numbers across both
stable and development packages. The version string for a development package
adheres to the following structure:
184
185


186
    <last_stable_tag>.post0.dev<date>[+<commit>]
187
188
189
190


where:

191
 - `<last_stable_tag>` is the version/tag for the most recent stable release
192
193
 - `<date>` is the date (`YYYYMMDD`) that the staging package was built
 - `<commit>` is the first 7 characters of the git commit that the staging
Paul McCarthy's avatar
Paul McCarthy committed
194
195
   package was built from. This may be omitted for externally hosted
   projects.
196
197


Paul McCarthy's avatar
Paul McCarthy committed
198
For example, if the latest stable release for the `fsl/base` project is
199
200
`2106.1`, and a new commit, with hash `ac28d14`, is subsequently pushed to the
`fsl/base` master branch on 8 June 2021, the resulting development package
201
202
203
will be assigned the version string `2106.1.post0.dev20210608+ac28d14`.


204
205
206
207
208
209
210
211
This convention ensures that development packages which are built after the
most recent stable package will be given higher priority, and will be
installed in preference to the stable package.


> Note: For official FSL releases, the versions of all FSL packages are pinned
> to specific stable releases, so development packages will not take priority
> in this instance.