Commit 4c9d93c9 authored by Paul McCarthy's avatar Paul McCarthy 🚵
Browse files

DOC: Section on post-link/pre-unlink

parent 97a7a7d7
......@@ -6,14 +6,16 @@ project repositories have an associated recipe repository, which contains
metadata describing how to build a conda recipe from the project repository.
You can create a conda recipe by hand, or you can use a script to
automatically generate an initial version of a repository for your project.
You can use a script to automatically generate an initial version of a
repository for your project (recommended), or you can create a conda recipe by
hand.
Conda recipes for all FSL projects, and some internally packaged dependencies,
are hosted in the [fsl/conda](https://git.fmrib.ox.ac.uk/fsl/conda/) gitlab
namespace. The name of a FSL conda recipe repository is the same as the name
of the FSL conda package - for example, the conda package for the
Conda recipes for all FSL projects, and some internally packaged/managed
third-party dependencies, are hosted in the GitLab
[fsl/conda](https://git.fmrib.ox.ac.uk/fsl/conda/) namespace. The name of a
FSL conda recipe repository is the same as the name of the FSL conda package -
for example, the conda package for the
[`fsl/avwutils`](https://git.fmrib.ox.ac.uk/fsl/avwutils/) project is named
`fsl-avwutils`, and is hosted at
[`fsl/conda/fsl-avwutils`](https://git.fmrib.ox.ac.uk/fsl/conda/fsl-avwutils/).
......@@ -34,6 +36,7 @@ with name `<project>` will have a corresponding conda package name of
(e.g. `fslvbm`, `fsl_deface`), the leading `fsl` will be dropped in the
construction of the corresponding conda-package name. For example:
| **FSL project name** | **Conda package name** |
| -------------------- | ---------------------- |
| `avwutils` | `fsl-avwutils` |
......@@ -61,7 +64,7 @@ To generate an initial version of a conda recipe for a new project:
2. Create a [GitLab personal access
token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#personal-access-tokens),
which will allow you to use the GitLab API.
which will allow you to programmatically interact with GitLab.
3. Clone the `fsl/fsl-ci-rules` repository, and configure your environment
so you can use `create_conda_recipe`:
......@@ -85,14 +88,27 @@ To generate an initial version of a conda recipe for a new project:
The `create_conda_recipe` script will automatically create a conda recipe
repository on GitLab, named according to the FSL conda package naming
conventions outlined above.
repository on GitLab, within the
[fsl/conda](https://git.fmrib.ox.ac.uk/fsl/conda) namespace, and named
according to the FSL conda package naming conventions outlined above. For
example, this command command:
```
fslpython -m create_conda_recipe -t <token> paulmc/my_new_project
```
will create a conda recipe for a GitLab project repository at
`https://git.fmrib.ox.ac.uk/paulmc/my_cool_project`, for a conda package
called `fsl-my_cool_project`, and will upload the recipe to a GitLab
repository at `https://git.fmrib.ox.ac.uk/fsl/conda/fsl-my_cool_project`.
The `create_conda_recipe` script has a few options allowing you to control its
behaviour (e.g. create a recipe repository locally insteaad of on GitLab,
create the recipe from a branch other than `master` from the project
repository) - run `fslpython -m create_conda_recipe -h` for help on the
repository); run `fslpython -m create_conda_recipe -h` for help on the
available options.
......@@ -100,18 +116,22 @@ available options.
We recommend using the `create_conda_recipe` script as outlined above, to
generate an initial version of the recipe for your FSL projec. This has the
generate an initial version of the recipe for your FSL project. This has the
advantage that recipes for all FSL projects will follow a few simple
conventions and standards. However, it is possible to create a conda recipe
by hand.
conventions and standards. However, it is possible to create a conda recipe by
hand.
The purpose of this section is not to provide full instructions on writing a
conda recipe, but rather on the specific issues that need to be considered
when writing a conda recipe for a FSL project. More general information on
creating conda recipes can be found at these websites:
Useful information on creating conda recipes can be found at these websites:
- https://docs.conda.io/projects/conda-build/en/latest/resources/define-metadata.html
- https://python-packaging-tutorial.readthedocs.io/en/latest/conda.html
A FSL conda recipe contains the following:
A FSL conda recipe is simply a flat directory containing the following files:
- `meta.yaml`: A YAML file which contains a description of the conda package,
including its name, current version, URL to the project repository, and
......@@ -131,12 +151,11 @@ A FSL conda recipe contains the following:
- `pre-unlink.sh`: This script is called when a package is *uninstalled* -
its job is to remove any wrapper scripts that were created by
`post-link.sh`. script is not required for projects which do not provide
any executables.
`post-link.sh`. This script is not required for projects which do not
provide any executables.
A FSL conda recipe is simply a flat directory containing the above files. FSL
projects are broadly divided into one of the following types:
FSL projects are broadly divided into one of the following categories:
- **`Makefile`-based project**: FSL projects which use a FSL-style
`Makefile` to compile and install their deliverables (shared libraries,
......@@ -147,12 +166,12 @@ projects are broadly divided into one of the following types:
a Python package.
The mechanisms by which projects of these types are built are slightly
different and, therefore, the conda recipe for projects of different types
will look slightly different. Examples of `Makefile`-based and
`setup.py`-based FSL projects, and associated conda recipes, can be found in
the `examples/cpp` and `examples/python` sub-directories. Some important
details are highlighted below.
The mechanisms by which projects in these categories are built are slightly
different and, therefore, the conda recipe for projects from different
category will look slightly different. Examples of `Makefile`-based and
`setup.py`-based FSL projects, and associated conda recipes, can be
respectively found in the `examples/cpp` and `examples/python`
sub-directories. Some important details are highlighted below.
### Writing the `meta.yaml` file
......@@ -190,6 +209,10 @@ package:
```
By following this convention, FSL conda recipes can be automatically updated
when a new version of a project is released.
**Project repository and git revision** The automated CI rules defined in
`fsl-ci-rules` allow the project repositoy and git revision (tag, branch, etc)
used to build a conda package to be overridden via the `FSLCONDA_REPOSITORY`
......@@ -209,6 +232,11 @@ source:
```
Following this convention allows conda packages for development and testing to
be built, both automatically, and when developing/testing locally, simply by
setting the `FSLCONDA_REPOSITORY` and `FSLCONDA_REVISION` variables.
**run_exports (C/C++ projects only)** When compiling a C/C++ project, any
shared library dependencies of the project must be present at the time of
compilation, and at run time. This means that the dependencies of your project
......@@ -217,9 +245,17 @@ may need to be listed twice within the `requirements` section - once under
`run_exports` in the recipes of the dependencies.
The
[`run_exports`](https://docs.conda.io/projects/conda-build/en/latest/resources/define-metadata.html#export-runtime-requirements)
section is a trick which can be used within `meta.yaml`, which essentially
allows us to define a dependency as a build-time dependency, and have it
automatically propagated as a run-time dependency. It is used simply to allow
us to avoid having to list shared library dependencies twice.
So if you are writing a conda recipe for a C/C++ project which provides shared
library files that will be used by other projects, add a `run_exports` section to
the `build` section like so:
library files that will be used by other projects, add a `run_exports` section
to the `build` section like so:
```
......@@ -231,23 +267,15 @@ build:
```
The
[`run_exports`](https://docs.conda.io/projects/conda-build/en/latest/resources/define-metadata.html#export-runtime-requirements)
section is a trick which can be used within `meta.yaml`, which essentially
allows us to define a dependency as a build-time dependency, and have it
automatically propagated as a run-time dependency. It is used simply to allow
us to avoid having to list shared library dependencies twice.
**`noarch` and `script` (`setup.py`-based projects only)** Conda recipes for
Python `setup.py`-based projects are built slightly differently to native
projects, as the Python `setuptools` machinery is integrated into the conda
build process. For `setup.py`-based projects, the build command is usually
specified within the `build` section of the `meta.yaml` file. Furthermore, if
you are packging a "pure" Python project, with no natively compiled code or
you are packging a *pure* Python project, with no natively compiled code or
extensions, you must label your recipe as being of type `noarch: python` -
this is also done within the `build` section. So a typical `build` section for
a `setup.py`-based project will resemble the following:
this is also specified within the `build` section. So a typical `build`
section for a `setup.py`-based project will resemble the following:
```
......@@ -262,7 +290,7 @@ build:
provides the fundamental elements of the contents of a FSL installation,
including FSL initialisation scripts and the `Makefile` machinery. As such it
must be installed in order to build `Makefile`-based FSL projects, and must be
present at run time for many FSL commands to function. All FSL conda recipes
present at run time for most FSL commands to function. All FSL conda recipes
must therefore list `fsl-base` as a requirement. C/C++ projects will also need
to have a C++ compiler installed at build time - the convention within conda
recipes is to list compilers as `host` dependencies. For example:
......@@ -310,5 +338,47 @@ If you are writing a recipe for a `setup.py`-based FSL project, a `build.sh`
script is generally not required.
### `post-link.sh` and `pre-unlink.sh` scripts.
Official/full FSL installations contain wrapper scripts for every FSL
executable in `$FSLDIR/share/fsl/bin/` - this is so that a user can add FSL
commands to their `$PATH` via this directory, rather than the `$FSLDIR/bin/`
directory, and avoid adding all of the other executables in `$FSLDIR/bin/` to
their `$PATH` (e.g. `python`).
These wrapper scripts are created/removed by two utility commands which are
provided by the `fsl-base` package - `createFSLWrapper` and
`removeFSLWrapper`. The conda recipes for any FSL projects which provide
executables need to call these scripts at the time of
installation/uninstallation to ensure that wrapper scripts for the executables
are created/removed.
This can be achieved by using [`post-link.sh` and
`pre-unlink.sh`](https://docs.conda.io/projects/conda-build/en/latest/resources/link-scripts.html)
scripts, which are conda-specific mechanisms allowing custom logic to be
executed when a conda package is installed or uninstalled.
For a FSL project which provides executables called `fsl_command1`,
`fsl_command2` and `fsl_command3`, the `post-link.sh` for the project recipe
should contain:
```
if [ -e ${FSLDIR}/share/fsl/sbin/createFSLWrapper ]; then
${FSLDIR}/share/fsl/sbin/createFSLWrapper fsl_command1 fsl_command2 fsl_command3
fi
```
The corresponding `pre-unlink.sh` script should contain:
```
if [ -e ${FSLDIR}/share/fsl/sbin/removeFSLWrapper ]; then
${FSLDIR}/share/fsl/sbin/removeFSLWrapper fsl_command1 fsl_command2 fsl_command3
fi
```
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment