Skip to content
Snippets Groups Projects

Enh/create wrapper

Merged Enh/create wrapper
enh/create_wrapper into master
3 unresolved threads
Merged Paul McCarthy requested to merge enh/create_wrapper into master
3 unresolved threads

When an "official" FSL distribution is installed via the fslinstaller script, wrapper scripts for all FSL executables are created in $FSLDIR/share/fsl/bin/, so that all FSL executables can be added to the user $PATH without all executables in $FSLDIR/bin/ being added.

The new wrapper creation/removal scripts are located in the share/fsl/sbin/ directory - the intent is that the pre-unlink.sh and post-link.sh scripts of a FSL conda recipe will call these scripts during installation/removal of a FSL conda package.

Note that thhis MR includes a few rudimentary tests, and CI infrastructure for automatically running those tests (and integration with the fsl/fsl-ci-rules logic for automated conda builds).

Background

I think this is currently the best option out of the ones that have been discussed (in the related [non-existent, but perhaps should be for future code changes] gitlab issue). To make things clear, we need a solution which supports:

  • "Official" installation of a full FSL distribution (via the future fslinstaller)
  • Direct installation of a sub-set of FSL into a user-managed conda environment, via conda install

The problems we are trying to solve are solely related to official FSL installations:

  • Not clobbering the user $PATH with non-FSL commands that are installed into the same $FSLDIR/bin/, such as python, pip, conda, tcl, etc
  • Running FSL python commands in isolated mode, so that other Python libraries available in the user's environment do not clobber the Python libraries installed into $FSLDIR

Generating a "wrapper script" for every FSL command, and installing those wrapper scripts into an isolated directory allows these problems to be overcome:

  • The user can add $FSLDIR/fsl/share/bin/ to their $PATH instead of $FSLDIR/bin/
  • Wrapper scripts for Python commands can explicitly pass -I to the python interpreter so that it runs in isolated mode

Other proposals which were discussed and rejected:

  • Install everything directly into $FSLDIR/fsl/share/bin/
    • Would require all FSL scripts which refer to $FSLDIR/bin/, which would be a fair amount of work, but is possible, and hence not a show stopper
    • Would break third-party/user scripts which refer to $FSLDIR/bin/
    • would potentially have library linking issues, so would possibly need the Make system to be adjusted to be able to install into either $FSLDIR/bin/ or to $FSLDIR/fsl/share/bin/, which could get messy
    • Not "officially" possible for python projects (from what I could tell), so we would need to manually mv/cp python entry points from $FSLDIR/bin/ anyway, which feels just as hacky as a wrapper script
  • Sym-link rather than use wrapper scripts
    • Does not solve the python isolation problem, nor the problem of long hashbangs (although the latter could be considered a problem for the user)
Edited by Paul McCarthy

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
share/fsl/sbin/createFSLWrapper 0 → 100755
1 #!/usr/bin/env bash
2 #
3 # Create wrapper scripts in $FSLDIR/share/fsl/bin/ which invoke commands that
4 # are installed in $FSLDIR/bin/.
5 #
6 # This script is used to create isolated versions of all executables provided
7 # by FSL projects, so they can be added to the user $PATH without any other
8 # executables that are installed into the FSL conda environment (for example,
9 # python, pip, tcl, etc).
10 #
11 # This script should only be invoked when FSL is being installed via the
12 # fslinstaller script - it is not intended to be used when individual FSL
13 # projects are explicitly installed into a custom conda environment.
14 #
15 # The fslinstaller script should ensure that the FSLDIR and
16 # FSL_CREATE_WRAPPER_SCRIPTS variables are set appropriately before creating
  • Taylor Hanayik
    Taylor Hanayik @thanayik ·
    Owner

    @paulmc Is there a mock up of this somewhere yet? I'm guessing this fslinstaller does not exist correct?

  • Paul McCarthy
    Paul McCarthy @paulmc ·
    Author Owner

    Yes, I'm referring to a not-yet-existent future fslinstaller. Ultimately the creation of wrapper scripts is controlled by FSL_CREATE_WRAPPER_SCRIPTS being set to some non-empty value, and by FSLDIR equalling PREFIX

  • Please register or sign in to reply
  • Taylor Hanayik
    Taylor Hanayik @thanayik started a thread on the diff
    • share/fsl/sbin/createFSLWrapper 0 → 100755
    1 #!/usr/bin/env bash
    2 #
    3 # Create wrapper scripts in $FSLDIR/share/fsl/bin/ which invoke commands that
    • Taylor Hanayik
      Taylor Hanayik @thanayik ·
      Owner

      Maybe I missed something from the various convos we have in email, slack, etc. Why don't we just install the compiled binaries to $FSLDIR/share/fsl/bin/?

    • Paul McCarthy
      Paul McCarthy @paulmc ·
      Author Owner

      Basically so we don't have to update every other FSL project to refer to $FSLDIR/share/fsl/bin/ - we can leave all existing projects as referring to $FSLDIR/bin/

    • Please register or sign in to reply
    • Matthew Webster
      Matthew Webster @mwebster ·
      Owner

      Hi @thanayik by "install" do you mean build- or install- time? The wrapper scripts should be more lightweight than the binaries themselves, and it means the script only has two modes ( python and non-python )

    • Paul McCarthy
      Paul McCarthy @paulmc ·
      Author Owner

      I think @thanayik is asking why we don't just install all executables directly into $FSLDIR/share/fsl/bin/ - reasons for not doing this being:

      • Backwards compatibility with existing FSL commands (and third party scripts) which refer to $FSLDIR/bin/
      • Possible issues with compiled binaries referring to dynamic libraries relative to their installation location
      • Inability to control the installation destination for setuptools-installed python applications
    • Taylor Hanayik
      Taylor Hanayik @thanayik ·
      Owner

      Yeah, this is what I was referring to (directly installing in $FSLDIR/share/fsl/bin. In that case this makes sense. Having $FSLDIR/bin still work as it always has is important I guess. Updating existing FSL commands would be a one off task though...hopefully. I just feel a bit hesitant to have wrappers for everything. It could have the effect of making an FSLDIR appear confusing.

      However, that is probably a risk no matter what we do if people are unfamiliar with an FSLDIR anyways.

      I guess I'm just trying to make sure wrapper scripts for everything in *bin solve more complexity than they create. It appears that the last two points

      Possible issues with compiled binaries referring to dynamic libraries relative to their installation location

      Inability to control the installation destination for setuptools-installed python applications

      are perhaps the most cumbersome to overcome so I guess wrappers are the ideal solution.

    • Matthew Webster
      Matthew Webster @mwebster ·
      Owner

      This secondary bin folder ( we should probably refer to it as something like an adjunct or sanitised folder ) will only really exist as a reference in people's profiles, which generally get set up automatically. Direct reference to $FSLDIR/share/fsl/bin should otherwise be minimised.

    • Taylor Hanayik
      Taylor Hanayik @thanayik ·
      Owner

      If you guys are happy with wrappers then I'm happy with wrappers :). Ultimately it does not change how people use FSL.

    • Please register or sign in to reply
  • Paul McCarthy added 1 commit

    added 1 commit

    Compare with previous version

  • Paul McCarthy
    Paul McCarthy @paulmc ·
    Author Owner

    Don't forget the intent to preserve backwards-compatibility with third party/user scripts that refer to $FSLDIR/bin/..

  • Paul McCarthy changed the description

    changed the description

  • Matthew Webster approved this merge request

    approved this merge request

  • Paul McCarthy changed the description

    changed the description

  • Paul McCarthy merged

    merged

  • Paul McCarthy mentioned in commit 3d9c6196

    mentioned in commit 3d9c6196

    • Please register or sign in to reply