Commit 089ab328 authored by Paul McCarthy's avatar Paul McCarthy 🚵
Browse files

Merge branch 'mnt/clean-up' into 'master'

MNT: Clean-up, turn into a package

See merge request fsl/conda/installer!40
parents e99a80cf af47d410
#!/usr/bin/env bash
set -e
git fetch origin master
thisver=$( cat fsl/installer/fslinstaller.py |
grep "__version__ = " |
cut -d " " -f 3 |
tr -d "'")
masterver=$(git show origin/master:fsl/installer/fslinstaller.py |
grep "__version__ = " |
cut -d " " -f 3 |
tr -d "'")
if [ "$thisver" = "$masterver" ]; then
echo "Version has not been updated!"
echo "Version on master branch: $masterver"
echo "Version in this MR: $thisver"
echo "The version number must be updated before this MR can be merged."
exit 1
else
echo "Version on master branch: $masterver"
echo "Version in this MR: $thisver"
fi
......@@ -11,11 +11,14 @@ set -e
# Make sure that the installer version matches the tag
scriptver=$(cat fslinstaller.py | grep "__version__ = " | cut -d " " -f 3 | tr -d "'")
scriptver=$(cat fsl/installer/fslinstaller.py |
grep "__version__ = " |
cut -d " " -f 3 |
tr -d "'")
if [ "$scriptver" != "$CI_COMMIT_TAG" ]; then
echo "Version in fslinstaller.py does not match tag! $scriptver != $CI_COMMIT_TAG"
exit 1
fi
cp fslinstaller.py $FSLINSTALLER_DEPLOY_DIRECTORY/
cp fsl/installer/fslinstaller.py $FSLINSTALLER_DEPLOY_DIRECTORY/
#!/usr/bin/env bash
# The fsl package is a native namespace package, meaning that it
# does not have a __init__.py. Native namespace packages are not
# supported in Python < 3.3.
#
# https://packaging.python.org/guides/packaging-namespace-packages/
if python -c 'import sys; sys.exit(sys.version_info[:2] >= (3, 3))'; then
touch fsl/__init__.py
fi
pip install --upgrade pip setuptools wheel
pip install pytest coverage pytest-cov mock typing
pytest -v --cov=fslinstaller test
pytest -v --cov=fsl test
# CI / CD for the fsl/conda/installer project.
#
# Unit tests are run every time commits are
# pushed to any branch.
#
# Release jobs are run every time a new tag
# is added. Release jobs assume that they
# Unit tests are run in the test stage every
# time commits are pushed to any branch.
# The check-version stage is run on
# non-master branches - it checks that the
# fslinstaller version has been changed.
# The release stage is run every time a new
# tag is added. Release jobs assume that they
# are running in an environment with a
# directory $FSLINSTALLER_DEPLOY_DIRECTORY
# into which the fslinstaller.py script can
# be copied.
# The fsl-ci stages are defined in the
# fsl/conda/fsl-ci-rules repository - they are
# used to build the fslinstaller conda package.
stages:
- test
- check-version
- release
- fsl-ci-pre
- fsl-ci-build
- fsl-ci-test
- fsl-ci-deploy
# Don't run pipeline on MRs
workflow:
......@@ -89,20 +102,7 @@ check-version:
rules:
- if: '$CI_COMMIT_TAG == null && $CI_COMMIT_BRANCH != "master"'
script:
- git fetch origin master
- thisver=$( cat fslinstaller.py | grep "__version__ = " | cut -d " " -f 3 | tr -d "'")
- masterver=$(git show origin/master:fslinstaller.py | grep "__version__ = " | cut -d " " -f 3 | tr -d "'")
- |
if [ "$thisver" = "$masterver" ]; then
echo "Version has not been updated!"
echo "Version on master branch: $masterver"
echo "Version in this MR: $thisver"
echo "The version number must be updated before this MR can be merged."
exit 1
else
echo "Version on master branch: $masterver"
echo "Version in this MR: $thisver"
fi
- bash ./.ci/check_version.sh
release-new-version:
......
......@@ -11,7 +11,7 @@ This repository is the home of `fslinstaller.py`, the installer script for
The `fslinstaller.py` script in this repository is the successor to the
`fslinstaller.py` script from the fsl/installer> repository. _This_ version
is for **conda-based** FSL release, from FSL version 6.0.6 onwards.
is for **conda-based** FSL release, from FSL version 6.1.0 onwards.
`fslinstaller.py` is a Python script which can run with any version of
Python from 2.7 onwards. Normal usage of `fslinstaller.py` will look like
......@@ -24,6 +24,11 @@ one of the following:
curl https://some_url/fslinstaller.py | python
```
The `fslinstaller` script is also built as a Python package, and importable
via the `fsl.installer` package.
In normal usage, the `fslinstaller.py` script performs the following tasks:
1. Downloads the FSL release manifest file from a hard-coded URL, which is a
JSON file containing information about available FSL releases.
......@@ -41,8 +46,8 @@ In normal usage, the `fslinstaller.py` script performs the following tasks:
their shell environment.
Several advanced options are available - run `python fslinstaller.py -h` for
more details.
Several advanced options are available - run `python fslinstaller.py -h`, and
read the `parse_args` function for more details on the advanced/hidden options.
# Managing `fslinstaller.py` versions and releases
......@@ -63,6 +68,7 @@ where:
All changes to the `fslinstaller.py` must be accompanied by a change to the
`__version__` attribute in the `fslinstaller.py` script.
New versions of the `fslinstaller.py` script can be released simply by
creating a new tag, containing the new version identifier, on the
fsl/conda/installer> GitLab repository. This will cause the following
......@@ -71,8 +77,13 @@ automated routines to run:
- The new version of the `fslinstaller.py` script is deployed to a web server,
available for download.
- A merge request is opened on the fsl/conda/manifest> repository, updating
the installer version number in the FSL release manifest JSON file.
- A merge request is opened on the fsl/conda/fsl-installer> conda recipe
repository, causing the new version to be built as a conda package.
- A merge request is optionally opened on the fsl/conda/manifest> repository,
updating the installer version number in the FSL release manifest JSON
file.
Note that the tag must be identical to the value of the `__version__`
attribute in the `fslinstaller.py` script.
#!/usr/bin/env python
#
# The fsl.installer package just imports all of the useful functions from
# the fslinstaller.py script, so that other scripts can "import fsl.installer"
# instaed of "fsl.installer.fslinstaller"
from fsl.installer.fslinstaller import *
#!/usr/bin/env python
import os.path as op
from setuptools import setup, find_namespace_packages
basedir = op.dirname(__file__)
version = {}
with open(op.join(basedir, 'fsl', 'installer', 'fslinstaller.py')) as f:
for line in f:
if line.startswith('__version__ = '):
exec(line, version)
break
version = version['__version__']
with open(op.join(basedir, 'README.md'), 'rt') as f:
readme = f.read()
setup(
name='fslinstaller',
version=version,
description='Scripts to install and update FSL',
long_description=readme,
long_description_content_type='text/markdown',
url='https://git.fmrib.ox.ac.uk/fsl/conda/installer',
author='Paul McCarthy',
author_email='paul.mccarthy@ndcn.ox.ac.uk',
license='Apache License Version 2.0',
classifiers=[
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
],
packages=['fsl', 'fsl.installer'],
entry_points={
'console_scripts' : [
'fslinstaller.py = fsl.installer.fslinstaller:main',
]
}
)
#!/usr/bin/env python
import os
import os.path as op
import shutil
import subprocess as sp
import sys
import textwrap as tw
# py3
try:
from unittest import mock
# py2
except ImportError:
import mock
import pytest
from . import onpath, server
import fslinstaller as inst
def test_Context_identify_plaform():
tests = [
[('linux', 'x86_64'), 'linux-64'],
[('darwin', 'x86_64'), 'macos-64'],
[('darwin', 'arm64'), 'macos-64'],
]
for info, expected in tests:
sys, cpu = info
with mock.patch('platform.system', return_value=sys), \
mock.patch('platform.machine', return_value=cpu):
assert inst.Context.identify_platform() == expected
def test_Context_get_admin_password():
sudo = tw.dedent("""
#!/usr/bin/env bash
echo -n "Password: "
read -e password
if [ "$password" = "password" ]; then exit 0
else exit 1
fi
""").strip()
with inst.tempdir() as cwd:
path = op.pathsep.join((cwd, os.environ['PATH']))
with open('sudo', 'wt') as f:
f.write(sudo)
os.chmod('sudo', 0o755)
# right password first time
with mock.patch.dict(os.environ, PATH=path), \
mock.patch('getpass.getpass', return_value='password'):
assert inst.Context.get_admin_password() == 'password'
# wrong, then right
returnvals = ['wrong', 'password']
def getpass(*a):
return returnvals.pop(0)
with mock.patch.dict(os.environ, PATH=path), \
mock.patch('getpass.getpass', getpass):
assert inst.Context.get_admin_password() == 'password'
# wrong wrong wrong
returnvals = ['wrong', 'bad', 'no']
def getpass(*a):
return returnvals.pop(0)
with mock.patch.dict(os.environ, PATH=path), \
mock.patch('getpass.getpass', getpass):
with pytest.raises(Exception):
inst.Context.get_admin_password()
......@@ -6,7 +6,7 @@ import contextlib
import shutil
import json
import fslinstaller as inst
import fsl.installer.fslinstaller as inst
try:
from unittest import mock
......@@ -129,7 +129,7 @@ def installer_server(cwd=None):
manifest = mock_manifest.format(
version=inst.__version__,
platform=inst.Context.identify_platform(),
platform=inst.identify_platform(),
url=srv.url,
conda_sha256=conda_sha256,
env610_sha256=env610_sha256,
......@@ -187,7 +187,7 @@ def check_install(homedir, destdir, version, envver=None):
def test_installer_normal_interactive_usage():
with inst.tempdir():
with installer_server() as srv:
with mock.patch('fslinstaller.FSL_RELEASE_MANIFEST',
with mock.patch('fsl.installer.fslinstaller.FSL_RELEASE_MANIFEST',
'{}/manifest.json'.format(srv.url)):
# accept rel/abs paths
for i in range(3):
......@@ -203,10 +203,10 @@ def test_installer_normal_interactive_usage():
def test_installer_list_versions():
platform = inst.Context.identify_platform()
platform = inst.identify_platform()
with inst.tempdir():
with installer_server() as srv:
with mock.patch('fslinstaller.FSL_RELEASE_MANIFEST',
with mock.patch('fsl.installer.fslinstaller.FSL_RELEASE_MANIFEST',
'{}/manifest.json'.format(srv.url)):
with inst.tempdir() as cwd:
with CaptureStdout() as cap:
......@@ -226,7 +226,7 @@ def test_installer_list_versions():
def test_installer_normal_cli_usage():
with inst.tempdir():
with installer_server() as srv:
with mock.patch('fslinstaller.FSL_RELEASE_MANIFEST',
with mock.patch('fsl.installer.fslinstaller.FSL_RELEASE_MANIFEST',
'{}/manifest.json'.format(srv.url)):
# accept rel/abs paths
......@@ -250,7 +250,7 @@ def test_installer_normal_cli_usage():
def test_installer_devrelease():
with inst.tempdir():
with installer_server() as srv:
with mock.patch('fslinstaller.FSL_DEV_RELEASES',
with mock.patch('fsl.installer.fslinstaller.FSL_DEV_RELEASES',
'{}/devreleases.txt'.format(srv.url)):
patch_manifest('manifest.json',
'manifest-6.1.0.20220518.abcdefg.master.json',
......
......@@ -15,7 +15,7 @@ import pytest
from . import onpath, server
import fslinstaller as inst
import fsl.installer.fslinstaller as inst
SUDO = """
......
......@@ -18,7 +18,21 @@ import pytest
from . import onpath, server
import fslinstaller as inst
import fsl.installer.fslinstaller as inst
def test_identify_plaform():
tests = [
[('linux', 'x86_64'), 'linux-64'],
[('darwin', 'x86_64'), 'macos-64'],
[('darwin', 'arm64'), 'macos-64'],
]
for info, expected in tests:
sys, cpu = info
with mock.patch('platform.system', return_value=sys), \
mock.patch('platform.machine', return_value=cpu):
assert inst.identify_platform() == expected
def test_Version():
......@@ -46,6 +60,47 @@ def test_read_fslversion():
assert inst.read_fslversion(cwd) == 'abcde'
def test_get_admin_password():
sudo = tw.dedent("""
#!/usr/bin/env bash
echo -n "Password: "
read -e password
if [ "$password" = "password" ]; then exit 0
else exit 1
fi
""").strip()
with inst.tempdir() as cwd:
path = op.pathsep.join((cwd, os.environ['PATH']))
with open('sudo', 'wt') as f:
f.write(sudo)
os.chmod('sudo', 0o755)
# right password first time
with mock.patch.dict(os.environ, PATH=path), \
mock.patch('getpass.getpass', return_value='password'):
assert inst.get_admin_password() == 'password'
# wrong, then right
returnvals = ['wrong', 'password']
def getpass(*a):
return returnvals.pop(0)
with mock.patch.dict(os.environ, PATH=path), \
mock.patch('getpass.getpass', getpass):
assert inst.get_admin_password() == 'password'
# wrong wrong wrong
returnvals = ['wrong', 'bad', 'no']
def getpass(*a):
return returnvals.pop(0)
with mock.patch.dict(os.environ, PATH=path), \
mock.patch('getpass.getpass', getpass):
with pytest.raises(Exception):
inst.get_admin_password()
def test_download_file():
with inst.tempdir() as cwd:
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment