ENH: Allow tests to be run on cluster via fsl_sub
This MR makes several changes to the behaviour of pyfeeds, and adds the capability to submit tests to a HPC cluster via fsl_sub. This can be achieved with the new --fslsub option. There is also a new --fslsubOptions option which allows additional per-test submission options to be specified (e.g. jobram=4, coprocessor=cuda, etc). These can be added to a pyfeeds configuration file under a [fslsubOptions] section, e.g.:
[fslsubOptions]
*eddy* = coprocessor=cuda
*feat* = jobram=4 jobtime=120
Note that when using pyfeeds run --fslsub on the FMRIB cluster, you must specify a --sandboxDir location which is on the shared file system, i.e. which is accessible to both the submission node and cluster nodes.
This MR also fixes a race condition which can occur when running nested tests in parallel. For example, with the following test structure:
test1/feedsRun
test1/test2/feedsRun
The output directory structure will also be nested, i.e.:
test1/feedsRun.log
test1/test2/feedsRun.log
If these tests are run in parallel, test2 may be executed before test1, meaning that the test1/test2/ directory may already exist.
A number of other changes have been made:
- Changes the method used to run tests in parallel from a
ThreadPoolto amultiprocessing.Pool- this is so that evaluation of test outputs can be run in parallel. AThreadPoolis still used when using the new--fslsuboption. - The final
pyfeedsresults table now distinguishes between tests which fail due to a non-zero exit code (reported asErrored), and tests which fail due to their outputs not matching their benchmark data (reported asFailed). - If
pyfeedsis called with the--overwriteflag, the test sandbox directory already exists, it is deleted. - Migrates the packaging configuration from using
setup.pytopyproject.toml. - Changes the version number management strategy to use
setuptools-scm.