UKB_UNCONFOUND_v2 merge requestshttps://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/merge_requests2021-02-28T15:53:26+00:00https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/merge_requests/9Fix FLIPPEDSWI call in gen_initial_workspace2021-02-28T15:53:26+00:00Tom NicholsFix FLIPPEDSWI call in gen_initial_workspaceIn `gen_initial_workspace/script_initial_workspace.sh`, before anything else is done, it is checked if `$CONFDATA/ID_FLIPPEDSWI.txt` has been created, and if not, a serial computation is called to create this file with `./gen_FLIPPEDSWI/...In `gen_initial_workspace/script_initial_workspace.sh`, before anything else is done, it is checked if `$CONFDATA/ID_FLIPPEDSWI.txt` has been created, and if not, a serial computation is called to create this file with `./gen_FLIPPEDSWI/script_FLIPPEDSWI_serial.sh`. However, the script is called directly without switching directories, resulting in a failure to find `./script_FLIPPEDSWI_subject.sh` when called within `script_FLIPPEDSWI_serial.sh`
This PR adds the needed change of directories to let this run.https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/merge_requests/8Union of all TN modifications2021-04-13T17:24:00+01:00Tom NicholsUnion of all TN modificationsI have tried to fastideously create a series of seperate MRs (merge requests) that isolate different changes. In the end, however, you may not want to bother with each individual set of changes. This MR encompasses all of the other MRs...I have tried to fastideously create a series of seperate MRs (merge requests) that isolate different changes. In the end, however, you may not want to bother with each individual set of changes. This MR encompasses all of the other MRs as follows:
- !1 Amend wrapper_funpack
- !2 Fix `generate_initial_data/README.md` formatting
- !3 Fix wrapper_funpack.sh
- !4 Update `generate_initial_data/README.md` (`bash_scripts`)
- !5 Add files to `bash_scripts`
- !6 Add absolute paths for FSL commands
- !7 Various changes
- !9 Fix call of `script_FLIPPEDSWI_serial.sh` from `script_initial_workspace.sh`
To be clear... *this* MR supercedes all the above MRs... if this MR is merged, the above MRs can be closed.https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/merge_requests/7Other changes needed to make a runnable Rescomp version2021-03-04T08:34:50+00:00Tom NicholsOther changes needed to make a runnable Rescomp versionWhile I have made various separate isolated Merge Requests, I eventually gave up and created this catch-all MR of various edits needed make a runnable version of the ukb_unconfound_v2 pipeline.While I have made various separate isolated Merge Requests, I eventually gave up and created this catch-all MR of various edits needed make a runnable version of the ukb_unconfound_v2 pipeline.https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/merge_requests/6Add absolute path to FSL commands2021-02-27T20:09:21+00:00Tom NicholsAdd absolute path to FSL commandsInspired by problems in `generate_initial_data/gen_STRUCTMOTION/script_STRUCTMOTION_subject.sh`, where multiple active versions of FSL lead to an old version of `smoothest` creating problems, I've gone through and looked for all instance...Inspired by problems in `generate_initial_data/gen_STRUCTMOTION/script_STRUCTMOTION_subject.sh`, where multiple active versions of FSL lead to an old version of `smoothest` creating problems, I've gone through and looked for all instances where FSL commands were called and added `$FSLDIR/bin/` to make the paths absolute.
The notable exception to this is (1) `fsl_sub`... it's own evil beast, I didn't want to touch that, and (2) places in Matlab code that I saw call FSL tools with absolute paths... just not sure enough to fix thoses.https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/merge_requests/5Add files to bash_scripts2021-02-27T20:09:20+00:00Tom NicholsAdd files to bash_scriptsAdd missing file `bool_files.sh` to `bash_scripts/` from `/well/win/projects/ukbiobank/scripts`Add missing file `bool_files.sh` to `bash_scripts/` from `/well/win/projects/ukbiobank/scripts`https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/merge_requests/4Update READMEmd: sh_utils -> bash_scripts2021-02-27T20:09:20+00:00Tom NicholsUpdate READMEmd: sh_utils -> bash_scriptsFix name of `bash_scripts` directory.Fix name of `bash_scripts` directory.https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/merge_requests/3Fix wrapper_funpack.sh [WIP]2021-02-27T20:09:20+00:00Tom NicholsFix wrapper_funpack.sh [WIP]Script `wrapper_funpack.sh` references nonexistant python environment with activation:
```
. $BB_BIN_DIR/bb_python/bb_python_funpack/bin/activate
```
Also, it appears that this script now no longer works with the current version of `fsl...Script `wrapper_funpack.sh` references nonexistant python environment with activation:
```
. $BB_BIN_DIR/bb_python/bb_python_funpack/bin/activate
```
Also, it appears that this script now no longer works with the current version of `fsl_sub` on Rescomp, as the environment is no longer automatically forwarded.
I'm currently chasing this down with WIN computing help... don't merge.https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/merge_requests/2Update generate_initial_data/README formating2021-02-27T20:09:20+00:00Tom NicholsUpdate generate_initial_data/README formatingPresently the crucial commands at the end of the https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/raw/master/generate_initial_data/README.md aren't treated as preformatted, and so all the commands are combined in a jumble. This P...Presently the crucial commands at the end of the https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/raw/master/generate_initial_data/README.md aren't treated as preformatted, and so all the commands are combined in a jumble. This PR makes these commands preformatted. (Also, one redirect is missing from the first line; that's fixed in the PR (but not in the example below).
E.g. compare
cat $subjDir/subj.txt | grep "^2" $CONFDATA/subj.txt
cat $CONFDATA/subj.txt | awk -F "" '{print $2$3$4$5$6$7$8}' > $CONFDATA/subj_2.txt
bool_files.sh $CONFDATA/subj.txt $SUBJECTS_DIR/subj.txt
cp A_and_B.txt $CONFDATA/FS_subj.txt
rm A_*txt B_not_A.txt
to
```
cat $subjDir/subj.txt | grep "^2" $CONFDATA/subj.txt
cat $CONFDATA/subj.txt | awk -F "" '{print $2$3$4$5$6$7$8}' > $CONFDATA/subj_2.txt
bool_files.sh $CONFDATA/subj.txt $SUBJECTS_DIR/subj.txt
cp A_and_B.txt $CONFDATA/FS_subj.txt
rm A_*txt B_not_A.txt
```https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/merge_requests/1Amend wrapper_funpack to match README2021-02-27T20:09:20+00:00Tom NicholsAmend wrapper_funpack to match READMEPresently [`generate_inital_data/README.md`](https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/blob/master/generate_initial_data/README.md) directs the user to set env var `NONIDPDATA` with a path to UKB csv files with tabular data...Presently [`generate_inital_data/README.md`](https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/blob/master/generate_initial_data/README.md) directs the user to set env var `NONIDPDATA` with a path to UKB csv files with tabular data. However, there are no references to `$NONIDPDATA` anywhere in the code, yet there is a `$input1` in [`wrapper_funpack.sh`](https://git.fmrib.ox.ac.uk/falmagro/ukb_unconfound_v2/-/blob/master/generate_initial_data/gen_nonIDPs/wrapper_funpack.sh) that is asking to be set to what is `$NONIDPDATA`.
This impliments that change.