Skip to content

MNT: Ignore additional messages emitted from qsub when scraping job ID

Paul McCarthy requested to merge mnt/qsub-output into master

Hi @duncan, Fidel and I just came across a situation where qsub is producing unexpected standard output when a job is submitted, e.g.:

[mza328@rescomp1 ~]$ LD_GOLD="what" qsub -v LD_GOLD -b y -q short.qc /bin/ls
removed environment variable LD_GOLD from submit environment - it is considered a security issue
Your job 81196363 ("ls") has been submitted

When run via fsl_sub this was resulting in an error of the form:

Traceback (most recent call last):
  File "/well/win/projects/ukbiobank/fbp/SWdir/fsl/envs/ukb/lib/python3.11/site-packages/fsl_sub_plugin_sge/__init__.py", line 733, in submit
    job_id = int(job_words[2].split('.')[0])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: 'variable'

During handling of the above exception, another exception occurred:

  File "/well/win/projects/ukbiobank/fbp/SWdir/fsl/envs/ukb/lib/python3.11/site-packages/fsl_sub/__init__.py", line 568, in submit
    job_id = queue_submit(
             ^^^^^^^^^^^^^
  File "/well/win/projects/ukbiobank/fbp/SWdir/fsl/envs/ukb/lib/python3.11/site-packages/fsl_sub_plugin_sge/__init__.py", line 735, in submit
    raise GridOutputError("Grid output was " + result.stdout)
fsl_sub.exceptions.GridOutputError: Grid output was removed environment variable LD_GOLD from submit environment - it is considered a security issue
Your job 81196271 ("struct_T1.T1_gdc_dummy_39") has been submitted

(this particular trace was from a programmatic invocation of fsl_sub.submit).

This MR proposes a small change to the job ID scraping logic in fsl_sub_plugin_sge.submit to work around this crash.

Thanks!

Merge request reports