Brain Imaging & Analysis Center

usage: batch_script datasetlist command [outname] [flag] [nnodes] [exptname]

BATCH_SCRIPT reads in a table of data set pointers and then creates a shell command file to run a standard processing script on each data set.

Its arguments are:

DATASETLIST – file containing a list of data sets to be processed.
Lines starting with a '#' are treated as comments and ignored.
If the second non-comment line is blank, the previous line is assumed to be a table header and is ignored.
All other lines are assumed to be a table of the form:

 DATALOC   DATADIR   WDFILE   DATASET  ARG1 ARG2 ARG3
  where:
    DATALOC - typically a BIAC experiment or pseudo-experiment name.
        Data files must be in the path pointed to by 'findexp DATALOC'.
    DATADIR - the directory path from there to the data files themselves.
    WDFILE - the workspace file, or it can be 0 if there is no WDF.
    DATASET - the name of a particular data set file to be processed.
    ARG1, ARG2, and ARG3 - optional arguments that may be used by the 
        particular command script being used (MACROSCRIPT).

COMMAND – the program or script to be executed in processing the named DATASETs.
If COMMAND includes a partial directory path it is assumed to be relative to the current directory (e.g. ../Analysis/Scripts/myscript).
If COMMAND starts with '/' it is a full path used as is.
Otherwise, COMMAND is assumed to be in an fScan EXEC directory. This will search the 3 standard fScan directory locations:

• a directory named “fscanhome” anywhere in the path above the current directory
• in your own home directory (e.g. /home/USER/fscanhome/EXEC)
• fScan's system files (e.g. /usr/local/packages/jvs/lib/FSCAN2/EXEC)

Note:If COMMAND calls any other files they must be readable from every sub directory that will be used for running COMMAND.

OUTNAME – name to use for the temporary batch file and output file. The default OUTNAME is 'jbatch'.

FLAG – batch processing options (combine options by adding flag values):

• 0 - default (create batch file and execute it)
• 1 - create batch file, but do not execute it.
• 2 - use QSUB to submit batch commands to Hugin cluster.
• 4 - Get DATALOC and root DATADIR from current path
• 8 - COMMAND is a standard shell command (don't look for script file)
• 16 - The first column of DATASETLIST is the path relative to current dir. The batch job will change to that directory before running COMMAND.
• 32 - Add DATALOC and DATADIR at front of each line in DLIST (with FLAG 4)
• 64 - Verbose mode
• 128 - omit actual COMMAND commands from qsub batch files (for testing)

NNODES – number of batch scripts to vreate for processing (default=1).
If NNODES >= 1, the list of datasets will be split into NNODES batch files.
If NNODES < 0, each dataset will be processed in a separate job.
If NNODES == 0 and FLAG&2 then all datasets will be sent to 1 node.

EXPTNAME – Experiment name for cluster batch scripts (ignored if not QSUB job).
If omitted, the experiment name will be extracted from the current data path (if FLAG 4), or it will be extracted as the DATALOC entry on the last line of the DATSETLIST file.

The batch processing job file will be named OUTNAME.SFX if NNODES is 1, or OUTNAME_X.SFX if submitting multiple NNODES. The SFX indicates the script type (e.g. csh, sh, perl).

BATCH_SCRIPT records each batch session in “batch_log.txt”.
QSUB job files will each generate their own log files (e.g. OUTNAME_X.SFX.jobnum.out).