usage: batch_script datasetlist command [outname] [flag] [nnodes] [exptname]
BATCH_SCRIPT reads in a table of data set pointers and then creates a
shell command file to run a standard processing script on each data set.
Its arguments are:
DATASETLIST – file containing a list of data sets to be processed.
Lines starting with a '#' are treated as comments and ignored.
If the second non-comment line is blank, the previous line is assumed to be a table header and is ignored.
All other lines are assumed to be a table of the form:
DATALOC DATADIR WDFILE DATASET ARG1 ARG2 ARG3 where: DATALOC - typically a BIAC experiment or pseudo-experiment name. Data files must be in the path pointed to by 'findexp DATALOC'. DATADIR - the directory path from there to the data files themselves. WDFILE - the workspace file, or it can be 0 if there is no WDF. DATASET - the name of a particular data set file to be processed. ARG1, ARG2, and ARG3 - optional arguments that may be used by the particular command script being used (MACROSCRIPT).
COMMAND – the program or script to be executed in processing the named DATASETs.
If COMMAND includes a partial directory path it is assumed to be relative to the current directory (e.g. ../Analysis/Scripts/myscript).
If COMMAND starts with '/' it is a full path used as is.
Otherwise, COMMAND is assumed to be in an fScan EXEC directory. This will search the 3 standard fScan directory locations:
Note:If COMMAND calls any other files they must be readable from every
sub directory that will be used for running COMMAND.
OUTNAME – name to use for the temporary batch file and output file.
The default OUTNAME is 'jbatch'.
FLAG – batch processing options (combine options by adding flag values):
NNODES – number of batch scripts to vreate for processing (default=1).
If NNODES >= 1, the list of datasets will be split into NNODES batch files.
If NNODES < 0, each dataset will be processed in a separate job.
If NNODES == 0 and FLAG&2 then all datasets will be sent to 1 node.
EXPTNAME – Experiment name for cluster batch scripts (ignored if not QSUB job).
If omitted, the experiment name will be extracted from the current data path (if FLAG 4), or it will be extracted as the DATALOC entry on the last line of the DATSETLIST file.
The batch processing job file will be named OUTNAME.SFX if NNODES is 1, or OUTNAME_X.SFX if submitting multiple NNODES. The SFX indicates the script type (e.g. csh, sh, perl).
BATCH_SCRIPT records each batch session in “batch_log.txt”.
QSUB job files will each generate their own log files (e.g. OUTNAME_X.SFX.jobnum.out).