User Tools

Site Tools


JVS Utilities: DMREAD

DMREAD -- Read data management (DM) archive file

usage: dmread infile [recflg outfile options]

DMREAD reads data archive files stored in the data management (DM) format used by CIGAL and fScan.
These files can contain multiple records of different types, containing either text or binary data. Individual records can be accessed either by record number or by record 'type'. The record type was defined when the data were added to the archive file; usually each kind of record in an archive will be assigned a unique record type. Record types allow individual records to be found based on the kind of data, regardless of what order the records are stored within the archive. Record numbers allow records to be read based on their sequential position in the archive, regardless of the kind of data they contain. Record numbers are also the preferred access method if you need to select from multiple records with the same type flag.

Each archive file contains an internal header table of contents. This header is a text record (usually 4096 bytes long), which indicates record type, data type (text, binary integers, binary floating point), data dimensions, and a short data description. The archive file can also have an external header file in XML format (based on the BXH/XCEDE schema “Datarec” element) that provides the same content information as the internal header.

Command arguments

The only command argument required by DMREAD is the name of the archive file to read.
If no other arguments are specified, DMREAD will display the archive
table of contents, obtained by reading the file header.

To read any particular record, a RECFLG number argument must be specified.
RECFLG can be:

  > 0 - The record type, defined when the record was written 
  < 0 - The negative record number (1st record is 1) 

The following OPTION arguments are recognized:

  1. s Convert time values to secs when listing RECFLG == 3.
  2. R recflg Display 2nd record, RECFLG, as second column of data (assumes -C 1).
  3. t time0 Subtract TIME0 (20 usec ticks) from all time values when

using -s option. Default is to get 'ltime0' parameter value from record 2.

  1. X Use XML format
  2. C N Print data in N columns
  3. z Don't display zero values.
  4. h Print header as separate file, pointing to data
  5. H Print header without reformatting
  6. mN val Only print out data records where the Nth value matches VAL

If VAL is negative, only print if the Nth value is not -VAL

            (N = 1-4). This only applies if RECFLG == 3. Up to 4 -m 
            options can be combined at a time. 
-dump nch  Dump text strings longer than NCH chars (NCH default = 8) 
-dump a n  Dump N bytes starting at address A, to OUTFILE 
-a         Print PTR and Size offsets with DUMP option 
-D         Don't assume CIGAL formats for RECFLG==3. 
-O         Display old format files with old header format 
-vN        Verbose flag -- print diagnostic messages. 


Assuming you have a DM archive file named “pdigmfile.xdm”, you could look at the archive's table of contents using:

dmread pdigmfile.xdm 

Read text record

To extract a text record such as the parameter settings in record type 2, enter:

dmread pdigmfile.xdm 2                       ; to view the record 
dmread pdigmfile.xdm 2 > pdigmrec2.txt       ; to store the record in a file 

Read CIGAL run log

For CIGAL “pdigm” output files the run-log is stored as record type 3. The run-log has 2 32-bit words per event, which contains the event time (in 20 us clock ticks) in the first word, then an 8-bit event code, an 8-bit user-defined flag, and then a 16-bit data value. DMREAD automatically recognizes record type 3 as this special 8-byte data format (unless you specify the -D option) when displaying run-log records. For example,

dmread pdigmfile.xdm 3                      ; display the log's 4 values per event 
dmread pdigmfile.xdm 3 -s                   ; display events with time in secs relative to task start 
dmread pdigmfile.xdm 3 -s -N                ; display events in secs with event codes named 
dmread pdigmfile.xdm 3 -s -N -m1 95 -m2 6   ; display events where code is 95 and flag is 6 
                                            ;   (these are subject responses in Showplay) 

Read CIGAL physiological data

Binary time course data are typically stored by CIGAL with time and data values in separate records (because multiple data channels may be sampled at each time point). To display analog time course data (e.g. subject respiration data) try:

dmread pdigmfile.xdm 7 -s -R 9              ; display time, rec type 7 (in task secs), and resp, rec type 9  

Common data record types

CIGAL and fScan use consistent record type flags to identify standard data records. For CIGAL, the standard record types are as follows (T= text, B= binary):

 1 T - a copy of the CIGAL script (usually Showplay.imp) that produced the pdigm file  
 2 T - the parameter variable settings (and other notes) used to produce this file 
 3 B - the real-time run-log (see notes above) 
 5 T - task timing information (image list format) 
 6 T - comment text 
 7 B - physiology time points 
 8 B - physiology cardiac values 
 9 B - physiology respiratory values 
10 B - physiology galvanic skin resistance (GSR) values 
11,12 B - more analog data 
13 B - eyetracker time points 
14 B - eyetracker X values 
15 B - eyetracker Y values 
16,17 B - more eyetracker values  
21 T - the Showplay .out output file 
22 T - the Showplay input file (user PPF) 
24 B - subject response codes  
25 B - subject response times 
27 B - times of stimuli expecting responses 
28 B - user_id of stimuli expecting responses 
29 T - realtime.tmp file used to actually run this paradigm 
30 B - stimulus onset order 
31 B - nominal stimulus start times 
32 B - actual stimulus start times 
33 B - stimulus offset order 
34 B - nominal stimulus stop times 
36 B - stimulus type codes 
37 B - stimulus ID (user-defined) codes 
38 B - stimulus flags 
39 B - stimulus response times 
40 B - stimulus response keys 
41 B - stimulus parameter table 
50-56 T - Showplay paradigm code modules used 

Not all record types shown will be present for every paradigm file created, and many other record types may be included. In general, the header table of contents should help document the actual file content.
The standard record type numbers are most useful for automated processing of DM archive data.

Extracting the header record

DMREAD files use a text format header, which can precede the data or
can have the data in a separate file. If the header is in the same file
as the data it can be separated from the data using:

dmread pdigmfile.xdm -h > hdrfile.txt 
dmread pdigmfile.xdm -h -X > hdrfile.bxh 

This HDRFILE can then be used to reference the data (i.e. as INFILE).
The header record can also be extracted for editting (although this is not recommended) using:

dmread pdigmfile.xdm  -H > hdrfile 

After editting, HDRFILE can be reinserted in the original data file using:

dmwrite pdigmfile.xdm hdrfile -h 

provided the total length of HDRFILE doesn't exceed the byte count found
on its first line (usually 4096).

See Also:

jvs/utilities/dmread.txt · Last modified: 2014/08/04 16:03 (external edit)