The IDAMS Setup File

3 The IDAMS Setup File

3.1 Contents and Purpose

To execute IDAMS programs, the user prepares a special file called the "Setup" file which controls the execution of the programs. This file contains IDAMS commands and control statements necessary for execution such as: reference to program to be executed, the names of files, the options to be selected for the program and variable transformation instructions, e.g.


     $RUN program name
     $FILES
         file specifications
     $SETUP
         program control statements
     $RECODE
         Recode statements

3.2 IDAMS Commands

These commands, which start with a "$", separate the different kind of information being provided for an IDAMS program execution. Available commands are:

$RUN program (name of program to be executed)
$FILES [RESET] (signals start of file specifications)
$RECODE (signals start of Recode statements)
$SETUP (signals start of program control statements)
$DICT (signals start of dictionary)
$DATA (signals start of data)
$MATRIX (signals start of a matrix)
$PRINT (turns printing on and off)
$COMMENT [text] (comments)
$CHECK [n] (checking if previous step terminated well).

The first line in a Setup file must always be a $RUN command identifying the IDAMS program to be executed. Other commands relating to this program execution (followed by associated control statements or data) can be placed in any order. These are then followed by the $RUN command for the next program (if any) to be executed and so on. The individual IDAMS commands are described below in alphabetical order.

$CHECK [n]. If this command is present, the program will not be executed if the immediately preceding program terminated with a condition code greater than n. If the command is present, but no value is supplied, the value of n defaults to 1.

All IDAMS programs terminate with a condition code of 16 if setup errors are encountered. For example, if TABLES is to be executed immediately after TRANS, but the user does not want to execute TABLES if a setup error occurred in the TRANS execution, a $CHECK command after the $RUN TABLES command will prevent execution of TABLES.
The $CHECK command may appear anywhere in the setup for the program, but is usually placed immediately after the $RUN command.

$COMMENT [text]. The "text" from this command is printed in the listing of the setup. This command has no effect on program execution.

$DATA. The $DATA command signals that the data follow.

This feature cannot be used if the program generates an output Data file and a DATAOUT file is not specified, i.e. the output data is allowed to default to a temporary file.
This feature cannot be used if the $MATRIX feature is used.
The record length of data in the setup cannot exceed 80 characters. If longer records or lines are input, only the first 80 characters will be used.
The print switch is turned off by the $DATA command. Thus, unless a $PRINT command immediately follows the $DATA command, the data will not be printed.

$DICT. The $DICT command signals that an IDAMS dictionary follows.

This feature cannot be used if the program generates an output dictionary and a DICTOUT file is not specified, i.e. if the output dictionary is allowed to default to a temporary file.
The print switch is turned off by the $DICT command. Thus, unless a $PRINT command immediately follows the $DICT command, the dictionary will not be printed.

$FILES [RESET]. This signals the start of file specifications. Default file names are attached to each file at the start of IDAMS program(s) execution through the use of a special file "idams.def". Any of these default names may be changed by introducing file specification statements after the $FILES command (see "File Specifications" below). To get back default file names for Fortran FT files (except FT06 and FT50), use "FILES RESET" command.

$MATRIX. The $MATRIX command signals that a matrix or set of matrices follows.

This feature cannot be used if the $DATA feature is used.
The print switch is turned off by the $MATRIX command. Thus, unless a $PRINT command immediately follows the $MATRIX command, the matrix input will not be printed.

$PRINT. The print switch is reversed; if it was on, $PRINT will turn it off; if it was off, $PRINT will turn it on. When printing is on, lines from the Setup file are listed as part of the program results.

When a $RUN command is encountered, the print switch is always turned on. The $DICT, $DATA, and $MATRIX commands automatically turn the print switch off.

$RECODE. The occurrence of this command signals that the IDAMS Recode facility is to be used. The Recode facility is described in the "Recode Facility" chapter of this manual.

The Recode statements normally follow the $RECODE command. If a new IDAMS command follows immediately after a $RECODE command, Recode statements from the setup for the preceding program will be used.

$RUN program. $RUN specifies the program to be executed and always is the first statement in the setup.

"program" is the 1 to 8 character name of the program.
All commands and statements following the $RUN command and up to the next $RUN command apply to the program named.
The print switch is turned on when $RUN is encountered. See the $PRINT description.

$SETUP. The $SETUP command signals the beginning of the program control statements, i.e. the filter, label, parameter statement, etc. (see below).

The $SETUP command is required even when program control statements follow immediately after the $RUN command.

3.3 File Specifications

The names of the files to be used are given following the $FILES command and take the following format:

ddname=filename [RECL=maximum record length] where:

ddname is the file reference name used internally by programs, e.g. DICTIN. The required files and the corresponding ddnames for a particular program are given in the program write-up in the section "Setup Structure".
filename is the physical file name. Enclose the name in primes if it contains blanks. See section "Folders in WinIDAMS" for additional explanation.
RECL must be used if the first record in a Data file is not the longest. If RECL is not specified the record length is taken as the record length of the first record. If a subsequent record is longer, an input error results.

Examples:


     DATAIN = A:ECON.DAT RECL=92
     PRINT  = RSLTS.LST
     FT02   = ECON.MAT
     DICTIN = \\nec0102\commondata\econ.dic

3.4 Examples of Use of $ Commands and File Specifications

Example A. Perform multiple executions of an analysis program, e.g. ONEWAY using the same data but with, for instance, different filters.


     $RUN ONEWAY
     $FILES
     DICTIN = CHEESE.DIC
     DATAIN = CHEESE.DAT
     $SETUP
          Filter 1
          Other control statements for ONEWAY
     $RUN ONEWAY
     $SETUP
          Filter 2
          Other control statements for ONEWAY

Example B. Execute TABLES and ONEWAY, using the same Dictionary and Data files for each and using the same Recode; do not list the Recode statements.


     $RUN TABLES
     $FILES
     DICTIN = ABC.DIC
     DATAIN = ABC.DAT  RECL=232
     $SETUP
          Control statements for TABLES
     $RECODE
     $PRINT
          Recode statements
     $RUN ONEWAY
     $SETUP
          Control statements for ONEWAY
     $RECODE
     $COMMENT THE RECODE STATEMENTS INPUT FOR TABLES WILL BE REUSED FOR ONEWAY

Example C. Execute TABLES using IDAMS Recode, dictionary in the setup, data on diskette. Print the input dictionary.


     $RUN TABLES
     $FILES
     DATAIN = A:MYDATA
     $RECODE
          Recode statements
     $SETUP
          Control statements for TABLES
     $DICT
     $PRINT
          Dictionary

Example D. Use the output from a data management program as input to analysis programs without retaining the output file, e.g. execute TRANS followed by TABLES using the output data from TRANS by specifying parameter INFILE=OUT. TABLES is not to be executed if the TRANS has control statement errors.


     $RUN TRANS
     $FILES
     DICTIN = MYDIC4
     DATAIN = MYDAT4
     $SETUP
          Control statements for TRANS
     $RECODE
          Recode statements
     $RUN TABLES
     $CHECK
     $SETUP
          Control statements for TABLES including parameter INFILE=OUT

3.5 Program Control Statements

3.5.1 General Description

IDAMS program control statements (which follow the $SETUP command) are used to specify the parameters for a particular execution. There are three standard control statements used by all programs:

the optional filter statement for selecting the cases from the data file to be used,
the mandatory label statement which assigns a title for the execution,
a mandatory parameter statement which selects the options for the program; some program options are standard across most programs, others are program specific.

Additional program control statements required by individual programs are described in the program write-up.

3.5.2 General Coding Rules

Control statements are entered on lines up to 255 characters long.
Lines may be continued by entering a dash at the end of one line and continuing on the next.
The maximum length of information that may be entered for one control statement is 1024 characters excluding the continuation characters.
Lower case letters, except for those occurring in strings enclosed in primes, are converted to upper case.
If character strings enclosed in primes are included on a control statement, these should be continued in one line.

3.5.3 Filters

Purpose. A filter statement is used to select a subset of data cases. It is expressed in terms of variables and the values assumed by those variables. For example, if variable V5 indicates "sex of respondent" in a survey and code 1 represents female, then "INCLUDE V5=1" is a filter statement which specifies female respondents as the desired subset of cases.

The main filter selects cases from an input Data file and applies throughout a program execution. These filters are available with all IDAMS programs which input a dictionary (except BUILD and SORMER). Some programs allow for additional subsetting. Such "local" filtering applies only to a specific program action, e.g. one frequency table.

Examples.


     1. INCLUDE V2=1-5 AND V7=23,27,35 AND V8=1,2,3,6
     2. EXCLUDE V10=2-3,6,8-9 AND V30=<5 OR -
           V91=25
     3. INCLUDE V50='FRAN','UK','MORO','INDI'

Placement. If a main filter is used, it is always the first program control statement. Each program write-up indicates whether "local" filters may also be used.

Rules for coding.

The filter statement begins with the word INCLUDE or EXCLUDE. Depending on which word is given, the filter statement defines the subset of cases to be used by the program (INCLUDE) or the subset to be ignored (EXCLUDE).
A statement may contain a maximum of 15 expressions. An expression consists of a variable number, an equals sign, and a list of possible values. The list of values can contain individual values and/or ranges of values separated by commas, e.g. V2=1,5-9. Open ended ranges are indicated by < or >, e.g. INCLUDE V1=0,3-5,>10; however the variable must always be followed by an = sign to begin with, e.g. V1>0 must be expressed V1=>0 and V1<0 as V1=<0.
Expressions are connected by the conjunctions AND and OR.
- AND indicates that a value from each of the series of expressions connected by AND must be found.
- OR indicates that a value from at least one of a series of expressions connected by OR must be found.
Expressions connected by AND are evaluated before expressions connected by OR. For example, "expression-1 OR expression-2 AND expression-3" is interpreted as "expression-1 OR (expression-2 AND expression-3)". Thus, in order for a case to be in the subset defined by these expressions, either a value from expression-1 occurs, values from both expression-2 and expression-3 occur, or a value from each of the three expressions occurs.
Parentheses cannot be used in the filter statement to indicate precedence of expression evaluation.
Variables may appear in any order and in more than one expression. However, note that "V1=1 OR V1=2" is equivalent to the single expression "V1=1,2". Note also that "V1=1 AND V1=2" is an impossible condition, as no single case can have both a '1' and a '2' as a value for variable V1.
A filter statement may optionally be terminated by an asterisk.
The variables in a filter.
- Numeric and alphabetic character type variables can be used.
- R-variables are not allowed in main filters. They are allowed in analysis specific or local filters. Note that the REJECT statement in Recode can be used to filter cases on R-variables.
The values in a filter for numeric variables.
- Numeric values may be integer or decimal, positive or negative, e.g. 1, 2.4, -10.
- Values are expressed singly or in ranges and are separated by commas, e.g. 1-5, 8, 12-13.
- For numeric filter variables, variable values in the data file are first converted to real binary mode using the correct number of decimal places from the dictionary and the comparison with the filter value is then done numerically. Note that this means that for a variable with decimals, filter values must be given with the decimal point in the correct place, e.g. V2=2.5-2.8.
- Cases for which a filter variable has a non-numeric value are always excluded from the execution.
The values in a filter for alphabetic variables.
- Values of 1-4 characters are expressed as character strings enclosed in primes, e.g. 'F'. Blanks on the right need not be entered, i.e. trailing blanks will be added.
- If the variable has a field width greater then 4, only the first 4 characters from the data are used for the comparison with the filter variable.
- Only single values, separated by commas are allowed; ranges of character strings cannot be used.

Note. The first statement following a $SETUP command is recognized as a main filter if it starts with INCLUDE or EXCLUDE. If the first non-blank characters are anything else, the statement is assumed to be a label.

3.5.4 Labels

Purpose. A label statement is used to title the results of a program execution. Some IDAMS programs print this label once at the start of the results, while others use it to title each page.

Examples.


     1. TABLES ON 1998 ELECTION DATA - JULY, 2000
     2. PRINTING OF CORRECTED A34 SURVEY DATA

Placement. A label statement is required by all IDAMS programs. The label is either the first or (if a filter is used), the second program control statement. If no special labeling is desired, it is still necessary to include a blank line.

Rules for coding.

The statement may be a string of any characters from which the first 80 characters are used, i.e. if a label longer than 80 characters is input, it is truncated to the first 80.
If the label is not enclosed in primes, lower case letters are converted to upper case and blanks are reduced to one blank.
The label should not begin with the words "INCLUDE" or "EXCLUDE".

3.5.5 Parameters

Purpose. All IDAMS programs have been designed in a fairly general way, allowing the user to select among several options. These options and values are generated by parameters and are supplied on program control statements, such as "parameters", "regression specifications", "table specifications", etc. Parameters are specified by the user in a standard keyword format with an English word or abbreviation being used to identify an option.

Examples.


     1. WRITE=CORR WEIGHT=V3, PRINT=(DICT, PAIR)
        (PEARSON - parameters)
     2. DEPV=V5 METHOD=STEP VARS=(R3-R9,V30) WRITE=RESID
        (REGRESSN - regression parameters)
     3. ROWV=(V3,V9,V10) COLV=(V4,V11,V19) CELLS=(FREQ,ROWPCT) STATS=(CHI,TAUA)
        (TABLES - table description)

Placement. The main parameter statement is required by all IDAMS programs and it must follow the label statement. If all defaults are chosen, a line with a single asterisk must be supplied. Each program write-up indicates the type and content of any other parameter lists that are required and indicates their position relative to other program control statements.

Presentation of keyword parameters in the program write-ups. All write-ups have a standard notation in the sections which describe the program parameters which are available. The basic notation is as follows:

A slash indicates that only one of the mutually exclusive items can be chosen, e.g. SAMPLE/POPUL or PRINT=CDICT/DICT.
A comma indicates that all, some, or none of the items may be chosen, e.g. STATS=(TAUA, TAUB, GAMMA).
When commas and slashes are combined, only one (or none) of the items from each group separated by commas and connected by slashes may be chosen, e.g. PRINT=(CDICT/DICT, LONG/SHORT).
Defaults, if any, are in bold, e.g. METHOD=STANDARD /STEPWISE/DESCENDING. A default is a parameter setting that the program assumes if an explicit selection is not made by the user.
When a parameter setting is obligatory but has no default, the words "No default" are used.
Words in upper case are keywords. Words or phrases in lower case indicate that the user should replace the word or phrase with an appropriate value, e.g. MAXCASES=n, VARS=(variable list).

Types of keywords. There are 5 types of keywords used for specifying parameters.

A keyword followed by a character string. This type of keyword identifies a parameter consisting of a string of characters, e.g.
INFILE=IN /xxxx

A 1-4 character ddname suffix for the input dictionary and data files.

A user might specify:
INFILE=IN2

(the ddnames would be DICTIN2 and DATAIN2)
A keyword followed by one or more variable numbers, e.g.
WEIGHT=variable number

The weight variable number if the data are to be weighted.
VARS=(variable list)

Use only the variables in the list; the numbers may be listed in any order with or without V-notation, i.e. VARS=(V1-V3) or VARS=(1-3). Note that the program write-ups always indicate whether V- and R-type variables or only V-type variables may be used.
A user might specify:
WEIGHT=V39

(the weight variable is V39)
VARS=(32,1,10)

(only the variables specified are to be used)
A keyword followed by one or more numeric values, e.g.
MAXCASES=n

Only the first n cases will be processed.
IDLOC=(s1,e1,s2,e2, ...)

Starting and ending columns of 1-5 case identification fields.
A user might specify:
MAXCASES=100

(only the first 100 cases will be used)
IDLOC=(1,3,7,9)

(case ID is located in columns 1-3 and 7-9)
A keyword followed by one or more keyword values. The keyword values may be a mixture of mutually exclusive options (separated by slashes) and independent options (separated by commas). For example:
PRINT=(OUTDICT /OUTCDICT/NOOUTDICT,DATA)

OUTD
Print the output dictionary without C-records.
OUTC
Print the output dictionary with C-records if any.
NOOU
Do not print output dictionary.
DATA
Print the values of the output variables.

A user might specify:
PRINT=(OUTC,DATA)

(full output dictionary is printed, and data values are printed)
PRINT=NOOUTDICT

(no output dictionary or data values are printed)
A set of mutually exclusive keywords. Only one of a set of options can be selected, e.g.
SAMPLE /POPULATION

SAMP
Compute the variance and/or standard deviation using the sample equation.
POPU
Use the population equation.

All keywords except the last type are followed by an equals sign. The character, numeric, and keyword values that follow the equals sign are called the "associated values".

Rules for coding.

Rules for specifying keywords

Only the first four letters of a keyword or an associated keyword need to be specified, although the whole keyword may be supplied. Thus, "TRAN" is an appropriate abbreviated form of the keyword "TRANSVARS". There are no abbreviations for keywords with four letters or less.

Rules for specifying associated values

Associated value is a list of items.
- The items in the list are separated by commas.
- If there are two or more items, the list must be enclosed in parentheses.
- Ranges of integer numeric values or variables are indicated by a dash.
- Ranges of decimal numeric values are not allowed.
For example:
```
     R=(V2,3,5)
     PRIN=(DICT,DATA,STAT)
     MAXC=5
     TRAN=(V5,V10-V25,V32)
     IDLOC=(1,3,7,8)
```
Associated value is a character string.
- The string must be enclosed in primes if it contains any non-alphanumeric characters, e.g. FNAME='EDUCATION: WAVE 1'. Note that blank, dot and comma are non-alphanumeric characters. When in doubt, use primes.
- Two consecutive primes (not a quotation mark) must be used to represent a prime, e.g, ANAME='KEVIN"S' (the extra prime is deleted, once the string is read).
- A string is better not split across lines.

Rules for specifying lists of keywords

Keywords (with or without associated values) are separated from one another by a comma or by one or more blanks, e.g.
```
     FNAME='FRED', TRAN=3    KAISER
```
Lists of keywords may spread across several lines but in this case there must be a dash (-) at the end of each line indicating continuation, e.g.
```
     FNAME='FRED' -
     TRAN=3 -
     KAISER
```
Keywords may be given in any order. If a keyword appears more than once in the list, then the last value encountered is used.
A keyword may not be split across lines.
Each list of keywords may optionally be terminated by an asterisk.
If all default options are chosen, a line with a single asterisk must be supplied.

Details of most common parameters not described fully in each program write-up.

BADDATA. Treatment of non-numeric data values.
BADDATA=STOP /SKIP/MD1/MD2

When non-numeric characters (including embedded blanks and all-blank fields) are found in numeric variables, the program should:

STOP
Terminate the execution.
SKIP
Skip the case.
MD1
Replace non-numeric values by the first missing data code (or 1.5 * 10⁹ if 1st missing data code is not specified).
MD2
Replace non-numeric values by the second missing data code (or 1.6 * 10⁹ if 2nd missing data code is not specified).

For SKIP, MD1, and MD2 a message is printed about the number of cases so treated.
MAXCASES. The maximum number of cases to be processed.
MAXCASES=n

The value given is the maximum number of cases that will be processed. If n=0, no cases are read; this option can be used to test setups without reading the data. If the parameter is not specified at all, all cases from the input file are processed.
MDVALUES. Specify which, if either, of the missing data codes are to be used to check for missing data in variable values. Note that some programs have, in addition, a MDHANDLING parameter to specify how data values which are missing are to be handled.
MDVALUES=BOTH /MD1/MD2/NONE

BOTH
Variable values will be checked against the MD1 codes and against the ranges of codes defined by MD2.
MD1
Variable values will be checked only against the MD1 codes.
MD2
Variable values will be checked only against the ranges of codes defined by MD2.
NONE
MD codes will not be used. All data values will be considered valid.

The default is always that both MD codes are used.
INFILE, OUTFILE. Specifying ddnames with which input and output dictionary and data files are defined.
INFILE=IN /xxxx
OUTFILE=OUT /yyyy

Input and output Dictionary and Data files for IDAMS programs are defined with ddnames DICTxxxx, DATAxxxx, DICTyyyy and DATAyyyy. These normally default to DICTIN, DATAIN, DICTOUT, DATAOUT. If several IDAMS programs are being executed in one setup, for example programs using different datasets as input, or when using the output from one program as input directly to another (chaining), then it is sometimes necessary to change these defaults.
WEIGHT. This parameter specifies the variable whose values are to be used for weighting data cases.
WEIGHT=variable number

The variable specified may be a V-type or R-type, integer or decimal valued. Cases with missing, zero, negative and non-numeric weight values are always skipped and a message is printed about the number of cases so treated. If the WEIGHT parameter is not specified, no weighting is performed.
VARS. This parameter and similar ones such as ROWVARS, OUTVARS, CONVARS, etc. are used to specify a list of variables.
VARS=(variable list)

If more than one variable is specified, the list must be enclosed in parentheses.

Rules for specifying variable lists
- Variables are specified by a variable "number" preceded by a V or an R. A V denotes a variable from an IDAMS dataset or matrix. An R denotes a resultant variable from a Recode operation. Note that internal to the programs and in the results, V- and R-type variables are distinguished by the sign of the variable number; positive numbers denote V-type variables and negative numbers denote R-type variables.
- To specify a set of contiguously numbered variables, such as V3, V4, V5, V6, connect two variable numbers, each preceded by a V, with a dash (e.g. V3-V6 is valid; V3-6 is invalid). Use ranges with caution if the dataset has gaps in the variable numbering, as all variables within the range must appear in the dataset or matrix, i.e. V6-V8 implies V6,V7,V8. If V7 is not in the dictionary, then an error message will result. V-type and R-type variables may not be mixed in a range, i.e. V2-R5 is invalid.
- Single variable numbers or ranges of variable numbers are separated by commas.
- In general, for data management programs, variables may be listed more than once, while for analysis programs specifying a variable more than once is inappropriate and will cause termination. See the program write-up for details.
- Blanks may be inserted anywhere in the list.
- In general, variables may be specified in any order. The order of variables may, however, have special meaning in some programs; check the program write-up for details.
Examples:
```
     VARS=(V1-V6, V9, V16, V20-V102, V18, V11, V209)
     OUTVARS=(R104, V7, V10-V12, R100-R103,  -
          V16, V1)
     CONVARS=V10
```

3.6 Recode Statements

The IDAMS Recode facility permits the temporary recoding of data during execution of IDAMS programs. Results from such recoding operations (together with variables transferred from the input file) can also be saved in permanent files using the TRANS program.

Recoding is invoked by the $RECODE command. This command and the associated Recode statements are placed after the $RUN command for the program with which the Recode facility is to be used. For example:


     $RUN program                    $RUN ONEWAY
     $FILES                          $FILES
     File definitions                DICTIN=MYDIC
                                     DATAIN=MYDAT
     $RECODE                         $RECODE
     Recode statements                R10 = BRAC(V3,0-10=1,11-20=2)
                                      R11 = SUM(V7,V8)
                                      NAME R10 'EDUC LEVEL', R11'TOTAL INCOME'
     $SETUP                          $SETUP
     Program control statements       INCOME BY EDUC,SEX
                                      BADDATA=SKIP
                                      CONVARS=(R10,V2) DEPVAR=R11

A complete description of the Recode facility is provided in the "Recode Facility" chapter.

$RUN program	(name of program to be executed)
$FILES [RESET]	(signals start of file specifications)
$RECODE	(signals start of Recode statements)
$SETUP	(signals start of program control statements)
$DICT	(signals start of dictionary)
$DATA	(signals start of data)
$MATRIX	(signals start of a matrix)
$PRINT	(turns printing on and off)
$COMMENT [text]	(comments)
$CHECK [n]	(checking if previous step terminated well).