Practical Data Management and Statistical Computing (BioEp691F)

Contacts

Outline

Assignments

Resources

---

Outline: Lec11 Lec12 Lec13 lec14 Lec15 Lec16 Lec17 Lec18 Lec19 Lec20
Lectures: Lec11 Lec12 Lec13 Lec14 Lec15 Lec16 Lec17 Lec18 Lec19 Lec20

---

Lecture 12

DOS (disk operating system) commands can be used in a SAS program to create directories. This can be used to your advantage to expedite processing. The commands begin with a KEYWORD X, and have the DOS commands in quotation marks. For example, we create a sub-directory with lec10p2.sas.

When running a DOS command from SAS, type EXIT to continue running the SAS program.

• As newer versions of SAS are written, the SAS data structure for a SAS Dataset changes. The data structures are written by SAS Engines. Changes in the structure are identified by Engines, and Suffixes for the SAS data sets:
  • Data created by SAS Versions 6.03-6.04 (suffix: *.ssd) Engine: V604
  • Data created by SAS Versions 6.08-6.12 (suffix: *.sd2) and Default in Version 7: Engine: V612
    • Includes ability to create "index" variables
    • Includes ability to create "views" of data
• SAS automatically uses the most recent engine, or the engine for the most recently created SAS data set in the LIBREF if none is specified.
• If you want to read older SAS data sets, you need to specify the engine.
  • The engine can be specified as an option in a LIBNAME statement.
    • For *.ssd SAS datasets: LIBNAME V604 old4 'c:\data\';
    • For *.sd2 SAS datasets: LIBNAME V612 old12 'c:\data\';
• More Details

Examples

• An introduction:  engine1.sas which reads in ancns2.ssd (You need to rename it to this after the download)
• Determining the Default SAS engine when Multiple SAS Data files Exist in the PATH for a given LIBREF: engine2.sas
• Mixtures of engines for SAS data sets: We re-initialize SAS on the computer to eliminate all of the LIBREFs, copy the three data files to the C:\temp\old directory, and then run the program engine3.sas . This program creates a "default" engine for the LIBREF "oldu" when a mixture of SAS engines created the files on the c:\temp\old directory.

A common format for data is the ASCII format. This format can be created by a variety of software, including editors, wordprocessors, spreadsheets, and database programs. ASCII files are often called "FLAT" files since they contain no formatting characters. The suffix *.DTA , *.DAT, or *.TXT are commonly used to denote ASCII data files. (Note that *.DTA is also used for STATA system data sets, so should be avoided.)

Locating the ASCII data file. There are two ways that the SAS program can locate the data. The two ways are:

• 1. Follow the DATA step with a CARDS; or a DATALINES; statement, followed by the data.
  • If data contain a semi-colon, use a CARDS4 statement, and four semi-colons (;;;;). Example: LEC13P1.SAS
• 2. Identify the name of a file where the ASCII data are stored. The file is identified with an INFILE statement of a FILENAME statement specifying the name of the file and the file's location.
  • The INFILE statement reads data from an ASCII file specified by the statement. Although very long lines of data can be read, the default record length for the INFILE statement is 256 columns. When the number of columns in the ASCII file exceeds this number, a larger record length needs to be specified using the option LRECL=xxxx (where xxxx stands for the maximum record length). An example of such a data file is created by LEC13P2.SAS is given by LEC13P1.TXT. This program illustrates several new features:
    • FILENAME statement defining a fileref
    • use of SAS statement DATA _NULL_;
    • FILE statement with fileref
      • LRECL =option
    • ARRAY statement
    • DO i=1 TO 5; loop with END;
    • PUT statement
    • repeated format (v1-v5) (+8)
  • We illustrate reading the data with LEC13P3.SAS.
  • The FILENAME statement improves organization, since it doesn't have to be in the DATA step. We will also use a FILENAME statement to identify all files at the beginning of the SAS program. Example LEC13P4.SAS
  • If LRECL is omitted, SAS will continue trying to read additional variables on subsequent lines. Example LEC13P6.SAS. To avoid going to a new line, use a MISSOVER option on the FILE (or INFILE) statement.
    • FILE statement with MISSOVER option.

What are Engines?

SAS data sets are stored in a special format. Only software that can recognize the special format can read SAS data files. There are two principal types of SAS data sets for PC's. The first type is created by SAS for DOS using SAS version 6.03 or 6.04 and is characterized by files that have the suffix *.ssd. The second type is created by SAS for Windows using Windows-SAS Version 6.08 (or later) and is characterized by files that have the suffix *.sd2. The manner in which data are stored differs between the two types. As a result, the type of SAS data set has to be specified when reading a SAS data set.

The manner in which SAS data sets are stored in Windows-SAS differs from that in SAS for DOS. The data sets saved using the SAS version 6.08 format or higher have additional features that were not included in the DOS-SAS data sets. The additional features are designed to speed processing, and include

• "index" variables
• the ability to create "views"

We will not discuss these features here. DOS-SAS can not create data sets that have the Windows-SAS features. However, Windows-SAS can create DOS-SAS data sets. When reading a SAS data set, the SAS system needs to identify the type of SAS data set. It does so by identifying with the data set a SAS version, called an "engine".

There is an "engine" for each major change in data sets corresponding to a version of SAS. However, since the DOS versions 6.03 and 6.04 of SAS create identical data sets, and the Windows versions (6.08-6.12) create nearly identical data sets, we can distinguish between two principal engines (up to 1999). The Engines are:

• V603/V604
  • Equiv. to DOS-SAS 6.03 or SAS 6.04, with data stored as *.ssd
• V608/V610/V612
  • Equivalent to Windows-SAS 6.08/6.10 and stored as *.sd2

While the version of SAS used to create a SAS data set is needed to read in a SAS data file, SAS data files using earlier SAS versions can be created using newer SAS engines.

SAS data files with the suffix *.ssd or *.sd2 do not uniquely identify the engine that created the data file, but are sufficient for SAS to read the data set. Engines are identified by specifying the "engine" option in a LIBNAME statements. For example,

LIBNAME old4 V604 ‘c:\temp\old';

will associate with the LIBREF "old4" the engine "V604". Thus, files saved in C:\TEMP\OLD with SAS versions 6.03 or 6.04 can be accessed using the LIBREF "old4".

More than one LIBREF can be assigned to the same path if SAS data sets that have been created with different engines are stored in the same location. For example, we may have an additional LIBREF defined for SAS 6.10 data sets given by :

 LIBNAME old10 V610 ‘c:\temp\old';

The LIBREF "old10" would be used to identify SAS data sets that were stored using SAS 6.10.

Note that the "engine" is an option in a LIBNAME statement, and not essential. When no "engine" is specified, SAS 6.10 will determine the "engine" it considers best to use. The engine is identified as follows:

• a. if there are no SAS data files in the LIBREF, then the engine is the current SAS version executing, V612 (or later).
• b. if SAS data files exist in the LIBREF that were created by the same engine, then that engine is assigned the LIBREF.
• c. if SAS data files exist in the LIBREF that were created using different engines, then the most recent SAS engine will be used (ie. V612). 

Once an "engine" is associated with a LIBREF in a LIBNAME statement, all files read from or written to the LIBNAME location are read or written using the "engine" assigned to the LIBNAME.