IRAF Mini-Tutorial

IRAF is the Image Reduction and Analysis Facility, a general purpose software system for the reduction and analysis of astronomical data. The core IRAF distribution includes a good selection of tasks for general image processing and graphics plus a large number of programs for the reduction and analysis of O/IR data in the "noao" package. The bulk of IRAF software in the community is in the form of individual tasks written by users and small to rather large external packages developed by groups outside of NOAO which handle data from other observatories (e.g. HST) and wavelength regimes (EUVE for extreme ultra-violet or ROSAT/AXAF for X-ray). These external packages are distributed separately from the main IRAF distribution but can be easily installed.

Why Use IRAF for VO Applications Development?

The IRAF system includes a complete programming environment for scientific applications, which includes a programmable Command Language (CL) scripting facility, a Fortran/C programming interface, and the full SPP/VOS programming environment in which the portable IRAF system and all applications are written. A complete description of these programming interfaces is not possible in a mini-tutorial such as this and so we will focus on the scripting capabilities of the system, and the many available tasks, for creating science applications.

Script tasks may also make use of host commands through the 'foreign' command interface, making it possible to develop CL script applications that mix IRAF analysis tasks with non-IRAF tasks in any variety of languages to access data and services from VO providers. On the server-side, tasks may be trivially turned into web-services via a simple configuration file using a system currently in development. The GALCAS/GALMORPH services used to support the science theme demonstrate both of these capabilities.

Installing and Starting IRAF

The NVOSS software distribution contains a setup script that will do a network-based install of IRAF on Mac OS X and Linux systems. This includes the core system plus several external packages (STSDAS/TABLES) and support software (i.e. DS9 display server and XGterm graphics terminal) used in the following examples.

Installing IRAF requires root permissions due to links made into the system by the installer, but it is as simple as

      # cd /some_dir/
      # $NVOSS_HOME/iraf/iraf_setup.csh

where some_dir is the directory on your machine where you will install the IRAF software tree. The setup script may also be executed with the 'sudo' command instead of logging in as the root user (most Mac OS X users will need this option).

Once the system is installed you can login using the commands: 

      % mkiraf		# Create the login.cl
      % ecl		# Start IRAF

The MKIRAF command is needed only the first time you start IRAF, it creates a "login.cl" file defining the system environment and a "uparm" directory to used to store modified task parameters. You should always login from a directory containing a 'login.cl' file, or else issue a new MKIRAF to create one. You should not copy the login.cl file created to other directories as environment variables in this file are path-relative. You can create a loginuser.cl file in the same directory to hold personal definitions or package loading, just be sure the last line in the file is keep.

The ECL is an enhanced CL for IRAF having error-handling and other new, tcsh/bash-like, features. To use the traditional CL, log into the system using the 'cl' command instead. Students on Windows machines who wish to use IRAF will be provided with an account on an NOAO system for use during the Summer School.

Navigating Around the System

Help/Context Commands
p/help get help on a particular task
references search for tasks by keyword
which/whereis print package containing a task
? show tasks in current package
?? show tasks in all loaded package
pack list currently loaded packages

Upon logging into the system you'll see the standard system banner and a menu of currently loaded packages. To load a new package simply type its name. Tasks are likewise executed but may contain additional arguments on the command-line to specify the parameters, required "query" parameters will be prompted for by the system (see below for an explanation of the parameter system) if not specified.

The phelp command will paginate output and like help takes as an argument the name of a task or package (in which case a one-line description of all tasks in that package is returned). The references task takes a string argument and returns a list of task descriptions matching that string, effectively allowing a keyword search. Additionally, if you are using XGterm as the terminal window, a GUI help browser is available that combines paging and searching. For example, 


    ecl> help implot dev=gui
    ecl> refer photometry
    apphot - Aperture Photometry Package  [digiphot]
    ccdtime - CCD photometry exposure time calculator  [obsutil]
    daophot - DAO Crowded-Field Photometry Package   [digiphot]
    digiphot - Digital stellar photometry package           [noao]
	:

CL Script Task Development

Basic CL Syntax

CL statements mostly follow C syntax conventions and have the same sorts of conditional and controls statements (i.e. "if-else" and "switch-case" statements, "while" and "for" loops, boolean expressions in conditionals, etc). Some differences C programmers may notice however are missing '++' and '--' operators to inc/decrement variables (although "i += 1" is valid), and fact that semi-colons are not required to terminate statements. Java programmers may notice that string concatenation may be done using a '+' sign although the double-slash '//' is more traditional in IRAF. The hash/crosshatch symbol ('#') is used to start a comment that continues until the end-of-line. Complete details of the language syntax are given in the "User's Introduction to the IRAF Command Language" and "CL Programmer's Manual" (both compressed Postscript files).

A few peculiarities of the CL deserve a bit more detailed discussion:

A "list directed parameter" is specified by prepending an asterisk to a parameter declaration of any type (but typically a string). These are used to read multiple values from a file with each reference returning the subsequent line from the file. For instance, 

    ecl> type testfile
    this is line 1
    this is line 2
    ecl> string *ld
    ecl> ld = "testfile"
    ecl> = ld
    this is line 1
    ecl> = ld
    this is line 2
    ecl> = ld
    EOF

Within a script these are typically used in a while() loop to read lines from a file, e.g.

    struct line
    real  ra, dec
    list = "testfile"
    while (fscan (list, ra, dec) != EOF)
        printf ("%H %h\n", ra, dec)

Note the declaration of line as a struct -- String variables terminate at whitespace while struct variables continue until the end of line. Also note the special formatting types '%H' and '%h' used in the printf() statement, these are peculiar to the CL and allow the formatting of sexigesimal numbers (as HMS or DMS respectively, a complete description of the formatting codes can be seen with "cl> help printf").

Similarly, sexigesimal values may be used in the CL directly since they are converted internally to floating-point values (always double-precision) 

    ecl> x = 12:34:56.7
    ecl> =x 
    12.582416666667
    ecl> = (x * 15)		# or " =(12:34:56.7 * 15)"
    188.73625			# to convert to RA in degrees
    ecl> x = 188.73625 ; y = -35.8324762
    ecl> printf ("%H %h\n", x, y) # to print degrees as RA and Dec
    12:34:56.7 -35:49:56.9

Note the use of semi-colons to put multiple statements on a single line and the use of sexigesimal values in expressions. Using an equal sign at the CL prompt serves to make the CL a handy calculator or simply to inspect the value of a variable or parameter.

Below we see how fscan() can be used to read values from a string. In the print() statements that follow we output the values using both the traditional '//' and '+' concatenation operators to construct the string, but when using '+' we need to explicitly convert the type. Note also that unlike the printf() used above, a print() will automatically output a newline after the string.

    ecl> string test = "word 17 3.14 now is the time"
    ecl> = fscan (test, s1, i, x, line)
    4
    ecl> print ("s1 = " // s1 // "  x = " // x)
    s1 = word  x = 3.14
    ecl> print ("s1 = " + s1 + "  x = " + str(x))
    s1 = word  x = 3.14

Next we'll see an example of the "scan-from-pipe" feature of the CL and how it can be used to cature the output of tasks to local variables. 

    ecl> imstat dev$pix format-
    dev$pix 262144 108.3 131.3 -1. 19936.
    ecl> imstat ("dev$pix", fields="mean,stddev", format-) | scan (x, y)
    ecl> printf ("%6.2f +/- %6.2f\n", x, y) | scan (line)
    ecl> = line
    108.32 +/- 131.30

The CL can be called as a task to interpret a command as with the Unix eval command by constructing a command and passing it to the CL for interpretation:

    ecl> printf ("imstat ('%s', fields='%s', format-)\n", s1, s2) | cl()
    108.3154 262144

    ecl> s1 = mktemp ("tmp$tmp")
    ecl> imhead ("dev$pix", lo+, > s1)
    ecl> printf ("!grep Overscan %s\n", osfn(s1)) | cl()
    BT-FLAG = 'Apr 22 14:11 Overscan correction strip is [515:544,3:510]'

        but, be sure you really need to do so, first:

    ecl> match ("Overscan", s1)
    BT-FLAG = 'Apr 22 14:11 Overscan correction strip is [515:544,3:510]'

In the first example we construct a call to the IMSTAT task for execution, but obviously one would simply make the call to an IRAF task directly. The second example is more realistic -- here we dump the image header to a file (named by the "s1" temp file) and use the '!' CL escape to call the unix grep utility to search for a string. In many (but not all) cases there is an alternative for common utilities like this already in the system.

Image Sections

Section Refers to Section Refers to
pix[]the whole image pix[i,j]the pixel value (scalar) at [i,j]
pix[*,*]whole image, two dimensions pix[-*,*]flip x-axis
pix[*,-*]flip y-axis pix[-*,-*]flip x and y-axis (180o rotate)
pix[*,*,b]band B of three dimensional image pix[*,*:s]subsample in y by S
pix[*,l]line L of image pix[c,*]column C of image
pix[i1:i2,j1:j2]subraster of image pix[i1:i2:sx,j1:j2:sy]subraster with subsampling

All IRAF programs which operate upon images may be used to operate on the entire image (the default) or any section of the image. A special notation is used to specify image sections. The section notation is appended to the file name of the image, much like an array subscript is appended to an array name in a conventional programming language. If no section is specified, the entire image will be used.

Unfortunately image transposition is not supported by the section syntax, but almost any other combination of flips, subsamples, or range notations are supported.


Parameter Syntax

Task parameters can be either query params that will be prompted for if not supplied on the command line, or hidden params that will use a default value defined in the task or the user-specified value on the command line. Boolean values in IRAF are yes or no values and may be abbreviated on the command line with '+' or '-', i.e. "cl> <task> bpar+" will turn on the parameter 'bpar'. Sexigesimal values are understood by the CL and will be converted to floating point before being passed to the task.

Essential Parameter Commands
epar edit task parameters
lpar list task parameters
dpar dump task parameters
unlearn reset task parameters
cl> task.param=value set a task parameter
cl> =task.param inspect the value of a param
All script args are query params
All variables between procedure/begin are script params

Parameters are learned between task calls and if a default value is changed it will be remembered the next time the task is used. Task params may be reset to their defaults using the unlearn command. Similarly, changing the number or name of params requires that you unlearn the task to pick up the change. Editing all of the params for a task is done using the epar command, simply list parameters can be done with the lpar or dpar commands.

The CL itself has parameters, some of which can be used as variables without declaring them explicitly in a script (e.g. 'i' is an int, 's1' is a string,etc). Other parameters control behavior of the CL, to see these use a command such as "cl> lpar cl".


Declaring Custom Tasks

The task command in the CL is used to define a new application for the system. In the case of script tasks we're considering here (all of which should be procedure scripts), the syntax is simply

The ".cl" extension on scripts is required and normally the name of the task is the name of the file as well. Tasks that have no parameters (i.e. arguments in the procedure statement) are required to be declared with a '$' before the task name, for example

Tasks with parameters do not use the leading '$' on the name. So-called foreign commands (i.e. host unix commands) obviously have no IRAF parameters and in addition are declared using either the reserved word foreign to indicate these commands will be found in the user's shell, or with a leading '$' on the command itself if they are declared as command strings. For example, to declare the host commands wget and sed as well as to create a command called tpipe that executes a method in a Java file we would do something like:

See the help page for the task statement for additional details. Note that in the tpipe declaration we are able to use shell environment variables, we also use the special string "$*" to pass through arguments to the task at the host level.

Task declarations and other environment variable settings may also be put into a loginuser.cl file in the same directory as the login.cl file, provided the last line of the file is keep to retain the definitions. A MKIRAF will overwrite the login.cl each time it is used, a loginuser.cl keeps from accidently overwriting custom changes.


Putting It All Together -- An Example Application

The following example implements a crude SIA table browser that allows the user to automatically download and examine the images found for a given position on the named SIA service. We first return the query result and then iterate over the resulting image references, using both foreign commands and native IRAF applications.

To run this example, cut-n-paste the code below and save to a file called siabrowser.cl in your IRAF login directory. Alternatively, the script may be downloaded from here.

Next, log into IRAF and declare the task as:

We will assume the wget/sed/tpipe task declarations from above are still valid. The use of "home$" illustrates the used if IRAF logical pathnames, where "home" is defined in your login.cl file as the iraf login directory.

Next, start the DS9 display server and execute the task using the coordinates of a known results (the Sextans B cluster in this case):

Note that we use the '!' escape to start ds9 as a host command in the background before running the task. Note also the use of sexigesimal values for the position; internally these are converted to decimal (10.0, 5.33) values but in the script we must still convert RA to degrees.

A loginuser.cl file with the needed definitions is available here, this should be put in the same directory as the login.cl file and will be read automatically when you login to IRAF.

1



5



10



15



20



25



30



35



40



45



50



55



60



65



70



75



80



85



90



95



 
# SIABROWSER -- Given a search position and size, query an SIA server for
# results.  Loop through the returned image table and download the FITS
# files, optionally displaying or analyzing them.

procedure siabrowser (ra, dec, size)

real    ra                { prompt = "RA (hours) of search" }
real    dec               { prompt = "DEC (degrees) of search" }
real    size              { prompt = "Size of search" }

string  siap = "http://archive.noao.edu/nvo/sim/voquery.php" \
                                 { prompt = "SIA Service" }

begin
    string url, ch, imname, res, tmplist, tabnam
    int    imnum = 0


    # Form the URL of the query using print() and scan to the result
    printf ("%s?POS=%g%%2C%g&SIZE=%g\n", 
        siap, (ra * 15.0), dec, size) | 
        sed ("-e", "'s/\?/\\\\?/g'", "-e", "'s/\&/\\\\&/g'") | scan (url)

    printf ("Querying server ....")
    res = mktemp ("res")
    wget ("-q", "-O", "-", url, >& res//".xml")

    # Convert the downloaded VOTable to a FITS bintable we can deal with.
    tabnam = res // ".fits"
    tpipe (res//".xml", "-ofmt", "fits-basic", "-o", tabnam)

    # Print some stats about the table.
    tinfo (tabnam, ttout-)
    printf ("Result:  %d rows   %d cols\n", tinfo.nrows, tinfo.ncols)

    # Create a list of the image access references.
    tmplist = mktemp ("tmp$sia")
    tprint (tabnam, columns="accessreference", showhdr-, showrow-, 
        showunits-, pwidth=1024, >& tmplist)


    # Loop over the image list, downloading the file and displaying it.
    list = tmplist
    while (fscan (list, s1) != EOF) {

        imnum = imnum + 1
        printf ("sia_%.1f_%.1f_%03d.fits\n", ra, dec, imnum) | scan (imname)

        # Filter the URL to make wget happy.
        print (s1) | 
            sed ("-e", "'s/\?/\\\\?/g'", "-e", "'s/\&/\\\\&/g'") | scan (url)

        # Download the image.
        printf ("Downloading image ...." // imname // " ")
        wget ("-q", url, "-O", imname)

        for (ch = "d"; ch != "n"; ch = cl.ukey) {
            print ("")
            switch (ch) {
            case "?":   print ("help for commands here...")
            case "c":   tlcol (tabnam)
            case "h":   imheader (imname, long+)
            case "e":   imexamine (imname, 1)
            case "d":   display (imname, 1, fill+)
            case "n":   break
            case "q":   goto cleanup
            case "r":   tprint (tabnam, rows=imnum, showrow-)
            case "s":   imstat (imname)
            }
            printf ("Command? ")
        }

    }

cleanup:
    list = ""                                   # cleanup list pointer
    delete (tmplist, ver-, >& "dev$null")       # delete temp files
end

Notes:



References and Useful Links

Introductory User's Guide to IRAF Scripts
IRAF Beginner's Guide
User's Introduction to the IRAF Command Language
CL Programmer's Manual

Recommended Documentation for every site
General Documentation and Beginner's Materials
Photometry
Spectroscopy
General Image Processing
Programming Documentation
Installation and Site Manager's Guides

Mike Fitzpatrick, NOAO
NVO Summer School 2005