|Space Telescope Science Institute|
|ACS Data Handbook V7.2|
2.1.1 Data Files and Suffixes
- Raw (raw.fits) files from Generic Conversion.
• A single fully-calibrated MAMA image is given the suffix flt.fits. (MAMA images do not have an overscan region, and they are not affected by cosmic rays and CTE.)
- After the bias image and bias level are subtracted, and the overscan regions trimmed, a temporary file, with the suffix blv_tmp.fits, is created. (By default, CTE-corrected images are also created for WFC data so calacs creates a temporary file with suffix blc_tmp.fits.)
- Upon completion of additional calibration steps (dark subtraction, flat-fielding, etc.), the temporary file is renamed with the suffix flt.fits. For WFC data, CTE-corrected images, with suffix flc.fits, are also created. The flt.fits and flc.fits files will later serve as input for AstroDrizzle.
- Raw images from a “CR-SPLIT” observation undergo bias image and bias level subtraction. These images are then combined at the cosmic ray rejection step in calacs to create a temporary image with suffix crj_tmp.fits. For WFC images with CTE corrections, a file with suffix is crc_tmp.fits is also created.
- Other basic calibrations are performed on the temporary combined image. It is then renamed with the crj.fits suffix, and crc.fits suffix for WFC CTE-corrected images.
- Individual calibrated images (flt.fits/flc.fits) are also created for each exposure in the “CR-SPLIT” observation.
- Calibrated flt.fits images are created for each sub-exposure.
- A summed flat-fielded image is created, with suffix sfl.fits. (MAMA images are not affected by cosmic rays and CTE.)Table 2.1: ACS File Suffixes
Trailer file containing calacs processing comments. (This is the same as .tra files generated during manual calibration.) Overscan-trimmed individual exposure (these will be renamed flt.fits or flc.fits after all basic calibrations are completed). CR-rejected combined image created using blv_tmp.fits and blc_tmp.fits for WFC CTE-corrected data. (these will be renamed crj.fits and crc.fits for WFC CTE-corrected images, after all basic calibrations are completed.) Calibrated, flat-fielded individual exposure.CTE-corrected images with suffix flc.fits are also created for WFC data. Calibrated, geometrically-corrected, dither-combined image (created by AstroDrizzle, which is not a part of calacs). The CTE-corrected version, for WFC images, has suffix drc.fits.
Intermediate calibrated products created by calacs, such as sfl.fits, blv_tmp.fits, blc_tmp.fits, crj_tmp.fits, crc_tmp.fits, crj.fits and crc.fits, are, by default, not delivered by the Archive.
Standard calibrated files delivered from the Archive contains these extensions: asn.fits, spt.fits, trl.fits, flt.fits, and drz.fits. For WFC images, flc.fits and drc.fits are also delivered.
To obtain intermediate calacs products from the Archive, enter specific extensions (e.g., crj, crj_tmp) in a field titled “File Extensions Requested” near the bottom of the Archive data request Web page.2.1.2 Association TablesAssociation tables describe and track the relationship or “associations” between data products for a set of observations. Such relationships include repeated exposures in observations, “CR-SPLIT” observations, and dithered observations. ACS association tables can be used to instruct calacs to create different levels of calibration products. These tables are particularly useful for keeping track of complex observations, for example, an observation at a specific dither position that may be additionally split into multiple exposures. Edited association tables can also be used with calacs to create non-default calibration products (see Example 3 in Section 3.5).
• A single image from an exposure or sub-exposure is the “atomic unit” of HST data.
• A dataset is a collection of files having a common rootname (first nine characters of the image name).
• A sub-product is created by combining a subset of the exposures in an association.
• A product is created by combining sub-products, or in some cases, individual exposures (before they were incorporated into a sub-product), of an association.An ACS association table has three primary columns: MEMNAME, MEMTYPE, and MEMPRSNT. (An example of an association table is shown in Table 2.3.)
• The column MEMNAME lists the name of each exposure in the association and names of calacs output products.
Same as “EXP-CRJ,” but used when there are multiple “CR-SPLIT” observations in an association. n is a numerical identification for each “CR-SPLIT” set (e.g., “EXP-CR1,” “EXP-CR2”). A calibrated and CR-rejected combined image created from a “CR-SPLIT” observation. (“PROD-CRJ” is used when there is only one “CR-SPLIT” observation in an association.) Same as “PROD-CRJ” but used when there are multiple “CR-SPLIT” observations in an association. n is a numerical identification for each “CR-SPLIT” set and its corresponding PROD-CRn combined image (e.g., “PROD-CR1,” “PROD-CR2”). Same as “EXP-RPT,” but used when there are multiple observations in an association, each containing a series of repeated sub-exposures. n is a numerical identification for each such observation (e.g., “EXP-RP1,” “EXP-RP2”). A calibrated summed MAMA image, created from an observation containing a series of repeated sub-exposures. (“PROD-RPT” is used when there is only one such observation in an association.) Same as “PROD-RPT,” but used when there are multiple observations in an association, each containing a series of repeated sub-exposures. n is a numerical identification for each such observation and its corresponding summed PROD-RPTn image (e.g., “PROD-RPT1,” “PROD-RPT-2”). An example of an association table is shown in Table 2.3. But first, to trace back its origins, begin by looking at the Phase II proposal commands that created the data. This example came from proposal 10605, visit 1, exposure log sheet number 1. The observations, using ACS/WFC, were taken as a two-point dither with a “CR-SPLIT=2” at each dither point. The proposal’s dither pattern specification looks like this:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------A standard calibrated data retrieval from the Archive for images taken by Visit 1, exposure 1, are listed below. (Intermediate calacs products from the Archive have to be specifically requested, therefore, the crj.fits and crc.fits files are not included in the delivery.)
There are two pairs of 4 single calibrated images, one pair with suffix flt.fits, and the other, corrected for CTE, with suffix flc.fits.j9cm01jvq_flt.fits and j9cm01jvq_flc.fitsj9cm01jwq_flt.fits and j9cm01jwq_flt.fitsj9cm01k2q_flt.fits and j9cm01k2q_flc.fitsj9cm01k4q_flt.fits and j9cm01k4q_flc.fits
• The combined image created by AstroDrizzle, in the pipeline, from four flt.fits and flc.fits images, respectively, are:j9cm01010_drz.fits and j9cm01010_drz.fitsTable 2.3: Contents of Association Table, j9cm01010_asn.fits
- “MEMPRSNT” set to “yes” indicates that those images underwent standard calibrations in calacs (which does not include geometric distortion corrections).
• Rows 3 and 4 describe the association’s second “CR-SPLIT” observation at dither point 2. The two images have rootnames “J9CM01K2Q” and “J9CM01K4Q.” “EXP-CR2” means they came from the second CR-SPLIT observation, and “yes” indicates that those images underwent standard calibrations in calacs (which does not include geometric distortion corrections).
• In row 5, “PROD-DTH” and “yes” indicate that the single exposure images in the association were drizzle-combined in the pipeline by AstroDrizzle, and named with the rootname specified in the MEMNAME column. In this example, four flt.fits images were drizzled-combined to produce j9cm01010_drz.fits, and four flc.fits images were drizzle-combined to create j9cm01010_drc.fits. (If “MEMPRSNT” had been set to “no,” the AstroDrizzle step would have been omitted in the pipeline.)
• In Row 6, “PROD-CR1” and “yes” indicate that the two images from the first “CR-SPLIT” (rows 1 and 2) were combined with cosmic ray rejection by calacs to create an image with the rootname specified in the MEMNAME column. This was done for flt.fits and flc.fits images to create the files J9CM01011_crj.fits and J9CM01011_crc.fits, respectively.
• In Row 7, “PROD-CR2” and “yes” indicate that the two images from the second “CR-SPLIT” (rows 3 and 4) were combined with cosmic ray rejection by calacs to create an image with the rootname specified in the MEMNAME column. This was done for flt.fits and flc.fits images to create the files j9cm01012_crj.fits and j9cm01012_crc.fits, respectively.2.1.3 Trailer FilesEach task in the calacs package creates messages that describe the progress of the calibration; these messages are directed to STDOUT (STanDard OUTput), which simply means that processing messages appear on the screen during the calacs run.In pipeline processing for first and second generation HST instruments, where data files were calibrated one at a time, trailer files were created by simply redirecting the STDOUT contents to a file. However, the ACS pipeline was designed to calibrate several images at a time (like those described in Section 2.1.2), and create different types of output files. Therefore, each task within the calacs package must decide how to populate the trailer files associated with each product.calacs will always overwrite information in existing trailer files from previous runs of calacs while preserving any comments generated by Generic Conversion. This ensures that the trailer files accurately reflect the most recent processing performed by calacs. After the generic conversion entries, the string “CALACSBEG” marks the first comment added to a trailer file by calacs. If the trailer file already exists, calacs searches for this string, then begins to write new processing comments from that point onwards, over-writing previous calacs comments. If “CALACSBEG” is not found in an existing trailer file, calacs will write that string at the end of the trailer file, then continue populating the trailer file with calibration processing comments.As each image is reprocessed, an accompanying trailer text file with the suffix “tra” (without the .fits extension) is created. (The trl.fits file from the Archive has the same content; it can be read using the PyRAF tdump command, e.g., “tdump jb5u34010_trl.fits”.)Following the processing hierarchy specified in the association table, information in trailer files belonging to images used for creating a higher level product will be included in the trailer file of that higher level product. In other words, the trailer file for any product processed by the pipeline will contain processing comments from trailers belonging to each input file.Linking trailer files together can result in multiple occurrences of the “CALACSBEG” string. Only the first, however, determines where calacs will begin overwriting comments if an observation is reprocessed.An “observation” set is a group of exposures under the umbrella of one unique ID. Each set has the same target, instrument configuration, operating mode, aperture, and spectral elements. An observation set created by the pipeline usually contains a set of dithered, repeated, and/or “CR-SPLIT” exposures taken within a visit. The data products in each calibrated observation set is described in its association table.