Parse the input file's ext=0 header for the CRREJTAB.
    A user-specified CRREJTAB may be provided instead.

    - update the input file's HISTORY
    - update the input file's CRREJTAB keyword to point to a new temporary file
    - temporarily store the old CRREJTAB value (if valid) in a new PREV_CRR keyword
    - read the CRREJTAB FITS table

    Parameters
    ----------
    input: str
        Name of the RAW FITS file to update.

    crrejtab: str or None, default=None
        User-supplied CRREJTAB to be used instead of the file specified in the ext=0
        header.

    Returns
    -------
        Astropy Table of CRREJTAB contents


Convert a dict to a 1-row Astropy Table with the correct column dtypes.

    Parameters
    ----------
    crr_par: dict
        Dictionary of CRREJECTAB contents

    Returns
    -------
    Astropy Table of CRREJECTAB contents


Compute a 2-D array of wavelengths, one value for each image pixel.

    Parameters
    ----------
    shape: tuple of two ints
        the number of rows and columns in the output image
    phdr:  FITS Header object
        primary header
    hdr: FITS Header object
        extension header
    helcorr: str
        "PERFORM" if heliocentric correction should be done

    Returns
    -------
    wavelengths: ndarray of float64
        an array of wavelengths, of the same shape (nrows, ncols) as
        the output image


Adjust the dispersion coefficients.

    The changes to the coefficients are for the incidence angle
    correction, the offset from the SHIFTA1 keyword, and the tilt
    of the slit.  The coefficients will be modified in-place.

    Parameters
    ----------
    ncoeff: int
        number of dispersion coefficients
    coeff: ndarray of float64
        array of dispersion coefficients, modified in-place
    delta_offset1: float
        incidence angle offset in degrees
    shifta1: float
        MSM offset (ref. pixels) in the dispersion direction
    delta_tan: float
        difference in tangents of slit angle and ref angle
    delta_row: float
        difference between current row number and CRPIX2
    binaxis1: float
        binning factor in dispersion direction
    inang_info: rec_array
        rows from the incidence-angle table


Computes an FFT on STIS CCD frames to evaluate fixed pattern noise.

    Fixed pattern noise is most obvious in a FFT of bias
    frames.  Optional filtering to correct the fixed pattern noise is
    provided through keywords boxcar, wipe, and window.  Filtered data
    can be saved as an output file.

    Parameters
    ----------
    infile: str
        STIS FITS file
    exten: int, optional [Default: 1]
        fits extension to be read
    dc: int, optional [Default: 1]
        the power in the first freq bin is set to zero for better
        plotting of the power spectrum.
    verbose: int, optional [Default: 1]
        set to 0 if you do not want brief information about each image.
    boxcar: int [default: 0]
        width of boxcar smoothing to be applied.
    wipe: ndarray or None
        a 3-element array, specifying how to modify the data in
        frequency space. If set, the image is converted to a 1-D time
        series, fourier transformed to frequency space, modified, inverse
        transformed back to time space, and converted back to a 2-D image.
        The first and second elements specify the range in frequencies to
        be scaled (in hz), and the third element specifies the scaling
        factor (should be 0-1).
    window: ndarray or None
        a 3 element array, specifying how to modify the data in
        frequency space.  The first element is the center of the window
        (in hz). The second element is the width of the window (in hz).
        The third element controls the tapering of the window - it is the
        scale (in hz) of the tapering width.  Specifically, a square
        bandstop is convolved with a gaussian having the FWHM given by the
        third parameter.
    outfile: str or None, optional
        name of filtered image file

    Returns
    -------
    noise_terms: tuple of arrays
        A tuple containing the arrays; namely, the arrays::

            freq  = frequency in power spectrum (hz)
            magn  = magnitude in power spectrum

    Notes
    -----
    Authors:
      - Original algorithm: Thomas M. Brown (STScI)
      - Python version: Paul Barrett (STScI)


Find the cross-dispersion location of the target spectrum.

    Parameters
    ----------
    input: ndarray
        The input science data array for the current image set.

    low_frac: float
        Fraction of image width for the start of a slice.

    high_frac: float
        Fraction of image width for the end of a slice.

    low_line_frac: float
        Fraction of image height for the start of a slice for limiting the
        region over which to search for the maximum of the slit profile.

    Returns
    -------
    target_locn: float or None
        The location (zero based pixel coordinate) of the target in the
        cross-dispersion direction.
        None will be returned if the quadratic fit to the cross-dispersion
        profile has zero curvature.


Normalize STIS CCD fringe flat.

    Based on the PyRAF `stsdas.hst_calib.stis.normspflat` task.

    Parameters
    ----------
    inflat: str
        Name of input fringe flat
    outflat: str
        Name of normalized fringe flat output or directory location.  (Default=".")
    do_cal: bool
        Perform bias and dark subtraction and CR rejection?  (Default=True)
    biasfile: str or None
        Name of superbias image.  If None, use BIASFILE in main header of the inflat.
    darkfile: str or None
        Name of superdark image.  If None, use DARKFILE in main header of the inflat.
    pixelflat: str or None
        Name of pixel-to-pixel flat.  If None, use PFLTFILE in main header of the inflat.
    wavecal: str or None
        Name of wavecal file [ONLY FOR G750M SPECTRA].  If None, use WAVECAL in main
        header of the inflat.

    Returns
    -------
    outname: str
        Fully qualified name of the outflat


Command line entry point for normspflat().


Main 1-d fitting function.  The parameters can be used exactly like they are in IRAF tasks
    like fit1d, response, etc.  The default behaviour is to fit a cubic spline with order 3
    (3 equal regions, 2 knots) with no sigma clipping.

    Parameters
    ----------

    x: 1-d ndarray
        The array of independent values

    y: 1-d ndarray
        The array of dependent values.  Must have same length as x, or function returns None

    weights: 1-d ndarray or None
        The array of weights.  If set to None, the points are given equal weight of 1.0.  If not
        set to None, must have the same length as x and y, or the function returns None

    naverage: int
        The number of adjacent elements of x, y, and weight that are averaged before
        fitting is done.  If naverage=1, no averaging is done.

    function: str
        Fitting function.  Currently only "spline1" and "spline3" are supported

    order: int
        Order of fitting fuction.  The number of knots in the spline fit is (order-1)

    low_reject: float
        Points with y values less than fit - low_reject*(rms deviation of data from fit) are rejected
        when the fit is iterated.  Ignored if niterate = 0.

    high_reject: float
        Points with y values greater than fit + low_reject*(rms deviation of data from fit) are rejected
        when the fit is iterated.  Ignored if niterate = 0.

    niterate: int
        Number of times sigma-clipping iterations are performed.  niterate=0 corresponds to no
        sigma clipping

    grow: float
        Used to calculate how many elements adjacent to reject pixels are also rejected.
        Not currently implemented, but included to duplicate the IRAF fitting parameter.

    Returns
    -------

    fit: scipy.interpolate.fitpack2.LSQUnivariateSpline object
        The result of the fit.  It can be used to calculate the fitted
        values for the input x values:

        fitted_values = fit(x)


Do a single fit of function to (x, y) data.

    Parameters
    ----------

    x: 1-d ndarray
        Input array of x-values

    y: 1-d array
        Input array of y-values

    weights: 1-d array
        Input array of weights

    function: str
        Function to be fitted.  Currently only "spline1" and "spline3" are supported

    order: int
        Order of function to be fitted.  Since only splines are currently supported,
        the order is the same as the number of equal reqions the data domain is split into,
        and 1 more than the number of equally-spaced knots.

    Returns
    -------

    fit: scipy.interpolate.fitpack2.LSQUnivariateSpline object
        This can be used to calculate the fitted values for the input x values:

        fitted_values = fit(x)


Fit with sigma-clipping

    Parameters
    ----------

    x: 1-d ndarray
        The array of independent values

    y: 1-d ndarray
        The array of dependent values.  Must have same length as x, or function returns None

    weights: 1-d ndarray or None
        The array of weights.  If set to None, the points are given equal weight of 1.0.  If not
        set to None, must have the same length as x and y, or the function returns None

    function: str
        Fitting function.  Currently only "spline1" and "spline3" are supported

    order: int
        Order of fitting fuction.  The number of knots in the spline fit is (order-1)

    low_reject: float
        Points with y values less than fit - low_reject*(rms deviation of data from fit) are rejected
        when the fit is iterated.  Ignored if niterate = 0.

    high_reject: float
        Points with y values greater than fit + low_reject*(rms deviation of data from fit) are rejected
        when the fit is iterated.  Ignored if niterate = 0.

    niterate: int
        Number of times sigma-clipping iterations are performed.  niterate=0 corresponds to no
        sigma clipping

    grow: float
        Used to calculate how many elements adjacent to reject pixels are also rejected.
        Not currently implemented, but included to duplicate the IRAF fitting parameter.

    Returns
    -------

    fit: scipy.interpolate.fitpack2.LSQUnivariateSpline object
        The result of the fit.  It can be used to calculate the fitted
        values for the input x values:

        fitted_values = fit(x)


Calculate the weighted RMS deviation between y and fitted(x)

    Parameters
    ----------

    x: 1-d ndarray
        Input array of x-values

    y: 1-d array
        Input array of y-values

    weights: 1-d array
        Input array of weights

    fitted: scipy.interpolate.fitpack2.LSQUnivariateSpline object
        The result from running the fitting routine

    Returns
    -------

    rms_deviation: float
        The rms deviation


Like the IRAF rg_wtbin.  Bin the x, y, and weight data by averaging nbin values.

    Parameters
    ----------

    x: 1-d ndarray
        The array of independent values

    y: 1-d ndarray
        The array of dependent values.  Must have same length as x, or function
        returns (None, None, None)

    weights: 1-d ndarray or None
        The array of weights.  If set to None, the points are given equal weight of 1.0.  If not
        set to None, must have the same length as x and y, or the function
        returns (None, None, None)

    nbin: int
        The number of adjacent elements of x, y, and weight that are averaged before
        fitting is done.  If nbin=1, no averaging is done and the input arrays are returned

    Returns
    -------

    (xout, yout, wout): 3-tuple of ndarray objects
        The binned/averaged x, y, and weight arrays


    RESPONSE -- Determine the response function for 2D spectra.

    A calibration image is divided by a normalization spectrum to form
    a response image.  The normalization spectrum is derived by averaging
    the normalization image across dispersion.  The normalization spectrum
    is then smoothed by curve fitting.  The smoothed normalization
    spectrum is divided into the calibration image to form the response
    function image.  The curve fitting may be performed interactively
    using the icfit package.  A response function is determined for each
    input image.  Image sections in the calibration image may be used to determine
    the response for only part of an image such as with multiple slits.

    The images are given by image templates.  The number of images must
    in each list must match.  Image sections are allowed in the calibration
    image.

    lpar response:
  calibration = "star_cont_sh1.fits[5:1020,505:515]" Longslit calibration images
 normalizatio = "star_cont_sh1.fits[5:1020,505:515]" Normalization spectrum images
     response = "norm_star_cont_sh1.fits" Response function images
 (interactive = yes)            Fit normalization spectrum interactively?
   (threshold = INDEF)          Response threshold
      (sample = "*")            Sample of points to use in fit
    (naverage = 2)              Number of points in sample averaging
    (function = "spline3")      Fitting function
       (order = 15)             Order of fitting function
  (low_reject = 3.0)            Low rejection in sigma of fit
 (high_reject = 3.0)            High rejection in sigma of fit
    (niterate = 2)              Number of rejection iterations
        (grow = 0.0)            Rejection growing radius
    (graphics = "stdgraph")     Graphics output device
      (cursor = "")             Graphics cursor input
        (mode = "al")


Like re_make in response.x


Like re_normalize in response.x


Defringe by dividing the science spectrum by the fringe flat.

    Based on the PyRAF `stsdas.hst_calib.stis.defringe` task.

    Parameters
    ----------
    science_file: str
        The name of the input science file.

    fringe_flat: str
        The name of the input fringe flat file.  This is the output from
        `mkfringeflat`.

    overwrite: bool
        The name of the output file will be constructed from the name of the
        input science file (`science_file`) by replacing the suffix with
        'drj' or 's2d'.  If the input name are the same a RuntimeError will
        be raised, rather than modifying the input in-place.
        If there is an existing file with the same name as the output name,
        the existing file will be overwritten if `overwrite` is True (the
        default is True).

    verbose: bool
        If True (the default), print more info.

    Returns
    -------
    drj_filename: str
        The name of the output file.  This will have suffix '_drj' if the
        input is G750L data, and the output name will have suffix '_s2d'
        if the input is G750M.


Calibrate STIS CCD G750L or G750M spectrum before defringing.

    Based on the PyRAF `stsdas.hst_calib.stis.prepspec` task.

    Parameters
    ----------
    inspec: str
        Name of input 'raw' science spectrum
    outroot: str
        Root for output file name.  (Default='./')
    darkfile: str or None
        Name of superdark image.  If None, use DARKFILE in main header of input spectrum.
    pixelflat: str or None
        Name of pixel-to-pixel flat.  If None, use PIXELFLAT in main header of input
        spectrum.
    initguess: str or None
        Method for initial value estimate for `ocrreject`: {None, 'minimum', 'median'}.
        (Default=None; Use the value in the CRREJTAB.)

    Returns
    -------
    outname: str
        Fully qualified name of prepared spectrum (CRJ or SX2 file)


Command line entry point for prepspec().


Takes an input science spectrum and a fringe flat that has been normalized using
    the task `normspflat`.  The fringe flat is shifted and scaled to produce the minimum
    RMS when divided into the science data.

    Based on the PyRAF `stsdas.hst_calib.stis.mkfringeflat` task.

    In `mkfringeflat`, the user can specify a range of shifts and scales for the routine
    to test creating an optimal fringe flat. `mkfringeflat` will go through the shift and
    scale dimensions separately and calculate the RMS using the following steps:

    1. For each shift step, apply the shift to the input flat field

    2. Divide the science data by the shifted flat

    3. Divide out the large-scale SED from the science image using a spline fit in order
       to isolate the fringing pattern (this is called the response image)

    4. Sum the response image along the columns within the RMS region

    5. Calculate the mean and standard deviation of the summed columns of the response
       image

    6. The RMS value for that shift is given by the standard deviation divided by the mean
       found in step 5

    7. Fit the RMS values with a quadratic polynomial weighted by the inverse RMS to find
       the optimal RMS value

    8. Apply the best shift determined in step 7 to the data and repeat steps 1-7 with the
       scale values to find the best scaling

    The RMS values are printed out for each scale and shift but the final best shift and
    best scale values do not necessarily correspond to the printed values. This is
    because the routine is calculating the RMS values based on a fit of the data at each
    scale and shift, rather than being calculated at each discrete step.

    Parameters
    ----------

    inspec: str
        Name of input science spectrum datafile

    inflat: str
        Name of input fringe flat file (usually the output from `normspflat`)

    outflat: str
        Name of output fringe flat to be used in the defringe task

    do_shift: bool
        Controls whether the shift between fringe flat and science data is
        to be calculated

    beg_shift: float
        Initial shift to apply to fringe flat

    end_shift: float
        Final shift to apply to fringe flat

    shift_step: float
        Step-size between shifts to be applied to fringe flat

    do_scale: bool
        Controls whether the scaling between fringe flat and science
        data is to be calculated

    beg_scale: float
        Initial scaling to apply to fringe flat

    end_scale: float
        Final scaling to appply to fringe flat

    scale_step: float
        Step-size between scaling values to be applied to fringe flat

    extrloc: float or None
        Extraction location.  If set to None, this will be calculated by
        parabolic interpolation of the peak of the cross-dispersion
        spectral sum

    extrsize: float or None
        Extraction size in pixels.  If set to None, this will be set to a
        reasonable value by this routine

    opti_spreg: list or array-like or None
        A list or array representing the section to be used in normalizing the spectrum
        of the science target before it is divided by the shifted/scaled fringe flat.
        If set to None, a reasonable range is chosen by this routine.  Should be
        specified like a Python slice, zero indexed.

    rms_region: list or array-like or None
        A list or array representing the section to be used in the rms calculation.  If set
        to None, a reasonable range is chosen by this routine.  Should be specified
        like a Python slice, zero indexed.


Command line entry point for mkfringeflat().


    Fold and bin light curve for input to LPP metric calculation

    data contains time, tzero, dur, priod,mes and flux (centered around zero)

    ntrfr -- number of transit fraction for binning around transit ~1.5
    npts -- number of points in the final binning.


    Normalize the rawTransitMetric value by those with the closest period.
    This part removes the period dependence of the metric at short periods.
    Plus it makes a value near one be the threshold between good and bad.

    newPerMes is the np.array([period, mes]) of the new sample


    Create the loop over individual transits and return
    array normalized lpp values, mean and std.
    Input TCE object and mapInfo object.

    It is unclear that this individual transit approach
    separates out several new false positives.
    It probably would require retuning for low SNR signals.


    Save array into GeoTIF file.
    Args:
        raster_f (str): input data filename
        segments (numpy.array): array with values
        out_tif (str): output filename
        ndval (int): no data value
    Return:
        save GeoTif to local disk
    ----------
    Example
    ----------
        arr_to_tif('inp.tif', segments, 'out.tif', ndval=-9999)


 doesnt work - some day figure this out
    sys.stdout.close()
    del logFile


Generate working directories.


Get dataset filenames for training.


Read dataset CSV from disk and load for preprocessing.


        Apply a function on each tile. Progress by batching the tiles.
        :param function: The function to apply on each batchs of tile.
        :type function: callable.
        :param progress_bar: choose if a progress_bar should be used.
        :type progress_bar: `bool`. Optional, default value is `False`.
        :param batch_size: Number of batchs to pass inside the function.
        :type batch_size: `int`. Optional, default value is `1`.
        :return: A subclass of `np.ndarray`, with shape of length either `4` or `5`.
        :rtype: `MightyMosaic`.


    Create a instance of `MightyMosaic` from a `np.ndarray`
    :param array: The array on which to apply the tiling.
    :type array: `np.ndarray`, it's shape should be of length either `2` or `3`.
    :param tile_shape: Number of pixels on each axis of the tile.
    :type tile_shape: `list` or `tuple`, should be of length `2`.
    :param overlap_factor: Overlapping of neighbor tiles. If iterable, the overlap can be different on the two axis.
    :type overlap_factor: `int`, `list` or `tuple`. If iterable, should be of length 2.
    :param fill_mode: Points outside the boundaries of the input are filled according to the given mode:
                        `constant`: kkkkkkkk|abcd|kkkkkkkk (cval=k)
                        `nearest`: aaaaaaaa|abcd|dddddddd
                        `reflect`: abcddcba|abcd|dcbaabcd
    :type fill_mode: `str`, one of `constant`, `nearest` or `reflect`. Optional.
    :param cval: Value used for points outside the boundaries when fill_mode = `constant`.
    :type cval: `int`. Optional.
    :return: A subclass of `np.ndarray`, with shape of length either `4` or `5`.
    :rtype: `MightyMosaic`


    Fill an array on the given indexes with respect to the `fill_mode`.
    :param array: Array to fill.
    :type array: `np.ndarray`.
    :param fill_mode: Points outside the boundaries of the input are filled according to the given mode:
                        `constant`: kkkkkkkk|abcd|kkkkkkkk (cval=k)
                        `nearest`: aaaaaaaa|abcd|dddddddd
                        `reflect`: abcddcba|abcd|dcbaabcd
    :type fill_mode: `str`, one of `constant`, `nearest` or `reflect`. Optional.
    :param cval: Value used for points outside the boundaries when fill_mode = `constant`.
    :type cval: `int`. Optional.
    :param i_begin: Index, on the first axis, of the real begin of the array. Index below are filled.
    :type i_begin: `int`. Optional, default value is `0`.
    :param i_end: Index, on the first axis, of the real end of the array. Index above are filled.
    :type i_end: `int`. Optional, default value is `-1`.
    :param j_begin: Index, on the second axis, of the real begin of the array. Index below are filled.
    :type j_begin: `int`. Optional, default value is `0`.
    :param j_end: Index, on the second axis, of the real end of the array. Index above are filled.
    :type j_end: `int`. Optional, default value is `-1`.
    :return:


    Save tif file from numpy to disk.
    :param rast: raster name to get metadata from
    :param prediction: numpy array with prediction output
    :param dtype type to store mask on
    :param output: raster name to save on
    :return: None, tif file saved to disk
    ----------
    Example
        raster.toraster(filename, raster_obj.prediction, outname)
    ----------


Compute a weight matrix that assigns bigger weight on pixels in center and
    less weight to pixels on image boundary.
    This weight matrix then used for merging individual tile predictions and helps dealing
    with prediction artifacts on tile boundaries.
    :param width: Tile width
    :param height: Tile height
    :return: Since-channel image [Width x Height]


        Accumulates single element
        :param sample: Predicted image of shape [C,H,W]
        :param crop_coords: Corresponding tile crops w.r.t to original image


        Accumulates batch of tile predictions
        :param batch: Predicted tiles  of shape [B,C,H,W]
        :param crop_coords: Corresponding tile crops w.r.t to original image


Save TIFF or TIF files, normally used from raster files.


Drop multiple bands to existing rasterio object


    wU-Net for comparison
    Total params: 9,282,246


    :param rastarr: xarray or numpy array object in the form (c, h, w)
    :param bands: list with strings of bands in the raster
    :param indices: indices to calculate and append to the raster
    :param factor: factor used for toa imagery
    :return: raster with updated bands list


    :param data: xarray or numpy array object in the form (c, h, w)
    :param bands: list with strings of bands in the raster
    :param factor: factor used for toa imagery
    :return: new band with DVI calculated


    :param data: xarray or numpy array object in the form (c, h, w)
    :param bands: number of the original bands of the raster
    :param factor: factor used for toa imagery
    :return: new band with NDVI calculated


    GPU Support
    :param data: xarray or numpy array object in the form (c, h, w)
    :param bands: number of the original bands of the raster
    :param factor: factor used for toa imagery
    :return: new band with FDI calculated


    GPU Support
    :param data: xarray or numpy array object in the form (c, h, w)
    :param bands: number of the original bands of the raster
    :param factor: factor used for toa imagery
    :return: new band with SI calculated


    :param data: xarray or numpy array object in the form (c, h, w)
    :param bands: number of the original bands of the raster
    :param factor: factor used for toa imagery
    :return: new band with DWI calculated


    :param data: xarray or numpy array object in the form (c, h, w)
    :param bands: number of the original bands of the raster
    :param factor: factor used for toa imagery
    :return: new band with SI calculated


    :param data: xarray or numpy array object in the form (c, h, w)
    :param bands: number of the original bands of the raster
    :param factor: factor used for toa imagery
    :return: new band with CS2 calculated


self._setup method: setup gpu and logging methods for the entire pipeline.


Create output directories for model output.


        Extract small patches for final dataset
        Args:
            img (numoy array - c, y, x): imagery data
            tile_size (tuple): 2D dimensions of tile
            random_state (int): seed for reproducibility (match image and mask)
            n_patches (int): number of tiles to extract


        Stretch the image histogram for each channel independantly. The image histogram
        is streched such that the lower and upper percentile are saturated.
        ------------
        INPUT
            |---- img (3D numpy array) the image to stretch (H x W x B)
            |---- percentile (tuple) the two percentile value to saturate
            |---- out_range (tuple) the output range value
        OUTPUT
            |---- img_adj (3D numpy array) the streched image (H x W x B)


        Normalize to zero mean and unit standard deviation along the given axis
        Args:
            img (numpy or cupy): array (w, h, c)
            axis (integer tuple): into or tuple of width and height axis
            c (float): epsilon to bound given std value
        Return:
            Normalize single image
        ----------
        Example
        ----------
            image_normalize(arr, axis=(0, 1), c=1e-8)


        Normalize batch to zero mean and unit standard deviation.
        Args:
            img (numpy or cupy): array (n, w, h, c)
            axis (integer tuple): into or tuple of width and height axis
            c (float): epsilon to bound given std value
        Return:
            Normalize batch of images.
        ----------
        Example
        ----------
            batch_normalize(arr, axis=(0, 1), c=1e-8)


Dice coefficient as a loss function.


Predicted mask according to threshold


Removes noise from a mask.
        Args:
            mask: the mask to remove noise from.
            eps: the morphological operation's kernel size for noise removal, in pixel.
        Returns:
            The mask after applying denoising.


Grows a mask to fill in small holes, e.g. to establish connectivity.
        Args:
            mask: the mask to grow.
            eps: the morphological operation's kernel size for growing, in pixel.
        Returns:
            The mask after filling in small holes.


Show side-by-side an input image,
        the ground truth and the prediction.

        Given a tensorflow dataset:
        for image, mask in dataset['train'].take(40):
            sample_image, sample_mask = image, mask
            self.display_sample([sample_image[0], sample_mask[0]])


It adds a Batch_normalization layer before a Relu


It adds a feedforward signal to the output of two following conv layers in contracting path


It is exactly like the identity_block plus a dropout layer. This block only uses in the valley of the UNet


It Is only the convolution part inside each expanding path's block


It is for adding two feed forwards to the output of the two following conv layers in expanding path


It improves the skip connection by using previous layers feature maps


    Make a 1D window function, then infer and return a 2D window function.
    Done with an augmentation, and self multiplication with its transpose.
    Could be generalized to more dimensions.


    Add borders to img for a "valid" border pattern according to "window_size" and
    "subdivisions".
    Image is an np array of shape (x, y, nb_channels).


    Undo what's done in the `_pad_img` function.
    Image is an np array of shape (x, y, nb_channels).


    Create tiled overlapping patches.
    Returns:
        5D numpy array of shape = (
            nb_patches_along_X,
            nb_patches_along_Y,
            patches_resolution_along_X,
            patches_resolution_along_Y,
            nb_output_channels
        )
    Note:
        patches_resolution_along_X == patches_resolution_along_Y == window_size


Merge tiled overlapping patches smoothly.


    Apply the `pred_func` function to square patches of the image, and overlap
    the predictions to merge them smoothly.
    See 6th, 7th and 8th idea here:
    http://blog.kaggle.com/2017/05/09/dstl-satellite-imagery-competition-3rd-place-winners-interview-vladimir-sergey/


Extract data CSV files to extract tiles from (data and golden tiles)


Read files from given directory.


        Augment data for semantic segmentation.
        Args:
            image (numpy.array): image numpy array
            label (numpy.array): image numpy array
        Return:
            augmented image and label
        ----------
        Example
        ----------
            data_augment(image, label)


Generate tiles from each individual raster.


Return data callbacks.


    Get a md5 checksum of a file

    :param filename:
    :return:

    THis function is adapted from https://stackoverflow.com/a/24847608


    Download a file from Google Drive using it's ID

    :param _id: File id on google drive
    :param destination: Location to save file

    This function is adapted from https://stackoverflow.com/a/39225039


    Collects the analyses cases to be run and launches processes for each of
    them.

    Args:
        run_type: A string representation of the run type (eg. verification)
        module: The module corresponding to the run.  Must have a run_suite function
        config: The configuration for the module


Helper method to launch processes and sync output


    Argparse helper function to specify a zero or positive integer as an
    argument type
    :param integer: A zero or positive integer
    :return: integer


    Handles the parsing of options for LIVVkit's command line interface

    Args:
        args: The list of arguments, typically sys.argv[1:]


Initialize some defaults


Create a series of nested LIVVDicts given a list


Set the value of nested LIVVDicts given a list


Merge two dictionaries and return the result


    Read a GPTL timing file and extract some data.

    Args:
        file_path: the path to the GPTL timing file
        var_list: a list of strings to look for in the file

    Returns:
        A dict containing key-value pairs of the livvkit
        and the times associated with them


    Search for a file in a directory, and return the first match.
    If the file is not found return an empty string

    Args:
        search_dir: The root directory to search in
        file_pattern: A unix-style wildcard pattern representing
            the file to find

    Returns:
        The path to the file if it was found, otherwise an empty string


Read in a json file and return a dictionary representation


    Write out data to a json file.

    Args:
        data: A dictionary representation of the data to write out
        path: The directory to output the file in
        file_name: The name of the file to write out


Find all cases and subcases of a particular run type


The jinja2 HTML template

        An attribute or property which holds the jinja2 template used to
        represent the element as HTML.

        Returns:
            str: The jinja2 HTML template


The jinja2 LaTeX template

        An attribute or property which holds the jinja2 template used to
        represent the element as LaTeX.

        Returns:
            str: The jinja2 LaTeX template


Represent this element as JSON

        Using the internal dictionary representation of this element, return a
        JSON representation of this element

        Returns:
            str: The JSON representation of this element


Represent this element as HTML

        Using the jinja2 template defined by ``self._html_template``, return an
        HTML representation of this element

        Returns:
            str: The HTML representation of this element


Represent this element as LaTeX

        Using the jinja2 template defined by ``self._latex_template``, return an
        LaTeX representation of this element

        Returns:
            str: The LaTeX representation of this element


Add a reference to the internal reference list

        Args:
            references: The references to add to this page's internal reference
                list. This can be a path to a bibtex file containing the
                references, or a list/set/tuple of bibtex files containing the
                references (Note: This will include the default LIVVkit
                references and ALL references inside the bibtex file(s)!).


Perform the file diff

        Args:
            context: An positive int indicating the number of lines of context
                to display on either side of each difference found

        Returns:
            (tuple): Tuple containing:
                difference: A str containing either a git-style diff of the
                    files if a difference was found or the original file in
                    full
                diff_status: A boolean indicating whether any differences were
                    found


Run the full suite of performance tests


Generates statistics from the timing summaries


Show some statistics from the run


Trim out some data to return for the index page


Provide some top level information for the summary


    Parse all of the timing files, and generate some statistics
    about the run.

    Args:
        file_list: A list of timing files to parse
        var_list: A list of variables to look for in the timing file

    Returns:
        A dict containing values that have the form:
            [mean, min, max, mean, standard deviation]


    Generate data for plotting weak scaling.  The data points keep
    a constant amount of work per processor for each data point.

    Args:
        timing_stats: the result of the generate_timing_stats function
        scaling_var: the variable to select from the timing_stats dictionary
                     (can be provided in configurations via the 'scaling_var' key)
        data_points: the list of size and processor counts to use as data
                     (can be provided in configurations via the 'weak_scaling_points' key)

    Returns:
         A dict of the form:
            {'bench' : {'mins' : [], 'means' : [], 'maxs' : []},
             'model' : {'mins' : [], 'means' : [], 'maxs' : []},
             'proc_counts' : []}


    Generate data for plotting strong scaling.  The data points keep
    the problem size the same and varies the number of processors
    used to complete the job.

    Args:
        timing_stats: the result of the generate_timing_stats function
        scaling_var: the variable to select from the timing_stats dictionary
                     (can be provided in configurations via the 'scaling_var' key)
        data_points: the list of size and processor counts to use as data
                     (can be provided in configurations via the 'strong_scaling_points' key)

    Returns:
        A dict of the form:
            {'bench' : {'mins' : [], 'means' : [], 'maxs' : []},
             'model' : {'mins' : [], 'means' : [], 'maxs' : []},
             'proc_counts' : []}


    Generate a scaling plot.

    Args:
        timing_data: data returned from a `*_scaling` method
        title: the title of the plot
        ylabel: the y-axis label of the plot
        description: a description of the plot
        plot_file: the file to write out to

    Returns:
        an image element containing the plot file and metadata


    Description

    Args:
        timing_stats: a dictionary of the form
            {proc_count : {model||bench : { var : { stat : val }}}}
        scaling_var: the variable that accounts for the total runtime
        title: the title of the plot
        description: the description of the plot
        plot_file: the file to write the plot out to
    Returns:
        an image element containing the plot file and metadata


Run the full suite of numerics tests


Run the full suite of verification tests


Runs all of the verification checks on a particular case


    Checks whether the given files have bit for bit solution matches
    on the given variable list.

    Args:
        model_path: absolute path to the model dataset
        bench_path: absolute path to the benchmark dataset
        config: the configuration of the set of analyses

    Returns:
        A dictionary created by the elements object corresponding to
        the results of the bit for bit testing


Create a bit for bit plot


Run the full suite of validation tests


    Parse a CISM output log and extract some information.

    Args:
        file_path: absolute path to the log file

    Return:
        A dictionary created by the elements object corresponding to
        the results of the bit for bit testing


    Convert the CISM configuration file to a python dictionary

    Args:
        file_path: absolute path to the configuration file

    Returns:
        A dictionary representation of the given file


Add input and output filenames to the parser.
        The filename of the output reference file is the first argument.
        The filename of the input image list is the second argument

        Parameters
        ----------
        parser : argparse.parser


Optional arguments common to all plugged in modules

        Parameters
        ----------
        parser : argparse.parser


Create argparser and populate with arguments

        Parameters
        ----------
        parser : argparse.parser

        usage : str
            Usage string for command line

        conflict_handler : str
            Method of resolving conflicting argument names. Input to argparse

        Returns
        -------
        parser : argparse.parser


Read in and define all arguments from the command line as well
        as the config file


Loop over the parameters that are present in both the config file
        and the extra_optional_arguments function above. If the parameter
        is set in extra_optional_arguments, then use that value in the call
        to the reference file generator. If not, use the value in the config
        ile.


Create a log entry for an instance of the class, dependong on
        whether the logger has been defined yet or not


Check if files in file_col exists, and fill the column file_exists_col with True or False


check if the reduced input files are already running in batch (Condor), and set 'already_in_batch' to True or False


 pick the commands to be executed!  execute_cmds can
        contain an initial one-dimensional input array of True or
        False. set execute_cmds to False if file_exists or
        already_in_batch=False.  if force_redo, then execute even if
        file_exists, and throw error message if already_in_batch


cleaning up old output files


dummy call to batch mode to return reasonable values


only keep fits file that match basenamepattern


check which entries are primary strun commands.


Construct the reflabel commands, get the input files


Any arguments added here will give the option of overriding
        the default argument values present in the config file. To override,
        call these arguments from teh command line in the call to mkrefs.py


Call your module. The only requirement is that the output
        reference file is saved as self.args.outputreffilename

        mkrefs.py will supply the input files in self.inputimages['output_name'].
        This will be a list containing the filenames to use as input. The
        file types (e.g. dark, flat) associated with each filename are
        contained in self.inputimages['imtype']. From this, you can specify
        the appropriate file names in the call to your module.


Enable "-" shorthand when talking about pipeline steps. For example,
    "rate-" will indicate all pipeline steps up to and including ramp-fitting.
    For a given step_name-, return the list of step names to be run.

    Parameters
    ----------

    step_name : str
        Name of step name (including the trailing '-')

    instrument : str
        Name of JWST instrument

    Returns
    -------

    step_list : list
        List of step names up to and including the input step


Once the search for ssb commands contained within other ssb commands has
        been completed, go through the list of sets and activate the top level rows
        that contain other commands. Remember that there can be nested values here.
        (e.g. row 4 contains [0], row 3 contains [0,4], row 5 contains [0,3,4]) In this
        example, we want to set the contained_within value to -1 for row 5, as that
        single run of the pipeline will output products needed by rows 0, 3, and 4 also.

        Rows that do not contain the command for any other rows will have an empty
        set. These need to be set to -1 as well, so that they are run.


Once the repeat search has been completed, activate only one copy of each
        repeated entry.


Create a generator to use for searching for files

        Returns
        -------
        generator : generator
            Generator from os.walk


Given a list of files, their calibration state
        and the required calibration state, choose the best file
        with which to proceed

        Paramters
        ---------
        filelist : list
            List of fits filenames

        req : dict
            Dictionary of pipeline steps containing boolean entries to describe
            the required calibration state of the file

        current : dict
            Nested dictionary: for each filename (key), the value is a dictionary
            of pipeline steps containing boolean entires to describe the current
            calibration state of the file

        Returns
        -------
        usefile : str
            Name of the fits file to begin further processing with


Search for repeated entries in the table input to calib_prep, and remove them.
        In this case, a repeated entry means that the filename is the same, and that the
        ssbsteps are the same OR one set are contained exactly within another (with no
        extra intervening steps). In this case, a single call to strun can be used to
        create the requested data for both cases. The only thing that needs to be done is
        to provide an intermediate output name to capture both versions of the calibrated
        file.


Identify and return the pipeline steps completed
        for the input file

        Parameters
        ----------
        input_file : str
            Name of input fits file

        Returns
        -------
        completed : dict


Check to see if the given pipeline configuration file exists
        in the output directory. If not, make a copy from the files in
        the repo

        Parameters
        ----------
        filename : str
            Name of the cfg file to check for (e.g. calwebb_detector1.cfg)


Create the output name of the pipeline-processed
        input file given the list of required steps

        Parameters
        ----------
        base : str
            Base name of the fits file. Should be the filename minus '.fits'

        req : dict
            Dictionary of pipeline steps, with boolean entries describing whether
            those steps should be completed or not.

        Returns
        -------
        output_filename : str
            Full path + filename of the output once all the required pipeline steps
            have been run. Should be the true base name + the suffix associated with
            the latest pipeline step to be run.

        true_base : str
            Base name of the fits file. Should be the filename minus '.fits' as
            well as any pipeline-generated suffix


Search for all versions of a particular
        file in the given directory

        Parameters
        ----------
        base : str
            Basename of the file to search for. This should be the filename minus
            the .fits as well as any pipeline-created suffix.

        generator_object : obj
            Generator constructed from os.walk on all input search directories

        Returns
        -------
        files : list
            List of found files containing the input base string


Search for all versions of a particular
        file in the given directory

        Parameters
        ----------
        base : str
            Basename of the file to search for. This should be the filename minus
            the .fits as well as any pipeline-created suffix.

        dir : str
            Directory to search. The search will look down the directory tree
            into all subdirectories below this level.

        Returns
        -------
        files : list
            List of found files containing the input base string


Check to see if the given file already
        exists. Remove if the user allows it.

        Parameters
        ----------
        filename : str
            Name of file to look for


Translate the input list of required pipeline steps
        into a dictionary with boolean entries

        Parameters
        ----------
        stepstr : str
            Comma-separated list of pipeline step abbreviations. These come from
            the origina config file.

        Returns
        -------
        req : dict
            Dictionary of pipeline steps with boolean values designating whether
            each step was requested in stepstr


Return a list of the pipeline steps that need
        to be run to bring the input file up to the state
        described by req. Return a dictionary of boolean
        values just like the req and current inputs

        Parameters
        ----------
        infile : str
            Input fits filename

        req : dict
            Dictionary of pipeline steps with boolean entries descibing whether
            each step should be complete on the final output file

        current : dict
            Dictionary of pipeline steps with boolean entries describing whether
            each step has already been completed on infile

        Returns
        -------
        torun : dict
            Dictionary of pipeline steps with boolean entires descrbing which
            required steps must still be run on infile


Create the necessary strun command to run the
        appropriate JWST calibration pipeline steps

        Parameters
        ----------
        input_files : astropy.table.Column
            astropy table column object listing input fits filenames

        steps_to_run : astropy.table.Column
            astropy table column object giving a comma-separated list of pipeline
            steps to run for each file

        outfile_name : astropy.table.Column
            astropy table column object listing pipeline ouput fits file name

        steps_to_save : astropy.table.Column
            astropy table column object giving a comma-separated list of pipeline
            steps whose output should be saved

        instrument : str
            Name of instrument. Used to collect the appropriate pipeline steps

        testing : bool
            Passed to copy_config_to_output_dir. Used for pytests.

        Returns
        -------
        cmds : list
            List of command line calls to the JWST calibration pipeline with appropriate
            flags to run/skip requested steps


Create a list of x,y coordinates to create a grid
        of boxes across the face of the detector


Calculate the sigma-clipped stdev through the stack of array,
        within each box defined by the x and y coordinate values

        Parameters
        ----------
        array : numpy.ndarray
            3D array containing a stack of CDS images

        x : numpy.ndarray
            1D array containing x coordinate values corresponding to the
            edges of the grid of boxes

        y : numpy.ndarray
            1D array containing y coordinate values corresponding to the
            edges of the grid of boxes

        clipthresh : int
            Sigma-clipping threshold to use. Number of sigma. Default=3

        Returns
        -------
        readnoise : numpy.ndarray
            2D array containing a map of the readnoise across the detector


Main function

        Parameters
        ----------
        out_dir : str
            Directory in which to save output file


Given an integration (or exposure with multiple
        integrations) create a series of CDS images. Assume
        the input comes from a DMS-formatted file, in which
        the data are in a 4D array (integration, group, y, x)

        Parameters
        ----------
        ramp1 : numpy.ndarray
            4-dimensional array of dark signals

        Returns
        -------
        cds_ramp : numpy.ndarray
            3-dimensional cube of difference images


Save the readnoise frame using JWST datamodel


Calculates the sigma-clipped mean through a stack of images.

    Parameters
    ----------
    stack : numpy.ndarray
        A 3D stack of images.

    clipping_sigma : float
        Number of sigmas to use when sigma-clipping the input stack.

    max_clipping_iters : int
        Maximum number of iterations to use when sigma-clipping the input
        stack.

    Returns
    -------
    mean_image : numpy.ndarray
        2D image of the sigma-clipped mean through the input stack.


Calculates the sigma-clipped standard deviation through a stack
    of images.

    Parameters
    ----------
    stack : numpy.ndarray
        A 3D stack of images.

    clipping_sigma : float
        Number of sigmas to use when sigma-clipping the input stack.

    max_clipping_iters : int
        Maximum number of iterations to use when sigma-clipping the input
        stack.

    Returns
    -------
    stddev_image : numpy.ndarray
        2D image of the sigma-clipped standard deviation through the
        input stack.


Creates a stack of CDS images (group difference images) using the
    input ramp data, combining multiple integrations if necessary.

    Parameters
    ----------
    data : numpy.ndarray
        The input ramp data. The data shape is assumed to be a 4D array in
        DMS format (integration, group, y, x).

    group_diff_type : str
        The method for calculating group differences. Options are:
        ``independent``: Each group is only differenced once (e.g. 6-5, 4-3,
                         2-1)
        ``consecutive``: Each group is differenced to its neighbors (e.g.
                         4-3, 3-2, 2-1)

    Returns
    -------
    cds_stack : numpy.ndarray
        A 3D stack of the group difference images.


The main function. Creates a readnoise reference file using the input
    dark current ramps. See module docstring for more details.

    Parameters
    ----------
    filenames : list
        List of dark current files. These should be calibrated ramp images.
        The data shape in these images is assumed to be a 4D array in DMS
        format (integration, group, y, x).

    method : str
        The method to use when calculating the readnoise. Options are:
        ``stack``: Creates a master stack of all CDS images by combining the
                   CDS images for each ramp and integration together. The
                   final readnoise map is calculated by taking the stddev
                   through this master stack.
        ``ramp``: Calculates the readnoise in each ramp individually. The
                  final readnoise map is calculated by averaging these
                  individual readnoise maps together.

    group_diff_type : str
        The method for calculating group differences. Options are:
        ``independent``: Each group is only differenced once (e.g. 6-5, 4-3,
                         2-1).
        ``consecutive``: Each group is differenced to its neighbors (e.g.
                         4-3, 3-2, 2-1).

    clipping_sigma : float
        Number of sigma to use when sigma-clipping.

    max_clipping_iters : int
        Maximum number of iterations to use when sigma-clipping.

    nproc : int
        The number of processes to use during multiprocessing.

    slice_width : int
        The width (in pixels) of the image slice to use during
        multiprocessing. The readnoise of each slice is calculated separately
        during multiprocessing and combined together at the end of
        processing. Only relevant if method==stack.

    single_value : bool
        Option to use a single readnoise value (the average of all readnoise
        values) for all pixels.

    outfile : str
        Name of the CRDS-formatted readnoise reference file to save the final
        readnoise map to.

    author : str
        CRDS-required name of the reference file author, to be placed in the
        referece file header.

    description : str
        CRDS-required description of the reference file, to be placed in the
        reference file header.

    pedigree : str
        CRDS-required pedigree of the data used to create the reference file.

    useafter : str
        CRDS-required date of earliest data with which this referece file
        should be used. (e.g. '2019-04-01T00:00:00').

    history : str
        CRDS-required history section to place in the reference file header.

    subarray : str
        CRDS-required subarray for which to use this reference file for.

    readpatt : str
        CRDS-required read pattern for which to use this reference file for.

    save_tmp : bool
        Option to save the final readnoise map before turning it into CRDS
        format. This is useful if the CRDS transformation fails; in this
        scenario, you won't lose all of the final readnoise results so all
        of the previous processing wasn't for nothing.


If value is a string containing 'none', then change it to a
    NoneType object.

    Parameters
    ----------
    value : str or NoneType

    Returns
    -------
    new_value : NoneType


Calculates the readnoise for the given input dark current ramp.

    Parameters
    ----------
    filename : str
        The dark current ramp file. The data shape in this image is assumed
        to be a 4D array in DMS format (integration, group, y, x).

    group_diff_type : str
        The method for calculating group differences. Options are:
        ``independent``: Each group is only differenced once (e.g. 6-5, 4-3,
                         2-1)
        ``consecutive``: Each group is differenced to its neighbors (e.g.
                         4-3, 3-2, 2-1)

    clipping_sigma : float
        Number of sigma to use when sigma-clipping.

    max_clipping_iters : int
        Maximum number of iterations to use when sigma-clipping.


Calculates the readnoise for a given slice in the input dark file
    ramps. Useful for multiprocessing and avoiding memory issues for large
    image stacks.

    Parameters
    ----------
    filenames : list
        List of dark current files. The data shape in these images is assumed
        to be a 4D array in DMS format (integration, group, y, x).

    group_diff_type : str
        The method for calculating group differences. Options are:
        ``independent``: Each group is only differenced once (e.g. 6-5, 4-3,
                         2-1)
        ``consecutive``: Each group is differenced to its neighbors (e.g.
                         4-3, 3-2, 2-1)

    clipping_sigma : float
        Number of sigma to use when sigma-clipping.

    max_clipping_iters : int
        Maximum number of iterations to use when sigma-clipping.

    column : int
        The index of the starting image column for each slice.

    slice_width : int
        The width of the slice in pixels.

    Returns
    -------
    readnoise : numpy.ndarray
        2D image of the calculated readnoise.


Saves a CRDS-formatted readnoise reference file.

    Parameters
    ----------
    readnoise : numpy.ndarray
        The 2D readnoise image.

    instrument : str
        CRDS-required instrument for which to use this reference file for.

    detector : str
        CRDS-required detector for which to use this reference file for.

    subarray : str
        CRDS-required subarray for which to use this reference file for.

    readpatt : str
        CRDS-required read pattern for which to use this reference file for.

    outfile : str
        Name of the CRDS-formatted readnoise reference file to save the final
        readnoise map to.

    author : str
        CRDS-required name of the reference file author, to be placed in the
        referece file header.

    description : str
        CRDS-required description of the reference file, to be placed in the
        reference file header.

    pedigree : str
        CRDS-required pedigree of the data used to create the reference file.

    useafter : str
        CRDS-required date of earliest data with which this referece file
        should be used. (e.g. '2019-04-01T00:00:00').

    history : str
        CRDS-required history section to place in the reference file header.

    fastaxis : int
        CRDS-required fastaxis of the reference file.

    slowaxis : int
        CRDS-required slowaxis of the reference file.

    substrt1 : int
        CRDS-required starting pixel in axis 1 direction.

    substrt2 : int
        CRDS-required starting pixel in axis 2 direction.

    filenames : list
        List of dark current files that were used to generate the reference
        file.


Replaces all readnoise image values with the average of all readnoise
    values.

    Parameters
    ----------
    readnoise : numpy.ndarray
        The 2D readnoise image.

    instrument : str
        CRDS-required instrument for which to use this reference file for.

    detector : str
        CRDS-required detector for which to use this reference file for.

    subarray : str
        CRDS-required subarray for which to use this reference file for.

    clipping_sigma : float
        Number of sigma to use when sigma-clipping.

    max_clipping_iters : int
        Maximum number of iterations to use when sigma-clipping.

    Returns
    -------
    readnoise_single_value : numpy.ndarray
        2D readnoise image where all of the readnoise values are the same
        (the average of all of the original readnoise values).


A wrapper around the readnoise_by_ramp function to allow for
    multiprocessing.

    Parameters
    ----------
    args : tuple
        A tuple containing the input arguments for the readnoise_by_ramp
        function. See readnoise_by_ramp docstring for more details.

    Returns
    -------
    readnoise : numpy.ndarray
        2D image of the calculated readnoise (i.e. the output from the
        readnoise_by_ramp function).


A wrapper around the readnoise_by_slice function to allow for
    multiprocessing.

    Parameters
    ----------
    args : tuple
        A tuple containing the input arguments for the readnoise_by_slice
        function. See readnoise_by_slice docstring for more details.

    Returns
    -------
    readnoise : numpy.ndarray
        2D image of the calculated readnoise (i.e. the output from the
        readnoise_by_slice function).


Any arguments added here will give the option of overriding
        the default argument values present in the config file. To override,
        call these arguments from the command line in the call to mkrefs.py


Call the readnoise algorithm. The only requirement is that the output
        reference file is saved as self.args.outputreffilename

        mkrefs.py will supply the input files in self.inputimages['output_name'].
        This will be a list containing the filenames to use as input. The
        file types (e.g. dark, flat) associated with each filename are
        contained in self.inputimages['imtype']. From this, you can specify
        the appropriate file names in the call to your module.


Identifies pixels with high superbias error values (i.e.
    UNRELIABLE_BIAS pixels).

    Parameters
    ----------
    error : numpy.ndarray
        A 2D superbias error image.

    bad_clipping_sigma : float
        Number of sigma to use when sigma-clipping the superbias error
        image to find UNRELIABLE_BIAS pixels.

    bad_max_clipping_iters : int
        Maximum number of iterations to use when sigma-clipping the superbias
        error image to find UNRELIABLE_BIAS pixels.

    bad_sigma_threshold : float
        Number of standard deviations above the mean of the superbias error
        image at which a pixel is flagged as UNRELIABLE_BIAS.

    plot : bool
        Option to save a histogram of the superbias error image values, with
        the threshold used for flagging UNRELIABLE_BIAS pixels also shown.

    Returns
    -------
    dq : numpy.ndarray
        A 2D DQ image that flags pixels with high superbias error values as
        UNRELIABLE_BIAS (2) and DO_NOT_USE (1).


Create a version of the superbias image where the values of high
    superbias pixels are replaced by their smoothed values.

    Parameters
    ----------
    superbias : numpy.ndarray
        A 2D superbias image.

    refpix_map : numpy.ndarray
        A 2D image where refpix have a value of 1 and all other pixels have
        a value of 0.

    bad_clipping_sigma : float
        Number of sigma to use when sigma-clipping to find UNRELIABLE_BIAS
        pixels.

    bad_max_clipping_iters : int
        Maximum number of iterations to use when sigma-clipping to find
        UNRELIABLE_BIAS pixels.

    bad_sigma_threshold : float
        Number of standard deviations above the mean at which a pixel is
        flagged as UNRELIABLE_BIAS.

    box_width : int
        Width in pixels of the box kernel to use to compute the smoothed
        superbias image.

    outfile : str
        Name of the CRDS-formatted superbias reference file to save the final
        superbias map to. This is only used in this function to name the
        filled output file.


Creates a map of the reference pixels based on the input fits file.

    Parameters
    ----------
    filename : str
        The fits image file to create the refpix map from. The PIXELDQ
        extension is used to find the refpix.

    Returns
    -------
    refpix_map : numpy.ndarray
        A map of the reference pixels (same dimensions as the input fits file
        image extensions), where refpix are 1 and all other pixels are 0.
        An array of all zeros is returned if no refpix are found or if no
        PIXELDQ extension exists.


The main function. Creates a superbias reference file using the input
    dark current ramps. See module docstring for more details.

    Parameters
    ----------
    filenames : list
        List of dark current ramp files. The data shape in these images is
        assumed to be a 4D array in DMS format (integration, group, y, x).

    clipping_sigma : float
        Number of sigma to use when sigma-clipping through the 0th frame
        stack to create the superbias image.

    max_clipping_iters : int
        Maximum number of iterations to use when sigma-clipping through
        the 0th frame stack to create the superbias error image.

    bad_clipping_sigma : float
        Number of sigma to use when sigma-clipping the superbias error image
        to find UNRELIABLE_BIAS pixels.

    bad_max_clipping_iters : int
        Maximum number of iterations to use when sigma-clipping the superbias
        error image to find UNRELIABLE_BIAS pixels.

    bad_sigma_threshold : float
        Number of standard deviations above the mean of the superbias error
        image at which a pixel is flagged as UNRELIABLE_BIAS.

    plot : bool
        Option to save a histogram of the superbias error image values, with
        the threshold used for flagging UNRELIABLE_BIAS pixels also shown.

    save_filled : bool
        Option to save an image where high superbias pixels are replaced by
        their smoothed values.

    box_width : int
        Width in pixels of the box kernel to use to compute the smoothed
        superbias image. Only relevant if save_filled=True.

    save_reffile : bool
        Option to save the generated superbias map into a CRDS-formatted
        reference file.

    outfile : str
        Name of the CRDS-formatted superbias reference file to save the final
        superbias map to.

    author : str
        CRDS-required name of the reference file author, to be placed in the
        referece file header.

    description : str
        CRDS-required description of the reference file, to be placed in the
        reference file header.

    pedigree : str
        CRDS-required pedigree of the data used to create the reference file.

    useafter : str
        CRDS-required date of earliest data with which this referece file
        should be used (e.g. '2019-04-01T00:00:00').

    history : str
        CRDS-required history section to place in the reference file header.

    subarray : str
        CRDS-required subarray for which to use this reference file for.

    readpatt : str
        CRDS-required read pattern for which to use this reference file for.


Saves a CRDS-formatted superbias reference file.

    Parameters
    ----------
    superbias : numpy.ndarray
        The 2D superbias image.

    error : numpy.ndarray
        The 2D superbias error image.

    dq : numpy.ndarray
        The 2D superbias data quality image.

    instrument : str
        CRDS-required instrument for which to use this reference file for.

    detector : str
        CRDS-required detector for which to use this reference file for.

    subarray : str
        CRDS-required subarray for which to use this reference file for.

    readpatt : str
        CRDS-required read pattern for which to use this reference file for.

    outfile : str
        Name of the CRDS-formatted superbias reference file to save the final
        superbias map to.

    author : str
        CRDS-required name of the reference file author, to be placed in the
        referece file header.

    description : str
        CRDS-required description of the reference file, to be placed in the
        reference file header.

    pedigree : str
        CRDS-required pedigree of the data used to create the reference file.

    useafter : str
        CRDS-required date of earliest data with which this referece file
        should be used. (e.g. '2019-04-01T00:00:00').

    history : str
        CRDS-required history section to place in the reference file header.

    fastaxis : int
        CRDS-required fastaxis of the reference file.

    slowaxis : int
        CRDS-required slowaxis of the reference file.

    substrt1 : int
        CRDS-required starting pixel in axis 1 direction.

    substrt2 : int
        CRDS-required starting pixel in axis 2 direction.

    filenames : list
        List of dark current files that were used to generate the reference
        file.


Create a default output filename for the bad pixel mask given
    instrument and detector names

    Parameters
    ----------
    inst_name : str
        Instrument name

    det_name : str
        Detector name

    Returns
    -------
    outfile : str
        Default bad pixel mask filename


Get the instrument and detector name from the header of the
    input file

    Parameters
    ----------
    filename : str
        Name of fits file

    Returns
    -------
    inst : str
        Instrument name

    det : str
        Detector name


Save a bad pixel map into a CRDS-formatted reference file

    Parameters
    ----------
    bad_pix_map : numpy.ndarray
        2D bad pixel array

    instrument : str
        Name of instrument associated with the bad pixel array

    detector : str
        Name of detector associated with the bad pixel array

    hdulist : astropy.fits.HDUList
        HDUList containing "extra" fits keywords

    files : list
        List of files used to create ``bad_pix_map``

    author : str
        Author of the bad pixel mask reference file

    description : str
        CRDS description to use in the final bad pixel file

    pedigree : str
        CRDS pedigree to use in the final bad pixel file

    useafter : str
        CRDS useafter string for the bad pixel file

    history_text : list
        List of strings to add as HISTORY entries to the bad pixel file

    outfile : str
        Name of the output bad pixel file


MAIN SCRIPT: Given a set of input files, create dead, low QE, and
    open pixel maps

    Parameters
    ----------
    input_files : list
        Fits files to be used in the search for dead/open pixels

    dead_search : bool
        Whether or not to search for DEAD pixels

    low_qe_and_open_search : bool
        Whether or not to search for LOW_QE, OPEN, and ADJ_OPEN pixels

    dead_search_type : str
        Type of search to use when looking for dead pixels. Options are:
        ``sigma_rate``: Using a normalized signal rate image, dead pixels
                        are defined as those with a rate smaller than
                        ``dead_sigma_threshold`` standard deviations below
                        the mean.
        ``absolute_rate``: Using a normalized signal rate image, dead pixels
                           are defined as those with a rate less than
                           ``max_dead_norm_signal``.

    sigma_threshold : float
        Number of standard deviations to use when sigma-clipping to
        calculate the mean slope image or the mean across the detector

    normalization_method : str
        Specify how the mean image is normalized prior to searching for
        bad pixels. Options are:
        'smoothed': Mean image will be smoothed using a
                    ``smoothing_box_width`` x ``smoothing_box_width``
                    box kernel. The mean image is then normalized by
                    this smoothed image.
        'none': No normalization is done. Mean slope image is used as is
        'mean': Mean image is normalized by its sigma-clipped mean

    smoothing_box_width : float
        Width in pixels of the box kernel to use to compute the smoothed
        mean image

    smoothing_typ : string
        Type of smoothing to do ``Box2D `` or ``median`` filtering

    smoothing_sigma : float
        Number of standard deviations to use when smoothing in a box defined
        by smoothing_box_width.

    dead_sigma_threshold : float
        Number of standard deviations below the mean at which a pixel is
        considered dead.

    max_dead_norm_signal : float
        Maximum normalized signal rate of a pixel that is considered dead

    run_dead_flux_check : bool
        Whether or not to check for dead pixels using an absolute flux value

    dead_flux_check_files : list
        List of ramp (uncalibrated) files to use to check the flux of average
        of last 4 groups. If None then the uncalibration files are not read in
        and no flux_check is done.

    flux_check: float
        Tolerance on average signal in last 4 groups. If dead_flux_check_files is
        a list of uncalibrated files, then the average of the last four groups
        for all the integrations is determined. If this average > flux_check
        then this pixel is not a dead pixel.

    max_low_qe_norm_signal: float
        The maximum normalized signal a pixel can have and be considered
        low QE.

    max_open_adj_norm_signal : float
        The maximum normalized signal a pixel adjacent to a low QE pixel can have
        in order for the low QE pixel to be reclassified as OPEN

    manual_flag_file : str
        Name of the ascii file containing a list of pixels to be added manually
        to the output bad pixel mask file. Default is 'default', in which case
        the file contained in the ``bad_pixel_mask`` directory of the repo will
        be used.

    do_not_use : list
        List of bad pixel types where the DO_NOT_USE flag should also be
        applied (e.g. ['DEAD', 'LOW_QE'])

    output_file : str
        Name of the CRDS-formatted bad pixel reference file to save the final
        bad pixel map into

    author : str
        CRDS-required name of the reference file author, to be placed in the
        referece file header

    description : str
        CRDS-required description of the reference file, to be placed in the
        reference file header

    pedigree : str
        CRDS-required pedigree of the data used to create the reference file

    useafter : str
        CRDS-required date of earliest data with which this referece file
        should be used. (e.g. '2019-04-01 00:00:00')

    history : str
        Text to be added to the HISOTRY section of the output bad pixel file

    quality_check : bool
        If True, the pipeline is run using the output reference file to be
        sure the pipeline doens't crash

    Returns
    -------
    final_map : numpy.ndarray
        2D array containing the combined bad pixel map


Given multiple separate bad pixel maps, combine using bit
    values matching those defined in the JWST calibraiton pipeline

    Parameters
    ----------
    bad_maps : dict
        Dictionary containing the various individual bad pixel maps to be
        combined. Format is (e.g.) {'DEAD': dead_map, 'LOW_QE': low_qe_map}

    Returns
    -------
    final_map : numpy.ndarray
        2D array containing the combined bad pixel map


Create the DQ definition data needed to populate the final bad pixel
    reference file

    Returns
    -------
    definitions : list
        Bad pixel bit definitions


Find the default manual bad pixel file for the given instrument
    and detector

    Parameters
    ----------
    inst_name : str
        Name of JWST instrument

    detector_name : str
        Name of detector

    Returns
    -------
    filename : str
        Full path and filename of the bad pixel file


Check that the flags in the input list are valid JWST bad pixel types

    Parameters
    ----------
    flag : list
        List of flag types (e.g. ['DEAD', 'LOW_QE'])

    Returns
    -------
    flag : list
        Modified list (all flags made uppercase)


Perform some basic checks on the input values to be placed in the
    header of the reference file. These check against CRDS requirements

    Parameters
    ----------

    Results
    -------


Create a fits HDU List to contain the "extra" fits header keywords
    that specific to this module and not included in the datamodels. This
    will be used to initialize the datamodel when creating the reference
    file

    Parameters
    ----------
    sigma_threshold : float
        Number of standard deviations to use when sigma-clipping to
        calculate the mean slope image or the mean across the detector

    smoothing_width : float
        Width in pixels of the box kernel to use to compute the smoothed
        mean image

    dead_sigma_threshold : float
        Number of standard deviations below the mean at which a pixel is
        considered dead.

    dead_max_rate : float
        Maximum normalized signal rate of a pixel that is considered dead

    low_qe_max_rate: float
        The maximum normalized signal a pixel can have and be considered
        low QE.

    open_adj_max_rate: float
        The maximum normalized signal a pixel adjacent to a low QE pixel can have
        in order for the low QE pixel to be reclassified as OPEN

    Returns
    -------
    hdu_list : astropy.io.fits.HDUList
        HDU List containing a header with just the extra keywords specific
        to this module


Create a map of dead pixels given a normalized rate image. In this
    case pixels are flagged as dead if their normalized signal rate is
    less than ``sigma`` standard deviations below the mean

    Parameters
    ----------
    rate_image : numpy.ndarray
        2D normalized rate image

    mean_rate : float
        Sigma-clipped mean value of the ``rate_image``

    stdev_rate : float
        Sigma-clipped standard deviation of the ``rate_image``

    sigma : float
        Number of standard deviations below the mean at which a pixel is
        considered dead.

    Returns
    -------
    dead_pix_map : numpy.ndarray
        2D map showing DEAD pixels. Good pixels have a value of 0.


Create a map of dead pixels given a normalized rate image. In this
    case pixels are flagged as dead if their normalized signal rate is
    less than ``max_dead_signal``.

    Parameters
    ----------
    rate_image : numpy.ndarray
        2D normalized rate image

    max_dead_signal : float
        Maximum normalized signal rate of a pixel that is considered dead

    Returns
    -------
    dead_pix_map : numpy.ndarray
        2D map showing DEAD pixels. Good pixels have a value of 0.


Check the dead_pix_map with the average_rate of the last 4 groups.
    We want to remove cases where a pixel was flagged as dead because
    it is a hot pixel that saturates on the first group causing the
    rate for this pixel to be close to 0.

    Parameters
    ----------
    dead_pix_map : numpy.ndarray
        2D map showing DEAD pixels. Good pixels have a value of 0.

    ave_group : numpy.ndaarray
        2D array of the average signal in the last 4 groups from
        an average of all integrations and exposures.

    flux_check : float
        if pixel have a ave_group > flux_check this is not a dead pixel

    Returns
    -------
    clean_dead_pix_map : numpy.ndarray
        2D map showing DEAD pixels. Good pixels have a value of 0.
        Clean up dead pixel map removing saturated pixels from dead
        flag.


Keep only the 10th group from each integration

    Parameters
    ----------
    data : numpy.ndarray
        3D or 4D array of data

    Returns
    -------
    data : numpy.ndarray
        3D array (10th group only)


Find the median of all the  group from each integration

    Parameters
    ----------
    data : numpy.ndarray
        3D or 4D array of data

    Returns
    -------
    data : numpy.ndarray
        3D array (10th group only)


Create maps of open (and adjacent to open) and low QE pixels given a
    normalized rate image

    Parameters
    ----------
    rate_image : numpy.ndarray
        2D normalized rate image

    mean_rate : float
        Sigma-clipped mean value of the ``rate_image``

    stdev_rate : float
        Sigma-clipped standard deviation of the ``rate_image``

    max_dead_signal : float
        The maximum normalized signal a pixel can have and be considered dead

    max_low_qe : float
        The maximum normalized signal a pixel can have and be considered
        low QE.

    max_adj_open : float
        The maximum normalized signal a pixel adjacent to a low QE pixel can have
        in order for the low QE pixel to be reclassified as OPEN

    Returns
    -------
    low_qe_map : numpy.ndarray
        2D map showing LOW QE pixels. Good pixels have a value of 0.

    open_pix_map : numpy.ndarray
        2D map showing OPEN pixels. Good pixels have a value of 0.

    adj_pix_map : numpy.ndarray
        2D map showing ADJ_OPEN pixels. Good pixels have a value of 0.


Calculate the sigma-clipped mean and standard deviation across the
    given image

    Parameters
    ----------
    image : numpy.ndarray
        2D image

    sigma : float
        Sigma threshold to use for sigma-clipping

    Returns
    -------
    mean_val : float
        Sigma-clipped mean value

    stdev_val : float
        Sigma-clipped standard deviation


Create a bad pixel mask from an input ascii file containing pixel
    indicies and flag types

    Parameters
    ----------
    filename : str
        Name of ascii file containing flags. File must contain columns
        'x', 'y', 'flags'

    map_shape : tup
        Shape of the bad pixel mask to create (y, x)

    no_use : list
        List of mnemonics to flag as DO_NOT_USE

    Returns
    -------
    mask : numpy.ndarray
        2D array of bad pixels, with bit values equal to the standard
        JWST values


Calculate the sigma-clipped mean and stdev images from a stack of
    input images

    Parameters
    ----------
    data : numpy.ndarray
        Stack of 2D images (i.e 3D array)

    sigma : float
        Threshold value to use for sigma clipping.

    Returns
    -------
    mean_image : numpy.ndarray
        2D array of the mean image

    stdev_image : numpy.ndarray
        2D array of the stdev image


Create a map that flags the shorted columns on the MIRI detector

    Parameters
    ----------
    dimensions : tup
        (y, x) dimensions, in pixels, of the map to create

    Returns
    -------
    shorted_map : numpy.ndarray
        2D map showing the locations of the shorted columns (1)


If value is a string containing 'none', then change it to a
    NoneType object

    Parameters
    ----------
    value : str or NoneType

    Returns
    -------
    new_value : NoneType


Fit the rate image with a 2-D surface plot

    Parameters
    ----------
    data : numpy.ndarray
        2D image

    deg: degree of polynomial fit

    Returns
    -------
    surface_image : numpy.ndarray
       2D image of polynomial fit


Pad the given image with an outer 4 rows and columns of (zeroed out)
    reference pixels.

    Parameters
    ----------
    data : numpy.ndarray
        2D image

    instrument_name : str
        Name of the JWST instrument associated with the data

    Returns
    -------
    padded : numpy.ndarray
        2D image with 4 rows and columns of reference pixels added


Run the dq_init step using the provided file, to check that the
    pipeline runs successfully

    Parameters
    ----------
    reference_filename : str
        Name of the bad pixel mask reference filename

    filename : str
        Name of the fit exposure on which to test the reference file


Read the data in the input files. Perform basic sanity checks
    to be sure data are consistent (same instrument, aperture, etc)

    Parameters
    ----------
    filenames : list
        List of fits filenames to be opened

    dead_search_type : str
        Type of search to use when looking for dead pixels. Options are:
        ``sigma_rate``: Using a normalized signal rate image, dead pixels
                        are defined as those with a rate smaller than
                        ``dead_sigma_threshold`` standard deviations below
                        the mean.
        ``absolute_rate``: Using a normalized signal rate image, dead pixels
                           are defined as those with a rate less than
                           ``max_dead_norm_signal``.
         ``saturation_check``: this option is set by the program if ``dead_flux_check_files``
                               is a list of ramp files. The average of the last four
                               groups is determined to test if pixel is close to saturation.

    Returns
    -------
    data : numpy.ndarray
        3D stack of data


Read in the text file containing the list of pixels to be flagged
    manually.

    Parameters
    ----------
    filename : str
        Name of ascii file to be read in

    Returns
    -------
    bad_table : astropy.table.Table
        Table of bad pixels


Create a map that flags all reference pixels as such

    Parameters
    ----------
    dimensions : tup
        (y, x) dimensions, in pixels, of the map to create

    instrument_name : str
        Name of JWST instrument associated with the data

    Returns
    -------
    ref_map : numpy.ndarray
        2D map showing the locations of reference pixels (1)


Turn a string containing a possible shorthand list of indexes
    into an array of the specified indexes. Support the notation that
    if no beginning or ending index is given that the range will be to
    the start or end of ``total_dimension``. Note that this function
    assumes that the input strings do not follow the python convention
    of using a maximum index value of one beyond the index you really
    want. (e.g. it assumes that '0:3' means columns 0 through 3 inclusive
    are what you are interested in (meaning indexing in python would
    need to use [0:4] to get all the values.))

    Parameters
    ----------
    table_cell : str or int
        Value to process (e.g. '34:36' or 93)

    total_dimension : int
        The maximum index number to be returned


Save a bad pixel map into a CRDS-formatted reference file

    Parameters
    ----------


Given a full frame image, strip off the reference pixels and return
    only the science pixels. At the moment, assume 4 rows and columns of
    reference pixels

    Parameters
    ----------
    data : numpy.ndarray
        2D image

    instrument_name : str
        Name of JWST instrument associated wit the data

    Returns
    -------
    data : numpy.ndarray
        2D image with outer 4 rows and columns removed


Create a smoothed version of the 2D input data

    Parameters
    ----------
    data : numpy.ndarray
        2D array of data to be smoothed

    box_width : int
        Width of the smoothing box, in pixels

    sigma: float
        value to use for sigma clipping

    Returns
    -------
    smoothed : numpy.ndarray
        A smoothed version of ``data``


Convert the indicies in a given row of the manual bad pixel ascii
    file into integers

    Parameters
    ----------
    line : astropy.table.row.Row
        Row containing x, y, and flag values

    colname : str
        Column name to examine

    Returns
    -------
    values : list
        If the input is an integer, the list will simply be [value]. If
        a range is given (e.g. 200:210), then values will be [200, 210].


Call the bpm algorithm. The only requirement is that the output
        reference file is saved as self.args.outputreffilename

        mkrefs.py will supply the input files in self.inputimages['output_name'].
        This will be a list containing the filenames to use as input. The
        file types (e.g. dark, flat) associated with each filename are
        contained in self.inputimages['imtype']. From this, you can specify
        the appropriate file names in the call to your module.


Make sure that all input files have the same dimensions,
        number of groups, etc


MAIN FUNCTION
        Creates a dark current reference file from a list of input dark
        current exposures


Read in groupdq extension from jump files, extract only the
        JUMP_DET flags, and collapse into an array that lists the group
        number of the first CR hit for each pixel in each integration

        Returns
        -------
        hit_indexes : dict
            key values are the filenames associated with the data. Each
            value is a 3D array (integ, y, x) of group numbers of CR hits


Save the given data in the official reference file format

        Parameters
        ----------
        data : numpy.ndarray
            3D array containing the mean dark current signals

        uncertainty : numpy.ndarray
            3D array containing associated uncertainties

        dq : numpy.ndarray
            2D array containing associated data quality flags


MAIN FUNCTION

    Parameters
    ----------
    filenames : list
        List of dark current slope files. These should be slope images.

    uncal_filenames : list
        List of uncal files. Should have a 1-to-1 correspondence to the
        files in ``filenames``. If None, the scipt will look in the same
        directory containing ``filenames``, and assume that the only
        difference in filename is that rate.fits is replaced with
        uncal.fits. Uncal files are only used when working with MIRI
        data.

    jump_filenames : list
        List of exposures output from the jump step of the pipeline.
        Should have a 1-to-1 correspondence to the
        files in ``filenames``. If None, the scipt will look in the same
        directory containing ``filenames``, and assume that the only
        difference in filename is that rate.fits is replaced with
        jump.fits

    fitopt_filenames : list
        List of exposures from the optional output from the ramp_fitting
        step of the pipeline. Should have a 1-to-1 correspondence to the
        files in ``filenames``. If None, the scipt will look in the same
        directory containing ``filenames``, and assume that the only
        difference in filename is that rate.fits is replaced with
        fitopt.fits

    clipping_sigma : int
        Number of sigma to use when sigma-clipping the 2D array of
        standard deviation values from the dark current slope files.
        The sigma-clipped mean and standard deviation are used to locate
        noisy pixels.

    max_clipping_iters : int
        Maximum number of iterations to use when sigma clipping to find
        the mean and standard deviation values that are used when
        locating noisy pixels.

    noisy_threshold : int
        Number of sigma above the mean noise (associated with the slope)
        to use as a threshold for identifying noisy pixels.

    max_saturated_fraction : float
        When identifying pixels that are fully saturated (in all groups
        of an integration), this is the fraction of integrations within
        which a pixel must be fully saturated before flagging it as HOT

    max_jump_limit : int
        The maximum number of jumps a pixel can have in an integration
        before it is flagged as a ``high jump`` pixel (which may be
        flagged as noisy later)

    jump_ratio_threshold : int
        Cutoff for the ratio of jumps early in the ramp to jumps later in
        the ramp. Pixels with a ratio greater than this value (and which
        also have a high total number of jumps) will be flagged as
        potential (I)RC pixels.

    early_cutoff_fraction : float
        Fraction of the integration to use when comparing the jump rate
        early in the integration to that across the entire integration.
        Must be <= 0.5

    pedestal_sigma_threshold : int
        Used when searching for RC pixels via the pedestal image. Pixels
        with pedestal values more than ``pedestal_sigma_threshold`` above
        the mean are flagged as potential RC pixels

    rc_fraction_threshold : float
        Used when searching for RC pixels. This is the fraction of input
        files within which the pixel must be identified as an RC pixel
        before it will be flagged as a permanent RC pixel

    low_pedestal_fraction : float
        This is the fraction of input files within which a pixel must be
        identified as a low pedestal pixel before it will be flagged as
        a permanent low pedestal pixel

    high_cr_fraction : float
        This is the fraction of input files within which a pixel must be
        flagged as having a high number of jumps before it will be flagged
        as permanently noisy

    flag_values : dict
        This dictionary maps the types of bad pixels searched for to the
        flag mnemonics to use when creating the bad pixel file. Keys are
        the types of bad pixels searched for, and values are lists that
        include mnemonics recognized by the jwst calibration pipeline
        e.g. {'hot': ['HOT'], 'rc': ['RC'], 'low_pedestal': ['OTHER_BAD_PIXEL'], 'high_cr': ["TELEGRAPH"]}

    do_not_use : list
        List of bad pixel types to be flagged as DO_NOT_USE
        e.g. ['hot', 'rc', 'low_pedestal', 'high_cr']

    plot : bool
        If True, produce plots of intermediate results.

    outfile : str
        Name of fits file to save the resulting bad pixel mask to


Place ``map`` within a larger array that contains the reference
    pixels.

    Parameters
    ----------
    array : numpy.ndarray
        2D array of bad pixels that does not contain reference pixels

    to_add : tup
        4-element tuple containing the number of rows and columns to
        add around the outside of the science pixels.
        (left cols, right cols, bottom rows, top rows)

    Returns
    -------
    array : numpy.ndarray
        2D array with rows and columns added


Beginning with a map indicating locations of a particular type of
    bad pixel, apply the bits specified in ``flag_list`` to come up with
    the ``jwst`` bad pixel value.

    Parameters
    ----------
    pixmap : numpy.ndarray
        2D array indicating bad pixels. 1 for a bad pixel and 0 for a
        good pixel

    flag_list : list
        List of bad pixel mnemonics to be applied. These mnemonics must
        be in the dictionary of bad pixel types recognized by the jwst
        calibration pipeline

    Returns
    -------
    pixmap : numpy.ndarray
        Map updated with proper bad pixel values


Basic metadata check for consistency from one file to another

    Parameters
    ----------
    hdr : astropy.fits.header
        Header read in from fits file

    comp : astropy.fits.header
        Baseline header to which the comparison is done


Copmbine individual maps of bad pixel types into a final bad pixel
    map, using flag values defined in ``dq_flags``.

    Parameters
    ----------
    sat_map : numpy.ndarray
        2D array giving the location of pixels saturated all the way up
        the ramp

    rc_map : numpy.ndarray
        2D array giving the location of RC pixels

    low_pededtal_map : numpy.ndarray
        2D array giving the location of pixels with abnormally low
        pedestal values

    high_cr_map : numpy.ndarray
        2D array giving the location of pixels with abnormally high
        numbers of jumps

    Returns
    -------
    final_map : numpy.ndarray
        2D array containing the bitwise combined bad pixel maps


Identify pixels that have an abnormal number of flagged jumps. Do
    this by finding the jump rate early in the ramp versus that later in
    the ramp.

    Parameters
    ----------
    jump_map : numpy.ndarray
        Map of jump flags for all pixels (e.g. output from ``dqflags.flag_map``)
        Assume we are working one integration at a time.

    max_jump_limit : int
        The maximum number of jumps a pixel can have in an integration
        before it is flagged as a ``high jump`` pixel (which may be
        flagged as noisy later)

    jump_ratio_threshold : int
        Cutoff for the ratio of jumps early in the ramp to jumps later in
        the ramp. Pixels with a ratio greater than this value (and which
        also have a high total number of jumps) will be flagged as
        potential (I)RC pixels.

    early_cutoff_fraction : float
        Fraction of the integration to use when comparing the jump rate
        early in the integration to that across the entire integration.
        Must be <= 0.5

    Returns
    -------
    high_jumps : numpy.ndarray
        Map of pixels that have more than ``max_jump_limit`` jumps.

    potential_rc : numpy.ndarray
        Map of pixels which have: 1) a large number of jumps, and
        2) a higher rate of jumps early in the ramp than later in
        the ramp.


 Create an array of pixel slopes which are clean and have no detected cosmic rays

    Parameters
    ----------
    slope : numpy.ndarray
        Array of pixel slopes for integration

    number_of_jumps : numpy.ndarray
        Array holding the number of jumps detected for each pixel ramp

    Returns
    -------
    clean_slope : numpy.ndarray
        array of slopes for an integration containing no cosmic rays
         slopes for a pixel ramp that have a cosmic ray detected are set to nan
    iclean_slope: numpy.ndarray
         an array of that holds if a good slope was detected. A value of 1 is
         assigned to array elements where there are no CRs


 Combine the stack of slopes and form the stanard deviation of the pixel slopes

    Parameters
    ----------
    slope_stack : list
        A list of slopes for full array stacked for each integration

    islope_stack: list
        A list of of 1 or 0 for each integration. A 1 is a good slope and 0 is slope
        with cosmic ray


Get statsitics on the pedestal array corresponding to an
    integration

    Parameters
    ----------
    pedestal_array : numpy.ndarray
        Array of pedestal values

    threshold : int
        Number of sigma above or below the mean at which a pedestal
        value is considered bad

    Returns
    -------
    suspicioius_pedestal : numpy.ndarray
        Map of bad pedestal values\