TRA Key Products

Real-Time Hyperspectral Processing

<<  Army Countermine Program
Back to Contents >>


Summary:
A Pentium 4 computer, running Windows, was chosen as the computer for the NVIS real-time processor.  This configuration was chosen because it offers an easily programmable platform with adequate computational resources.   All programming was done in Microsoft Visual C++, with programs developed on computers other than the final processor.   Visual C++ is a familiar programming environment for Technical Research Associates, Inc.  It has been used for several previous real-time sensor-processing systems, such as the Airborne Hyperspectral Imager (AHI) processor.   Many display and data handling functions developed earlier are basis of the NVIS processor.  
NVIS-R Hardware
The processor configuration used for NVIS consists of:
Pentium IV CPU
Intel Motherboard
2 gigabytes of Memory
BitFlow Frame Grabber Cards

The system is packaged into a rack-mounted case for aircraft installation.  Streamlining many of the N-FINDR functions first reduced the processing requirement for the N-FINDR/STD algorithm.   In addition to the Pentium 4 processor, the NVIS system also includes the Dual Real Time Recorder (DRTR), which performs real-time streaming of raw data to a hard drive array.  The DRTR is currently configured with a 6-bay drive array, which provides approximately 100GB of storage when populated with 18 GB drives or over 400 GB with 72 GB drives.

The NVIS Sensor:
The Night Vision Imaging Spectrometer (NVIS) is an improved copy of the TRWIS III hyperspectral system developed by TRW (One Space Park, Redondo Beach, CA).  NVIS was developed for the US Army Night Vision and Electronic Sensors Directorate (NVESD) and has been successfully used as the solar reflective sensor for various programs and several types of measurements.  The NVIS system was designed for a variety of airborne platforms in a nadir push-broom mode. The system produces 384 simultaneous spectral bands with 256 cross-track pixels. It consists of two, separate co-aligned imaging spectrometers, which together cover the 0.4-2.35 micrometer spectral region.  The dual spectrometer design was used to optimize grating efficiency, and the use of two focal planes provides the required spectral coverage.  Each spectrometer has approximately 0.9 milliradian spatial resolution with 12.8° full lateral view.

The primary purpose of the NVIS on-board processor is to perform anomaly detection processing in real-time, as well as to save data for later processing, and display relevant outputs for the sensor operator.  The NVIS on-board processor builds upon several recent successful military hyperspectral demonstrations.  The processing flow can be divided into preprocessing, frame processing, cube processing, and point processing.   

Preprocessing:
 Preprocessing includes all of the focal plane electronics, the A/D conversion, and the buffering of the frame.  The custom preprocessor developed by TRW includes the function of combining the outputs of the two spectrometers into one 384 band by 256 frame.  This data is output from the preprocessor to a frame grabber in Band Interleaved by Line (BIL). Processing Steps A second frame grabber card acquires the data from the DALSA linescanner.  Frame processing is a series of functions that are performed on the data to prepare it for the detection algorithm oriented cube processing.  Frame processing functions include band discard and binning, calibration, rotation from Band Interleaved by Line (BIL) to Band Interleaved by Pixel (BIP), and bad detector removal.  For cube processing, multiple frames of data (usually 400-1200 frames) must be buffered for the detection algorithm since it must have access to both spatial and spectral data.  Point processing includes those functions that are performed on the data after the detection decision (threshold).  These include clustering of detections, prioritization of the detections, calculation of coordinates, and generating high-resolution image “chips” (256x256 image centered on the detection).   The overall flow of the processing is shown in figure above.  In addition to the detection algorithm oriented function, the option is provided to calibrate and save data as they are being acquired.

Frame Processing:
The purpose of frame processing is to remove any sensor defects from the data, apply an accurate radiometric calibration, and discard and bin spectral bands.   Target detection algorithms are sensitive to the presence of sensor defects.  For example, a bad detector that has not been corrected will result in a stripe in the data in one of the bands.  Since the detection algorithms find targets because they are spectral anomalies, this defect will likely lead to many false alarms.  Other effects such as shading or broad striping will also decrease the performance of the detection algorithms.  For this reason it very important to remove these defects that can result from focal plane non-uniformities or bad detectors.The formation of calibration files and bad detector maps is not a real-time function.   In the simplest implementation, the white file and dark file are processed into a gain and bias file for linear correction to each detector.   Detectors that do not perform well in the calibration step are flagged for replacement.    The formation of the calibration files and the bad detector maps can be done on demand.  Typically, this will be done immediately prior to each collection run.

The first step in the real-time frame processing is bad detector repair.   Referencing a bad detector map (formed as part of the calculation of the calibration files) and then using linear interpolation in the spectral direction to calculate a replacement value accomplishes bad detector repair.   Following the repair of bad detectors, the data rate is reduced by band discard and binning.  Certain bands, particularly in the SWIR are in water absorption bands, and are of no use for target detection.  several bands have very low response and are not useful.  These bands, once identified, are removed from the data frame.  Spectral binning is an important step to reduce the data rate and to increase SNR.  Generally, it is desirable to keep full spectral resolution in most of the VNIR portion of the spectrum, while the binning of two to four bands is performed in the SWIR. 

The calibration step is the most important step in frame processing.  Calibration performs two functions: field flattening and conversion to radiometric traceable units.  Regardless whether the calibration is relative or absolute, this step consists of a linear operation using gain and bias coefficients for each detector. 

Cube Processing:
With the calibration data applied, the data must be buffered into a data cube for processing by the detection algorithms.  While the size of this buffer is minimum for local anomaly detection algorithms like Reed-Xiaoli (RX), the buffer must be at least several hundred frames (lines on the ground) for a global spectral anomaly detection algorithm.  One approach to finding targets as global spectral anomalies is to use an algorithm based on linear mixture analysis.   These algorithms are based upon the assumption that the spectrum of each pixel is a linear combination of constituent spectra, called “endmembers”. N-FINDR was developed by Technical Research Associates, Inc. as an alternative approach for the autonomous determination of spectral endmembers.  Once these endmember spectra have been found, the image cube can be unmixed using a least-squares approach into a map of fractional abundances of each endmember material in each pixel.    The targets can be found since they are isolated to a sparsely populated abundance map. 

N-FINDR alone cannot perform the target detection function since the output in each case is a set of fractional abundance planes and endmember spectra, not candidate detections.  An anomaly detection algorithm, Stochastic Target Detector, or STD, was developed for the backend of N-FINDR to identify global spectral anomalies based upon the statistics of the fractional abundance planes.  The STD algorithm exploits the fact that the endmembers provide information about the materials in the scene and the abundance planes provide information about the fraction of each material within each pixel.  The output of the STD algorithm is a detection statistic map that can be thresholded.   With the existence of the STD detection backend to the N-FINDR algorithm, the N-FINDR/STD combination becomes an autonomous global anomaly detection algorithm. The N-FINDR autonomous endmember determination and unmixing algorithm, coupled with the Stochastic Target Detection (STD) algorithm has been chosen for the primary NVIS anomaly detection algorithm.

Point Processing:
After thresholding, the data stream has been basically reduced to detected targets (and false alarms).  A clustering step is employed to group detections from the same objects into one point.  Using the positional information (from the GPS/INS), coordinates of the detection are calculated.  Since the line scanner data has been similarly geo-referenced, the coordinates can be sent to the line scanner computer.  A “image chip” centered on those coordinates is then cut from the line scanner data and transmitted back to the processor computer where it is displayed for the operator.

Display:
Real-time display for the operator is also available.  For each cube of data (typically 400-1200 frames or lines on the ground), the operator can see either three endmember abundance planes or three bands of input data as a color image.  The location of detections by the anomaly detection algorithm is shown on the color images.  Image chips corresponding to the best three detections (highest value of the STD detection statistic) are also displayed.   As an alternative in the “calibrate and save” mode, the operator can see a reduced version of the DALSA line scanner data displayed as a waterfall.