Hyperspectral Derivative Analysis

by Fuan Tsai, School of Civil and Environmental Engineering, Cornell University

Table of Contents



Introduction

Until relatively recently, imaging remote sensing systems have been limited to multispectral devices. Numerous effective methods, usually derived from established methods in multivariate statistics, have been developed and applied for spatial or spectral analysis of multispectral remote sensing data.[1][2] However, when higher resolution, spectrally continuous remote sensing data become available, researchers in remote sensing tended to select suitable bands to optimize the existing algorithms or to generate new algorithms based on traditional multispectral concepts.[3]-[5] Typical multispectral analysis methods treat each spectral band as an independent variable--a reasonable assumption for multispectral data but not really appropriate for hyperspectral data. Only a few researchers have tried to employ approaches commonly used in spectroscopy or have manipulated data as truly spectrally continuous data.[7]-[9]

Among the techniques developed in spectroscopy, derivative spectroscopy is particularly promising for use with remotely sensed data. Derivatives of the second order or higher should be relatively insensitive to variations in illumination. At the typical spectral sampling interval of hyperspectral systems, derivatives should also be relatively insensitive to sunlight and skylight. Nonetheless, few researchers so far have addressed applications of spectral derivatives in remote sensing.[6][9][10] Although some of these studies have used high order derivatives, [11][12] first and second order derivatives have been the most common. In addition, most of the studies using spectral derivative analysis were directly toward specific applications.[10][13][14]

The purpose of this project is to develop an algorithm for derivative analysis of hyperspectal data and then implement modules for IBM Data Explorer as a general hyperspectral derivative tool that will treat hyperspectal data as truly spectrally continuous data based on the algorithm. It is hoped that with this exploratory tool, the user would be able to proceed general hyperspectral derivative analysis and extract useful spectral features from the spectra for advanced analysis. More important, these operations would be performed with no needs to assume that the data consisted of homogeneous pixels or were generated in highly controlled environments.


Methodology

In hyperspectral analysis, derivatives are very sensitive to noise. Therefore, minimizing random noise is often the first step after the spectra is imported. Among various methods for smoothing spectral data, mean filter is a simple but direct approach. A mean filter locally smooths data within a predetermined moving smoothing window by calculating the mean value of samples within the smoothing window as the new value of the middle sampling point in the smoothing window. The algorithm can be represented as the following equation:
Mean filter equation(1)
where n (number of sampling points) is the size of filter, and j is the index of the midpoint. In this study, n is determined by the half-bandwidth, hbw, i.e. n=2*hbw+1.

After smoothing, the resultant smoothing spectra can be then passed into the derivative module. In this study, derivatives are estimated using a finite divided difference scheme, or "finite approximation". The advantage of finite approximation is that the derivatives can be computed according to different finite band resolutions (band separations) to extract special spectral features of interest at different spectral scales. The first and second derivatives are estimated by:
Equation for 1st. derivative(2)
Equation for 2nd. derivative(3)

Accordingly, higher order derivatives are computed iteratively and any order of derivative is accessible using finite approximation. In general, the n-th order of derivative is:
General equation of finite approximation (4)
where j=(2*i+n)/2, if (2i+n) is even, or j=(2*i+n+1)/2, if (2i+n) is odd, i.e. if the resultant derivative falls between sampling points, it is assigned to the point at next larger wavelength or wave number. The coefficients, Ck are calculated using an iteration scheme.
As the band separation increases, the magnitude of derivatives is usually depressed -a quasi-smoothing effect.[15] This is because the derivatives are normalized by a power of band separation (Equation-4). To facilitate visual comparison of the derivatives as band separation increases, the derivatives can be "enhanced" by replacing the denominator of Equation-4 with the band separation, regardless the order of derivatives.


DX Implementation

In this project, three (3) DX modules (categorized as HyperSpec) have been developed to accomplish hyperspectral derivative analysis. They are dxInputSpc, dxMeanFilter and dxDerivative. The functions of these modules are to input the spectra, smooth the spectra and compute derivatives of the spectra. When smoothing the spectra, the user can specify the half bandwidth of the smoothing window. In dxDerivative, the user can either compute derivatives of the whole spectra at a specified band separation or pick a member from the spectra and calculate derivatives at a range of band separations. In either case, the user can also specify using normal derivative procedure or enhanced derivative algorithm and specify a wavelength range of interest. Each module will output the result as a group object (where each member represents a single (derivative) spectrum) and as a single two-dimensional field representing the whole spectra as well as other useful data.

The key issue of the implementation is to allow users to flexibly assign various parameters, such as bandwidth for (smoothing), band separation (for derivative computation), to proceed derivative analysis for extracting desired spectral features or exploring the spectra to detect useful information. If you want to try these modules, please go to the program page to download the archives.


Result

After generating the modules, one can use them in a DX network to proceed hyperspectral derivative analysis. A suggested approach is to create a control panel for specifying various parameters in Data Explorer Graphical User Interfaces. Fig-1 is an example of this control panel, which is used in a DX net to read in reflectance spectra of a soybean plant subjected to different manganese treatment and then to smooth the spectra and compute derivatives.
Fig-2 and Fig-3 are the displays of the soybean spectra in line and surface mode respectively. The line display is created from the group object output of dxInputSpc, whereas the surface display is from the two-dimensional field output. Fig-4 is the result of the smoothing at a half bandwidth of five sampling points. Fig-5 and Fig-6 present the second order derivative spectra at band separation equals to 3 (Fig-5) and 11 (Fig-6) sampling intervals.
Fig-7 shows the second derivatives of a single spectrum at band separations ranging from 3 to 20 sampling intervals. From this figure, it is noticed that as the band separation increases (toward the front of the figure), the magnitude of derivative is damped. To overcome this quasi-smoothing effect, enhanced finite approximation should be applied. This is like applying a dynamic scaling factor to counteract the natural scaling effect resulting from a wide band separation. The enhanced derivative of Fig-7 is shown in Fig-8. One thing to note is that this enhancement is an empirical choice based on experience and should be used only to simplify visual interpretations when comparing spectra analyzed at different band separations.

Conclusion

The implementation of HyperSpec modules in Data Explorer provides researchers in the field of remote sensing the ability to treat hyperspectral data as truly continuous data. With these modules users in performing hyperspectral analysis can optimize noise reduction and to better match the scale of spectral features of interest by adjusting various factors used in the analysis, such as the half bandwidth and the band separation. The superior visualization ability of Data Explorer also allows the users to explore the spectra in a more flexible way to extract subtle information.

Modules developed in this study have been limited to deal with hyperspectral data so far. Improvement and extensions for algorithms toward a more solid analysis tool would be worthwhile. These include the improvement of programming techniques to accelerate the process and reduce memory requirement, incorporating other smoothing and derivative methods, and extend the modules for more general applications on spectral analysis. It is also desired to reinforce these modules for dealing with hyperspectral imagery such as the AVIRIS (Airborne Visible/Infrared Imaging Spectrometer) images.


Acknowledge

This project is conducted under the instruction of Prof. Bruce Land as an independent research project (CS 790). Thank Prof. Land for giving me all helps and suggestions. I would also like to thank Chris Pelkie at Cornell Theory Center, who gave me advises on DX programming.

References

[1] Duda, R. O., &;Hart, P. E., Pattern Classification and Scene Analysis, John Wiley &;Sons, New York, 1973.
[2] Richards, J. A., Remote Sensing DIgital Image Analysis. (2nd. Edition), Springer Verlag, New York, 1993.
[3] Hoffbeck, J. P., &;Landgrebe, D. A., "Classification of high dimensional multispectral image data", the Fourth Annual JPL Airborne Geoscience Workshop, Washington D.C., 1993.
[4] Rock, B. N., Williams, D. L., Moss, D. M., Lauten, G. N., &;Kim, M., "High spectral Resolution Field And Laboratory Optical Reflectance Measurements of Red Spruce And Eastern Hemlock Needles And Branches", Remote Sens. Environ., 47:176-189, 1994.
[5] Chappelle, E. W., Kim, M. S., &;McMurtrey, J. E. I., "Ratio Analysis of Reflectance Spectra (RARS): an algorithm for the remote estimation of the concentrations of chlorophyll A, chlorophyll B, and carotenoids in soybean leaves", Remote Sens. Environ., 39:239-247, 1992.
[6] Peñuelas, J., Gamon, J. A., Fredeen, A. L., Merino, J., &;Field, C. B., "Reflectance Indices Associated With Physiological Changes in Nitrogen- and Water-limited Sunflower Leaves", Remote Sens. Environ., 48:135-146, 1994.
[7] Talsky, G., Derivative Spectrophotometry, Weihein, Verlagsgesellschaft, Weihein, Germany, 1994
[8] Curran, P. J., Dungan, J. L., Macler, B. A., Plummer, S. E., &;Peterson, D. L., "Reflectance Spectroscopy of Fresh Whole Leaves for the Estimation of Chemical Concentration", Remote Sens. Environ., 39:153-166, 1992.
[9] Demetriades-Shah, T. H., Steven, M. D., &;Clark, J. A., "High Resolution Derivatives Spectra in Remote Sensing", Remote Sens. Environ., 33:55-64, 1990.
[10] Philpot, W. D., "The Derivative Ratio Algorithm: Avoiding Atmospheric Effects in Remote Sensing", IEEE Transactions on Geoscience &;Remote Sensing, 29(3):350 357, 1991.
[11] Butler, W. L., &;Hopkins, D. W., "Higher Derivative Analysis of Complex Absorption Spectra", Photochemistry and Photobiology, 12:439-450, 1970.
[12] Fell, A. F., &;Smith, G., "Higher Derivative Methods in Ultraviolet, Visible and Infrared Spectrophotometry", the Anal. Proc., 1982.
[13] Dick, K., &;Miller, J. R., "Derivative Analysis Applied to High Resolution Optical Spectra of Freshwater Lakes", the 14th Canadian Symposium on Remote Sensing, Calgary, Alberta, CA, 1991.
[14] Chen, Z., &;Curran, P. J., "Derivative Reflectance Spectroscopy to Estimate Suspended Sediment Concentration", Remote Sens. Environ.,40:67, 1992.
[15] Tsai, Fuan, Derivative Analysis of Hyperspectral Data, M.S. thesis, Cornell University, Ithaca, NY, 1996.
[16] IBM Data Explorer 3.0 user manuals.