0. Introduction:

This package, HyperSpec, is for derivative analysis of hyperspectral data using IBM Data Explorer. The main components of this package are programs for making three DX modules, dxInputSpc, dxMeanFilter, and dx Derivative, to use in DX Graphical User Interfaces. Details and usage of the three modules are described later in this document.

NOTICE:

The development of HyperSpec is still in alpha phase. Bugs and unsatisfactory performance are expected. In no event, the author of HyperSpec will be liable for any result or damage caused by using any part of this package.

1. List:

This package consists of following files.

ReadMe.html --> this document
HyperSpec.h --> header file
HyperSpec.inboard.mdf --> (inboard) module description file
Makefile.inboard --> makefile for making inboard modules
dxInputSpc.c --> main program for dxinputSpc module
dxMeanFilter.c --> main program for dxMeanFilter module
dxDerivative --> main program for dxDerivative module
der_coef.c, get_der.c, mean.c, mean_filter.c, wl2idx.c --> secondary level routines
MnRef.spc --> sample spectra
HyperSpec.net --> sample DX net for using these modules

2. Installation

After ungziping and untaring the archives, you should see files listed above. Before you start making the modules, there are several things you need to know. First of all, the module description file HyperSpec.inboard.mdf is for building inboard module (as the filename says). If you want to build outboard or runtime modules, please modify the mdf or provide your own. Second, this package was developed and tested on IBM RS6000 machines running AIX 4.1. The Makefile.inboard is specific for this platform and for inboard module construction. If you are using a different platform and/or you want to create outboard modules, please make necessary changes of the Makefile. For details about module description file (mdf) and makefile, please refer to your DX references.

Assume you have make all necessary modifications, you can now cook your HyperSpec modules by typing following command:
%make -f Makefile.inboard

Please replace Makefile.inboard with the real filename of your makefile. If you encounter any error during the making process, first check your path environment variable to make sure you have include the suitable path to DX programs and libraries or consult your system adm.

3. Module description and usage:

If you choose to stick on inboard modules, you should see an executable dxexec generated after the making process. To use these modules in DX, type the following command to star the DX VPE (assume you are in the same directory of the newly generated dxexec):
%dx -exec ./dxexec -mdf HyperSpec.inboard.mdf

Please replace the HyperSpec.inboard.mdf with your mdf, if you have changed it.

By default, the three modules will appear in a category labelled as HyperSpec in your DX Visual Program Editor. Use these modules in the way you did for other DX modules.

A. dxInputSpc:

dxInputSpc let you input spectra from a formatted ASCII file by specifying the file name. The input tag of dxInputSpc accepts a string input of the file name. The format of the ASCII spectra file is described as below:
The first row of the file must give the number of samples (ns) of each band and number of bands (nb) of the spectra. These two numbers are provided as two integers and separated by spaces/tabs.
The rest of the file is basically a (ns x nb+1) matrix where the first column is the wavelength, and each column of the rest represents values of a single spectrum. For example, spectra consisting of three bands and each band (a spectrum) has 10 sampling points would be represented as:
10 3
1 1 1.2 0.9
2 1.2 1.5 1.1
...........
10 -0.3 -0.2 -0.5

There are five output tags of dxInputSpc:

spc_grp the spectra as a group object, each member represents a single spectrum

spc_fld a two-dimensional field of the spectra where x-coord is the wavelength, y- coord is the band number and the data component is the spectral value.

wl a scaler list of the wavelength

ns number of samples (an integer) for each band

nb number of samples (an integer) for each band

B. dxMeanFilter:

dxMeanFilter smooth the spectra using mean filter. It takes five inputs:

spc	a group object where each member of the group represents a band (for example, the first output of dxInputSpc)
wl	a scaler list of the wavelength (for example, the third output of dxInputSpc)
ns	number of samples (an integer) for each band
nb	number of samples (an integer) for each band
half_bw	an integer indicating the half bandwidth of the smoothing window; if the value is zero that means no smoothing

There are two output tags of dxMeanFilter:

spc_grp	a group object of the smoothed spectra where each member of the group is a band
spc_fld	a two-dimensional field of the smoothed spectra.

C. dxDerivative:

dxDerivative computes the derivatives of the spectra. It takes 8 input arguments:

spc	a group object of the spectra (for example, the first output of dxInputSpc or dxMeanFilter)
wl	scaler list of the wavelength
ord	order of derivative (integer)
bs	integer or integer list; band separation or a range of band separations; the range is represented as from "bs1 to bs2 by bs3" instead of an explicit list of all band separations.
enh_flag	if 0, use normal finite difference to compute derivatives, otherwise compute "enhanced" derivatives
mode	if 0, compute a band (a spectrum) at a range of band separations; otherwise, compute the whole spectra at a band separation (the first bs input)
target_spc	which band to compute if mode is 0
wl_range	scaler list, the minimum and maximum range of wavelength of interest

There are also two output tags of dxDerivative:

spc_grp	a group object of resultant derivatives
spc_fld	a two-dimensional field of resultant derivatives, where the x-coord is the wavelength, y-coord is the band number (mode 1) or band separation (mode 0)

4. Known problems:

Memory leaking: OK, I am not a good C programmer or I am abused by huge- memory equipped machines. I often forget to release the memory allocation. If you encounter memory leaking, please disconnect DX from server and then restart it. Occasionally, you may need to quit the entire DX then restart it. The most common symptom of this memory leaking is a segmentation fault error after several executions (even without changing any parameter).

Floating errors: Although in theory HyperSpec allows you to compute any order of derivatives but because I declared variables as float instead of double in the programs, you may encounter floating errors. The most common evidence of this problem is an unreasonable high or low derivative value comparing to the wavelength, so your display of the derivatives looks like a single line.

Please send your bug reports and comments to tsai@tc.cornell.edu.

5. Future work:

The first thing to do is to change all variables to double precision and fix the memory leaking problem. Then I will work on the performance issues-- try better programming schemes to expedite the process and reduce memory usage. Finally, I would like to seek the possibility to expend these modules for dealing with hyperspectral images.

spc_grp	the spectra as a group object, each member represents a single spectrum
spc_fld	a two-dimensional field of the spectra where x-coord is the wavelength, y- coord is the band number and the data component is the spectral value.
wl	a scaler list of the wavelength
ns	number of samples (an integer) for each band
nb	number of samples (an integer) for each band