[UPDATE: Since this article was written, both the ENHG and XIONG algorithms described in this article have been fully automated in PANCROMA into a single step process. See the User Manual for more information.]

Pan sharpening is a powerful technique for increasing the resolution of color satellite imagery. This is done by combining the color data from the resolution satellite bands with a higher resolution panchromatic band. A common method for doing this is the hue, saturation, intensity (HSI) transform, whereby the red, green and blue (RGB) chromatic bands are transformed to a corresponding hue, saturation intensity model. The hue and saturation bands (containing most of the color information) are doubled and interpolated and the intensity band is discarded, being replaced by the panchromatic band. The doubled HSI model is then reverse-transformed back to an expanded RGB model, yielding the pan sharpened image at double (or multiple) resolution.

There are two fundamental and related problems with this technique (shared by other transformation methods for pan sharpening as well). Firstly, the three HSI color vectors are practically speaking not orthogonal, meaning that the characteristics of the intensity band influence the color tone of the image, not just the overall brightness. The second problem lies with the nature of the panchromatic band. Despite its higher resolution, it differs from the discarded intensity band in subtle but important ways. The result is that when the panchromatic band is substituted for the intensity band, the color tones of the pan sharpened image may not match those of the lower resolution RGB image. Often this takes the form of a blue cast to vegetated (green) areas of the image and other spectral distortions. For some pan sharpened images this can be pronounced to the point of producing an unacceptable image. In general the effect most pronounced for temporal forested areas, less pronounced for mountainous areas, and practically non-existent for desert regions.

To see why this problem can occur, it is helpful to examine the sensors the record the panchromatic image. The Landsat panchromatic sensor records radiated energy with wavelengths between 0.52 microns and 0.90 microns. This essentially spans the green, red, and the band 4 near infrared (NIR), but does not include any of the blue band. As a result of the skewed-ness, when shoe-horned into the RGB reverse transform, the green band essentially becomes the blue, the red becomes the green and the NIR becomes the red. This can result in some interesting but unnatural looking color tones in the pan sharpened image.

You may think that the solution is as simple as uniformly displacing the entire histogram toward the red wavelengths. Unfortunately, this seldom yields good results as the panchromatic histogram differs from the computed intensity histogram in a non linear fashion. Neither does selectively adjusting the blue and green channel levels very often solve the problem. Histogram matching tries to correct for this by applying a non-linear correction to the panchromatic histogram. However, deficiencies in the algorithms used to compute the histogram match (which is actually performed on the cumulative histograms) often result in corrections that are less than perfect and that still exhibit a blue bias, although generally less so than the unprocessed image. As a result, secondary corrections are often required for troublesome cases.

PANCROMA currently offers a choice of two correction algorithms: the Enhanced Green (ENHG) method and the XIONG algorithm. Both of these use the additional information contained in the NIR band to help correct unnatural vegetation colors. The ENHG method uses the NIR band to correct band 2 (the green band) prior to pan sharpening. The XIONG algorithm uses the same NIR band to correct the panchromatic band.

The ENHG method computes the NDVI index and uses it to identify probable vegetation areas. It then increases the value of the green channel pixels in these areas. The NDVI index is given by:

NDVI= (NIR - RED) / (NIR + RED)

Currently the PANCROMA ENHG process is manual and is implemented as a preprocessing step. An ENHG band file is first computed and saved. Then the pan sharpening process is conducted as usual, substituting ENHG band for the green band. The result is often an image with more realistic vegetation tones.

The XIONG algorithm offers another alternative to correct the blue spectral distortion inherent in the panchromatic file. This algorithm often produces better results than the ENHG algorithm described above. Like ENHG the XIONG process also uses the NIR band and is also manual. A XIONG file only requires two inputs, a low resolution NIR band and the high resolution panchromatic band. The XIONG is also computed as a pre-processing step and the output file (which has the same resolution as the panchromatic band) is saved for subsequent processing. When the pan sharpening process is conducted as usual, the preprocessed XIONG file is substituted for the panchromatic band

The images to the right illustrate the result of using the various methods to correct a particularly troublesome Landsat image. The first image is the RGB composite reference image, which presumably shows the 'natural' color that is the target.

The next figure shows the pan sharpened image without any image processing at all. As a result of the characteristics of the panchromatic band for this file, the ocean areas are almost black and the vegetation is distinctly blue.

The next image shows the result of applying a nonlinear histogram match. Now the ocean is much more naturally colored, but the vegetation is still blue.

The next image shows the result of applying ENHG plus histogram matching. This is almost an acceptable image. The ocean has the proper blue tone and the land vegetation is much improved over the histogram match only image.

The final image shows the result of applying XIONG and histogram matching. Although there are some minor flaws that could be improved with a little more image processing, this image matches the reference RGB image very closely. The color tones of the ocean, land and vegetation are perfectly balanced and the image is very pleasing.

This article has demonstrated the inherent limitations on pan sharpening imposed by the nature of the panchromatic band. It has also demonstrated how the image processing techniques in the PANCROMA pan sharpening application can have a beneficial effect on processed image quality.