Image interpretation and classification

Chapter 6 of Remote sensing.

Visual interpretation

Three kinds of information in an optical image are used to interpret it:

• Radiometric information: the values in each band, read as colour and brightness, and as spectral signatures.
• Texture: the pattern of variation between neighbouring pixels, especially in high-resolution images. In an image of a plantation, mature trees in rows, closed canopy and open grass can have the same colour but different texture. Texture can also be measured numerically and used in automatic classification.
• Geometric and contextual information: shape, size, pattern, location and association with neighbouring features. Using it requires some prior knowledge of the area. Fields and city blocks can both be rectangular, but blocks are smaller, lie in a regular grid and have a different colour; fields follow the terrain.

Interpretation can be wrong. The only certainty is ground truth: going to the place and seeing what is there.

Ground truth

Positional and thematic field observations are a precondition for most image interpretation. Field measurements of reflectance with a handheld spectrometer link what is on the ground to what the sensor measures, and are used to develop new methods, such as vegetation indices, that quantify surface properties from reflectance.

Image classification

An image classification assigns each pixel to one category, a land cover class, from its values in the bands used. The bands need not be the original ones: they can be an index such as NDVI, the result of a statistical transformation, data from another sensor, or the same bands from another season (seven bands from a summer image and seven from a winter image).

Each pixel is a point in a space with one dimension per band: with six bands, a six-dimensional spectral space. Ideally, pixels of the same land cover form clusters in that space, separate from the others. In practice the clusters overlap: a scatter plot of a NIR band against a red band shows clouds of points with no clear borders.

Unsupervised classification

An algorithm finds the k most distinct clusters in the spectral space, and the analyst then names them. Nothing guarantees that the clusters match useful land cover classes; they are only spectrally different. The most common algorithm is k-means:

1. Place k cluster seeds in the spectral space at random.
2. Assign every pixel to the nearest seed, forming clusters.
3. Move each seed to the mean of the pixels in its cluster.
4. If any seed moved, go back to step 2; otherwise the clusters are the result.

Supervised classification

The analyst first marks training areas, pixels or polygons known to be of each class, and an algorithm assigns the remaining pixels to the classes they resemble. Nothing guarantees that the classes can be told apart in the spectral space. A class that varies a lot is often better split into homogeneous subclasses (sea and lakes, or deep and shallow water) and merged afterwards.

• Minimum distance: computes the mean of each class’s training pixels and assigns each pixel to the class whose mean is nearest in the spectral space.
• Classification and regression tree (CART): a decision tree that splits the spectral space step by step at the band value that best separates the classes (“B8 below 0.029”), until a stopping rule is met. Trees are very sensitive to the training data.
• Random forest: many trees, each trained on a random subsample of the training data (often 50–60 %). Each pixel goes to the class most trees vote for. Even a small forest is more robust than a single tree.

A classifier can in principle be applied to any image with the same bands. Applying it to another date than it was trained on works best with surface reflectance, so that differences in light between, say, summer and winter do not look like differences in land cover.

Evaluating a classification

The ground truth is split in two: one part trains the classifier, the other (often about 30 %) is kept for validation. The validation pixels give a confusion matrix: the true class along one axis, the class assigned along the other. In this example, of 112 pixels that are truly class 4, 97 were classified as 4, 1 as 1, 13 as 2 and 1 as 3:

True class	→ 0	→ 1	→ 2	→ 3	→ 4	Producer’s accuracy
0	2818	0	0	0	0	1.00
1	0	75	0	0	0	1.00
2	1	0	173	0	3	0.98
3	0	4	0	224	2	0.97
4	0	1	13	1	97	0.87
User’s accuracy	1.00	0.94	0.93	0.99	0.95

Three measures follow from it:

1. Overall accuracy: the share of all validation pixels classified correctly.
2. Producer’s accuracy, per class: the share of the pixels truly in the class that were classified as it (row by row; errors of omission).
3. User’s accuracy, per class: the share of the pixels classified as the class that truly belong to it (column by column; errors of commission). It tells a map user how far to trust the class on the map.

A large class that is easy to classify, like class 0 here, lifts the overall accuracy; the per-class accuracies show where the classification fails.