ABSTRACT

Many supervised classification algorithms have been proposed, however, they are rarely evaluated for specific application. This research examines the performance of machine learning classifiers support vector machine (SVM), neural network (NN), Random Forest (RF) against maximum classifier (MLC) (traditional supervised classifier) in forest resources and land cover categorization, based on combination of Advanced Land Observing Satellite (ALOS) Phased Array type L-band Synthetic Aperture Radar (PALSAR) and Landsat Thematic Mapper (TM) data, in Northern Tanzania. Various data categories based on Landsat TM surface reflectance, ALOS PALSAR backscattering and their derivatives were generated for various classification scenarios. Then a separate and joint processing of Landsat and ALOS PALSAR data were executed using SVM, NN, RF and ML classifiers. The overall classification accuracy (OA), kappa coefficient (KC) and F₁ score index values were computed. The result proves the robustness of SVM and RF in classification of forest resource and land cover using mere Landsat data and integration of Landsat and PALSAR (average OA = 92% and F₁ = 0.7 to 1). A two sample t-statistics was utilized to evaluate the performance of the classifiers using different data categories. SVM and RF indicate there is no significance difference at 5% significance level. SVM and RF show a significant difference when compared to NN and ML. Generally, the study suggests that parametric classifiers indicate better performance compared to parametric classifier.

Keywords:

Supervised Classifier, Landsat, ALOS PALSAR, Support Vector Machine, Maximum Likelihood, Neural Network, Random Forest, Land Cover Classification

1. Introduction

Classification of satellite image is a very significant part of remote sensing image analysis, object and pattern recognition, mapping and monitoring of forest covers and natural resources. The process is commonly utilized for generation of thematic maps like forest, land cover/use maps and spatial pattern maps. Forest and land cover types classification using satellite data has been adopted extensively. Many supervised image classification algorithms have been developed and utilized for forest and land cover mapping, ranging from machine learning algorithms to traditional classifiers [1] [2] . Most of the algorithms have been reported to perform reasonably and to enhance the classification accuracy [1] [2] [3] . However, it is difficult to identify the best image classification algorithm which suits a particular environment. This is simply because numerous factors tend to affect the results: scheme of classification, satellite data in use, image pre-processing, training and validation sample selection and collection, learning algorithm and post processing approaches and validation techniques [4] . For that reason, evaluation of commonly applied machine learning algorithms is essential using same satellite dataset and scheme of classification to aid the selection of suitable algorithm for a particular application. With the advancement of the remote sensing technology, new classification algorithms are developed and invented rapidly. Consequently, it is very important to assess their performance in various kinds of environment using various types of remote sensing datasets [2]

The main objective of this study therefore, is to evaluate the capability of the widely applied parametric and non-parametric supervised machine learning algorithms for forest resource and land cover mapping in tropical environment using SAR and optical datasets. Specifically to assess which classification algorithm gives better results using independent and integrated Landsat TM and ALOS POLSAR datasets for categorization of forest resource and land cover mapping.

2. Study Area

The satellite image utilized for this study is of Bereko and Duru-Haitemba forest reserve in Babati, Tanzania. Lying between latitude 4˚15' and 4˚30' South, and between longitude 35˚35' and 35˚50' East (Figure 1). The area is categorized into six main land cover/use types: water (e.g. lakes), shrubs, natural dense forest and moderate forests [5] (Figure 1).

3. Dataset and Methods

3.1. Dataset and Training Samples

Both Optical and SAR satellite images has been utilized. Landsat 5 Thematic Mapper (TM) 30 m spatial resolution of November 4^th, 2009 and ALOS PALSAR L band [6] of September 13^th, 2009. The Shuttle Radar Topographic Mission (SRTM) Digital Elevation Model (DEM) 90 m resolution [7] [8] has been applied

for image preprocessing. In addition, a set of points based on Global Positioning System (GPS) and knowledge-based information acquired in October 2009, Normalized Difference Vegetation Index (NDVI) [9] and Google Earth images were used for ground truthing.

Training and validation samples for all land cover classes (i.e. water, shrubs, natural dense forest and moderate forests were) selected based on ground truth data, GPS based point locations and knowledge based information acquired on the site. The collected samples were divided into two groups, first as test sample (70% of the collected sample) and as second validation sample (30% of the collected sample).

3.2. Data Processing

ALOS PALSAR HH and HV polarization images were collected in slant range single look complex format. The images were transformed from slant range to ground range resolution using a multi-looking procedure of 9 × 2 (i.e., nine looks in azimuth and two looks in range) [10] [11] [12] . Resulting images were of 29.9 m × 27.7 m resolution in range and azimuth consecutively. This procedure improves radiometric resolution and squares the pixels in ground range geometry, which is same as Landsat TM spatial resolution (e.g. 30 m). For speckle reduction a refined Lee spatial filter [13] with a 7 × 7 window size was adapted. The topographic effects on the ALOS PALSAR backscattering was accounted by applying a radiometric terrain correction to convert backscattering in sigma-nought σ˚ value to the improved backscattering in gamma-nought γ˚ [1] [14] [15] . Landsat-5 TM digital numbers were converted to surface reflectance (SR) and normalized using Atmospheric and Topographic Correction 3 [16] .

3.2.1. Landsat TM and ALOS PALSAR Derivatives

Several ALOS PALSAR and Landsat TM derivatives were extracted. Especially vegetation indices (VI), Principal Component Analysis (PCA), SAR quotient bands HH/HV and HV/HH [17] and Radar Forest Deforestation Index (RFDI) [18] and the Grey-Level Co-Occurrence Matrix (GLCM) [19] textural feature measures. Normalized Vegetation Index (NDVI), (IR − R)/(IR + R) [9] , and Soil leaf Area Vegetation Index (SLAVI $\text{NIR}/\left(\text{Red}+\text{MIR}2\right)$ ) [20] (Table 1).

3.2.2. Integration of ALOS PALSAR and Landsat TM data

Various input bands were prepared ready for image classification. A multi-sensor integration image fusion approach was adapted [1] [5] . The multi-sensor integration approach combines n images in n different layers algorithmically, without creating a new set of images [21] [22] [23]

3.2.3. Classifiers under Study

For classification of forest covers and land cover mapping of the independent bands and integrated ALOS PALSAR/Landsat data and their derivatives Three non-parametric and one-parametric classifiers were tested on their ability. Random Forest [24] , Support Vector Machine (SVM) [1] [12] [25] [26] and Neural Network (NN) [27] supervised classifiers. Maximum likelihood (ML) [28] parametric classifier was used for comparison purpose. The ML is a conventional

classification algorithm that uses Gaussian distribution principle for data segmentation. The technique is robust and well-known for general classification problems. However, it may have some difficulties in classifying data coming from different sources, such as optical and SAR data. MLC is one of the extensively utilized classifier in the field.

The SVM is basically a binary class classification method based on machine learning and using support vector in the data classification. [12] [25] . Linear, polynomial, radial basis function and sigmoid are the four common kernels available in remote sensing packages. A careful selection of parameter setting can improve the performance of the SVM [29] . The Gaussian radial basis kernel function and a penalty parameter of 100 were selected based on trial and error. However, the kernel and penalty parameter selected are recommended to be the best for land cover classification [29] .

NN classifier has arbitrary decision boundary abilities and could adapt to various data types and input structures easily, fuzzy output values and suitable generalization for use when integrating manifold images [27] . The classifier benefits from parallel computation, the capability to estimate the non-linear relationship between the input data and desired outputs, and fast generalization ability [1] [30] [31] . The NN parameter setting was adapted based on trial and errors. As an activation function a logistic function was chosen, one hidden layer and 1000 training iterations were also designated.

RF is a machine ensemble approach based on classification and regression trees and can be used for both image classification and regression analysis [1] [24] [32] It makes use of multiple self-learning decision trees to parameterize models and use them for estimating categorical or continuous variables [33] [34] . The number of trees is a user-defined parameter. RF normally gives higher overall cross-validation accuracies compared to other classification approaches [33] Generally, Non-parametric classifiers yield higher classification accuracy compared to parametric classifiers [1] [30] [31] . To run the classification process and assess the potential of parametric and non-parametric classifiers. The data are grouped into three major groups A-C (Table 2), Group A, consists of surface reflectance bands from Landsat 5 TM image and its derivatives (i.e. Vegetation index, GLCM textures and PCA). Group B, comprises of individual ALOS PALSAR backscattering and derivatives. Group C, involve the integration of surface reflectance, backscattering and their derivatives. However, to maximize the overall classification accuracy the best blend of textures, indices and features were identified. The selection of relevant integration bands was carried out based on trial and error classification.

3.2.4. Classification Accuracy Assessment

To test the capability of parametric and non-parametric classifiers a validation dataset was used for accuracy assessment. Three terms that describe the classification accuracy were utilized (i.e. overall accuracy (OA), kappa coefficient (κ)) [35] and F₁ score index [36] . The overall classification accuracy is the percentage of the pixels that have been classified correctly in the validation dataset [35] . The Kappa coefficient is a metric that compares an observed Accuracy with an expected accuracy. It is used not only to assess a single classifier, but also to assess classifiers amongst themselves. F₁ score index merges producer’s and user’s accuracy into a fused quantity was computed (Equation (1)) [36] . Producer’s accuracy is used to estimate the omission error to a certain class and it is the probability that a reference site has been classified correctly. User’s accuracy is used to estimate the commission error and it is the probability that a pixel classified on the image signifies the actual class on the ground. F₁ score enables a better evaluation of the land cover class-wise accuracies. The score varies between 0 and 1 where by 0 signifies the worst results, and 1 is the best accuracy achieved.

${\text{F}}_1\text{ score}=2\times \frac{\text{precision}\times \text{recall}}{\text{precision}+\text{recall}}=2\times \frac{\text{user's accuracy}\times \text{producer's accuracy}}{\text{user's accuracy}+\text{producer's accuracy}}$ (1)

To compare the capability of the four classifiers under study, a two-sample t-test [37] was applied on the overall classification accuracy obtained using different data categories (Table 1). The influence of surface reflectance and backscattering derivatives on the classification accuracy the two-sample t-test was utilized. The two-sample t-test assesses whether two sample means unrelated. A difference in mean indicates that the two samples are dissimilar. The test is normally applied when the test makes use of a small sample size, the variances of two normal distributions are unknown and the experimentation involve a small sample size.

4. Results

4.1. Classification Results Based on the Four Classifiers

The classification results attained based on different data groups (A-C) (Table 1) and tested classifiers are presented in Figure 2 for Overall accuracy, Figure 3 for Kappa coefficients and Table 2 for F₁ score attained for every land cover type. Using maximum likelihood classifier, data group A, group A1 surface reflectance and derivatives depicts higher overall classification accuracy (average OA = 93.35%) and higher F₁ score index values (F₁ = 0.95 - 1) for all land cover types. Group B, backscattering values and derivatives, depicts lower overall classification accuracy (average OA = 53.92) and lower F₁ score index (F₁ = 0.18 - 0.53) values for dense forest, moderate forest and bare soil land cover classes. Group C, integration of surface reflectance, backscattering and derivatives provides good overall classification accuracy (average OA = 87.25%) and higher F₁ score index values (F₁ = 0.77 - 1) for all land cover classes (Figure 2 and Table 2). Using, support vector machine, both category A and C provides the best classification accuracy in terms of overall classification accuracy (average OA = 95.82% and 97.20% respectively) and F₁ score index values varied between 0.94 and 1 for all land cover types. Category B indicates poor lower overall classification accuracy (average OA = 57.9%) and lower F₁ score index values are obtained for dense forest, moderate forest and bare soil covers ranging from 0.07 to 0.68 (Table 2).

For Random Forest both data group A and C provides the best classification accuracy in terms of overall accuracy (average OA = 95.7% and 96.9% respectively). Higher F₁ score index values are obtained for all land cover types ranging between 0.94 and 1 (Table 2). Group B indicates higher overall classification accuracy compares to other classifier (average OA = 61.08%) and Lower F₁ score index values varying between 0.38 and 0.61 are obtained for dense forest, moderate forest and bare soil covers. For neural network classifier, like other classifier best results are obtained with data group A and C (average OA = 91.03% and 89.02% respectively). Lower F₁ score values for dense forest, moderate forest and bare soil covers ranging from 0.1 to 0.56 (Figure 2 and Table 2). For all classifiers when using SAR data water is the only land cover type classified with very higher F₁ score index values followed by shrubs (Table 2).

4.2. Evaluation of RF, SVM, NN and MLC Classifiers

The non-parametric classifiers (RF, SVM and NN) are assessed together with the maximum likelihood classifier (MLC) on different data subgroups. Figure 3 indicates the performance of the three classifiers in terms of Kappa coefficient (KC). The results of data group A and C Landsat based surface reflectance, PALSAR backscattering and derivatives, indicate that the three tested machine learning classifiers as well as MLC have good performance (average KC = 0.89 and 0.91 respectively) and there is no statistically significant difference at 95% confidence interval in their results (Figure 3).

However, MLC provides the poorest accuracy compared to the machine learning classifiers. In these groups, SVM and RF have better performance at 95% confidence interval compared to NN and MLC classifiers. For group B, SAR

backscattering and derivatives, all classifiers displayed poorer performance (Average KC = 0.50), though in most cases machine learning algorithm performed better compared to MLC (Figure 3). In this group, SVM and RF have more or less same performance and there is no substantial difference in their performances at 95% confidence level. Generally, in all data categories, SVM and RF produce better classification value at 95% confidence interval compared to NN.

Table 3 presents classifier comparison results of the three classification algorithm utilized based on the two sample t-test. The performances of all classifiers within each group are compared at 5% significance level. The results indicates that there is a statistical significant difference between SVM and NN classifiers (p-value = 0.05) at 5% significant level. There is no statistical significant difference between SVM and RF classifiers (p-value = 0.834) at 5% significance level. Both SVM and RF classifiers indicate a significance difference when compared to MLC (p-value = 0.001). RF and NN classifiers are statistically significant different at 5% significance level (p-value = 0.012). NN and MLC indicates that there is no statistically significant different at 5% significance level (p = 0.622) (Table 3).

5. Discussion

In this research a comparison of supervised learning algorithm using independent and integrated landsat TM and ALOS PALSAR data has been carried out. The assessment of the performances of the four classifiers under study shows that both parametric and non-parametric classifiers have good performance when using Landsat TM data (Figure 2 and Figure 3). Attarchi and Golaguen [1] attained same results indicating that both parametric and non parametric classifiers performs well for Landsat based surface reflectance and derivatives. SAR data and derivatives were effectively well classified by RF and SVM classifiers compared to MLC at 95% confidence level. This is probably due to the fact that SAR data and their derived parameters usually do not follow a Gaussian distribution, which is a basic assumption for several classification approaches.

Notes: A p-value ≤ 0.05 indicates the two samples are statistically significant different at 5% significance level. The p-value of greater than 0.05 implies that there is no significant difference between the two samples on comparison.

On the integration of SAR and Landsat data all classifiers indicate good performance, however, SVM and RF has the best performance in relation to NN and MLC at 95% confidence interval. Based on previous studies, parametric classifiers like MLC are not worthy when using multi-source remote sensing data. The superior performance of SVM and RF compared to NN could be due to the fact that SVM and RF has the potential to handle high dimensional data [1] .

Looking on the performance of classifiers based on data category, results in category A, subgroup A1-A3, Landsat surface reflectance and its derivatives indicates that non-parametric classifiers (SVM, RF and NN) as well as MLC performs well (Figure 2 and Figure 3). However NN indicate the lowest performance for data group A2. In category B, subgroup B1-B5 MLC has lowest performance. In this category MLC and NN have the more or less same performance for data group B2 and B4 and there is no significant difference in their performances at 5% confidence level. Subgroup A3 and C1 are the only data categories where all four classifiers indicates nearly similar performances (Figure 2 and Figure 3). In all categories A-C, SVM and RF provides the best performance. In data category C, specifically for C2-C4 NN display lowest performance compared to SVM and RF as well as MLC (Figure 2 and Figure 3). Attarchi and Gloaguen [1] , also indicated a poorest performance of NN compared to SVM and RF. As depicted in Figure 3, SVM and RF appear to have more or less same performance. This is simply because accuracy improvement of land cover mapping by new algorithms are hardly observable [2] [38] . Li et al [2] attained more or less similar classification accuracy values for SVM and RF.

The performances of all classifiers within each group are compared at 5% significance level. Comparing all classifiers using the two sample test, the results indicates that there is no statistical significant difference between SVM and RF classifiers at 5% confidence interval. Both SVM and RF classifiers indicate a significance difference when compared to NN and MLC. RF and SVM show a statistically significant different at 5% significance level when compared to MLC. NN and MLC indicates that there is no statistically significant different at 5% significance level (Table 3).

6. Conclusion and Recommendations

The potential of parametric and non parametric classifiers has been examined based on integration of Landsat TM and ALOS PALSAR data. All classifiers under study performs well in terms of overall accuracy when using Landsat TM and derivatives, however SVM and RF are superior compared to others. For SAR data SVM, RF and NN performs well compared to MLC. On integration of Landsat and PALSAR data SVM and RF seems to be very powerful compared to NN and MLC especially when combining TM derivative, backscattering and GLCM textures. Generally, the overall results indicates the robustness SVM and RF at 5% significance level for land cover classification in tropical area. However, the process of selecting a suitable classifier for a certain task depends much on tradeoffs among classification accuracy, time consumption, and computing resources. Based on the results attained the researcher recommends that, the performance of other classification algorithms, especially object based classification should be tested in tropics and semi-arid environments. This will show their potential ability in terms of differentiating forest resource and land cover mapping. Additionally, since new classification algorithms are developed rapidly it is very essential to evaluate their performance and sensitivity in different environs using various types of remote sensing datasets and high quality samples. If a comprehensive assessment of algorithms on various kinds of environment types were carried out it would be more suitable to select an algorithm for a specific remote sensing application.

Acknowledgements

The author thanks Dr. Veraldo Liesenberg for facilitating the acquisition of ALOS PALSAR L band data. The data was acquired under Cat.1-Proposal 6242 through the European Space Agency (ESA) Third Party Mission. The Landsat TM data was downloaded from the US Geological Survey (USGS) website.

Cite this paper

Deus, D. (2018) Assessment of Supervised Classifiers for Land Cover Categorization Based on Integration of ALOS PALSAR and Landsat Data. Advances in Remote Sensing, 7, 47-60. https://doi.org/10.4236/ars.2018.72004

References

Abbreviations

The following abbreviations are used mostly in this manuscript: