Biostatistics — 14. Topic 10. Discriminant Analysis

←

[IMG_1] D.A. Shabanov, M.A. Kravchenko. Statistical Analysis of Data in Zoology and Ecology

→

Topic 9. Principal Component Analysis

Topic 10. Discriminant Analysis

Topic 11. Some Methods Specific to Zoology and Ecology

Biostatistics-13

Biostatistics-14

Biostatistics-15

Topic 10. Discriminant Analysis 10.1. Purpose and Basic Logic of Discriminant Analysis Discriminant analysis is a statistical method designed to study differences between two or more groups of objects using data on the variation of several features that distinguish these objects from each other. A typical task for discriminant analysis is to determine which features best discriminate (distinguish) objects belonging to different groups. Once the best ways to discriminate between the existing groups have been determined (i.e., the interpretation of differences between them has been performed), this method of analysis allows for the classification of samples whose group membership is not known in advance. Discriminant analysis was developed by Ronald Fisher (1890–1962), a classic of biometry and evolutionary biology. Explaining how discriminant analysis can be used is easiest with examples. Suppose we are interested in the differences between female and male skeletons (or between the body shape of diploid and triploid green frogs). We consider a set of male and female skeletons (or morphological features of diploids and triploids) and determine which features best discriminate these groups. After that, we can use our results to determine the sex of skeletons whose sex is initially unknown (or to determine the ploidy of frogs based on their body shape). The discriminant analysis algorithm considers a multidimensional feature space in which the studied objects are located (the state of features for each object determines its position in such a space). In this space, a canonical discriminant function is selected that best reflects the differences between groups of objects. This procedure resembles the procedure used in principal component analysis, with the exception that in component analysis, principal components are selected onto which the maximum information about the variation of all objects is projected, while discriminant analysis maximizes the differences between their predetermined groups. After the first such function is selected, a second canonical discriminant function is selected based on the remaining information. On the other hand, discriminant analysis is similar to analysis of variance. Its task can be formulated as follows. In discriminant analysis, a discriminant function (a variable or a linear combination of variables) is selected that allows distinguishing groups from each other and whose value can be used to predict which group each object belongs to. The situation can also be considered in the terminology of one-way analysis of variance (ANOVA). Are the differences between groups in the values of the discriminant function statistically significant? Which of the discriminant functions shows the most statistically significant differences between groups? How to select a discriminant function so that it best reflects the differences between groups? The discriminant function is better the more tightly the objects of each group are clustered around the centroids ("centers of gravity") of the groups, and the farther the centroids are from each other.Each subsequent function will contribute less and less to the discrimination of the groups under consideration. Each discriminant function is some linear combination of discriminant variables, i.e., features characterizing the objects under consideration. The maximum number of discriminant functions is one less than the number of discriminant variables and does not exceed the number of groups. The discriminant analysis algorithm is based on two fairly important assumptions. It is assumed that the discriminant variables have a normal distribution, and that their variance and covariance are homogeneous across groups. Small deviations from the mathematical validity of these conditions are quite acceptable. "The most important criterion for the correctness of the constructed classifier is practice." Khalafyan, 2007 10.2. Example of Performing Discriminant Analysis: Morphometric Features of Frogs The easiest way to master the discriminant analysis procedure is to use it for analyzing data in the file Pelophylax_example.sta. Let us determine which variables best discriminate between frogs belonging to five different genotypes. For this, we will use the Discriminant Analysis module in the Multivariate Exploratory Techniques submenu from the Statistics menu, as shown in Fig. 10.2.1. [IMG_2] Fig. 10.2.1. Calling the discriminant analysis module Let us start by considering all seven morphometric features included in our analysis. Select them in the Variables window as shown in Fig. 10.2.2. [IMG_3] Fig. 10.2.2. Selecting the grouping variable (genotype) and discriminant variables In this case, we will perform the analysis with default settings. Its results are shown in Fig. 10.2.3. [IMG_4] Fig. 10.2.3. After clicking the OK button in the dialog shown in the previous figure, the Discriminant Function Analysis Result window appeared. If the Summary button is clicked in this window, a table with results appears: Discriminant Function Analysis Summary, also shown in this figure Now we should consider the main statistics used for interpreting the results of discriminant analysis. The most important of them are shown in the upper part of the Discriminant Function Analysis Result window, and more detailed results concerning each of the variables are reflected in the Discriminant Function Analysis Summary window (Fig. 10.2.3). Wilk's Lambda is computed as the ratio of the determinant of the within-groups variance (covariance) matrix to the determinant of the total covariance matrix. We will not delve into the details of what this means and will use a simple definition instead. Wilk's Lambda is the ratio of the measure of within-group variation to the measure of total variation. Within-group variation is part of the total variation, which means that Wilk's Lambda can take values from 0 (groups are completely homogeneous) to 1 (splitting objects into groups does not result in within-group variation being less than the total). Therefore, the smaller the value of Wilk's Lambda, the better the separation into groups in discriminant analysis. The upper part of the Discriminant Function Analysis Result window shows the overall Wilk's Lambda value for the discriminant analysis taking into account all involved variables. In the first column of the Discriminant Function Analysis Summary window, opposite each variable, the Wilk's Lambda value is shown for the analysis in which that variable is not used. If excluding a particular variable from the analysis led to a significant deterioration in the result, we can state that this variable made an important contribution to it. Therefore: the higher the Wilk's Lambda value in the first column of the Discriminant Function Analysis Summary window, the more important this feature, and the lower the overall value of this statistic shown in the Discriminant Function Analysis Result window, the better the separation of groups was performed. Try not to get confused! Partial Lambda is the ratio of Wilk's Lambda after adding this variable to Wilk's Lambda before adding the variable. If a variable makes at least some contribution to the separation of groups, Wilk's Lambda should decrease after its addition. Consequently, the smaller the value of Partial Lambda, the more valuable this feature is. F-remove is the F-criterion associated with excluding this feature from the analysis, and p-level is its statistical significance. If excluding the feature leads to a statistically significant change in the ratio of variances, then this feature makes an important contribution to the discrimination of groups. Finally, Toler. is tolerance. This is a measure of redundancy of the feature, computed as 1-R², where R² is the coefficient of multiple correlation of this feature with all other features used in the analysis. The lower the tolerance, the stronger this feature is correlated with all the others. However, for the multiple correlation coefficient indicated in the last column, the situation is the opposite. The higher the R², the stronger this feature is correlated with the others used in the model. Another important measure that allows evaluating the quality of separation of objects into groups is the percentage of correctly classified objects. To find it out, in the Discriminant Function Analysis Result window, go to the Classification tab (Fig. 10.2.4). [IMG_5] Fig. 10.2.4. The Classification tab in the Discriminant Function Analysis Result window. To evaluate the correctness of the distribution into groups obtained as a result of the analysis, click the Classification Matrix button (see the next figure)

[IMG_6] Fig. 10.2.5. Correctness of classification based on the data used in the analysis. Rows show which groups the studied specimens actually belong to. Columns reflect how these objects would be classified (using the variables employed), if their group membership were unknown In Fig. 10.2.5, we can see that our analysis would have correctly identified two-thirds of the studied specimens. Specimens with the genotype RR (Pelophylax ridibundus) are identified best, and those with LL (Pelophylax lessonae) are identified worst. However, the peculiarities of human perception are such that no statistics (for which one has to remember whether an increase or decrease indicates an improvement in the quality of separation) can compare with a clear graph showing the distribution of objects relative to each other. To obtain a clear picture, canonical analysis should be performed (Fig. 10.2.6). [IMG_7] Fig. 10.2.6. In the Advanced tab in the Discriminant Function Analysis Result window, canonical analysis can be called (Perform canonical analysis)

[IMG_8] Fig. 10.2.7. The window with canonical analysis results. In it, you need to go to the Canonical scores tab (canonical roots)... [IMG_9] Fig. 10.2.8. ...and click the Scatterplot of canonical scores button [IMG_10] Fig. 10.2.9. By selecting the necessary canonical roots (usually the first and second), one can see how the studied specimens are positioned in their space The obtained picture can already be interpreted. We can see that the first canonical root reflects the differences between Pelophylax ridibundus and Pelophylax lessonae. Hybrids are located between the parental species, "lined up" in the order of the ratio of parental genomes. The separation is not absolute and includes some overlap (it is precisely this that accounts for the cases when classification would be incorrect). I would like to remind you that canonical roots are precisely those canonical discriminant functions that are selected so that they best reflect the differences between groups of objects. These roots are linear combinations of discriminant variables (in the extreme case, they correspond to some of the variables). To see how canonical roots and discriminant variables are related, click the Factor structure button in the Advanced tab in the Canonical Analysis window. There, correlation coefficients linking each canonical root with each discriminant variable are shown. 10.3. Searching for More Effective Ways of Separating Groups In the previous section, "raw" morphometric features were used in discriminant analysis. It can be assumed that this is not the best solution. Animal species and intraspecific forms very often differ from each other in proportions—the ratios of different morphometric features. In many cases in zoological research, the following variant proves successful: use some feature that characterizes overall size, in its absolute form, and all others—as proportions, their ratio to the feature characterizing size. In our example, naturally, the feature L—body length—characterizes size. The remaining six features can be used as proportions—the quotient of dividing their absolute sizes by L. Let us perform the analysis with such a set of features. Its results are shown in Fig. 10.3.1. [IMG_11] Fig. 10.3.1. Results of discriminant analysis in which body length and ratios of other features to body length were used. Notations like T_L correspond to proportions computed as T/L When using absolute values of features, Wilk's Lambda was equal to 0.20. When transitioning to proportions, it decreased slightly and became equal to 0.19 (0.18952). [IMG_12] Fig. 10.3.2. Correctness of classification when using ratios of morphometric features to body length The correctness of classification also improved slightly. It now amounts to 73.7% (when using "raw" data, it was 66.6%). [IMG_13] Fig. 10.3.3. Distribution of the studied objects in the plane of the first two canonical discriminant functions when using data on body length of specimens and the ratios of their measurements to body length The quality of separation of specimens in the plane of the two canonical roots changed insignificantly. We failed to significantly improve the quality of classification, but we confirmed that by using ratios of data instead of raw data, we can improve the result. But how to find the optimal set of features that should be used to search for differences between representatives of different genotypes?