BioStatistics — 08. Topic 5. Brief Introduction to Analysis of Variance

←

Д. А. Шабанов, М. А. Кравченко. Статистичний аналіз даних у зоології та екології

→

Topic 4 (continued). Non-parametric tests for sample comparison

Topic 5. A Brief Introduction to Analysis of Variance

Topic 5 (continued). Multifactorial Analysis of Variance

Biostatistics-07

Biostatistics-08

Biostatistics-09

Topic 5. A brief introduction to analysis of variance 5.1. What is analysis of variance? Analysis of variance was developed in the 1920s by the English mathematician and geneticist Ronald Fisher. According to a survey of scientists to determine who had the greatest impact on 20th-century biology, Sir Fisher received first place (for his achievements, he was awarded a knighthood—one of the highest honors in Great Britain); in this regard, Fisher is comparable to Charles Darwin, who had the greatest impact on 19th-century biology. Analysis of variance (ANOVA) is currently a separate branch of statistics. It is based on Fisher's discovery: the measure of variability of a studied quantity can be decomposed into parts corresponding to the factors influencing this quantity and random deviations. To understand the essence of analysis of variance, we will perform the same calculations twice: 'manually' (with a calculator) and using the Statistica program. To simplify the task, we will work not with the results of a real study of the diversity of green frogs, but with a fictional example concerning the comparison of women and men in humans. Let's consider the height variation of 12 adult individuals: 7 women and 5 men. Table 5.1.1. Example for one-way analysis of variance: data on sex and height of 12 people

Sex

Growth

Sex

Growth

Sex

Growth

Male

186

Female

172

Female

163

Female

169

Female

179

Male

162

Female

166

Female

165

Female

162

Male

188

Male

174

Male

190

{"translated_text": "[IMG_1]\n\n\n\n←\n\n\nD.A. Shabanov, M.A. Kravchenko. Statistical Data Analysis in Zoology and Ecology\n\n\n→\n\n\n\n\nTopic 4 (continuation). Nonparametric Tests for Sample Comparison\n\n\nTopic 5. A Brief Introduction to Analysis of Variance\n\n\nTopic 5 (continuation). Multifactor Analysis of Variance\n\n\n\n\nBiostatistics-07\n\n\nBiostatistics-08\n\n\nBiostatistics-09\n\n\nTopic 5. A Brief Introduction to Analysis of Variance\n5.1. What is Analysis of Variance?\nAnalysis of variance was developed in the 1920s by the English mathematician and geneticist Ronald Fisher. According to a survey among scientists asking who had the greatest impact on 20th-century biology, Sir Fisher took first place (for his achievements he was awarded knighthood — one of the highest honors in Great Britain); in this respect, Fisher is comparable to Charles Darwin, who had the greatest influence on 19th-century biology.\nAnalysis of variance (ANOVA) is now a separate branch of statistics. It is based on the discovery by Fisher that the measure of variability of the studied quantity can be decomposed into parts corresponding to the factors influencing this quantity and random deviations.\nTo understand the essence of analysis of variance, we will perform the same calculations twice: \"manually\" (with a calculator) and using the Statistica program. To simplify our task, we will not work with the results of an actual description of green frog diversity, but with a fictional example concerning the comparison of women and men in humans. Let us consider the height diversity of 12 adults: 7 women and 5 men.\nTable 5.1.1. Example for one-way analysis of variance: data on sex and height of 12 people\n\nSex\n\nGrowth\n\n\n\n\n\n\nSex\n\nGrowth\n\n\n\n\n\n\nSex\n\nGrowth\n\n\n1\n\nMale\n\n186\n\n5\n\nFemale\n\n172\n\n9\n\nFemale\n\n163\n\n\n2\n\nFemale\n\n169\n\n6\n\nFemale\n\n179\n\n10\n\nMale\n\n162\n\n\n3\n\nFemale\n\n166\n\n7\n\nFemale\n\n165\n\n11\n\nFemale\n\n162\n\n\n4\n\nMale\n\n188\n\n8\n\nMale\n\n174\n\n12\n\nMale\n\n190\n\nWe will perform one-way analysis of variance: we will compare whether men and women in the described group differ in height in a statistically significant way.\n5.2. Test for Normality of Distribution\nFurther reasoning is based on the assumption that the distribution in the considered sample is normal or close to normal. If the distribution is far from normal, variance is not an adequate measure of its variability. However, analysis of variance is relatively robust to deviations from normality.\nTesting these data for normality can be done in two different ways. The first: Statistics / Basic Statistics/Tables / Descriptive statistics / Normality tab. In the Normality tab, you can select the normality tests to be used. Clicking the Frequency tables button will produce a frequency table, and the Histograms button — a histogram. The table and histogram will show the results of various tests.\nThe second method involves using the corresponding capabilities when constructing histograms. In the histogram construction dialog (Graphs / Histograms...), select the Advanced tab. In its lower part there is a Statistics block. Check Shapiro-Wilk test and Kolmogorov-Smirnov test on it, as shown in the figure.\n[IMG_2]\nFig. 5.2.1. Statistical tests for normality of distribution in the histogram construction dialog\nAs can be seen from the histogram, the distribution of height in our sample differs from normal (in the middle there is a \"dip\").\n[IMG_3]\nFig. 5.2.2. Histogram constructed with the parameters specified in the previous figure\nThe third line in the graph title indicates the parameters of the normal distribution to which the observed distribution was closest. The population mean is 173, the population standard deviation is 10.4. At the bottom, in the inset on the graph, the results of normality tests are shown. D is the Kolmogorov-Smirnov test, and SW-W is the Shapiro-Wilk test. As can be seen, for all the tests used, the differences in height distribution from normal distribution turned out to be statistically insignificant (p in all cases greater than 0.05).\nThus, formally speaking, the tests for conformity to normal distribution did not \"prohibit\" us from using a parametric method based on the assumption of normal distribution. As already mentioned, analysis of variance is relatively robust to deviations from normality, so we will still use it.\n\n5.3. One-Way Analysis of Variance: Manual Calculations\nTo characterize the variability of human height in the given example, let us calculate the sum of squares of deviations (denoted in English as SS, Sum of Squares or [IMG_4]) of individual values from the mean: [IMG_5]. The mean value for height in the given example is 173 centimeters. Based on this,\nSS = (186–173)2 + (169–173)2 + (166–173)2 + (188–173)2 + (172–173)2 + (179–173)2 + (165–173)2 + (174–173)2 + (163–173)2 + (162–173)2 + (162–173)2 + (190–173)2;\nSS = 132 + 42 + 72 + 152 + 12 + 62 + 82 + 12 + 102 + 112 + 112 + 172;\nSS = 169 + 16 + 49 + 225 + 1 + 36 + 64 + 1 + 100 + 121 + 121 + 289 = 1192.\nThe obtained value (1192) is a measure of variability of the entire data set. However, they consist of two groups, for each of which its own mean can be distinguished. In the given data, the mean height of women is 168 cm, and of men is 180 cm.\nLet us calculate the sum of squares of deviations for women:\nSSf = (169–168)2 + (166–168)2 + (172–168)2 + (179–168)2 + (163–168)2 + (162–168)2;\nSSf = 12 + 22 + 42 + 112 + 32 + 52 + 62 = 1 + 4 + 16 + 121 + 9 + 25 + 36 = 212.\nAlso let us calculate the sum of squares of deviations for men:\nSSm = (186–180)2 + (188–180)2 + (174–180)2 + (162–180)2 + (190–180)2;\nSSm = 62 + 82 + 62 + 182 + 102 = 36 + 64 + 36 + 324 + 100 = 560.\nWhat does the studied quantity depend on according to the logic of analysis of variance?\nThe two calculated values, SSf and SSm, characterize the within-group variance, which in analysis of variance is conventionally called \"error.\" The origin of this name is related to the following logic.\nWhat does human height depend on in the considered example? First of all, on the average height of people in general, regardless of their sex. Second — on sex. If people of one sex (male) are taller than another (female), this can be represented as adding some value, the effect of sex, to the \"universal\" mean. Finally, people of the same sex differ in height due to individual differences. Within the model describing height as the sum of the universal mean and a correction for sex, individual differences are unexplainable, and they can be considered as \"error.\"\nThus, according to the logic of analysis of variance, the studied quantity is determined as follows: [IMG_6], where xij is the i-th value of the studied quantity for the j-th value of the studied factor; [IMG_7] is the population mean; Fj is the influence of the j-th value of the studied factor; [IMG_8] is the \"error,\" the contribution of the individuality of the object to which the value xij relates.\nBetween-group sum of squares\nThus, SSerror = SSf + SSm = 212 + 560 = 772. This value describes the within-group variability (when grouping by sex). But there is also a second part of variability — the between-group, which we will call SSeffect (since it concerns the effect of dividing the set of considered objects into women and men).\nThe mean of each group differs from the overall mean. When calculating the contribution of this difference to the overall measure of variability, we must multiply the difference between the group and overall means by the number of objects in each group.\nSSeffect = [IMG_9] = 7×(168–173)2 + 5×(180–173)2 = 7×52 + 5×72 = 7×25 + 5×49 = 175 + 245 = 420.\nHere the principle of constancy of sum of squares discovered by Fisher manifested: SS = SSeffect + SSerror, that is, for this example, 1192 = 440 + 722.\nMean squares\nWhen comparing the between-group and within-group sums of squares in our example, we can see that the first is associated with variation of two groups, and the second — with 12 values in 2 groups. The degrees of freedom (df) for some parameter can be defined as the difference between the number of objects in the group and the number of dependencies (equations) that connect these values.\nIn our example, dfeffect = 2–1 = 1, and dferror = 12–2 = 10.\nWe can divide the sums of squares by their degrees of freedom, obtaining mean squares (MS, Means of Squares). By doing this, we can establish that MS is nothing other than variances (\"dispersions,\" the result of dividing the sum of squares by the degrees of freedom). After this discovery, we can understand the structure of the analysis of variance table. For our example, it will have the following form.\n\nSS\n\ndf\n\nMS\n\nF\n\nP\n\n\nEffect\n\n420.0\n\n1\n\n420.0\n\n5.440\n\n0.041874\n\n\nError\n\n772.0\n\n10\n\n77.2\n\n\nMSeffect and MSerror are estimates of between-group and within-group variance, and therefore they can be compared using the F test (Snedecor's test, named after Fisher), intended for comparing variances. This test is simply the quotient of dividing the larger variance by the smaller. In our case this is 420 / 77.2 = 5.440.\nDetermining statistical significance of Fisher's test from tables\nIf we were to determine the statistical significance of the effect manually, from tables, we would need to compare the obtained F test value with the critical one, corresponding to a certain level of statistical significance at given degrees of freedom.\n[IMG_10]\nFig. 5.3.1. Fragment of the table with critical values of the F test\nAs can be verified, for the statistical significance level p=0.05, the critical value of the F test is 4.96. This means that in our example, the effect of the studied sex was registered at the statistical significance level of 0.05.\nThe obtained result can be interpreted as follows. The probability of the null hypothesis, according to which the mean height of women and men is the same, and the registered difference in their height is due to chance in sample formation, is less than 5%. This means that we must choose the alternative hypothesis, according to which the mean height of women and men differs.\n\n5.4. One-Way Analysis of Variance (ANOVA) in Statistica\nIn cases when calculations are performed not manually, but with the help of appropriate programs (for example, the Statistica package), the p value is determined automatically. It can be verified that it is somewhat higher than the critical value.\nTo analyze the discussed example using the simplest version of analysis of variance, you need to run the Statistics / ANOVA procedure for the file with the corresponding data and select One-way ANOVA in the Type of analysis window, and Quick specs dialog in the Specification method window.\n[IMG_11]\nFig. 5.4.1. General ANOVA/MANOVA dialog\nIn the opened quick dialog window, in the Variables field, you need to specify the columns that contain the data whose variability we are studying (Dependent variable list; in our case — the Growth column), as well as the column containing the values that break down the studied quantity into groups (Categorical predictor (factor); in our case — the Sex column). In this version of analysis, unlike multivariate analysis, only one factor can be considered.\n[IMG_12]\nFig. 5.4.2. One-Way ANOVA dialog\nIn the Factor codes window, you should specify those values of the considered factor that need to be processed in this analysis. All existing values can be viewed using the Zoom button; if, as in our example, all values of the factor need to be considered (and for sex in our example there are only two), you can press the All button. When the processed columns and factor codes are set, you can press the OK button and go to the quick analysis of results window: ANOVA Results 1, to the Quick tab.\n[IMG_13]\nFig. 5.4.3. Quick tab of the analysis of variance results window\nThe All effects/Graphs button allows you to see how the means of the two groups relate. Above the graph, the number of degrees of freedom is indicated, as well as the F and p values for the considered factor.\n[IMG_14]\nFig. 5.4.4. Graphical representation of analysis of variance results\nThe All effects button allows you to obtain an analysis of variance table similar to the one described above (with some significant differences).\n[IMG_15]\nFig. 5.4.5. Table with analysis of variance results (compare with the similar table obtained \"manually\")\nThe bottom row of the table indicates the sum of squares, degrees of freedom, and mean squares for error (within-group variability). The row above — similar indicators for the studied factor (in this case — the Sex trait), as well as the F test (ratio of mean squares of effect to mean squares of error), and the level of its statistical significance. That the effect of the considered factor turned out to be statistically significant is shown by highlighting in red.\nAnd in the first row, data on the \"Intercept\" indicator are given. This row presents a puzzle for users who are getting acquainted with Statistica version 6 or later. The Intercept value (intersection, intercept) is probably related to the decomposition of the sum of squares of all data values (i.e., 1862 + 1692 … = 360340). The F test value indicated for it was obtained by dividing MSIntercept/MSError = 353220 / 77.2 = 4575.389, and naturally gives a very low p value. Interestingly, in Statistica-5, this value was not calculated at all, and guides on using later versions do not comment on its introduction in any way. Probably, the best thing a biologist working with Statistica-6 and subsequent versions can do is simply ignore the Intercept row in the analysis of variance table.\n\n5.5. ANOVA and Student's and Fisher's Tests: Which is Better?\nAs you may have noticed, the data we compared using one-way analysis of variance could also be studied using Student's and Fisher's tests. Let us compare these two methods. For this, we will calculate the difference in height of men and women using these tests. For this, we will have to go through Statistics / Basic Statistics / t-test, independent, by groups. Naturally, Dependent variables is the Growth variable, and Grouping variable is the Sex variable.\n[IMG_16]\nFig. 5.5.1. Comparison of data processed using ANOVA, by Student's and Fisher's tests\nAs can be verified, the result is the same as when using ANOVA. p = 0.041874 in both cases, as shown in Fig. 5.4.5, and as shown in Fig. 5.5.2 (verify this yourself!).\n[IMG_17]\nFig. 5.5.2. Analysis results (detailed breakdown of the results table — in the section devoted to Student's test)\nIt is important to emphasize that although the F test from a mathematical point of view in the considered analysis by Student's and Fisher's tests is the same as in ANOVA (and expresses the ratio of variances), its meaning in the analysis results presented by the summary table is quite different. When comparing by Student's and Fisher's tests, comparison of mean values of samples is carried out by Student's test, and comparison of their variability is carried out by Fisher's test. In the analysis results, not the variance itself is output, but its square root — the standard deviation.\nIn analysis of variance, on the contrary, Fisher's test is used for comparing means of different samples (as we discussed, this is done by dividing the sum of squares into parts and comparing the mean sum of squares corresponding to between- and within-group variability).\nHowever, the given difference concerns rather the presentation of the results of statistical research than its essence. As Glantz (1999, p. 99) notes, for example, comparison of groups by Student's test can be considered a special case of analysis of variance for two samples.\nThus, comparison of samples by Student's and Fisher's tests has one important advantage over analysis of variance: in it, samples can be compared from the point of view of their variability. But the advantages of analysis of variance are still more significant. Among them, for example, is the possibility of simultaneous comparison of several samples."}

Effect

420,0

5,440

0,041874

Error

772,0

77,2