Biostatistics — 05. Topic 4. Comparison of Samples

←

D.A. Shabanov, M.A. Kravchenko. Statistical Analysis of Data in Zoology and Ecology

→

Topic 3 (continued). Working with graphs

Topic 4. Comparison of Samples

Topic 4 (continued). Multiple comparisons

Biostatistics-04

Biostatistics-05

Biostatistics-06

Topic 4. Comparison of samples 4.1. In what situations might it be necessary to compare samples? A significant part of statistical research follows one simple scheme. Very often it is necessary to establish whether two samples belong to the same general population or to different ones. Here are examples of such studies. — Samples of certain animals are collected from a number of locations. It is necessary to determine if there is a significant difference between them that allows us to say that they belong to different species. The question can also be rephrased as: do these samples belong to the same general population, or to different ones? — Does the sex of individuals affect the length of their tails (i.e., do individuals of different sexes differ in tail length)? Another formulation of the problem: can it be assumed that the values of a given trait in individuals of different sexes represent samples from different general populations (tails of females and tails of males) or can they be considered as a sample from one population (tails of individuals of a given species without considering their sex). — How does some factor affect a certain trait (e.g., is the survival rate of experimental animals the same in pure water and in a mixture of pure water and tap water)? Are the samples of experimental animals that developed in pure and test water taken from different populations? — Does a new drug improve the condition of patients compared to those suffering from the same diseases who receive traditional treatment? Are the samples of patients treated traditionally and with new methods taken from the same or different general populations? Thus, based on the composition of the samples, it should be understood whether they are taken from the same general population or from different ones. Such a comparison fully corresponds to the humorous example used to discuss the concept of statistical significance in the first topic.

{ "text": "[IMG_1]\n \n←\n\nD.A. Shabanov, M.A. Kravchenko. Statistical Analysis of Data in Zoology and Ecology\n\n→\n\nTopic 3 (continued). Working with Graphs\n\nTopic 4. Sample Comparison\n\nTopic 4 (continued). Multiple Comparisons\n\n\nBiostatistics-04\n\nBiostatistics-05\n\nBiostatistics-06\n\nTopic 4. Sample Comparison\n4.1. In what situations might it be necessary to compare samples?\nA significant portion of statistical research follows a simple scheme. It is often necessary to determine whether two samples belong to the same population or to different ones. Let us provide examples of such studies.\n— Samples of specific animals are collected from several sites. It is necessary to determine whether there is a significant difference between them that would indicate they belong to different species. This question can be rephrased as follows: do these samples belong to the same population or to different ones?\n— Does the sex of individuals affect their tail length (i.e., do individuals of different sexes differ in tail length)? Another formulation of the problem: can we consider the values of this trait in individuals of different sexes as samples from different populations (female tails and male tails), or can they be treated as a sample from a single population (tails of individuals of the given species regardless of their sex)?\n— How does a certain factor affect a specific trait (e.g., is the survival rate of experimental animals the same in pure water and in a mixture of pure water and tap water)? Are the samples of experimental animals that developed in pure water and in the test water samples from different populations?\n— Does a new drug improve the condition of patients compared to those suffering from the same diseases who receive traditional treatment? Are the samples of patients treated traditionally and with new methods drawn from the same or different populations?\nThus, based on the composition of the samples, we need to determine whether they are drawn from the same population or from different ones. This comparison aligns well with the humorous example used to discuss the concept of statistical significance in the first topic.\n \n4.2. Sample Comparison Using Student's t-test\nIn 1908, the English mathematician W. Gosset, who worked for a brewing company and published his works under the pseudonym Student, described a function that governs the distribution of sample means around the population mean. For large samples, the distributions of sample means follow a normal distribution, while for small samples they are flatter.\nUsing this function, one can determine the probability that two samples are drawn from the same population. The t-test, also known as Student's t-test, is calculated for this purpose. [IMG_2] . In this formula, the numerator is the difference between the means, and the denominator is the standard error of this difference.\nWhen the sample sizes are equal (n1 = n2 = n), calculating the standard error of the difference between means is relatively simpler than in the following case. It is calculated as the square root of the sum of the squared standard deviations of the compared samples:\n [IMG_3].\nIf the sample sizes are unequal, then [IMG_4].\nFor each number of degrees of freedom df (df = n1 + n2 - 2), Student's t-test takes higher values the more significant the differences between the samples. These values are compared with tabulated critical values to determine the statistical significance level of the observed results.\nExample of such a table:\nCritical Values of Student's t-test for Different Statistical Significance Levels\n\nDegrees of\nFreedom df\n\n\np\n\n\nDegrees of\nFreedom df\n\np\n\n\n\n\n0.05\n\n\n0.01\n\n\n0.05\n\n\n0.05\n\n\n0.01\n\n\n0.001\n\n\n\n\n1\n\n\n12.71\n\n\n63.66\n\n\n64.60\n\n\n4\n\n\n2.78\n\n\n4.60\n\n\n8.61\n\n\n\n\n2\n\n\n4.30\n\n\n9.92\n\n\n31.60\n\n\n5\n\n\n2.57\n\n\n4.03\n\n\n6.87\n\n\n\n\n3\n\n\n3.18\n\n\n5.84\n\n\n12.92\n\n\n6\n\n\n2.45\n\n\n3.71\n\n\n5.96\n\n\nThus, the algorithm for \"manual\" use of Student's t-test is quite simple: for two samples, their means are determined, Student's t-test is calculated, and then the statistical significance level of the observed differences is determined using tables (for the corresponding degrees of freedom).\nDue to its simplicity, Student's t-test has become very widely used, far more widely than it should be. The fact is that by its nature, it requires the compared samples to follow a normal distribution. In any case, it is necessary to understand how this test works. Let us use it to compare the body length of male and female frogs described in the file Pelophylax_example.sta. \nTo access one of the sample analysis options using Student's t-test, we follow the path Statistics / Basic Statistics.\n[IMG_5]\nFig. 4.2.1. Statistics Menu\n[IMG_6]\nFig. 4.2.2. Statistics / Basic Statistics Menu\nIn the Statistics / Basic Statistics menu, select the appropriate option for sample comparison using Student's t-test. The dialog box that opens offers four options for such comparison.\nStudent's t-test Application Options\n\nName\n\n\nIcon\n\n\nApplication\n\n\n\n\nt-test, independent, by groups\n\n\nTwo groups of data stacked vertically\n\n\nHomogeneous measurements in a single column, grouped by values in another column\n\n\n\n\nt-test, independent, by variables\n\n\nTwo groups of data in adjacent columns, not necessarily matched (may differ in sample size)\n\n\nMeasurements in two different columns\n\n\n\n\nt-test, dependent samples\n\n\nTwo matched groups of data (equal sample size) in adjacent columns\n\n\nPaired comparisons (e.g., comparing the length of the right and left arm in different individuals)\n\n\n\n\nt-test, single sample\n\n\nSingle group of data\n\n\nDifferences between the group mean and a specified value (e.g., 0)\n\nNaturally, our case corresponds to the \"t-test, independent, by groups\" option. It is clear that the Dependent variables (the variable(s) whose variability is being analyzed) is variable L, and the Grouping variable is variable Sex, which contains the values that split the values of variable L into groups corresponding to females and males.\n[IMG_7]\nFig. 4.2.3. T-Test Independent Sample by Groups Dialog Box\nClicking the Summary button in the upper right corner of the T-Test Independent Sample by Groups dialog box opens a separate page with the analysis results.\n[IMG_8]\nFig. 4.2.4. Analysis Results\nThe results table shown in Fig. 4.2.4 should be examined in more detail. Let us go through all its columns in order.\nVariable— the variable for which data are provided (if we had specified several Dependent variables, the table would have multiple rows).\nMean female— Mean value for the first sample.\nMean male— Mean value for the second sample.\nt-value— Value of Student's t-test (for a given number of degrees of freedom, it can be compared with tabulated critical values).\ndf— Number of degrees of freedom.\np— Statistical significance level of differences between the compared samples using Student's t-test.\nValid N female— Sample size of the first sample.\nValid N male— Sample size of the second sample.\nStd. Dev. female— Standard deviation of the first sample.\nStd. Dev. male— Standard deviation of the second sample.\nF-ratio Variances— Fisher's F-ratio: the ratio of the larger variance (square of the standard deviation) to the smaller variance.\np Variances— Statistical significance level of differences between the compared samples using Fisher's F-test.\nIn field biological research, it is customary that the alternative hypothesis is accepted (and the null hypothesis rejected) when the statistical significance level is less than 0.05. What does this mean in the case of comparison using Student's t-test? It means that differences between means as large as those observed occur due to random sampling error no more than 5% of the time (i.e., no more than once in twenty cases). In our example, p=0.76. This means that differences as large as those we observed occur in more than three-quarters of cases purely due to random sampling error. Such differences are statistically non-significant. There is no basis to reject the null hypothesis.\nDoes this mean that the body length of female and male green frogs does not differ? No. In fact, analysis of much larger samples shows that female green frogs are larger than males. However, in this study, using the material examined, the differences proved statistically non-significant. Increasing the sample size several times or examining more homogeneous material (e.g., individuals of the same genotype from a single habitat) would likely yield statistically significant differences.\n \n4.3. Using Fisher's F-test for Sample Comparison\nAs you can see, the results of our sample comparison shown in Fig. 4.3.4 also include calculation of measures of variability (standard deviations) and Fisher's F-test (variance ratio). This test was proposed by the American statistician George Snedecor, who named it in honor of Fisher. It is the same test used in analysis of variance (ANOVA, discussed in the next topic), but in this case it is used for different purposes.\nFirst of all, it is necessary to understand the rationale for using Fisher's F-test.\n[IMG_9]\nFig. 4.3.1. Comparison of Three Distributions. Distributions 1 and 2 differ in mean values, while 1 and 3 differ in variability level\nAs already mentioned, Student's t-test is a parametric method based on the assumption that the populations from which the compared samples are drawn follow a normal distribution. Normal distributions (see Section 1.5) are defined by two parameters: [IMG_10] (population mean) and [IMG_11] (population standard deviation). If two normal distributions differ from each other, then either their means, their standard deviations, or both differ. Therefore, to determine the probability that the compared samples are drawn from the same population, we must compare both their means and their measures of variability.\nIn our example, the statistical significance level of differences between the compared samples using Fisher's F-test is also below the critical value; p=0.86.\n \n4.4. Box-and-Whisker Plots in the t-test Module\nFrom the T-Test Independent Sample by Groups dialog box, you can open not only the window with numerical comparison results, but also a plot that allows comparing two distributions. Clicking the Box & whisker plot button will generate the following box-and-whisker plot.\n[IMG_12]\nFig. 4.4.1. Box-and-Whisker Plot Constructed from Body Length Data of Female and Male Frogs\nExplanations of the notations are given next to the plot. The dot in the middle of each \"box\" represents the mean of each sample, the \"box edges\" represent the mean plus/minus the standard error (SE), and the \"whiskers\" represent the mean plus/minus SE multiplied by 1.96. Standard Error (SE) is a measure of variability defined as [IMG_13] . Its use is not recommended by modern statistical guidelines; it is unclear why this particular measure is used in this module. A more \"robust\" measure for assessing the variability of the variable under study is the confidence interval (calculated for a specified statistical significance level). The confidence interval (CI) shows the range within which the population mean, estimated from the variability of the sample drawn from that population, lies with a given probability.\nTo display measures other than the default ones on the box-and-whisker plot, right-click on the graph, select the Graph Properties (All Options)… option, in the window that opens select the Plot: Box/Whisker tab, and then click the More button (while marveling at how deeply important settings are hidden).\n[IMG_14]\nFig. 4.4.2. Editing the Box-and-Whisker Plot Constructed from Body Length Data of Female and Male Frogs\nIn the window that opens, you can, for example, set the box to display confidence interval values, and the whiskers to display minimum and maximum values (do not forget to correct the coefficient 1.96 by which these values are multiplied, replacing it with 1). \n[IMG_15]\nFig. 4.4.3. Final Version of the Box-and-Whisker Plot Reflecting Comparison of Female and Male Frogs by Body Length" }

Number of degrees of freedom df

0,05

0,01

0,05

0,01

0,001

12,71

63,66

64,60

2,78

4,60

8,61

4,30

9,92

31,60

2,57

4,03

6,87

3,18

5,84

12,92

2,45

3,71

5,96

So, the algorithm for 'manual' use of the Student's t-test is quite simple: for two samples, their means are determined, the Student's t-test is calculated, and then from tables (for the corresponding number of degrees of freedom) the level of statistical significance of the registered differences is determined. Due to its simplicity, Student's t-test has become very widely used, much more widely than it should have been. The fact is that it essentially requires that the compared samples have a normal distribution. In any case, it is necessary to understand how this criterion works. Let's compare the body length of male and female frogs described in the file Pelophylax_example.sta using it. To call one of the sample analysis options using Student's t-test, we must go through Statistics / Basic Statistics. Fig. 4.2.1. Statistics Menu Fig. 4.2.2. Statistics / Basic Statistics Menu In the Statistics / Basic Statistics menu, you need to select the appropriate option for comparing samples using Student's t-test. There are four options for such comparison in the proposed window. Options for applying Student's t-test

Name

Icon

Application

t-test, independent, by groups

Two groups of data one above the other

Homogeneous measurements in one column, divided into groups by values in another column

t-test, independent, by variables

Two groups of data in adjacent columns, not necessarily corresponding to each other (may differ in size)

Measurements in two different columns

t-test, dependent samples

Corresponding to each other (equal size) two groups of data in adjacent columns

Paired comparisons (e.g., comparing the lengths of the right and left hands of different people)

t-test, single sample

A single group of data

Difference between the mean of a data group and some value (e.g., 0)

Obviously, our case corresponds to the option 't-test, independent, by groups'. It is clear that Dependent variables (the variable or variables whose variability is being analyzed) is the variable L, and Grouping variable is the variable Sex, which contains the values that divide the values of variable L into groups corresponding to females and males. Fig. 4.2.3. T-Test Independent Sample by Groups Dialog Clicking the Summary button, located in the upper right corner of the T-Test Independent Sample by Groups dialog, will open a separate page with the analysis results. Fig. 4.2.4. Analysis Result The table with the results shown in Fig. 4.2.4 should be examined in detail. Let's examine all its columns sequentially. Variable— the variable for which the data is presented (if we had specified several Dependent variables, there would be several rows in the table). Mean female— Mean value for the first sample. Mean male— Mean value for the second sample. t-value— Student's t-test value (for a certain number of degrees of freedom, it can be compared with tabular values). df— Number of degrees of freedom. p— Level of statistical significance of the differences between the compared samples using Student's t-test. Valid N female— Size of the first sample. Valid N male— Size of the second sample. Std. Dev. female— Standard deviation of the first sample. Std. Dev. male— Standard deviation of the second sample. F-ratio Variances— Fisher's F-test: the ratio of the larger variance (square of the standard deviation) to the smaller variance. p Variances— Level of statistical significance of the differences between the compared samples using Fisher's F-test. In biological research, it is accepted that the basis for accepting the alternative hypothesis (and rejecting the null hypothesis) is a statistical significance level less than 0.05. What does this mean in the case of comparison using Student's t-test? That differences in mean values, such as those registered, occur randomly during sample formation no more often than with a probability of 0.05 (i.e., no more often than in one case out of twenty). In our example, p=0.76. This means that the same differences as those we registered occur in more than three-quarters of cases simply due to randomness in sample formation. Such differences are statistically insignificant. There are no grounds for rejecting the null hypothesis. Does this mean that the body length of male and female green frogs does not differ? No. In fact, based on the analysis of much larger samples, it can be stated that female green frogs are larger than males. However, in this study, based on the material examined, the differences turned out to be statistically insignificant. By increasing the sample size several times or by examining more homogeneous material (e.g., representatives of the same genotype from the same habitat), we would likely obtain statistically significant differences.

4.3. Using the Fisher's F-test for Sample Comparison As you can see, the results of our sample comparison, shown in Fig. 4.3.4, also include the calculation of the measure of variability (standard deviations) and Fisher's F-test (variance ratio). This test was proposed by the American statistician George Snedecor, who named it in honor of Fisher. It is the same test used in analysis of variance (discussed in the next topic), but in this case, it is applied for different purposes. First of all, one needs to understand why Fisher's F-test is used. Fig. 4.3.1. Comparison of three distributions. Distributions 1 and 2 differ in mean values, while 1 and 3 differ in variability level. As already mentioned, Student's t-test is a parametric method based on the assumption that the populations from which the compared samples are drawn have a normal distribution. Normal distributions (see section 1.5) are defined by two parameters: \( \mu \) (population mean) and \( \sigma \) (population standard deviation). If two normal distributions differ from each other, it means that either their means or their standard deviations (or both simultaneously) differ. Therefore, to determine the probability that the compared samples are obtained from the same population, one must compare both their mean values and their measures of variability. In the example we considered, the level of statistical significance of the differences between the compared samples using Fisher's F-test is also below the critical value; p=0.86.

4.4. Box Plots in the t-Test Module From the T-Test Independent Sample by Groups dialog box, you can access not only the window with numerical results of data comparison but also a graph that allows comparing two distributions. Clicking the Box & whisker plot button will generate the following box plot. Fig. 4.4.1. Box plot generated from data on the body length of female and male frogs. The legends are provided next to the graph. The dot in the middle of each 'box' represents the mean of each sample, the 'walls of the box' represent the mean plus or minus the standard error (SE), and the 'whiskers' represent the mean plus or minus the SE multiplied by 1.96. Standard Error (SE) is a measure of variability, defined as . Its use is not recommended by modern statistical guidelines; why it is used in this module is not entirely clear. A more 'qualitative' measure for assessing the variability of the studied variable is the confidence interval (calculated for a specific level of statistical significance). Confidence intervals show the limits within which the population mean, estimated from the variability of a sample taken from this population, lies with a given probability. To display measures other than the default ones on the box plot, right-click on the graph, select the Graph Properties (All Options)… option, and in the opened window, select the Plot: Box/Whisker tab, and then the More button (marveling at how deeply hidden important settings are). Fig. 4.4.2. Editing the box plot generated from data on the body length of female and male frogs. In the opened window, you can, for example, set the confidence interval values for the box and the minimum and maximum values for the whiskers (not forgetting to change the coefficient 1.96, by which these values will be multiplied, replacing it with 1). Fig. 4.4.3. The final version of the box plot showing the comparison of female and male frogs by body length.