{"id":360,"date":"2019-03-12T07:07:34","date_gmt":"2019-03-12T07:07:34","guid":{"rendered":"http:\/\/esp14.epgpbooks.inflibnet.ac.in\/?post_type=chapter&p=360"},"modified":"2019-03-12T07:32:09","modified_gmt":"2019-03-12T07:32:09","slug":"f-distribution-and-tests-of-significance-based-on-f-distribution","status":"publish","type":"chapter","link":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/chapter\/f-distribution-and-tests-of-significance-based-on-f-distribution\/","title":{"rendered":"F-distribution and tests of significance based on F distribution"},"content":{"raw":"
\r\n\r\n\u00a0 \u00a0 1.\u00a0Introduction<\/strong>\r\n\r\n \r\n

F-distribution is an important probability distribution which is used in a number of statistical tests of significance, most famous among which is ANOVA used to compare means of more than two groups. The main problem with multiple t-tests is that significance level applies to each comparison and as number of comparisons increases, chance of false positives greatly increases and test results would become unreliable. One way ANOVA tests whether means of all groups are equal. If ANOVA returns a P value <0.05 (or any other threshold significance level that we decide as part of the experimental design), it would mean that at least one group mean is different from the rest. In most cases we are more interested to know which group means are significantly different from the rest rather than ANOVA P value itself. This can be tested by multiple comparisons posttests, famous one being Tukey\u2019s HSD. Tukey\u2019s HSD calculates 95% CI of the difference between each pairs of group means adjusted for multiple comparisons using a critical value from q-distribution. Tukey\u2019s HSD depends not merely of the two groups that the test analyses, but also all other groups and in fact every single value. Interpretation of Tukey\u2019s HSD Confidence Intervals are straight forward; if the range includes zero-the null hypothesis of no difference, then the difference in means of the pair of groups are significant. Other posttests include Dunnett\u2019s test which is highly useful to compare multiple groups with a group defined as a control group (each test group will be compared with control group, but there wont be any comparison between test groups), and Scheffe\u2019s test to compare \u2018contrasting\u2019 sets of group means (for example, groups {A, B and C} versus groups {D and E}).<\/p>\r\n \r\n

2.\u00a0<\/strong>Learning Outcome:<\/strong><\/p>\r\n \r\n

a. To learn about the properties of F-distribution and statistical tests of significance based upon F distribution.<\/span><\/p>\r\n

b. To learn why uncorrected multiple comparisons like multiple t-tests to compare means of groups is problematic.<\/span><\/p>\r\n

c. To learn <\/span>principles and assumptions of one-way ANOVA and how to compute it by hand<\/span><\/p>\r\n

d. To know more about other kinds of ANOVA including two-way ANOVA, Repeated measures ANOVA, Random Effects ANOVA, MANOVA and AMOVA<\/span><\/p>\r\n

e.\u00a0To learn how to perform multiple comparison post hoc tests including Tukey\u2019s HSD and\u00a0<\/b><\/span>Bonferroni adjustment<\/span><\/p>\r\n \r\n

3.<\/strong>\u00a0<\/span>F Distribution<\/strong><\/p>\r\n \r\n

F-distribution (Fisher distribution) is a type of continuous probability distribution with exact probabilities of every F ratios under the assumption of <\/span>null hypothesis. This enables us to calculate <\/span>P value from a given F-ratio. We have seen how F-ratio is calculated in module 18 while testing for the assumption that two groups have <\/span>same variance, prior to performing an unpaired t-test. This is calculated by the formula:<\/span><\/p>\r\n \r\n

(s1\/s2)2<\/span><\/p>\r\n \r\n

Where s1 and s2 are standard deviations of group 1 and group 2 respectively. More generally, each of these standard deviations is divided by the respective degrees of freedom of each group. Distribution of these F-ratios plotted as in a probability histogram is called F-distribution. Like lognormal distribution and Chi-Square distribution, F-distribution is right-skewed (with a long tail towards <\/span>right. The shape of F-distribution depends only on degrees of freedom (df) of <\/span>numerator (DFn) and df of <\/span>denominator (DFd).<\/span><\/p>\r\n\r\n<\/div>\r\n

\r\n\r\n\"\"\r\n\r\n \r\n

As one can see, when both of the df is small, <\/span>shape is distinctly skewed to the right. With high df, distribution becomes bell-shaped, although it would still be skewed to the right.<\/span><\/p>\r\n \r\n

At each combination of <\/span>these two df, F-distribution can be calculated using <\/span>intricacies of calculus. At each of these distributions at each combination of DFn and DFd, we can calculate the area under the significance level (the threshold P value, alpha) at the tails. These values can be presented in a tabular format, the so called F-distribution tables, where one can lookup a value called F-critical. One can compare this F-critical value found from the table and the F-ratio calculated from the data to make inferences about statistical significance. If F-critical at <\/span>significance level of 0.05 is less than F-ratio, we can infer that P value must be less than 0.05 and can reject the null hypothesis.<\/span><\/p>\r\n \r\n

4. Tests of significance based on F distribution<\/strong><\/p>\r\n \r\n

F distribution is used to test whether two groups have equal variances, one of the assumptions of <\/span>unpaired t-test, as we have seen in module 18. The most important use of F-distribution is for F-test, also known as ANOVA (Analysis of Variance), for comparing means of three or more groups, as discussed in this module. Keep in mind that ANOVA merely compares group means and informs us whether all groups have identical means or not. Means would not be identical if <\/span>mean of one group is different from the rest. To know which group means are significantly different, we should do multiple comparisons posttest, as explained later in this module.<\/span><\/p>\r\n \r\n

5.<\/strong>\u00a0<\/span>Problem with multiple t-tests<\/strong><\/p>\r\n \r\n

Consider three groups of patients (X, Y <\/span>and Z), each group treated with <\/span>different drug. An intuitive approach to compare group means would be performing multiple t-tests. For example, t-test to compare means of X and Y, another one between Y-Z, yet another one between X and Z. In any of these t-tests if P<0.05 was obtained, we can conclude that group means as not identical. If all of these t-tests returns P>0.05, we can conclude that group means are identical. Though sound appealing, multiple <\/span>comparison like these should not be performed as it drastically increases random <\/span>error .In one t-test at 0.05 significance level, we still expect to find 5% of \u201cfalse positives\u201d (a blood test tells you are HIV+ but in reality you are HIV-); i.e, t-test yielding a significant P value even if two group means are identical due to chance alone. With two or more t-tests, <\/span>proportion of expected false positives are not merely 5% but well above this. Therefore multiple comparisons, like multiple t-tests, should never be used.<\/span><\/p>\r\n \r\n

6.<\/strong>\u00a0<\/span>One-way ANOVA<\/strong><\/p>\r\n \r\n

One-way ANOVA, also called one-factor ANOVA compares means of three or more groups to find differences between means of these groups are statistically significant or not. ANOVA can also be used to compare means of two groups, but in this <\/span>case ANOVA is indistinguishable from unpaired t-test explained previously and returns identical P values from <\/span>t-test. ANOVA was developed by population geneticist and founder of statistics, Ronald Fisher.<\/span><\/p>\r\n \r\n

7.<\/strong>\u00a0<\/span>Assumptions for ANOVA<\/strong><\/p>\r\n \r\n

Like t-test, One-way ANOVA is based on a set of familiar assumptions:<\/span><\/p>\r\n

1.\u00a0\u00a0\u00a0\u00a0\u00a0 Random samples<\/span><\/p>\r\n

2.\u00a0\u00a0\u00a0\u00a0\u00a0 Independent measurements<\/span><\/p>\r\n

3.\u00a0\u00a0\u00a0\u00a0\u00a0 Accurate Data<\/span><\/p>\r\n

4.\u00a0\u00a0\u00a0\u00a0\u00a0 Data are sampled from populations that are approximately Gaussian<\/span><\/p>\r\n

5.\u00a0\u00a0\u00a0\u00a0\u00a0 Variances, or Standard deviations, of populations from which the samples came <\/span>from are\u00a0<\/span>identical<\/span><\/p>\r\n \r\n

To test <\/span>4th assumption, a formal statistical test like D\u2019Agostino Pearson Omnibus K2 test can be performed. In case the populations significantly deviate from Gaussian, <\/span>first option should be trying to see any normalizations transforms the distributions to normal (normalization). For example, a number of biological distributions like enzyme kinetics and wherever the term \u2018half-maximal\u2019 is used are lognormal. It can be normalized by converting each <\/span>values to <\/span>logarithm. If none of the normalizations works, then a non-parametric test like Kruskal-Wallis test, followed by Dunn\u2019s posttest should be preferred. For paired (matched) data that came from non-Gaussian, Friedman\u2019s test followed by Dunn\u2019s posttest should be preferred. A statistical package like Graphpad Prism can be used for both of these non-parametric tests. <\/span>5th assumption of equal variance is not usually tested prior to ANOVA, as testing for this assumption is built-in ANOVA itself.<\/span><\/p>\r\n \r\n

8.\u00a0\u00a0 One-way ANOVA by hand<\/strong><\/p>\r\n \r\n

Let us consider an example, Increase in mouse colorectal tumor by four different heavy metals after feeding for a period of time.<\/span><\/p>\r\n\r\n<\/div>\r\n

\r\n\r\n\"\"\r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\nLet us first define our null hypothesis and alternative hypotheses:\r\n\r\nH0: All group means are equal\r\n\r\nHa: At least one group mean is different from the rest\r\n\r\n \r\n

As we can see, there are four groups (arranged by columns: U, Pb, As and Hg) and four technical replicate measurements (also called levels) for each group, arranged by rows (only three for As though). The numbers (elements, measurements) means <\/span>increase in <\/span>tumour, so in <\/span>effect there is only one factor (increase in <\/span>tumour mass) across four groups. That is why the ANOVA is called one-way or one factor ANOVA. A factor is an independent treatment variable whose settings (values) are controlled and varied by the experimenter. The intensity setting of a factor is the level. Levels may be quantitative numbers or, in many cases, simply \"present\" or \"not present\" (\"0\" or \"1\"). Had there been another factor like <\/span>age of mouse, then we would need to do two-factor ANOVA. Use one-way for comparing <\/span>1 factor means of >2 groups. For eg., average heights of neem trees at three villages; average ground water As levels at 4 districts in Gujarat.<\/span><\/p>\r\n \r\n

For one-way ANOVA, the first step is to calculate mean, standard deviation and variance for each of these groups, as already computed in the table above. We can also compute overall (total) mean, Standard Deviation and variance of our entire Dataset (n=15). ANOVA starts with <\/span>calculation of \u201cSum of Squares,\u201d SS. SS is defined as the square of deviations of values from <\/span>sample mean (\u2211(x-x\u0304)2). Remember that we have used this while calculating standard deviation from raw data.<\/span><\/p>\r\n \r\n

First let us calculate <\/span>SS<\/strong>total<\/strong>, Sum of Squares of difference between each of these 15 values and the overall mean (x\u0304, 77.79).<\/span><\/p>\r\n \r\n

(60.8-77.79)2+ (67-77.79)2+\u2026\u2026\u2026..(90.3-77.79)2<\/span><\/p>\r\n\r\n<\/div>\r\n

\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
<\/td>\r\n<\/td>\r\nx-x\u0304<\/td>\r\n(x-x\u0304)2<\/td>\r\n<\/tr>\r\n
1<\/td>\r\n60.8<\/td>\r\n-16.99<\/td>\r\n288.6601<\/td>\r\n<\/tr>\r\n
2<\/td>\r\n67<\/td>\r\n-10.79<\/td>\r\n116.4241<\/td>\r\n<\/tr>\r\n
3<\/td>\r\n54.6<\/td>\r\n-23.19<\/td>\r\n537.7761<\/td>\r\n<\/tr>\r\n
4<\/td>\r\n61.7<\/td>\r\n-16.09<\/td>\r\n258.8881<\/td>\r\n<\/tr>\r\n
5<\/td>\r\n78.7<\/td>\r\n0.91<\/td>\r\n0.8281<\/td>\r\n<\/tr>\r\n
6<\/td>\r\n77.7<\/td>\r\n-0.09<\/td>\r\n0.0081<\/td>\r\n<\/tr>\r\n
7<\/td>\r\n76.3<\/td>\r\n-1.49<\/td>\r\n2.2201<\/td>\r\n<\/tr>\r\n
8<\/td>\r\n79.8<\/td>\r\n2.01<\/td>\r\n4.0401<\/td>\r\n<\/tr>\r\n
9<\/td>\r\n92.6<\/td>\r\n14.81<\/td>\r\n219.3361<\/td>\r\n<\/tr>\r\n
10<\/td>\r\n84.1<\/td>\r\n6.31<\/td>\r\n39.8161<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/div>\r\n
\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
11<\/td>\r\n90.5<\/td>\r\n12.71<\/td>\r\n161.5441<\/td>\r\n<\/tr>\r\n
12<\/td>\r\n86.9<\/td>\r\n9.11<\/td>\r\n82.9921<\/td>\r\n<\/tr>\r\n
13<\/td>\r\n82.2<\/td>\r\n4.41<\/td>\r\n19.4481<\/td>\r\n<\/tr>\r\n
14<\/td>\r\n83.7<\/td>\r\n5.91<\/td>\r\n34.9281<\/td>\r\n<\/tr>\r\n
15<\/td>\r\n90.3<\/td>\r\n12.51<\/td>\r\n156.5001<\/td>\r\n<\/tr>\r\n
<\/td>\r\n<\/td>\r\nTotal<\/strong><\/td>\r\n1923.41<\/strong><\/td>\r\n<\/tr>\r\n
<\/td>\r\n<\/td>\r\n(SS<\/strong>total<\/strong>)<\/strong><\/td>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n \r\n

Let us then consider SS<\/strong>between<\/strong>, \u201cSum of Squares\u201d between group means. We have 4 groups, so as four group means. To compute SS<\/strong>between<\/strong>, we have to subtract each group mean (x) with overall mean (x\u0304, 77.79), square this number, and multiplied with size of each of these groups (to weigh by sample size). Sum of all these values in our 5th column is SS<\/strong>between<\/strong>.<\/p>\r\n\"\"\r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\nWe need one more Sum of Squares, i.e., within groups SS<\/strong>within<\/strong>.\r\n\r\n \r\n\r\nSS<\/strong>total<\/strong> = SS<\/strong>between<\/strong> and SS<\/strong>within<\/strong> (derivation of which is omitted for the sake of simplicity)\r\n\r\n\u2234\u00a0\u00a0\u00a0\u00a0 SS<\/strong>within<\/strong>= SS<\/strong>total<\/strong> - SS<\/strong>between<\/strong>\r\n\r\n=1923.41- 1761.128\r\n\r\n=162.282\r\n\r\nThis SS<\/strong>within<\/strong> is also called \u2018residual sum of squares\u2019 or \u2018error sum of squares\u2019\r\n\r\n \r\n

Next step is to calculate degree of freedom for each of these three Sum of Squares. For SS<\/strong>between<\/strong>, we have four groups (K, total number of groups); as df=(n-1), df is 4-1= 3<\/p>\r\n \r\n

Degree of freedom for SS<\/strong>within<\/strong>, is overall sample size (N) minus number of groups (K). Here overall sample size is 15 and number of groups is 4. Therefore, 15-4= 11<\/p>\r\n \r\n

Degree of freedom for <\/span>SS<\/strong>total<\/strong> is N-1, 15-1= 11 (also, df for between groups + df for within groups = df for total)<\/span><\/p>\r\n \r\n

Let us put all these numbers in the <\/span>summary table<\/strong> for one-way ANOVA:<\/span><\/p>\r\n\r\n<\/div>\r\n

\r\n\r\n\"\"\r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n

The fourth column is Mean Square (MS, also called variance), which is Sum of Squares divided by df for each of these sources. Sum of MS to calculate total MS makes no sense, so this is not done. Please note that MSwithin is perhaps the most important statistic in an ANOVA test; it tells us about the variance due to error and is important for calculating \u2018margin of error\u2019 required for standardized error computations in posttests (like Tukey\u2019s) to know which means are significantly different, as explained later in this module. Finally, the fifth column is F-ratio, which is the ratio of these two MS values (MSbetween\/MSwithin).<\/p>\r\n \r\n

As in the case of F-test to detect the equality of variance (assumption of homoscedasticity) we did prior to unpaired t-test, here too we should look up F table for critical F value. Df for F Ratio is expressed with DFn =3 and DFd=11. Critical F value corresponding to these two df at 0.05 significance level is:<\/p>\r\n \r\n\r\n\"\"\r\n\r\n<\/div>\r\n \r\n

As table value (critical F, 3.587) is far less than our obtained F ratio (39.7917), we can conclude that P<0.05, null hypothesis of equal means are rejected and concluded that at least one of the group mean is significantly different from the rest. There are many F tables corresponding to different significance levels (these are accessible at <\/span>http:\/\/www.socr.ucla.edu\/applets.dir\/f_table.html<\/a>). Perhaps we are interested whether this F ratio is still higher than the critical F at lower significance levels, let us say 0.01 or 0.001. We can see that even at <\/span>significance level of 0.001 level, F critical from <\/span>table (11.56) is still less than our obtained F ratio, so our actual P value must be very less. One-way ANOVA for the above data done using excel returns a P value of 3.36E-06 (read this value as 3.36 x 10-6 i.e, 0.00000336). We can also use an online calculator like the one at <\/span>http:\/\/stattrek.com\/online-calculator\/f-<\/a>distribution.aspx <\/a>(keep in mind that this calculator returns a cumulative probability; we have to subtract this value from 1 to get empirical probability). A low P value <\/span>indicate that the group means are significantly different. It could be because one of the four means is different from the rest three, or two of the group means different from the rest two. It could also be due to all four means different from each other. None of these finer details are revealed by one-way ANOVA P value. To find those out we have to perform an appropriate multiple comparison posttest as explained below.<\/span><\/p>\r\n \r\n

9.<\/strong>\u00a0<\/span>Other types of ANOVA<\/strong><\/p>\r\n \r\n

a)\u00a0<\/strong>Repeated Measures ANOVA: <\/em><\/strong>As a paired t-test is used to compare two groups with paired\u00a0<\/span>(matched or dependant) measurements, we use \u2018Repeated Measures ANOVA for matched measurements for more than 2 groups (these are called matched sets or matched blocks). Whenever a value in any particular group is expected to be closer to a specific value in another group than to a random value from our whole experiment (because of the way we designed the experiment), we should use repeated measures ANOVA. To find those out which groups are significantly different, we have to perform an appropriate multiple comparison posttest as explained below, similar to normal one-way ANOVA.<\/span><\/p>\r\n \r\n

b)\u00a0<\/strong>Two-way ANOVA and more: <\/em><\/strong>Two-way (or two-factor) ANOVA is used for comparison involving two factors. As already explained, in addition to heavy metals, we might also be interested to know the effects of their age on <\/span>tumour mass (so variable remains only one). Use two-way ANOVA for comparing <\/span>2 factor means of >2 groups. Examples are average heights of neem trees grown using 4 different fertilizers at three villages<\/span>; and average As levels at shallow\/moderate\/deep wells in 4 districts. If we add yet another factor, like sex, we will have to perform 3 <\/span>factor ANOVA and so on. In the case of One-way ANOVA, <\/span>null hypothesis is that means of groups are <\/span>same. In case of Two-way ANOVA, null hypothesis takes up three possibilities: 1) There is no difference in the means of factor A across the groups, 2) There is no difference in means of factor B across the groups and 3) There is no interaction between factors A and B<\/span><\/p>\r\n \r\n

c)\u00a0<\/strong>Multivariate ANOVA (MANOVA): <\/em><\/strong>Manova is used for comparing the effect of <\/span>single categorical variable (exposure to tobacco) on the averages of two or more continuous variables (for eg: heart rate and blood pressure)<\/span><\/p>\r\n \r\n

d)\u00a0<\/strong>Random Effects ANOVA: <\/em><\/strong>Normal ANOVA is technically called fixed-effects or Type-I ANOVA; in this <\/span>case the test assess means of our selected groups only without any further extrapolations. In the case of Random Effects ANOVA, also called Type-II ANOVA, the test assumes that the groups that we selected are randomly selected representatives (our sample groups) of an infinite number of groups (our unknown population) and the test compares our sample group means to infer whether our infinite population group means are significantly different or not.<\/span><\/p>\r\n \r\n

e)\u00a0<\/strong>Analysis of Molecular Variance (AMOVA): <\/em><\/strong>This is a method to detect population differentiation (a large contiguous population fragmenting to small isolated populations using molecular markers. While AMOVA had been developed in lines of ANOVA, two methods are quite different.<\/span><\/p>\r\n \r\n

10. Multiple Comparisons Posttests<\/strong><\/p>\r\n \r\n

One-way ANOVA informs us whether the overall differences in the means of our groups are significant or not. As explained already, a low P value in one-way ANOVA could be due to the fact that only one of the mean is significantly deferring from the rest, or groups of means differing from other groups of <\/span>mean (the so called \u2018contrast\u2019), or every <\/span>mean differing from every other means significantly. In most of the situations, ANOVA P value is not what we want to know; we would be more interested to know which means are significantly different. To find that out, we should perform a multiple comparisons posttest (also called Post Hoc test).<\/span><\/p>\r\n \r\n

Multiple <\/span>t-test is one way to compare pairs of means after ANOVA; however, as already explained this should not be used as expected false positives (Type I error) would be far higher than 5% level. This is because <\/span>significance level (for example, 0.05) is for each comparison involved, not for the entire family of comparisons. Between four groups, 6 comparisons are possible (no. of possible comparisons can be calculated by combination formula, nCr= n! \/(n-r)!.r!), so after performing these comparisons, our significance level would be far higher than 0.05 ( that would mean a number of false positives in our results). In case multiple <\/span>t-test is used, each individual P value should be multiplied with <\/span>total number of such comparisons. For example, an unadjusted P value of 0.01 is obtained for one pair of comparison in a group of 4. As <\/span>total number of possible comparisons in a group of four is 6, the unadjusted P value <\/span>need to be multiplied with 6 to get the corrected P value. 0.01*6 = 0.06. This correction is known as <\/span>Bonferroni adjustment<\/strong>. Bonferroni adjustment is fine in case you want to compare a specific pair after the ANOVA provided that this pair is clearly specified as part of the experimental design. As already mentioned, after seeing the data and choosing the highest and lowest means, you would be effectively comparing not merely <\/span>this two means but all six means in a group of four.<\/span><\/p>\r\n \r\n

In case we would like to compare all combinations of means, which is by far most of us would like to do after ANOVA, the best option is Tukey\u2019s Honestly Signified Differences (Tukey\u2019s HSD). Bonferroni\u00a0<\/span>test should not be used for this purpose, as the statistical power for which is far less than Tukey\u2019s HSD. As already explained, Tukey\u2019s HSD depends on a key output of ANOVA, MSwithin. As in Confidence Interval of mean explained in module 16, or CI of differences between means explained in module 18, Tukey\u2019s test returns CIs for all pairwise combinations of the differences between group means. For four groups, there would be six pairwise comparisons (<\/span>difference between means) and these would be expressed in Confidence Intervals at our selected Confidence Level (for example 95%).<\/span><\/p>\r\n \r\n

The formula for Tukey\u2019s HSD is:<\/span><\/p>\r\n\r\n

\r\n\r\n\"\"\r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n

Where x\u0304i and x\u0304j are means of groups i and j. q is a critical value from q distribution, MSw is MSwithin from ANOVA output, and ni and nj are sizes of groups I and j respectively. The above equation is very similar to other equations of Confidence Intervals; difference between sample means \u00b1 w (the width of CI). The width of CI is calculated from standardized (instead of \u2018standard\u2019) error (the square-root term in above equation) multiplied by a critical value from q distribution (instead of t*). The critical value q depends upon three things; the chosen level of significance (\u03b1), total number of groups (r) and degree of freedom for MSwithin (dfw). q Distribution is studentized range for \u03b1. Also note that though we are only comparing means of two of our groups (i and j), all groups and every single value in our dataset is important, as the value MSwithin depends upon the whole dataset. This is needed to account for the perils of multiple comparisons as already explained.<\/p>\r\n \r\n

Let us go back to our original example. To compare means of Pb vs U (Lead vs Uranium, our first two groups), we need to first calculate the difference between their respective group means. Mean for Pb is 78.13 and mean for U is 61.03, so difference is 17.1.<\/p>\r\n \r\n\r\nLet us first define our null hypothesis and alternative hypotheses:\r\n\r\nH0: Difference between two means is zero\r\n\r\nHa: Difference between two means is not zero\r\n\r\n \r\n\r\nNext step is to calculate the width of 95% CI (so \u03b1=0.05). To calculate it we need the following information: MSwithin, dfw, nPb, nU, \u03b1, r and q. MSwithin<\/sub>=14.753, dfw<\/sub>=11, nPb=4, nU=4. \u03b1=0.05, r=4 and dfw=11. Plugging the values in:<\/span>\r\n\r\n \r\n\r\nw=[\u221a(14.753\/2)x (1\/4+1\/4)] x q (\u03b1=0.05, r=4 and df<\/span>w<\/sub>=11)<\/span>\r\n\r\n \r\n\r\nq- <\/span>table is not normally provided in <\/span>statistics tests of tables, as it is hard to calculate. One version of <\/span>online calculator can be found at <\/span>http:\/\/www.vassarstats.net\/tabs.html#q<\/a>\r\n\r\n<\/div>\r\n

\r\n\r\n\"\"\r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\nAs seen, Q at \u03b1=0.05 is 4.27. Plugging this in the above equation:\r\n\r\n=[\u221a(14.753\/2)x (1\/4+1\/4)] x 4.27\r\n\r\n=\u221a (7.365x 0.5) x 4.27\r\n\r\n=1.919 x 4.27\r\n\r\n=8.2\r\n\r\n \r\n

The above value is the width, it extends both sides of the difference between means (17.1). So, 95% Confidence Interval for this difference is:<\/p>\r\n(17.1 \u2013 8.2) to (17.1 + 8.2)\r\n\r\n8.9 to 25.3\r\n\r\n \r\n

Does this range include our null hypothesis of no difference in means, i.e., zero? No, it doesn\u2019t. Therefore, P<0.05 and the difference is statistically significant.<\/p>\r\n \r\n

We can perform the same pairwise computation of 95% CI for all six combinations of group means as well to find how many of these combinations returns statistically significant differences. Confidence Interval of difference between means tells us whether P value is significant at our chosen significance level. Some softwares (like GraphPad prism) also returns an exact P value, called \u2018multiplicity-adjusted P value\u2019 (that had been adjusted for multiple comparisons).<\/p>\r\n \r\n

It is possible that the one-way ANOVA returns a P<0.05 yet none of the pairs of mean differences are significant. It is also possible that ANOVA returns P>0.05, yet some of the difference between group means are significant.<\/span><\/p>\r\n \r\n

Other post hoc multiple comparison methods other than Bonferroni\u2019s test and Tukey\u2019s HSD) include Dunnett\u2019s test which is highly useful to compare multiple groups with a group defined as a control group (each test group will be compared with <\/span>control group, but <\/span>there wont be any comparison between test groups), Scheffe\u2019s test to compare \u2018contrasting\u2019 sets of group means (for example, groups {A, B and C} versus groups {D and E}) and Holm\u2019s test.<\/span><\/p>\r\n \r\n

An online calculator for ANOVA with Tukey\u2019s HSD is available at <\/span>http:\/\/astatsa.com\/OneWay_Anova_with_TukeyHSD\/<\/a><\/p>\r\n \r\n

The results of Tukey\u2019s HSD for our example are as follows:<\/span><\/p>\r\n\r\n<\/div>\r\n

\r\n\r\n\"\"\r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n

This calculator also include results of other multiple comparison tests, including Bonferroni\u2019s test, Scheff\u00e9 test, and Holm\u2019s test.<\/p>\r\n \r\n

Also note that even though ANOVA returns and overall P value >0.05 (and differences not statistically significant), results of multiple comparisons like Tukey\u2019s HSD, Bonferroni test and Dunn\u2019s test are still valid.<\/p>\r\n \r\n\r\n11. Summary<\/strong>\r\n\r\n<\/div>\r\n

    \r\n \t
  1. For One way ANOVA, we first have to calculate SStotal, SSbetween and SSwithin. With degrees of freedom for all of these Sum of Squares, we can calculate Mean Square by dividing Sum of Squares with df. Finally we can calculate F-ratio by MSbetween\/MSwithin. From two df, number of groups and F-ratio, finally we can calculate P value. One way ANOVA can easily be calculated using a software or many web-based calculators.<\/li>\r\n \t
  2. For matched or paired data, the test analogous to matched t-test for groups more than 2 is repeated measures ANOVA<\/li>\r\n \t
  3. Two-way ANOVA determines how a response is affected by two variables, and any association between the variables.<\/li>\r\n \t
  4. A common mistake is analysis of ANOVA groups by multiple t-tests, but this is highly problematic as <\/span>chance of false positives greatly increases this way.<\/span><\/li>\r\n \t
  5. A popular and powerful multiple comparison post hoc test is Tukey\u2019s HSD that calculate\u00a0<\/span>95% CI of differences between means of pairs of groups. In most cases, results of Post hoc tests like Tukey\u2019s is a lot more informative than ANOVA itself, but Tukey\u2019s test is dependent on results of ANOVA.<\/li>\r\n \t
  6. Even though ANOVA returns and overall P value >0.05 (and differences not statistically significant), results of multiple comparisons like Tukey\u2019s HSD, Bonferroni test and Dunn\u2019s test are still valid.<\/li>\r\n<\/ol>\r\n\u00a0 \u00a0 Quadrant-III: Learn More\/ Web Resources \/ Supporting Materials:<\/strong>\r\n
      \r\n \t
    1. F tables arranged by significance levels http:\/\/www.socr.ucla.edu\/applets.dir\/f_table.html<\/a><\/li>\r\n<\/ol>\r\n
        \r\n \t
      1. One-way ANOVA online calculator http:\/\/stattrek.com\/online-calculator\/f-distribution.aspx<\/a><\/li>\r\n<\/ol>\r\n
          \r\n \t
        1. Online calculator to find q-value from df and k (number of groups) http:\/\/www.vassarstats.net\/tabs.html#q<\/a><\/li>\r\n<\/ol>\r\n
            \r\n \t
          1. online calculator for ANOVA with Tukey\u2019s HSD is available at http:\/\/astatsa.com\/OneWay_Anova_with_TukeyHSD\/<\/a><\/li>\r\n<\/ol>\r\n
              \r\n \t
            1. Online calculator to calculate exact P from F ratio https:\/\/www.graphpad.com\/quickcalcs\/pValue1\/<\/a><\/li>\r\n<\/ol>","rendered":"
              \n

              \u00a0 \u00a0 1.\u00a0Introduction<\/strong><\/p>\n

               <\/p>\n

              F-distribution is an important probability distribution which is used in a number of statistical tests of significance, most famous among which is ANOVA used to compare means of more than two groups. The main problem with multiple t-tests is that significance level applies to each comparison and as number of comparisons increases, chance of false positives greatly increases and test results would become unreliable. One way ANOVA tests whether means of all groups are equal. If ANOVA returns a P value <0.05 (or any other threshold significance level that we decide as part of the experimental design), it would mean that at least one group mean is different from the rest. In most cases we are more interested to know which group means are significantly different from the rest rather than ANOVA P value itself. This can be tested by multiple comparisons posttests, famous one being Tukey\u2019s HSD. Tukey\u2019s HSD calculates 95% CI of the difference between each pairs of group means adjusted for multiple comparisons using a critical value from q-distribution. Tukey\u2019s HSD depends not merely of the two groups that the test analyses, but also all other groups and in fact every single value. Interpretation of Tukey\u2019s HSD Confidence Intervals are straight forward; if the range includes zero-the null hypothesis of no difference, then the difference in means of the pair of groups are significant. Other posttests include Dunnett\u2019s test which is highly useful to compare multiple groups with a group defined as a control group (each test group will be compared with control group, but there wont be any comparison between test groups), and Scheffe\u2019s test to compare \u2018contrasting\u2019 sets of group means (for example, groups {A, B and C} versus groups {D and E}).<\/p>\n

               <\/p>\n

              2.\u00a0<\/strong>Learning Outcome:<\/strong><\/p>\n

               <\/p>\n

              a. To learn about the properties of F-distribution and statistical tests of significance based upon F distribution.<\/span><\/p>\n

              b. To learn why uncorrected multiple comparisons like multiple t-tests to compare means of groups is problematic.<\/span><\/p>\n

              c. To learn <\/span>principles and assumptions of one-way ANOVA and how to compute it by hand<\/span><\/p>\n

              d. To know more about other kinds of ANOVA including two-way ANOVA, Repeated measures ANOVA, Random Effects ANOVA, MANOVA and AMOVA<\/span><\/p>\n

              e.\u00a0To learn how to perform multiple comparison post hoc tests including Tukey\u2019s HSD and\u00a0<\/b><\/span>Bonferroni adjustment<\/span><\/p>\n

               <\/p>\n

              3.<\/strong>\u00a0<\/span>F Distribution<\/strong><\/p>\n

               <\/p>\n

              F-distribution (Fisher distribution) is a type of continuous probability distribution with exact probabilities of every F ratios under the assumption of <\/span>null hypothesis. This enables us to calculate <\/span>P value from a given F-ratio. We have seen how F-ratio is calculated in module 18 while testing for the assumption that two groups have <\/span>same variance, prior to performing an unpaired t-test. This is calculated by the formula:<\/span><\/p>\n

               <\/p>\n

              (s1\/s2)2<\/span><\/p>\n

               <\/p>\n

              Where s1 and s2 are standard deviations of group 1 and group 2 respectively. More generally, each of these standard deviations is divided by the respective degrees of freedom of each group. Distribution of these F-ratios plotted as in a probability histogram is called F-distribution. Like lognormal distribution and Chi-Square distribution, F-distribution is right-skewed (with a long tail towards <\/span>right. The shape of F-distribution depends only on degrees of freedom (df) of <\/span>numerator (DFn) and df of <\/span>denominator (DFd).<\/span><\/p>\n<\/div>\n

              \n

              \"\"<\/p>\n

               <\/p>\n

              As one can see, when both of the df is small, <\/span>shape is distinctly skewed to the right. With high df, distribution becomes bell-shaped, although it would still be skewed to the right.<\/span><\/p>\n

               <\/p>\n

              At each combination of <\/span>these two df, F-distribution can be calculated using <\/span>intricacies of calculus. At each of these distributions at each combination of DFn and DFd, we can calculate the area under the significance level (the threshold P value, alpha) at the tails. These values can be presented in a tabular format, the so called F-distribution tables, where one can lookup a value called F-critical. One can compare this F-critical value found from the table and the F-ratio calculated from the data to make inferences about statistical significance. If F-critical at <\/span>significance level of 0.05 is less than F-ratio, we can infer that P value must be less than 0.05 and can reject the null hypothesis.<\/span><\/p>\n

               <\/p>\n

              4. Tests of significance based on F distribution<\/strong><\/p>\n

               <\/p>\n

              F distribution is used to test whether two groups have equal variances, one of the assumptions of <\/span>unpaired t-test, as we have seen in module 18. The most important use of F-distribution is for F-test, also known as ANOVA (Analysis of Variance), for comparing means of three or more groups, as discussed in this module. Keep in mind that ANOVA merely compares group means and informs us whether all groups have identical means or not. Means would not be identical if <\/span>mean of one group is different from the rest. To know which group means are significantly different, we should do multiple comparisons posttest, as explained later in this module.<\/span><\/p>\n

               <\/p>\n

              5.<\/strong>\u00a0<\/span>Problem with multiple t-tests<\/strong><\/p>\n

               <\/p>\n

              Consider three groups of patients (X, Y <\/span>and Z), each group treated with <\/span>different drug. An intuitive approach to compare group means would be performing multiple t-tests. For example, t-test to compare means of X and Y, another one between Y-Z, yet another one between X and Z. In any of these t-tests if P<0.05 was obtained, we can conclude that group means as not identical. If all of these t-tests returns P>0.05, we can conclude that group means are identical. Though sound appealing, multiple <\/span>comparison like these should not be performed as it drastically increases random <\/span>error .In one t-test at 0.05 significance level, we still expect to find 5% of \u201cfalse positives\u201d (a blood test tells you are HIV+ but in reality you are HIV-); i.e, t-test yielding a significant P value even if two group means are identical due to chance alone. With two or more t-tests, <\/span>proportion of expected false positives are not merely 5% but well above this. Therefore multiple comparisons, like multiple t-tests, should never be used.<\/span><\/p>\n

               <\/p>\n

              6.<\/strong>\u00a0<\/span>One-way ANOVA<\/strong><\/p>\n

               <\/p>\n

              One-way ANOVA, also called one-factor ANOVA compares means of three or more groups to find differences between means of these groups are statistically significant or not. ANOVA can also be used to compare means of two groups, but in this <\/span>case ANOVA is indistinguishable from unpaired t-test explained previously and returns identical P values from <\/span>t-test. ANOVA was developed by population geneticist and founder of statistics, Ronald Fisher.<\/span><\/p>\n

               <\/p>\n

              7.<\/strong>\u00a0<\/span>Assumptions for ANOVA<\/strong><\/p>\n

               <\/p>\n

              Like t-test, One-way ANOVA is based on a set of familiar assumptions:<\/span><\/p>\n

              1.\u00a0\u00a0\u00a0\u00a0\u00a0 Random samples<\/span><\/p>\n

              2.\u00a0\u00a0\u00a0\u00a0\u00a0 Independent measurements<\/span><\/p>\n

              3.\u00a0\u00a0\u00a0\u00a0\u00a0 Accurate Data<\/span><\/p>\n

              4.\u00a0\u00a0\u00a0\u00a0\u00a0 Data are sampled from populations that are approximately Gaussian<\/span><\/p>\n

              5.\u00a0\u00a0\u00a0\u00a0\u00a0 Variances, or Standard deviations, of populations from which the samples came <\/span>from are\u00a0<\/span>identical<\/span><\/p>\n

               <\/p>\n

              To test <\/span>4th assumption, a formal statistical test like D\u2019Agostino Pearson Omnibus K2 test can be performed. In case the populations significantly deviate from Gaussian, <\/span>first option should be trying to see any normalizations transforms the distributions to normal (normalization). For example, a number of biological distributions like enzyme kinetics and wherever the term \u2018half-maximal\u2019 is used are lognormal. It can be normalized by converting each <\/span>values to <\/span>logarithm. If none of the normalizations works, then a non-parametric test like Kruskal-Wallis test, followed by Dunn\u2019s posttest should be preferred. For paired (matched) data that came from non-Gaussian, Friedman\u2019s test followed by Dunn\u2019s posttest should be preferred. A statistical package like Graphpad Prism can be used for both of these non-parametric tests. <\/span>5th assumption of equal variance is not usually tested prior to ANOVA, as testing for this assumption is built-in ANOVA itself.<\/span><\/p>\n

               <\/p>\n

              8.\u00a0\u00a0 One-way ANOVA by hand<\/strong><\/p>\n

               <\/p>\n

              Let us consider an example, Increase in mouse colorectal tumor by four different heavy metals after feeding for a period of time.<\/span><\/p>\n<\/div>\n

              \n

              \"\"<\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

              Let us first define our null hypothesis and alternative hypotheses:<\/p>\n

              H0: All group means are equal<\/p>\n

              Ha: At least one group mean is different from the rest<\/p>\n

               <\/p>\n

              As we can see, there are four groups (arranged by columns: U, Pb, As and Hg) and four technical replicate measurements (also called levels) for each group, arranged by rows (only three for As though). The numbers (elements, measurements) means <\/span>increase in <\/span>tumour, so in <\/span>effect there is only one factor (increase in <\/span>tumour mass) across four groups. That is why the ANOVA is called one-way or one factor ANOVA. A factor is an independent treatment variable whose settings (values) are controlled and varied by the experimenter. The intensity setting of a factor is the level. Levels may be quantitative numbers or, in many cases, simply “present” or “not present” (“0” or “1”). Had there been another factor like <\/span>age of mouse, then we would need to do two-factor ANOVA. Use one-way for comparing <\/span>1 factor means of >2 groups. For eg., average heights of neem trees at three villages; average ground water As levels at 4 districts in Gujarat.<\/span><\/p>\n

               <\/p>\n

              For one-way ANOVA, the first step is to calculate mean, standard deviation and variance for each of these groups, as already computed in the table above. We can also compute overall (total) mean, Standard Deviation and variance of our entire Dataset (n=15). ANOVA starts with <\/span>calculation of \u201cSum of Squares,\u201d SS. SS is defined as the square of deviations of values from <\/span>sample mean (\u2211(x-x\u0304)2). Remember that we have used this while calculating standard deviation from raw data.<\/span><\/p>\n

               <\/p>\n

              First let us calculate <\/span>SS<\/strong>total<\/strong>, Sum of Squares of difference between each of these 15 values and the overall mean (x\u0304, 77.79).<\/span><\/p>\n

               <\/p>\n

              (60.8-77.79)2+ (67-77.79)2+\u2026\u2026\u2026..(90.3-77.79)2<\/span><\/p>\n<\/div>\n

              \n\n\n\n\n\n\n\n\n\n\n\n\n\n
              <\/td>\n<\/td>\nx-x\u0304<\/td>\n(x-x\u0304)2<\/td>\n<\/tr>\n
              1<\/td>\n60.8<\/td>\n-16.99<\/td>\n288.6601<\/td>\n<\/tr>\n
              2<\/td>\n67<\/td>\n-10.79<\/td>\n116.4241<\/td>\n<\/tr>\n
              3<\/td>\n54.6<\/td>\n-23.19<\/td>\n537.7761<\/td>\n<\/tr>\n
              4<\/td>\n61.7<\/td>\n-16.09<\/td>\n258.8881<\/td>\n<\/tr>\n
              5<\/td>\n78.7<\/td>\n0.91<\/td>\n0.8281<\/td>\n<\/tr>\n
              6<\/td>\n77.7<\/td>\n-0.09<\/td>\n0.0081<\/td>\n<\/tr>\n
              7<\/td>\n76.3<\/td>\n-1.49<\/td>\n2.2201<\/td>\n<\/tr>\n
              8<\/td>\n79.8<\/td>\n2.01<\/td>\n4.0401<\/td>\n<\/tr>\n
              9<\/td>\n92.6<\/td>\n14.81<\/td>\n219.3361<\/td>\n<\/tr>\n
              10<\/td>\n84.1<\/td>\n6.31<\/td>\n39.8161<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n
              \n\n\n\n\n\n\n\n\n\n
              11<\/td>\n90.5<\/td>\n12.71<\/td>\n161.5441<\/td>\n<\/tr>\n
              12<\/td>\n86.9<\/td>\n9.11<\/td>\n82.9921<\/td>\n<\/tr>\n
              13<\/td>\n82.2<\/td>\n4.41<\/td>\n19.4481<\/td>\n<\/tr>\n
              14<\/td>\n83.7<\/td>\n5.91<\/td>\n34.9281<\/td>\n<\/tr>\n
              15<\/td>\n90.3<\/td>\n12.51<\/td>\n156.5001<\/td>\n<\/tr>\n
              <\/td>\n<\/td>\nTotal<\/strong><\/td>\n1923.41<\/strong><\/td>\n<\/tr>\n
              <\/td>\n<\/td>\n(SS<\/strong>total<\/strong>)<\/strong><\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

               <\/p>\n

              Let us then consider SS<\/strong>between<\/strong>, \u201cSum of Squares\u201d between group means. We have 4 groups, so as four group means. To compute SS<\/strong>between<\/strong>, we have to subtract each group mean (x) with overall mean (x\u0304, 77.79), square this number, and multiplied with size of each of these groups (to weigh by sample size). Sum of all these values in our 5th column is SS<\/strong>between<\/strong>.<\/p>\n

              \"\"<\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

              We need one more Sum of Squares, i.e., within groups SS<\/strong>within<\/strong>.<\/p>\n

               <\/p>\n

              SS<\/strong>total<\/strong> = SS<\/strong>between<\/strong> and SS<\/strong>within<\/strong> (derivation of which is omitted for the sake of simplicity)<\/p>\n

              \u2234\u00a0\u00a0\u00a0\u00a0 SS<\/strong>within<\/strong>= SS<\/strong>total<\/strong> – SS<\/strong>between<\/strong><\/p>\n

              =1923.41- 1761.128<\/p>\n

              =162.282<\/p>\n

              This SS<\/strong>within<\/strong> is also called \u2018residual sum of squares\u2019 or \u2018error sum of squares\u2019<\/p>\n

               <\/p>\n

              Next step is to calculate degree of freedom for each of these three Sum of Squares. For SS<\/strong>between<\/strong>, we have four groups (K, total number of groups); as df=(n-1), df is 4-1= 3<\/p>\n

               <\/p>\n

              Degree of freedom for SS<\/strong>within<\/strong>, is overall sample size (N) minus number of groups (K). Here overall sample size is 15 and number of groups is 4. Therefore, 15-4= 11<\/p>\n

               <\/p>\n

              Degree of freedom for <\/span>SS<\/strong>total<\/strong> is N-1, 15-1= 11 (also, df for between groups + df for within groups = df for total)<\/span><\/p>\n

               <\/p>\n

              Let us put all these numbers in the <\/span>summary table<\/strong> for one-way ANOVA:<\/span><\/p>\n<\/div>\n

              \n

              \"\"<\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

              The fourth column is Mean Square (MS, also called variance), which is Sum of Squares divided by df for each of these sources. Sum of MS to calculate total MS makes no sense, so this is not done. Please note that MSwithin is perhaps the most important statistic in an ANOVA test; it tells us about the variance due to error and is important for calculating \u2018margin of error\u2019 required for standardized error computations in posttests (like Tukey\u2019s) to know which means are significantly different, as explained later in this module. Finally, the fifth column is F-ratio, which is the ratio of these two MS values (MSbetween\/MSwithin).<\/p>\n

               <\/p>\n

              As in the case of F-test to detect the equality of variance (assumption of homoscedasticity) we did prior to unpaired t-test, here too we should look up F table for critical F value. Df for F Ratio is expressed with DFn =3 and DFd=11. Critical F value corresponding to these two df at 0.05 significance level is:<\/p>\n

               <\/p>\n

              \"\"<\/p>\n<\/div>\n

               <\/p>\n

              As table value (critical F, 3.587) is far less than our obtained F ratio (39.7917), we can conclude that P<0.05, null hypothesis of equal means are rejected and concluded that at least one of the group mean is significantly different from the rest. There are many F tables corresponding to different significance levels (these are accessible at <\/span>http:\/\/www.socr.ucla.edu\/applets.dir\/f_table.html<\/a>). Perhaps we are interested whether this F ratio is still higher than the critical F at lower significance levels, let us say 0.01 or 0.001. We can see that even at <\/span>significance level of 0.001 level, F critical from <\/span>table (11.56) is still less than our obtained F ratio, so our actual P value must be very less. One-way ANOVA for the above data done using excel returns a P value of 3.36E-06 (read this value as 3.36 x 10-6 i.e, 0.00000336). We can also use an online calculator like the one at <\/span>http:\/\/stattrek.com\/online-calculator\/f-<\/a>distribution.aspx <\/a>(keep in mind that this calculator returns a cumulative probability; we have to subtract this value from 1 to get empirical probability). A low P value <\/span>indicate that the group means are significantly different. It could be because one of the four means is different from the rest three, or two of the group means different from the rest two. It could also be due to all four means different from each other. None of these finer details are revealed by one-way ANOVA P value. To find those out we have to perform an appropriate multiple comparison posttest as explained below.<\/span><\/p>\n

               <\/p>\n

              9.<\/strong>\u00a0<\/span>Other types of ANOVA<\/strong><\/p>\n

               <\/p>\n

              a)\u00a0<\/strong>Repeated Measures ANOVA: <\/em><\/strong>As a paired t-test is used to compare two groups with paired\u00a0<\/span>(matched or dependant) measurements, we use \u2018Repeated Measures ANOVA for matched measurements for more than 2 groups (these are called matched sets or matched blocks). Whenever a value in any particular group is expected to be closer to a specific value in another group than to a random value from our whole experiment (because of the way we designed the experiment), we should use repeated measures ANOVA. To find those out which groups are significantly different, we have to perform an appropriate multiple comparison posttest as explained below, similar to normal one-way ANOVA.<\/span><\/p>\n

               <\/p>\n

              b)\u00a0<\/strong>Two-way ANOVA and more: <\/em><\/strong>Two-way (or two-factor) ANOVA is used for comparison involving two factors. As already explained, in addition to heavy metals, we might also be interested to know the effects of their age on <\/span>tumour mass (so variable remains only one). Use two-way ANOVA for comparing <\/span>2 factor means of >2 groups. Examples are average heights of neem trees grown using 4 different fertilizers at three villages<\/span>; and average As levels at shallow\/moderate\/deep wells in 4 districts. If we add yet another factor, like sex, we will have to perform 3 <\/span>factor ANOVA and so on. In the case of One-way ANOVA, <\/span>null hypothesis is that means of groups are <\/span>same. In case of Two-way ANOVA, null hypothesis takes up three possibilities: 1) There is no difference in the means of factor A across the groups, 2) There is no difference in means of factor B across the groups and 3) There is no interaction between factors A and B<\/span><\/p>\n

               <\/p>\n

              c)\u00a0<\/strong>Multivariate ANOVA (MANOVA): <\/em><\/strong>Manova is used for comparing the effect of <\/span>single categorical variable (exposure to tobacco) on the averages of two or more continuous variables (for eg: heart rate and blood pressure)<\/span><\/p>\n

               <\/p>\n

              d)\u00a0<\/strong>Random Effects ANOVA: <\/em><\/strong>Normal ANOVA is technically called fixed-effects or Type-I ANOVA; in this <\/span>case the test assess means of our selected groups only without any further extrapolations. In the case of Random Effects ANOVA, also called Type-II ANOVA, the test assumes that the groups that we selected are randomly selected representatives (our sample groups) of an infinite number of groups (our unknown population) and the test compares our sample group means to infer whether our infinite population group means are significantly different or not.<\/span><\/p>\n

               <\/p>\n

              e)\u00a0<\/strong>Analysis of Molecular Variance (AMOVA): <\/em><\/strong>This is a method to detect population differentiation (a large contiguous population fragmenting to small isolated populations using molecular markers. While AMOVA had been developed in lines of ANOVA, two methods are quite different.<\/span><\/p>\n

               <\/p>\n

              10. Multiple Comparisons Posttests<\/strong><\/p>\n

               <\/p>\n

              One-way ANOVA informs us whether the overall differences in the means of our groups are significant or not. As explained already, a low P value in one-way ANOVA could be due to the fact that only one of the mean is significantly deferring from the rest, or groups of means differing from other groups of <\/span>mean (the so called \u2018contrast\u2019), or every <\/span>mean differing from every other means significantly. In most of the situations, ANOVA P value is not what we want to know; we would be more interested to know which means are significantly different. To find that out, we should perform a multiple comparisons posttest (also called Post Hoc test).<\/span><\/p>\n

               <\/p>\n

              Multiple <\/span>t-test is one way to compare pairs of means after ANOVA; however, as already explained this should not be used as expected false positives (Type I error) would be far higher than 5% level. This is because <\/span>significance level (for example, 0.05) is for each comparison involved, not for the entire family of comparisons. Between four groups, 6 comparisons are possible (no. of possible comparisons can be calculated by combination formula, nCr= n! \/(n-r)!.r!), so after performing these comparisons, our significance level would be far higher than 0.05 ( that would mean a number of false positives in our results). In case multiple <\/span>t-test is used, each individual P value should be multiplied with <\/span>total number of such comparisons. For example, an unadjusted P value of 0.01 is obtained for one pair of comparison in a group of 4. As <\/span>total number of possible comparisons in a group of four is 6, the unadjusted P value <\/span>need to be multiplied with 6 to get the corrected P value. 0.01*6 = 0.06. This correction is known as <\/span>Bonferroni adjustment<\/strong>. Bonferroni adjustment is fine in case you want to compare a specific pair after the ANOVA provided that this pair is clearly specified as part of the experimental design. As already mentioned, after seeing the data and choosing the highest and lowest means, you would be effectively comparing not merely <\/span>this two means but all six means in a group of four.<\/span><\/p>\n

               <\/p>\n

              In case we would like to compare all combinations of means, which is by far most of us would like to do after ANOVA, the best option is Tukey\u2019s Honestly Signified Differences (Tukey\u2019s HSD). Bonferroni\u00a0<\/span>test should not be used for this purpose, as the statistical power for which is far less than Tukey\u2019s HSD. As already explained, Tukey\u2019s HSD depends on a key output of ANOVA, MSwithin. As in Confidence Interval of mean explained in module 16, or CI of differences between means explained in module 18, Tukey\u2019s test returns CIs for all pairwise combinations of the differences between group means. For four groups, there would be six pairwise comparisons (<\/span>difference between means) and these would be expressed in Confidence Intervals at our selected Confidence Level (for example 95%).<\/span><\/p>\n

               <\/p>\n

              The formula for Tukey\u2019s HSD is:<\/span><\/p>\n

              \n

              \"\"<\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

              Where x\u0304i and x\u0304j are means of groups i and j. q is a critical value from q distribution, MSw is MSwithin from ANOVA output, and ni and nj are sizes of groups I and j respectively. The above equation is very similar to other equations of Confidence Intervals; difference between sample means \u00b1 w (the width of CI). The width of CI is calculated from standardized (instead of \u2018standard\u2019) error (the square-root term in above equation) multiplied by a critical value from q distribution (instead of t*). The critical value q depends upon three things; the chosen level of significance (\u03b1), total number of groups (r) and degree of freedom for MSwithin (dfw). q Distribution is studentized range for \u03b1. Also note that though we are only comparing means of two of our groups (i and j), all groups and every single value in our dataset is important, as the value MSwithin depends upon the whole dataset. This is needed to account for the perils of multiple comparisons as already explained.<\/p>\n

               <\/p>\n

              Let us go back to our original example. To compare means of Pb vs U (Lead vs Uranium, our first two groups), we need to first calculate the difference between their respective group means. Mean for Pb is 78.13 and mean for U is 61.03, so difference is 17.1.<\/p>\n

               <\/p>\n

              Let us first define our null hypothesis and alternative hypotheses:<\/p>\n

              H0: Difference between two means is zero<\/p>\n

              Ha: Difference between two means is not zero<\/p>\n

               <\/p>\n

              Next step is to calculate the width of 95% CI (so \u03b1=0.05). To calculate it we need the following information: MSwithin, dfw, nPb, nU, \u03b1, r and q. MSwithin<\/sub>=14.753, dfw<\/sub>=11, nPb=4, nU=4. \u03b1=0.05, r=4 and dfw=11. Plugging the values in:<\/span><\/p>\n

               <\/p>\n

              w=[\u221a(14.753\/2)x (1\/4+1\/4)] x q (\u03b1=0.05, r=4 and df<\/span>w<\/sub>=11)<\/span><\/p>\n

               <\/p>\n

              q- <\/span>table is not normally provided in <\/span>statistics tests of tables, as it is hard to calculate. One version of <\/span>online calculator can be found at <\/span>http:\/\/www.vassarstats.net\/tabs.html#q<\/a><\/p>\n<\/div>\n

              \n

              \"\"<\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

              As seen, Q at \u03b1=0.05 is 4.27. Plugging this in the above equation:<\/p>\n

              =[\u221a(14.753\/2)x (1\/4+1\/4)] x 4.27<\/p>\n

              =\u221a (7.365x 0.5) x 4.27<\/p>\n

              =1.919 x 4.27<\/p>\n

              =8.2<\/p>\n

               <\/p>\n

              The above value is the width, it extends both sides of the difference between means (17.1). So, 95% Confidence Interval for this difference is:<\/p>\n

              (17.1 \u2013 8.2) to (17.1 + 8.2)<\/p>\n

              8.9 to 25.3<\/p>\n

               <\/p>\n

              Does this range include our null hypothesis of no difference in means, i.e., zero? No, it doesn\u2019t. Therefore, P<0.05 and the difference is statistically significant.<\/p>\n

               <\/p>\n

              We can perform the same pairwise computation of 95% CI for all six combinations of group means as well to find how many of these combinations returns statistically significant differences. Confidence Interval of difference between means tells us whether P value is significant at our chosen significance level. Some softwares (like GraphPad prism) also returns an exact P value, called \u2018multiplicity-adjusted P value\u2019 (that had been adjusted for multiple comparisons).<\/p>\n

               <\/p>\n

              It is possible that the one-way ANOVA returns a P<0.05 yet none of the pairs of mean differences are significant. It is also possible that ANOVA returns P>0.05, yet some of the difference between group means are significant.<\/span><\/p>\n

               <\/p>\n

              Other post hoc multiple comparison methods other than Bonferroni\u2019s test and Tukey\u2019s HSD) include Dunnett\u2019s test which is highly useful to compare multiple groups with a group defined as a control group (each test group will be compared with <\/span>control group, but <\/span>there wont be any comparison between test groups), Scheffe\u2019s test to compare \u2018contrasting\u2019 sets of group means (for example, groups {A, B and C} versus groups {D and E}) and Holm\u2019s test.<\/span><\/p>\n

               <\/p>\n

              An online calculator for ANOVA with Tukey\u2019s HSD is available at <\/span>http:\/\/astatsa.com\/OneWay_Anova_with_TukeyHSD\/<\/a><\/p>\n

               <\/p>\n

              The results of Tukey\u2019s HSD for our example are as follows:<\/span><\/p>\n<\/div>\n

              \n

              \"\"<\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

               <\/p>\n

              This calculator also include results of other multiple comparison tests, including Bonferroni\u2019s test, Scheff\u00e9 test, and Holm\u2019s test.<\/p>\n

               <\/p>\n

              Also note that even though ANOVA returns and overall P value >0.05 (and differences not statistically significant), results of multiple comparisons like Tukey\u2019s HSD, Bonferroni test and Dunn\u2019s test are still valid.<\/p>\n

               <\/p>\n

              11. Summary<\/strong><\/p>\n<\/div>\n

                \n
              1. For One way ANOVA, we first have to calculate SStotal, SSbetween and SSwithin. With degrees of freedom for all of these Sum of Squares, we can calculate Mean Square by dividing Sum of Squares with df. Finally we can calculate F-ratio by MSbetween\/MSwithin. From two df, number of groups and F-ratio, finally we can calculate P value. One way ANOVA can easily be calculated using a software or many web-based calculators.<\/li>\n
              2. For matched or paired data, the test analogous to matched t-test for groups more than 2 is repeated measures ANOVA<\/li>\n
              3. Two-way ANOVA determines how a response is affected by two variables, and any association between the variables.<\/li>\n
              4. A common mistake is analysis of ANOVA groups by multiple t-tests, but this is highly problematic as <\/span>chance of false positives greatly increases this way.<\/span><\/li>\n
              5. A popular and powerful multiple comparison post hoc test is Tukey\u2019s HSD that calculate\u00a0<\/span>95% CI of differences between means of pairs of groups. In most cases, results of Post hoc tests like Tukey\u2019s is a lot more informative than ANOVA itself, but Tukey\u2019s test is dependent on results of ANOVA.<\/li>\n
              6. Even though ANOVA returns and overall P value >0.05 (and differences not statistically significant), results of multiple comparisons like Tukey\u2019s HSD, Bonferroni test and Dunn\u2019s test are still valid.<\/li>\n<\/ol>\n

                \u00a0 \u00a0 Quadrant-III: Learn More\/ Web Resources \/ Supporting Materials:<\/strong><\/p>\n

                  \n
                1. F tables arranged by significance levels http:\/\/www.socr.ucla.edu\/applets.dir\/f_table.html<\/a><\/li>\n<\/ol>\n
                    \n
                  1. One-way ANOVA online calculator http:\/\/stattrek.com\/online-calculator\/f-distribution.aspx<\/a><\/li>\n<\/ol>\n
                      \n
                    1. Online calculator to find q-value from df and k (number of groups) http:\/\/www.vassarstats.net\/tabs.html#q<\/a><\/li>\n<\/ol>\n
                        \n
                      1. online calculator for ANOVA with Tukey\u2019s HSD is available at http:\/\/astatsa.com\/OneWay_Anova_with_TukeyHSD\/<\/a><\/li>\n<\/ol>\n
                          \n
                        1. Online calculator to calculate exact P from F ratio https:\/\/www.graphpad.com\/quickcalcs\/pValue1\/<\/a><\/li>\n<\/ol>\n","protected":false},"author":3,"menu_order":19,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":["dr-felix-bast"],"pb_section_license":""},"chapter-type":[],"contributor":[59],"license":[],"class_list":["post-360","chapter","type-chapter","status-publish","hentry","contributor-dr-felix-bast"],"part":3,"_links":{"self":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/wp-json\/pressbooks\/v2\/chapters\/360","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/wp-json\/wp\/v2\/users\/3"}],"version-history":[{"count":4,"href":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/wp-json\/pressbooks\/v2\/chapters\/360\/revisions"}],"predecessor-version":[{"id":372,"href":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/wp-json\/pressbooks\/v2\/chapters\/360\/revisions\/372"}],"part":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/wp-json\/pressbooks\/v2\/parts\/3"}],"metadata":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/wp-json\/pressbooks\/v2\/chapters\/360\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/wp-json\/wp\/v2\/media?parent=360"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/wp-json\/pressbooks\/v2\/chapter-type?post=360"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/wp-json\/wp\/v2\/contributor?post=360"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/ebooks.inflibnet.ac.in\/esp14\/wp-json\/wp\/v2\/license?post=360"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}