Are You Performing

The Correct ANOVA?

So far, we have primarily discussed designs with fixed factors only, which are the simplest form of experimental designs. The Analysis of Variance (ANOVA) for the models associated with fixed factors is known as Analysis of Variance Type I or ANOVA Type I or ANOVA Model I. An analysis of variance associated with all random factors is known as the ANOVA Model II or ANOVA Type II. An analysis of variance associated with multiple factor types, including random, fixed, nested, is known as a mixed model and the ANOVA is known as ANOVA Type III or ANOVA Model III.

In the ANOVA model I, the test statistics were developed by simply dividing by the experimental error as in Equation 1.

Equation 1

Note. Mean Squares follow a chi-square distribution. The ratio of two chi-square distributions follows an f-distribution. Therefore, the test statistics can be precisely called F-Statistics, f-tests or f-value.

Simply, the test statistics can be interpreted as “how strong the treatment is as compared to the random variations in the experimental units.” The stronger/larger the treatment effect, MSTretment is as compared to the natural variations in the experimental units, MSE; the stronger the possibility of observing a significant result. In the ANOVA model I, there is only one random error and that is the natural variations in the experimental units. However, in both the ANOVA Model II and III, there are more than just the random variations in the experimental units as random factors are included in the study. Therefore, the appropriate F-Statistics, especially for ANOVA model II & III, is not as straightforward to develop as compared to the ANOVA Model I. In some cases, an appropriate exact F-Statistics does not even exist. Therefore, many software packages, such as MINITAB, SAS, SPSS, Design Expert, and JMP will not automatically produce the F-Statistics for a complex design of experiment. Even if statistical software packages produce results, understanding and interpretation of the results in the context of the problems require the understanding of the correct/appropriate random error associated with the treatment/effect. In this module, we will show a very simple step by step technique to develop the correct F-Statistics flawlessly (Hicks, 1982; Kutner et al., 2005; Underwood, 1996). First, we will start with the fixed effect model, ANOVA Model I as we are familiar with it. Second, we will discuss the pure random effect model, ANVOA Model II. Finally, we will discuss the mixed model (ANOVA model III), including fixed, random and nested factors.

Understanding the process of developing the correct F-Statistics will help design any complex experiments, including, split-plot, split-split-plot, sampling, sub-sampling, repeated measure, nested, nested-nested, and hierarchical design of experiments. The design and analyses of experiments are guided by the factor types (e.g., fixed, random, and nested), rather than the name of the experiments. The experimental design names are more associated with the field of study, such as “repeated measure” in the human/animal subject study, while a similar design is known as “split-plot” in the agriculture field. Factor types determine the appropriate ANOVA Model (e.g., Model III) and the correct F-Statistics.

Equation 2

To determine the correct Analysis of Variance (ANOVA), factors are classified as either (1) Fixed, (2) Random, or (3) Nested. The ANOVA models are classified as either (1) Model I with fixed factors only, (2) Model II with random factors only, or (3) Model III with mixed or combinations of factors. Again, the names for ANOVA Models are irrelevant in determining the correct F-Statistics (Equation 2). The mean square for treatment (MStreatment Effect) is the same for any design or ANOVA model. However, the MSRandom Error Associated with the Effect is NOT simply the experimental error like in a fixed effect model, which is the ANOVA model I. In this module, we will discuss how to determine the correct MSRandom Error Associated with the Effect (the divisor for correct F-Statistics) for any design of experiments, especially the complex random (ANOVA Model II) and mixed effect (ANOVA Model III) models.