Video 1. One-Way Single-Factor Completely Randomized Design Analysis of Variance ANOVA Using MS Excel

Fuel type study data is provided in Table 1.

Table 1. Fuel Economy in Miles Per Gallon Using Three Different Fuel

This data set is different from the data set used in the video.

Any dedicated statistical software would be time saving in performing the post-hoc pairwise comparison analyses. Video 2 shows the analysis and explanation, including the post-hoc pairwise comparisons.

Video 2. One Way Single Factor Analysis of Variance ANOVA Completely Randomized Design Analysis in Minitab.

Four Steps

Fixed Effect Model Design of Experiments

In any design of experiment or any research study, regardless of the discipline, the following four steps are generally followed.

Step #1. Research question (hypothesis)

Step #2. Appropriate Method

As we are interested in the mean difference in the fuel economy from three different fuels, the means model or the population means model in Equation 2 is utilized. As the levels of the fuel type factor are fixed, the model is also called the fixed effects model. Analysis can be performed either using MS Excel or Minitab shown in Video 1 or Video 2, respectively.

Step #3. Analysis Results with Statistical Explanations

The analysis output is provided in Figure 1.

Figure 1. Analysis of Variance ANOVA results from One-Way Fixed Effect Model Analysis

Statistical Explanation of the Result

We reject the null hypothesis as the p-value (0.000) is smaller than the level of significance alpha (=0.05).

[p-value is defined by the observed probably for the null hypothesis to happen. As the probability (p-value < 0.001) is lower than our rejection criteria alpha (=.05), we reject the null hypothesis. As there are two choices between accepting the null or the alternative, we accept the alternative in this case because the null is rejected]

Step #4. Contextual Conclusion: Results Explanation in the Context of the Problem

Statistically, at least one fuel type is different from another fuel type with respect to the fuel economy in miles per gallon. [Simply, rewrite the accepted hypothesis in the context of the problem. The alternative hypothesis is accepted in this case.]

Post-Hoc Analysis

From the previous four-steps analysis of variance, we only know that there is a difference in fuel type with respect to the fuel economy in miles per gallon. Therefore, the next question would be which one is the best or worst so that the best ingredient can be used to make the fuel that provides the best fuel economy. When results are observed to be significant from the analysis of variance, additional analysis called the post-hoc analysis is performed to determine the best or worst level of the factor with respect to the mean responses. The name post-hoc analysis is used as this analysis is performed after the analysis of variance (ANOVA) results are observed to be significant.

Pairwise Comparisons Tests

As the alternative hypothesis is accepted when the results are observed to be significant, the post-hoc is focused on the alternative hypothesis. The alternative hypothesis contains three pairs of means in this fuel type study. Any of the two fuel types could be different with respect to the mean fuel economy. Therefore, the pairwise comparisons tests are performed as the post-hoc analyses to determine the best or worst level for the factor with respect to the response. The name pairwise comparisons test is used as the process compares the pairs one by one to find the best or worst level of the factor with respect to the response.

There are few different types of pairwise comparisons tests, including the Fisher LSD, Tukey, Dunnett, Bonferroni, etc. While the Fisher LSD is the most basic, the Tukey is a bit more conservative, meaning that it may not show the significance as easy as the Fisher LSD. Nevertheless, the difference between these various pairwise comparisons tests are more theoretical interests. In practice, simply the Fisher LSD can be performed to determine the best or worst level of the factor. The post-hoc pairwise comparison tests are sometimes field-specific too. For example, the research study in human factors and ergonomics uses Tukey more than any other pairwise comparison tests. The reader can see a couple of journal articles to know the most commonly used (or accepted) pairwise comparison tests for their field of study.

Most statistical software such as Minitab will produce these pairwise comparisons results. Video 3 shows the analysis using Minitab. While the MS Excel works fine in analyzing the one-way ANOVA, performing the pairwise comparisons are time consuming in MS Excel. However, readers can check Video 3 to understand the procedure for the pairwise comparisons using MS Excel.

Video 3. One Way Single Factor Analysis of Variance ANOVA Post Hoc Pairwise Comparison Analysis in MS Excel

Explanation of the Post-hoc Pairwise Comparison Tests

The pairwise comparisons test using both the Fisher LSD and the Tukey are provided in Figure 2 and Figure 3. Both Fisher LSD and Tukey pairwise comparison analyses show that Fuel Type 1 is the best with respect to the mean fuel economy in miles per gallon. Fuel 1 provides 32.37 miles per gallon, while the worst type 3 provide only 22.83 miles per gallon.

Figure 2. Fisher LSD Method Pairwise Comparisons Results