This is one of the following sixteen articles on Single-Factor ANOVA in Excel

Overview of Single-Factor ANOVA

Single-Factor ANOVA in 5 Steps in Excel 2010 and Excel 2013

Shapiro-Wilk Normality Test in Excel For Each Single-Factor ANOVA Sample Group

Kruskal-Wallis Test Alternative For Single Factor ANOVA in 7 Steps in Excel 2010 and Excel 2013

Levene’s and Brown-Forsythe Tests in Excel For Single-Factor ANOVA Sample Group Variance Comparison

Single-Factor ANOVA - All Excel Calculations

Overview of Post-Hoc Testing For Single-Factor ANOVA

Tukey-Kramer Post-Hoc Test in Excel For Single-Factor ANOVA

Games-Howell Post-Hoc Test in Excel For Single-Factor ANOVA

Overview of Effect Size For Single-Factor ANOVA

ANOVA Effect Size Calculation Eta Squared in Excel 2010 and Excel 2013

ANOVA Effect Size Calculation Psi – RMSSE – in Excel 2010 and Excel 2013

ANOVA Effect Size Calculation Omega Squared in Excel 2010 and Excel 2013

Power of Single-Factor ANOVA Test Using Free Utility G*Power

# Overview Post-Hoc

Testing For Single-Factor

ANOVA

The F-test in ANOVA is classified as an omnibus test. An omnibus test is one that-Tests the overall significance of a model to determine whether a difference exists but not exactly where the difference is. ANOVA test the Null Hypothesis that all of the group means are the same. When this Null Hypothesis is rejected, further testing must be performed to determine which pairs of means are significantly different. That type of testing is called Post-Hoc testing.

Post-Hoc testing is a pairwise comparison. Groups means are compared two at a time to determine whether the difference between the pair of means is significant.

## The Many Types of Post-Hoc Tests

Available

There are many types of Post-Hoc tests available in most major statistical software packages but two have become the preferred tests. These two are the Tukey-Kramer test and the Games-Howell test. The Tukey-Kramer test should be used when group variances are similar. The Games-Howell test should be used with group variances are dissimilar. Both tests can be used when group sizes are unequal.

The Tukey-Kramer test is a slight variation of the well-known Tukey HSD test. The Tukey-Kramer can be used when group sizes are unequal, which the Tukey HSD test is not designed for.

### Post-Hoc Tests Used When Group Variances Are Equal

SPSS lists the following Post-Hoc tests or corrections available when groups variances are equal:

LSD

Bonferroni

Sidak

Scheffe

REGWF

REGWQ

S-N-K

**Tukey (Tukey’s HSD or Tukey-Kramer)**

Tukey’s b

Duncan

Hochberg’s GT2

Gabriel

Waller-Duncan

Dunnett

Of all of the Post-Hoc tests available when groups variances are found to be similar, Tukey’s HSD test is used much more often than the others. A slight variation of Tukey’s HSD called the Tukey-Kramer test is normally used when group variances are the same but group sample sizes are different. Tukey’s HSD can only be used when group sizes are exactly the same.

The Tukey test (Tukey’s HSD test or the Tukey-Kramer test) is generally a good choice when group variances are similar. Hochberg’s GT2 produces the best result when group sizes are very different. REGWQ is slightly more accurate than the Tukey test but should only be used when group sizes are the same.

### Post-Hoc Tests Used When Group Variances Are Not Equal

SPSS lists the following Post-Hoc tests available when groups variances are not equal:

Tamhane’s T2

Dunnett’s T3

**Games-Howell**

Dunnett’s C

The Games-Howell test is the most widely used of the above and is generally a good choice when group variances are not similar. The Games-Howell test can be used when group sizes are not the same.

### Tukey’s HSD (Honestly Significant Difference) Test

**Used When Group Sizes and Group Variances Are Equal**

Tukey’s HSD test compares the difference between each pair of group means to determine which differences are large enough to be considered significant.

Tukey’s HSD test is very similar to a t-test except that it makes a correction for the experiment-wide error rate that a t-test doesn’t. The experiment-wide error rate is the increased probability of type 1 errors (false positives) when multiple comparisons are made.

Tukey’s HSD test can be summarized as follows:

The means of all groups are arranged into as many unique pair combinations as possible. The pair combination with the largest difference between the two means is tested first. A test statistic for this pair of means is calculated as follows:

*(Click Image To See a Larger Version)*

where

*(Click Image To See a Larger Version)*

n = number of samples in any group (all groups must be of equal size for Tukey’s HSD Post-Hoc test)

This test statistic is compared to q_{Critical }. The critical q values are found on the Studentized Range q table. A unique critical q value is calculated for each unique combination of level of significance (usually set at 0.05), the degrees of freedom, and the total number of groups in the ANOVA analysis.

Tukey’s test calculates degrees of freedom as follows:

df = Degrees of freedom = (total number of samples in all groups combined) – (total number of groups)

The difference between the two means is designated as significant if its test statistic q is larger than the critical q value from the table.

If the difference between the means with the largest difference is found to be significant, the next inside pair of means is tested. This step is repeated until an innermost pair is found to have a difference that is not significant. Once an inner pair of means is found to have a difference that is not large enough to be significant, no further testing needs to be done because all untested pairs will be inside this one and have even smaller differences between the means.

### Tukey-Kramer Test

**Used When Group Variances Are Equal But Group Sizes Are Unequal**

A slightly variation of Tukey’s HSD test should be used when group sizes are not the same. This variation of Tukey’s HSD test is called the Tukey-Kramer test.

This Tukey-Kramer test will normally be performed instead Tukey’s HSD test by most statistical packages. The Tukey-Kramer test produces the same answer as Tukey’s HSD when group sizes are the same and can be used when group sizes are different (unlike Tukey’s HSD).

Recall that the Tukey’s HSD test statistic for a pair of means is calculated as follows:

*(Click Image To See a Larger Version)*

where

*(Click Image To See a Larger Version)*

The Tukey-Kramer test makes the following adjustment to standard error to account for unequal group sizes. The pooled variance MS** _{Within_Groups}** is multiplied by the average of ( 1/n

_{i }+ 1/n

_{j}) instead of 1/n.

*(Click Image To See a Larger Version)*

As with Tukey’s HSD test, the Tukey-Kramer test calculates Test Statistic q for each pair of means. This Test Statistic is compared to q_{Critical }. The critical q values are found on the Studentized Range q table. A unique critical q value is calculated for each unique combination of level of significance (usually set at 0.05), the degrees of freedom, and the total number of groups in the ANOVA analysis. An Excel lookup function can be conveniently used to obtain the critical q value. The easiest Excel lookup function in this case is Index(array, row, column).

The Tukey-Kramer test calculates degrees of freedom in the same way as Tukey’s HSD test as follows:

df = Degrees of freedom = (total number of samples) – (total number of groups)

The difference between the two means is designated as significant if its test statistic q is larger than the critical q value from the table.

The Tukey-Kramer test will be performed on the sample data shortly.

### Games-Howell Test

**Used When Groups Variances Are Not Equal. Groups Sizes Can Be Unequal.**

The Games-Howell test is the Post-Hoc test used when group variances cannot be confirmed to be homogeneous (similar). The Games-Howell test can be used whether or not sample sizes are the same.

When group variances are shown to be dissimilar, the Single-factor ANOVA test should be replaced by either Welch’s ANOVA or the Brown-Forsythe F-Test. Both of these tests will be performed on the sample data at the end of this chapter.

The two main tests used to evaluate samples for homogeneity (sameness) of variance are Levene’s test and the Brown-Forsythe test.

** Levene’s test** is an ANOVA test the compares distances between sample values and group means.

** The Brown-Forsythe test** is an ANOVA test that compares distances between sample values and group medians. The Brown-Forsythe test is more robust because it is less affected by outliers since it is based on the median and not the mean as Levene’s test is.

The ** F-test** is not a good test to compare variances because it is extremely sensitive to non-normality of sample data.

The Games-Howell Post-Hoc test is performed in the same manner as Tukey’ HSD and the Tukey-Kramer test. The only differences are the formulas used to calculate standard error and the degrees of freedom.

Recall that the Tukey’s HSD test statistic for a pair of means is calculated as follows:

*(Click Image To See a Larger Version)*

where

*(Click Image To See a Larger Version)*

The Tukey-Kramer test makes the following adjustment to standard error to account for unequal group sizes. The pooled variance MS** _{Within_Groups}** is multiplied by the average of ( 1/n

_{i }+ 1/n

_{j}) instead of 1/n.

*(Click Image To See a Larger Version)*

Notice that both Tukey’s HSD and the Tukey-Kramer test use a pooled variance MS** _{Within_Groups}** because groups variances are similar. The Games-Howell test assumes dissimilar groups variances and calculates the standard error using individual variances of each group as follows:

*(Click Image To See a Larger Version)*

The Games-Howells test calculates degrees of freedom in a different way as well. The formula for df is as follows:

*(Click Image To See a Larger Version)*

In Excel terms, the formula is expressed as the following:

df = **(**** (** (Var_{1}/n_{1}) + (Var_{2}/n_{2}) **)**^2 **) **/ **(** **(**(Var_{1}/n_{1})^2 / (n_{1} - 1) **)** + **( **(Var_{2}/n_{2})^2 / (n_{2}-1) } **)**

This is the same formula used to calculate degrees of freedom for a two-independent sample, unpooled t-test. This t-test is known as Welch’s t-test. As mentioned, when group variances are unequal, Single-Factor ANOVA is replaced by Welch’s ANOVA or the Brown-Forsythe F-Test.

As with Tukey’s HSD test and the Tukey-Kramer test, the Games-Howell test calculates Test Statistic q for each pair of means. This Test Statistic is compared to q_{Critical}_{ }. The critical q values are found on the Studentized Range q table. A unique critical q value is calculated for each unique combination of level of significance (usually set at 0.05), the degrees of freedom, and the total number of groups in the ANOVA analysis. An Excel lookup function can be conveniently used to obtain the critical q value. The easiest Excel lookup function in this case is Index(array, row, column).

The difference between the two means is designated as significant if its test statistic q is larger than the critical q value from the table.

The Games-Howell test will be performed on the sample data shortly.

**Excel Master Series Blog Directory**

Statistical Topics and Articles In Each Topic

- Histograms in Excel
- Bar Chart in Excel
- Combinations & Permutations in Excel
- Normal Distribution in Excel
- Overview of the Normal Distribution
- Normal Distribution’s PDF (Probability Density Function) in Excel 2010 and Excel 2013
- Normal Distribution’s CDF (Cumulative Distribution Function) in Excel 2010 and Excel 2013
- Solving Normal Distribution Problems in Excel 2010 and Excel 2013
- Overview of the Standard Normal Distribution in Excel 2010 and Excel 2013
- An Important Difference Between the t and Normal Distribution Graphs
- The Empirical Rule and Chebyshev’s Theorem in Excel – Calculating How Much Data Is a Certain Distance From the Mean
- Demonstrating the Central Limit Theorem In Excel 2010 and Excel 2013 In An Easy-To-Understand Way

- t-Distribution in Excel
- Binomial Distribution in Excel
- z-Tests in Excel
- Overview of Hypothesis Tests Using the Normal Distribution in Excel 2010 and Excel 2013
- One-Sample z-Test in 4 Steps in Excel 2010 and Excel 2013
- 2-Sample Unpooled z-Test in 4 Steps in Excel 2010 and Excel 2013
- Overview of the Paired (Two-Dependent-Sample) z-Test in 4 Steps in Excel 2010 and Excel 2013

- t-Tests in Excel
- Overview of t-Tests: Hypothesis Tests that Use the t-Distribution
- 1-Sample t-Tests in Excel
- 1-Sample t-Test in 4 Steps in Excel 2010 and Excel 2013
- Excel Normality Testing For the 1-Sample t-Test in Excel 2010 and Excel 2013
- 1-Sample t-Test – Effect Size in Excel 2010 and Excel 2013
- 1-Sample t-Test Power With G*Power Utility
- Wilcoxon Signed-Rank Test in 8 Steps As a 1-Sample t-Test Alternative in Excel 2010 and Excel 2013
- Sign Test As a 1-Sample t-Test Alternative in Excel 2010 and Excel 2013

- 2-Independent-Sample Pooled t-Tests in Excel
- 2-Independent-Sample Pooled t-Test in 4 Steps in Excel 2010 and Excel 2013
- Excel Variance Tests: Levene’s, Brown-Forsythe, and F Test For 2-Sample Pooled t-Test in Excel 2010 and Excel 2013
- Excel Normality Tests Kolmogorov-Smirnov, Anderson-Darling, and Shapiro Wilk Tests For Two-Sample Pooled t-Test
- Two-Independent-Sample Pooled t-Test - All Excel Calculations
- 2- Sample Pooled t-Test – Effect Size in Excel 2010 and Excel 2013
- 2-Sample Pooled t-Test Power With G*Power Utility
- Mann-Whitney U Test in 12 Steps in Excel as 2-Sample Pooled t-Test Nonparametric Alternative in Excel 2010 and Excel 2013
- 2- Sample Pooled t-Test = Single-Factor ANOVA With 2 Sample Groups

- 2-Independent-Sample Unpooled t-Tests in Excel
- 2-Independent-Sample Unpooled t-Test in 4 Steps in Excel 2010 and Excel 2013
- Variance Tests: Levene’s Test, Brown-Forsythe Test, and F-Test in Excel For 2-Sample Unpooled t-Test
- Excel Normality Tests Kolmogorov-Smirnov, Anderson-Darling, and Shapiro-Wilk For 2-Sample Unpooled t-Test
- 2-Sample Unpooled t-Test Excel Calculations, Formulas, and Tools
- Effect Size for a 2-Independent-Sample Unpooled t-Test in Excel 2010 and Excel 2013
- Test Power of a 2-Independent Sample Unpooled t-Test With G-Power Utility

- Paired (2-Sample Dependent) t-Tests in Excel
- Paired t-Test in 4 Steps in Excel 2010 and Excel 2013
- Excel Normality Testing of Paired t-Test Data
- Paired t-Test Excel Calculations, Formulas, and Tools
- Paired t-Test – Effect Size in Excel 2010, and Excel 2013
- Paired t-Test – Test Power With G-Power Utility
- Wilcoxon Signed-Rank Test in 8 Steps As a Paired t-Test Alternative
- Sign Test in Excel As A Paired t-Test Alternative

- Hypothesis Tests of Proportion in Excel
- Hypothesis Tests of Proportion Overview (Hypothesis Testing On Binomial Data)
- 1-Sample Hypothesis Test of Proportion in 4 Steps in Excel 2010 and Excel 2013
- 2-Sample Pooled Hypothesis Test of Proportion in 4 Steps in Excel 2010 and Excel 2013
- How To Build a Much More Useful Split-Tester in Excel Than Google's Website Optimizer

- Chi-Square Independence Tests in Excel
- Chi-Square Goodness-Of-Fit Tests in Excel
- F Tests in Excel
- Correlation in Excel
- Pearson Correlation in Excel
- Spearman Correlation in Excel
- Confidence Intervals in Excel
- z-Based Confidence Intervals of a Population Mean in 2 Steps in Excel 2010 and Excel 2013
- t-Based Confidence Intervals of a Population Mean in 2 Steps in Excel 2010 and Excel 2013
- Minimum Sample Size to Limit the Size of a Confidence interval of a Population Mean
- Confidence Interval of Population Proportion in 2 Steps in Excel 2010 and Excel 2013
- Min Sample Size of Confidence Interval of Proportion in Excel 2010 and Excel 2013

- Simple Linear Regression in Excel
- Overview of Simple Linear Regression in Excel 2010 and Excel 2013
- Complete Simple Linear Regression Example in 7 Steps in Excel 2010 and Excel 2013
- Residual Evaluation For Simple Regression in 8 Steps in Excel 2010 and Excel 2013
- Residual Normality Tests in Excel – Kolmogorov-Smirnov Test, Anderson-Darling Test, and Shapiro-Wilk Test For Simple Linear Regression
- Evaluation of Simple Regression Output For Excel 2010 and Excel 2013
- All Calculations Performed By the Simple Regression Data Analysis Tool in Excel 2010 and Excel 2013
- Prediction Interval of Simple Regression in Excel 2010 and Excel 2013

- Multiple Linear Regression in Excel
- Basics of Multiple Regression in Excel 2010 and Excel 2013
- Complete Multiple Linear Regression Example in 6 Steps in Excel 2010 and Excel 2013
- Multiple Linear Regression’s Required Residual Assumptions
- Normality Testing of Residuals in Excel 2010 and Excel 2013
- Evaluating the Excel Output of Multiple Regression
- Estimating the Prediction Interval of Multiple Regression in Excel
- Regression - How To Do Conjoint Analysis Using Dummy Variable Regression in Excel

- Logistic Regression in Excel
- Logistic Regression Overview
- Logistic Regression in 6 Steps in Excel 2010 and Excel 2013
- R Square For Logistic Regression Overview
- Excel R Square Tests: Nagelkerke, Cox and Snell, and Log-Linear Ratio in Excel 2010 and Excel 2013
- Likelihood Ratio Is Better Than Wald Statistic To Determine if the Variable Coefficients Are Significant For Excel 2010 and Excel 2013
- Excel Classification Table: Logistic Regression’s Percentage Correct of Predicted Results in Excel 2010 and Excel 2013
- Hosmer- Lemeshow Test in Excel – Logistic Regression Goodness-of-Fit Test in Excel 2010 and Excel 2013

- Single-Factor ANOVA in Excel
- Overview of Single-Factor ANOVA
- Single-Factor ANOVA in 5 Steps in Excel 2010 and Excel 2013
- Shapiro-Wilk Normality Test in Excel For Each Single-Factor ANOVA Sample Group
- Kruskal-Wallis Test Alternative For Single Factor ANOVA in 7 Steps in Excel 2010 and Excel 2013
- Levene’s and Brown-Forsythe Tests in Excel For Single-Factor ANOVA Sample Group Variance Comparison
- Single-Factor ANOVA - All Excel Calculations
- Overview of Post-Hoc Testing For Single-Factor ANOVA
- Tukey-Kramer Post-Hoc Test in Excel For Single-Factor ANOVA
- Games-Howell Post-Hoc Test in Excel For Single-Factor ANOVA
- Overview of Effect Size For Single-Factor ANOVA
- ANOVA Effect Size Calculation Eta Squared in Excel 2010 and Excel 2013
- ANOVA Effect Size Calculation Psi – RMSSE – in Excel 2010 and Excel 2013
- ANOVA Effect Size Calculation Omega Squared in Excel 2010 and Excel 2013
- Power of Single-Factor ANOVA Test Using Free Utility G*Power
- Welch’s ANOVA Test in 8 Steps in Excel Substitute For Single-Factor ANOVA When Sample Variances Are Not Similar
- Brown-Forsythe F-Test in 4 Steps in Excel Substitute For Single-Factor ANOVA When Sample Variances Are Not Similar

- Two-Factor ANOVA With Replication in Excel
- Two-Factor ANOVA With Replication in 5 Steps in Excel 2010 and Excel 2013
- Variance Tests: Levene’s and Brown-Forsythe For 2-Factor ANOVA in Excel 2010 and Excel 2013
- Shapiro-Wilk Normality Test in Excel For 2-Factor ANOVA With Replication
- 2-Factor ANOVA With Replication Effect Size in Excel 2010 and Excel 2013
- Excel Post Hoc Tukey’s HSD Test For 2-Factor ANOVA With Replication
- 2-Factor ANOVA With Replication – Test Power With G-Power Utility
- Scheirer-Ray-Hare Test Alternative For 2-Factor ANOVA With Replication

- Two-Factor ANOVA Without Replication in Excel
- Randomized Block Design ANOVA in Excel
- Repeated-Measures ANOVA in Excel
- Single-Factor Repeated-Measures ANOVA in 4 Steps in Excel 2010 and Excel 2013
- Sphericity Testing in 9 Steps For Repeated Measures ANOVA in Excel 2010 and Excel 2013
- Effect Size For Repeated-Measures ANOVA in Excel 2010 and Excel 2013
- Friedman Test in 3 Steps For Repeated-Measures ANOVA in Excel 2010 and Excel 2013

- ANCOVA in Excel
- Normality Testing in Excel
- Creating a Box Plot in 8 Steps in Excel
- Creating a Normal Probability Plot With Adjustable Confidence Interval Bands in 9 Steps in Excel With Formulas and a Bar Chart
- Chi-Square Goodness-of-Fit Test For Normality in 9 Steps in Excel
- Kolmogorov-Smirnov, Anderson-Darling, and Shapiro-Wilk Normality Tests in Excel

- Nonparametric Testing in Excel
- Mann-Whitney U Test in 12 Steps in Excel
- Wilcoxon Signed-Rank Test in 8 Steps in Excel
- Sign Test in Excel
- Friedman Test in 3 Steps in Excel
- Scheirer-Ray-Hope Test in Excel
- Welch's ANOVA Test in 8 Steps Test in Excel
- Brown-Forsythe F Test in 4 Steps Test in Excel
- Levene's Test and Brown-Forsythe Variance Tests in Excel
- Chi-Square Independence Test in 7 Steps in Excel
- Chi-Square Goodness-of-Fit Tests in Excel
- Chi-Square Population Variance Test in Excel

- Post Hoc Testing in Excel
- Creating Interactive Graphs of Statistical Distributions in Excel
- Interactive Statistical Distribution Graph in Excel 2010 and Excel 2013
- Interactive Graph of the Normal Distribution in Excel 2010 and Excel 2013
- Interactive Graph of the Chi-Square Distribution in Excel 2010 and Excel 2013
- Interactive Graph of the t-Distribution in Excel 2010 and Excel 2013
- Interactive Graph of the t-Distribution’s PDF in Excel 2010 and Excel 2013
- Interactive Graph of the t-Distribution’s CDF in Excel 2010 and Excel 2013
- Interactive Graph of the Binomial Distribution in Excel 2010 and Excel 2013
- Interactive Graph of the Exponential Distribution in Excel 2010 and Excel 2013
- Interactive Graph of the Beta Distribution in Excel 2010 and Excel 2013
- Interactive Graph of the Gamma Distribution in Excel 2010 and Excel 2013
- Interactive Graph of the Poisson Distribution in Excel 2010 and Excel 2013

- Solving Problems With Other Distributions in Excel
- Solving Uniform Distribution Problems in Excel 2010 and Excel 2013
- Solving Multinomial Distribution Problems in Excel 2010 and Excel 2013
- Solving Exponential Distribution Problems in Excel 2010 and Excel 2013
- Solving Beta Distribution Problems in Excel 2010 and Excel 2013
- Solving Gamma Distribution Problems in Excel 2010 and Excel 2013
- Solving Poisson Distribution Problems in Excel 2010 and Excel 2013

- Optimization With Excel Solver
- Maximizing Lead Generation With Excel Solver
- Minimizing Cutting Stock Waste With Excel Solver
- Optimal Investment Selection With Excel Solver
- Minimizing the Total Cost of Shipping From Multiple Points To Multiple Points With Excel Solver
- Knapsack Loading Problem in Excel Solver – Optimizing the Loading of a Limited Compartment
- Optimizing a Bond Portfolio With Excel Solver
- Travelling Salesman Problem in Excel Solver – Finding the Shortest Path To Reach All Customers

- Chi-Square Population Variance Test in Excel
- Analyzing Data With Pivot Tables
- SEO Functions in Excel
- Time Series Analysis in Excel
- VLOOKUP

## No comments:

## Post a Comment