Wednesday, May 28, 2014

2-Factor ANOVA w/Rep – Test Power With G-Power Utility

This is one of the following seven articles on Two-Factor ANOVA With Replication in Excel

Two-Factor ANOVA With Replication in 5 Steps in Excel 2010 and Excel 2013

Variance Tests: Levene’s and Brown-Forsythe For 2-Factor ANOVA in Excel 2010 and Excel 2013

Shapiro-Wilk Normality Test in Excel For 2-Factor ANOVA With Replication

2-Factor ANOVA With Replication Effect Size in Excel 2010 and Excel 2013

Excel Post Hoc Tukey’s HSD Test For 2-Factor ANOVA With Replication

2-Factor ANOVA With Replication – Test Power With G-Power Utility

Scheirer-Ray-Hare Test Alternative For 2-Factor ANOVA With Replication

 

Power of the Two-Factor

ANOVA Test With

Replication Using Free

Utility G*Power

The accuracy of a statistical test is very dependent upon the sample size. The larger the sample size, the more reliable will be the test’s results. The accuracy of a statistical test is specified as the Power of the test. A statistical test’s Power is the probability that the test will detect an effect of a given size at a given level of significance (alpha). The relationships are as follows:

α (“alpha”) = Level of Significance = 1 – Level of Confidence

α = probability of a type 1 error (a false positive)

α = probability of detecting an effect where there is none

Β (“beta”) = probability of a type 2 error (a false negative)

Β = probability of not detecting a real effect

1 - Β = probability of detecting a real effect

Power = 1 - Β

Power needs to be clarified further. Power is the probability of detecting a real effect of a given size at a given Level of Significance (alpha) at a given total sample size and number of groups.

The term Power can be described as the accuracy of a statistical test. The Power of a statistical test is related with alpha, sample size, and effect size in the following ways:

The larger the sample size, the larger is a test’s Power because a larger sample size increases a statistical test’s accuracy.

The larger alpha is, the larger is a test’s Power because a larger alpha reduces the amount of confidence needed to validate a statistical test’s result. Alpha = 1 – Level of Confidence. The lower the Level of Confidence needed, the more likely a statistical test will detect an effect.

The larger the specified effect size, the larger is a test’s Power because a larger effect size is more likely to be detected by a statistical test.

If any three of the four related factors (Power, alpha, sample size, and effect size) are known, the fourth factor can be calculated. These calculations can be very tedious. Fortunately there are a number of free utilities available online that can calculate a test’s Power or the sample size needed to achieve a specified Power. One very convenient and easy-to-use downloadable Power calculator called G-Power is available at the following link at the time of this writing:

http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/

Power calculations are generally used in two ways:

1) A priori - Calculation of the minimum sample size needed to achieve a specified Power to detect an effect of a given size at a given alpha. This is the most common use of Power analysis and is normally conducted a priori (before the test is conducted) when designing the test. A Power level of 80 percent for a given alpha and effect size is a common target. Sample size is increased until the desired Power level can be achieved. Since Power equals 1 – Β, the resulting Β of the targeted Power level represents the highest acceptable level of a type 2 error (a false negative – failing to detect a real effect). Calculation of the sample size necessary to achieve a specified Power requires three input variables:

a) Power level – This is often set at .8 meaning that the test has an 80 percent to detect an effect of a given size.

b) Effect size - Effect sizes are specified by the variable f. Effect size f is calculated from a different measure of effect size called η2 (eta square). η2 = SSBetween_Groups / SSTotal These two terms are part of the ANOVA calculations found in the Single-factor ANOVA output.

The relationship between effect size f and effect size η2 is as follows:

anova, two-way anova, two way anova, two-factor anova, two factor anova, test power, statistics, excel, excel 2010, excel 2013

As mentioned, effect sizes are often generalized as follows:

η2 = 0.01 for a small effect. A small effect is one that not easily observable.

η2 = 0.05 for a medium effect. A medium effect is more easily detected than a small effect but less easily detected than a large effect.

η2 = 0.14 for a small effect. A large effect is one that is readily detected with the current measuring equipment.

The above values of η2 produce the following values of effect size f:

f = 0.1 for a small effect.

f = 0.25 for a medium effect.

f = 0.4 for a large effect.

c) Alpha – This is commonly set at 0.05.

Calculating Power With Online

Tool G*Power

An example of a priori Power calculation would be the following. Power calculations are normally used a priori to determine the total ANOVA sample size necessary to achieve a specific Power level for detecting an effect of a specified size at a given alpha.

Power will be calculated separately for each of the Main Effects F Tests.

Power Calculation For the Factor 1 Main Effects F Test

The Factor 1 Main Effect F Test has the following parameters:

Number of Groups in the Factor 1 Main Effects F Test = k = 3

Numerator df = k – 1 = 2

Total number of groups = (number of levels of Factor 1) X (number of levels of Factor 2) = 3 X 2 = 6

Total sample size = total number of data observations contains in all level data groups that are part of the Factor 1 Main Effects F Test = 24

Determining the power of this F Test to detect a large effect (f = 0.4) at an alpha level of 0.05 would be calculated using the G*Pwer utility as follows:

anova, two-way anova, two way anova, two-factor anova, two factor anova, test power, statistics, excel, excel 2010, excel 2013 (Click On Image To See a Larger Version)

The preceding G*Power dialogue box and output shows the power of this F Test to be 0.345. That means that this F Test has a 34.5 percent chance of detect a large effect (f = 0.4) at an alpha level of 0.05. Determining the power of the current test is post hoc analysis. The type of analysis selected in the dialogue box is the Post Hoc selection.

A priori analysis can also be performed with the G*Power utility. A priori analysis would be used to determine the sample size necessary to achieve a given power level. When a prior analysis is selected on G*Power, the following chart can be generated which indicates the total sample size necessary to generate various power levels for the test using the current parameters.

anova, two-way anova, two way anova, two-factor anova, two factor anova, test power, statistics, excel, excel 2010, excel 2013 (Click On Image To See a Larger Version)

This diagram shows that a total sample size of at least 63 or 64 would be necessary this 3-level F Test within this two-factor ANOVA test to generate a power level of 0.8 to detect a large effect (f = 0.4) at an alpha level of 0.05. Four replicates in each of the six unique treatment cells means that the current total sample size is 24. At least 11 replicates would be needed in each treatment cell for this F Test to achieve a power level of 0.8. A power level of 0.8 means that a test has an 80 percent chance of detecting an effect of the specified size at the given alpha level.

 

Power Calculation For the Factor 2 Main Effects F Test

The Factor 1 Main Effect F Test has the following parameters:

Number of Groups in the Factor 1 Main Effects F Test = k = 2

Numerator df = k – 1 = 1

Total number of groups = (number of levels of Factor 1) X (number of levels of Factor 2) = 3 X 2 = 6

Total sample size = total number of data observations contains in all level data groups that are part of the Factor 1 Main Effects F Test = 24

Determining the power of this F Test to detect a large effect (f = 0.4) at an alpha level of 0.05 would be calculated using the G*Power utility as follows:

anova, two-way anova, two way anova, two-factor anova, two factor anova, test power, statistics, excel, excel 2010, excel 2013 (Click On Image To See a Larger Version)

The preceding G*Power dialogue box and output shows the power of this F Test to be 0.458. That means that this F Test has a 45.8 percent chance of detect a large effect (f = 0.4) at an alpha level of 0.05. Determining the power of the current test is post hoc analysis. The type of analysis selected in the dialogue box is the Post Hoc selection.

A priori analysis can also be performed with the G*Power utility. A priori analysis would be used to determine the sample size necessary to achieve a given power level. When a prior analysis is selected on G*Power, the following chart can be generated which indicates the total sample size necessary to generate various power levels for the test using the current parameters.

anova, two-way anova, two way anova, two-factor anova, two factor anova, test power, statistics, excel, excel 2010, excel 2013 (Click On Image To See a Larger Version)

This diagram shows that a total sample size of at least 50 would be necessary this 3-level F Test within this two-factor ANOVA test to generate a power level of 0.8 to detect a large effect (f = 0.4) at an alpha level of 0.05. Four replicates in each of the six unique treatment cells means that the current total sample size is 24. At least 9 replicates would be needed in each treatment cell for this F Test to achieve a power level of 0.8. A power level of 0.8 means that a test has an 80 percent chance of detecting an effect of the specified size at the given alpha level.

 

Power Calculation For the Interaction Effect F Test

All F Tests that are part of the same ANOVA test use nearly all of the same input parameters for the G*Power utility. The only input parameter that varies for different F Test is the Numerator df. The Numerator df for the interaction effect equals (number of Factor 1 levels – 1) X (Number of Factor 2 levels – 1). In this case, the following calculation is performed:

Numerator df = (3 – 1) X (2 – 1) = 2

This is the same Numerator df as used by G*Power for the Factor 1 Main Effects F Test. The G*Power output will therefore be the same for both F Tests.

 

Excel Master Series Blog Directory

Statistical Topics and Articles In Each Topic

 

Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)