During its history of almost 40 years, the SCU has collected a big archive and some of that material is still relevant today. The Letters of SCU series will bring out some of these masterpieces to bring age-old problems back in the limelight.
Dear SCU,
Please help me with this intransigent referee! I have two rye grass genotypes that show different responses to dry conditions. The growth pattern is delayed in Genotype B (p=0.001) but not in Genotype A (p-value n.s.). The referee keeps referring to “interactions”, but I think my data are clear enough. Can you help me respond to put an end to his endless whining?
Signed: Rye-thing in Despair
Dear Rye-thing,
I’m sorry to hear about your difficulties with the referee. However, he is correct that you have not yet shown statistically that there is a difference in genotype responses to differing conditions. But we’re here to help, and I can show you how to statistically compare the normal-dry contrast between genotypes.
You’ve measured the percent change in leaf mass for each plant, some of which were exposed to normal conditions, and the others to dry conditions. You can summarise the results of this experiment in a 2×2 table, e.g. Table 1:
Table 1. Mean percent change in leaf mass
| Genotype A | Genotype B | Total | |
| Normal conditions | 19.9 | 22.0 | 20.9 |
| Dry conditions | 17.3 | 17.7 | 17.5 |
| Mean Difference | 2.6 | 4.3 | 3.4 |
The Total column on the right shows the overall mean percent leaf mass growth under normal (20.9) and dry conditions (17.5) as well as the mean difference (3.4). The question you need to address whether the normal-dry contrast is similar between genotypes, or whether there is evidence of a DIFFERENCE in the normal-dry difference (2.6 vs 4.3).
Within-genotype comparisons of normal vs dry don’t address this question of a between-genotype DIFFERENCE in the differences. In this example, t-tests for each genotype comparing normal vs dry might mislead the researcher into believing that Genotype A and Genotype B respond differently to changes in water availability.
> t.test(dat$leaf[1:9],dat$leaf[10:18]) #Genotype A, normal vs dry
t = 1.8486, df = 15.875, p-value = 0.083 (n.s.)
> t.test(dat$leaf[19:27],dat$leaf[28:36]) #Genotype B, normal vs dry
t = 3.9166, df = 15.915, p-value = 0.001 (**)
In this example, a combination of a more pronounced mean difference between normal and dry conditions in Genotype B, and somewhat smaller standard errors resulted in the much smaller p-values for this comparison. But standard error estimates may not be robust for small sample sizes. It would be more prudent to pool variances and consider an analysis (such as a two-way ANOVA) that uses more robust standard error estimates.
A difference in the normal-dry contrast between genotypes is called a genotype by condition interaction (a DIFFERENCE of differences). This is what the referee was referring to.
So how do we compare the normal-dry contrast between genotypes statistically? An estimate of both mean differences, as well as the standard error of the difference in the mean difference, would allow us to make this comparison statistically. But a good statistical software package can do most of the work for us.
For example in R, we can specify a model where Leaf_growth is the response variable and Genotype and condition are the factors of interest:
Leaf_growth~Genotype+condition+Genotype*condition
Fitting a linear model to the data can test for evidence of a genotype by condition interaction:
> check<-lm(leaf~condition+Genotype+condition*Genotype, data=dat)
> summary(check)
Estimate Std. Error t value Pr(>|t|)
(Intercept) 17.3096 0.8808 19.653 <2e-16 ***
conditionNormal 2.5428 1.2456 2.041 0.0495 *
GenotypeB 0.3757 1.2456 0.302 0.7649
condNormal:GenoB 1.7673 1.7615 1.003 0.3233
—
This more robust analysis suggests that evidence for a genotype by condition interaction (p=0.32) is lacking. In this case, we may want to get estimates of a Genotype effect and a condition effect in the absence of an interaction:
>check<-lm(leaf~condition+Genotype, data=dat)
> summary(check)
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 16.8678 0.7628 22.11 < 2e-16 ***
conditionNormal 3.4264 0.8809 3.89 0.000459 ***
GenotypeB 1.2593 0.8809 1.43 0.162223
This model assumes a common Normal:Dry contrast across genotypes. The common mean difference in percent leaf growth is 3.4 (SE 0.88) and this is statistically significant (p=0.00046). There is insufficient evidence to conclude that the two genotypes have different mean percent leaf growth (p=0.16).
For more discussion of interactions and models, check out these references:
A. Berrington de González and D. R. Cox, Interpretation of Interaction: A Review, The Annals of Applied Statistics, Vol. 1, No. 2 (Dec., 2007), pp. 371-385
Interpreting interactions in regression
Editor’s note
The Letters to SCU were originally written by Dr. Terry Neeman and are republished here with permission.
Terry has more than two decades of experience as a Biostatistician working with Biologists, Biomedical Researchers and Clinicians in both Industry and Academia. Terry enjoys the statistical challenges associated with all aspects of experimental work; from experimental design to data exploration and statistical modelling.