Higher Degree Research

Writing a statistical methods section

November is Academic Writing Month, #AcWriMo and the StatsCome2U team is getting on board with a blog post about how to write the statistical methods section of your paper.

In some journals it is customary to put the Methods section in small font, or even to move it after the main Introduction-Results-Discussion-Conclusion sections. I think the justification for this is that readers don’t care or don’t need to know how the research was done, just what happened in the end. This doesn’t seem right, as an understanding of the process can be as revealing as examination of the outcomes. For me it’s often a deep dive into the Methods section in order to see if this research is a good example for a colleague to follow, or to illustrate a lecture, for instance.

The Methods section should be written in such a way that another researcher could pick up your paper and reproduce the analysis – there’s sufficient detail to explain what the analysis process was. Remember that meme going around about how to draw an owl? Draw two circles, then draw the rest of the owl? A poor Methods section is exactly like that meme – you get a sketchy idea and then the rest is just left to your imagination. Not very helpful when you’re trying to build your methods on that example!

Another great tip for your Methods section is to try not to focus purely on what tests were done to analyse the data. Behind each test is a model of the data generating process and those theoretical readers who are aiming to follow your methods in their research will need that level of detail.

There are a number of checklists available in different disciplines and for different study designs that can also be helpful to ensure that your Methods section covers everything that it needs to. One example is the STROBE statement, for STrengthening the Reporting of Observational studies in Epidemiology. Other examples of good practice in reporting Methods in the medical and biological sciences can be found here and here. I found these examples as a result of a Stack Exchange conversation and if you have any great examples from your discipline, please let me know!

A feature of these two scientific papers is that they are highly tailored to the situation at hand – there is very little feel of sentences copied and pasted from previous research. This copy-paste approach – sometimes known as the boiler-plate approach – to a statistical methods section is risky though. White and colleagues use the term trope to refer to phrases such as “A p-value of less than 0.05 was considered statistically significant”. That was a really interesting choice of name, avoiding more loaded terms such as clichĂ©, and conjuring up some wonderful images and connections for me a life-long interest in languages.

So what of the term “trope”? It’s derived from the Greek word tropos meaning to turn. We find it in the mathematical term entropy, capturing the notion of “turning in”. We also find it in medieval church music, where a trope is a piece of text and music added into a Biblical plainchant. Sometimes the text is explanatory, putting the Bible passage into context. Sometimes is purely decorative, associated with long flowing passages of notes of great beauty but only a small amount of relevance to the text around it.

We even find it in the word tropical, where it refers to the parts of the Earth where the Sun “turns back” after reaching its most northern or southern point. So a trope is a phrase for which the meaning has turned away from the literal to metaphorical. When we say “Stop and smell the roses” we don’t usually mean to literally, stop and smell the roses, we mean slow down and enjoy the good things in the here and now.

I think White et al are therefore suggesting that the phrase “A p-value of less than 0.05 was considered statistically significant” doesn’t actually mean that, rather it is being use in the sense of “we didn’t really think very much about the research question or model, we did some statistical tests and will let you to decide if p < 0.05 means anything in terms of scientific importance.”

The statisticians in the Statistical Support Network are here to help you tease out the statistical heart of your Methods section from the tropes, and to contribute to a paper that is reproducible as well as telling a compelling story about the results of your research. Don’t just stop and smell the roses, think about your data generating process and the models you build to describe it!

Associate Professor Alice Richardson is Lead of the Statistical Support Network at the Australian National University. Her research interests are in linear models and robust statistics; statistical properties of data mining methods; and innovation in statistics education. In her role at the SSN she applies statistical methods to large and small data sets, especially for research questions in population health and the biomedical sciences.

Leave a Reply

Your email address will not be published. Required fields are marked *

*