November is Academic Writing Month at ANU. No better time thus to put the translation process of data to words in the spotlight.
“The secret language of statistics, so appealing in a fact minded culture, is employed to sensationalise, inflate, confuse, and oversimplify. Statistical methods and statistical terms are necessary in reporting the mass data of social and economic trends, business conditions, “opinion” polls, the census. But without writers who use the words with honesty and understanding and readers who know what they mean, the result can only be semantic nonsense.”
From: How to lie with Statistics – Darrell Huff (Chapter 1, p. 10)
This rather cynical quote points to the responsibility of any academic author, whether it is a thesis or a journal paper, to ensure that their writing is with the reader’s understanding in front of mind. At the same time, they should never loose sight of the objectivity and preciseness required to fulfil the integrity of the scientific process.
More specifically, when writing down the statistical methods and results section of an academic piece, a few of the goals that we aim for are: (1) reproducibility, (2) objectivity and clarity, and (3) preempt misunderstanding.
Reproducibility
In the context of academic writing, reproducibility refers to the ability for an independent reviewer/researcher to replicate the reported results based on the information provided in the paper.
Writing reproducible methods starts with conducting reproducible analyses. The National Academy of Sciences has published several guides to aid this process but a good starting point would be to document every single step of the analysis process. For science researchers, this would be akin to maintaining a lab book. In essence, it is a record of every data manipulation and calculation that was performed before obtaining the end result.
Statistical software can be of great help here. By avoiding copy/paste or point-and-click processes, but instead utilising the syntax or coding functions of the software you have an immediate record of all the steps performed during the analysis process. Ideally, this code will include every manipulation and estimation that were performed on the raw data.
Objectivity and clarity
“The real purpose of the scientific method is to make sure nature hasn’t misled you into thinking you know something you actually don’t know.”
Robert Pirsig, Zen and the Art of Motorcycle Maintenance
When reporting statistics your field of research or intended journal for publication will typically have guidelines as to what exactly needs to be reported. In general, the common rule would be that as a writer you need to report all the numbers required for a reader to have a full picture and enable them to draw their own conclusion.
For example, when reporting p-values you could simply go with stating whether a p < 0.05 or p > 0.05, indicating that you have evaluated the actual number against an arbitrary threshold. However, you rob the reader of the more precise information on the exact p-value.
Of course, often p-values have too many decimals which would hamper the flow to the text. But a good practice would be to always report exact p-values up to 3 decimals and when a p-value is too small, state p < 0.001. This way you compromise between the flow of your writing whilst maintaining the objectivity and clarity of your writing.
As a side note, it would be even better practice to accompany your p-values with supporting statistics of which the p-value was derived, as well as confidence intervals and/or effect sizes where appropriate.
Preempt misunderstanding
“… When we reason informally – call it intuition, if you like – we use rules of thumb which simplify problems for the sake of efficiency. Many of these shortcuts have been well characterised in a field called ‘heuristics’, and they are efficient ways of knowing in many circumstances.
This convenience comes at a cost – false beliefs – because there are systematic vulnerabilities in these truth-checking strategies which can be exploited. …”
From: Bad Science – Ben Goldacre (Chapter 13: Why clever people believe stupid things)
Everyone of us comes with their own set of beliefs. As a writer it is important to be conscious and open about your own set of beliefs. Simultaneously, you will need to have an understanding of your readers’ beliefs as conflicting belief systems may lead to incompatible interpretations.
When your writing is reproducible and objective, you have already taken important steps towards avoiding your research being misunderstood as the reader has all the necessary tools to draw their own conclusions. When your conclusions are supported by the data and your statistical analyses are sound, your readers will inevitably concur.
Avoiding misunderstanding also relates to the description of the statistical methods you used. All too often we see papers in which well-known statistical analysis techniques are referred to by their software name, or more obscure statistical analyses are not well referenced or justified. Readers will have confidence in your results when they belief that those results were obtained in the most appropriate way.
As a word of caution though, belief systems come also into play when selecting statistical methods for data analysis and while there is often more than one way to analyse your data, not every single method is always appropriate. But that is probably leading us too far from the writing aspect.
For further reading on how to translate your data for understanding, check out this post by The Writing Center and this online video tutorial.
The Statistical Consulting Unit values research integrity and we see writing as an integral part of that through appropriate representation of statistical analyses and results.
Marijke joined the Statistical Consulting Unit in May 2019. She is passionate about explaining statistics, especially to those who deem themselves not statistically gifted.