NARIC’s Research In Focus series highlights new and interesting findings from NIDILRR-funded studies, presented in lay language summaries. The series covers a wide array of topics, and aims to present peer-reviewed research in readable formats, so our readers can learn about issues that affect them every day. The Research in Brief companion series breaks down some of the concepts readers might come across when exploring research in disability and rehabilitation. This issue introduces the concepts of statistical significance, associations and p-values.
Oftentimes, when researching science questions, we are looking for unambiguous answers. Does smoking cause lung cancer? Will drinking cranberry juice prevent UTI’s? But rarely in science is the answer simple and clear cut. Instead, you may come across terminology like: this X is significantly associated with Y.
For example, a recent NIDILRR-funded study found that, among persons receiving community mental health services, there was a significant association with having a serious mental illness and experiencing PTSD and symptoms of prolonged grief. Does this mean that having a serious mental illness will always result in also having PTSD?
No. And here’s why it’s important to understand what it means when an author says something is significantly associated with something else. Statistical significance gives you a clearer understanding of scientific results without making wrong assumptions.
In short, a statistical significance indicates that the relationship between two variables cannot be explained by chance alone. In other words, something other than chance is responsible for the association between the two variables.
Going back to our earlier example, the researchers found that the probability of some persons with mental illness also experiencing PTSD and prolonged grief could not be explained by purely chance. However, this doesn’t necessarily mean that one thing automatically leads to another. In fact, there may be an entirely different variable that is causing the relationship we are seeing. The study’s authors noted that people with mental illness are more likely to experience trauma or the sudden loss of a loved one than the general population, which could result in PTSD. What the results tell us is that there is a relationship there and it isn’t due to chance.
Scientists use statistical hypothesis testing to calculate statistical significance. They are testing whether the null hypothesis (that the results are due to chance alone) is true. In a scientific article, you may see something called a p-value. The p-value is used to determine statistical significance. If you are looking at the results section of a scientific article, look for the p-values listed on the tables. Many times, articles will either italicize or bolden significant p-values. These are the results that are further explained in the results and discussion sections of papers.
Generally, if the p-value is 5% or lower then the null hypothesis is rejected and the results are not due to chance. However, a p-value that is higher than 5% is interpreted to mean that there is not enough data to disprove or reject the null hypothesis.
The next time you come across a study that shows a statistically significant association between two things, remember it doesn’t mean that X causes Y. Instead, it is simply showing an important relationship there that is not due to chance.
Hafsa Abdirahman MPH is a public health scientist and freelance medical writer and editor. She believes that access to evidence-based, quality health information can save lives and she’s worked throughout her career to put this belief into practice.
Collection Spotlight from the National Rehabilitation Information Center 