Absence of Evidence Is Not Evidence of Absence
Carl Sagan popularised this phrase, and it applies beautifully to dissertation statistics.
A common issue I see in students’ dissertations is this:
They run a statistical test, get a non-significant result… and then bend over backwards to explain their unexpected, disappointing finding. What they often don’t realise is that their study was simply underpowered.
In other words, the sample was too small to detect the effect in the first place.
You see, absence of evidence is not evidence of absence.
A non-significant result does not automatically mean
• “there is no relationship,”
• “the intervention didn’t work,” or
• “the theory is disproved.”
Sometimes it simply means: your study wasn’t strong enough to pick up the signal.
Why this matters for your dissertation
When a test is underpowered, you can easily get a false negative, a test result that says “no effect” even though one does exist. And then your final chapter may drift off-point, trying to interpret a result that cannot statistically be interpreted.
This weakens the whole dissertation, leading to shaky conclusions and recommendations (needlessly).
What you should do instead
- Do a power analysis before data collection.
This is ideal. Consider the main objective of your study and the statistical test you will use to evaluate its hypothesis. Then use the appropriate formula to estimate the sample size needed to give your study a fair chance of detecting a moderate-sized effect.
A power analysis simply answers:
“How big does my sample need to be to give my study a fighting chance?”
- At the very least, do a post-hoc power check.
If your result is non-significant, calculate the power post hoc. You may discover that your test lacked the strength to detect a significant relationship.
In those cases, it is perfectly appropriate to write something like:
“The test did not reach significance, and the low power suggests that the sample size may have been too small to detect the effect.”
You can then acknowledge any other contributors, such as methodological limitations, measurement weaknesses, or theoretical possibilities, on solid ground.
What strong dissertations do
A strong student will write, for example:
• “This non-significant finding should be interpreted with caution due to the limited sample size and low statistical power.”
• “While the analysis did not detect a significant relationship, this does not rule out the possibility that one exists.”
• “Future studies with larger samples are recommended.”
This is clear, transparent, and trustworthy.
Because: absence of evidence is not evidence of absence.
And recognising this is a hallmark of good research.
If you need help with your statistical analysis, research design, or questionnaire construction, contact me at [email protected].
#dissertationstatistics #statisticalanalysis #quantitativedissertation #statistics