Looking for more info and free resources for QCE? Check out these resources!
Inferential statistics are absolutely important for QCE psychology and science in general.
To understand why, let’s jump into an example. You’ve just finished an experiment testing the impact of using black paper and white text for tests rather than white paper with black text. The group using black paper scored an average of 80% and the group using the white paper scored an average of 60%. Great! There was a difference. Now let’s think about the limitations… ok, so we could’ve used more participants to ensure our sample was representative – maybe there was no actual difference due to the papers and it was just due to participant differences? So here’s what we learnt from doing the experiment: maybe using black paper makes a difference and maybe it doesn’t. Hang on a second, that’s exactly what we knew before running the experiment. What. Was. The. Point.
p<0.05
It turns out that there is a point – and that’s because of inferential statistics. Inferential statistics tell us how likely it is that the null hypothesis is true (i.e. that differences in results are just due to chance (including random participant differences)). We consider the results we obtained significant if there is less than 0.05 probability (p<0.05) that the null hypothesis is true. In other words if our experiment produced significant results, there’s at least a 95% probability that using the black or white paper made a difference. Different tests of statistical inference use different methods to figure out what the p value is, and some have higher statistical power than others – you want to use the test with the highest statistical power that’s available to you.
The catch
In order for a test of statistical inference to give you the correct probability of the null hypothesis being true (aka p-value), the assumptions of the test need to be true. This guide on the forums provides information on how to pick the right statistical test, including assumptions for each test. You might not meet the assumptions perfectly, but they’ll give you an idea of what to look out for and how accurate the results of the test will be. Picking the wrong statistical test increases the probability of getting a type 1 error (rejecting the null hypothesis when you shouldn’t) or type 2 error (not rejecting the null hypothesis when you should) so it’s important to pick the right test for the job.
Final thoughts
Inferential statistics aren’t all-powerful, and you still need to consider confounding variables, but they’re great for quantifying the probability that the results you’re seeing are random. Keep in mind that not being able to support the experimental hypothesis is completely fine – it’s all about how well you understand the concepts, carry out the experiment, and analyse it. If you design your methodology well, enact it well, understand what the data means & communicate it effectively those are the hallmarks of a good (& high scoring) scientific report/poster – NOT whether your data says what you want it to. Good luck and happy experimenting! If you want feedback on your work, to request a resource, or to ask a psych question please let us know 😊
