An Illustrative Guide to Statistical Power, Alpha, Beta, and Critical Values

From my interactions with undergraduate students, it seems that even though these definitions are easy to recite, they are difficult to be integrated into a comprehensive whole. I hope here to show how to conceptually integrate them into a cohesive picture.

This post was originally created for Psychology in Action, and edited by her new blog master, Tawny Tsui. See her posts here.

Everything begins with reality: the “Reality Continuum”

I call this green line “Reality Continuum” (rather grand, no?) because you will take your ideas, and do a reality check against it via data analysis (within the traditional statistical framework–it is definitely NOT the only framework on the market, but I digress).

Odds Are: On the difference between odds, probability, and risk ratio.

Odds, Probability, Chance, Risks: Interchangeable?
Not so much.

What does it mean to say “smokers are X times more likely to get lung cancer than non-smokers?” What about
when the weather channel says, “there is a 10% chance of rain?” The odds of 1 to 10 of winning?

These words are often used in casual conversations as somewhat interchangeable, and can be rather confusing. I remember being very excited to learn about them for the first time, so hopefully you will find this as interesting (or at least as clarifying!) as I did!

A little test!

In which of the following scenario are you most likely to find dessert happiness? Which ones are saying the same thing?

A. The odds against you eating a cupcake are 1 to 5.

B. Your odds of/on eating a cupcake are 1 to 5.

C. The probability of you eating a cupcake is 20%.

D. You have a 20% chance of eating a cupcake.

Answers, in short: A is the most likely-for-cupcake scenario, and C and D are saying the same thing.