Don't make such black and white interpretations

Factors Determining the p-value

Test Statistic

Misinterpretations of the p-value

Alpha (Level of Significance)

p-hacking

The 2015 Ban on Null Hypothesis Testing

Alpha Criterion

What does a p value represent in the context of psychological research?

If a psychology researcher calculates a p-value of 0.08, they will typically declare the experimental result to be statistically significant.

A social psychologist is analyzing data from four different studies. Based on the standard thresholds for statistical significance provided in the table, match each study's obtained p value with its correct interpretation.

Misinterpretation of the p Value

Determinants of the $p$ Value

A researcher obtains a $p$ value of $0.02$ and labels the result 'highly significant' ($**$). According to standard significance thresholds, where $* = p \leq 0.05$ and $** = p \leq 0.01$, this labeling is _____.

A cognitive psychologist conducts an experiment on learning styles and calculates a $p$ value of $0.04$. Applying the standard threshold of significance ($p = 0.05$) described in the text, the researcher should declare this result to be statistically significant.

Define what a $p$ value represents in the context of an experiment and state the standard threshold for declaring an experimental result to be statistically significant.

Reporting Effect Sizes in Null Hypothesis Significance Testing

Using the convention stated in this question for marking results with asterisks (a cut-off value itself counts as meeting it), match each label with the range of $p$ values it marks.

By a common convention in psychology, a result is declared statistically significant when $p \leq 0.05$. For a result exactly at this threshold ($p = 0.05$), results at least as extreme would occur, under the test's model (the null hypothesis plus the test's assumptions), $1$ time in how many?

By a common convention in psychology, a result is declared statistically significant when $p \leq 0.05$. For a result exactly at this threshold ($p = 0.05$), if the null hypothesis and the test's assumptions were true, results at least as extreme would occur 1 time in _____.

A psychologist has $p$ values from four different studies. Arrange the $p$ values in ascending numerical order, from the smallest to the largest. (This orders the numbers only: $p$ values from different studies, each calculated under its own model, do not by themselves rank how strong the evidence is.)

A psychology student runs an analysis on their study's data and obtains a $p$ value of $0.03$. Using the common convention of a significance level of $0.05$ (a result is significant when $p \leq 0.05$), what conclusion should the student draw about their experimental result, and why?

Based on the definition of a $p$ value and a significance level of $0.05$ set in advance, explain in your own words what it means if, assuming the null hypothesis and the test's assumptions were true, a result at least as extreme as the researchers' would occur less than 1 time in 20, and what this does not establish.