Tag: statistics

Most psychology undergraduates do not have a statistics problem. They have a sequencing problem: they collected the data first and are now trying to reverse-engineer a test that fits it. The test you need is determined by three things you decided long before you opened your dataset — what your hypothesis claims, how you measured your variables, and how your participants were allocated. Work through those in order and the test chooses itself.

This guide gives you that order as six steps. It is written for a UK undergraduate psychology dissertation, where you are typically working with one or two independent variables, a sample recruited through your department, and a marking rubric that cares more about whether your analysis is justified than whether it is clever.

Step 1: Decide whether you are testing a difference or an association

Almost every undergraduate psychology hypothesis is one of two shapes. Either you predict that groups differ on some measure, or you predict that variables move together. This single distinction splits the entire test family tree in half, and it is the first branch used by the decision guides published by university libraries, including the University of Leeds “Find a test” guide, which separates tests of difference from tests of association before considering anything else.

Write your hypothesis as a sentence and look at the verb:

• Difference: “Students who revise with self-testing will recall more items than students who reread.” Two groups, one outcome. You need a test of difference.
• Association: “Higher trait anxiety will be associated with poorer sleep quality.” Two continuous measures on the same people. You need a correlation or regression.
• Prediction: “Trait anxiety and rumination will predict sleep quality.” More than one predictor, one outcome. You need multiple regression.

If you cannot write your hypothesis in one of these shapes, that is the finding — your research question is not yet operationalised, and no test will rescue it. Fix the question before you touch the data.

Step 2: Identify the level of measurement of your outcome variable

The second branch is what kind of thing your dependent variable actually is. The UCLA Office of Advanced Research Computing statistical methods guide organises its whole selection table around exactly this: the number of dependent variables, the nature and number of independent variables, and whether the outcome is interval and normal, ordinal, or categorical.

• Continuous — reaction times in milliseconds, scores on a validated scale, hours of sleep. Opens the parametric family.
• Ordinal — ranked positions, single Likert items, ordered categories such as “never / sometimes / often”. Points you towards non-parametric tests.
• Categorical — diagnosed or not, chose option A or option B. Points you towards chi-square.

One judgement call trips up a large share of psychology dissertations: a single Likert item is ordinal, but a summed or averaged scale built from many items is conventionally treated as continuous. If you are analysing a validated multi-item questionnaire, you are almost certainly in the continuous column. Say so explicitly in your methods chapter and give your reason; markers reward the justification far more than the choice itself.

Step 3: Count your groups and check whether they are independent or repeated

Now describe your design in three numbers: how many independent variables, how many levels each has, and whether the same people appear in more than one level.

• Independent (between-subjects): different people in each condition.
• Repeated (within-subjects): the same people measured more than once.

Getting this wrong is the single most expensive error available to you, because an independent-samples test run on repeated-measures data throws away the very thing that makes a within-subjects design powerful — the pairing. It will usually make a real effect disappear.

Step 4: Check the parametric assumptions before you commit

Parametric tests buy you statistical power in exchange for assumptions. The British Psychological Society’s supplementary guidance on research methods for accredited undergraduate and conversion programmes expects students to be able to detect differences in sample means using tests such as t tests and ANOVA, and relationships between variables using chi-square, correlation and regression — and, importantly, to show “familiarity with robust alternatives when those assumptions are not met (e.g. non parametric alternatives)”. Knowing the alternative is part of the competence being assessed.

For most undergraduate designs you need to check:

1. Distribution of the outcome within each group, judged from a histogram and a normality test together rather than either alone.
2. Homogeneity of variance across groups, usually via Levene’s test.
3. Independence of observations — a design question, not a statistical one. If participants worked in pairs or discussed the task, this assumption is already compromised.
4. Outliers, identified and handled by a rule you state before looking at whether removing them helps your p value.

If assumptions fail, you do not have a disaster. You have a non-parametric equivalent and a sentence to write explaining why you used it.

Step 5: Read the test off the table

With steps 1 to 4 answered, the choice is mechanical.

The right-hand column is not a consolation prize. Reporting a Mann–Whitney because your data were skewed, and saying so, reads as competence. Reporting a t test on visibly skewed data reads as not having looked.

Step 6: Report it so a marker can verify it

A result is only worth the marks if someone can reconstruct it. Report the test, the degrees of freedom, the test statistic, the exact p value and an effect size — the last of these is what turns a significance claim into a meaningful one. Jacob Cohen’s widely used conventions, set out in his 1992 paper “A power primer” in Psychological Bulletin, give benchmarks of .20, .50 and .80 for small, medium and large d, and .10, .30 and .50 for r. Cohen was candid that “the definitions were made subjectively”, describing a medium effect as one “likely to be visible to the naked eye of a careful observer” — so treat them as reference points, not verdicts.

A worked example of how the sentence should look:

Participants in the self-testing condition (M = 24.10, SD = 4.32) recalled significantly more items than those in the rereading condition (M = 19.85, SD = 5.01), t(58) = 3.52, p = .001, d = 0.91.

On referencing style, check your handbook rather than assuming. APA style — currently the seventh edition of the Publication Manual of the American Psychological Association, published in 2020 — is the working convention across UK psychology and is required by BPS journals. But the BPS’s own supplementary guidance for accredited undergraduate programmes states plainly in a footnote that “in respect to referencing of work, alternatives to the APA style are acceptable”. Your department decides. Ask, and follow what it says.

Which software should you actually use?

All three of the packages UK psychology departments commonly point at will run everything in the table above.

• IBM SPSS Statistics is commercial software, currently at version 32, and is the package most UK psychology departments teach; access normally comes through a university site licence rather than a personal purchase.
• jamovi is free and open source — released under the AGPL3, with its analysis package jmv under GPL2+ — and is “powered by the R statistical language”. It runs the standard undergraduate test set through a point-and-click interface very close to SPSS.
• R is a free software environment for statistical computing maintained by the R Core Team, with copyright held by the R Foundation for Statistical Computing. It has the steepest learning curve and the longest payoff.

If your analysis is the standard undergraduate set and you have lost access to a campus licence over the summer, jamovi is the pragmatic answer: it is free, it is legitimate to cite, and its output maps onto what your supervisor expects to see.

Where students actually lose marks

Not on the test. On the justification. A methods chapter that says “a t test was conducted” earns less than one that says “an independent-samples t test was selected because the design compared two independent groups on a continuous outcome, and inspection of histograms and Levene’s test indicated the parametric assumptions were tenable.” Same analysis, visibly different competence.

If you have your design settled and the blank methods chapter is the thing standing between you and a submission, you can start drafting it in Tesify — it works from your own design decisions and your own data, so the argument stays yours. The writing is 100% written by you; what you get is structure and momentum.

Frequently asked questions

Do I need to run a normality test if my sample is large?

Report one, but do not let it decide alone. Normality tests become very sensitive at larger sample sizes and will flag trivial departures as significant. Read the test alongside a histogram and, where relevant, skewness and kurtosis values, and say in your write-up what you looked at.

Can I use a parametric test on Likert data?

On a single Likert item, no — it is ordinal. On a summed or averaged multi-item scale, it is conventional in psychology to treat the total as continuous. State which you have and justify it in one sentence.

What if my assumptions fail and there is no non-parametric equivalent?

This happens most often with factorial designs. Your options are to transform the outcome variable, to use a robust or bootstrapped procedure, or to simplify the design to one your data can support. Whichever you choose, report what failed and why you responded as you did.

Is a non-significant result a failed dissertation?

No. Undergraduate projects are frequently underpowered, and a well-designed study with a null result and an honest discussion of power is a legitimate piece of work. What loses marks is presenting a null result as though the hypothesis had been confirmed, or quietly running additional tests until something reaches significance.

How many statistical tests should a psychology dissertation contain?

As many as your hypotheses require, and no more. Each hypothesis should map to one planned analysis. Running a large number of unplanned comparisons inflates your false-positive rate, and markers can see it in your results section.

Should I report effect sizes even when the result is not significant?

Yes. An effect size with a confidence interval tells the reader how precise your estimate was, which is exactly the information a null result needs in order to be interpretable.