Tag: statistical power

Key finding: published management research achieves an average response rate of about 52.7% when surveying individuals and 35.7% when surveying organisations, but those figures come from funded studies with institutional backing — an unfunded undergraduate survey distributed by cold email should plan for a single-figure percentage and size its distribution list accordingly.

The gap between those two realities is where business dissertations get into trouble. A student reads a paper reporting 48% response, assumes a list of 200 contacts will yield 96 replies, sends the survey, receives 14, and arrives at the analysis chapter with a sample too small to support the regression the proposal promised.

No national dataset tracks response rates for UK student dissertation surveys specifically. What follows combines the published methodological literature — which is real, cited and dated — with clearly labelled practitioner estimates for the student case, so you can see which is which.

The benchmark figures from published research

Two things follow. First, the headline benchmarks describe surveys with resources behind them: institutional letterhead, follow-up mailings, sometimes incentives, and frequently a pre-existing relationship with respondents. Second, even within that literature, surveying managers rather than employees costs roughly 17 percentage points — a finding that should shape who you decide to survey.

Why the published averages do not transfer to your project

Four differences separate a published management study from an undergraduate dissertation survey.

• Sponsorship. A survey introduced by a professor at a named business school carries authority a student email does not. Sending from your university address rather than a personal one recovers some of this, and costs nothing.
• Follow-up. Published studies typically send two or three reminders. Dillman’s Tailored Design Method treats sequenced contact as central, and reminders routinely add more responses than the original mailing. Student surveys frequently go out once.
• Relationship. Response rates rise sharply when a respondent has some connection to the researcher — an employer, a placement organisation, a professional network.
• Incentives. Prize draws and small payments raise response. Most students have no budget, and any incentive must be declared to your ethics committee before you offer it.

The practical implication is not that you should give up on survey research. It is that you should plan the arithmetic backwards from the sample you need rather than forwards from the list you happen to have.

Working backwards: how big does your distribution list need to be?

Start from the analysis, not the survey. Common undergraduate business analyses and the sample sizes they require:

Now apply attrition. Suppose you need 85 usable responses for a regression with four predictors:

1. Usable responses needed: 85
2. Allow for incomplete and careless responses — typically 10–20% of submissions are unusable — so target roughly 100 submissions.
3. At a 20% response rate (realistic for a warm list — your own workplace, a placement employer, a professional association that has agreed to circulate it): distribute to about 500 people.
4. At a 5% response rate (realistic for cold outreach to strangers): distribute to about 2,000 people.

That final line is the one that changes projects. Very few undergraduates can assemble a genuine list of 2,000 relevant business contacts. Confronting this arithmetic in week three rather than week ten is the difference between adjusting your design and salvaging a broken one. The general logic of sizing a study — and the difference between what is statistically required and what your department actually expects — is set out in the guide to what sample size an undergraduate dissertation needs.

Five design decisions that materially raise response

1. Shorten it. Response falls as length rises. Under ten minutes, and preferably five. Every scale you add costs you respondents, and a smaller sample damages your analysis more than a missing variable does.
2. Send from your university address. Institutional domains carry credibility and are less likely to be filtered as spam.
3. Plan two reminders. Schedule them at roughly one and two weeks, and get the wording approved in your ethics application so you are not blocked later. Reminders frequently deliver as many responses as the initial send.
4. Use a gatekeeper. One email from an HR manager to 200 staff will outperform 200 cold emails from a student by an order of magnitude. Trade body newsletters, alumni networks and LinkedIn groups with an active moderator work on the same principle.
5. Survey employees rather than executives where the question allows. The 2008 meta-analysis puts roughly 17 percentage points between the two. If your research question can be answered at individual level, answer it there.

What to do when the responses do not arrive

If you are three weeks in with 22 responses, you have four options and they are not equally good.

Option 1: extend and broaden the sampling frame. Best if time allows. Note that widening from one sector to several changes your population, and you must say so in your methodology and limitations.

Option 2: change the analysis to match the sample. Perfectly legitimate. A well-executed correlational study with n = 45 beats an underpowered five-predictor regression with n = 38. Report descriptive statistics, correlations and effect sizes with confidence intervals, and state explicitly that the study was underpowered for multivariate analysis.

Option 3: convert to a qualitative or mixed design. Eight to twelve semi-structured interviews can answer many business research questions better than a thin survey. This normally requires an ethics amendment, so speak to your supervisor before switching.

Option 4: use secondary data. Frequently the strongest rescue for business projects, because the datasets are large, free and already cleaned — company financials, ONS business statistics, sector reports. The subject-by-subject list in the guide to free UK data sources for a dissertation is the fastest place to look for a viable replacement.

What you must not do is quietly proceed with the underpowered analysis and hope nobody checks. Markers do check, and an unacknowledged n = 31 regression with six predictors reads as a student who does not understand their own method.

Reporting response rate in your methodology chapter

State four numbers, in this order: how many people the survey was distributed to, how many responses were received, how many were retained after cleaning, and the resulting response rate as a percentage. Then say how that compares with published benchmarks.

A worked example: “The questionnaire was distributed to 480 employees across three retail organisations. 96 responses were received, of which 87 were retained after removing incomplete submissions and two cases failing the attention check, giving an effective response rate of 18.1%. This falls below the 52.7% average reported by Baruch and Holtom (2008) for individual-level organisational research, which is consistent with the absence of institutional sponsorship and incentives in the present study.”

That paragraph does something important: it pre-empts the criticism by naming it. Markers reward a student who identifies the weakness in their own data before the examiner does. The same reflective habit shapes the whole chapter, and the structure expected of it is covered in the guide to writing the methodology chapter of a business or management dissertation.

Non-response bias: the paragraph almost everyone omits

Low response is a problem chiefly because of who does not respond. If dissatisfied employees ignore your engagement survey, your results are biased upward regardless of sample size.

You cannot eliminate this, but you can address it credibly in two ways. Compare your sample’s demographics against the known population — if your organisation is 60% female and your sample is 61%, say so, as it is weak but real evidence of representativeness. And compare early with late respondents, since late respondents are conventionally treated as proxies for non-respondents; if their scores do not differ significantly, that is reportable evidence.

One short paragraph doing this demonstrates methodological awareness that most undergraduate dissertations lack.

Before you analyse: check the data will support the test

A small sample makes assumption violations both more likely and more consequential. Normality tests behave erratically at low n, outliers exert more leverage, and a single careless respondent can move a correlation noticeably. Screen the data before running anything, and if the assumptions do not hold, the options for proceeding honestly are set out in the guide on what to do when your statistical assumptions fail.

One further practical point: run your survey on a platform your university has approved, since data collected on an unapproved consumer tool can fail institutional UK GDPR requirements and occasionally cannot be used at all. The options available to UK students are compared in the review of survey platforms you can actually use for a student dissertation.

Turn the numbers into a written chapter

Once your responses are in, the methodology and results chapters are largely a matter of reporting the figures in the expected order. Tesify can take your distribution figures, response counts and analysis plan and draft the sampling, response rate and data screening sections in the structure markers expect, with citations formatted as you go. The design decisions and the interpretation stay yours — the drafting stops eating the week.

Frequently asked questions