Tag: benchmarking

The evaluation chapter is where computer science dissertations are won and lost. Implementation chapters are usually fine — students enjoy writing about what they built. Evaluation is harder, because it requires you to argue against your own work, and a large share of UK undergraduate CS projects submit a chapter that describes the system a second time instead of testing whether it succeeded.

Marking rubrics in UK computing departments almost always weight evaluation and critical analysis heavily — often a quarter or more of the total, and typically more than implementation. A project that works but is not evaluated will land in the low 2:1 band. A project that is partly broken but rigorously evaluated frequently scores higher. This guide gives you the sequence.

Step 1: Work out what kind of evaluation your project needs

Before writing anything, identify which of four evaluation types matches your project. Mixing them without saying so is the fastest route to an incoherent chapter.

1. Performance or benchmark evaluation. For algorithms, data structures, compilers, distributed systems, anything where speed, memory, throughput or accuracy is the point. You compare measured numbers against a baseline.
2. Functional evaluation against requirements. For build-focused software projects. You revisit the requirements you specified earlier and give evidence for each: met, partially met, or not met.
3. User evaluation. For anything with an interface. Real people attempt real tasks, and you measure completion, time, errors and satisfaction.
4. Model evaluation. For machine learning projects. Accuracy, precision, recall, F1, confusion matrices, cross-validation, all measured on data the model has never seen.

Expected output of this step: one sentence naming your evaluation type and why it fits, written at the top of the chapter. For example: “Because the contribution of this project is a user-facing scheduling tool rather than a novel algorithm, evaluation combines functional verification against the requirements in Chapter 3 with a task-based user study of eight participants.”

Most good projects use two types. Two is fine. Four is a project that never decided what it was.

Step 2: Define your success criteria before you report any results

Criteria written after seeing the results are not criteria, they are excuses, and markers spot them instantly. Set thresholds in advance and state them plainly.

Weak: “The system should be fast.”

Strong: “The system should return search results for a 10,000-record dataset in under 200 ms on the reference hardware, a threshold chosen because Nielsen identifies 0.1 s as the limit for the interaction feeling instantaneous.”

Every threshold needs a source. Cite the literature, cite an industry standard, or cite the baseline system you are improving on. A threshold with no justification is a number you invented.

Expected output: a table of criteria with three columns — criterion, target value, and the justification with citation. Place this table before any results.

Step 3: Describe the experimental setup so it can be repeated

Reproducibility carries genuine marks in computing, and it is the cheapest section to write well. Specify:

• Hardware: exact CPU model, RAM, storage type. “A laptop” is not a specification.
• Software versions: operating system, language runtime, every significant library with its version number. Python 3.12.4, PyTorch 2.4.0, PostgreSQL 16.3.
• Dataset: source, size, licence, and how it was split. If you generated synthetic data, explain how and why.
• Repetitions: how many times each measurement was taken. A single timing run is noise, not data. Ten runs reporting mean and standard deviation is a measurement.
• Controls: what you held constant — background processes closed, cold versus warm cache, network conditions.

Two paragraphs here can lift a chapter from adequate to credible, because it shows you understand that a number without conditions attached means nothing.

Step 4: Report results as evidence, not as a diary

The commonest failure in this chapter is chronological narration — “first I tested the login page, then I tested the database” — instead of structured evidence.

Organise results by criterion, not by chronology. For each criterion: state the target, give the measured result, say whether it was met, then interpret it in one or two sentences. Interpretation is the part students skip and markers reward.

Presentation rules that hold across CS projects:

1. Tables for precise values, graphs for trends. Do not present the same numbers twice in both forms.
2. Label every axis with units. An unlabelled axis reads as carelessness in a discipline that is meant to be precise.
3. Report variability. A mean with no standard deviation or confidence interval hides how stable your result is.
4. Never screenshot the terminal. Raw console output pasted as an image is not a result. Extract the numbers into a table.
5. Include the failures. A criterion that was not met, reported honestly and analysed, earns more credit than a suspiciously perfect set of results.

The structural logic here — result first, interpretation second, no new theory — is the same discipline required in any empirical project, and the worked patterns in the guide to writing the results chapter when it is still blank transfer directly to computing data.

Step 5: Run a user evaluation properly, if your project has an interface

If you built something a person uses, you need people to use it. Reporting only that “the system was tested and worked as expected” tells a marker you evaluated nothing.

A defensible undergraduate user study:

1. Recruit five to ten participants. Nielsen’s finding that five users surface roughly 85% of usability problems is the standard justification, and you should cite it rather than picking a number silently. Sample size logic is worth understanding properly, and the reasoning in what sample size an undergraduate dissertation actually needs applies to usability work as much as to survey research.
2. Write task scenarios, not instructions. “You have booked a room for Tuesday and need to move it to Thursday” is a scenario. “Click the reschedule button” tells them the answer and measures nothing.
3. Measure something objective. Task completion rate, time on task, number of errors, requests for help. Satisfaction alone is too soft to carry a chapter.
4. Add a standard instrument. The System Usability Scale gives you ten Likert items and a 0–100 score with published interpretation norms — roughly 68 is average, above 80 is good. Using a validated instrument rather than questions you wrote yourself is a clear quality signal.
5. Report per-task, not just in aggregate. “Seven of eight participants completed Task 3, but the mean time of 94 s against an expected 30 s indicates the filter control is not discoverable” is analysis. A single overall satisfaction figure is not.

Anything involving human participants — even eight coursemates in a lab — needs ethical clearance at essentially every UK university, and clearance must be obtained before you collect a single data point. Retrospective approval does not exist, and unapproved data usually cannot be included. Check the requirements in the guide to ethics approval for an undergraduate dissertation the week you decide to run a study, not the week you want to run it.

Step 6: Use statistics only where they are warranted

Computing students tend to make one of two errors: no statistics at all where comparison demands them, or elaborate significance testing on six data points.

Reasonable practice at undergraduate level:

• Comparing two algorithms across many runs? An independent-samples t-test is appropriate if the assumptions hold; the procedure and reporting format are covered in the walkthrough of running an independent-samples t-test and reporting it properly.
• Comparing performance across three or more configurations? One-way ANOVA, with a post-hoc test if it is significant.
• Eight users, one interface, no comparison group? Descriptive statistics only. Report medians and ranges and stop. Inferential tests on that design are not defensible.
• Classification model? Never accuracy alone on imbalanced data. Give precision, recall, F1 and a confusion matrix, and say which error type matters more for your application.

Always report effect size alongside significance. A statistically significant 3 ms improvement is statistically real and practically irrelevant, and saying so yourself demonstrates exactly the judgement being marked.

Step 7: Write the critical reflection — the highest-value section

This is where firsts are made. After the results, write a section that does four things:

1. State what the evidence supports. Not what you hoped, what the numbers show.
2. Identify threats to validity. Participants recruited from your own course are not representative users. A synthetic dataset may not reflect production distributions. Testing on one machine tells you nothing about other hardware. Name these yourself.
3. Compare against existing work. Your result must sit against something published. “The 180 ms mean query time compares favourably with the 240 ms reported by [author] for a comparable index size, though their dataset was an order of magnitude larger” — that single sentence does more than a page of description.
4. Say what you would do differently. Concrete and technical: a different indexing strategy, a larger participant pool, cross-validation instead of a single hold-out split.

Markers are explicitly looking for a student who can see the limits of their own work. Defending a project as flawless reads as inability to evaluate.

Referencing technical sources correctly

CS dissertations cite things other disciplines do not — conference papers, standards, documentation, repositories — and these are where Harvard bibliographies go wrong. Two worked examples in Harvard style, though you must check your own module handbook because departments vary:

Conference paper:
Vaswani, A., Shazeer, N. and Parmar, N. (2017) ‘Attention is all you need’, Advances in Neural Information Processing Systems 30. Long Beach, CA, 4–9 December. Red Hook, NY: Curran Associates, pp. 5998–6008.

Software or repository:
Pedregosa, F. et al. (2011) scikit-learn: machine learning in Python (Version 1.5.1). Available at: https://scikit-learn.org (Accessed: 14 October 2026).

Note that conference papers cite the proceedings and location, not a journal, and that software citations need a version number and access date. If your bibliography mixes formats or omits access dates, the full rules are set out in the complete UK guide to Harvard referencing for dissertations.

A checklist before you submit the chapter

• Evaluation type named and justified in the opening paragraph
• Success criteria stated with sources, before any results appear
• Hardware, software versions and dataset fully specified
• Repetition count stated; variability reported alongside every mean
• Results organised by criterion rather than chronologically
• Every figure axis labelled with units
• At least one criterion honestly reported as unmet or partially met
• Ethics approval referenced if human participants were involved
• Comparison against at least one published result
• Threats to validity named explicitly
• No new implementation detail introduced in this chapter

Draft it while the numbers are in front of you

The evaluation chapter is easiest to write in the week you run your tests and hardest to write a month later, when you can no longer remember which library version produced which figure. If you have results but no prose, Tesify can turn your criteria table, measurements and study notes into a structured evaluation chapter draft with the setup, results and critical reflection sections already in place. The measurements and the judgement remain yours; what disappears is the blank page.

Frequently asked questions