AI-Assisted Marking with Human Moderation in Mark

The marking season problem

Every exam season, lecturers carry home piles of handwritten booklets. Marking runs into the night, consistency slips by the hundredth script, and results reach Senate late. Students wait weeks for a mark with no explanation.

AI-enabled marking promises relief, but examiners are right to be cautious. A mark that cannot be explained cannot be defended at an appeal or to an external examiner.

Oxonia Mark is designed around that caution. It does the slow, repetitive reading and leaves every judgement that matters to people. Read the wider case in how to mark thousands of exam scripts without burnout.

Scanning scripts, even without signal

Booklets are photographed on a phone in the exam office. The scanning screen works offline, and uploads resume automatically when the signal returns, so a weak connection in the exam hall does not stop the work.

Each answer is then read from the handwriting and attached to the right question and candidate. Where words cannot be read, Mark marks them as illegible rather than guessing.

Scanning in the exam office, rather than carrying booklets home, also keeps scripts in one place, with every later step recorded against the script.

A rubric agreed before marking starts

Marking starts from the rubric, not from the AI. The examiner sets out the points each question rewards. The rubric is versioned, tried on sample scripts and approved before marking begins, so everyone is marking to the same, fixed standard.

Mark also watches the human markers. Each marker's calibration compares their marks with the first pass and with hidden seed scripts whose marks are already known, question by question and over time. It notes approvals made in under twenty seconds. The point is not to police markers but to spot drift early and hold a short standardisation conversation before it spreads.

Each change to the rubric is saved as a new version, with a snapshot of the rubric, the reason for the change and the name of the person who made it, rather than quietly altering the old one. It is always clear which standard a script was marked against. That matters when an external examiner or an appeal panel asks.

Two passes, and evidence for every mark

Each answer is marked twice, independently. Every mark awarded quotes the words in the student's answer that earned it, with a confidence score. A mark without an evidence quote always goes to a person, however low the university sets its confidence threshold.

The evidence quote is what makes AI-assisted marking defensible. An examiner looking at a mark sees exactly which sentence earned which rubric point, and can agree or overrule in seconds. A student who asks why sees the same thing.

Two independent passes are a simple but powerful check. When they agree, the mark is more likely to be right. When they differ, Mark does not average them or pick one; it sends the script to a person, with the size of the difference shown.

What goes to a person

After the first pass, each script is either ready for approval or sent to the review queue. A script goes to review when any rule fires:

• the two marking passes disagree;
• confidence is below the exam's threshold, or a mark has no evidence quote;
• illegible words touch the evidence for a mark;
• no answer was read but the page is written on;
• the total sits close to a grade boundary or the pass mark;
• the answer appears to address the marker, a possible attempt to influence AI marking, or a possible collusion flag is open;
• the candidate answered more questions than allowed;
• or the script is drawn in a random audit sample.

Scripts that are ready still need a person: approving them, even in bulk, is a human action.

The university decides the confidence threshold, the borderline margin around each grade boundary and the size of the random audit sample for each exam. A cautious faculty can send more scripts to people; a confident one fewer. Either way the rules are written down and applied the same way to every script.

Moderation, external examiners and release

Moderation in Mark samples 10% of scripts by default, and at least ten. It takes risk first: scripts where a person overrode the marks and borderline scripts, then scripts where the two passes disagreed or the marking was unsure. Moderators mark blind, without seeing the original marks, and Mark reports how closely the two agree.

External examiners moderate their sample and sign their report. Any adjustment to a whole cohort is recorded with its reason. Results are computed from the approved marks and the university's grade scale, and nothing is released until a named examiner approves.

For Senate, Mark drafts a short note per course from its own statistics, such as results, the weakest question, moderation agreement, overrides, appeals and integrity flags. The exams officer edits it before it goes into the pack.

What students see, and appeals

Once results are released, each student can see their marked script with feedback for each question, including the evidence behind each mark. That answers most questions before they become complaints.

Requests for a re-mark and appeals are tracked in Mark, with their deadlines, so none is lost in an inbox. Read more in marking handwritten exams to a rubric, with evidence, or see how results reach students in Oxonia Learn.

For lecturers, the biggest change is where their time goes. Instead of reading every answer to every question, they read the answers that genuinely need judgement: the unclear, the borderline and the unusual. That is where an expert examiner adds most, and it is where Mark sends them.

Products in this article

Questions people ask