A student submits an exercise on Tuesday. If they get it back on Thursday, they remember what they were thinking and the comment lands. If they get it back a fortnight later, they have moved on, and the comment is archaeology.
That is not a scheduling problem, it is the whole problem. So the question we actually faced was not “which grading tool should we use” but “what would have to be true for feedback to arrive while it still matters”.
What we tried first
Marking by hand. It works beautifully and it does not scale past about thirty students per marker, which is roughly one cohort and one very tired person.
Off-the-shelf autograders next. Most of them can tell a student that a test failed. Very few can tell them why, in the voice of a person who has taught the thing before, and none of the ones we could actually buy handled the specific case we care most about: a student in a place with intermittent electricity submitting scanned handwritten work.
What we ended up building
Poseidon runs student code in containers per language, so a failing test comes back with output a beginner can read rather than a stack trace. It handles scanned PDFs with annotation, because paper is still how a lot of assessment happens here. Marking is rubric-based, which is the part that makes a grade defensible when a student asks why. There is a regrade workflow, because sometimes the student is right. And it is multi-institution, because a club inside a partner university has to be able to run without seeing anybody else’s students.
The part nobody asks about
The most valuable thing it produces is not the grade. It is the record.
Because every submission and every rubric decision is stored, we can answer questions about our own teaching that we previously had to guess at: which exercise is quietly defeating everybody, whether a cohort is falling behind, how long marking is actually taking. Most teaching organisations do not know these things about themselves. It turns out you cannot buy that, because it is not a feature, it is a consequence of owning the pipe.
It is also why every certificate we issue can be checked by a stranger without asking us for permission. That falls out of the same record.