Assessing code

Why it matters

Assessment in CS has two jobs: telling students how they are doing so they can improve (formative), and recording what they have achieved (summative). Programming work is easy to check for "does it run?" and hard to check for "does the student understand it?". Good CS teachers combine automatic testing, human judgement and well-designed tasks to get a fair picture without drowning in marking.

Autograding: what it is good at

An autograder runs student code against test cases and reports what passed. Tools include GitHub Classroom, Gradescope, CodeHS, Runestone and many learning platforms; you can also write a simple one yourself (the Do tab).

Autograders are good at:

• Correctness on specified inputs: does average(2, 4) return 3?
• Fast feedback: students can test, fix and resubmit many times.
• Consistency: every submission is checked the same way.
• Scale: 150 submissions take seconds.

What autograders miss

• Design and readability: names, structure, comments, duplication.
• Understanding: a student may have copied or generated code they cannot explain.
• Partial progress: a program that fails one test may show more learning than one that passes by accident.
• Edge cases you did not think of: tests only check what you wrote.

So combine autograded tests with a short rubric for quality and process, and with ways to check understanding (see below).

Writing good test cases

• Typical cases: the ordinary inputs (average(2, 4)).
• Boundary cases: zero, empty lists, one item, the largest allowed value.
• Tricky cases linked to misconceptions: integer vs float results, off-by-one ranges, negative numbers.
• Clear names: "handles an empty list" tells a student more than "test 4".
• Hidden and visible tests: show some tests so students can work toward them, keep a few hidden to discourage hard-coding answers.

Worked example: feedback that teaches

A student's function:

def average(a, b):
    return a + b / 2

A bare autograder says: 1/3 tests passed. A better one says:

1 of 3 tests passed. First failure: inputs (2, 4), expected 3, got 4.0

That message points at the bug (operator precedence: b / 2 happens first) without giving away the fix. You can add hint messages to specific tests: "Check the order of operations: does division happen before addition?".

Checking understanding

Because code can be copied, shared or generated by AI tools, many teachers add:

• Code explanations: a short written or spoken explanation of how their code works.
• Modify-on-the-spot tasks: "change your function so it averages three numbers" in class.
• Code reading questions on tests: tracing, predicting output, finding a bug.
• Process evidence: commit history, test logs, planning notes.

Be clear about your policy on collaboration and AI tools, and design tasks where the learning is visible.

Grading fairly

• Separate correctness, quality and process in your rubric.
• Allow resubmission for formative work: the goal is learning, not one shot.
• Check that your tests are correct and fair before releasing them (run them on your own solution and on a deliberately buggy one).
• Watch for bias: do students who had coding at home score higher on "polish" criteria that you never taught?

Common mistakes

• Only autograding. Students learn to satisfy tests, not to write good code.
• Only hidden tests. Students cannot tell what is expected.
• Unhelpful messages. "Test 3 failed" teaches nothing.
• Tests that crash on student errors. A good grader catches exceptions and reports them.
• Grading what you did not teach. If style counts, teach and model it first.

Quick recap

• Autograders give fast, consistent feedback on correctness; they miss design and understanding.
• Write typical, boundary and misconception-based tests with clear names and useful messages.
• Combine tests with a quality rubric and checks of understanding.
• Allow resubmission, verify your tests, and be clear about collaboration and AI policies.

Try it

Open the Do tab and write a small autograder that runs test cases and produces student feedback.