Red Shell · Common Errors

All mistakes/Section 4: A rubric that separates two runs

4.3Weights that do not separate difficulty from lookup

Costs pointsRubric Criteria - Overall Rubric Quality

The mistake

Weights that ignore difficulty: a cross source derivation weighted like a direct read, or one operation at 5 in one row and 3 in the next.

Do this instead

Weight by axis count, never importance. A lookup at 5, or a chained derivation at 1, is a two level miss.

Where it happened

One real task. Each shows the spot and what it should have been; click any line to open the task there.

the words at faultthe row or field at faultmissing from the taskwhat is righta task file

Example 18Small Business & Personal Docs · 3 Leg A turns

What is wrongObjective criterion 12, the BendFilm fee worked out from a ticket order, weighs 1, like objective criterion 17, a page count read off the inventory. A calculation from one source takes +3.

Leg A turn 1 sets the BendFilm fee at four times the order processing fee of the June Timbers vs Sounders game: $5.00 on the Ticketmaster order, so $20.

Where it shows

In vantol_films_2026_submission_dashboard.html, the BendFilm Festival fee is shown as $20.worked out: the June game's order found, its fee read, times four
In vantol_films_2026_submission_dashboard.html, the RACC project treatment associated with SetInLead_Treatment_Final.pdf is classified as Ready.one read: the treatment PDF's page count
…For BendFilm, it will be four times the order processing fee from that same game…

What it should have been

a hidden field lookup and a calculation: two axes
Criterion 17+1Keep it

How it breaks the standardObjective criterion 12 takes a hidden field and a calculation: find that order, take its order processing fee rather than its $15.00 service fee, and multiply by four. Objective criterion 17 checks one listed page count against a limit. At 1 each, a Model A run that misses the fee loses no more than one that misreads a page count.

The rule it breaksGuidelines · 5.2 Rubric Weights Weight is difficulty, never importance. Priced by four axes (source coordination, reasoning depth, modality, discovery effort): three or more is +5, two is +3, zero or one is +1.

Catch it in your own task

Count the axes each criterion exercises and read off its weight.

Where the correct guidance is written

Section titles as they print, so you can search for them in the document.

Guidelines · 5.2 Rubric Weights

Weight is difficulty, never importance. How hard the criterion is to satisfy, or for a negative, how tempting the failure is.

Priced by four axes (source coordination, reasoning depth, modality, discovery effort): three or more is +5, two is +3, zero or one is +1.

QC spec appendix · Rubric Quality Definitions · Incorrect Weights - Major

Criteria that are objectively incorrectly weighted by two levels, e.g., 1 is selected when 5 is appropriate or vice versa.

A Moderate issue, despite its name.

Why it matters

The scale exists to separate discovery from lookup. When it does not, the final score says nothing about whether the model did the hard part, and a run that nails every lookup while missing every derivation still scores well.

Every example taken apart, with the task, how it breaks the standard and what it should have been: 4.3 Weights that do not separate difficulty from lookup, in the explanations →