Red Shell · Common Errors

All mistakes/Section 4: A rubric that separates two runs

4.5A run that does not fail enough of the rubric

Fails the taskHard Client Requirements for Every Task

The mistake

The criteria Model A fails carry under half the rubric's weight, both blocks counted. Everything looks sound; only the arithmetic shows it.

Do this instead

Under 50%, make the scenario harder and run both legs again. Never inflate the weights or pad the set.

Where it happened

One real task. Each shows the spot and what it should have been; click any line to open the task there.

the words at faultthe row or field at faultmissing from the taskwhat is righta task file

Example 20Commerce & Product · 4 Leg A turns

What is wrongModel A fails 13 of 49 criteria, 43 of the 123 total weight with both blocks counted: 35.0%. The floor is half the weight, 62.

Leg A turns 1 to 4 build oven_decision.html, a one page case for a deck oven: its cost, the landlord's allowance, the building's power and the monthly commitments.

Where it shows

Model A fails 13 of 49 criteria, 43 of 123 weight: 35.0%.
The floor
At least half the weight, both blocks counted: 62 of 123.

What it should have been

The scenario
Harder before the run, with sources that disagree, until Model A fails at least half the weight. Never heavier weights or extra rows.

How it breaks the standardThat is 35.0%; the floor needs 62, so the task fails. Nothing else flags it: the rubric is sound and the golden passes it. Weights cannot fix it: three of the four weight 5 criteria are already failed, and objective criteria 1, 26 and 34, image reads Model A passed, belong at 3, not 1.

The rule it breaksGuidelines · Hard Client Requirements for Every Task clearly fails at least 50% of the final rubric score

Catch it in your own task

Divide the weight of the criteria Model A fails by the total, both blocks counted, signs ignored.

Where the correct guidance is written

Section titles as they print, so you can search for them in the document.

Guidelines · Hard Client Requirements for Every Task

Model A has to fail: Enough difficulty that the Model A trajectory clearly fails at least 50% of the final rubric score

Guidelines · 8.5 Rating Against both Models and Justifications

Rate each subjective rubric as Present or Not Present for Model A.

So the 50% counts both blocks.

Why it matters

Every task has to be hard enough that the Model A trajectory clearly fails at least 50% of the rubric score. That is a hard client requirement, not a target. A run that sails through most of the set is not separating a strong model from a weak one, and the data is worth less.

Every example taken apart, with the task, how it breaks the standard and what it should have been: 4.5 A run that does not fail enough of the rubric, in the explanations →