All mistakes/Section 2: What you may not tell Model A or Model B
2.2A Leg A prompt that answers its own attachment
Fails the taskMulti-Turn - Simulator Answer Leak
The mistake
A Leg A prompt that hands Model A a file also states what the file exists to make it read, such as a count or which document is current. The same words in Leg B leak it to Model B too.
Do this instead
State the requirement and leave the value to the file, then run both legs again.
Where it happened
One real task. Each shows the spot and what it should have been; click any line to open the task there.
the words at faultthe row or field at faultmissing from the taskwhat is righta task file
Example 5Property & Space · 4 Leg A turns
What is wrongLeg A turn 2 releases IMG_4392.jpeg and tells Model A the count it exists to supply: "five big ones and one small one". Leg B turn 4 repeats it to Model B.
How it breaks the standardIMG_4392.jpeg shows one large plate by the rulers, a stack behind it and a small one; counting the stack is the read the photo exists to force. Given the count, Model A and Model B only have to measure one plate of each size. A count is a value, so this fails the task.
The rule it breaksQC spec · Multi-Turn - Simulator Answer LeakOne or more user turns supply a value, count, position, or conclusionAny user turn, in either leg. A count is a value.
Read each prompt beside the file it hands over, in both legs: any value the model should read from that file is a leak.
Where the correct guidance is written
Section titles as they print, so you can search for them in the document.
QC spec · Multi-Turn - Simulator Answer Leak
One or more user turns supply a value, count, position, or conclusion (see notes for exceptions) the agent was supposed to derive from the multimodal inputs
Any user turn, in either leg. A count is a value.
Guidelines · 7.1 What Changes from Leg A
The same intent expressed the same way, and the same persona from first turn to last.
Unchanged in Leg B, so the turn that hands Model B the file repeats the value.
Why it matters
The no leak rule covers every turn you type, in Leg A and in Leg B, and QC treats a count as a value: one of its own failing examples is “there are five receipts”. Once the value is in a prompt, the attachment is decoration: no criterion that depends on reading it can separate a run that opened the file from one that did not, for Model A or for Model B.
Example 5A Leg A prompt that answers its own attachmentFails the task
Property & Space4 Leg A turnstask 6a9e4478f55775e9124002e5
From the taskLeg A prompt, turn 2, to Model A
the mistake
Leg A turn 2
Sorted the plates, the photo is fine to use now. There are five big ones and one small one, and they need to be stored flat somewhere in the unit. Add that into the HTML rather than starting over.
From the taskFrom the Leg B conversation, with Model B
You, as the user · Leg B turn 4
Sorted the plates, the photo is fine to use now. There are five big ones and one small one, and they need to be stored flat somewhere in the unit. Add that into the HTML rather than starting over.
From the taskLeg A prompt, turn 1, to Model A
the mistake
Leg A turn 1
Hi, I need a plan for the built-in on the front room wall, the one between the corner and the door. Halvard agreed to build it, but he needs a proper cut list as the wall isn't a perfect square. I measured it and took photos at the floor, middle and ceiling levels + made a sketch, and keep in mind that the photos are the ground truth for the measurements if something's off. I have a voice memo of what plywood he has and a sticky note with his shop rules attached so stick to those. It has to fit my drafting table, my bound notebooks(standing up) + my printing plates. I included a picture of my notebooks to see their sizes from and leave space for two more seasons of notebooks(my calendar has info on how many I do per season). I still have to verify something on the photo of the plates, so skip that for now. Write all up as front_room_builtin.html - the cut list in mm with each part sized to the real wall, how to scribe the edges that meet the wall and ceiling and a cutting diagram for each sheet.
Task fileIMG_4392.jpeg
Look at The photo Leg A turn 2 releases to Model A, and Leg B turn 4 to Model B. Count the plates yourself, including the stack at the back: that count is what both turns gave away.
IMG_4392.jpeginput file
Click the image to see it at full size, and again to fit it back.