Implement validation, retry, and feedback loops for extraction quality
What you should be able to do
- Retry with the specific validation errors appended, not with the same prompt again.
- Tell a retryable failure from one where the information is simply absent from the source.
- Design self-checking extractions that surface their own inconsistencies.
- Record what triggered a finding so dismissal patterns can be analysed.
Exam traps (4)
Retry the extraction when validation fails.
With no new information the retry succeeds only by luck, at full price, and the failures it does clear teach the pipeline nothing.
Send the document, the failed extraction and the specific errors, and ask for correction.
Retry harder when a required value keeps coming back empty.
If the value lives in a document that was never supplied, no number of retries can find it — and a retried model may eventually invent one.
Retries fix format and structure. Absent information is a pipeline problem, not a prompting one.
Validate totals in application code after extraction.
It catches the error but not what caused it, and the model gets no signal it could have used to correct itself.
Have the extraction report both the stated and the calculated value, and flag the discrepancy as data.
Track how many findings developers dismiss.
A dismissal rate is a real health signal and a dead end for diagnosis: it says the output is noisy without saying which construct produces the noise, so there is nothing to act on.
Record the pattern that triggered each finding, so dismissals point at a cause.
Primary sources
- Claude API — tool usechecked 2026-08-15
- Prompt engineering — chain promptschecked 2026-08-15
Know cold
- Validator
- Retry
- Calculated vs stated total
- Structured output
6 practice questions in the bank are tagged to this task statement.
Practise this task in context: open it inside the interactive study guide, which carries the concept cards, the mock quiz and the practice simulation.