Validation, retry & feedback loops
Implement validation, retry, and feedback loops for extraction quality.
This subtopic (4.4) sits in Prompt Engineering & Structured Output (D4) on Anthropic's Claude Certified Architect — Foundations (CCA-F) exam. The bank holds 21 practice questions here — 5 easy, 12 medium, and 4 hard — with 5 free to try, answers and explanations included. 3 of the free questions are below; the rest are in the practice stream.
What the exam tests here
- retry-with-error-feedback — append validation errors to prompt on retry
- semantic validation errors vs schema syntax errors
- detected_pattern field for tracking false positive patterns
- retries ineffective when information absent from source document
Free practice questions: Validation, retry & feedback loops
A CI/CD pipeline uses --output-format json to extract review results. Claude returns malformed JSON on 15% of runs. What is the correct retry strategy?
Show answer & explanation
Correct answer: A. Append the validation error message to the prompt before retrying
Appending validation errors gives Claude specific feedback so it can correct its output on retry. 'Increase the max_tokens limit' addresses length, not malformed structure. 'Switch to plain text' abandons structured output needed by the pipeline. 'Retry without modification' repeats the same failure condition with no corrective signal.
Your CI job invokes Claude Code with --output-format json and --json-schema to extract structured review findings for PR comments. About 4% of runs return JSON that parses but fails schema validation (missing severity enum, wrong file_path type). You want a second attempt to succeed without re-running the whole review. What is the most effective retry design?
Show answer & explanation
Correct answer: A. Send a follow-up prompt containing the original output plus the specific schema validator errors, asking for a corrected JSON document.
Retry-with-error-feedback appends the concrete validator messages to a follow-up turn, letting the model localize and fix the exact violations without redoing the review work. 'Re-invoke the same command with a higher temperature' just randomizes output. 'Tighten the --json-schema definition' is upstream prevention, not retry. 'Add few-shot schema-compliant examples to CLAUDE.md' shifts defaults but does not recover the current failing run.
Your CI/CD pipeline uses --output-format json to extract review findings. After two weeks, you notice certain lint warnings are repeatedly flagged as code quality issues in PR comments despite being intentional suppressions. Which approach best uses a detected_pattern field to reduce these false positives?
Show answer & explanation
Correct answer: B. Log each suppression pattern in detected_pattern and filter extracted findings against it before posting PR comments.
Tracking known suppression patterns in detected_pattern lets the validation layer filter them out before findings reach PR comments — a structured feedback loop. 'Increase the review prompt's temperature' introduces nondeterminism without addressing the root cause. 'Discards any finding mentioning suppression keywords' is brittle keyword matching, not pattern-aware filtering. 'Retry until counts drop' is a retry loop without a principled exit criterion.
2 more free questions on this subtopic in the practice stream, plus 16 in the full bank. Keep practicing →