Skip to content

F. Verification and review quiz

Read the examples first.

A team's CI run for a new customer portal is green. The product owner has not seen the portal yet, and the manager says, "Tests pass, so mark it done." What does the team still need before handover?

Select all that apply.

Show answer
  • Correct: A recorded acceptance by the product owner against the owner's requirements.. Right. Validation needs a stakeholder acceptance as well as a test result (P54).
  • Correct: An end-to-end test that runs all parts together, such as DNS, certificates and monitoring.. Right. P54 asks for a commissioning test of all parts working together before handover.
  • Incorrect: Nothing more, because a green CI run shows that the portal meets the need.. Wrong. Passing tests is verification. It does not show that the work meets the owner's need (P54).
  • Incorrect: The same unit tests run again in a clean environment.. Wrong. A second verification run adds no validation (P54). A clean environment helps with P57, but it does not replace acceptance.
  • Incorrect: A second look from the author before the portal ships.. Wrong. A review by the author is neither an acceptance by a stakeholder (P54) nor an independent check (P56).
Three services each pass their own tests. On the first joint deployment, the certificate service rejects the new DNS name. Which of the following conclusions are right?

Select all that apply.

Show answer
  • Correct: Parts that pass alone can fail together, so an end-to-end test of the whole system is needed before handover.. Right. This is the failure the commissioning part of P54 is meant to catch.
  • Correct: The joint test should run at the first stage where all the parts exist, not after release.. Right. P55 asks each check to run at the first stage where it can run, and a joint test can run once the parts exist.
  • Incorrect: Passing unit tests shows that every service meets the owner's needs, so this failure is a one-off.. Wrong. Passing alone does not mean passing together (P54).
  • Incorrect: The team should move its integration test to after the release, because the fix costs the same at any stage of the work.. Wrong. The cost of a fix rises at each later stage, so P55 says to find defects earlier.
A type error reaches production and is found by a customer at 2 am. The team's lint and unit tests run only nightly, after the release branch is cut. Which of the following changes fit P55?

Select all that apply.

Show answer
  • Incorrect: Add more monitoring in production to catch such errors sooner.. Wrong. Monitoring finds the defect after release, which is the latest stage. P55 asks for the earliest stage.
  • Correct: Run lint, type checks and unit tests on every pull request before merge.. Right. Each check then runs at the first stage where it can run (P55).
  • Incorrect: Require a second approver on every release.. Wrong. An independent checker is P56, and it is a different control. It does not move any check earlier (P55).
  • Correct: Run the type check in a pre-commit hook on the developer's machine.. Right. A type check can run before the commit, which is earlier than CI (P55).
  • Incorrect: Keep the nightly run, because a fix costs the same at every stage.. Wrong. P55 rests on the cost of a fix rising at each later stage.
A product lead wants to commit twelve people for a quarter to a new initiative. The only document is a one-page proposal that nobody has reviewed. Which of the following actions fit P55?

Select all that apply.

Show answer
  • Incorrect: Start the work and review the business case at the end-of-quarter retrospective with the sponsor.. Wrong. The review comes after the money is spent. P55 asks for plans to be reviewed before people and money are committed.
  • Correct: Review the proposal and the business case before any people or money are committed.. Right. This is the earliest stage at which the defect can be found (P55).
  • Incorrect: Let the author of the proposal approve it, to keep the review quick.. Wrong. Authors do not see their own errors, and P56 asks for a checker who is not the author.
  • Incorrect: Plan an acceptance review for after delivery.. Wrong. An acceptance review is useful (P54), but it is the latest stage. It does not find a plan defect early (P55).
A payments team changes the code that authorizes card transactions. A failure would be costly. Which of the following practices meet P56?

Select all that apply.

Show answer
  • Correct: A second engineer approves every pull request before it is merged.. Right. Every change has a checker who is not the author (P56).
  • Incorrect: The author re-reads the change the next day and merges it.. Wrong. Authors do not see their own errors. A self-check is the lowest level of independence (P56).
  • Correct: An external organization runs a penetration test on the payment flow.. Right. P56 raises independence with the consequence of failure, from self to peer to external organization.
  • Incorrect: The two engineers who designed the change together approve each other's work.. Wrong. Shared assumptions hide shared errors (P56). Co-authors are not independent checkers.
  • Incorrect: A second person must also authorize the deploy command.. Wrong. A second person at the moment of action resembles the two-person rule (P42), which is for critical, irreversible actions. It is not a review of the work (P56).
A company wants its budgets and accounts checked in proportion to the consequence of an error. Which of the following choices fit P56?

Select all that apply.

Show answer
  • Incorrect: The head of a department prepares the budget and approves it after a careful second reading.. Wrong. The preparer is the checker. P56 needs a checker who is not the author.
  • Correct: Finance checks a team lead's budget.. Right. A check by someone other than the author fits a low-consequence budget (P56).
  • Correct: An external auditor checks the company's accounts, and long-serving auditors are rotated.. Right. Independence rises with consequence, up to an external audit, and P56 asks to rotate long-serving external auditors.
  • Incorrect: The finance team of the company is the only checker of the yearly accounts.. Wrong. For a high-consequence record, P56 raises independence beyond the company's own people.
A team must review two changes, a new layout for an internal dashboard and the deletion of ten years of production audit logs. How independent should each checker be?

Select all that apply.

Show answer
  • Correct: Dashboard layout - a peer who did not write it reviews the change.. Right. A low-consequence, easily reversed change needs a checker who is not the author, and a peer is enough (P56).
  • Incorrect: Both changes - the same peer review, because every change gets identical scrutiny.. Wrong. P56 raises the checker's independence with the consequence of failure, so the two changes differ.
  • Correct: Audit-log deletion - a checker from outside the team, with more independence than a peer.. Right. An irreversible, high-consequence change calls for a more independent checker, up to an external one (P56).
  • Incorrect: Dashboard layout - an external organization reviews it.. Wrong. External review for a low-consequence change is more independence than P56 asks for.
  • Incorrect: Audit-log deletion - the author's own check is enough, because the script is short.. Wrong. Short work does not lower the consequence of failure. Authors do not see their own errors (P56).
A release passes in staging and fails in production. Staging runs an image built separately from the release, with a smaller replica count and TLS switched off. Which of the following actions fit P57?

Select all that apply.

Show answer
  • Incorrect: Add more unit tests, since they run earlier than staging.. Wrong. Earlier tests help find defects (P55), but they do not make the staging setup match operation (P57).
  • Correct: Deploy the same image digest to staging and to production.. Right. Testing a different build than the one deployed is the violation P57 names.
  • Incorrect: Have a second engineer sign off the staging result.. Wrong. A second checker (P56) does not remove the gap between the test setup and real operation (P57).
  • Correct: List each remaining difference from production, such as replica count, and justify it.. Right. P57 is met when the differences between the test setup and operation are listed and each is justified.
  • Incorrect: Report the spread over several staging runs.. Wrong. Reporting spread is P58. It does not make the configuration match production (P57).
A load test fails because of a connection-pool bug. The team applies a one-line fix and wants to ship, since the earlier passing runs covered everything else. What does P57 ask?

Select all that apply.

Show answer
  • Incorrect: Ship the fix, because the earlier passing runs still count.. Wrong. P57 asks for the test to run again after every repair.
  • Correct: Run the load test again after the fix, in the same configuration as production.. Right. P57 asks for tests in operating conditions, and again after every repair.
  • Incorrect: Test the fix on a developer laptop, where the bug is easy to reproduce.. Wrong. A laptop differs from real operation. P57 asks for the same configuration and conditions.
  • Incorrect: Ask the author to confirm that the fix works.. Wrong. The author's word is neither a test in operating conditions (P57) nor an independent check (P56).
A team reports that a new cache is 12 percent faster. The figure comes from one run on a laptop and one run on a CI runner. Which of the following changes make the comparison fit P58?

Select all that apply.

Show answer
  • Correct: Run both versions on the same instance type with a fixed load and a warm-up.. Right. P58 asks for measurement under stated, controlled reference conditions.
  • Incorrect: Report only the mean, to keep the report short.. Wrong. A result without its uncertainty is incomplete (P58). The report needs the spread too.
  • Correct: Report the spread over several runs.. Right. P58 asks for the uncertainty with every result, so a reader can see whether 12 percent exceeds the noise.
  • Incorrect: Keep the single runs, because 12 percent is larger than any plausible noise.. Wrong. Without a measured spread that is an assumption. P58 asks for the spread to be reported.
  • Incorrect: Add an independent reviewer of the claim, without changing the runs.. Wrong. A reviewer (P56) cannot make two runs on different machines comparable (P58).
Two directors want to rank teams by lead time. One team measures it from ticket creation, and another from the first commit. Which of the following actions fit P58?

Select all that apply.

Show answer
  • Incorrect: Let each team keep its own definition, since it knows its work best.. Wrong. P58 asks for each indicator to be defined once so that results are comparable.
  • Correct: Define lead time once, with its formula, period, population and data source.. Right. A single definition makes the teams' numbers comparable (P58).
  • Incorrect: Set a lead-time target for each team and compare progress.. Wrong. A target does not make the measures comparable (P58). It also invites gaming (P19).
  • Correct: Show how much lead time normally varies before comparing teams or periods.. Right. P58 asks to show normal variation before comparing teams or periods.
  • Incorrect: Compare only the last two weeks, because they are the freshest data.. Wrong. Two weeks may sit inside normal variation. P58 asks to show that variation first.
Firmware for an infusion pump is safety-critical and regulated. Its traceability report lists three requirements with no test and five tests that link to no requirement. What should the team do?

Select all that apply.

Show answer
  • Correct: Write a test for each of the three untested requirements, or link an existing one.. Right. P59 requires that no requirement is without a test.
  • Incorrect: Cite 97 percent line coverage as proof of complete coverage.. Wrong. Line coverage measures code. P59 asks for coverage of the requirements.
  • Correct: Link each of the five orphan tests to the requirement it verifies, or add the missing requirement.. Right. P59 requires that no test is without a requirement.
  • Incorrect: Link requirements to design only, since tests are optional.. Wrong. P59 links every requirement in both directions to design and to test.
  • Incorrect: Skip traceability, since a passing compliance scan covers it.. Wrong. A compliance scan (P60) does not show that every requirement has a test (P59).
A company has two projects. One is the braking software of a rail vehicle under a safety standard. The other is a plugin for an internal wiki, with no regulator. Which of the following statements about P59 are right?

Select all that apply.

Show answer
  • Correct: The braking software needs links from each requirement to design and to test, in both directions.. Right. The system is safety-critical, so the trigger of P59 holds.
  • Incorrect: The wiki plugin needs the same full traceability, because P59 applies to every system.. Wrong. P59 applies when the system is safety-critical or regulated. Elsewhere it is costly.
  • Incorrect: The braking software may add traceability after certification.. Wrong. The standards require proof of complete coverage, which P59 asks for as part of the work.
  • Correct: The wiki plugin falls outside the trigger of P59, unless it becomes regulated.. Right. P59 is risk-scaled. Its condition is a safety-critical or regulated system.
A Kubernetes cluster passes the CIS Benchmark scan. The team lead says, "We are secure, so no further analysis is needed." Which of the following statements are right?

Select all that apply.

Show answer
  • Incorrect: A passing scan means the cluster is safe, so the other rules can wait.. Wrong. Treating compliance as proof of safety is the violation of P60.
  • Correct: A passing scan meets the minimum, and the other rules still apply.. Right. P60 says to meet the code as the minimum and then apply the other rules.
  • Correct: The scan result is the check that shows the code is met.. Right. A compliance scan that passes is the check for P60.
  • Incorrect: The team may skip the benchmark where its own analysis goes further.. Wrong. The code is the minimum. Better analysis is added on top of it, not instead of it (P60).
  • Incorrect: An independent reviewer can replace the scan.. Wrong. A reviewer (P56) is a different control and does not show that the code is met (P60).
A team builds a new kind of device for which no code or standard exists yet. How does P60 apply?

Select all that apply.

Show answer
  • Correct: The condition of P60 is not met, because no code exists to meet as a minimum.. Right. P60 applies if a code or standard exists, and a novel technology is its stated counterexample.
  • Correct: The other rules still apply to the work.. Right. P60 only says what to do about a code. It does not switch off the other rules.
  • Incorrect: The team picks the nearest code of another technology and treats compliance as proof of safety.. Wrong. That repeats the violation of P60, because compliance is a minimum and never proof of safety.
  • Incorrect: The team waits for a code to appear before it verifies anything.. Wrong. Verification and validation are always required (P54), whether or not a code exists.
A team's backup procedure was written in 2022 and has never been restored. Any on-call engineer may run it during an outage. Which of the following actions fit P61?

Select all that apply.

Show answer
  • Correct: Prove the procedure with a real restore and keep the test record.. Right. P61 asks for the procedure to be proven by a test before it is used.
  • Incorrect: Have a colleague sign the document after re-reading it.. Wrong. A review of the text is a check (P56). It does not prove that the procedure works (P61).
  • Correct: Qualify each performer through a restore drill and keep a qualification record.. Right. P61 asks to qualify each person who performs the procedure.
  • Incorrect: Add a checklist for the restore steps instead of testing the procedure.. Wrong. A checklist (P71) helps a person follow a procedure. It does not prove the procedure (P61).
  • Incorrect: Monitor that the nightly backup job succeeds.. Wrong. A successful job does not show that a restore works. P61 asks for a restore test.
A team looks for places where a proven procedure and a qualified person are required. Which of these results fit P61?

Select all that apply.

Show answer
  • Incorrect: The length of a simple machined bracket, checked afterwards with a caliper.. Wrong. This can be fully inspected afterwards, which is the counterexample to P61.
  • Correct: The quality of a weld inside a pressure vessel wall, which cannot be inspected afterwards.. Right. A defect that inspection cannot find must be prevented by a proven procedure and a qualified person (P61).
  • Incorrect: A typo fix in a README, checked by reading the diff.. Wrong. The whole result can be checked afterwards, so P61 does not apply.
  • Correct: A production database restore, which shows its flaws only when it is needed.. Right. The result cannot be fully checked in advance, so P61 asks for a proven procedure and a qualified person.
A team proves its restore procedure by restoring a backup into a test cluster. The first restore failed on a missing configuration step, which the team then added. The test cluster runs a different database version from production. Which of the following statements are right?

Select all that apply.

Show answer
  • Incorrect: The test record from the failed first run is enough, since the step is now in the document.. Wrong. P57 asks to test again after every repair.
  • Correct: The team should run the restore again after adding the step.. Right. P57 asks for a new test after every repair, and P61 asks for a procedure proven by a test.
  • Incorrect: The version difference needs no note, because a restore is a restore.. Wrong. P57 asks to list the differences between the test setup and operation and justify each one.
  • Correct: The team should list the version difference and justify it or remove it.. Right. This keeps the test close to real operation (P57), so the proof of the procedure counts (P61).
A company hires a platform engineer. Each interviewer asks their own questions about 'culture fit', and the raters decide in a joint discussion. Which of the following changes fit P62?

Select all that apply.

Show answer
  • Correct: Ask every candidate the same job-related questions.. Right. P62 asks for the same questions for each candidate.
  • Incorrect: Let each interviewer probe the background of the candidate in front of them.. Wrong. Different questions per candidate are the unstructured interview that P62 replaces.
  • Correct: Rate the answers on scales anchored to the job requirements, with raters scoring independently first.. Right. P62 asks for anchored scales, and its check is that raters score independently before the discussion.
  • Incorrect: Rank the candidates against each other during the discussion.. Wrong. P62 rates each answer on scales anchored to the job, and raters score independently before any discussion. Ranking during the discussion skips both.
  • Correct: Add a debugging exercise that every candidate does.. Right. P62 adds work samples or job-knowledge tests to the structured interview.
A manager says unstructured interviews show the real person and that structured interviews predict nothing extra. A colleague quotes the evidence behind P62. Which of the following statements are right?

Select all that apply.

Show answer
  • Incorrect: Unstructured interviews predict job performance about as well, and they are more flexible.. Wrong. P62 gives a mean validity of .42 for structured interviews and .19 for unstructured ones.
  • Correct: Structured interviews predict job performance better than other common methods.. Right. This is the reason given for P62.
  • Incorrect: A work sample is redundant once the structured interview is done.. Wrong. P62 asks to add work samples or job-knowledge tests.
  • Correct: P62 covers promotions as well as hiring.. Right. The trigger of P62 is selecting a person for a job or a promotion.
An engineer deploys a service, and the runbook is missing the rollback step. The lead wants to give feedback. Which of the following statements fit P63?

Select all that apply.

Show answer
  • Correct: The runbook is missing the rollback step. Add it and test it.. Right. This names the task and how to improve it (P63).
  • Incorrect: You are careless. Be more careful next time.. Wrong. This judges character, which P63 says to avoid.
  • Incorrect: Your incident count is the highest on the team this quarter.. Wrong. This ranks people. P63 asks for feedback on the task, not a comparison.
  • Correct: At the next review, check whether the runbook now has a tested rollback step.. Right. P63 asks to check afterwards whether performance improved.
  • Incorrect: Thank the engineer for the effort and let the matter drop.. Wrong. Without a named improvement and a later check, P63 is not met.
A team lead keeps written feedback records. Which of the following records meet the check of P63?

Select all that apply.

Show answer
  • Incorrect: Strongest developer on the team, ranked 2 of 8.. Wrong. A ranking compares people. P63 asks the record to name a task and an improvement.
  • Correct: Alert thresholds in the report were stale. Update them by Friday. Re-check at the next review.. Right. It names a task and an improvement, and it plans a check of the effect (P63).
  • Incorrect: Too negative in meetings. Improve your attitude.. Wrong. This targets character, not a task. P63 asks for feedback on the task.
  • Incorrect: Feedback always improves performance, so no follow-up is needed.. Wrong. In over one third of cases feedback decreased performance, so P63 asks to check the effect.