← Insights
Note 005 3 min read 632 words

What a retest proves that a report cannot

A penetration test report describes a moment. The retest is the only part of the engagement that shows the fix actually holds.

offensivepentest

A penetration test report is a snapshot. It says what we could do to your environment on the days we tested it. The day after delivery it starts going stale, and the most important question it raises is one it cannot answer: did the fixes work?

That question is what the retest is for, and it is the part of an engagement most often cut when budgets get tight.

A closed ticket is not a closed finding

When a finding comes back marked fixed, it usually means someone changed something. That is different from the attack path being gone. In our experience the gap shows up in a few predictable ways:

  • The fix addressed the example, not the class. We reported one injectable parameter. The developer sanitized that parameter. Three others in the same handler still pass input straight to the query.
  • The fix landed in one place. The vulnerable configuration was corrected on the host we used, but the same image or template built six other hosts that never got the change.
  • The fix moved the problem. A broad permission was removed from a service account, and a teammate restored access by granting it to a group the account belongs to.
  • The compensating control was never tested. A web application firewall rule was added in front of the issue. Nobody checked whether a trivial encoding change gets around it.

None of these are negligence. They are what happens when the person fixing an issue has a ticket, a deadline and no attacker’s view of the system.

What we do on a retest

A retest is not a rescan. We take each finding and attack it again the way we did originally, then we try the variations an adversary would try next:

  1. Reproduce the original exploit exactly, using the steps from the report.
  2. Try the obvious neighbours: sibling parameters, other hosts built from the same source, the same permission reached by a different path.
  3. Check that the fix lives in the source of truth (the template, the image, the policy as code) and not only on the box we touched.
  4. Record evidence for each result, closed or still open, so your assessor can read it without taking our word for it.

The output is a short letter per finding: what we tried, what happened, and whether the attack path is closed. That letter is what you hand to an authorizing official or a program office when they ask whether the high findings were remediated.

Plan the retest before the test starts

The retest works best when it is scoped at the beginning, not bargained for at the end. We put it in the original statement of work with a window, usually 30 to 60 days after the report, so remediation has a deadline and the people fixing issues know someone will check.

If you are buying a test, ask the vendor three questions:

  • Is a retest included, and how long is the window?
  • Will the same engineers who found the issue verify the fix?
  • Will the retest results be written up separately, finding by finding?

If the answer to any of those is no, you are buying a description of your problems rather than a way to close them.

Where this fits with your own testing

Your team should still verify their own fixes before we come back. Internal checks catch the easy misses and keep the retest focused on the hard ones. What an independent retest adds is the attacker’s habit of asking “what else did this fix not cover?” That habit is hard to keep when you also wrote the fix.

The engagement ends when our exploits stop working. Not when the report is delivered.

End of note 005 · 5 of 6