How To Use Pen-Test Reports to Improve Security

Whether you engage a penetration tester to do an internal network penetration test, external network test, source code review, web application test or perhaps even a red teaming engagement, the tester will provide a report with distinct recommendations designed to be read, absorbed and prioritized for policy, process and operational changes.

This piece explains what to expect from a typical pen-test report and provides a step-by-step plan for using the recommendations outlined in the report to continually reduce risk and improve your security program.

Interpreting Pen-Test Reports

Most pen-test reports begin with an executive section outlining both business and technical risks, followed by specific findings with detailed technical explanations for technical staff to consume. The report should also include a detailed methodology so technical staff can replicate and verify a finding, as needed.

Although there will be some variation among different information security consultants, you will likely have three specific actionable sections in a penetration test report:

1. Key findings for executive consumption
2. Strategic recommendations for executive consumption
3. Technical findings

What is a Pen-Test Report?

It is important not to interpret a penetration-test report as simply a rundown of software vulnerabilities that need to be patched. While it may include a sub-component of such items, it should be expected the penetration tester will actively exploit weak processes, policies and misconfigurations, and resort to software vulnerabilities only when nothing else works.

It is also important to understand penetration-testing activities have a specific scope and time period associated with them. Be sure and take that into account and recognize there will likely be blind spots that may not have been covered, due to scope interpretation or time restrictions.

Pen-Test Postmortem

Consider a postmortem/post engagement discussion after the penetration test is complete. This should occur after all relevant parties have read the report, and at the discretion of the person who commissioned the activity. Considering involving multiple IT support and security personnel.

The postmortem discussion should focus on the penetration tester’s impressions of your overall environment. Experienced testers should have an intuitive feel for how your operation measures up against the many others they’ve tested in the past.

Security and IT stakeholders, such as systems administrators, should take advantage of this opportunity to engage the tester and ask questions to clarify recommendations. If they feel a recommendation is unreasonable or unrealistic, consider asking questions about compensating controls that might be considered. Example questions might include, but should not be limited to:

• What account password policy do you think is realistic for most organizations, and what have you seen with other clients?
• We would like to replicate this specific attack; can you help me understand from your methodology how we can go about doing that?
• How can we better detect password-spraying attacks?

Pen-Test Report Key Findings

The key findings of a report are designed to provide an opportunity for the CISO or specific individual commissioning the testing to discuss with IT management about what the findings mean and how they relate to ongoing organizational risk. It is important to frame this discussion in a context of ongoing information security projects and priorities already identified.

The strategic guidance section, unfortunately, is an area where the reported information may not be as customized to your organization as you would like. A penetration-testing firm is an outside entity that lives within the context of your environment for only a short period of time. The testers cannot know your specific business pressure points and organizational culture.

If reported well, the strategic guidance section will address the key findings in such a way as to indicate what levers executive management can engage to help facilitate remediation of reported findings. The language may be fairly generalized, for example:

• Provide operational priority for applying application patches in a timely manner.
• Provide financial support to implement technology upgrades to specific services.
• Provide financial support to enhance the security team personnel resources to implement or enhance the vulnerability management program.

Prioritize Pen-Test Report Takeaways

Organizations should consider leveraging the postmortem discussion, key findings and strategic guidance to ultimately re-prioritize or introduce new security priority areas to be addressed. Key findings and strategic guidance may also be a driver for information security policy changes and employee training program enhancements.

The classic example of a change spurred by a pen test revolves around strengthening authentication, usually implementing a stronger passphrase policy with accompanying 2FA mechanisms. This sort of change impacts policy, procedure, process and training.

The takeaways might also spur different actions depending on the maturity of the security program:

• Mature information security programs may find the results of a penetration test help reinforce a list of remediation efforts that are already recognized.
• Less mature organizations may be likely to find penetration-testing efforts end up generating a list of items that were not recognized or budgeted for, and in fact, may be a surprise and present a far more difficult executive discussion.

If you find yourself in a situation of shock and surprise, it may be your organization was not ready for the level of penetration testing that occurred. In that case, it may be better to fall back to assessing program maturity and providing recommendations to build the baseline program items first, before coming back for a repeated penetration-testing effort.

Address Technical Findings with a Detailed Methodology

After the information security group and executive discussions are complete, consider the following:

• Produce a short list of high-priority action items to address in the near term. For example, if there are multiple unpatched operation system or application vulnerabilities, consider moving these to the top of the list along with simple misconfiguration corrections with little to no operating impact.
• Create a longer-term, actionable, prioritized risk reduction list. The longer-term list should be factored into budgetary processes for technology and personnel acquisition. The penetration test report is unlikely to be the exclusive driver of longer-term priorities, but rather a contributor document that supports and enhances the organizational risk reduction strategy.
• Consider making the technical findings and detailed methodology available to all technical staff responsible for replicating the findings, patching and remediating the technology, configuration and/or processes.
• Create a checklist – and monitor it. With a checklist, remediation actions taken can be tracked through change management processes by the information security office and marked off the list as completed or enhanced, depending on the issue at hand.

Focus on Continuous Risk Reduction and Improvement

Ultimately, your pen-test results should be used to foster continuous improvement in your defensive security posture. You should understand the reported information reflects an outsider’s lack of familiarity and context, but that it also reveals potential exploitable weaknesses in your IT operation. When you receive this information, it may be a shock to the system, or it may be a reinforcement of what you already know. Regardless, to ensure you gain the maximum value from your penetration-testing engagements:

• Listen to what the penetration tester is saying in discussion. There are similarities across all IT environments, and an experienced tester can spot weaknesses they have encountered before. Furthermore, attackers have the same knowledge and could potentially target those same weaknesses.
• Spend time to absorb the reported information. Gain a clear understanding technically, tactically, procedurally and strategically of what is being communicated.
• Consider a level of openness and transparency. Make sure your operational teams know penetration-testing activities will occur, and that it is not adversarial but rather a bi-directional learning process.
• Take action on the policy, process and technical recommendations wherever possible. Doing so can hopefully result in a stronger, more defendable environment.
• Schedule repeat pen tests. We suggest performing pen-tests on an annual or semi-annual basis to help can validate remediation and adjust to new technology vulnerabilities as they emerge.
• Be prepared for new findings. The penetration-testing field is in a constant state of improvement. As time goes on, you might find testers using new techniques that bring forth new findings. In addition, introduction of any new technology in your environment will bring new security challenges to accommodate in your risk portfolio. It’s important to be aware of these issues and keep the level of engagement high, even as new findings arise.

Although reasonable efforts will be made to ensure the completeness and accuracy of the information contained in our blog posts, no liability can be accepted by IANS or our Faculty members for the results of any actions taken by individuals or firms in connection with such information, opinions, or advice.