Automated penetration testing tools sit in a sweet spot between vulnerability scanners and manual penetration tests. While nothing can take the place of the expertise and customized approach of a manual
penetration tester, automated pen testing tools excel at finding vulnerabilities that can be exploited through automated scripting, and they can be a worthwhile way to increase the frequency/value of automated checks between manual pen tests.
This piece explains the difference between automated scans and automated pen test tools and recommends ways to get the most from automated pen testing tools.
Automated Scanning vs. Manual Testing
“Penetration testing” has always been a very nebulous category. The industry never provided a clear definition, and the offerings became even more fuzzy with labels such as vulnerability scanning, vulnerability assessments, application
testing, black box, grey box, white box, red teaming, purple teaming, PCI testing, etc. To complicate things further, vendors and consultants define each of these terms differently to address the latest marketing buzz.
What are the differences between automated scanning and manual testing:
- Automated Scanning: involves software-based solutions (scanners) that run various vulnerability checks against targets to determine whether the asset is vulnerable or not based on the specific check. The results are binary: yes = vulnerable; no
= not vulnerable. These results are then cross-checked against industry benchmarks such as the CVSS in an attempt to determine the severity of the vulnerability based on a numerical scoring metric involving various factors (access vector, access complexity,
authentication, confidentiality, integrity and availability). With the predefined scoring, the vulnerability is then categorized as low, medium, high or critical. Automated testing offers simplicity, speed, cost and coverage. However, the main drawback
is that it provides static checks with very little ability to apply context, validate or demonstrate the true risk to the environment (observed exploitation).
- Manual Testing: Relies on the expertise of the testers and their ability to properly apply a methodology, chain together identified vulnerabilities for exploitation and pivot throughout an environment to truly demonstrate an effective attack lifecycle.
This process exposes the true risks of the vulnerabilities within the environment. The disadvantages of manual testing include time, expense, lack of frequency, and lack of full vulnerability discovery and, sometimes, coverage due to scoping restrictions.
READ: How to Use Pen-Test Reports to Improve Security
What is Automated Penetration Testing?
The gaps between automated scanning and manual testing have provided an opportunity for a hybrid offering known as “automated penetration testing.” Manual testing always involves some level of automated scanning to find the low-hanging fruit.
The results of these scans are then used to construct the attack plan for the manual testing. For example, experienced attackers will use the information from the scanner to choose the appropriate exploit, script and proper application.
Automated penetration testing works in a similar manner. It takes the results of vulnerability scans and constructs an applicable automated attack for the vulnerability identified (usually using information from tools like MITRE ATT&CK). Automated pen test tools provide a good way to increase the value of vulnerability scanners and the frequency of scripted penetration testing.
Network/Host Automated Scanning
Automated vulnerability scanning capabilities are foundational to any information security program to identify known vulnerabilities across the environment. Over the past decades, we’ve seen numerous offerings in this space, but the market has leveled
out.
Automated Application Scanning
While the network-and host-level scanners do a great job of addressing network and operating system vulnerabilities, testing applications at Layer 7 of the OSI model requires a different approach. As with network and host testing, application testing
starts with automated scanners, but often also requires manual testing and extensive experience to uncover related risks.
The challenge with web application scanners is there are often discrepancies in the findings based on the logic of the scanner. It is often wise to use multiple web application scanners to ensure optimal coverage.
Automated Penetration Testing Tools
As previously mentioned, “automated” penetration tools enable organizations to fill the gap of additional automation and scripting to exploit the vulnerabilities identified by the scanners without the need for manual expertise. It uses the
skill and expertise of annual pen testers and published frameworks such as MITRE ATT&CK to develop the methodology and automation to validate and demonstrate the risks of the vulnerabilities—without manual intervention.
Tips for Automated Pen Testing
Many experienced security professionals have strong opinions that penetration testing cannot, by definition, ever be fully automated, because anything that is fully automated is just a different type of vulnerability scanning. However, there is the opportunity
to bridge the gap between automated scanners and manual testing. To improve your chances of success with these tools, it’s important to understand:
- The true end goal: An important factor to consider with tools that advertise themselves as “automated penetration testing” is that their goal is not really comprehensive vulnerability detection, but to find vulnerabilities that can
be exploited through automated scripting. However, they can offer value for organizations that want to ensure they invest as little as possible toward addressing non-exploitable vulnerabilities.
- The lack of customization: In essence, you are buying a product with an automated scanner for mass use. It is not on par with manual testing, where you invest in the tester’s expertise and receive custom testing based on your environment.
- The fact that no automated tool can replace a human tester: Automated penetration testing tools have a huge opportunity to increase the frequency of tests that offer greater value than conventional vulnerability scanners. For security organizations
looking to get more from their automated vulnerability scanning, these tools can provide value, given their methodology aligns with expectations and the vendor has mature technologies (vs. simply large advertising budgets). However, “automated”
technologies cannot currently match the expertise and value of manual pen testing. Organizations should still use periodic manual penetration tests for best coverage.
Although reasonable efforts will be made to ensure the completeness and accuracy of the information contained in our blog posts, no liability can be accepted by IANS or our Faculty members for the results of any actions taken by individuals or firms in
connection with such information, opinions, or advice.