Manual Security Testing vs Automated Scanning?
The art of penetration testing has evolved over the years. What began with testing arrows on armour, has now become testing tools and techniques on systems and applications. Without a doubt, we are still mostly using manually driven techniques, however this can be slow, cumbersome, and subject to the human element which can result in faults and missed opportunities.
Over the last decade or so, tools to aid and automate security testing have rapidly entered the fray and are increasingly taking the burden off some of the more time-intensive tasks in the cyber security sphere, such as scanning, brute-forcing, or even full-fledged attacks commanded with single line commands. Tools such as BurpSuite, Nmap, SQLMap, Metasploit, and Nessus, among many others, have certainly sped up the discovery and exploitation of vulnerabilities, allowing more in-depth testing within often limited test windows.
Looking at the bounty of tools available to us, you may start to wonder why manual testing is required anymore. Here is a quick rundown on some of the benefits and disadvantages of both, and how using both on engagements, big and small, can be greatly beneficial.
Manual Testing – The Old Reliable
Manual testing, simply put, is the act of using little to no automation for tasks. A great example of this would be the manual exploration of a website while data is being captured by BurpSuite, where the tester can manually analyse the headers and requests as its own task later, rather than immediately after every click.
Manual testing also extends to the exploitation stage of an engagement, where the tester may need to utilise very specific commands or customised scripts to achieve the desired result.
While manual testing can be very meticulous, and provide a detailed and deep understanding of the subject of the test, it can be very time-consuming, possibly taking days longer than an automation-driven test. There are some vulnerabilities that just simply can’t be automated entirely, or are very prone to false positives if automated, and therefore will require further investigation, possibly using more time than if done entirely manually from the beginning.
Some examples of vulnerabilities that require manual testing to correctly identify and safely exploit are:
- Social Engineering
- Access Control Violations
- Password Spraying and Credential Stuffing Attacks
- SQL Injection
- Cross-Site Request Forgery (CSRF)
Another advantage of manual testing over automation is the ability to find, and use, newly or not yet discovered zero-day exploits, which can take a significant amount of time to be implemented into commonly used tools.
Automated Testing – The Shiny New Tools
Automated penetration testing is really what is written on the package – it is the process of utilizing automation tools, such as applications, platforms, and scripts, rather than the expertise and efforts of a human tester. It can be significantly cheaper and far more time efficient (which also adds to cost efficiency) than manual testing by one or more human ethical hackers.
Automation tools can perform actions such as content discovery, vulnerability analysis, and brute forcing, in a matter of minutes or seconds, where it could take a manual tester hours or days to get the same results. Automated tools, namely scanning, can be left to run in the background while manual testing is also performed, or set to periodically scan for issues, such as Tenable Nessus keeping an eye on things and providing reports at set intervals or upon request.
When it comes to regular penetration testing, companies factor in cost, and it can be rather expensive to hire human penetration testers for regular tests or as in-house, so it can be more cost-effective to have automated tools do the day-to-day, then infrequently have a human run further tests and analysis.
There is no doubt that automation is the way of the future, and will continuously improve; however, there are many tasks that are best suited to manual testing, either due to the simple inability to automate or due to the hassle of false positives (and false negatives).
Another advantage to automation is consistency, in both its actions and results, and with the reporting at the end. As the scans and processes run are mostly, if not entirely hands-off, there is less room for human error or deviation, and therefore don’t require a highly trained expert to perform the required tasks, which ultimately can save money for the organization. Automation, however, is often unable to fully assess a threat and how it can impact you in context to your application, platform, infrastructure, network, or organisation as a whole, which is something a sufficiently trained human penetration tester can do, and make new actions accordingly. A vulnerability that may be picked up and reported as a low finding by an automation tool, could have much more critical consequences when chained with other low, or even informational, vulnerabilities.
So, what’s better? A manual or automated approach?
Simply put, both manual and automated testing methods have their place, and should always be used in penetration tests of all kinds. The level of detail and effectiveness provided by manual testing is unsurpassed, as well as contextual reporting and risk analysis that simply cannot be provided by even the best automation tools on the market. However, where speed and consistency of tasks are concerned, automation wins without question.
Although both methods can provide you with a satisfactory outcome in terms of vulnerability identification, what is best for your organization will come down to what level of detail and quality your organisation requires, the frequency of the tests, and the cost factor.
Ultimately, a combination of both manual and automated testing is the best way to get the highest quality outcome of a penetration test, with the most efficient use of time and money, to bring you a greater assurance of security and peace of mind that your assets are secure from malicious attack.