An automated scanner that finds all OWASP Top 10 security flaws? Really?

Many security software vendors claim that their web application security scanning tool can identify every vulnerability in the OWASP Top 10. It’s a bold claim – but can it be true?

Marketing activities can lead many organizations, keen to secure their web applications, to believe that some automated web application security testing tools can detect all vulnerabilities and security issues listed in the OWASP Top 10 list. We’re often asked if our Netsparker web application security scanner can do the same.

The short answer is: There is no automated tool that can detect all the security flaws listed in the OWASP Top 10 list. This applies both to dynamic application security testing tools (DAST) and other automated solutions, such as SAST. Statements that claim otherwise can mislead even experienced security professionals and decision-makers. But even though OWASP itself states that any vendor claims of complete Top 10 coverage cannot be true, a simple “no” is not enough to fully answer the question. This article provides a detailed explanation of why no security software can automatically identify all vulnerabilities listed in the OWASP Top 10 list.

What is the OWASP Top 10 list?

For those who are new to web application security testing and vulnerability scanning, OWASP is short for Open Web Application Security Project. OWASP is a non-profit organization that raises web application security awareness and encourages organizations to develop secure applications. Every three years, they publish the OWASP Top 10 list of critical web application security risks. It highlights the most commonly exploited vulnerabilities and security problems found in web applications, web services, and APIs.

The list itself is not a formal standard for web application security, but it does serve as a guideline for organizations, helping them ensure that their web applications are not vulnerable to the most commonly exploited security issues and web application vulnerabilities. There are many other vulnerabilities and security issues that can be found in web applications that are not listed in the OWASP Top 10 list. Ideally, however, all of them should be addressed in time.

Understanding the security flaws listed in the OWASP Top 10:2017

This section outlines the categories and vulnerabilities from the 2017 edition of the OWASP Top 10.

A1: Injection

Injection vulnerabilities, such as SQL injection and OS command injection, are technical vulnerabilities that can be detected by an automated web application security scanner, also known as a web vulnerability scanner.

A2: Broken Authentication

While A1 deals with a specific list of vulnerabilities, A2 refers instead to a number of security issues related to the design of web applications that may result in authentication vulnerabilities.

This is the first category in the OWASP Top 10 that lists a number of security issues that cannot be automatically identified through black-box testing. For example, there is the problem of insecure storage of user credentials. This means that passwords are not hashed but stored in plain text or are only encrypted. It’s also possible that an unsuitable hashing algorithm has been used. Neither an automated web vulnerability scanning tool nor any other automated security tool can determine how user authentication details are stored in a web application backend database. Such checks can only be conducted manually.

However, some of the security issues listed in this category can be identified automatically by web application security scanners. Examples include Session IDs exposed in URLs or the transmission of usernames and passwords over an unencrypted connection.

A3: Sensitive Data Exposure

The criticality of the security issues listed in this category varies depending on an organization’s scope. While some data is considered sensitive by a particular business, it might not be sensitive for other businesses. Therefore, most of the security issues in this category cannot be identified automatically using software testing tools, because it is impossible to cater for all scenarios. Else, test results could be flooded with false positives. Only someone familiar with the scope of the web page or web application is in a position to determine whether some data should be available to a visitor or specific user.

However, there is, of course, data – such as cardholder data, social security numbers, user credentials and similar – that is always considered as sensitive, and automated web application security tools can easily automatically identify that.

A4: XML External Entities (XXE)

Similar to injections A1, XML External Entity security vulnerabilities can be reliably detected using web application scanning. In fact, this type of vulnerability is among the flaws in web applications that can be detected and reported with a very high of certainty (in most cases), given the possibility of issuing outbound requests. It is, therefore, one of the most easily-identifiable vulnerabilities on the list.

A5: Broken Access Control

This category refers to a number of logical security problems in web applications. The fact that they are called logical issues is already an indication that they most likely cannot be detected by automated web vulnerability scanners, as explained in the Identifying Logical Vulnerabilities section of our Getting Started with Web Application Security guide.

In this category, there are a number of security issues, where a sensitive object or resource is not properly protected. For example, a user might gain unauthorized access to information. To prevent this, the application should verify whether an authenticated user has the permission to access it, before granting access. This type of problem will never be identified by an automated web security tool because such tools cannot determine whether a specific user in an organization should have access to a specific resource or not.

Let’s consider an example: web-based accounting software. The accounts clerk should not have access to make money transfers. However, due to a logical flaw, they are able to make transfers. In this scenario, only a person who is familiar with the organization operations and business scope of the web application can determine if such behavior is a flaw.

However, there are a few exceptions where web application scanners can identify authentication problems. For example, if an admin portal is accessible anonymously, an automated tool can warn of the problem, because it can detect a specific pattern in the URL, such as /admin/ or /private/. But a web security tool would not alert you if a logged-in user could see records that they should not be able to see or is able to access restricted sections of a website.

As this example demonstrates, an automated tool cannot differentiate between what two users (such as a Book-keeper versus a Financial Controller) should be able to access. Therefore, these types of security issues or vulnerabilities cannot be identified automatically. They can only be identified manually through penetration testing, and even then (unless the penetration tester is familiar with the scope of the web application they’re auditing) it is very difficult to highlight such problems.

A6: Security Misconfiguration

This category refers to a number of security issues that are the result of a misconfiguration between the server or the software and the framework. Many of these security issues can be identified with an automated web vulnerability scanner. However, most of them still need to be verified by a human during a pen test before being confirmed as security issues, as explained below.

Unnecessary Network Services

You can identify running network services such as FTP, DNS and SMTP on your web server using a port scanner. The scanner will report on open ports, but your security researchers will still need to determine whether the reported network services are needed or not.

Out of Date Software

A web vulnerability scanner will alert you if any of the software being used to run your web application is outdated and, therefore, probably vulnerable. A scanner can also identify the webserver version, the development framework (PHP, .NET) version, and even the version of well-known web applications (WordPress, Drupal).

Security Settings of the Development Framework

A web vulnerability scanner can also notify you of some configuration issues in development frameworks that might leave your web application exposed to malicious attacks. For example, a scanner will alert you if debugging is enabled, or if error messages disclose sensitive information.

Default Accounts and Passwords

By default, web vulnerability scanners launch brute force attacks against login forms using a dictionary to identify weak credentials. This is limited by what you are scanning. For example, if you use weak credentials to access the web server itself, the scanner will never be able to identify them. This is why it is important to audit every single component that makes up your web farm.

A7: Cross-Site Scripting (XSS)

There are many different variants of cross-site scripting vulnerabilities, such as reflected, persistent and DOM XSS. Since all are technical vulnerabilities, they can be detected using an automated web application security scanner. However, it’s worth testing your web application security scanner before using it to detect vulnerabilities, because some of them have shortcomings, especially when it comes to detecting DOM XSS vulnerabilities.

A8: Insecure Deserialization

Due to the nature of this vulnerability, a web application scanner can only identify insecure deserialization in certain cases. These might include pattern matching (detecting a serialized object in a cookie), or probing the application using a serialized object that triggers malicious behavior in a specific library or application. An attacker must have detailed knowledge of the source code of an application to find out how to exploit a deserialization vulnerability.

Web application security scanners can find this vulnerability by sending payloads for specific applications and frameworks. But they can neither exploit nor confirm a vulnerability in a proprietary application that was written from scratch.

A9: Using Components with Known Vulnerabilities

You might think that common sense would prevent software creators from using components with known vulnerabilities. But a surprisingly large amount of IT people still run old, and vulnerable, software.

Web vulnerability scanners run a number of checks to determine whether the web server, database server, and other server components are susceptible to known vulnerabilities. Typically, automated scanners also have a number of checks for well-known web applications and components, such as WordPress, Joomla! and Drupal, for example.

Since it is virtually impossible for a web vulnerability scanner to have a list of all possible vulnerable components and software, it is up to users to ensure that they always use the latest version of any particular software.

A10: Insufficient Logging and Monitoring

The final category of the OWASP Top 10 refers to the insufficient logging and monitoring of user actions. On occasion, developers may not have implemented functionality that records login attempts together with the IP address that was in use. This can help an attacker that is determined to conduct a brute-force attack to find out valid login credentials or log into an account with a stolen password.

If such logs exist and are monitored, either manually or, preferably, automatically, website owners can react quickly. Webmasters with applications that don’t implement this type of mechanism will be unlikely to detect an intrusion in time.

This is another vulnerability that web application scanners cannot detect automatically. It’s impossible to tell whether or not logs are stored by an application. Even when the administrator panel is scanned and no logs are detected, it is still possible that they exist either on the local filesystem or remote servers where they are monitored with dedicated tools.

Identifying all OWASP Top 10 security issues and vulnerabilities in your web applications

As this article explains, many (but not all) vulnerabilities and security flaws in the OWASP Top 10 list can be identified with an automated web application security scanner. To identify every issue in a web application, users should perform a complete web vulnerability assessment, which includes a combination of automated tools and manual security checks and test cases.

It is also crucial that the person conducting the manual audit is familiar with the business scope of the web application. Otherwise, organizations run the risk that not all security issues will be identified, especially vulnerabilities related to business logic.

So the next time someone blithely announces that their solution can automatically identify all the vulnerabilities listed in the OWASP Top 10 list, ask them to send you a report that demonstrates that. Alternatively, save them some time and stress and simply send them a link to this article!