Application and Service Discovery

• As soon as you register with Invicti Enterprise, the system begins the discovery process with your commercial email, immediately suggesting websites that might also belong to you.
• Once you start adding websites, the system makes new suggestions based on those websites.
• Invicti analyzes your configuration and data, then suggests additional websites that may also belong to you.

This topic explains how Invicti Enterprise discovers web assets and services.

To manage the Discovery Service in Invicti Enterprise, see Managing Discovery Service in Invicti Enterprise. In order to create websites via the discovery service, see Creating Websites via Discovery Service.

How the Discovery Service Works

There are three main resources that Invicti uses to discover your web assets.

• Your Email’s Domain
• Second-Level Domain of Existing Websites
• Knowledge Base

Email Domain

As soon as you register with Invicti Enterprise, the system begins the discovery process with your email address, immediately suggesting websites that might also belong to you.

The service takes the domain name (e.g. invicti from user@invicti.com) and starts querying. At the same time, the service queries the IP address of discovered websites. For example, the service queries the IP address of Invicti and lists the results in the discovered websites section.

Second Level Domain

While the domain name of your email address is used to query the discovery service, Invicti further uses this domain address to look for additional websites. For example, when the service discovers invicti.com from your email address, it also starts looking for websites, such as api.invicti.com and test.invicti.com. Then, it lists these websites under the Discovered Websites.

Links in the Knowledge Base

In addition to the second level domain, you can utilize the Out of Scope Links Node in the Knowledge Base. The node lists those links that are excluded from a scan in the Scan Scope settings.

Although Invicti aims to crawl every part of the target web application to identify vulnerabilities, the scanner still allows you to determine the scope of the scan. Even if you do, Invicti lists these websites to inform you which links remain uncrawled.

You can use these links and feed the discovery service so that Invicti tries to find additional websites. Once you do that, you will be able to see additional websites in the discovered websites.

When you add, for example, netsparker as a second-level domain into Invicti, the discovery service starts querying and populating the discovered websites as the following:

Public Data Sources

Three main sources act as information for Invicti to discover websites that may be related to you. But, which service does Invicti use to inquiry in order to list these websites?

Firstly, the Discovery Service is a separate service that works completely independent from Invicti Enterprise and currently runs here: https://services.invicti.cloud/

Invicti queries the discovery server and lists the results in the application. There is a public source where the Discovery Service collects this data: Certificate Transparency Logs.

Certificate Transparency Logs

This is a registration system in which all certificate authorities have to register every SSL certificate they sign. In this registration system, logs are kept as binary. For example, in the following query, logs of record number 696712242 (which is associated with invicti.com) can be seen: https://ct.googleapis.com/rocketeer/ct/v1/get-entries?start=696712242&end=696712242

The parsed view of this record is as follows: https://crt.sh/?id=1509541883

Discovery Service downloads these logs, parses them and saves them into the database. By doing this, for example, when “www.google.com” is added to Invicti Enterprise, Discovery Service gets the Organization (O) and Subject Common Name values from this website’s SSL certificate and filters websites that match the organization name or subject common name in that SSL certificate from the backend database and shows them as a discovered website in the UI.

For example, these records will be listed under the Discovered Websites section when www.google.com is added from the New Websites page:

Discovery Service FAQ

Question: I have example.com. However, Invicti Discovery Service could not find this domain. Why?

• As specified above, the Discovery Service is a separate service that works completely independent from Invicti Enterprise. Invicti inquiries third-party databases to identify websites that may be related to you.
• Secondly, the Discovery Service does not provide a 100% guarantee that Invicti will discover all of your websites. If only third-party databases have information related to your website, Invicti can discover and list them.
• Also, the Discovery Service can find those websites that are public even if they are not publicly accessible.

Question: In order to utilize the Discovery Service in Invicti On-Premises, which URL/port should I permit?

While using Invicti On-Premises, you should select the Enable Discovery Service under the General Settings.

Also, you should enter https://services.netsparker.cloud to the Discovery Service URL so that Invicti can carry out the query to discover websites.