Configuring the Discovery Service

The Application and Service Discovery Settings allow you to configure multiple components of the discovery service, which informs the web applications that are dynamically added to your Discovered Websites list. You can also make manual adjustments to refine the search criteria employed by the discovery service or use the Knowledge Base component of your scan reports as a hint to customizing your discovery service settings.

This guide explains how to access and configure the discovery service in Invicti Enterprise.

How to access the Application and Service Discovery Settings

1. Log in to Invicti Enterprise.
2. Select Discovery > Settings from the left-side menu.
3. Select a tab from the following options to access specific settings:

• Match Settings
• Main Domains
• Second Level Domains
• Organizations
• IP Addresses
• Excluded Domains
• Excluded Second Level Domains
• Excluded Top Level Domains
• Excluded Organization Names
• Excluded IP Addresses
• Risk Scoring

4. Adjust settings according to your preference, then click Save & Recrawl at the bottom of the page.

Match Settings

Uncheck the box next to any of the following settings to disable the matching option.

• Email Matching: By default, the discovery service will use the domain name of your email account to suggest websites that might belong to you.

• Website Matching: By default, the discovery service will use the domain name of any assets already listed in the Websites & APIs page as a search term to suggest websites that might belong to you. The discovery service can use a maximum of 32 websites in your website list to build the search query.

• There is no practical limit to the number of results that can be discovered.
• If your list contains more than 32 websites, this feature is transparently disabled to prevent massive queries from overloading the service.

• Reverse IP Lookup: By default, the discovery service will use the IP Addresses of any assets already listed in the Websites & APIs page as a search term to suggest other websites that are known to be hosted on the same web host.

• Organization Name Matching: By default, the discovery service will use the Organization Name in the SSL Certificate of any assets already listed in the Websites & APIs page to search for websites that have the same Organization Name in their SSL Certificate.

• Only show results with IP address available: By default, the discovery service will not show any results that do not have an IP address. In most cases, you can only scan websites that have an actual IP address. With this setting enabled, you reduce the risk of unnecessarily using up a license credit by creating a target website that cannot be scanned.

Manual Adjustments

To refine the search criteria used by the discovery service, select any of the following tabs to enter specific inclusion or exclusion criteria.

• Main Domains: Adding specific main domains helps you discover more relevant apps or websites related to your organization. This option is enabled by default. Main domains can include letters, numbers, hyphens, and extensions like .com or .net. 
• Second Level Domains: Adding second-level domain names amplifies the results generated by the discovery service. For example, if your main company domain name is example.com, but you wish to discover any additional assets that may be using the word alternative in the name, you can configure the discovery service to also search for websites that match this search parameter.
• Organizations: This section allows you to configure additional SSL certificate or copyright organization names to amplify the results generated by the discovery service. For example, if your company has a policy to include the organization name Example Inc in all its SSL Certificates, and you wish to discover any additional assets that may be using an SSL Certificate with this organization name, you can configure the discovery service to also search for websites that match this search parameter by adding them to the free text box.
• IP Addresses: If your web setup uses one or more specific IP Addresses to host multiple web assets, you can expand the scope of the discovery service by including one or more IP Addresses or IP Address ranges so the discovery service can also search for websites that are hosted on web servers on the configured list of IP Addresses.
• Excluded Domains: Here you can specify any domain names that you want the discovery service to ignore. Your search results will exclude each domain you add as well as excluding any related sub-domains. Excluded domains can include letters, numbers, hyphens, and extensions such as .com or net. 
• Excluded Second Level Domains: This option allows you to refine your search to explicitly exclude second-level domain names to reduce any unnecessary results generated by the discovery service. For example, you may have a defunct number of domains, such as revoked.com, revoked.eu, revoked.us, and possibly other combinations with the word revoked in the second-level domain name. You can configure the discovery service to exclude any results that match this search parameter.
• Excluded Top Level Domains: Excluding specific top-level domain names helps reduce any unnecessary results generated by the discovery service. For example, you may want to exclude all results in the .gov, .mil, and .gov.uk top-level domains. To achieve this, you would enter those top-level domains in the free text box to exclude them from your discovery service results.
• Excluded Organization Names: You can refine your search to explicitly exclude organization names to reduce any unnecessary results generated by the discovery service. For example, if your company has sold off a business unit that used SpinOff Inc as the organization name in its SSL Certificates, you can configure the discovery service to exclude any results that match this search parameter.
• Excluded IP Addresses: This option allows you to refine your search to explicitly exclude IP Addresses and reduce any unnecessary results generated by the discovery service. For example, if your company has a web server that only hosts test websites that you do NOT intend to scan, you can configure the discovery service to exclude any results that match this search parameter.
• Risk Scoring: This setting enables or disables the predictive risk scoring functionality for discovered web assets. For more information, refer to our Introduction to Predictive Risk Scoring documentation.

Using the Knowledge Base to customize discovery results

The Scan Report Knowledge Base can be a powerful ally to help you track down web assets that may be missing from your discovery results. Whenever a scan is made, the scan report will list paths that are out of the scope of the scan and were, therefore, not crawled or tested. You can use this list as a source of information to add domains to your discovery service settings.

To access the Knowledge Base in a scan report:

1. Go to Scans > Recent Scans.
2. Click Report on the right-hand side of one of your recent scans.
3. Scroll down to the Technical Report section and select the Knowledge Base tab.  
4. Select Out of Scope Links, then expand the Out of Entered Path and Below Scope section.