Crawling Performance Node

This document is for:

Invicti Standard, Invicti Enterprise On-Premises, Invicti Enterprise On-Demand

Crawling is a crucial stage in a Invicti scan, enabling vulnerable points in the target web application to be discovered during the Attacking stage.

Invicti clicks every link and button in the web application in order to access every corner of the target web application. It also submits discovered forms and traverses them to access the pages that are typically only accessible once the form is submitted. This is done so that it can populate the link pool, then attack these links to identify vulnerabilities.

The scanner lists these links in the Knowledge Base and provides details on how it found them.

This table lists the sources that are used to determine the number of links.

Source	Description
Start Link	This is the number of links that are entered by the user to initialize the scan. It is basically the target URL.
Text Parser	This is the number of links identified by the text parser while parsing the responses’ source code.
Text Parser Form	This is the number of links identified through HTML forms to which the forms are submitted.
AJAX/XMLHttpRequests	This is the number of links identified as AJAX requests.
Related Link	This is the number of links identified by the scanner through the analysis of other crawled links.
Resource Finder	This is the number of links identified by the Common Files and Directories checks, which look for hidden resources that are not visible to the public.
Unspecified	This is the number of links for which the scanner could not determine the Parsing Source.
ASP.NET Project Importer	This is the number of links identified from the ASP.NET Project (.csproj or .vbproj) file.
Backup Resource	This is the number of links identified by the Backup Modifier, which tries to find backup (.bak, .old) files
Burp Importer	This is the number of links identified from the Burp Saved Items (*.xml).
CSV Importer	This is the number of links identified from comma-separated values.
DOM Parser	This is the number of links identified by the DOM Parser, which parses HTML or XML files.
DOM Parser Extracted Resource	This is the number of links identified by the DOM Parser Extracted Resource, which extracts resources like image and frame.
DOM Parser Navigate	This is the number of links identified by the DOM Parser Navigate, which intercepts navigate calls.
DOM Parser New Window	This is the number of links identified by the DOM Parser, which intercepts new window calls.
.DS_Store Modifier	This is the number of links identified from the .DS_Store file.
Fiddler Importer	This is the number of links identified from the Fiddler Session Archive (*.saz) file.
Form Authentication Sequence	This is the number of links discovered while performing form authentication requests.
HTTP Archive Importer	This is the number of links identified from the HTTP Archive (*.har) file.
HTTP Request Importer	This is the number of links that are identified by parsing sources.
I/O Docs Importer	This is the number of links identified from the I/O Docs (*.json) file.
Link Importer	This is the number of links that are identified using the Link Importer tool.
Mod Negotiation Resource	This is the number of links that are identified from content negotiation provided by the mod_negotiation module.
Invicti Session Importer	This is the number of links that are identified from the Invicti Session (*.nss) file.
OWASP ZAP Importer	This is the number of links that are identified from the OWASP ZAP file.
Postman Importer	This is the number of links that are identified from the Postman file.
Proxy	This is the number of links that are crawled using the proxy (Manual Crawling) feature.
RAML Importer	This is the number of links that are identified from the RESTful API Modeling Language (*.raml) file.
Resource Finder	This is the number of links identified by brute-forcing hidden resources.
Robots.txt Sitemap	This is the number of links that are identified from robots.txt or sitemap.xml files.
SOAP Web Service Parser	This is the number of links that are links identified from SOAP Web Service parser.
Swagger Importer	This is the number of links that are identified from the OpenAPI (formerly Swagger) (.json, .yaml, *.yml) file.
WADL Importer	This is the number of links that are identified from the Web Application Description Language (*.wadl) file.
WordPress Importer	This is the number of links that are identified from the WordPress REST API (*.json) file.
WSDL Importer	This is the number of links that are identified from the Web Services Description Language (*.wsdl) file.

Once the scan is completed, all crawling performance information is listed under the Crawling Performance node in the Knowledge Base, highlighted in red and bold. You can access the same information in the Knowledge Base Report and Knowledge Base Tab.

For further information, see Knowledge Base Nodes.

How to View the Crawling Performance Node in Invicti Enterprise

Log in to Invicti Enterprise.
From the main menu, click Scans, then Recent Scans. The Recent Scans window is displayed.
Next to the relevant website, click Report.
From the Technical Report section, click the Knowledge Base tab.
Click Crawling Performance. The information is displayed in a Crawling Performance tab.

How to View the Crawling Performance Node in Invicti Standard

Open Invicti Standard
Start a Scan or open a previously saved scan.
The Knowledge Base is displayed on the right of the Scan Summary Dashboard. (If it is hidden, display it again using the Knowledge Base icon on the View tab on the ribbon. Alternatively, click the Reset Layout icon on the View tab, then close the Activity/Progress/Logs panes to give maximum viewing space.)

Ensure that the Knowledge Base Viewer is also displayed. (If it is hidden, you can display it again using the Knowledge Base Viewer button on the View tab. You may also want to close the Activity/Progress/Logs panes.)
Click the Crawling Performance node in the Knowledge Base. All detected Crawling Performance information is displayed in the Knowledge Base Viewer.