Crawling Performance Node
This document is for:
Invicti Standard, Invicti Enterprise On-Premises, Invicti Enterprise On-Demand
Crawling is a crucial stage in a Invicti scan, enabling vulnerable points in the target web application to be discovered during the Attacking stage.
Invicti clicks every link and button in the web application in order to access every corner of the target web application. It also submits discovered forms and traverses them to access the pages that are typically only accessible once the form is submitted. This is done so that it can populate the link pool, then attack these links to identify vulnerabilities.
The scanner lists these links in the Knowledge Base and provides details on how it found them.
This table lists the sources that are used to determine the number of links.
| Source | Description |
| Start Link | This is the number of links that are entered by the user to initialize the scan. It is basically the target URL. |
| Text Parser | This is the number of links identified by the text parser while parsing the responses’ source code. |
| Text Parser Form | This is the number of links identified through HTML forms to which the forms are submitted. |
| AJAX/XMLHttpRequests | This is the number of links identified as AJAX requests. |
| Related Link | This is the number of links identified by the scanner through the analysis of other crawled links. |
| Resource Finder | This is the number of links identified by the Common Files and Directories checks, which look for hidden resources that are not visible to the public. |
| Unspecified | This is the number of links for which the scanner could not determine the Parsing Source. |
| ASP.NET Project Importer | This is the number of links identified from the ASP.NET Project (*.csproj or *.vbproj) file. |
| Backup Resource | This is the number of links identified by the Backup Modifier, which tries to find backup (*.bak, *.old) files |
| Burp Importer | This is the number of links identified from the Burp Saved Items (*.xml). |
| CSV Importer | This is the number of links identified from comma-separated values. |
| DOM Parser | This is the number of links identified by the DOM Parser, which parses HTML or XML files. |
| DOM Parser Extracted Resource | This is the number of links identified by the DOM Parser Extracted Resource, which extracts resources like image and frame. |
| DOM Parser Navigate | This is the number of links identified by the DOM Parser Navigate, which intercepts navigate calls. |
| DOM Parser New Window | This is the number of links identified by the DOM Parser, which intercepts new window calls. |
| .DS_Store Modifier | This is the number of links identified from the .DS_Store file. |
| Fiddler Importer | This is the number of links identified from the Fiddler Session Archive (*.saz) file. |
| Form Authentication Sequence | This is the number of links discovered while performing form authentication requests. |
| HTTP Archive Importer | This is the number of links identified from the HTTP Archive (*.har) file. |
| HTTP Request Importer | This is the number of links that are identified by parsing sources. |
| I/O Docs Importer | This is the number of links identified from the I/O Docs (*.json) file. |
| Link Importer | This is the number of links that are identified using the Link Importer tool. |
| Mod Negotiation Resource | This is the number of links that are identified from content negotiation provided by the mod_negotiation module. |
| Invicti Session Importer | This is the number of links that are identified from the Invicti Session (*.nss) file. |
| OWASP ZAP Importer | This is the number of links that are identified from the OWASP ZAP file. |
| Postman Importer | This is the number of links that are identified from the Postman file. |
| Proxy | This is the number of links that are crawled using the proxy (Manual Crawling) feature. |
| RAML Importer | This is the number of links that are identified from the RESTful API Modeling Language (*.raml) file. |
| Resource Finder | This is the number of links identified by brute-forcing hidden resources. |
| Robots.txt Sitemap | This is the number of links that are identified from robots.txt or sitemap.xml files. |
| SOAP Web Service Parser | This is the number of links that are links identified from SOAP Web Service parser. |
| Swagger Importer | This is the number of links that are identified from the OpenAPI (formerly Swagger) (*.json, *.yaml, *.yml) file. |
| WADL Importer | This is the number of links that are identified from the Web Application Description Language (*.wadl) file. |
| WordPress Importer | This is the number of links that are identified from the WordPress REST API (*.json) file. |
| WSDL Importer | This is the number of links that are identified from the Web Services Description Language (*.wsdl) file. |
Once the scan is completed, all crawling performance information is listed under the Crawling Performance node in the Knowledge Base, highlighted in red and bold. You can access the same information in the Knowledge Base Report and Knowledge Base Tab.
For further information, see Knowledge Base Nodes.
How to View the Crawling Performance Node in Invicti Enterprise
- Log in to Invicti Enterprise.
- From the main menu, click Scans, then Recent Scans. The Recent Scans window is displayed.
- Next to the relevant website, click Report.
- From the Technical Report section, click the Knowledge Base tab.
- Click Crawling Performance. The information is displayed in a Crawling Performance tab.
How to View the Crawling Performance Node in Invicti Standard
- Open Invicti Standard
- Start a Scan or open a previously saved scan.
- The Knowledge Base is displayed on the right of the Scan Summary Dashboard. (If it is hidden, display it again using the Knowledge Base icon on the View tab on the ribbon. Alternatively, click the Reset Layout icon on the View tab, then close the Activity/Progress/Logs panes to give maximum viewing space.)
- Ensure that the Knowledge Base Viewer is also displayed. (If it is hidden, you can display it again using the Knowledge Base Viewer button on the View tab. You may also want to close the Activity/Progress/Logs panes.)
- Click the Crawling Performance node in the Knowledge Base. All detected Crawling Performance information is displayed in the Knowledge Base Viewer.
