Exclude paths from scanning

There are situations where you may need to configure Invicti Platform to exclude a portion of a web application from crawling and scanning. This might be required if the web application being scanned is too large, or if scanning part of the site might trigger unwanted actions such as submitting data. For more information on crawling options, refer to the crawling options overview document.

This document explains how you can specify paths for exclusion based on regular expressions. Excluded paths are added to individual targets on the Scan configure target page. 

Add an excluded path

The Excluded paths option allows you to specify a list of directories and files to be excluded from crawling and scanning. Multiple paths can be excluded for each target.

1. Select Inventory > Targets from the left-side menu.
2. Click the ⋮ > Edit target by the selected target to access its settings page.
3. Click Crawling Options from the settings menu and scroll Excluded paths section.
4. In the Excluded Paths field, enter a RegEx for the path you want to exclude from scanning. Refer to the information below these instructions to learn about formatting requirements for excluded paths.
5. Click Save target configuration when you are finished.

Excluded paths formatting requirements

Excluded paths need to be configured using regular expressions (RegEx). This is useful in situations where you want to exclude a URL pattern rather than a single URL. Invicti Platform accepts the widely used Perl Compatible Regular Expressions (PCRE) syntax for defining RegEx.

The format for creating exclusions is with a forward slash at the front (/) followed by the path that should be after the target URL. Once a path is excluded from scanning, all its subdirectories will also be excluded from the scan because once a directory is not crawled, the scanner cannot know that there is anything below that directory that has been ignored.

Example

• Target URL = www.example.com
• Directory to exclude = /dir2 which is in directory /dir1 (www.example.com/dir1/dir2)
• Excluded path = /dir1/dir2 where /dir2 will be ignored by the scanner. Note that /dir1 and everything in it (except /dir2) will still be scanned.
• RegEx = /dir1/dir2(/.*)?$

The table below provides examples of regular expressions you can configure in Invicti to restrict URL patterns.