Bot Protection Policies

What Are Bot Protection Policies?

Bot Protection Policies are the profiles that detect the automated (bot) traffic in the requests arriving at a vService with AGS enabled with a trigger and apply an action to the matching requests. The trigger runs on every request (Static), is based on the number of requests counted in a certain scope (Requests) or on the number of failed authentications (Auth Failures), or uses the Threat Protection bot score of the vService (Bot Score). The action is rejecting the request (Deny), slowing it down (Tarpit), blocking the source for a duration (Block), limiting the bandwidth (Bandwidth Limit), a CAPTCHA verification (Captcha) or showing a custom content page (Show Content).

The policies are selected from the Bot Protection Policies list in the AGS settings of the vService; more than one policy works together. Six built-in profiles come with TR7; a new vService comes with Standard Rate Limiting, Bot Score CAPTCHA Challenge and Bot Score Deny by default. The built-in profiles do not appear in the list of this screen, they are selected directly in the selector in the vService settings.

4

Trigger Type

Static · Requests · Auth Failures · Bot Score

6

Action

Deny · Tarpit · Block · Bandwidth Limit · Captcha · Show Content

Scope

Counting Scope

Global · IP Address · IP + User Agent · Composite (a smart variable combination)

6

Built-in Profiles

Static CAPTCHA · Standard/Strict Rate Limiting · Rate Limiting with CAPTCHA · Bot Score CAPTCHA · Bot Score Deny

TP

The Bot Score Source

The Bot Score trigger uses the Threat Protection bot score of the vService

Live

Live Tables

The counting and block tables are monitored with the "Bot: <name>" title on the Bot Protection tab of the vService

How does it work?

1. The policy is defined on this screen: the Trigger Type and its threshold, the Policy Scope for the counting, the optional Policy Condition and the Action to be applied are selected.
2. The policy is added to the Bot Protection Policies list in the vService AGS settings.
3. When the request arrives, the condition (if there is one) and the trigger are evaluated: the Requests and Auth Failures triggers count per scope key in the Trigger Window Duration window, the Bot Score trigger compares the bot score of the request with the threshold, and Static is triggered on every request.
4. The action is applied on the triggered request; in the Block action the source is taken into the block table for the Block Duration, and in the Captcha action the request is directed to the verification of the selected provider.

Built-in Profiles

Profile	Trigger Type	Threshold	Policy Scope	Action
Static TR7 CAPTCHA Protection	Static	-	-	Captcha (Standard TR7 CAPTCHA)
Standard Rate Limiting (the vService default)	Requests	300 requests / 1 min	IP Address	Deny 429
Strict Rate Limiting	Requests	100 requests / 1 min	IP Address	Block 5 min
Rate Limiting with TR7 CAPTCHA	Requests	150 requests / 1 min	IP Address	Captcha (Standard TR7 CAPTCHA)
Bot Score CAPTCHA Challenge (the vService default)	Bot Score	≥ 60	IP Address	Captcha (Standard TR7 CAPTCHA)
Bot Score Deny (the vService default)	Bot Score	≥ 85	IP Address	Deny 403

Since the built-in profiles are not in the list, they are not edited or deleted from this screen.

Operational Best Practices

• Threat Protection for Bot Score. The Bot Score trigger works only while the Threat Protection Bot Protection protection of the vService is enabled (the monitor mode is sufficient); otherwise the policy is not triggered.
• A graduated action. Two policies that use Captcha at a low threshold and Deny or Block at a high threshold (such as the built-in Bot Score profiles) stop the attacker while leaving the legitimate user a chance of verification.
• Shared addresses. The IP Address scope collects the user groups behind NAT in a single counter; for corporate networks the thresholds must be raised accordingly or the IP + User Agent or the Composite scope must be preferred.
• Tarpit carefully. Tarpit holds the request and consumes connection resources; it is suitable only for small volume attacks.
• The counting tables are limited. Every counting policy and every Block action uses a table on the vService; if the table capacity is full, the policy is not taken into the configuration. The number of policies must be kept reasonable.
• The vServices before deleting. The delete confirmation does not show the usage locations; the vServices that use the policy must be checked on the Used in tab in the preview.

How to Add?

A new Bot Protection Policies profile is added through the TR7 web management interface with the following steps:

1. The Configuration Menu is opened by clicking the TR7 logo at the bottom left.
2. The AGS → Protection Policies → Bot Protection Policies → Add steps are followed.
3. A unique and descriptive name is entered for the profile.
4. The Trigger Type, the threshold, the Policy Scope and the Action are selected; the fields specific to the action are filled in.
5. The profile is added by clicking the Add button.

TIP

Reaching the page via quick search

Instead of navigating the menu step by step, the related page is reached directly by using the search bar in the bottom left corner of the screen. The search bar opens with the Alt + T shortcut on Windows and Opt + T on macOS; typing the name of the feature being searched for is then sufficient.

The created profile is used in the Bot Protection Policies list of one or more vServices with AGS enabled. The profile is also defined without leaving the vService settings with the add icon next to this field.

Bot Protection Policies Listing Screen

The list of all Bot Protection Policies profiles that are added on the TR7 device is reached by following the AGS → Protection Policies → Bot Protection Policies path. Profile adding, editing and deleting operations are performed on this screen. The built-in profiles are not listed.

Bot Protection Policies Profiles Listing Form

The table opens with the following columns by default:

Column	Description
Bot Protection Policy Name	The name of the profile.
Trigger Type	The trigger: Static, Requests, Auth Failures or Bot Score.
Policy Scope	The counting scope; since no scope is used in the Static trigger, - is displayed.
Action	The action applied to the triggered request.

There is no column that is hidden by default. The table is not sorted when it opens; when a column header is clicked, first a descending and then an ascending sort is applied, and on the third click the sort is removed.

The items in the toolbar are arranged from left to right in the following order; the last two items (Page Information and Page Size) are located in the information strip below the table.

Columns

Which columns are shown in the table is selected; all four of the columns come on. In the same panel the columns are reordered by dragging and the alignment is selected per column; the row height is fitted to the page with the Fit rows to the page button and the column layout is returned to the default with the Reset button. When the width badge next to a manually widened column is clicked, the column returns to the automatic width.

Filter

The Filter button is used to filter the profiles shown in the list according to certain criteria. There are four cards in the filter strip; the number next to the card name shows how many different values are found for that card in the profiles in the list:

Card	What it filters
Trigger Type	By the trigger type.
Policy Scope	By the counting scope.
Action	By the action.
Used in	The vServices that use the profiles and the Unused Entries value, which lists the profiles that are not used anywhere.

Export

The table is exported as Excel, CSV or PDF by clicking the Export button. In the window that opens, the Filename, Format and Data Source, together with the Columns to be exported, are determined; the remaining fields are displayed according to the format: CSV Separator for CSV, Excel Sheet Name for Excel, and Page Orientation and Page Size for PDF.

Add

By clicking the Add button, the form for creating a new profile opens in the Add tab.

Edit

By clicking the Edit button, the edit form of the selected profile opens in the Edit tab. The button becomes enabled only when a single profile is selected.

The edit form is the same as the add form; all the fields including the trigger and the action are changed and the button at the bottom of the form carries the name Edit. The change is saved without asking for confirmation and the configuration of the vServices that use the profile is applied again.

Delete

When one or more profiles are selected and the Delete button is clicked, the related profiles are deleted. Before deleting, the "Are you sure?" confirmation is asked for.

WARNING

A profile that is in use can also be deleted and the confirmation window does not show the usage locations.

The Bot Protection Policies reference of the vServices that list the profile is not cleared automatically; a policy that cannot be found at runtime is skipped, the reference stays in the list and it must be removed manually from the AGS settings of the vService. The delete operation is written to the user operation record.

Search

Used to search all information in the table.

RegExp Search

Used to search all the information in the table with a regular expression (RegExp). While the usage filter is active, this button is hidden.

Column Based Search

Used to perform a column-based search on the information in the table.

Usage Filter

Used to filter to which vServices the profiles are bound. While the filter is active, a selector takes the place of the search box; when the icon is clicked again, the search box returns.

Page Information

Contains information about the total number of defined profiles and the number of profiles displayed in the list: Showing N-M of T records and the number of selected profiles (N selected, or Nothing selected when there is no selection).

Page Size

The maximum number of profiles to be displayed on the page is selected. On this screen the default is 100; the options are 5, 10, 20, 50, 100, 200, 500 and All.

Right-Click Menu

When a row is right-clicked, a menu with the profile's name in its title opens. Both operations in the menu apply only to the right-clicked profile; a selection made in the list does not affect the menu.

Operation	What it does
Edit	Opens the editing form of the profile in the Edit tab; it is the same as the form opened by the Edit button in the toolbar.
Delete	Deletes the right-clicked profile; the same confirmation question as the deletion in the toolbar is displayed.

Preview of the Selected Profile

When a row in the list is clicked, the preview of the profile opens on the right of the screen; when the selected row is clicked again, the panel closes. The header contains the robot icon, the name of the profile and the Bot Protection Policy subtitle, and the single operation is at the top right.

Preview of the Selected Profile

Operation	What it does
Edit	Opens the editing form of the profile in the Edit tab.

There are three tabs in the panel; Used in is displayed only if the profile is used on at least one vService:

Details

Lists the settings of the profile row by row (the table below).

Log

Lists the user operations on the profile (adding, editing, deleting) as a timeline.

Used in

Lists the vServices that use the profile. The badge next to the tab name shows the usage count.

On the Details tab only the rows belonging to the selected trigger and action are displayed:

Row	Description	Visibility condition
Bot Protection Policy Name	The name of the profile.	Always
Trigger Type	The trigger type.	Always
Policy Condition	The condition tree (read-only).	If a condition is defined
Request Threshold	The request threshold.	Requests
Failed Auth Threshold	The failed verification threshold.	Auth Failures
Bot Score Threshold	The bot score threshold.	Bot Score
Trigger Window Duration	The counting window, with its duration and unit.	Other than Static
Policy Scope	The counting scope.	Other than Static
Scope Components	The component smart variables.	Composite
Action	The action.	Always
Deny Status Code	The HTTP status code and its description.	Deny, Tarpit
Block Status Code	The HTTP status code and its description.	Block
Block Duration	The block duration.	Block
Upload Limit (kbps)	The upload limit.	Bandwidth Limit, if a value is entered
Download Limit (kbps)	The download limit.	Bandwidth Limit, if a value is entered
Captcha Provider	A link to the provider profile; the summary of the profile and the edit form are opened with the icons next to it.	Captcha
Content Page	A link to the Content Pages profile; the summary of the profile and the edit form are opened with the icons next to it.	Show Content
Status Code	The response status code.	Show Content

Profile Add Form

A new Bot Protection Policies profile is added on the TR7 device by following the AGS → Protection Policies → Bot Protection Policies → Add steps. The form opens with the Static trigger and the Deny action selected; when the Trigger Type and the Action are changed, the related fields are displayed. Below, first the common fields, then the own fields of each trigger and finally of each action are explained.

Common Fields

INFO

Bot Protection Policy Name Required

The name of the profile; it identifies the profile in the selection in the vService settings, in the live statistics tables and in the logs. The name must be unique on the device; a second profile with the same name is not saved and the Name is used warning is displayed. When it is left empty, the Required field warning is displayed.

INFO

Trigger Type Required Default: Static

When the policy is triggered is selected:

Option	Meaning
Static	It is triggered on every request; if a condition is defined, only on the requests that match the condition. The details: Static.
Requests	It is triggered when the number of requests per scope key reaches the threshold. The details: Requests.
Auth Failures	It is triggered when the number of failed authentications per scope key reaches the threshold. The details: Auth Failures.
Bot Score	It is triggered when the Threat Protection bot score of the request reaches the threshold. The details: Bot Score.

INFO

Policy Condition Default: Disabled

It is displayed in all the triggers. When the switch is moved to the Enabled position, the condition builder opens; the policy is evaluated only on the requests that match the condition. The conditions are selected from the HTTP conditions defined on the Conditions screen.

Static

The Static trigger does not count; it applies the action on every request (if a condition is defined, only on the requests that match the condition). It is used together with Policy Condition in scenarios such as always showing a CAPTCHA or closing the access for a certain path or client group. In this trigger the threshold, Trigger Window Duration and Policy Scope fields are not present; Policy Condition and Action are located directly under the Trigger Type field.

The Form While Static Is Selected

Requests

The Requests trigger counts the requests arriving per key determined with the Policy Scope (e.g. per IP address) in the Trigger Window Duration window; when the count reaches the Request Threshold value, the action is applied. It is used for rate limiting scenarios.

The Form While Requests Is Selected

INFO

Request Threshold Required

The number of requests allowed within the window; an integer between 1 and 10000 is entered. No default value comes in the form.

INFO

Trigger Window Duration Required Default: 1 minutes

The window in which the counting is performed; a value and a unit (seconds, minutes or hours) are entered, and a duration between 1 second and 24 hours is accepted.

INFO

Policy Scope Required Default: Global

The key for which the counting is performed is selected:

Option	Meaning
Global	All the requests arriving at the vService are counted in a single counter.
IP Address	A separate counter is kept for each source IP address.
IP + User Agent	A separate counter is kept for the combination of the source IP address and the User-Agent header; the different browsers behind the same NAT are separated.
Composite	A separate counter is kept for the combination of the Smart Variables values selected in the Scope Components field.

Scope Components Required (Composite)

Displayed only in the Composite scope. An HTTP request Smart Variables variable is selected on each row (e.g. the source address, a certain header or cookie); at least two rows are present, the rows do not repeat and the counter key consists of the combination of these values.

Auth Failures

The Auth Failures trigger counts the failed authentications the access engine marks per key determined with the Policy Scope in the Trigger Window Duration window; when the count reaches the Failed Auth Threshold value, the action is applied. It is used against password guessing (brute force) and credential stuffing attacks.

The Form While Auth failures Is Selected

INFO

Failed Auth Threshold Required

The number of failed authentications allowed within the window; an integer between 1 and 100 is entered. No default value comes in the form.

INFO

Trigger Window Duration Required Default: 1 minutes

The window in which the counting is performed; a value and a unit (seconds, minutes or hours) are entered, and a duration between 1 second and 24 hours is accepted.

INFO

Policy Scope Required Default: Global

The key for which the counting is performed is selected:

Option	Meaning
Global	All the requests arriving at the vService are counted in a single counter.
IP Address	A separate counter is kept for each source IP address.
IP + User Agent	A separate counter is kept for the combination of the source IP address and the User-Agent header; the different browsers behind the same NAT are separated.
Composite	A separate counter is kept for the combination of the Smart Variables values selected in the Scope Components field.

Scope Components Required (Composite)

Displayed only in the Composite scope. An HTTP request Smart Variables variable is selected on each row (e.g. the source address, a certain header or cookie); at least two rows are present, the rows do not repeat and the counter key consists of the combination of these values.

Bot Score

The Bot Score trigger compares the bot score of the request coming from the Threat Protection bot protection of the vService with the Bot Score Threshold; if the score is equal to or higher than the threshold, the action is applied. The score is between 0 (clean) and 100 (bad bot). This trigger works only while the Threat Protection bot protection is enabled on the vService.

The Form While Bot Score Is Selected

INFO

Bot Score Threshold Required Default: 60

The lowest bot score that triggers the action; an integer between 1 and 100 is entered. The ready buttons next to the field set the threshold to the lower limit of the device-wide bot score bands:

Button	Threshold	Band
Suspicious	40	Suspicious (40–59)
Bot	60	Bot (60–84)
Bad Bot	85	Bad Bot (85–100)

The colored bar below the field shows the place of the selected threshold on the bands; the bar is at the same time a draggable slider and changes the threshold between 0 and 100 one by one. The row below the bar explains which band the threshold falls into and is written with one of three forms according to the position of the threshold within the band:

State	The description row
The threshold is equal to the floor of the band	60 → Bot (60–84) · exactly the band floor · this score and above trigger
The threshold is above the floor of the band	70 → Bot (60–84) · 10 points above the band floor · this score and above trigger
The threshold is in the 1–39 range	20 → Clean (0–39) · the clean-traffic band · a threshold here fires on nearly every visitor

The 0–39 range is the Clean band.

INFO

Trigger Window Duration Required Default: 1 minutes

It is displayed in the form while Bot Score is selected as well and is required to be filled in; since the score comparison is made per request, the counting window is not used in this trigger.

INFO

Policy Scope Required Default: Global

For which key the block table is kept in the Block action is selected:

Option	Meaning
Global	All the requests arriving at the vService are kept in a single record.
IP Address	A separate record is kept for each source IP address.
IP + User Agent	A separate record is kept for the combination of the source IP address and the User-Agent header; the different browsers behind the same NAT are separated.
Composite	A separate record is kept for the combination of the Smart Variables values selected in the Scope Components field.

Scope Components Required (Composite)

Displayed only in the Composite scope. An HTTP request Smart Variables variable is selected on each row (e.g. the source address, a certain header or cookie); at least two rows are present, the rows do not repeat and the record key consists of the combination of these values.

Actions

INFO

Action Required Default: Deny

The operation to be applied to the triggered request is selected; the following fields are displayed according to the selection:

Option	Meaning
Deny	The request is rejected with the Deny Status Code.
Tarpit	The request is held for the connection timeout of the vService and is then rejected with the Deny Status Code.
Block	The source is taken into the block table for the Block Duration; the requests during this duration are rejected with the Block Status Code. As long as the trigger condition is satisfied, every request restarts the block duration from the beginning.
Bandwidth Limit	The upload and download speed of the source is limited.
Captcha	The request is directed to the verification of the selected Captcha Provider.
Show Content	The selected Content Page is returned with the specified Status Code.

Deny Status Code Default: 429

The HTTP status code of the rejection response in the Deny and Tarpit actions; a value between 400 and 599 is entered.

Block Status Code Default: 403

The HTTP status code returned during the block in the Block action; a value between 400 and 599 is entered.

Block Duration Default: 5 minutes

The duration for which the source stays in the block table in the Block action; a value and a unit (seconds, minutes or hours) are entered, and a duration between 30 seconds and 24 hours is accepted.

Upload Limit (kbps)

The upload speed limit of the source in the Bandwidth Limit action; a value between 1 and 1048576 kbps is entered. When it is left empty, the upload direction is not limited.

Download Limit (kbps)

The download speed limit of the source in the Bandwidth Limit action; a value between 1 and 1048576 kbps is entered. When it is left empty, the download direction is not limited. While both fields are empty, the No bandwidth limit given placeholder text is displayed inside the box.

Captcha Provider Required (Captcha)

The verification provider to which the request is directed in the Captcha action; the built-in and defined CAPTCHA Providers profiles are listed.

Content Page Default: Access denied page (English)

The content page to be returned in the Show Content action is selected from the Content Pages profiles; a new content page is defined with the + icon next to it. If the content page is empty, only the status code is returned.

Status Code Default: 403

The HTTP status code returned together with the content page in the Show Content action; a code that carries a body is selected (100, 101, 102, 103, 204 and 304 are not accepted on saving).

INFO

Add

The profile is added by clicking the Add button; the tab closes and the profile is displayed in the list.

Operating Mechanism

INFO

• Distribution. The policies are written into the vService configuration as rules; a counting table according to the scope key is created for the Requests and Auth Failures triggers and additionally a block table for the Block action. There are 14 table slots per vService. The Requests and Auth Failures policies use one slot and the Block action adds one more slot; the Static and Bot Score policies use a slot only in the Block action; if the capacity is full, the policy is skipped entirely with an error record. The Bandwidth Limit action uses its own separate table and does not count a slot; this table keeps the instantaneous upload and download speed per scope key for one hour and the kbps values are converted while they are being applied.
• The triggers. Requests counts the request rate within the window and Auth Failures the failed verifications the access engine marks; Bot Score reads the Threat Protection enabled bot score of the vService (with the trust discounts applied) and is evaluated only if the bot protection is configured; Static matches on every request. If a Policy Condition is defined, only the matching requests are triggered; the counters work independently of the condition: Requests counts all the requests and Auth Failures counts all the failed verifications without looking at the condition.
• The actions. Deny and Tarpit respond with the Deny Status Code; Block takes the source into the block table and returns the Block Status Code during the duration; Bandwidth Limit applies the upload and download speed filters; Captcha shows the verification page by carrying the provider to the access engine; Show Content returns the selected content page with the specified status code.
• The counting key. IP Address is the source address, IP + User Agent is the source address and the User-Agent, and Composite is the combination of the component values; Global uses a single key. The counting table is kept for the window duration in Auth Failures and for ten times the window in Requests. The Auth Failures counting is performed at the response stage, with a plain counter; the window runs from the last failed attempt.
• The statistics. The tables of the counting and blocking policies are monitored with the Bot: <the policy name> entries on the Bot Protection tab of the vService; the provider counters of the policies with the Captcha action are shown on the Live Stats tab.
• Cluster. The profiles are synchronized between the devices with the LB configuration; the counting and block tables are not synchronized, the thresholds are evaluated separately on each device. The tables are preserved when the configuration is applied again; the counters and the active blocks are not deleted.
• Logs. Adding, editing and deleting are monitored on the User Logs screen as the bot protection policy object.

Scope and Limits

INFO

• License. The screen depends on the AGS license; for the details the Products & Licensing page must be consulted.
• Permission. The AGS screens depend on the Load Balancer permission area; for a user who does not have the permission the You don't have access to this page card is displayed instead of the page.
• The name is unique. A second profile carrying the same name is not saved.
• The built-in profiles are not listed, they are not edited or deleted from this screen.
• It is applied only on HTTP vServices.
• The Bot Score trigger is not triggered without the Threat Protection bot protection of the vService.
• The Static and Bot Score triggers do not use a counting table; the Block action requires a block table in every case.
• There is no scope in the Static trigger. Since the Policy Scope field is not displayed, the Block action uses a block table with a single record and the Bandwidth Limit action a single common bucket: when the policy is triggered, the block and the speed limit are applied to all the clients together.
• The scope selector lists options that are not accepted by the record as well. In the selector, next to the four scopes, Username and Device ID are displayed as well; a record cannot be saved with these two values.
• Deleting does not perform a reference check; a profile in use is deleted as well.

Usage Locations

Usage location	Field	Purpose
A vService with AGS enabled → AGS Management	Bot Protection Policies (multiple selection)	The bot policies to be applied to the vService; Standard Rate Limiting, Bot Score CAPTCHA Challenge, Bot Score Deny on a new vService
A vService with AGS enabled → Bot Protection	The Bot: <the policy name> tables	The live monitoring and the clearing of the counting and block tables
CAPTCHA Providers	Captcha Provider (the Captcha action)	The verification provider to which the triggered request is directed