Metric Persistence Filter

The metric persistence filter, configurable in the Metric Editor, can be used to discard incoming series commands according to a filter expression. Commands for which the expression returns false are not stored in the database.

Expression Syntax

The filter is a boolean condition that can include fields, operators, and functions.

Fields

Name Type Description Example
entity string Entity name. entity LIKE "?tsd"
value number Numeric value. value > 0
message string Text value (annotation). message = "a"
timestamp number series command timestamp. timestamp < 1522683114614
tags.{name} or tags['name'] string Value of command tag with name name.
Tag names are case-insensitive.
tags.location NOT IN ('a', 'b', 'c')
tags['fs'] LIKE "ext*".

Operators

Comparison operators: =, ==, !=, LIKE.

Logical operators: AND, OR, NOT as well as && , ||, !

Wildcards

  • Wildcard * means zero or more characters.
  • Wildcard ? means any character.

Functions

The following built-in functions can be used in the filter expression:

Filter expression can also include Math functions.

Math.sin(Math.toRadians(value)) < 0.5

collection

collection(string s) [string]

Returns an array of strings which have been loaded with the specified string s.

Named collections are listed on the Data > Named Collections page.

To check the size of the collection, use the .size() method.

To access the n-th element in the collection, use square brackets as in [index] or the get(index) method (starting with 0 for the first element).

entity = collection('hosts')[0]

tags.request_ip = collection('ip_white_list').get(1)

list

list(string s[, string p]) [string]

Splits string s using separator p (default is comma ',') into a collection of string values. The function discards duplicate items by preserving only the first occurrence of each element.

To access the n-th element in the collection, use square brackets as in [index] or the get(index) method (starting with 0 for the first element).

Examples:

entity = list('atsd,nurswgvml007').get(0)

likeAll

likeAll(object s, [string] c) boolean

Returns true, if the first argument s matches every element in the collection of patterns c. The collection c can be specified inline as an array of strings or reference a named collection.

Examples:

likeAll(tags.request_ip, ['192.0.*', '192.0.2.?'])

likeAny

likeAny(object s, [string] c) boolean

Returns true, if the first argument s matches at least one element in the collection of patterns c. The collection c can be specified inline as an array of strings or reference a named collection.

Examples:

likeAny(tags.os, ['Ubuntu*', 'CentOS*'])

likeAny(tags.request_ip, collection('ip_white_list'))

matches

matches(string p, [string] c) boolean

Returns true if one of the elements in collection c matches (satisfies) the specified pattern p. The collection c can be specified inline as an array of strings or reference a named collection.

The pattern supports wildcard characters asterisk (*) and question mark (?).

Example:

matches(entity, collection('hosts'))

matches(message, ['OK', 'stable'])

startsWithAny

startsWithAny(object s, [string] c) boolean

Returns true, if the first argument s starts with any of strings from collection c. Collection c can be specified inline as an array of strings or reference a named collection.

Examples:

startsWithAny(entity, ['a', 'nur'])

contains

[string].contains(string s) boolean

Returns true if s is contained in the collection. The collection c can be specified inline as an array of strings or reference a named collection.

Example:

collection('ip_white_list').contains(tags.request_ip)

collection_contains

collection_contains(object v, [] c) boolean

Returns true, if collection c contains object v. The collection c can be specified inline as an array of strings or reference a named collection.

Examples:

NOT collection_contains(tags['os'], collection('ignore_os'))

collection_intersects

collection_intersects([] f, [] s) boolean

Returns true, if collection f has elements in common with collection s. The collections can be specified inline as an arrays of strings or reference a named collections.

Examples:

collection_intersects(tags.values(), collection('ip_white_list'))

upper

upper(string s) string

Converts s to uppercase letters.

lower

lower(string s) string

Converts s to lowercase letters.

size

[].size() integer

Returns the number of elements in the collection.

Note

The function can be applied to collections of any type (string, number) as well as to maps.

Examples:

tags.size() > 1

isEmpty

[].isEmpty() boolean

Returns true if the number of elements in the collection is zero.

Note

The function can be applied to collections of any type (string, number) as well as maps.

Example:

NOT tags.isEmpty()

IN

string s IN (string a[, string b[...]]) boolean

Returns true if s is contained in the collection of strings enclosed in round brackets.

Examples:

entity IN ('nurswgvml007', 'nurswgvml008')
tags.location IN ('NUR', 'SVL')

Using Persistence Filter

To reduce disk space utilization, open the Settings > Receive Statistics page to view metrics with the highest number of inserted commands.

The Series icon opens a list of unique tags for the selected metric.

If some of the incoming data is of low value it can be ignored when received.

To stop storing such series, create a collection with filter patterns on the Data > Named Collections page.

Open metric editor and enter a filter expression to ignore matching series.

!likeAny(tags.command, collection('ignore-collector-process-commands'))

As a result, the number of stored series is reduced.