Derived Commands

Overview

Derived command actions store new calculated metrics in the database by creating and processing custom commands defined using the Network API syntax.

Command Template

When configuring a command action, you need to specify a template consisting of command name, command fields and command values.

Supported commands

Fields

The template can include plain text and placeholders.

series e:${entity} m:jvm_memory_free_avg_percent=${round(100 - avg(), 3)}

The calculated metrics can reference other metrics using db_last and db_statistic functions.

series e:${entity} m:jvm_memory_used_bytes=${value * db_last('jvm_memory_total_bytes') / 100.0}

Tags

A special placeholder ${commandTags} is provided to print out all window tags in the Network API syntax. It allows appending all tags to the command without knowing the tag names in advance.

series e:${entity} m:disk_free=${100 - value} ${commandTags}

Assuming the incoming command was series e:test m:disk_used=25 t:mount_point=/ t:file_system=sda:

series e:test m:disk_free=75 t:mount_point=/ t:file_system=sda

Time

Current Server Time

To store derived commands with the current server time, omit the date/time fields (ms, s, d) from the derived command.

series e:${entity} m:disk_free=${100 - value} ${commandTags}

Alternatively, use the now placeholder to access the current server time.

series e:${entity} m:disk_free=${100 - value} ${commandTags} ms:${now.getMillis()}

To store commands with seconds precision, round the current time using the floor function and the seconds field s::

series e:${entity} m:disk_free=${100 - value} ${commandTags} s:${floor(now.getMillis()/1000)}

Received Time

To store derived commands with exactly the same time as the incoming command, set the millisecond field ms: to the timestamp field. The timestamp field represents the timestamp of the command that caused the window status event.

series e:${entity} m:disk_free=${100 - value} ${commandTags} ms:${timestamp}

If the 'Check On Exit' option is enabled for time-based window, some of the events are caused by exiting commands and the timestamp field returns the time of the oldest command, rounded to seconds.

To round the input time to seconds, use the seconds field s: and the floor function:

series e:${entity} m:disk_free=${100 - value} ${commandTags} s:${floor(timestamp/1000)}

Frequency

Derived commands can be stored each time a command is received or removed from the window by setting the Repeat parameter to 'All'.

The frequency can be lowered by adjusting the repeat interval.

The produced commands are queued in memory and are persisted to the database once per second.

Multiple Commands

Multiple commands, including commands of different type, can be specified at the same time. Each command must be specified on a separate line.

series e:${entity} m:jvm_memory_free_avg_percent=${round(100 - avg(), 3)}
series e:${entity} m:jvm_memory_free_min_percent=${round(100 - max(), 3)}

To create multiple metrics within the same command, use the for loop to iterate over a collection or an array.

series e:${entity} @{s = ""; for (stat : stats) {s = s + " m:" + stat.split(":")[0] + "=" + stat.split(":")[1];} return s;}

Assuming the stats collection is equal ['a:10', 'b:20', 'c:30'], the produced command looks as follows:

series e:entity1 m:a=10 m:b=20 m:c=30

Condition

If the rule purpose is to only create derived series, without any alerting, set the Condition field to a static true value to minimize the processing overhead.

Examples

Moving Average (Last N Count)

  • Window type: count-based
  • Window length: 10
  • Condition: true
  • Frequency: All
  • Command Template:
  series e:${entity} m:${metric}_movavg=${avg()} ${commandTags}

Moving Average (Last N Time)

  • Window type: time-based
  • Window length: 10 minute
  • Condition: true
  • Frequency: All or Every N Minutes = 1 minute
  • Command Template:
  series e:${entity} m:${metric}_movavg=${avg()} ${commandTags}

Roll-up (all matching entities)

  • Window type: time-based
  • Window length: 1 minute
  • Group by entity: NO
  • Condition: true
  • Frequency: Every N Minutes = 1 minute
  • Command Template:
  series e:total m:${metric}_sum=${sum()}

Reverse/Inverse Metric

  • Window type: count-based
  • Window length: 1
  • Condition: true
  • Frequency: All
  • Command Template:
  series e:${entity} m:${metric}_rev=${100-value} ${commandTags}
  series e:${entity} m:${metric}_inv=${value = 0 ? 0 : 1/value} ${commandTags}

Ratio / Percentage

  • Window type: count-based
  • Window length: 1
  • Condition: true
  • Frequency: All
  • Command Template:
  series e:${entity} m:${metric}_percent=${100 * value/value('total')} ${commandTags}

Message to Series

  • Window type: count-based
  • Window length: 1
  • Condition: true
  • Frequency: All
  • Command Template:
  series e:${entity} m:job_execution_time=${tags.job_execution_time.replaceAll("[a-zA-Z]", "").trim()}