Monthly Change Log: December 2017

ATSD

Issue Category Type Subject
4831 security Support Update dependencies to next minor versions.
4829 rule engine Bug ElUtils.preProcess fails while processing empty list.
4824 email Bug Enforce max limit to the message subject.
4819 api-rest Feature Implement incoming webhook functionality.
4817 rule engine Feature Add administrative setting to allow command execution.
4816 api-rest Bug Series query: fix PREVIOUS interpolation.
4815 entity Feature Entity groups: add field to specify the default tag template.
4811 portal Feature Use entity and metric labels in the default rule engine portal.
4808 rule engine Feature Implement access to metric values by name if metrics are in the same series command.
4807 rule engine Feature Webhooks: implement AWS API webhook.
4803 sql Bug Refactor setting to allow filling interpolated values with any value.
4790 rule engine Bug Webhooks: UI Enhancements.
4784 rule engine Bug Improve the contents and formatting of the default alert details table.
4765 rule engine Feature Implement derived commands.
4749 UI Bug Delete entity error after delete is cancelled.
4711 UI Feature UI: Multiple Enhancements.
3560 core Feature Implement units field in the Metric class.
2613 portal Feature Implement a redirect to view portal by name as well as by id.
4809 administration Bug Enforce UTF-8 file encoding in ATSD start scripts.
4808 rule engine Feature Implement value('metricName') function to access other metrics in the same series command by name.
4798 rule engine Feature Outgoing webhooks: add support for proxy and reverse-proxy network connections to external cloud services.
4796 rule engine Feature Portal: add Freemarker functions to check user role and group membership.
4795 rule engine Feature Implement BETWEEN AND clause.
4794 rule engine Feature Implement random selector functions.
4793 rule engine Feature Implement statistical functions with interval/count: avg('1 minute'), max(10).
4792 rule engine Feature Implement statistical functions with conditions: countIf, sumIf, avgIf.
4785 rule engine Feature Implement AWS SQS notification.
4741 rule engine Feature Implement AWS SNS notification.
4705 administration Bug ATSD does not start after update via docker exec.
4788 rule editor Feature Add endpoint links in the webhook drop-down.
4787 rule editor Bug Set default values for empty fields in CUSTOM webhook type.
4782 security Bug Prevent browser auto-complete to submit user credentials on the outgoing webhook form.
4779 api-rest Bug Series query: invalid JSON returned if addMeta parameter for unknown metric is specified.
4778 rule editor Feature Implement placeholders for the Details Table.
4776 rule engine Feature Extract functions from the db_last and db_statistic functions to include in webhook charts.
4775 rule editor Bug Raise error on invalid syntax in database functions.
4773 api-rest Bug Series query: fix for empty response when seriesLimit is enforced.
4772 rule editor Feature Add setting Group by All Tags.
4767 rule engine Bug Webhooks: Timeout waiting for connection from pool on CircleCI.
4765 rule engine Feature Implement derived commands.
4757 rule engine Bug Remove excessive rounding in variables.
4754 rule engine Feature Add window viewer.
4753 rule engine Bug Alert does not reopen when Message on Cancel is empty.
4728 portal Bug Charts and portals: ensure LTR layout regardless of the browser setting.
4717 api-rest Bug Series query: wrong below median PERCENTILE calculations (all equal MIN).
4713 api-rest Bug Series query with FORECAST type: add support for addMeta parameter.
4781 rule engine Feature Implement HipChat Data Center notifications.
4738 rule engine Feature Implement WEBHOOK notifications.
4579a rule engine Feature Implement CUSTOM notifications.
4579b rule engine Feature Implement SLACK notifications.
4579c rule engine Feature Implement TELEGRAM notifications.
4579d rule engine Feature Implement DISCORD notifications.
4774 rule engine Bug Invalid URL validation error for .site domain.
4769 rule engine Bug Rule does not update when the linked webhook changes.
4767 rule engine Bug CUSTOM webhook: timeout waiting for connection from pool.
4764 rule engine Bug Do not show entity information in entity non-grouped alert details.
4762 rule engine Feature Simplify DISCORD webhook configuration.
4761 rule editor Bug Add override link not creating a row in the Overrides table.
4760 api-rest Bug series query method: entity expression error when using name field.
4759 rule editor Bug Buttons disappear when the rule configuration contains errors.
4758 rule editor Bug Error when saving webhook using HTTP protocol.
4756 api-rest Bug Data API: Minimum date for series query with FORECAST type.
4755 rule engine Feature Implement db_message functions for correlation.
4744 rule engine Bug Add chart link to webhook message if chart error is raised.
4725 api-rest Bug Processing error in series query method with END_TIME alignment.
4713 api-rest Bug series query with FORECAST type ignores addMeta parameter.

Charts

Issue Category Type Subject
4805 api Feature Interpolation field in series configuration.

Collector

Issue Category Type Subject
4830 core Bug Security Fixes.
4780 docker Bug Collector CPU is abnormally high if TCP checks are enabled.
4742 docker Bug Add atsd-url parameter validation.
4552 json Feature JSON job: add support for thousand items and more.

Issue 4755

The db_message_count and db_message_last functions allow one to correlate different types of data, time series and messages.

db_message_count Function

  • Calculate the number of messages matching the specified parameters.
db_message_count(S interval, S type, S source [, S tags, [S entity]])

tags and entity arguments are optional. If the type, source, or tags fields are set to empty string, they are ignored when matching messages. If the entity is not specified, the request retrieves messages for the current entity.

Example:

// Check if the average exceeds 20 and the 'compaction' message is not received within the last hour for the current entity.
avg() > 20 && db_message_count('1 hour', 'compaction', '') == 0
// Check if the average exceeds 80 and there is an event with type=backup-error received within the last 15 minutes for entity 'nurswgvml006'.
avg() > 80 && db_message_count('15 minute', 'backup-error', '', '', 'nurswgvml006') > 0

db_message_last Function

  • Returns the most recent message object matching the specified parameters.
db_message_last(S interval, S type, S source[, S tags, [S entity]])

tags and entity arguments are optional. If the type, source, or tags fields are set to empty string, they are ignored when matching messages. If the entity is not specified, the request retrieves messages for the current entity.

The returned object contains type, source, and tags.{name} fields of string type and the date field of long data type. The date field is returned as Unix time in milliseconds.

Example:

last_msg = db_message_last('60 minute', 'logger', '')
// Check that the average exceeds 50 and the severity of the last message with type 'logger' for the current entity is greater or equal `ERROR`.
value > 50 && last_msg != null && last_msg.severity.toString() >= "6"

Issue 4808

The value functions provide access to other metrics submitted within the same series command or parsed from the same row in the CSV file.

value > 1.5 && value('temperature') > 50
series e:sensor01 m:pressure=3.5 m:temperature=80

Assuming the rule is created for the pressure metric, the below condition evaluates to true for the above series command.

3.5 > 1.5 && 80 > 50

Issue 4796

The functions simplify the design of role-based portals.

// Returns true if the user belongs to the specified user group.
userInGroup(String userGroupname)

// Returns true if the user has read permissions for the specified entity.
userAllowEntity(String entityName)

// Returns true if the user has the specified role: USER, EDITOR, etc.
userHasRole(String roleName)

// Returns true if the user has READ permissions to the specified entity group.
userAllowEntityGroup(String entityGroupName)

Issue 4795

value BETWEEN 75 AND 90
dateStr BETWEEN '18:30:00' AND '18:34:59'

Issue 4794

randomKey(replacementTable('oncall-person'))
randomKey(excludeKeys(replacementTable('oncall-person'),['John Doe']))
randomItem(collection('ip_white_list'))

Issue 4793

avg('1 minute') / avg() > 2
max(5) == max()

Issue 4792

countIf('value > 10') < 5

Issue 4785

  • SQS

Issue 4741

  • SNS

Issue 4778

The alert details table can embedded into the target document using the same format as the host document.

${detailsTable('markdown')}
${detailsTable('ascii')}
${detailsTable('html')}
${detailsTable('property')}
${detailsTable('csv')}
${detailsTable('json')}

Issue 4772

The new settings automatically creates as many series as there are incoming series to simplify the initial configuration. For message and property commands, the setting is set to No Tags.

Issue 4765

The derived command action allows storing new calculated metrics in the database by executing custom command templates in the Network API syntax.

series e:${entity} m:jvm_memory_free_avg_percent=${round(100 - avg(), 3)}

The new commands can be executed at the frequency of incoming commands or decreased with a custom repeat interval.

Issue 4781

HipChat Data Center notifications.

Issue 4579b

Slack notifications.

Issue 4579c

Telegram notifications.

Issue 4579d

Discord notifications.