# Smoothing Time Series

## Overview

Smoothing is a transformation that reduces the noise in an underlying time series. Unlike period aggregation which returns series with new timestamps set to period start times, the smoothing transformation retains the original timestamps.

For each timestamp in the underlying series, the smoothed value is calculated in two steps:

- A sequence of preceding samples up to the current timestamp is retrieved from the underlying series. This sequence ends with the current sample and is called a
**rolling**(or sliding) window. - A new value at the current timestamp is calculated by applying a smoothing function, such as average, to the retrieved sequence.

Supported smoothing functions:

`AVG`

: Average`COUNT`

: Count`SUM`

: Sum`WAVG`

: Weighted Average`WTAVG`

: Weighted-time Average`EMA`

: Exponential Moving Average

The size of the rolling window is based on either time duration or the number of samples. Window size is controlled by the `interval`

and `count`

settings.

The example below calculates the simple moving average with a 15 minute rolling window.

```
"smooth": {
"type": "AVG",
"interval": {"count": 15, "unit": "MINUTE"}
}
```

The implemented smoothing functions differ in how they assign weight to samples in the sliding window. Simple average `AVG`

assigns equal weight to all samples. Weighted average functions take the sample index or time distance between samples into account and assign greater weight to more recent samples.

## Fields

Name | Type | Description |
---|---|---|

`type` | string | [Required] Smoothing function.Available functions: `AVG` , `COUNT` , `SUM` , `WAVG` , `WTAVG` , `EMA` . |

`count` | number | Number of samples in a count-based rolling window. |

`interval` | object | Duration of a time-based window specified with `count` and time `unit` .For example: `{"count": 3, "unit": "HOUR"}` . Supported units: `MILLISECOND` , `SECOND` , `MINUTE` , `HOUR` . |

`minimumCount` | number | Minimum number of samples in a window required to apply the smoothing function. Default value is `1` .If a window is incomplete (count is below minimum), the smoothing function returns `incompleteValue` for the current timestamp. |

`incompleteValue` | string | Number returned in an incomplete window for the current timestamp if the sample count is below `minimumCount` .Possible values: `null` (default), `NaN` , or a constant numeric value.`null` values are omitted from the response. |

Parameters

`count`

and`interval`

are ignored by the`EMA`

function which weighs all loaded samples.

The longer the time window, the more smoothing is performed on the returned series.

## Smoothing Process

Received series samples are processed sequentially in ascending time order.
The ordered sequence is called the **rolling window** or a **sliding window**. Initially the window is empty.

For each series sample the following steps are executed sequentially:

- If the sample value is
`NaN`

, include the sample in the response unchanged. Skip remaining steps. - Append the sample to the end of the window.
- Remove outdated samples:
- For
**time-based**windows: Remove any samples with timestamp equal to or earlier than the current timestamp minus the interval. - For
**count-based**windows: Remove the first (oldest) sample if window length exceeds`count`

.

- For
- If the number of samples is below
`minimumCount`

, return the value specified in the`incompleteValue`

parameter. - Return the value calculated by the smoothing function.

The timestamp of the returned samples is the same as the timestamp of the input sample.

## Smoothing Functions

### Average

`type = AVG`

Calculates average value within rolling window. Sum of values divided by number of values.

Window values:

Calculation formula:

### Count

`type = COUNT`

Calculates the count of values in the rolling window.

### Sum

`type = SUM`

Calculates the sum of values in the rolling window.

### Weighted Average

`type = WAVG`

Calculates weighted average where the weight of the value equals the sample index.

Window values:

Calculation formula:

### Weighted Time Average

`type = WTAVG`

.

Calculates weighted average where the weight of the sample value is equal to the distance between the first timestamp and the current sample timestamp.

Window samples, with timestamps measured in milliseconds:

Calculation formula:

The function returns sample value () if .

### Exponential Moving Average

`type = EMA`

Calculates exponentially smoothed weighted average. The function uses a window which consists of **all** samples preceding the current timestamp, therefore it does not require `count`

and `interval`

parameters. The contribution of a sample to the calculated smoothed value decreases exponentially for more distant samples.

`EMA`

Specific Fields

Name | Type | Description |
---|---|---|

`factor` | number | Smoothing factor. A number within the `(0, 1)` range.A smaller smoothing factor decreases variance. Default value: `0.25` |

`range` | number | Controls steepness of the smoothing function. Typically used to smooth irregular series. A larger range decreases variance. |

Input series samples:

The smoothed series contains samples with the same timestamps:

Calculations using

`factor`

where is the value of smoothing

`factor`

.Calculations using

`range`

The calculation applies the same formulas

`(1)`

and`(2)`

, using a`factor`

calculated based on series timestamps and the specified`range`

:where is the value of

`range`

.For regular series,

`range`

can be calculated from`factor`

:where is the time interval between consecutive samples, measured in milliseconds.

If the interval between samples is

`1000`

milliseconds, and smoothing factor is`0.5`

, the`range`

is`1443`

.

#### References

- A. Eckner, Algorithms for Unevenly Spaced Time Series: Moving Averages and Other Rolling Operators, Section 4.1,
`EMA_next`

. - U. Muller, Specially Weighted Moving Averages with Repeated Application of the EMA Operator, Formulas
`2.7`

-`2.14`

.