Network API provides a set of plain text commands for inserting numeric time series, key=value properties, and tagged messages into the database via TCP and UDP network protocols.
You can use
bash pseudo-device files, or a programming language such as Java that implements the TCP/UDP stack.
Extended Data Commands
By default, the database server listens for incoming commands on the following ports:
To encrypt TCP traffic, setup an SSH tunnel or create a VPN connection.
Authentication and authorization are not supported for plain text commands received over TCP and UDP protocols.
Utilize the HTTP command to send plain-text commands over http/https protocols with authentication and authorization enabled.
To send a single command, connect to an ATSD server, send the command in plain text, and terminate the connection.
echo -e "series e:station_1 m:temperature=32.2 m:humidity=81.4 d:2016-05-15T00:10:00Z" | nc -w 1 atsd_host 8081
printf 'series e:station_2 m:temperature=32.2 m:humidity=81.4 s:1463271035' | nc -w 1 atsd_host 8081
echo -e "series e:station_3 m:temperature=32.2 m:humidity=81.4" > /dev/tcp/atsd_host/8081
bashpseudo-device files which can be used in redirection.
- telnet: one line
telnet atsd_host 8081 << EOF series e:station_4 m:temperature=32.2 m:humidity=81.4 EOF
- telnet: session
$ telnet atsd_host 8081 Trying atsd_host... Connected to atsd_host. Escape character is '^]'. series e:station_5 m:temperature=32.2 m:humidity=81.4 ^C Connection closed by foreign host.
- java: socket
Socket s = new Socket("atsd_host", 8081); PrintWriter writer = new PrintWriter(s.getOutputStream(), true); writer.println("series e:station_6 m:temperature=32.2"); s.close();
The above examples insert timestamped temperature and humidity metric samples for station entities.
Separate commands by a line feed symbol
0x0A) when sending a batch containing multiple commands over the same connection.
A trailing line feed is not required for the last command in the batch.
-e option in
echo commands to enable interpretation of backslash escapes.
echo -e "series e:station_1 m:temperature=32.2 m:humidity=81.4 d:2016-05-15T00:10:00Z\nseries e:station_1 m:temperature=32.1 m:humidity=82.4 d:2016-05-15T00:25:00Z" | nc -w 1 atsd_host 8081
Socket s = new Socket("atsd_host", 8081); PrintWriter writer = new PrintWriter(s.getOutputStream(), true); writer.println("series e:station_6 m:temperature=30.1\nseries e:station_7 m:temperature=28.7"); s.close();
A client application can establish a persistent connection to continuously write commands, one command per line, and close the connection.
Trailing line feed is not required for the last command when the connection is closed.
Commands are processed as they are received by the server, without buffering.
To prevent the connection from timing out the client may send a
ping command at a regular interval.
Clients can submit different types of commands over the same connection.
$ telnet atsd_host 8081 Trying atsd_host... Connected to atsd_host. Escape character is '^]'. series e:station_1 m:temperature=32.2 property e:station_2 t:location v:city=Cupertino v:state=CA v:country=USA ^C Connection closed by foreign host.
Note that the server terminates the connection if it receives an unsupported or malformed command.
$ telnet atsd_host 8081 Trying atsd_host... Connected to atsd_host. Escape character is '^]'. unknown_command e:station_1 m:temperature=32.2 Connection closed by foreign host.
If the connection is terminated due to client error, all valid commands sent prior to the first invalid command are stored.
Due to the fact that closing the channel due to client error may take some time, the database may also store a few valid commands received after the discarded command.
valid command - stored ... - stored valid command - stored invalid command - discarded -> initiate channel closing valid command - possibly stored if present in buffer ...
The above behavior can be modified by changing the
input.disconnect.on.error setting to
No on the Settings > Server Properties page.
This causes the database to maintain a client connection even if one of the received commands is malformed or unknown.
The UDP protocol does not guarantee delivery but may have a higher throughput compared to TCP due to lower overhead.
In addition, sending commands with UDP datagrams decouples the client application from the server to minimize the risk of blocking I/O time-outs.
echo -e "series e:station_3 m:temperature=32.2 m:humidity=81.4" | nc -u -w 1 atsd_host 8082
printf 'series e:station_3 m:temperature=32.2 m:humidity=81.4' | nc -u -w 1 atsd_host 8082
Unlike TCP, the last command in a multi-command UDP datagram must be terminated with the line feed character.
echo -e "series e:station_33 m:temperature=32.2\nseries e:station_34 m:temperature=32.1 m:humidity=82.4\n" | nc -u -w 1 atsd_host 8082
Multiple commands with the same timestamp and key fields may override each others value.
If such commands are submitted at approximately the same time, there is no guarantee that they are processed in the order received.
- Duplicate example: same key, same current time
echo -e "series e:station_1 m:temperature=32.2\nseries e:station_1 m:temperature=42.1" | nc atsd_host 8081
- Duplicate example: same key, same time
echo -e "series e:station_1 m:temperature=32.2 d:2016-05-15T00:10:00Z\nseries e:station_1 m:temperature=42.1 d:2016-05-15T00:10:00Z" | nc atsd_host 8081
TCP Client Examples
A command must start with a name such as
series followed by space-separated fields each identified with a prefix, followed by
: colon, and
- The order of fields is not important.
- Refer to the
ABNFrules of a particular command for its exact rules.
- A field name can contain only printable characters.
- If the field name contains a double-quote (") or an equal (=) sign, it must be enclosed in double quotes. For example:
- Any double quote character in the value must be escaped with another double quote.
- A field value can contain printable and non-printable characters including space, line breaks, tab.
- If the field value contains a double-quote (") or equal (=) sign or a non-printable character, it must be enclosed in double quotes. For example:
- Any double quote character in the value must be escaped with another double quote.
Use CSV escaping methods in core libraries where available, for example
StringEscapeUtils.escapeCsv in Java.
- Field names are case-insensitive and are converted to lower case when stored in the database.
- Field values are case-sensitive and are stored as submitted, except for entity names, metric names, and property types, which are converted to lower case.
# input command series e:nurSWG m:Temperature=38.5 t:Degrees=Celsius # stored record series e:nurswg m:temperature=38.5 t:degrees=Celsius
The command length cannot exceed 128 kilobytes.
The client must split a command that is too long into multiple commands.
Tag Count Limit
The number of tags included in the command cannot exceed the following limit:
|series||1024 series tags|
|property||1024 keys and tags|
|message||1024 message tags|
- New entities, metrics, and tags are created automatically when inserting data.
- The number of unique identifiers is subject to the following limits:
The timestamp field records the time of an observation or an event as determined by the source and can be specified with
|ms||long||Unix milliseconds since 1970-01-01T00:00:00Z|
|s||integer||Unix seconds since 1970-01-01T00:00:00Z|
|d||string||ISO 8601 date format. Supported formats:|
UTC time zone (Z) =
Time zone offset =
- The minimum time that can be stored in the database is
0milliseconds from Unix time.
- The maximum date that can be stored by the database is
4294969199999milliseconds from Unix time.
- If the timestamp field is not specified, time is set to current server time.
- The decimal separator is a period (
- No thousands separator is allowed.
- No digit grouping is allowed.
- Negative numbers use the negative sign (
-) at the beginning of the number.
- Not-a-Number is literal
Enable Debug Mode
By default, ATSD does not return acknowledgements to the client after processing data commands.
debug command at the start of the line to instruct the server to respond with
ok for each processed command.
debugwith valid command
$ echo -e "debug series e:station_1 m:temperature=32.2" \ | nc -w 1 atsd_host 8081 ok
debugwith unknown command
$ echo -e "debug my_command e:station_1 m:temperature=32.2" \ | nc -w 1 atsd_host 8081 >no response, connection closed
Review Client Commands
To review a sequence of commands sent by the client, launch the
netcat utility in server mode, reconfigure the client to send data to the netcat port.
nc -lk 0.0.0.0 2081 > command-in.log &
echo -e "series e:station_1 m:temperature=32.2 m:humidity=81.4 d:2016-05-15T00:10:00Z" \ | nc -w 1 localhost 2081
Reasons why ATSD server can drop commands:
- Entity, metric, or tag names are not valid.
- Timestamp is negative or earlier than
- Timestamp field
ms:is not numeric or if the
dfield is not in ISO format.
- Metric value could not be parsed as a number using
.as the decimal separator. Scientific notation is supported.
- Multiple data points for the same entity, metric, and tags have the same timestamp in which case commands are considered duplicates and some of them are dropped. This could occur when commands with the same key are sent without a timestamp.
- Data is sent using the UDP protocol and the client UDP send buffer or the server UDP receive buffer overflows.
- Value is below 'Min Value' or above 'Max Value' limit specified for the metric and the 'Invalid Value Action' is set to
- Last command in a multi-line UDP packed does not terminate with line feed symbol.
To review dropped commands,
open command*.log files in ATSD.