coercer transform

Accepts `log` events and allows you to coerce log fields into fixed types.

The coercer transform accepts log events and allows you to coerce log fields into fixed types.

Config File

vector.toml (simple)
vector.toml (advanced)
[transforms.my_transform_id]
type = "coercer" # must be: "coercer"
inputs = ["my-source-id"]
# For a complete list of options see the "advanced" tab above.

Examples

Given the following input event:

log
{
// ... existing fields
"bytes_in": "5667",
"bytes_out": "20574",
"host": "5.86.210.12",
"message": "GET /embrace/supply-chains/dynamic/vertical",
"status": "201",
"timestamp": "19/06/2019:17:20:49 -0400",
"user_id": "zieme4647"
}

And the following configuration:

vector.toml
[transforms.<transform-id>]
type = "coercer"
[transforms.<transform-id>.types]
bytes_in = "int"
bytes_out = "int"
timestamp = "timestamp|%m/%d/%Y:%H:%M:%S %z"
status = "int"

A log event will be emitted with the following structure:

{
// ... existing fields
"bytes_in": 5667,
"bytes_out": 20574,
"host": "5.86.210.12",
"message": "GET /embrace/supply-chains/dynamic/vertical",
"status": 201,
"timestamp": <19/06/2019:17:20:49 -0400>,
"user_id": "zieme4647"
}

How It Works

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Types

By default, extracted (parsed) fields all contain string values. You can coerce these values into types via the types table as shown in the Config File example above. For example:

[transforms.my_transform_id]
# ...
# OPTIONAL - Types
[transforms.my_transform_id.types]
status = "int"
duration = "float"
success = "bool"
timestamp = "timestamp|%s"
timestamp = "timestamp|%+"
timestamp = "timestamp|%F"
timestamp = "timestamp|%a %b %e %T %Y"

The available types are:

Type

Desription

bool

Coerces to a true/false boolean. The 1/0 and t/f values are also coerced.

float

Coerce to 64 bit floats.

int

Coerce to a 64 bit integer.

string

Coerces to a string. Generally not necessary since values are extracted as strings.

timestamp

Coerces to a Vector timestamp. strftime specificiers must be used to parse the string.

Troubleshooting

The best place to start with troubleshooting is to check the Vector logs. This is typically located at /var/log/vector.log, then proceed to follow the Troubleshooting Guide.

If the Troubleshooting Guide does not resolve your issue, please:

  1. If encountered a bug, please file a bug report.

  2. If encountered a missing feature, please file a feature request.

  3. If you need help, join our chat/forum community. You can post a question and search previous questions.

Alternatives

Finally, consider the following alternatives:

Resources