add_fields transform

Accepts `log` events and allows you to add one or more log fields.

The add_fields transform accepts log events and allows you to add one or more log fields.

Config File

vector.toml (simple)
vector.toml (advanced)
[transforms.my_transform_id]
# REQUIRED - General
type = "add_fields" # must be: "add_fields"
inputs = ["my-source-id"]
# REQUIRED - Fields
[transforms.my_transform_id.fields]
my_string_field = "string value"
my_env_var_field = "${ENV_VAR}"
my_int_field = 1
my_float_field = 1.2
my_bool_field = true
my_timestamp_field = 1979-05-27T00:32:00Z
my_nested_fields = {key1 = "value1", key2 = "value2"}
my_list = ["first", "second", "third"]
# For a complete list of options see the "advanced" tab above.

Examples

Given the following configuration:

vector.toml
[transforms.my_transform]
type = "add_fields"
inputs = [...]
[transforms.my_transform.fields]
field1 = "string value"
field2 = 1
field3 = 2.0
field4 = true
field5 = 2019-05-27T07:32:00Z
field6 = ["item 1", "item 2"]
field7.nested = "nested value",
field8 = "#{HOSTNAME}"

A log event will be emitted with the following structure:

log
{
// ... existing fields
"field1": "string value",
"field2": 1,
"field3": 2.0,
"field4": true,
"field5": <timestamp:2019-05-27T07:32:00Z>,
"field6.0": "item1",
"field6.1": "item2",
"field7.nested": "nested value",
"field8": "my.hostname.com"
}

While unrealistic, this example demonstrates the various accepted types and how they're repsented in Vector's internal log structure.

How It Works

Arrays

The add_fields transform will support TOML arrays. Keep in mind that the values must be simple type (not tables), and each value must the same type. You cannot mix types:

[transforms.<transform-id>]
# ...
[transforms.<transform-id>.fields]
my_array = ["first", "second", "third"]

Results in:

{
"my_array.0": "first",
"my_array.1": "second",
"my_array.2": "third"
}

Learn more about how log events are structured.

Complex Transforming

The add_fields transform is designed for simple key additions. If you need more complex transforming then we recommend using a more versatile transform like the lua transform.

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Key Conflicts

Keys specified in this transform will replace existing keys.

Nested Fields

The add_fields transform will support dotted keys or TOML tables. We recommend the dotted key syntax since it is less verbose for this usecase:

[transforms.<transform-id>]
# ...
[transforms.<transform-id>.fields]
parent.child.grandchild = "my_value"

Results in:

{
"parent.child.grandchild": "my_value"
}

Learn more about how log events are structured.

Removing Fields

See the remove_fields transform.

Special Characters

Aside from the special characters listed in the Data Model doc, Vector does not restrict the characters allowed in keys. You can wrap key names in " " quote to preserve spaces and use \ to escape quotes.

Types

All supported configuration value types are accepted. This includes primitivate types (string, int, float, boolean) and special types, such as arrays and nested fields.

Troubleshooting

The best place to start with troubleshooting is to check the Vector logs. This is typically located at /var/log/vector.log, then proceed to follow the Troubleshooting Guide.

If the Troubleshooting Guide does not resolve your issue, please:

  1. If encountered a bug, please file a bug report.

  2. If encountered a missing feature, please file a feature request.

  3. If you need help, join our chat/forum community. You can post a question and search previous questions.

Alternatives

Finally, consider the following alternatives:

Resources