json_parser transform

Accepts `log` events and allows you to parse a log field value as JSON.

The json_parser transform accepts log events and allows you to parse a log field value as JSON.

Config File

vector.toml (simple)
vector.toml (advanced)
[transforms.my_transform_id]
type = "json_parser" # must be: "json_parser"
inputs = ["my-source-id"]
drop_invalid = true
# For a complete list of options see the "advanced" tab above.

Examples

Simple
Wrapped

Given the following log event:

{
"message": "{"key": "value"}"
}

You can parse the JSON with:

[transforms.json]
inputs = ["<source_id>"]
type = "json_parser"
field = "message"

This would produce the following event as output:

{
"key": "value"
}

By default, Vector drops fields after parsing them via the drop_field option.

It is possible to chain json_parser transforms to effectively "unwrap" nested JSON documents. For example, give this log event:

{
"message": "{"parent": "{\"child\": \"value2\"}"}"
}

You could unwrap the JSON with the following transforms:

[transforms.root_json]
inputs = ["<source_id>"]
type = "json_parser"
field = "message"
[transforms.parent_json]
inputs = ["root_json"]
type = "json_parser"
field = "parent"
[transforms.child_json]
inputs = ["parent_json"]
type = "json_parser"
field = "child"

This would produce the following event as output:

{
"child": "value2"
}

By default, Vector drops fields after parsing them via the drop_field option.

How It Works

Chaining / Unwrapping

Please see the I/O section for an example of chaining and unwrapping JSON.

Correctness

The json_parser source has been involved in the following correctness tests:

Learn more in the Correctness sections.

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Invalid JSON

If the value for the specified field is not valid JSON you can control keep or discard the event with the drop_invalid option. Setting it to true will discard the event and drop it entirely. Setting it to false will keep the event and pass it through. Note that passing through the event could cause problems and violate assumptions about the structure of your event.

Key Conflicts

Any key present in the decoded JSON will override existin keys in the event.

Nested Fields

If the decoded JSON includes nested fields it will be deep merged into the event. For example, given the following event:

{
"message": "{"parent": {"child2": "value2"}}",
"parent": {
"child1": "value1"
}
}

Parsing the "message" field would result the following structure:

{
"parent": {
"child1": "value1",
"child2": "value2"
}
}

Troubleshooting

The best place to start with troubleshooting is to check the Vector logs. This is typically located at /var/log/vector.log, then proceed to follow the Troubleshooting Guide.

If the Troubleshooting Guide does not resolve your issue, please:

  1. If encountered a bug, please file a bug report.

  2. If encountered a missing feature, please file a feature request.

  3. If you need help, join our chat/forum community. You can post a question and search previous questions.

Alternatives

Finally, consider the following alternatives:

Resources