json_parser transform

Accepts `log` events and allows you to parse a field value as JSON.

The json_parser transform accepts log events and allows you to parse a field value as JSON.

Config File

vector.toml (example)
vector.toml (schema)
vector.toml (specification)
[transforms.my_transform_id]
type = "json_parser" # must be: "json_parser"
inputs = ["my-source-id"]
drop_invalid = true
field = "message" # default

Options

Key

Type

Description

REQUIRED

type

string

The component type required must be: "json_parser"

inputs

[string]

A list of upstream source or transform IDs. See Config Composition for more info. required example: ["my-source-id"]

drop_invalid

bool

If true events with invalid JSON will be dropped, otherwise the event will be kept and passed through. See Invalid JSON for more info. required example: true

OPTIONAL

field

string

The field decode as JSON. Must be a string value. See Invalid JSON for more info. default: "message"

Examples

Simple
Wrapped

Given the following log event:

{
"message": "{"key": "value"}"
}

You can parse the JSON with:

[transforms.json]
inputs = ["<source_id>"]
type = "json_parser"
field = "message"

This would produce the following event as output:

{
"key": "value"
}

By default, Vector drops fields after parsing them via the drop_field option.

It is possible to chain json_parser transforms to effectively "unwrap" nested JSON documents. For example, give this log event:

{
"message": "{"parent": "{\"child\": \"value2\"}"}"
}

You could unwrap the JSON with the following transforms:

[transforms.root_json]
inputs = ["<source_id>"]
type = "json_parser"
field = "message"
[transforms.parent_json]
inputs = ["root_json"]
type = "json_parser"
field = "parent"
[transforms.child_json]
inputs = ["parent_json"]
type = "json_parser"
field = "child"

This would produce the following event as output:

{
"child": "value2"
}

By default, Vector drops fields after parsing them via the drop_field option.

How It Works

Chaining / Unwrapping

Please see the I/O section for an example of chaining and unwrapping JSON.

Correctness

The json_parser source has been involved in the following correctness tests:

Learn more in the Correctness sections.

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Invalid JSON

If the value for the specified field is not valid JSON you can control keep or discard the event with the drop_invalid option. Setting it to true will discard the event and drop it entirely. Setting it to false will keep the event and pass it through. Note that passing through the event could cause problems and violate assumptions about the structure of your event.

Key Conflicts

Any key present in the decoded JSON will override existin keys in the event.

Nested Fields

If the decoded JSON includes nested fields it will be deep merged into the event. For example, given the following event:

{
"message": "{"parent": {"child2": "value2"}}",
"parent": {
"child1": "value1"
}
}

Parsing the "message" field would result the following structure:

{
"parent": {
"child1": "value1",
"child2": "value2"
}
}

Troubleshooting

The best place to start with troubleshooting is to check the Vector logs. This is typically located at /var/log/vector.log, then proceed to follow the Troubleshooting Guide.

If the Troubleshooting Guide does not resolve your issue, please:

  1. If encountered a bug, please file a bug report.

  2. If encountered a missing feature, please file a feature request.

  3. If you need help, join our chat/forum community. You can post a question and search previous questions.

Alternatives

Finally, consider the following alternatives:

Resources