lua transform

Accepts `log` events and allows you to transform events with a full embedded Lua engine.

The lua transform is in beta. Please see the current enhancements and bugs for known issues. We kindly ask that you add any missing issues as it will help shape the roadmap of this component.

The lua transform accepts log events and allows you to transform events with a full embedded Lua engine.

Config File

vector.toml (simple)
vector.toml (advanced)
[transforms.my_transform_id]
type = "lua" # must be: "lua"
inputs = ["my-source-id"]
source = """
require("script") # a `script.lua` file must be in your `search_dirs`
if event["host"] == nil then
local f = io.popen ("/bin/hostname")
local hostname = f:read("*a") or ""
f:close()
hostname = string.gsub(hostname, "\n$", "")
event["host"] = hostname
end
"""
# For a complete list of options see the "advanced" tab above.

Examples

Add fields
Remove fields
Drop event

Add a field to an event. Supply this as a the source value:

# Add root level field
event["new_field"] = "new value"
# Add nested field
event["parent.child"] = "nested value"

Remove a field from an event. Supply this as a the source value:

# Remove root level field
event["field"] = nil
# Remove nested field
event["parent.child"] = nil

Drop an event entirely. Supply this as a the source value:

# Drop event entirely
event = nil

How It Works

Dropping Events

To drop events, simply set the event variable to nil. For example:

if event["message"].match(str, "debug") then
event = nil
end

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Global Variables

When evaluating the provided source, Vector will provide a single global variable representing the event:

Name

Type

Description

event

table

The current [log event]. Depending on prior processing the structure of your event will vary. Generally though, it will follow the default event schema.

Note, a Lua table is an associative array. You can read more about Lua types in the Lua docs.

Nested Fields

As described in the Data Model document, Vector flatten events, representing nested field with a . delimiter. Therefore, adding, accessing, or removing nested fields is as simple as added a . in your key name:

# Add nested field
event["parent.child"] = "nested value"
# Remove nested field
event["parent.child"] = nil

Search Directories

Vector provides a search_dirs option that allows you to specify absolute paths that will searched when using the Lua require function.

Troubleshooting

The best place to start with troubleshooting is to check the Vector logs. This is typically located at /var/log/vector.log, then proceed to follow the Troubleshooting Guide.

If the Troubleshooting Guide does not resolve your issue, please:

  1. If encountered a bug, please file a bug report.

  2. If encountered a missing feature, please file a feature request.

  3. If you need help, join our chat/forum community. You can post a question and search previous questions.

Alternatives

Finally, consider the following alternatives:

Resources