VRL expression reference

Syntax

VRL programs can be constructed with the following syntax rules.

Comment

A comment serves as program documentation and is identified with #. Each line must be preceeded with a # character. VRL currently does not allow for block comments.

Examples

# comment
# multi-line
# comment

Keywords

Keywords are reserved words that are used for primitive language features, such as if, and cannot be used as variable assignments or other custom directives. The following words are reserved:

  • abort
  • as
  • break
  • continue
  • else
  • false
  • for
  • if
  • impl
  • in
  • let
  • loop
  • null
  • return
  • self
  • std
  • then
  • this
  • true
  • type
  • until
  • use
  • while

Whitespace

Whitespace is any non-empty string as defined by the Unicode White_Space property.

VRL is a “free-form” language, meaning that all forms of whitespace serve only to separate tokens in the grammar, and have no semantic significance.

Literal expressions

As in most other languages, literals in VRL are values written exactly as they are meant to be interpreted. Literals include things like strings, Booleans, and integers.

Array

An array literal is a comma-delimited set of expressions that represents a contiguous growable array type.

Examples

[]
["first", "second", "third"]
["mixed", 1, 1.0, true, false, {"foo": "bar"}]
["first-level", ["second-level", ["third-level"]]
[.field1, .field2, to_int!("2"), variable_1]
[
  "expressions",
  1 + 2,
  2 == 5,
  true || false
]

Boolean

A Boolean literal represents a binary value which can only be either true or false.

Examples

true
false

Float

A float literal is a decimal representation of a 64-bit floating-point type (specifically, the “binary64” type defined in IEEE 754-2008).

A decimal floating-point literal consists of an integer part (decimal digits), a decimal point, a fractional part (decimal digits).

Examples

1_000_000.01
1000000.01
1.001

Characteristics

Limits
Floats in VRL can range from -1.7976931348623157E+308f64 to 1.7976931348623157E+308f64. Floats outside that range are wrapped.
Underscores
Floats can use underscore (_) characters instead of , to make them human readable. For example, 1_000_000.

Integer

An integer literal is a sequence of digits representing a 64-bit signed integer type.

Examples

1_000_000
1000000

Characteristics

Limits
Integers in VRL can range from -9223372036854775807 to 9223372036854775807. Integers outside that range are wrapped.
Underscore
Integers can use underscore (_) characters instead of , to make them human readable. For example, 1_000_000.

Null

A null literal is the absence of a defined value.

Examples

null

Object

An object literal is a growable key/value structure that is syntactically equivalent to a JSON object.

A well-formed JSON document is a valid VRL object.

Examples

{
	"field1": "value1",
	"field2": [ "value2", "value3", "value4" ],
	"field3": { "field4": "value5" }
}
{
	"field1": .some_path,
	"field2": some_variable,
	"field3": { "subfield": "some value" }
}

Characteristics

Ordering
Object fields are ordered alphabetically by the key in ascending order. Therefore, operations like encoding into JSON produce a string with keys that are in ascending alphabetical order.

Regular Expression

A regular expression literal represents a Regular Expression used for string matching and parsing.

Regular expressions are defined by the r sigil and wrapped with single quotes (r'...'). The value between the quotes uses the Rust regex syntax.

Examples

r'^Hello, World!$'
r'^Hello, World!$'i
r'^\d{4}-\d{2}-\d{2}$'
r'(?P<y>\d{4})-(?P<m>\d{2})-(?P<d>\d{2})'

Characteristics

Flags

Regular expressions allow for flags. Flags can be combined, as in r'(?ixm)pattern', r'(?im)pattern', etc.

To learn more about regular expressions in Rust—and by extension in VRL—we strongly recommend the in-browser Rustexp expression editor and tester.

Enum options
OptionDescription
USwap the meaning of x* and x*?
iCase insensitive
mMulti-line mode
sAllow . to match
uUnicode support (enabled by default)
xIgnore whitespace
Named Captures

Regular expressions support named capture groups, allowing extractions to be associated with keys. Named captures should be preceded with a ?P<name> declaraction. This regex, for example…

r'(?P<y>\d{4})-(?P<m>\d{2})-(?P<d>\d{2})'

…extracts captures with the y, m, and d keys.

String

A string literal is a UTF-8–encoded string. String literals can be raw or interpreted.

Raw string literals are composed of the uninterpreted (implicitly UTF-8-encoded) characters between single quotes identified with the s sigil and wrapped with single quotes (s'...'); in particular, backslashes have no special meaning and the string may contain newlines.

Interpreted string literals are character sequences between double quotes ("..."). Within the quotes, any character may appear except newline and unescaped double quote. The text between the quotes forms the result of the literal, with backslash escapes interpreted as defined below.

Examples

"Hello, world! 🌎"
"Hello, world! \\u1F30E"
s'Hello, world!'
s'{ "foo": "bar" }'

Characteristics

Backslash escapes
Special characters, such as newlines, can be expressed with a backslash escape.
Enum options
OptionDescription
`\"`Double quote
`\'`Single quote
`\0`Null
`\\`Backslash
`\n`Newline
`\r`Carriage return
`\t`Tab
`\u{7FFF}`24-bit Unicode character code (up to 6 digits)
Concatenation
Strings can be concatenated with the + operator.
Invalid Characters
Invalid UTF-8 sequences are replaced with the character.

Timestamp

A timestamp literal defines a native timestamp expressed in the RFC 3339 format with a nanosecond precision.

Timestamp literals are defined by the t sigil and wrapped with single quotes (t'2021-02-11T10:32:50.553955473Z').

Examples

t'2021-02-11T10:32:50.553955473Z'
t'2021-02-11T10:32:50.553Z'
t'2021-02-11T10:32:50.553-04:00'

Characteristics

Timezones
As defined in RFC 3339 format, timestamp literals support UTC and local offsets.

Dynamic expressions

VRL is an expression-oriented language. A VRL program consists entirely of expressions and every expression returns a value.

Abort

An abort expression causes the VRL program to terminate, aborting any modifications made to the event.

Grammar

abort

Examples

Ignoring invalid events
Vector event
{
  "log": {
    "message": "hello world"
  }
}
VRL program
if contains(string!(.message), "hello") {
	abort
}
.message = "not hello world"
Resulting event
{
  "message": "hello world"
}

Arithmetic

An arithmetic expression performs an operation on two expressions (operands) as defined by the operator.

Although arithmetic is commonly applied to numbers, you can use it with other types as well, such as strings.

Grammar

expression ~ operator ~ expression
ArgumentMeaning
expressionThe expression can be any expression that returns a valid type as defined by the operator.
operatorThe operator defines the operation performed on the left-hand- and right-hand-side operands.

Examples

Sum (int)
VRL program
1 + 1
Resulting event
2
Sum (float)
VRL program
1.0 + 1.0
Resulting event
2
Sum (numeric)
VRL program
1 + 1.0
Resulting event
2
Sum (string)
VRL program
"Hello" + ", " + "World!"
Resulting event
Hello, World!
Difference (int)
VRL program
2 - 1
Resulting event
1
Difference (float)
VRL program
2.0 - 1.0
Resulting event
1
Difference (numeric)
VRL program
2.0 - 1
Resulting event
1
Multiplication (int)
VRL program
2 * 1
Resulting event
2
Multiplication (float)
VRL program
2.0 * 1.0
Resulting event
2
Multiplication (numeric)
VRL program
2.0 * 1
Resulting event
2
Float division (int)
VRL program
2 / 1 ?? null
Resulting event
2
Float division (float)
VRL program
2.0 / 1.0 ?? null
Resulting event
2
Float division (numeric)
VRL program
2.0 / 1 ?? null
Resulting event
2
Remainder
VRL program
3 % 2
Resulting event
1

Assignment

An assignment expression assigns the result of the right-hand-side expression to the left-hand-side target (path or variable).

Grammar

target ~ ("," ~ error)? ~ operator ~ expression
ArgumentMeaning
errorThe error allows for optional assignment to errors when the right-hand-side expression is fallible. This is commonly used when invoking fallible functions.
expression

If the target is a variable, the expression can be any expression.

If the target is a path, the expression can be any expression that returns a supported object value type (i.e. not a regular expression).

operatorThe operator delimits the target and expression and defines assignment conditions.
targetThe target must be a path, with an optional second variable for error handling if the right-hand side is fallible.

Examples

Path assignment
VRL program
.message = "Hello, World!"
Nested path assignment
VRL program
.parent.child = "Hello, World!"
Double assignment
VRL program
.first = .second = "Hello, World!"
Array element assignment
VRL program
.array[1] = "Hello, World!"
Variable assignment
VRL program
my_variable = "Hello, World!"
Resulting event
Hello, World!
Object merge assignment
VRL program
my_variable = {"message": "Hello, World!"}
my_variable |= {"level": "info"}
Resulting event
{
  "level": "info",
  "message": "Hello, World!"
}
Fallible assignment (success)
VRL program
.parsed, .err = parse_json("{\"Hello\": \"World!\"}")
Fallible assignment (error)
VRL program
.parsed, .err = parse_json("malformed")

Block

A block expression is a sequence of one or more expressions within matching brace brackets.

Blocks can’t be empty. Instead, empty blocks ({}) are treated as blank objects.

Grammar

"{" ~ NEWLINE* ~ expressions ~ NEWLINE* ~ "}"
ArgumentMeaning
expressionsOne or more expresions.

Examples

Simple block
VRL program
{
	message = "{\"Hello\": \"World!\"}"
	parse_json!(message)
}
Resulting event
{
  "Hello": "World!"
}
Assignment block
VRL program
.structured = {
	message = "{\"Hello\": \"World!\"}"
	parse_json!(message)
}

Coalesce

A coalesce expression is composed of multiple expressions (operands) delimited by a coalesce operator, short-circuiting on the first expression that doesn’t violate the operator condition.

Grammar

expression ~ (operator ~ expression)+
ArgumentMeaning
expressionThe expression (operand) can be any expression.
operatorThe operator delimits two or more expressions.

Examples

Error coalescing
VRL program
parse_syslog("not syslog") ?? parse_common_log("not common") ?? "malformed"
Resulting event
malformed

Comparison

A comparison expression compares two expressions (operands) and produces a Boolean as defined by the operator.

Grammar

expression ~ operator ~ expression
ArgumentMeaning
expressionThe expression (operand) can be any expression that returns a valid type as defined by the operator.
operatorThe operator defines the operation performed on the left-hand and right-hand side operations.

Examples

Equal
VRL program
1 == 1
Resulting event
true
Not equal
VRL program
1 != 2
Resulting event
true
Greater than or equal
VRL program
2 >= 2
Resulting event
true
Greater than
VRL program
2 > 1
Resulting event
true
Less than or equal
VRL program
2 <= 2
Resulting event
true
Less than
VRL program
1 < 2
Resulting event
true

Function call

A function call expression invokes built-in VRL functions.

Grammar

function ~ abort? ~ "(" ~ arguments? ~ ")"
ArgumentMeaning
abort

abort represents a literal ! that can optionally be used with fallible functions to abort the program when the function fails:

result = f!()

Otherwise, errors must be handled:

result, err = f()

Failure to handle errors from fallible functions results in compile-time errors. See the error reference for more info.

argumentsThe arguments are comma-delimited expressions that can optionally be prefixed with the documented name.
functionfunction represents the name of the built-in function.

Examples

Positional function invocation
VRL program
split("hello, world!", ", ")
Resulting event
[
  "hello",
  "world!"
]
Named function invocation (ordered)
VRL program
split("hello, world!", pattern: ", ")
Resulting event
[
  "hello",
  "world!"
]
Named function invocation (unordered)
VRL program
split(pattern: ", ", value: "hello, world!")
Resulting event
[
  "hello",
  "world!"
]

If

An if expression specifies the conditional execution of two branches according to the value of a Boolean expression. If the Boolean expression evaluates to true, the “if” branch is executed, otherwise the “else” branch is executed (if present).

Grammar

"if" ~ predicate ~ block ~ ("else if" ~ predicate ~ block)* ~ ("else" ~ block)?
ArgumentMeaning
predicateThe predicate must be an expression that resolves to a Boolean. If a Boolean isn’t returned, a compile-time error is raised.

Examples

True if expression
VRL program
if true {
	"Hello, World!"
}
Resulting event
Hello, World!
False if expression
VRL program
if false {
	# not evaluated
	null
}
If/else expression
VRL program
if false {
	# not evaluated
	null
} else {
	"Hello, World!"
}
Resulting event
Hello, World!
If/else if/else expression
VRL program
if false {
	# not evaluated
	null
} else if false {
	# not evaluated
	null
} else {
	"Hello, World!"
}
Resulting event
Hello, World!

Index

An index expression denotes an element of an array. Array indices in VRL start at zero.

Grammar

"[" ~ index ~ "]"
ArgumentMeaning
indexThe index represents the zero-based position of the element.

Examples

Array index expression
Vector event
{
  "log": {
    "array": [
      "first",
      "second"
    ]
  }
}
VRL program
.array[0]
Resulting event
first

Logical

A logical expression compares two expressions (operands), short-circuiting on the last expression evaluated as defined by the operator.

Grammar

expression ~ operator ~ expression
ArgumentMeaning
expressionThe expression (operand) can be any expression that returns a valid type as defined by the operator.
operatorThe operator defines the operation performed on the left-hand- and right-hand-side operations.

Examples

AND
VRL program
true && true
Resulting event
true
OR (boolean)
VRL program
false || "foo"
Resulting event
foo
OR (null)
VRL program
null || "foo"
Resulting event
foo
NOT
VRL program
!false
Resulting event
true

Path

A path expression is a sequence of period-delimited segments that represent the location of a value within an object.

Grammar

"." ~ path_segments
ArgumentMeaning
"."The "." character represents the root of the event. Therefore, all paths must begin with the . character, and . alone is a valid path.
path_segmentspath_segments denote a segment of a nested path. Each segment must be delimited by a . character and only contain alpha-numeric characters and _ (a-zA-Z0-9_). Segments that contain characters outside of this range must be quoted.

Examples

Root path
Vector event
{
  "log": {
    "message": "Hello, World!"
  }
}
VRL program
.
Resulting event
{
  "message": "Hello, World!"
}
Top-level path
Vector event
{
  "log": {
    "message": "Hello, World!"
  }
}
VRL program
.message
Resulting event
Hello, World!
Nested path
Vector event
{
  "log": {
    "parent": {
      "child": "Hello, World!"
    }
  }
}
VRL program
.parent.child
Resulting event
Hello, World!
Nested path coalescing
Vector event
{
  "log": {
    "grand_parent": {
      "parent2": {
        "child": "Hello, World!"
      }
    }
  }
}
VRL program
.grand_parent.(parent1 | parent2).child
Resulting event
Hello, World!
Array element path (first)
Vector event
{
  "log": {
    "array": [
      "first",
      "second"
    ]
  }
}
VRL program
.array[0]
Resulting event
first
Array element path (second)
Vector event
{
  "log": {
    "array": [
      "first",
      "second"
    ]
  }
}
VRL program
.array[1]
Resulting event
second
Quoted path
Vector event
{
  "log": {
    "parent.key.with.special characters": {
      "child": "Hello, World!"
    }
  }
}
VRL program
."parent.key.with.special characters".child
Resulting event
Hello, World!

Variable

A variable expression names variables. A variable is a sequence of one or more letters and digits. The first character in a variable must be a letter.

Grammar

first ~ (trailing)*
ArgumentMeaning
firstThe first character can only be an alpha-numeric character (a-zA-Z0-9).
trailingThe trailing characters must only contain ASCII alpha-numeric and underscore characters (a-zA-Z0-9_).

Examples

Simple variable
VRL program
my_variable = 1
Resulting event
1
Variable with path
VRL program
my_object = { "one": 1 }
my_object.one
Resulting event
1