aws_cloudwatch_metrics sink

Streams `metric` events to AWS CloudWatch Metrics via the `PutMetricData` API endpoint.

The aws_cloudwatch_metrics sink is in beta. Please see the current enhancements and bugs for known issues. We kindly ask that you add any missing issues as it will help shape the roadmap of this component.

The aws_cloudwatch_metrics sink streams metric events to AWS CloudWatch Metrics via the PutMetricData API endpoint.

Config File

vector.toml (simple)
vector.toml (advanced)
[sinks.my_sink_id]
type = "aws_cloudwatch_metrics" # must be: "aws_cloudwatch_metrics"
inputs = ["my-source-id"]
namespace = "service"
region = "us-east-1"
# For a complete list of options see the "advanced" tab above.

Options

Key

Type

Description

REQUIRED

type

string

The component type required must be: "aws_cloudwatch_metrics"

inputs

[string]

A list of upstream source or transform IDs. See Config Composition for more info. required example: ["my-source-id"]

namespace

string

A namespace that will isolate different metrics from each other. required example: "service"

region

string

The AWS region of the target CloudWatch stream resides. required example: "us-east-1"

OPTIONAL

endpoint

string

Custom endpoint for use with AWS-compatible services. no default example: "127.0.0.0:5000"

healthcheck

bool

Enables/disables the sink healthcheck upon start. See Health Checks for more info. default: true

How It Works

Authentication

Vector checks for AWS credentials in the following order:

  1. Environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

  2. The credential_process command in the AWS config file. (usually located at ~/.aws/config)

  3. The AWS credentials file. (usually located at ~/.aws/credentials)

  4. The IAM instance profile. (will only work if running on an EC2 instance with an instance profile/role)

If credentials are not found the healtcheck will fail and an error will be logged.

Obtaining an access key

In general, we recommend using instance profiles/roles whenever possible. In cases where this is not possible you can generate an AWS access key for any user within your AWS account. AWS provides a detailed guide on how to do this.

Delivery Guarantee

This component offers an at least once delivery guarantee if your pipeline is configured to achieve this.

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Health Checks

Health checks ensure that the downstream service is accessible and ready to accept data. This check is performed upon sink initialization.

If the health check fails an error will be logged and Vector will proceed to start. If you'd like to exit immediately upon health check failure, you can pass the --require-healthy flag:

vector --config /etc/vector/vector.toml --require-healthy

And finally, if you'd like to disable health checks entirely for this sink you can set the healthcheck option to false.

Metric Types

CloudWatch Metrics types are organized not by their semantics, but by storage properties:

  • Statistic Sets

  • Data Points

In Vector only the latter is used to allow lossless statistics calculations on CloudWatch side.

The following matrix outlines how Vector metric types are mapped into CloudWatch metrics types.

Vector Metrics

CloudWatch Metrics

Counter

Data Point

Gauge

Data Point

Gauge Delta [1]

Data Point

Histogram

Data Point

Set

N/A

  1. Gauge values are persisted between flushes. On Vector start up each gauge is assumed to have

    zero (0.0) value, that can be updated explicitly by the consequent absolute (not delta) gauge

    observation, or by delta increments/decrements. Delta gauges are considered an advanced feature

    useful in distributed setting, however it should be used with care.

Streaming

The aws_cloudwatch_metrics sink streams data on a real-time event-by-event basis. It does not batch data.

Troubleshooting

The best place to start with troubleshooting is to check the Vector logs. This is typically located at /var/log/vector.log, then proceed to follow the Troubleshooting Guide.

If the Troubleshooting Guide does not resolve your issue, please:

  1. If encountered a bug, please file a bug report.

  2. If encountered a missing feature, please file a feature request.

  3. If you need help, join our chat/forum community. You can post a question and search previous questions.

Resources