ObserveNow
...
Integrations
Cloud Providers

Amazon Web Services (AWS)

Collecting AWS CloudWatch Logs

Logs generated by AWS managed services like RDS, ElastiCache, MKS etc are available only in CloudWatch Logs. These logs can be pulled into the OpsVerse Observability stack using the log-shipper lambda function.

This enables you to bring all your logs to one central system.

This lambda function is delivered as a terraform script. Please download this zip file and unzip it on a computer from where you can run terraform.

Structure

When unzipped, it should look like this:

Shell


Running the Lambda

To run the lambda, please provide the necessary variables present in the variables.tf. Run the lambda script using the following two commands -

Shell


Here's a sample run:

Shell


Please run once for each region you need this for. The first time you run it in your account, make sure to have create_execution_role = true set. Remove this in subsequent runs of different regions in the same account (because IAM resources are global, not region-specific - there's no need to re-create them in subsequent runs).

Summary of What It Creates

  • A lambda function (log-shipper) by pulling our function from S3. The supported AWS regions are us-east-1, us-west-2, eu-central-1 and ap-south-1
  • The corresponding roles and policies to allow the function to both:
    • Allow the function specific permissions needed on the log groups passed in
    • create its own log group (for debug/output)
    • A subscription (trigger) of each of the log groups to the newly created Lambda function# AWS CloudWatch Metrics


Collecting AWS CloudWatch Metrics

Metrics generated by AWS managed services like RDS, ElastiCache, MKS etc are available only in AWS CloudWatch. OpsVerse agent can pull those metrics and bring them into the same metrics backend enabling you to visualize all your metrics in one place.

Tag your AWS resources

Set the following tags in AWS for the resources you want to monitor. This allows you to easily identify the AWS metrics from your resources and correlate them with the rest of the telemetry data.

  • RDS :
    • Key: opsverse-database-name
    • Value:
  • CloudFront:
    • Key: opsverse-monitor
    • Value:

Setting Up Authentication

Following types of authentication can be set up for the cloudwatch agent:

  • Configure using an aws role
  • Configure using aws_access_key_id and aws_secret_access_key
  • Configure using pre-created secret in which AWS credentials are stored.
  • Service Account based authentication

Using AWS access and secret keys

Using existing secret

Service Account based Auth

Configuring values.yaml and Installing the chart

Copy the template in a cloudwatch-values.yaml named file and configure the values appropriately.

If using aws role, aws access keys, precreated secret or the service account for authentication, update the values file accordingly.

Only one of the authentication methods needs to be configured. Please remove the other authentication configurations from the values.



YAML


Run the following command in a Kubernetes cluster:

Shell