Installing and Configuring Agents

This section describes the installation of Foglight for Redshift. Follow the steps in the order mentioned below:

Configuring the AWS Environment

In order to allow full monitoring of Redshift, the agent will require an IAM user with sufficient privileges to execute system queries as well as access to metrics through CloudWatch. Skip any steps in user or policy creation that have already been performed.

IAM Policy Setup

  1. Using the IAM service, create a policy called ClusterCredentialsPermission and use the following configuration
    1. Service – Select Redshift
    2. Actions – In Access Level > Write select GetClusterCredentials. In Permissions Management select CreateClusterUser and JoinGroup
    3. Resources – Select All Resources
  2. Review policy and save

IAM User Setup

  1. Using the IAM service, create an IAM user
  2. Create a name for the user and select the “Programmatic Access Option” (uses access ID and secret access key)
  3. When configuring permissions, select the “Attach existing policies directly” option and search for and add CloudWatchReadOnlyAccess and ClusterCredentialsPermission
  4. Finish creating the user

Creating and Configuring Agents

The Agent Status page can be used to create new agents and configure and manage existing agents. To access the page from the navigation pane, click Administration > Agents > Agent Status.

To create a new agent instance:

Deploy the Redshift agent package to the FglAM before creating the agent if it has not been deployed yet. Click Deploy Agent Package on the Agent Status or Agent Managers page to perform this.

  1. Navigate to Administration > Agents > Agent Status.
  2. Click Create Agent.
  3. Select the hosts to which you want to deploy agent packages.
    Considerations for this may include physical or virtual locality to the monitored instance, allocated resources, or grouping with other agents of the same type or monitored environment.
  4. Click Next.
  5. Select the RedshiftAgent type. Then, select the Specify Name radio button and enter agent instance name. Click Next.
  6. Click Finish.
  7. Once the agent has been created, click the checkbox next to the Redshift agent.
  8. Click Edit Properties.
  9. Select Modify the default properties for this agent.
  10. Edit the agent properties for the Redshift agent instance:
  1. Click the Activate button.

To modify the properties of an existing agent, go to step 3, deactivate the agent, update the configuration, and then reactivate the agent.

Configuring Agent Properties

This is a full list and explanation of the configurable properties of the Foglight for Redshift agent. The Agent Installer Wizard provides access to the essential subset of available properties. To modify other properties or modify the agent configuration after creation, use the Agent Status dashboard.

This section includes the following key areas:

Setting Cluster Connection (mandatory)

The agent requires a connection to the cluster in order to gather information about the cluster and data structure. It is very important that the values provided here exactly match the details specified in the Cluster Properties of the AWS console for this Redshift cluster.

  • Cluster Identifier – The name of the Redshift cluster. Referred to as the cluster identifier in AWS, this is a unique key that identifies the cluster.
  • Region – The EC2 Availability Zone that the cluster was created in. Described as Zone in the Cluster Properties.
  • IP or Hostname – The Endpoint address for the cluster minus the port number.
  • Port – The port number included in the Endpoint address.
  • Database – Name of a database to connect to. All databases will be monitored, but data gathered from system tables and views available to the entire cluster will be through this database.
  • Username – Redshift database user that must have at a minimum SELECT privilege on all system tables and views.
  • Access Key ID – Access Key ID of the IAM user created during pre-configuration or existing user with necessary attached policies.
  • Secret Access Key – Secret access key for the user described above.
  • SSL Mode – Enforced security for making a connection to the Redshift cluster. If require_ssl is set to true for the cluster parameter group, this must be set to at least Enable.

Setting Collection Intervals (optional)

The Collection Interval fields in the agent properties are used to set the sample frequencies. You can turn off a collection by setting the interval to 0. The defaults are set based on the type of data being collected for relevancy. Data gathered from CloudWatch will always be collected at 5-minute intervals.

  • DB List Refresh – Checks existing databases in order to remove collection tasks for dropped databases or schedule new collections for newly created databases.
  • Availability – Checks ability of the agent to connect to and request data from the cluster.
  • Cluster – Retrieves cluster-level data from the database.
  • Tables – Retrieves table structure and statistics from each database.
  • Configuration – Retrieves configuration values from pg_settings.

Setting DB Overrides (optional)

The DB override properties allow you finer-grained control over database collections. The default list is populated with system databases that have restricted use and will cause the agent to generate errors if access attempts are made. If a database is not specified here, it will use the parameters set in the Collection Intervals section.

  • Database Name – Name of the database where collections are being overridden.
  • Ignore Tables – If set to true, table information for this database will not be collected.
  • Interval – This collection interval will override the default Tables setting in the Collection Intervals section for this database.

Setting Statement Tracking (optional)

  • Enable Statement Tracking – Enables or disables use of statement tracking.
  • # of Top Statements – The maximum number of statements for the agent to collect during each sample period.
  • Order By – The statements are sorted in order to gather the # of top statements specified in the previous property. If all statements are being gathered, this field is unimportant. Otherwise, several self-explanatory options are available based on existing or calculated column data in the view.
  • Enable S3 Query Tracking – Enables statement tracking specifically for S3 queries.
  • # of Top S3 Queries – The maximum number of S3 queries for the agent to collect during each sample period.

Setting Options (optional)

  • FMS Time Offset – This option allows conversion of timestamp information from various collections. Timestamp information from Redshift is generally in UTC time. The default value of SYSTEM will automatically convert timestamps based on the offset of the machine that hosts the agent from UTC.