Redshift

Jitsu supports Redshift as a destination. For more information about Redshift see docs. Redshift destination can work in stream and batch modes. In stream mode Jitsu uses plain insert statements for storing data. In batch mode Jitsu writes incoming events in formatted file on the AWS S3 and uses COPY command to store data from AWS S3 files into Redshift.

Configuration#

Redshift destination config consists of the following schema:

destinations:
  my_redshift:
    type: redshift
    datasource:
      host: redshift.amazonaws.com
      db: my-db
      port: 5439
      schema: myschema
      username: user
      password: pass
      parameters:
        connect_timeout: 300
    s3:
      access_key_id: abc123
      secret_access_key: secretabc123
      bucket: my-bucket
      region: us-west-1
      folder: my_redshift

'datasource' field#

's3' section#

AWS S3#

Create AWS S3 bucket and add IAM permissions for using Redshift in batch mode:

• Create your AWS S3 bucket.
• Create your AWS IAM User and save its credentials: Access key ID and Secret Access Key.
• Create IAM Policy with clear name (e.g. jitsu-redshift-policy) and attach it to created user with the following JSON:

Replace YOUR_BUCKET_ARN with your bucket ARN:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:DeleteObject"
      ],
      "Resource": [
        "arn:aws:s3:::YOUR_BUCKET_ARN",
        "arn:aws:s3:::YOUR_BUCKET_ARN/*"
      ]
    }
  ]
}