This is a Jitsu.Classic documentation. For the lastest version, please visit docs.jitsu.com. Read about differences here.

📜 Configuration

Configuration UI

Jitsu Internals

Jitsu Server

Snowflake

Jitsu supports Snowflake as a destination. For more information about Snowflake see docs. Snowflake destination can work in stream and batch modes. In stream mode Jitsu uses plain insert statements for storing data. In batch mode Jitsu writes incoming events in formatted file on the AWS S3 or Google Cloud Storage (intermediate Stage layer) and uses COPY command to store data from Stage files into Snowflake.

Please carefully read Snowflake docs regarding System Usage & Billing, especially the Virtual Warehouse Credit Usage part.
Remember that using Snowflake destination in stream mode is highly likely to be charged as full-time warehouse usage.
For batch mode, we recommend enabling Automating Warehouse Suspension using ALTER WAREHOUSE query. E.g.:
ALTER WAREHOUSE WAREHOUSE_NAME set AUTO_SUSPEND = 1

Configuration

Snowflake destination in batch mode can be configured via S3 or Google Cloud Storage. In the stream mode, it can be configured without any. The config consists of the following schema:

destinations:
  my_snowflake_via_aws_s3_or_gcp:
    type: snowflake
    snowflake:
      account: hha56552.us-east-1
      schema: MYSCHEMA
      warehouse: my_warehouse
      db: mydb
      username: user
      password: pass
      parameters:
        name: value
      stage: my_stage
# via s3
    s3:
      access_key_id: ...
      secret_access_key: ...
      bucket: ...
      region: ...
# or via gcp
    google:
      gcs_bucket: ...
      key_file: ...

snowflake section

Field (*required)TypeDescriptionDefault value
account*stringSnowflake Account from URL https://"SNOWFLAKE_ACCOUNT".snowflakecomputing.com/-
portintPort of destination.443
db*stringDatabase of destination.-
schemastringSchema of destination.PUBLIC
username*stringUsername for authorization in a destination.-
passwordstringPassword for authorization in a destination.-
warehouse*stringSnowflake warehouse name.
parametersobjectConnection parameters.client_session_keep_alive=true
stage**stringName of Snowflake stage. It is required in batch mode.-

s3 section

S3 configuration
/docs/destinations-configuration/s3

google section

Google (GCS) configuration
/docs/destinations-configuration/bigquery

AWS S3 Stage

For using Snowflake in batch mode with AWS S3 stage you should create AWS S3 bucket and add IAM permissions:

  • Create your AWS S3 bucket.
  • Create your AWS IAM User and save its credentials: Access key ID and Secret Access Key.
  • Create IAM Policy with clear name (e.g. jitsu-snowflake-policy) and attach it to created user with the following JSON:

Replace YOUR_BUCKET_ARN with your bucket ARN:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:DeleteObject"
      ],
      "Resource": [
        "arn:aws:s3:::YOUR_BUCKET_ARN",
        "arn:aws:s3:::YOUR_BUCKET_ARN/*"
      ]
    }
  ]
}

Google Cloud Storage Stage

For using Snowflake in batch mode with Google Cloud Storage stage you should create a GCP bucket, an Integration and Stage between Snowflake and GCP bucket. Also, you should configure IAM permissions in created by Snowflake Google Service Account.