This is a Jitsu.Classic documentation. For the lastest version, please visit docs.jitsu.com. Read about differences here.

📜 Configuration

Configuration UI

Jitsu Internals

Jitsu Server

BigQuery

Jitsu supports BigQuery as a destination. For more information about BigQuery see docs. BigQuery destination can work in stream and batch modes. In stream mode Jitsu uses BigQuery Streaming API. In batch mode Jitsu writes incoming events in formatted file on the Google Cloud Storage and creates a loading job to store data from GCP files into BigQuery.

Using BigQuery destination in the batch mode might decrease your Google bill. Read more about BigQuery Pricing.

Configuration

BigQuery destination config consists of the following schema:

destinations:
  my_bigquery:
    type: bigquery
    google:
      gcs_bucket: google_cloud_storage_bucket
      bq_project: big_query_project
      bq_dataset: big_query_dataset
      key_file: path_to_bqkey.json # or json string of key e.g. "{"service_account":...}"

google

Field (*required)TypeDescriptionDefault value
gcs_bucket*stringGoogle cloud storage bucket.-
bq_project*stringBigQuery project.-
bq_datasetstringBigQuery dataset.default
key_file*stringJSON string with Google key or file path to a file.-

Google Cloud Storage

For using BigQuery in batch mode you should configure custom Google Service Account with permissions on Google Cloud Storage bucket and BigQuery:

  • Create Google Service Account
  • Set IAM Permissions (BigQuery Data Owner and BigQuery Job User) to created Google Service Account.
  • Create Google Cloud Storage bucket
  • Give Storage Object Admin permission to your Google Service Account in created bucket.