This is a Jitsu.Classic documentation. For the lastest version, please visit docs.jitsu.com. Read about differences here.
Events Cache
Jitsu supports caching last events in storage (currently only Redis is supported). All income events will be stored in meta storage as well as processed events with DB data types and errors if occurred. Default cache size is 100 events per destination. It is configured in server.cache.events.size.
This feature requires meta.storage configuration.
server:
cache:
#true by default. If false Jitsu doesn't cache any events
enabled: true
events:
#default value. Amount of events which are saved per token or destination.
#If there are more events, old ones will be replaced with new ones.
size: 100
#by default Jitsu applies rate limiting for not saving all events into the cache.
#Jitsu saves in cache ${size} events per ${time_window_sec}. Other events are skipped.
time_window_sec: 60
#Every ${trim_interval_ms} value Jitsu will delete events from cache
#if size of the cache collection by token or destination is greater than ${size}
trim_interval_ms: 500
#When Jitsu receives malformed event JSON, event will be cut to ${max_malformed_event_size_bytes} bytes (10KB) and saved to cache
max_malformed_event_size_bytes: 10000
pool:
#number of goroutines which are process events cache
size: 10
destinations:
...
meta:
storage:
redis:
host: redis_host
port: 6379
password: secret_passwordEndpoint#
Jitsu has an endpoint that return last events from cache. Last event structure:
| Field | Type | Description |
|---|---|---|
| malformed | string | Malformed event as a string in case when Jitsu can't parse input JSON events because it is not valid JSON |
| original | json object | Raw event. |
| success | json object | Successfully processed event (after enrichments, flattening, mappings, typecasting). Contains destination_id, table_name, and array of fields where every field contains name, type, value. |
| error | string | Error if occurred. |
GET/api/v1/events/cache#
Method returns last events per destination in time interval with limit
limit
optional
Query String Parameter
integer
Limit per destination
ids
optional
Query String Parameter
string
Comma-separated api keys or destination ids array
namespace
optional
Query String Parameter
string
Namespace of the events. Can be destination or token constants. Default value is destination.
status
optional
Query String Parameter
string
Events status filter. If no status provided all events will be returned. Available value is 'error'. If error provided - only events with error status will be returned.
Response#
Return last events with DB schema and errors
{
"last_minute_limited": 0,
"cache_capacity_per_interval": 100,
"interval_seconds": 60,
"total_events": 95,
"response_events": 2,
"events": [
{
"original": {
"api_key": "api_secret",
"event_type": "pageview",
"eventn_ctx": {
"click_id": {},
"event_id": "dna3b213b",
"ids": {},
"local_tz_offset": "0",
"page_title": "Jitsu: Open-source data integration and event collection",
"referer": "https://referer-site.com/",
"url": "https://jitsu.com/",
"user": {
"anonymous_id": "enoq4o12"
},
"user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0",
"utc_time": "2020-11-30T21:48:12.754000Z",
"utm": {}
},
"eventn_data": {},
"src": "eventn"
},
"error": "Unsupported column [local_tz_offset] type changing from: INT64 to: STRING"
},
{
"original": {
"api_key": "api_secret",
"event_type": "pageview",
"eventn_ctx": {
"click_id": {},
"event_id": "213ndisand",
"ids": {},
"local_tz_offset": 0,
"page_title": "Jitsu: Open-source data integration and event collection",
"referer": "https://referer-site.com/",
"url": "https://jitsu.com/",
"user": {
"anonymous_id": "dano2131"
},
"user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0",
"utc_time": "2020-11-30T21:48:12.754000Z",
"utm": {}
},
"eventn_data": {},
"src": "eventn"
},
"success": {
"destination_id": "my_destination",
"table": "jitsu_events",
"record": [
{
"field": "eventn_ctx_utc_time",
"type": "timestamp",
"value": "2020-11-30T21:48:12.754000Z"
},
{
"field": "eventn_ctx_user_anonymous_id",
"type": "character varying(8192)",
"value": "dano2131"
},
{
"field": "eventn_ctx_page_title",
"type": "character varying(8192)",
"value": "Jitsu: Open-source data integration and event collection"
},
{
"field": "eventn_ctx_location_latitude",
"type": "numeric(38,18)",
"value": "50.8757"
},
{
"field": "eventn_ctx_location_city",
"type": "character varying(8192)",
"value": "Verwood"
},
{
"field": "_timestamp",
"type": "timestamp",
"value": "2020-11-30T21:48:13.609858Z"
},
{
"field": "eventn_ctx_parsed_ua_ua_family",
"type": "character varying(8192)",
"value": "Firefox"
},
{
"field": "eventn_ctx_parsed_ua_ua_version",
"type": "character varying(8192)",
"value": "78.0."
},
{
"field": "eventn_ctx_parsed_ua_os_family",
"type": "character varying(8192)",
"value": "Linux"
},
{
"field": "eventn_ctx_event_id",
"type": "character varying(8192)",
"value": "213ndisand"
},
{
"field": "eventn_ctx_referer",
"type": "character varying(8192)",
"value": "https://referer-site.com/"
},
{
"field": "eventn_ctx_location_zip",
"type": "character varying(8192)",
"value": "BH31"
},
{
"field": "eventn_ctx_location_region",
"type": "character varying(8192)",
"value": "ENG"
},
{
"field": "eventn_ctx_location_country",
"type": "character varying(8192)",
"value": "GB"
},
{
"field": "src",
"type": "character varying(8192)",
"value": "eventn"
},
{
"field": "event_type",
"type": "character varying(8192)",
"value": "pageview"
},
{
"field": "eventn_ctx_local_tz_offset",
"type": "bigint",
"value": "0"
},
{
"field": "eventn_ctx_user_agent",
"type": "character varying(8192)",
"value": "Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0"
},
{
"field": "eventn_ctx_url",
"type": "character varying(8192)",
"value": "https://jitsu.com/"
},
{
"field": "source_ip",
"type": "character varying(8192)",
"value": "86.149.97.182"
},
{
"field": "api_key",
"type": "character varying(8192)",
"value": "api_secret"
},
{
"field": "eventn_ctx_location_longitude",
"type": "numeric(38,18)",
"value": "-1.8721"
}
]
}
}
]
}