CLI

Jitsu CLI reads files with events JSONs from local file system and send them to Jitsu via Bulk API where 1 file = 1 HTTP request with synchronous response. Common use case is uploading archive events into new destination. Use a certain --api-key for routing events payload into a certain destination. All big files larger than max chunk size are divided into parts with size smaller or equal to max chunk size configuration and parts are sent separately: 1 part = 1 HTTP request.

Run#

The easiest way to use Jitsu CLI is run it via docker container. There are two key points:

• Mount dir with files to docker container: since Docker has own dirs structure for passing files to the CLI and uploading them to Jitsu you should mount directory with files to the container with flag:

-v <local_path_to_dir>:/home/eventnative/data/upload

• Specify Jitsu Host: since Docker doesn't have access to host machine's localhost and if Jitsu is deployed locally, you should use --host flag with value depend on you OS:

1. for Mac OS: --host http://host.docker.internal:8000
2. for Linux: --host http://172.17.0.1:8000

Command structure:

docker run --rm -it -v <dir_with_files_to_upload>:/home/eventnative/data/upload jitsucom/jitsu <command> [flags] <list of files to upload or bash pattern>

Example:

docker run --rm -it -v /tmp/my_dir_with_files/:/home/eventnative/data/upload jitsucom/jitsu replay --api-key s2s.dai213sad.dasdpwneqe --host http://myhost:8000 '/home/eventnative/data/upload/*'

List of files can be a bash expression with wildcard. All directories in the list will be read recursively.

Usage#

Please use the following command for displaying all flags and usage:

docker run --rm -it jitsucom/jitsu replay -h

State#

Jitsu CLI supports state file to prevent resending the same file on each run. Names of already uploaded files are stored into state file and CLI won't resend them. Make sure that state directory is writable. Giving permissions for writing state file may be needed:

#Ubuntu/Mac OS
chmod -R 777 /tmp/my_dir_with_files/

Use --state flag and make sure that path to state file is mounted to container as well:

docker run --rm -it -v /tmp/my_dir_with_files/:/home/eventnative/data/upload jitsucom/jitsu replay --api-key s2s.dai213sad.dasdpwneqe --state /home/eventnative/data/upload/cli_state.state --host http://myhost:8000 '/home/eventnative/data/upload/*'

In the example state file cli_state.state is in /home/eventnative/data/upload dir which is mounted to the container. Jitsu CLI filters state files from uploading to Jitsu API.

Filtering files by date#

Jitsu CLI supports filtering files by date. CLI will read file names and filter files that contains date in name and this date isn't in start-end interval. --start and --end flags can be used together or separately and should be in YYYY-MM-DD format. --start flag will be treated as the beginning of the day UTC YYYY-MM-DD 00:00:00.000Z and --end flag will be treated as the end of the day UTC YYYY-MM-DD 23:59:59.999Z:

docker run --rm -it -v /tmp/my_dir_with_files/:/home/eventnative/data/upload jitsucom/jitsu replay --api-key s2s.dai213sad.dasdpwneqe --start 2021-01-01 --end 2021-01-08 --state /home/eventnative/data/upload/cli_state.state --host http://myhost:8000 '/home/eventnative/data/upload/*'

Max chunk size#

Max chunk size parameter controls request payload size. If a file is smaller than max chunk size Jitsu CLI will send it as one HTTP request. If a file is larger - CLI will divide the file into parts where 1 part size is smaller or equal max chunk size and send them to Jitsu.

You can override max chunk size configuration with --chunk-size flag and value in bytes:

• chunk size 100KB = 100 * 1024: --chunk-size 102400
• chunk size 10MB = 10 * 1024 * 1024: --chunk-size 10485760

docker run --rm -it -v /tmp/my_dir_with_files/:/home/eventnative/data/upload jitsucom/jitsu replay --api-key s2s.dai213sad.dasdpwneqe --chunk-size 10485760 --state /home/eventnative/data/upload/cli_state.state --host http://myhost:8000 '/home/eventnative/data/upload/*'