Bulk Data Insertion API endpoints

Once you have met all prerequisites, have a correctly formatted file, and have checked your file for syntax errors with our validation tool, you can begin making API calls to Adobe's ingestion endpoint for this API. For the complete machine-readable specification of every endpoint, see the Bulk Data Insertion API reference.

data-variant=info

data-slots=text

Primary ingestion endpoint

The file ingestion endpoint is found here.

POST https://analytics-collection.adobe.io/aa/collect/v1/events

Include all of the required headers with each API call:

• Authorization: Required for authentication with the API. Format is the string "Bearer {ACCESS_TOKEN}".
• x-api-key: Required for authentication with the API. This is the Client ID (API key) from your OAuth Server-to-Server credential in the Adobe Developer Console.
• x-adobe-vgid: Required for this endpoint. The visitor group ID. See Visitor groups.
• x-adobe-idempotency-key: Optional. Allows you to store a reference to your unique file identifier for the upload. This value can be used for duplication protection in cases when your request does not return a response from Adobe.

You must also add the file, which should be compressed in gzip format, and included as multipart/form-data.

A cap of one request per second is enforced for all ingest API calls. If your implementation exceeds this limit, consolidate your data into fewer, larger files so that your rate stays under the cap. See Visitor groups for guidance on sizing files and parallelizing uploads.

The https://analytics-collection.adobe.io domain automatically routes your API call to the best region for processing. However, if you are legally required to have your data processed in a specific part of the world, you can use one of the following regional host names for API calls:

• US processing: https://analytics-collection-us.adobe.io
• European processing: https://analytics-collection-eu.adobe.io

data-slots=heading, code

data-repeat=2

data-languages=CURL,JSON

curl -X POST -H "accept: application/json" \
    -H "Authorization: Bearer {ACCESS_TOKEN}" \
    -H "x-api-key: {CLIENTID}" \
    -H "x-adobe-vgid: example_group" \
    -F file=@/tmp/ingest_file.gz \
    "https://analytics-collection.adobe.io/aa/collect/v1/events"

{
  "file_id": "5c04f400-fca5-420c-9c36-b94aaec55e69",
  "visitor_group_id": "example_group",
  "size": 355600,
  "received_date": 1506553535,
  "rows": 10000,
  "invalid_rows": 42,
  "upload_name": "ingest_file.gz",
  "status": "string",
  "status_code": "UPLOADED",
  "processing_log": "string",
  "idempotency_key": "5c04f400-fca5-420c-9c36-b94aaec55e69"
}

Response object details

When you upload a file to this endpoint, Adobe returns a JSON object that can contain the following fields:

Validate

Before uploading your first file, Adobe strongly recommends running the file through the validation endpoint. You should not send every file to this endpoint. This endpoint exists to validate file format before you begin uploading files to the events endpoint. Files uploaded to this endpoint are not stored on Adobe's servers or processed. This API is synchronous and returns an immediate reply that states the file's validation status. If a file fails validation, the reason is also returned. See Troubleshoot bulk data uploads for more information.

POST https://analytics-collection.adobe.io/aa/collect/v1/events/validate

data-slots=heading, code

data-repeat=3

data-languages=CURL,JSON,JSON

curl -X POST -H "accept: application/json" \
    -H "Authorization: Bearer {ACCESS_TOKEN}" \
    -H "x-api-key: {CLIENTID}" \
    -H "x-adobe-vgid: example_group" \
    -F file=@/tmp/ingest_file.gz \
    "https://analytics-collection.adobe.io/aa/collect/v1/events/validate"

{
    "success": "File is valid"
}

{
    "error": "File has 2 rows that do not conform to the required CSV format! (Ex: row #59)"
}

Idempotency key lookup

The API offers duplication protection through the use of an idempotency_key. This unique value is generated on the client side, then passed in through the x-adobe-idempotency-key header field. If your POST request submission does not return a response, you can call this lookup endpoint to verify whether the file was received by Adobe.

If you do not include an idempotency key when creating a file upload, it defaults to the file_id generated when uploading a file. Since this idempotency key is generated after a file is uploaded, automatically generated idempotency keys lose their effectiveness in preventing duplicate uploads.

GET https://analytics-collection.adobe.io/aa/collect/v1/events/key/{idempotency_key}

data-slots=heading, code

data-repeat=3

data-languages=CURL,JSON,JSON

curl -X GET -H "accept: application/json" \
    -H "Authorization: Bearer {ACCESS_TOKEN}" \
    -H "x-api-key: {CLIENTID}" \
    "https://analytics-collection.adobe.io/aa/collect/v1/events/key/{IDEMPOTENCY_KEY}"

{
    "file_id": "22eea6b6-eecf-4d13-b70f-abbb8a81efa2",
    "size": 1687458,
    "received_date": 1650323049,
    "rows": 10212,
    "idempotency_key":"{IDEMPOTENCY_KEY}"
}

{
    "error": "File not found"
}

When does data appear?

For data with timestamps less than 24 hours old, ingested data typically follows a standard latency of 20-50 minutes. Some data, such as page views, is available within minutes.

Data with timestamps older than 24 hours can take two hours or longer, depending on the age of the data. See Features that depend on latency for more information.