Setting up Marketo Lead Activity Data Stream with Adobe I/O Events

These instructions describe how to set up and get started using Adobe I/O Events for Marketo lead-based activities and change events.

Introduction

The Lead Activity Data Stream provides near real-time streaming of audit tracking Lead Activities where large volumes of Lead Activities can be sent to a customer’s external system. Streams enable customers to effectively audit Lead-related events, usage patterns, provide views into Lead changes and trigger processes and workflows based upon the different types of Lead Events.

Setting up the Data Stream in Marketo

The following are required before you can subscribe to this data stream:

• Marketo Engage subscription that is IMS-enabled
• Marketo Engage subscription provisioned with a Performance or Ultimate tier package
• Developer or System Admin account in Adobe Experience Cloud

Please contact your Adobe representative with your Munchkin ID and request to enable this data stream.

Setup Adobe I/O

See Getting Started with Adobe I/O Events

For basic instructions for this use case, starting from console.adobe.io:

When prompted, click the designated button to proceed:

• Select Create new project

  

• Select Add event

  

• Filter by Experience Cloud

• Select Marketo Lead Activity Data Stream

  

• Subscribe to the lead-driven change events of your choosing

  

• Set up OAuth Server-to-Server Credentials

  

• Set up Event Registration

  

  • Provide a name and description for this event subscription

  • Optionally choose whether to enable Webhook or Runtime action

    • Enable Webhook

      • We recommend batch over single webhooks
      • For Webhook URL a public https endpoint must be provided
      • The endpoint must be able to handle get and post requests
      • The get request must respond with the challenge query if it exists
      • The post request must respond that it received the message or the webhook will re-attempt to send several times before giving up and automatically disabling the webhook sends

    • Enable Runtime action

      • See Setting up your Runtime Environment
      • Select a pre-made runtime action/runtime namespace

• After Saving

  

  • Verify that the Status is Active
  • If Webhook was selected, verify that it successfully passed the challenge without errors

Developer Guidelines

When setting up a project to subscribe to events, there are three ways to interact with those event subscriptions in order to receive the events. The first is Journaling, which provides a pull model in which events can be pulled via API and stores up to 7 days of past events. The second is Webhooks, which can be configured to send events either as single events or batched to a webhook endpoint in near real-time with the event occurrence. Third is Runtime, where you can set up your own custom function within Adobe that events will automatically run through near-real time.

Journaling

Getting Started with Journaling

Important Takeaways:

• Stores up to 7 days of history
• Can be iterated through from any previous event within the history
• Will still receive and store events even if webhook is failing
• Useful for fetching events that were missed due to webhook issues or for a pulling mechanism instead of webhook push

Webhooks

Getting Started with Adobe I/O Events Webhooks

Webhook Endpoint Requirements:

• Handle GET and POST requests

• Respond with a 200-type response within a reasonable time period

• Challenge Request

  • GET request with challenge query parameter
  • Must respond with value of challenge query parameter

• Webhook Events

  • POST request with JSON data body with one or more events
  • Recommended to set up webhook as batch

Multiple Marketo Instances

If you have multiple Marketo instances which fall under a single Adobe Organization (such as a production and a sandbox instance), all subscribed events from every enabled instance will appear in the data stream. You can identify which instance the event came from by inspecting the munchkinId field in the event payload.

Event Data Structure

Events are structured in JSON format using the CloudEvents spec

Example Event (batch):

[
  {
    "eventid": "ec2c6092-1e94-45e9-b2dd-0b6bad0baa6f",
    "specversion": "1.0",
    "type": "com.adobe.platform.marketo.activity.standard.openemail",
    "source": "urn:marketo_activity_stream",
    "id": "a1764870-4f4b-4f8c-82d6-59b0fff2d39c",
    "time": "2024-12-17T18:39:33Z",
    "datacontenttype": "application/json",
    "data": {
      "munchkinId": "123-ABC-456",
      "leadId": "1234",
      "activityDate": "2024-12-17T18:39:33Z",
      "activityTypeId": 10,
      "activityType": "Open Email",
      "activityLogItemId": 1234567890,
      "primaryAttributeValueId": 1234,
      "primaryAttributeValue": "Attribute Value",
      "attributes": [
        {
          "name": "Campaign Run ID",
          "dataType": "integer",
          "value": 1234
        },
        {
          "name": "Platform",
          "dataType": "string",
          "value": "Platform Name"
        },
        {
          "name": "Device",
          "dataType": "string",
          "value": "Device Name"
        },
        {
          "name": "Mailing ID",
          "dataType": "object",
          "value": 1234
        },
        {
          "name": "User Agent",
          "dataType": "string",
          "value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
        },
        {
          "name": "Is Mobile Device",
          "dataType": "boolean",
          "value": false
        }
      ]
    },
    "recipientclientid": "<your_client_id>"
  },
  {
    "eventid": "97980214-3c8f-4964-88a3-75b42833a1f9",
    "specversion": "1.0",
    "type": "com.adobe.platform.marketo.activity.standard.visitwebpage",
    "source": "urn:marketo_activity_stream",
    "id": "a434d7ee-fe91-4b96-aaf2-61f1c6f4c46e",
    "time": "2024-12-17T18:42:49Z",
    "datacontenttype": "application/json",
    "data": {
      "munchkinId": "123-ABC-456",
      "leadId": "1234",
      "activityDate": "2024-12-17T18:42:49Z",
      "activityTypeId": 1,
      "activityType": "Visit Webpage",
      "activityLogItemId": 1234567890,
      "primaryAttributeValueId": 1234,
      "primaryAttributeValue": "Attribute Value",
      "attributes": [
        {
          "name": "Client IP Address",
          "dataType": "string",
          "value": "11.22.33.44"
        },
        {
          "name": "Query Parameters",
          "dataType": "string",
          "value": ""
        },
        {
          "name": "Referrer URL",
          "dataType": "string",
          "value": ""
        },
        {
          "name": "User Agent",
          "dataType": "string",
          "value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
        },
        {
          "name": "Webpage ID",
          "dataType": "object",
          "value": 1234
        },
        {
          "name": "Webpage URL",
          "dataType": "string",
          "value": "/lp/123-ABC-456/MyLandingPage.html"
        }
      ]
    },
    "recipientclientid": "<your_client_id>"
  }
]

Example Event (single):

{
  "eventid": "4109a990-da28-4c5b-bad9-22df8903ed9a",
  "specversion": "1.0",
  "type": "com.adobe.platform.marketo.activity.standard.clicklink",
  "source": "urn:marketo_activity_stream",
  "id": "570b455e-203f-4983-91a8-5a03f597fda5",
  "time": "2024-12-17T18:43:56Z",
  "datacontenttype": "application/json",
  "data": {
    "munchkinId": "123-ABC-456",
    "leadId": "1234",
    "activityDate": "2024-12-17T18:43:56Z",
    "activityTypeId": 3,
    "activityType": "Click Link",
    "activityLogItemId": 1234567890,
    "primaryAttributeValueId": 1234,
    "primaryAttributeValue": "Attribute Value",
    "attributes": [
      {
        "name": "Link ID",
        "dataType": "object",
        "value": 1234
      },
      {
        "name": "Client IP Address",
        "dataType": "string",
        "value": "11.22.33.44"
      },
      {
        "name": "Query Parameters",
        "dataType": "string",
        "value": ""
      },
      {
        "name": "Referrer URL",
        "dataType": "string",
        "value": "https://marketo.com/lp/123-ABC-456/MyLandingPage.html"
      },
      {
        "name": "User Agent",
        "dataType": "string",
        "value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
      },
      {
        "name": "Webpage ID",
        "dataType": "object",
        "value": 1234
      }
    ]
  },
  "recipientclientid": "<your_client_id>"
}

Data Field Definitions

Many of the fields are common across the different types of events. The data object will contain the specific details of the event. In particular, the attributes array will contain a subset of all possible attributes (or none at all!) depending on the event type, and the event context.

The type field will always be the Event Type name, all lowercase with spaces removed, appended to the string "com.adobe.platform.marketo.activity.standard". For example, for the "Open Email" Event Type, the type string will be "com.adobe.platform.marketo.activity.standard.openemail".

All customer-defined activity types will be delivered with an identical event type "com.adobe.platform.marketo.activity.custom", with no suffix.

The data field contains the following data:

And finally, all Attribute objects in the attributes array will contain the following fields:

Some events may include additional metadata with their response. You are always able to emit a sample event for every event type from within the Adobe Developer Console to see an example of the event structure.

Event List

These are all of the currently supported Lead Activity Event Types. If you have a "firehose" endpoint configured that is subscribed to more than one event type, you can identify the type of event in three different ways.

Here are examples using the "Open Email" event type:

1. Text match on the activityType field under the data object (e.g. payload.data.activityType = "Open Email")
2. Number match on the actitityTypeId field under the data object (e.g. payload.data.activityTypeId = 10)
3. Text match on the type field with the concatenated, lower-cased Event Type (e.g. payload.type = "com.adobe.platform.marketo.activity.standard.openemail")

Debug

Debug Tracing

Once you have successfully completed your setup and event subscription registration, events should start being stored in the journal. In addition, if you have webhooks or runtime set up, the events will go through those flows. From the project's page in the event registration details, you should see a tab for Debug Tracing. For webhooks, this will show a record of failed and successful challenge attempts as well as webhook attempts. Each request includes the request/response details to help debug.

There is also an Event Browser tab which lets you manually look at events in the Journal from the Developer Console UI