Edit in GitHubLog an issue

Execute feature tests with attributes

Summary of steps#

  1. Enable On-Device Decisioning for your organization
  2. Create an A/B activity
  3. Define your A and B
  4. Add an audience
  5. Set traffic allocation
  6. Set traffic distribution to variations
  7. Set up reporting
  8. Add metrics for tracking KPIs
  9. Implement code to execute feature tests with attributes
  10. Implement code to track conversion events
  11. Activate your feature tests with attributes

NOTE: Suppose you are a retail e-commerce company. You want to increase the conversion rate when customers browse and sort through your catalog of products. You have a hypothesis that certain sorting algorithms and pagination strategies yield better results than others. To test this theory, you decide to run a feature test that involves the redesign of the sorting widget using different sorting options for your end users. You want to ensure this feature test is executed at near-zero latency so it doesn't negatively impact user experiences and skew the results.

Enable On-Device Decisioning for your organization#

Enabling on-device decisioning ensures an A/B activity is executed at near-zero latency. To enable this feature, navigate to Administration > Implementation > Account details in Adobe Target, and enable the On-Device Decisioning toggle.

alt image

NOTE: You must have the Admin or Approver user role to enable or disable the On-Device Decisioning toggle.

After enabling the On-Device Decisioning toggle, Adobe Target begins generating rule artifacts for your client.

Create an A/B activity#

  1. In Adobe Target, navigate to the Activities page, then select Create Activity > A/B test.

    alt image

  2. In the Create A/B Test Activity modal, leave the default Web option selected (1), select Form as your experience composer (2), select Default Workspace with No Property Restrictions (3), and click Next (4).

    alt image

Define your feature flag and attribute values#

  1. In the Experiences step of activity creation, provide a name for your activity (1) and add a second experience, Experience B, by clicking the Add Experience (2) button. Enter the name of the location (3) within your application where you want to execute your feature test with attributes. In the example shown below, product-results-page is the location defined for Experience A. (It is also the location defined for Experience B.)

    alt image

    Experience A will contain the JSON that signals your business logic to do the following:

    • Initiate the sorting algorithm feature via the test_sorting feature flag
    • Execute the recommended sorting algorithm defined in the sorting_algorithm _**_attribute
    • Return 50 products per page as defined by the pagination strategy defined in the pagination_limit
  2. In Experience A, click to change the content from Default Content to the JSON by selecting Create JSON Offer as shown below (1).

    alt image

  3. Define the JSON with test_sorting, sorting_algorithm, and pagination_limit flags and attributes that will be used to initiate the recommended sorting algorithm with a pagination limit of 50 products.

    NOTE: When Adobe Target buckets a user to see Experience A, the JSON with the defined attributes in the example will be returned. In your code, you will need to check the value of the feature flag test_sorting to see whether the sorting feature should be turned on. If so, you will use the recommended value of the sorting_algorithm attribute to show recommended products in the product list view. The limit of products to show for your application will be 50, since that is the value of the pagination_limit attribute.

    alt image

    Experience B will define the JSON that signals your business logic to do the following:

    • Initiate the sorting algorithm feature via the test_sorting feature flag
    • Execute the best_sellers sorting algorithm defined in the sorting_algorithm _**_attribute
    • Return 50 products per page as defined by the pagination strategy defined in the pagination_limit

    NOTE: When Adobe Target buckets a user to see Experience B, the JSON with the defined attributes in the example will be returned. In your code, you will need to check the value of the feature flag test_sorting to see whether the sorting feature should be turned on. If so, you will use the best_sellers value of the sorting_algorithm attribute to show best selling products in the product list view. The limit of products to show for your application will be 50, since that is the value of the pagination_limit attribute.

    alt image

Add an audience#

In the Targeting step, keep the All Visitors audience. This will enable you to understand the impact of your sorting feature, as well as which algorithm and number of items best influence the results.

alt image

Set traffic allocation#

Define the percentage of your visitors against which you want to test your sorting algorithms and pagination strategy. In other words, to what percentage of your users do you want to roll out this test? In this example, to deploy this test to all logged-in users, keep the traffic allocation at 100%.

alt image

Set traffic distribution to variations#

Define the percentage of your visitors that will see the recommended versus the best sellers sorting algorithm, with a limit of 50 products per page. In this example, keep the traffic distribution as a 50/50 split between Experiences A and B.

alt image

Set up reporting#

In the Goals & Settings step, choose Adobe Target as the Reporting Source to view your A/B test results in the Adobe Target UI, or choose Adobe Analytics to view them in the Adobe Analytics UI.

alt image

Add metrics for tracking KPIs#

Choose a Goal Metric to measure the feature test with attributes. In this example, success is based on whether the user purchases a product, depending on the sorting algorithm and pagination strategy they were shown.

Implement feature tests with attributes into your application#

Node.js

Copied to your clipboard
1const TargetClient = require("@adobe/target-nodejs-sdk");
2const options = {
3 client: "testClient",
4 organizationId: "ABCDEF012345677890ABCDEF0@AdobeOrg",
5 decisioningMethod: "on-device",
6 events: {
7 clientReady: targetClientReady
8 }
9};
10const targetClient = TargetClient.create(options);
11
12function targetClientReady() {
13 return targetClient.getAttributes(["product-results-page"]).then(function(attributes) {
14 const test_sorting = attributes.getValue("product-results-page", "test-sorting");
15 const sorting_algorithm = attributes.getValue("product-results-page", "sorting_algorithm");
16 const pagination_limit = attributes.getValue("product-results-page", "pagination_limit");
17 });
18}

Java

Copied to your clipboard
1import com.adobe.target.edge.client.ClientConfig;
2import com.adobe.target.edge.client.TargetClient;
3import com.adobe.target.delivery.v1.model.ChannelType;
4import com.adobe.target.delivery.v1.model.Context;
5import com.adobe.target.delivery.v1.model.ExecuteRequest;
6import com.adobe.target.delivery.v1.model.MboxRequest;
7import com.adobe.target.edge.client.entities.TargetDeliveryRequest;
8import com.adobe.target.edge.client.model.TargetDeliveryResponse;
9
10ClientConfig config = ClientConfig.builder()
11 .client("testClient")
12 .organizationId("ABCDEF012345677890ABCDEF0@AdobeOrg")
13 .build();
14TargetClient targetClient = TargetClient.create(config);
15MboxRequest mbox = new MboxRequest().name("product-results-page").index(0);
16TargetDeliveryRequest request = TargetDeliveryRequest.builder()
17 .context(new Context().channel(ChannelType.WEB))
18 .execute(new ExecuteRequest().mboxes(Arrays.asList(mbox)))
19 .build();
20Attributes attributes = targetClient.getAttributes(request, "product-results-page");
21String testSorting = attributes.getString("product-results-page", "test-sorting");
22String sortingAlgorithm = attributes.getString("product-results-page", "sorting_algorithm");
23String paginationLimit = attributes.getString("product-results-page", "pagination_limit");

Implement code to track conversion events#

Node.js

Copied to your clipboard
1//... Code removed for brevity
2
3//When a conversion happens
4TargetClient.sendNotifications({
5 targetCookie,
6 "request" : {
7 "notifications" : [
8 {
9 type: "click",
10 timestamp : Date.now(),
11 id: "conversion",
12 mbox : {
13 name : "product-results-page"
14 }
15 }
16 ]
17 }
18})

Java

Copied to your clipboard
1ClientConfig config = ClientConfig.builder()
2 .client("acmeclient")
3 .organizationId("1234567890@AdobeOrg")
4 .build();
5TargetClient targetClient = TargetClient.create(config);
6
7Context context = new Context().channel(ChannelType.WEB);
8
9ExecuteRequest executeRequest = new ExecuteRequest();
10
11NotificationDeliveryService notificationDeliveryService = new NotificationDeliveryService();
12
13Notification notification = new Notification();
14notification.setId("conversion");
15notification.setImpressionId(UUID.randomUUID().toString());
16notification.setType(MetricType.CLICK);
17notification.setTimestamp(System.currentTimeMillis());
18notification.setTokens(
19 Collections.singletonList(
20 "IbG2Jz2xmHaqX7Ml/YRxRGqipfsIHvVzTQxHolz2IpSCnQ9Y9OaLL2gsdrWQTvE54PwSz67rmXWmSnkXpSSS2Q=="));
21
22TargetDeliveryRequest targetDeliveryRequest =
23 TargetDeliveryRequest.builder()
24 .context(context)
25 .execute(executeRequest)
26 .notifications(Collections.singletonList(notification))
27 .build();
28
29TargetDeliveryResponse offers = targetClient.getOffers(request);
30notificationDeliveryService.sendNotification(request);
31
32Attributes attributes = targetClient.getAttributes(request, "product-results-page");
33String testSorting = attributes.getString("product-results-page", "test-sorting");
34String sortingAlgorithm = attributes.getString("product-results-page", "sorting_algorithm");
35String paginationLimit = attributes.getString("product-results-page", "pagination_limit");

Activate your feature tests with attributes#

alt image

Was this helpful?
  • Privacy
  • Terms of Use
  • Do not sell my personal information
  • AdChoices
Copyright © 2022 Adobe. All rights reserved.