Test results

Submissions go through several reviews before being released on the Commerce Marketplace. You can use the following endpoints to get more information about the current status of a submission. See Get package details for details about these endpoints.

GET /rest/v1/products/packages/:submission_id/status
GET /rest/v1/products/packages/sku/:url_encoded_sku/status
GET /rest/v1/products/packages/item/:item_id/status
GET /rest/v1/products/packages/:submission_id/status/:tool_name/:tool_run_id
Copied to your clipboard
GET /rest/v1/products/packages/:submission_id/status
GET /rest/v1/products/packages/sku/:url_encoded_sku/status
GET /rest/v1/products/packages/item/:item_id/status
GET /rest/v1/products/packages/:submission_id/status/:tool_name/:tool_run_id

In this example, submission f4eacd72be failed the marketing review. Use this request to get a summary report of the review process:

Request

Response

curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/products/packages/f4eacd72be/status
Copied to your clipboard
curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/products/packages/f4eacd72be/status

[
    {
        "tool_run_id": "5",
        "name": "phpcpd",
        "status": "pass",
        "date": "2020-01-07 18:43:32"
    },
    {
        "tool_run_id": "5",
        "name": "phpcs",
        "status": "pass",
        "date": "2020-01-07 18:52:25"
    },
    {
        "tool_run_id": "5",
        "name": "varnish",
        "status": "pass",
        "date": "2020-01-07 18:45:03"
    },
    {
        "tool_run_id": "6",
        "name": "marketing",
        "status": "fail",
        "date": "2020-01-08 16:57:55"
    }
]
Copied to your clipboard
[
    {
        "tool_run_id": "5",
        "name": "phpcpd",
        "status": "pass",
        "date": "2020-01-07 18:43:32"
    },
    {
        "tool_run_id": "5",
        "name": "phpcs",
        "status": "pass",
        "date": "2020-01-07 18:52:25"
    },
    {
        "tool_run_id": "5",
        "name": "varnish",
        "status": "pass",
        "date": "2020-01-07 18:45:03"
    },
    {
        "tool_run_id": "6",
        "name": "marketing",
        "status": "fail",
        "date": "2020-01-08 16:57:55"
    }
]

The status field indicates the overall result of the review, which can be pass, fail, error, or needs_review.

Additional details for each review can be found using the status API with the tool name and the tool run ID.

Request

Response

curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/f4eacd72be/status/marketing/6
Copied to your clipboard
curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/f4eacd72be/status/marketing/6

{
    "tool": "marketing",
    "status": "FAIL",
    "reports": {
        "tool": "marketing",
        "vendor": "acme",
        "extension": "one-click-checkout",
        "version" : "1.1.5",
        "status": "FAIL",
        "date": "2020-01-08 16:57:55",
        "details": {
            "reasons": ["Long Description: Your extension long description contains several errors. View marketing review guidelines..."],
            "message": "Please correct grammar and spelling issues. See attached screenshots for more information.",
            "attachment": "https://commercedeveloper-sandbox-static.adobe.com/user/attachments/ha/8d/ha8d83d9482un439d8s09xnc88/prv/jj/t1/jjt18m0u38ru0v4v8y76q5e2xre5vrecz/spelling.zip"
        }
    }
}
Copied to your clipboard
{
    "tool": "marketing",
    "status": "FAIL",
    "reports": {
        "tool": "marketing",
        "vendor": "acme",
        "extension": "one-click-checkout",
        "version" : "1.1.5",
        "status": "FAIL",
        "date": "2020-01-08 16:57:55",
        "details": {
            "reasons": ["Long Description: Your extension long description contains several errors. View marketing review guidelines..."],
            "message": "Please correct grammar and spelling issues. See attached screenshots for more information.",
            "attachment": "https://commercedeveloper-sandbox-static.adobe.com/user/attachments/ha/8d/ha8d83d9482un439d8s09xnc88/prv/jj/t1/jjt18m0u38ru0v4v8y76q5e2xre5vrecz/spelling.zip"
        }
    }
}

The top-level status field contains the final decision made for this test. All status values in the reports array must be PASS in order for the test to pass.
The reports array can contain multiple entries. For automated tests, there will be additional information about the environment used for testing.
The details array structure will be different for each test.

Test result structures

The following examples demonstrate failure reports for each test type.

Code Sniffer (PHPCS)

A code sniffer will be run against every PHP version the extension is compatible with. The reports array contains a separate entry for each environment. The details array contains the output generated by the Code Sniffer tool.

Additional fields:

Field Name	Type	Value
`php_version`	string	PHP version number

Request

Response

curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/f4eacd72be/status/phpcs/8
Copied to your clipboard
curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/f4eacd72be/status/phpcs/8

{
    "tool": "phpcs",
    "status": "FAIL",
    "reports": [
        {
            "tool": "phpcs",
            "vendor": "acme",
            "extension": "one-click-checkout",
            "version" : "1.1.5",
            "php_version" : "7.1.16",
            "status": "FAIL",
            "date": "2020-01-07 18:52:25",
            "details": {
                "totals": {
                    "errors": 1,
                    "warnings": 0,
                    "fixable": 0
                },
                "files": {
                    "\/eqp-automation\/temp\/acme\/one-click-checkout\/acme-one-click-checkout-1.1.5\/registration.php": {
                        "errors": 1,
                        "warnings": 0,
                        "messages": [
                            {
                                "message": "\"resource\" is a reserved word in PHP 7",
                                "source": "MEQP2.NamingConventions.ReservedWords.FoundReservedWord",
                                "severity": 10,
                                "type": "ERROR",
                                "line": 3,
                                "column": 36,
                                "fixable": false
                            }
                        ]
                    }
                }
            }
        },
        {
            "tool": "phpcs",
            "vendor": "acme",
            "extension": "one-click-checkout",
            "version" : "1.1.5",
            "php_version" : "5.6.38",
            "status": "PASS",
            "date": "2020-01-07 18:52:25",
            "details": {
                "totals": {
                    "errors": 0,
                    "warnings": 0,
                    "fixable": 0
                },
                "files": {
                  "\/eqp-automation\/temp\/acme\/one-click-checkout\/acme-one-click-checkout-1.1.5\/registration.php": {
                        "errors": 0,
                        "warnings": 0,
                        "messages": []
                    }
                }
            }
        }
    ]
}
Copied to your clipboard
{
    "tool": "phpcs",
    "status": "FAIL",
    "reports": [
        {
            "tool": "phpcs",
            "vendor": "acme",
            "extension": "one-click-checkout",
            "version" : "1.1.5",
            "php_version" : "7.1.16",
            "status": "FAIL",
            "date": "2020-01-07 18:52:25",
            "details": {
                "totals": {
                    "errors": 1,
                    "warnings": 0,
                    "fixable": 0
                },
                "files": {
                    "\/eqp-automation\/temp\/acme\/one-click-checkout\/acme-one-click-checkout-1.1.5\/registration.php": {
                        "errors": 1,
                        "warnings": 0,
                        "messages": [
                            {
                                "message": "\"resource\" is a reserved word in PHP 7",
                                "source": "MEQP2.NamingConventions.ReservedWords.FoundReservedWord",
                                "severity": 10,
                                "type": "ERROR",
                                "line": 3,
                                "column": 36,
                                "fixable": false
                            }
                        ]
                    }
                }
            }
        },
        {
            "tool": "phpcs",
            "vendor": "acme",
            "extension": "one-click-checkout",
            "version" : "1.1.5",
            "php_version" : "5.6.38",
            "status": "PASS",
            "date": "2020-01-07 18:52:25",
            "details": {
                "totals": {
                    "errors": 0,
                    "warnings": 0,
                    "fixable": 0
                },
                "files": {
                  "\/eqp-automation\/temp\/acme\/one-click-checkout\/acme-one-click-checkout-1.1.5\/registration.php": {
                        "errors": 0,
                        "warnings": 0,
                        "messages": []
                    }
                }
            }
        }
    ]
}

Copy-Paste Detector

Copy-Paste Detector (PHPCPD) reports cannot be directly requested. Only the summary data is available through the API.

Varnish

The Varnish test installs the extension into a Magento instance and attempts to access several pages to determine if Varnish caching is working as expected. The test will be run against all combinations of Magento versions, Magento editions, and PHP versions.

Additional fields:

Field Name	Type	Value
`php_version`	string	PHP version number
`magento_platform`	string	M1 or M2
`magento_edition`	string	CE or EE
`magento_version`	string	Magento version number

Request

Response

curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/f4eacd72be/status/varnish/8
Copied to your clipboard
curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/f4eacd72be/status/varnish/8

{
    "tool" : "varnish",
    "status" : "FAIL",
    "reports" : [
        {
            "tool": "varnish",
            "vendor": "acme",
            "extension": "one-click-checkout",
            "version": "1.1.5",
            "magento_platform" : "M2",
            "magento_edition" : "CE",
            "magento_version" : "2.2.3",
            "php_version" : "7.1.16",
            "status": "FAIL",
            "date": "2020-01-07 19:45:42",
            "details" : [
                {
                    "command": "composer require acme\/one-click-checkout:1.1.5 2>&1",
                    "output" : "<string output of composer require command goes here>"
                },
                {
                    "command": "php bin\/magento setup:upgrade",
                    "output" : "<string output of setup:upgrade command goes here>"
                },
                ...
            ]
        },
        ...
    ]
}
Copied to your clipboard
{
    "tool" : "varnish",
    "status" : "FAIL",
    "reports" : [
        {
            "tool": "varnish",
            "vendor": "acme",
            "extension": "one-click-checkout",
            "version": "1.1.5",
            "magento_platform" : "M2",
            "magento_edition" : "CE",
            "magento_version" : "2.2.3",
            "php_version" : "7.1.16",
            "status": "FAIL",
            "date": "2020-01-07 19:45:42",
            "details" : [
                {
                    "command": "composer require acme\/one-click-checkout:1.1.5 2>&1",
                    "output" : "<string output of composer require command goes here>"
                },
                {
                    "command": "php bin\/magento setup:upgrade",
                    "output" : "<string output of setup:upgrade command goes here>"
                },
                ...
            ]
        },
        ...
    ]
}

The last command/output step in the details section is the command that failed to execute successfully.

Marketing and Manual QA

The Marketing and Manual QA reviews share the same report structure. There should only be one entry in the reports array for these reviews. The tool names are marketing and manual_qa respectively.

Request

Response

curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/f4eacd72be/status/manual_qa/9
Copied to your clipboard
curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/f4eacd72be/status/manual_qa/9

{
  "tool": "manual_qa",
  "status": "FAIL",
  "reports": {
    "tool": "manual_qa",
    "vendor": "acme",
    "extension": "one-click-checkout",
    "version": "1.1.5",
    "status": "FAIL",
    "date": "2020-01-14 07:21:57",
    "details": {
      "reasons": ["Tech: Your extension does not function as described in the user guide."],
      "message": "Missing feature - your extension was missing a feature when tested on Magento 2.2.3. See attached screenshots for more information.",
      "attachment": "https://commercedeveloper-sandbox-static.adobe.com/user/attachments/ha/8d/ha8d83d9482un439d8s09xnc88/prv/jj/t1/jjt18m0u38ru0v4v8y76q5e2xre5vrecz/missing_feature.zip"
    }
  }
}
Copied to your clipboard
{
  "tool": "manual_qa",
  "status": "FAIL",
  "reports": {
    "tool": "manual_qa",
    "vendor": "acme",
    "extension": "one-click-checkout",
    "version": "1.1.5",
    "status": "FAIL",
    "date": "2020-01-14 07:21:57",
    "details": {
      "reasons": ["Tech: Your extension does not function as described in the user guide."],
      "message": "Missing feature - your extension was missing a feature when tested on Magento 2.2.3. See attached screenshots for more information.",
      "attachment": "https://commercedeveloper-sandbox-static.adobe.com/user/attachments/ha/8d/ha8d83d9482un439d8s09xnc88/prv/jj/t1/jjt18m0u38ru0v4v8y76q5e2xre5vrecz/missing_feature.zip"
    }
  }
}