Test results

Submissions go through several reviews before being released on the Commerce Marketplace. You can use the following endpoints to get more information about the current status of a submission. See Get package details for details about these endpoints.

GET /rest/v1/products/packages/:submission_id/status
GET /rest/v1/products/packages/sku/:url_encoded_sku/status
GET /rest/v1/products/packages/item/:item_id/status
GET /rest/v1/products/packages/:submission_id/status/:tool_name/:tool_run_id

In this example, submission f4eacd72be failed the marketing review. Use this request to get a summary report of the review process:

data-slots=heading, code

data-repeat=2

data-languages=CURL, JSON

Request

curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/products/packages/f4eacd72be/status

Response

[
    {
        "tool_run_id": "5",
        "name": "phpcpd",
        "status": "pass",
        "date": "2020-01-07 18:43:32"
    },
    {
        "tool_run_id": "5",
        "name": "phpcs",
        "status": "pass",
        "date": "2020-01-07 18:52:25"
    },
    {
        "tool_run_id": "5",
        "name": "varnish",
        "status": "pass",
        "date": "2020-01-07 18:45:03"
    },
    {
        "tool_run_id": "6",
        "name": "marketing",
        "status": "fail",
        "date": "2020-01-08 16:57:55"
    }
]

The status field indicates the overall result of the review, which can be pass, fail, error, or needs_review.

Additional details for each review can be found using the status API with the tool name and the tool run ID.

data-slots=heading, code

data-repeat=2

data-languages=CURL, JSON

Request

curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/f4eacd72be/status/marketing/6

Response

{
    "tool": "marketing",
    "status": "FAIL",
    "reports": {
        "tool": "marketing",
        "vendor": "acme",
        "extension": "one-click-checkout",
        "version" : "1.1.5",
        "status": "FAIL",
        "date": "2020-01-08 16:57:55",
        "details": {
            "reasons": ["Long Description: Your extension long description contains several errors. View marketing review guidelines..."],
            "message": "Please correct grammar and spelling issues. See attached screenshots for more information.",
            "attachment": "https://commercedeveloper-sandbox-static.adobe.com/user/attachments/ha/8d/ha8d83d9482un439d8s09xnc88/prv/jj/t1/jjt18m0u38ru0v4v8y76q5e2xre5vrecz/spelling.zip"
        }
    }
}

Test result structures

The following examples demonstrate failure reports for each test type.

Code Sniffer (PHPCS)

A code sniffer will be run against every PHP version the extension is compatible with. The reports array contains a separate entry for each environment. The details array contains the output generated by the Code Sniffer tool.

Additional fields:

Field Name
Type
Value
php_version
string
PHP version number
data-slots=heading, code

data-repeat=2

data-languages=CURL, JSON

Request

curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/f4eacd72be/status/phpcs/8

Response

{
    "tool": "phpcs",
    "status": "FAIL",
    "reports": [
        {
            "tool": "phpcs",
            "vendor": "acme",
            "extension": "one-click-checkout",
            "version" : "1.1.5",
            "php_version" : "7.1.16",
            "status": "FAIL",
            "date": "2020-01-07 18:52:25",
            "details": {
                "totals": {
                    "errors": 1,
                    "warnings": 0,
                    "fixable": 0
                },
                "files": {
                    "\/eqp-automation\/temp\/acme\/one-click-checkout\/acme-one-click-checkout-1.1.5\/registration.php": {
                        "errors": 1,
                        "warnings": 0,
                        "messages": [
                            {
                                "message": "\"resource\" is a reserved word in PHP 7",
                                "source": "MEQP2.NamingConventions.ReservedWords.FoundReservedWord",
                                "severity": 10,
                                "type": "ERROR",
                                "line": 3,
                                "column": 36,
                                "fixable": false
                            }
                        ]
                    }
                }
            }
        },
        {
            "tool": "phpcs",
            "vendor": "acme",
            "extension": "one-click-checkout",
            "version" : "1.1.5",
            "php_version" : "5.6.38",
            "status": "PASS",
            "date": "2020-01-07 18:52:25",
            "details": {
                "totals": {
                    "errors": 0,
                    "warnings": 0,
                    "fixable": 0
                },
                "files": {
                  "\/eqp-automation\/temp\/acme\/one-click-checkout\/acme-one-click-checkout-1.1.5\/registration.php": {
                        "errors": 0,
                        "warnings": 0,
                        "messages": []
                    }
                }
            }
        }
    ]
}

Copy-Paste Detector

data-variant=info

data-slots=text
Copy-Paste Detector (PHPCPD) reports cannot be directly requested. Only the summary data is available through the API.

Varnish

The Varnish test installs the extension into a Magento instance and attempts to access several pages to determine if Varnish caching is working as expected. The test will be run against all combinations of Magento versions, Magento editions, and PHP versions.

Additional fields:

Field Name
Type
Value
php_version
string
PHP version number
magento_platform
string
M1 or M2
magento_edition
string
CE or EE
magento_version
string
Magento version number
data-slots=heading, code

data-repeat=2

data-languages=CURL, JSON

Request

curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/f4eacd72be/status/varnish/8

Response

{
    "tool" : "varnish",
    "status" : "FAIL",
    "reports" : [
        {
            "tool": "varnish",
            "vendor": "acme",
            "extension": "one-click-checkout",
            "version": "1.1.5",
            "magento_platform" : "M2",
            "magento_edition" : "CE",
            "magento_version" : "2.2.3",
            "php_version" : "7.1.16",
            "status": "FAIL",
            "date": "2020-01-07 19:45:42",
            "details" : [
                {
                    "command": "composer require acme\/one-click-checkout:1.1.5 2>&1",
                    "output" : "<string output of composer require command goes here>"
                },
                {
                    "command": "php bin\/magento setup:upgrade",
                    "output" : "<string output of setup:upgrade command goes here>"
                },
                ...
            ]
        },
        ...
    ]
}

The last command/output step in the details section is the command that failed to execute successfully.

Marketing and Manual QA

The Marketing and Manual QA reviews share the same report structure. There should only be one entry in the reports array for these reviews. The tool names are marketing and manual_qa respectively.

data-slots=heading, code

data-repeat=2

data-languages=CURL, JSON

Request

curl -X GET \
     -H 'Authorization: Bearer baGXoStRuR9VCDFQGZNzgNqbqu5WUwlr.cAxZJ9m22Le7' \
     https://commercedeveloper-sandbox-api.adobe.com/rest/v1/f4eacd72be/status/manual_qa/9

Response

{
  "tool": "manual_qa",
  "status": "FAIL",
  "reports": {
    "tool": "manual_qa",
    "vendor": "acme",
    "extension": "one-click-checkout",
    "version": "1.1.5",
    "status": "FAIL",
    "date": "2020-01-14 07:21:57",
    "details": {
      "reasons": ["Tech: Your extension does not function as described in the user guide."],
      "message": "Missing feature - your extension was missing a feature when tested on Magento 2.2.3. See attached screenshots for more information.",
      "attachment": "https://commercedeveloper-sandbox-static.adobe.com/user/attachments/ha/8d/ha8d83d9482un439d8s09xnc88/prv/jj/t1/jjt18m0u38ru0v4v8y76q5e2xre5vrecz/missing_feature.zip"
    }
  }
}