Assertions

Assertions serve to pass or fail the test if a condition is not met. These assertions will look familiar to you if you've used any other testing framework, like PHPUnit.

All assertions contain the same common actions attributes: stepKey, before, and after.

Most assertions contain a message attribute that specifies the text of an informational message to help you identify the cause of the failure.

Principles

The principles for actions are also applicable to assertions.

Assertion actions have nested self-descriptive elements, <expectedResult> and <actualResult>. These elements contain a result type and a value:

• type
  • const (default)
  • int
  • float
  • bool
  • string
  • variable
  • array
• value

If variable is used, the test transforms the corresponding value to $variable. Use the stepKey of a test, that returns the value you want to use, in assertions:

actual="stepKeyOfGrab" actualType="variable"

To use variables embedded in a string in expected and actual of your assertion, use the {$stepKey} format:

actual="A long assert string {$stepKeyOfGrab} with an embedded variable reference." actualType="variable"

In case of assertContains actions, <expectedResult> is the needle and <actualResult> is the haystack.

Example

The following example shows a common test that gets text from a page and asserts that it matches what we expect to see. If it does not, the test will fail at the assert step.

Copy
<!-- Grab a value from the page using any grab action -->
<grabTextFrom selector="#elementId" stepKey="stepKeyOfGrab"/>

<!-- Ensure that the value we grabbed matches our expectation -->
<assertEquals message="This is an optional human readable hint that will be shown in the logs if this assert fails." stepKey="assertEquals1">
   <expectedResult type="string">Some String</expectedResult>
   <actualResult type="string">A long assert string {$stepKeyOfGrab} with an embedded variable reference.</actualResult>
</assertEquals>
Copied to your clipboard
<!-- Grab a value from the page using any grab action -->
<grabTextFrom selector="#elementId" stepKey="stepKeyOfGrab"/>

<!-- Ensure that the value we grabbed matches our expectation -->
<assertEquals message="This is an optional human readable hint that will be shown in the logs if this assert fails." stepKey="assertEquals1">
   <expectedResult type="string">Some String</expectedResult>
   <actualResult type="string">A long assert string {$stepKeyOfGrab} with an embedded variable reference.</actualResult>
</assertEquals>

Elements reference

assertElementContainsAttribute

The <assertElementContainsAttribute> asserts that the selected html element contains and matches the expected value for the given attribute.

Example:

Copy
<assertElementContainsAttribute stepKey="assertElementContainsAttribute">
    <expectedResult selector=".admin__menu-overlay" attribute="style" type="string">color: #333;</expectedResult>
</assertElementContainsAttribute>
Copied to your clipboard
<assertElementContainsAttribute stepKey="assertElementContainsAttribute">
    <expectedResult selector=".admin__menu-overlay" attribute="style" type="string">color: #333;</expectedResult>
</assertElementContainsAttribute>

assertArrayIsSorted

The <assertArrayIsSorted> asserts that the array is sorted according to a specified sort order, ascending or descending.

Example:

Copy
<assertArrayIsSorted sortOrder="asc" stepKey="assertSorted">
    <array>[1,2,3,4,5,6,7]</array>
</assertArrayIsSorted>
Copied to your clipboard
<assertArrayIsSorted sortOrder="asc" stepKey="assertSorted">
    <array>[1,2,3,4,5,6,7]</array>
</assertArrayIsSorted>

It contains an <array> child element that specifies an array to be asserted for proper sorting. It must be in typical array format like [1,2,3,4,5] or [alpha, brontosaurus, zebra].

assertArrayHasKey

See assertArrayHasKey docs on codeception.com

assertArrayNotHasKey

See assertArrayNotHasKey docs on codeception.com.

assertContains

See assertContains docs on codeception.com.

assertStringContainsString

See assertStringContainsString docs on codeception.com.

Example:

Copy
<assertStringContainsString stepKey="assertDropDownTierPriceTextProduct1">
    <expectedResult type="string">Buy 5 for $5.00 each and save 50%</expectedResult>
    <actualResult type="variable">DropDownTierPriceTextProduct1</actualResult>
</assertStringContainsString>
Copied to your clipboard
<assertStringContainsString stepKey="assertDropDownTierPriceTextProduct1">
    <expectedResult type="string">Buy 5 for $5.00 each and save 50%</expectedResult>
    <actualResult type="variable">DropDownTierPriceTextProduct1</actualResult>
</assertStringContainsString>

assertStringContainsStringIgnoringCase

See assertStringContainsStringIgnoringCase docs on codeception.com.

Example:

Copy
<assertStringContainsStringIgnoringCase stepKey="verifyContentType">
    <actualResult type="variable">grabContentType</actualResult>
    <expectedResult type="string">{{image.extension}}</expectedResult>
</assertStringContainsStringIgnoringCase>
Copied to your clipboard
<assertStringContainsStringIgnoringCase stepKey="verifyContentType">
    <actualResult type="variable">grabContentType</actualResult>
    <expectedResult type="string">{{image.extension}}</expectedResult>
</assertStringContainsStringIgnoringCase>

assertCount

See assertCount docs on codeception.com.

assertEmpty

See assertEmpty docs on codeception.com.

Example:

Copy
<assertEmpty stepKey="assertSearchButtonEnabled">
    <actualResult type="string">$grabSearchButtonAttribute</actualResult>
</assertEmpty>
Copied to your clipboard
<assertEmpty stepKey="assertSearchButtonEnabled">
    <actualResult type="string">$grabSearchButtonAttribute</actualResult>
</assertEmpty>

assertEquals

See assertEquals docs on codeception.com.

Example:

Copy
<assertEquals message="ExpectedPrice" stepKey="assertBundleProductPrice">
    <actualResult type="variable">grabProductPrice</actualResult>
    <expectedResult type="string">$75.00</expectedResult>
</assertEquals>
Copied to your clipboard
<assertEquals message="ExpectedPrice" stepKey="assertBundleProductPrice">
    <actualResult type="variable">grabProductPrice</actualResult>
    <expectedResult type="string">$75.00</expectedResult>
</assertEquals>

assertEqualsWithDelta

See assertEqualsWithDelta docs on codeception.com.

assertEqualsCanonicalizing

See assertEqualsCanonicalizing docs on codeception.com.

assertEqualsIgnoringCase

See assertEqualsIgnoringCase docs on codeception.com.

assertFalse

See assertFalse docs on codeception.com.

assertFileExists

See assertFileExists docs on codeception.com.

assertFileNotExists

See assertFileNotExists docs on codeception.com.

assertGreaterOrEquals

See assertGreaterOrEquals docs on codeception.com.

Example:

Copy
<assertGreaterOrEquals stepKey="checkStatusSortOrderAsc" after="getOrderStatusSecondRow">
    <actualResult type="const">$getOrderStatusSecondRow</actualResult>
    <expectedResult type="const">$getOrderStatusFirstRow</expectedResult>
</assertGreaterOrEquals>
Copied to your clipboard
<assertGreaterOrEquals stepKey="checkStatusSortOrderAsc" after="getOrderStatusSecondRow">
    <actualResult type="const">$getOrderStatusSecondRow</actualResult>
    <expectedResult type="const">$getOrderStatusFirstRow</expectedResult>
</assertGreaterOrEquals>

assertGreaterThan

See assertGreaterThan docs on codeception.com.

Example:

Copy
<assertGreaterThan stepKey="checkQuantityWasChanged">
    <actualResult type="const">$grabEndQuantity</actualResult>
    <expectedResult type="const">$grabStartQuantity</expectedResult>
</assertGreaterThan>
Copied to your clipboard
<assertGreaterThan stepKey="checkQuantityWasChanged">
    <actualResult type="const">$grabEndQuantity</actualResult>
    <expectedResult type="const">$grabStartQuantity</expectedResult>
</assertGreaterThan>

assertGreaterThanOrEqual

See assertGreaterThanOrEqual docs on codeception.com.

Example:

Copy
<assertGreaterThanOrEqual stepKey="checkStatusSortOrderAsc" after="getOrderStatusSecondRow">
    <actualResult type="const">$getOrderStatusSecondRow</actualResult>
    <expectedResult type="const">$getOrderStatusFirstRow</expectedResult>
</assertGreaterThanOrEqual>
Copied to your clipboard
<assertGreaterThanOrEqual stepKey="checkStatusSortOrderAsc" after="getOrderStatusSecondRow">
    <actualResult type="const">$getOrderStatusSecondRow</actualResult>
    <expectedResult type="const">$getOrderStatusFirstRow</expectedResult>
</assertGreaterThanOrEqual>

assertInstanceOf

See assertInstanceOf docs on codeception.com.

assertIsEmpty

See assertIsEmpty docs on codeception.com.

assertLessOrEquals

See assertLessOrEquals docs on codeception.com.

Example:

Copy
<assertLessOrEquals stepKey="checkHeightIsCorrect">
    <actualResult type="variable">getImageHeight</actualResult>
    <expectedResult type="variable">getSectionHeight</expectedResult>
</assertLessOrEquals>
Copied to your clipboard
<assertLessOrEquals stepKey="checkHeightIsCorrect">
    <actualResult type="variable">getImageHeight</actualResult>
    <expectedResult type="variable">getSectionHeight</expectedResult>
</assertLessOrEquals>

assertLessThan

See assertLessThan docs on codeception.com.

Example:

Copy
<assertLessThan stepKey="assertLessImages">
    <expectedResult type="variable">initialImages</expectedResult>
    <actualResult type="variable">newImages</actualResult>
</assertLessThan>
Copied to your clipboard
<assertLessThan stepKey="assertLessImages">
    <expectedResult type="variable">initialImages</expectedResult>
    <actualResult type="variable">newImages</actualResult>
</assertLessThan>

assertLessThanOrEqual

See assertLessThanOrEqual docs on codeception.com.

Example:

Copy
<assertLessThanOrEqual stepKey="checkHeightIsCorrect">
    <actualResult type="variable">getImageHeight</actualResult>
    <expectedResult type="variable">getSectionHeight</expectedResult>
</assertLessThanOrEqual>
Copied to your clipboard
<assertLessThanOrEqual stepKey="checkHeightIsCorrect">
    <actualResult type="variable">getImageHeight</actualResult>
    <expectedResult type="variable">getSectionHeight</expectedResult>
</assertLessThanOrEqual>

assertNotContains

See assertNotContains docs on codeception.com.

Example:

Copy
<assertNotContains stepKey="assertCustomerGroupNotInOptions">
    <actualResult type="variable">customerGroups</actualResult>
    <expectedResult type="string">{{customerGroup.code}}</expectedResult>
</assertNotContains>
Copied to your clipboard
<assertNotContains stepKey="assertCustomerGroupNotInOptions">
    <actualResult type="variable">customerGroups</actualResult>
    <expectedResult type="string">{{customerGroup.code}}</expectedResult>
</assertNotContains>

assertStringNotContainsString

See assertStringNotContainsString docs on codeception.com.

Example:

Copy
<assertStringNotContainsString stepKey="checkoutAsGuest">
    <expectedResult type="string">{{CaptchaData.checkoutAsGuest}}</expectedResult>
    <actualResult type="variable">$formItems</actualResult>
</assertStringNotContainsString>
Copied to your clipboard
<assertStringNotContainsString stepKey="checkoutAsGuest">
    <expectedResult type="string">{{CaptchaData.checkoutAsGuest}}</expectedResult>
    <actualResult type="variable">$formItems</actualResult>
</assertStringNotContainsString>

assertStringContainsStringIgnoringCase

See assertStringNotContainsStringIgnoringCase docs on codeception.com.

Example:

Copy
<assertStringContainsStringIgnoringCase stepKey="verifyContentType">
    <actualResult type="variable">grabContentType</actualResult>
    <expectedResult type="string">{{image.extension}}</expectedResult>
</assertStringContainsStringIgnoringCase>
Copied to your clipboard
<assertStringContainsStringIgnoringCase stepKey="verifyContentType">
    <actualResult type="variable">grabContentType</actualResult>
    <expectedResult type="string">{{image.extension}}</expectedResult>
</assertStringContainsStringIgnoringCase>

assertNotEmpty

See assertNotEmpty docs on codeception.com.

Example:

Copy
<assertNotEmpty stepKey="checkSwatchFieldForAdmin">
    <actualResult type="const">$grabSwatchForAdmin</actualResult>
</assertNotEmpty>
Copied to your clipboard
<assertNotEmpty stepKey="checkSwatchFieldForAdmin">
    <actualResult type="const">$grabSwatchForAdmin</actualResult>
</assertNotEmpty>

assertNotEquals

See assertNotEquals docs on codeception.com.

Example:

Copy
<assertNotEquals stepKey="assertNotEquals">
    <actualResult type="string">{$grabTotalAfter}</actualResult>
    <expectedResult type="string">{$grabTotalBefore}</expectedResult>
</assertNotEquals>
Copied to your clipboard
<assertNotEquals stepKey="assertNotEquals">
    <actualResult type="string">{$grabTotalAfter}</actualResult>
    <expectedResult type="string">{$grabTotalBefore}</expectedResult>
</assertNotEquals>

assertNotEqualsWithDelta

See assertNotEqualsWithDelta docs on codeception.com.

assertNotEqualsCanonicalizing

See assertNotEqualsCanonicalizing docs on codeception.com.

assertNotEqualsIgnoringCase

See assertNotEqualsIgnoringCase docs on codeception.com.

assertNotInstanceOf

See assertNotInstanceOf docs on codeception.com.

assertNotNull

See assertNotNull docs on codeception.com.

assertNotRegExp

See assertNotRegExp docs on codeception.com.

Example:

Copy
<assertNotRegExp stepKey="simpleThumbnailIsNotDefault">
    <actualResult type="const">$getSimpleProductThumbnail</actualResult>
    <expectedResult type="const">'/placeholder\/thumbnail\.jpg/'</expectedResult>
</assertNotRegExp>
Copied to your clipboard
<assertNotRegExp stepKey="simpleThumbnailIsNotDefault">
    <actualResult type="const">$getSimpleProductThumbnail</actualResult>
    <expectedResult type="const">'/placeholder\/thumbnail\.jpg/'</expectedResult>
</assertNotRegExp>

assertNotSame

See assertNotSame docs on codeception.com.

assertNull

See assertNull docs on codeception.com.

assertRegExp

See assertRegExp docs on codeception.com.

Example:

Copy
<assertRegExp message="adminAnalyticsMetadata object is invalid" stepKey="validateadminAnalyticsMetadata">
    <expectedResult type="string">#var\s+adminAnalyticsMetadata\s+=\s+{\s+("[\w_]+":\s+"[^"]*?",\s+)*?("[\w_]+":\s+"[^"]*?"\s+)};#s</expectedResult>
    <actualResult type="variable">$pageSource</actualResult>
</assertRegExp>
Copied to your clipboard
<assertRegExp message="adminAnalyticsMetadata object is invalid" stepKey="validateadminAnalyticsMetadata">
    <expectedResult type="string">#var\s+adminAnalyticsMetadata\s+=\s+{\s+("[\w_]+":\s+"[^"]*?",\s+)*?("[\w_]+":\s+"[^"]*?"\s+)};#s</expectedResult>
    <actualResult type="variable">$pageSource</actualResult>
</assertRegExp>

assertSame

See assertSame docs on codeception.com.

assertStringStartsNotWith

See assertStringStartsNotWith docs on codeception.com.

assertStringStartsWith

See assertStringStartsWith docs on codeception.com.

assertTrue

See assertTrue docs on codeception.com.

expectException

See expectException docs on codeception.com.

fail

See fail docs on codeception.com.

Last updated 9/11/2023

Was this helpful?

Yes

No