Edit in GitHubLog an issue

Bulk operations

Bulk operations are actions that are performed on a large scale. Example bulk operations tasks include importing or exporting items, changing prices on a mass scale, and assigning products to a warehouse.

For each individual task of a bulk operation, the system creates a message that is published in a message queue. A consumer runs in the background and processes the messages that it receives. Because tasks are processed in the background through the message queue system, when a merchant launches a bulk operation from the Admin panel, control is quickly returned to the merchant. In previous releases, the merchant could not use the Admin panel until all tasks were completed.

The primary Bulk Operation interface is OperationInterface. It defines the getter and setter methods the bulk operation uses to create and process messages. The following interfaces are also used:

InterfaceDescription
BulkManagementInterfaceSchedules and deletes bulk operation requests
BulkStatusInferfaceReturns the status of bulk operations
BulkSummaryInterfaceProvides details about a bulk operation
OperationManagementInterfaceChanges the status of an operation

Three clients call bulk operation APIs:

  • A publisher, which pushes messages to the message queue
  • A consumer, which handles each specific operation
  • A client that gets the status of the bulk operation and shows the list of failed operations

Publish bulk operations#

The BulkManagementInterface::scheduleBulk is responsible for publishing bulk operations. The following table describes its arguments.

ParameterTypeDescription
$bulkUuidstringBulk identifier generated by Magento\Framework\DataObject\IdentityGeneratorInterface::generateId
$operationsarray
  • topic_name - Must be defined in the communication.xml and queue configuration files. It can also be used in one of the Asynchonous API pre-generated topics.
  • bulk_uuid - A bulk identifier
  • status - The default operation status OperationInterface::STATUS_TYPE_OPEN
  • serialized_data - An array of serialized data with the following required keys:
    • entity_id - Your entity ID
    • entity_link - Link to your entity
    • meta_info - String that describes your entity. For example, "SKU: Simple_Product"

    This data is required to display the results of operations couldn't be executed for any non-recoverable reason. These results are displayed in the failed operations grid.

    You also can add any data needed to execute operations. For example, if you are conducting a mass price update, you can add price data.

$bulkDescriptionstringA description of the bulk operation. This value is displayed in the Your Bulk Operations grid.
$userIdintThe Admin user ID that executes this bulk operation.

See Create a publisher for a detailed example of a publisher.

Consume messages#

When a consumer processes a message, it must notify the system of its status. The status can be OPEN, COMPLETE, RETRIABLY_FAILED, and NOT_RETRIABLY_FAILED.

To send this notification, use OperationManagementInterface::changeOperationStatus($operationId, $status, $errorCode = null, $message = null, $data = null).

Handling recoverable exceptions#

Adobe Commerce and Magento Open Source provide database exception classes to simplify the process of identifying recoverable database errors in client code. In most cases, such errors happen due to some environment issues and can be fixed. The full path to these classes is Magento\Framework\DB\Adapter\<class_name>. These exceptions extend generic \Zend_Db_Adapter_Exception.

Exception classDescription of database error(s)
ConnectionExceptionSQLSTATE[HY000]: General error: 2006 MySQL server has gone away
SQLSTATE[HY000]: General error: 2013 Lost connection to MySQL server during query
LockWaitExceptionSQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded
DeadlockExceptionSQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock

The following pseudocode illustrates how to recover from database-related errors.

Copied to your clipboard
1<?php
2namespace example;
3use Magento\Framework\DB\Adapter\LockWaitException;
4
5// ...
6try {
7 // do something
8} catch (LockWaitException $exception) {
9 // try to recover from exception
10}

See Create a consumer for a detailed example of a consumer.

Get the status of operations#

Use getBulkStatus(UuidInterface $bulkId) to get the status of the overall bulk operation. Possible values are

ValueConstant
0NOT_STARTED
1IN_PROGRESS
2FINISHED_SUCCESSFULLY
3FINISHED_WITH_FAILURE
Was this helpful?
  • Privacy
  • Terms of Use
  • Do not sell my personal information
  • AdChoices
Copyright © 2022 Adobe. All rights reserved.