magento2: Deadlock Errors during product save
Preconditions
- Magneto 2.0.2
- PHP 5.6, MySQL 5.6
My Scene
- To Save/Update large amount(45K) products in Magento from amazon on daily basis.
- I have developed a custom script to save products in Magento
- Product is saving properly if i save them one by one.
Steps to reproduce
- saved 10 products at a time by using pcntl_fork(php function).
- another scenario is NodeJS async call to Magento API’s {each api call will save one product} and i limit the calls to 5 at a time. but still same error.
- i also change indexing on “Update by Schedule”. and done indexing manually by command at last.
Expected result
- product should be save
Actual result
Some of the products are saved successfully but some failed in ratio of 70%-30% respectively. Log Errors-
- Asymmetric Transaction rollback.
- Deadlock Errors.
- Serialization failure. These are the general message of errors. one of my friend also suggest me to change innoDB lock or some serialization properties of MySQL, but as i thought these changes will make some other effect on magento application.
Description
i am saving custom product in magento 2.0.x. it is working fine when i am saving them one by one. but as i have to save 45k products and to make my process fast, i saved multiple product at same time. at this time magento is throwing exception of deadlock.
Exception line :- /vendor/magento/zendframework1/library/Zend/Db/Statement/Pdo.php235
Exception :-
SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction, query was: INSERT INTO catalog_product_entity (attribute_set_id, type_id, sku, created_at, updated_at) VALUES (?, ?, ?, ‘2016-06-29 12:29:57’, ‘2016-06-29 12:29:57’)
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 30 (11 by maintainers)
@deepanshu27193, we are closing this issue due to inactivity. If you’d like to update it, please reopen the issue.
This issue is still available, when Magento team will stop closing issue for “inactivity” ?! We have no choice to fix by ourselves some troubles because you don’t give us the solutions… 😕 It really sucks.
We’re also seeing this issue when saving multiple configurable products in parallel. Do you have any update on this Magento?
I’ve fixed this issue by changing the tx_isolation to “READ-COMMITTED” instead of the default value of InnoDB.
The only solution we found was using async. This happened on a (new) Magento 2.3.6 version with MSI default source. Using REST API PUT …/rest/V1/products/[sku] and JSON. Function pushes about 20 products at a time, result: one goes fine, one deadlocks, etc. We tried about everything without success. So we decided to use async: …/rest/async/V1/products/[sku] and it works, no more deadlocks. Worth a try. Be sure to enable the consumer: bin/magento queue:consumers:start async.operations.all Another advice: set all indexers to update on schedule.
Hi @deepanshu27193 , We have fixed deadlocks on product save in many scenarios. Could you please try to reproduce it with your cases? Please note, if you use EE edition, then fixes will be available in one of next releases.
CC: @peleq-smile, @careys7, @maheshb4u , @w130pmpo