magento2: Product Duplication crash

@orlangur: would be great if a fix could get backported to version 2.1.x though, it’s a really serious problem…

We had a similar issue with 2.1.4. I was able to isolate it to the way url rewrites were getting generated in the copying process. The error is a bit misleading it has to do with url_path not url_key. When copying a product a new url_key was being generated from the old one with a “-1” appended to it however the duplicate product still has a value in the url_path attribute. When the url rewrite is getting generated the url_path takes precedence in over the url_key for some reason (I’m not totally sure why both of these are needed.)

To fix we just need to force the url_path attribute to null during the copping process. This is my new copy() function from \Magento\Catalog\Model\Product\Copier:

    /**
     * Create product duplicate
     *
     * @param \Magento\Catalog\Model\Product $product
     * @return \Magento\Catalog\Model\Product
     */
    public function copy(\Magento\Catalog\Model\Product $product)
    {
        $product->getWebsiteIds();
        $product->getCategoryIds();

        /** @var \Magento\Catalog\Model\Product $duplicate */
        $duplicate = $this->productFactory->create();
        $duplicate->setData($product->getData());
        $duplicate->setOptions([]);
        $duplicate->setIsDuplicate(true);
        $duplicate->setOriginalId($product->getEntityId());
        $duplicate->setStatus(\Magento\Catalog\Model\Product\Attribute\Source\Status::STATUS_DISABLED);
        $duplicate->setCreatedAt(null);
        $duplicate->setUpdatedAt(null);
        $duplicate->setId(null);
        //url path must be set to null to generate url_rewrites correctly
        $duplicate->setUrlPath(null);
        $duplicate->setStoreId(\Magento\Store\Model\Store::DEFAULT_STORE_ID);

        $this->copyConstructor->build($product, $duplicate);
        $isDuplicateSaved = false;


        $i = 0;
        do {
            $urlKey = $duplicate->getUrlKey();
            $urlKey = preg_match('/(.*)-(\d+)$/', $urlKey, $matches)
                ? $matches[1] . '-' . ($matches[2] + 1)
                : $urlKey . '-1';
            $duplicate->setUrlKey($urlKey);

            try {
                $duplicate->save();
                $isDuplicateSaved = true;
            } catch (\Magento\Framework\Exception\AlreadyExistsException $e) {
                $error = $e->getRawMessage();
                //if errored 20 times break
                if($i > 20){
                    $isDuplicateSaved = true;
                }
            }
            //count to aviod run away loop
            $i ++;
        } while (!$isDuplicateSaved);

        $this->getOptionRepository()->duplicate($product, $duplicate);
        $metadata = $this->getMetadataPool()->getMetadata(ProductInterface::class);
        $product->getResource()->duplicate(
            $product->getData($metadata->getLinkField()),
            $duplicate->getData($metadata->getLinkField())
        );
        return $duplicate;
    }

you can also see that I’ve added a counter for the do-while loop so it will break after 20 attempts, this do-while loop is the whats causing the images to be copied over and over, so I figured it would be safer to just cut this off at 20 even if I fixed the url_key issue.

It probably should also be investigated why the images are being processed before the save is complete.

Magento 2.1.4 still has this very big problem…

@jwallb40 I am having same issue. When we duplicate any product, it generates almost 4000 images. We have only 13 products in store and we tried to duplicate some of them and suddenly our media folder size increase to 130GB from 80MB. This is super strange and suspecious issue.

Anybody found solution for this issue?

Has anyone come up with a fix for this? We’re seeing the same thing on products that were migrated to magento 2. After a client attempted to duplicate a pre-migration product, some loop caused it to duplicate the image thousands of times, leaving us filling our entire disk.

My Products were originally created in Magento 2.0.3… never in a 1.x install

I never had a problem with product duplication myself until I upgraded to 2.1.0 recently… now when I Save an Duplicate a new product the behavior is the same as described here in this post.

If I should open a new post just holler… I was on the fence as the behavior seems the same as is noted here… the major difference being I never had my products exported from a 1.x site

-Open existing Product -Save and Duplicate -Spinner/Overlay comes up while it saves, then disapears -Page/Browser loader spings -Eventually a Server Error 500 displays

Nothing in my Apache Error Log for the 500 response either unfortunately

Save+Duplicate does appear to Save the Product… but it does not Duplicate currently for most of my current products

I tested on a variety of Products… some original Products, some previously Duplicated Products… it “seems” that products I originally created I can still duplicate… but products I have that I once duplicated from another product cannot themselves be duplicated… I haven’t exactly kept tracked of which ones were once duplicates… most of products are duplicates as that’s often the easiest way to build out a catalog of new/existing products.

OK – so it appears to be related to “Product in Websites” options… if I uncheck all Websites, save the product, I can then Save + Duplicate on a product on couldn’t have otherwise… My “old”/originally created products seem to work fine with Save + Duplicate and a Website selected but my “newer”/duplicated products can only Save+Duplicate if I first uncheck all associated Websites

Thanks for the update @VirtusB!

It looks like they re-added catching that Magento\UrlRewrite\Model\Exception\UrlAlreadyExistsException exception in 2.2.4 in: https://github.com/magento/magento2/commit/ac4dd33ea39e3d25226291282560ca3083635e7b which I explained about over here

@orlangur: I’m re-opening ticket #9466 if you don’t mind, since the issue in 2.2.x was never really solved and was accidentally hidden away and has now re-appeared in 2.2.4 (not tested myself, but the evidence seems to point in that direction). Issue #9466 seems to be more relevant then this issue for the disk space eating bug, so not re-opening this one.

I’ve built a module to cleanup product, category and swatches media which is not used. Let me know if it helps: https://github.com/ThomasNegeli/M2CliTools Currently testet with 2.1.5. Hope it works with 2.1.7 too.

Hey @FreederFree,

Yes, we added it to a custom module and overwrote the class with dependency injection, if your not familiar here’s is a pretty good tutorial: http://inchoo.net/magento-2/overriding-classes-magento-2/

Here is my di.xml

<?xml version="1.0"?>
<!--
/**
 * @author Amasty Team
 * @copyright Copyright (c) 2017 Amasty (https://www.amasty.com)
 * @package Amasty_Stockstatus
 */
-->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">
    
	<preference for="Magento\Catalog\Model\Product\Copier" type="NAMESPACE\MODULE_NAME\Model\Product\Copier" />
      
</config>

and here is the overriding class NAMESPACE/MODULE_NAME/Model/Product/Copier.php

<?php
/**
 * Catalog product copier. Creates product duplicate
 *
 * Copyright © 2016 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
namespace NAMESPACE\MODULE_NAME\Model\Product;

use \Magento\Catalog\Api\Data\ProductInterface;
use \Magento\Catalog\Model\Product\CopyConstructorInterface;

class Copier extends \Magento\Catalog\Model\Product\Copier
{
    /**
     * @var Option\Repository
     */
    protected $optionRepository;

    /**
     * @var CopyConstructorInterface
     */
    protected $copyConstructor;

    /**
     * @var \Magento\Catalog\Model\ProductFactory
     */
    protected $productFactory;

    /**
     * @var \Magento\Framework\EntityManager\MetadataPool
     */
    protected $metadataPool;

    /**
     * @param CopyConstructorInterface $copyConstructor
     * @param \Magento\Catalog\Model\ProductFactory $productFactory
     */
    public function __construct(
        CopyConstructorInterface $copyConstructor,
        \Magento\Catalog\Model\ProductFactory $productFactory
    ) {
        $this->productFactory = $productFactory;
        $this->copyConstructor = $copyConstructor;
    }

    /**
     * Create product duplicate
     *
     * @param \Magento\Catalog\Model\Product $product
     * @return \Magento\Catalog\Model\Product
     */
    public function copy(\Magento\Catalog\Model\Product $product)
    {
        $product->getWebsiteIds();
        $product->getCategoryIds();

        /** @var \Magento\Catalog\Model\Product $duplicate */
        $duplicate = $this->productFactory->create();
        $duplicate->setData($product->getData());
        $duplicate->setOptions([]);
        $duplicate->setIsDuplicate(true);
        $duplicate->setOriginalId($product->getEntityId());
        $duplicate->setStatus(\Magento\Catalog\Model\Product\Attribute\Source\Status::STATUS_DISABLED);
        $duplicate->setCreatedAt(null);
        $duplicate->setUpdatedAt(null);
        $duplicate->setId(null);
        $duplicate->setUrlPath(null);
        $duplicate->setStoreId(\Magento\Store\Model\Store::DEFAULT_STORE_ID);

        $this->copyConstructor->build($product, $duplicate);
        $isDuplicateSaved = false;


        $i = 0;
        do {
            $urlKey = $duplicate->getUrlKey();
            $urlKey = preg_match('/(.*)-(\d+)$/', $urlKey, $matches)
                ? $matches[1] . '-' . ($matches[2] + 1)
                : $urlKey . '-1';
            $duplicate->setUrlKey($urlKey);

            try {
                $duplicate->save();
                $isDuplicateSaved = true;
            } catch (\Magento\Framework\Exception\AlreadyExistsException $e) {
                $error = $e->getRawMessage();
                //if errored 20 times break
                if($i > 20){
                    $isDuplicateSaved = true;
                }
            }
            //count to aviod run away loop
            $i ++;
        } while (!$isDuplicateSaved);

        $this->getOptionRepository()->duplicate($product, $duplicate);
        $metadata = $this->getMetadataPool()->getMetadata(ProductInterface::class);
        $product->getResource()->duplicate(
            $product->getData($metadata->getLinkField()),
            $duplicate->getData($metadata->getLinkField())
        );
        return $duplicate;
    }

    /**
     * @return Option\Repository
     * @deprecated
     */
    private function getOptionRepository()
    {
        if (null === $this->optionRepository) {
            $this->optionRepository = \Magento\Framework\App\ObjectManager::getInstance()
                ->get('Magento\Catalog\Model\Product\Option\Repository');
        }
        return $this->optionRepository;
    }

    /**
     * @return \Magento\Framework\EntityManager\MetadataPool
     * @deprecated
     */
    private function getMetadataPool()
    {
        if (null === $this->metadataPool) {
            $this->metadataPool = \Magento\Framework\App\ObjectManager::getInstance()
                ->get('Magento\Framework\EntityManager\MetadataPool');
        }
        return $this->metadataPool;
    }
}

Magento 2.1.1 - same issue. New product, then save & duplicate - works fine. Migrated product, the save & duplicate - fail, php takes 100% CPU and POSTs data.

On the exception log I have this Next exception ‘Zend_Db_Statement_Exception’ with message 'SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction, query was: INSERT INTO `$