magento2: Moving category in hierarchy causes url_path to be incorrect when using different url keys in multiple storeviews

Hi!

This is yet another issue in the seemingly endless catalog of Magento2 url bugs with categories and products when using multiple storeviews.

Preconditions

Tested with Magento 2.2.4 & Magento 2.3-develop branch (commit: 2910d8b03e8), and most likely also all Magento 2.1.x and 2.0.x releases are affected (untested)
PHP 7.1.18
Updates on Dec 23, 2020: Issue still actual for 2.4.x https://github.com/magento/magento2/issues/16202#issuecomment-750051749 https://github.com/magento/magento2/issues/16202#issuecomment-750393505

Steps to reproduce

Setup a clean Magento installation, I’m using commit 2910d8b03e8 here
Make sure you have 2 storeviews: English (code: en) & French (code: fr)
Set Stores > Configuration > General > Web > Url Options > Add Store Code to Urls to Yes
Flush caches
Add new Categories, in this exact structure:

  Default Category
├── All 1
│   └── All 2
├── All 3
└── All 4

Start translating these category names & url keys, as following (uncheck ‘Create permanent redirect for old url’ every single time), I’ve also added the category id from my case, as it might be useful to understand the expected & actual results:

Store view	Category name	Category url key	Category ID
All Store Views	All 1	all-1	3
English	EN 1	en-1	3
French	FR 1	fr-1	3
All Store Views	All 2	all-2	4
English	EN 2	en-2	4
French	FR 2	fr-2	4
All Store Views	All 3	all-3	5
English	EN 3	en-3	5
French	FR 3	fr-3	5
All Store Views	All 4	all-4	6
English	EN 4	en-4	6
French	FR 4	fr-4	6

So far, so good, all values in the catalog_category_entity_varchar for the attributes url_key and url_path are correct. And all entries in the url_rewrite table are good. Feel free to review them at this point.
Switch back to ‘All Store Views’ in the Categories listing if you aren’t already on that store view
Drag category ‘All 2’ out of ‘All 1’, and make its new parent ‘Default Category’
Drag category ‘All 4’ inside of ‘All 3’, so that becomes its new parent.
You now have this category structure:

  Default Category
├── All 1
├── All 3
│   └── All 4
├── All 2

At this point the values in the catalog_category_entity_varchar for the attribute url_path are already incorrect on store view level, but the problem doesn’t manifest itself yet on the frontend, since the url rewrites are still ok somehow. So let’s start to break something on the frontend now.
Add a new subcategory ‘All 5’ under the category ‘All 2’ (no need to create translations for this one), ID: 7, see below
Add a new subcategory ‘All 6’ under the category ‘All 4’ (no need to create translations for this one), ID: 8, see below
You now have this category structure:

  Default Category
├── All 1
├── All 3
│   └── All 4
│   │   └── All 6
├── All 2
│   └── All 5

Expected result

Contents of the catalog_category_entity_varchar table for the url_path attribute (I took the liberty to “translate” the attribute_id and store_id to make the table easier to understand):

attribute	store_code	entity_id	value
url_path	all	3	all-1
url_path	all	4	all-2
url_path	all	5	all-3
url_path	all	6	all-3/all-4
url_path	all	7	all-2/all-5
url_path	all	8	all-3/all-4/all-6
url_path	en	3	en-1
url_path	en	4	en-2
url_path	en	5	en-3
url_path	en	6	en-3/en-4
url_path	en	7	en-2/all-5
url_path	en	8	en-3/en-4/all-6
url_path	fr	3	fr-1
url_path	fr	4	fr-2
url_path	fr	5	fr-3
url_path	fr	6	fr-3/fr-4
url_path	fr	7	fr-2/all-5
url_path	fr	8	fr-3/fr-4/all-6

When you look at the frontend of the shop and click on the following categories in the menu and look at the url, you’ll see:

Category name	Category url
EN 1	/en/en-1.html
EN 2	/en/en-2.html
EN 3	/en/en-3.html
EN 4	/en/en-3/en-4.html
EN 5	/en/en-2/all-5.html
EN 6	/en/en-3/en-4/all-6.html

(exact same for French storeview)

Actual result

Contents of the catalog_category_entity_varchar table for the url_path attribute:

attribute	store_code	entity_id	value
url_path	all	3	all-1
url_path	all	4	all-2
url_path	all	5	all-3
url_path	all	6	all-3/all-4
url_path	all	7	all-2/all-5
url_path	all	8	all-3/all-4/all-6
url_path	en	3	en-1
url_path	en	4	en-1/en-2
url_path	en	5	en-3
url_path	en	6	en-4
url_path	en	7	en-1/en-2/all-5
url_path	en	8	en-4/all-6
url_path	fr	3	fr-1
url_path	fr	4	fr-1/fr-2
url_path	fr	5	fr-3
url_path	fr	6	fr-4
url_path	fr	7	fr-1/fr-2/all-5
url_path	fr	8	fr-4/all-6

When you look at the frontend of the shop and click on the following categories in the menu and look at the url, you’ll see:

Category name	Category url
EN 1	/en/en-1.html
EN 2	/en/en-2.html
EN 3	/en/en-3.html
EN 4	/en/all-3/en-4.html
EN 5	/en/en-1/en-2/all-5.html
EN 6	/en/en-4/all-6.html

(exact same for French storeview)

Further discussion

I think this is because in the methods afterChangeParent and updateUrlPathForChildren of the class Magento\CatalogUrlRewrite\Model\Category\Plugin\Category\Move should loop over all store id’s and recalculate the url_path for every store view separately.

Now, I don’t think that fixing the code alone will be enough. You have to keep in mind that people have been running into this bug (and a lot of other url bugs) in the past 2 to 3 years. So it won’t be enough to just fix the code.

I believe Magento should get some new command line commands for bin/magento which looks at all category’s url_path attributes, and fix those if incorrect, and then also (maybe in a separate command) re-generate url rewrites (with an optional flag for creating 301 redirects, for shops already live vs shops in development). You could argue to write upgrade scripts which do this automatically when updating to a new version of Magento, but I don’t think that would be a good idea, I think store owners need to execute these things manually, it might have SEO consequences if it happens automatically and store owners might not expect that to happen.

@akaplya: since you are responsible for the UrlRewrites components according to the wiki, I’m including you in here. It might also be nice to get some kind of update from you regarding how it’s going with fixing all those other store view related bugs with url rewrites for products and categories which still haven’t been properly fixed (at least not in 2.2.4). If you don’t know what I’m talking about, let me know, I can send you a whole list of issues.

Thanks!

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 7
Comments: 22 (18 by maintainers)

Most upvoted comments

For the people interested, MC-40371 fixed part of the problem, but it started breaking again after step 16. https://github.com/magento/magento2/pull/32598 then solved the remainder, and now the bug no longer occurs as far as I can see. All category url_path’s (and url rewrites having to do with these) are being generated correctly if you start dragging categories around in the hierarchy.

If people are interested, here is a zip file with two patches that can be applied with https://github.com/cweagans/composer-patches or https://github.com/vaimo/composer-patches/ (use patch level 1 please): issue-16202-patches.zip

These patches are tested on Magento 2.4.2 but could in theory also be applied to Magento 2.4.0 or 2.4.1. I haven’t looked into backporting these all the way to 2.3.6 though (and maybe never will …)

Big thanks to @victorpetryk and @nikita-shcherbatykh and possibly some others involved here for finally fixing this huge bug that has existed since Magento 2 was released! 🥳

hostep on Apr 12, 2021

@hostep thank you for the clarification! I have raised the priority of the picket to P1 since the flow to manage the categories you mentioned is awful.

sidolov on Dec 23, 2020

✅ Confirmed by @sdzhepa Thank you for verifying the issue. Based on the provided information internal tickets MC-40780 were created

Issue Available: @sdzhepa, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

magento-engcom-team on Feb 5, 2021

I can very easily break the frontend again. Here are some extra steps to perform after the steps from my opening post.

Edit the ‘All 2’ category on the English & French storeview, change the url-key temporarily to be able to uncheck the ‘Create Permanent Redirect for old URL’ checkbox, then check ‘Use Default Value’ so it uses all-2 again as url_key, and save the category.
Reindex & flush caches
Go to homepage on frontend in English & French and click the ‘All 5’ category, now check the url

Expected

Category name	Url in English	Url in French
All 2	/en/all-2.html	/fr/all-2.html
All 5	/en/all-2/all-5.html	/fr/all-2/all-5.html

Actual:

Category name	Url in English	Url in French
All 2	/en/all-2.html	/fr/all-2.html
All 5	/en/en-2/all-5.html	/fr/fr-2/all-5.html

$ bin/magento catalog:category:integrity:urlpath
+-------------+-------+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Category ID | Name  | Store ID | Problem                                                                                                                                                          |
+-------------+-------+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
...
| 7           | All 5 | 1        | Category has an incorrect url_path value "en-2/all-5". It should be "all-2/all-5". Unfortunately this is not easily fixable using the backend of Magento.        |
| 7           | All 5 | 2        | Category has an incorrect url_path value "fr-2/all-5". It should be "all-2/all-5". Unfortunately this is not easily fixable using the backend of Magento.        |

...
+-------------+-------+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Discussion

For the developers who want to work on this issue, note that the bug starts happening between step 9 and 12, once you start dragging categories around, the url_path is not re-calculated correctly on storeview scope 0.

In my opinion this should become a higher priority, shopowners using multiple storeviews and translated url_key’s for categories still can’t drag categories around in the hierarchy without breaking their url structure. Which is really bad for SEO. We have to tell our clients to delete a category, re-create it, re-add all children categories and re-assign all products, if they want to position it in another position, which is ridiculous. (/cc @sidolov)

@engcom-Delta: can you re-open please? Also I have no powers to re-open my own tickets, so please don’t suggest that I can, because we have very limited rights unfortunatly 🙁

hostep on Dec 23, 2020

@engcom-Delta, @sdzhepa: I just retested on the latest 2.4-develop branch (using commit: fadbf696918f990f6c743fd45661b2ce51cb4a0c). The situation is improved a lot but it’s not fully fixed yet. I would like to ask you to re-open and further investigate please! The end result with the url rewrites seems to be correct but some category’s url_path attributes are still incorrect. There are fewer problems then initially reported, but they are still here. It looks like it happens only on storeview ID 0 which might explain why the frontend looks fine since that uses storeview id’s > 0. But these incorrect values might cause issues later on when creating more categories or products or translating url keys.

It will help if you install https://github.com/baldwin-agency/magento2-module-url-data-integrity-checker/ and run bin/magento catalog:category:integrity:urlpath. This command will calculate the correct url_path for categories and compare with the situation in the database.

After step 12, it will output:

$ bin/magento catalog:category:integrity:urlpath
+-------------+-------+----------+------------------------------------------------------------------------------------------------------------------------------------------------------+
| Category ID | Name  | Store ID | Problem                                                                                                                                              |
+-------------+-------+----------+------------------------------------------------------------------------------------------------------------------------------------------------------+
| 4           | All 2 | 0        | Category has an incorrect url_path value "all-1/all-2". It should be "all-2". Unfortunately this is not easily fixable using the backend of Magento. |
| 6           | All 4 | 0        | Category has an incorrect url_path value "all-4". It should be "all-3/all-4". Unfortunately this is not easily fixable using the backend of Magento. |
+-------------+-------+----------+------------------------------------------------------------------------------------------------------------------------------------------------------+

After step 15, it will output:

$ bin/magento catalog:category:integrity:urlpath
+-------------+-------+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Category ID | Name  | Store ID | Problem                                                                                                                                                          |
+-------------+-------+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 4           | All 2 | 0        | Category has an incorrect url_path value "all-1/all-2". It should be "all-2". Unfortunately this is not easily fixable using the backend of Magento.             |
| 6           | All 4 | 0        | Category has an incorrect url_path value "all-4". It should be "all-3/all-4". Unfortunately this is not easily fixable using the backend of Magento.             |
| 7           | All 5 | 0        | Category has an incorrect url_path value "all-1/all-2/all-5". It should be "all-2/all-5". Unfortunately this is not easily fixable using the backend of Magento. |
| 8           | All 6 | 0        | Category has an incorrect url_path value "all-4/all-6". It should be "all-3/all-4/all-6". Unfortunately this is not easily fixable using the backend of Magento. |
+-------------+-------+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+

So please re-open and have somebody investigate what is going on and fix it, thanks! 🙂

hostep on Dec 23, 2020