magento2: After running setup:upgrade, setup:db:check still says: Declarative Schema is not up to date

Preconditions

Magento 2.3.0
Hava a database running on MariaDB 10.2

Steps to reproduce

Run command: bin/magento setup:db:status
Run command: bin/magento setup:upgrade --keep-generated
Run command: bin/magento setup:db:status

Expected result

Since we have updated, the setup:db:status command should return that everything is up to date.

Actual result

Command setup:db:status always returns: “Declarative Schema is not up to date” and that we need to run “setup:upgrade”

My logfiles from the command line:

[root@web01 current]# bin/magento setup:db:status
Declarative Schema is not up to date
Run 'setup:upgrade' to update your DB schema and data.
[root@web01 current]# bin/magento setup:upgrade --keep-generated
Schema creation/updates:
Module 'Magento_Store':
Module 'Magento_Directory':
Module 'Magento_Theme':
Module 'Cream_Backend':
Module 'Magento_Backend':
Module 'Magento_Variable':
Module 'Magento_Eav':
Module 'Magento_AdminNotification':
Module 'Magento_Indexer':
Module 'Magento_Customer':
Module 'MSP_ReCaptcha':
Module 'Magento_Config':
Module 'Magento_Cms':
Module 'Magento_AdvancedPricingImportExport':
Module 'Magento_Search':
Module 'Magento_Amqp':
Module 'Magento_Security':
Module 'Magento_User':
Module 'Magento_Authorization':
Module 'Magento_Quote':
Module 'Magento_Robots':
Module 'Magento_Backup':
Module 'Magento_Rule':
Module 'Magento_Catalog':
Module 'Magento_GraphQl':
Module 'Magento_BundleImportExport':
Module 'Magento_CatalogRule':
Module 'Magento_CacheInvalidate':
Module 'Magento_SalesSequence':
Module 'Magento_Payment':
Module 'Magento_Integration':
Module 'Magento_EavGraphQl':
Module 'Magento_CatalogImportExport':
Module 'Magento_CatalogInventory':
Module 'Magento_CatalogInventoryGraphQl':
Module 'Magento_Cron':
Module 'Magento_Msrp':
Module 'Magento_CatalogSearch':
Module 'Magento_UrlRewrite':
Module 'Magento_Sales':
Module 'Magento_CatalogUrlRewrite':
Module 'Magento_CatalogUrlRewriteGraphQl':
Module 'Magento_Widget':
Module 'Magento_Ui':
Module 'Magento_Checkout':
Module 'Magento_Vault':
Module 'Magento_CmsGraphQl':
Module 'Magento_Tax':
Module 'Magento_CmsUrlRewrite':
Module 'Magento_CmsUrlRewriteGraphQl':
Module 'MSP_TwoFactorAuth':
Module 'Magento_ConfigurableImportExport':
Module 'Magento_Downloadable':
Module 'Magento_StoreGraphQl':
Module 'Magento_ConfigurableProduct':
Module 'Magento_SampleData':
Module 'Magento_Contact':
Module 'Magento_Cookie':
Module 'Magento_Email':
Module 'Magento_CurrencySymbol':
Module 'Magento_Paypal':
Module 'Magento_Analytics':
Module 'Magento_CustomerGraphQl':
Module 'Magento_CustomerImportExport':
Module 'Magento_CustomerSampleData':
Module 'Magento_Deploy':
Module 'Magento_Developer':
Module 'Magento_Dhl':
Module 'Magento_GroupedProduct':
Module 'Magento_Weee':
Module 'Magento_CatalogGraphQl':
Module 'Magento_ImportExport':
Module 'Magento_Bundle':
Module 'Klarna_Core':
Module 'Magento_BundleGraphQl':
Module 'Magento_AdvancedSearch':
Module 'Magento_Newsletter':
Module 'Magento_EncryptionKey':
Module 'Magento_Fedex':
Module 'Magento_GiftMessage':
Module 'Magento_GoogleAdwords':
Module 'Magento_GoogleAnalytics':
Module 'Magento_GoogleOptimizer':
Module 'Magento_ConfigurableProductGraphQl':
Module 'Magento_GroupedImportExport':
Module 'Magento_CatalogSampleData':
Module 'Magento_GroupedProductGraphQl':
Module 'Magento_CatalogRuleConfigurable':
Module 'Magento_DownloadableImportExport':
Module 'Amazon_Core':
Module 'Magento_InstantPurchase':
Module 'Magento_CatalogAnalytics':
Module 'Magento_Inventory':
Module 'Magento_InventoryAdminUi':
Module 'Magento_InventoryApi':
Module 'Magento_InventoryBundleProduct':
Module 'Magento_InventoryBundleProductAdminUi':
Module 'Magento_InventoryCatalog':
Module 'Magento_InventorySales':
Module 'Magento_InventoryCatalogAdminUi':
Module 'Magento_InventoryCatalogApi':
Module 'Magento_InventoryCatalogSearch':
Module 'Magento_InventoryConfigurableProduct':
Module 'Magento_InventoryConfigurableProductAdminUi':
Module 'Magento_InventoryConfigurableProductIndexer':
Module 'Magento_InventoryConfiguration':
Module 'Magento_InventoryConfigurationApi':
Module 'Magento_InventoryGroupedProduct':
Module 'Magento_InventoryGroupedProductAdminUi':
Module 'Magento_InventoryGroupedProductIndexer':
Module 'Magento_InventoryImportExport':
Module 'Magento_InventoryIndexer':
Module 'Magento_InventoryLowQuantityNotification':
Module 'Magento_InventoryLowQuantityNotificationAdminUi':
Module 'Magento_InventoryLowQuantityNotificationApi':
Module 'Magento_InventoryMultiDimensionalIndexerApi':
Module 'Magento_InventoryProductAlert':
Module 'Magento_InventoryReservations':
Module 'Magento_InventoryReservationsApi':
Module 'Magento_InventoryCache':
Module 'Magento_InventorySalesAdminUi':
Module 'Magento_InventorySalesApi':
Module 'Magento_InventorySalesFrontendUi':
Module 'Magento_InventoryShipping':
Module 'Magento_Shipping':
Module 'Magento_InventorySourceDeductionApi':
Module 'Magento_InventorySourceSelection':
Module 'Magento_InventorySourceSelectionApi':
Module 'Magento_LayeredNavigation':
Module 'Magento_Marketplace':
Module 'Magento_MediaStorage':
Module 'Magento_MessageQueue':
Module 'Magento_GroupedProductSampleData':
Module 'Magento_MsrpSampleData':
Module 'Magento_Multishipping':
Module 'Magento_MysqlMq':
Module 'Magento_NewRelicReporting':
Module 'Magento_ProductAlert':
Module 'Magento_OfflinePayments':
Module 'Magento_SalesRule':
Module 'Magento_OfflineShipping':
Module 'Magento_PageCache':
Module 'Magento_Authorizenet':
Module 'Magento_Braintree':
Module 'Magento_Persistent':
Module 'Magento_Reports':
Module 'Magento_BundleSampleData':
Module 'Magento_ProductVideo':
Module 'Klarna_Ordermanagement':
Module 'Magento_QuoteAnalytics':
Module 'Magento_QuoteGraphQl':
Module 'Magento_ReleaseNotification':
Module 'Magento_Review':
Module 'Magento_RequireJs':
Module 'Magento_SendFriend':
Module 'Magento_ReviewAnalytics':
Module 'Magento_DownloadableSampleData':
Module 'Magento_Sitemap':
Module 'Magento_Rss':
Module 'Magento_Captcha':
Module 'Amazon_Login':
Module 'Magento_SalesAnalytics':
Module 'Magento_SalesInventory':
Module 'Magento_OfflineShippingSampleData':
Module 'Magento_ConfigurableSampleData':
Module 'Magento_TaxSampleData':
Module 'Amazon_Payment':
Module 'Magento_ThemeSampleData':
Module 'Magento_Elasticsearch':
Module 'Magento_CustomerAnalytics':
Module 'Magento_Wishlist':
Module 'Cream_DutchShop':
Module 'Magento_Signifyd':
Module 'Cream_BaseShop':
Module 'Magento_ConfigurableProductSales':
Module 'Magento_DownloadableGraphQl':
Module 'Magento_Webapi':
Module 'Magento_SwaggerWebapi':
Module 'Magento_SwaggerWebapiAsync':
Module 'Magento_Swatches':
Module 'Magento_SwatchesGraphQl':
Module 'Magento_SwatchesLayeredNavigation':
Module 'Magento_SwatchesSampleData':
Module 'Magento_CatalogRuleSampleData':
Module 'Magento_TaxGraphQl':
Module 'Magento_TaxImportExport':
Module 'Magento_SalesSampleData':
Module 'Magento_CheckoutAgreements':
Module 'Magento_CmsSampleData':
Module 'Magento_Tinymce3':
Module 'Magento_Translation':
Module 'Magento_InventoryShippingAdminUi':
Module 'Magento_Ups':
Module 'Magento_ProductLinksSampleData':
Module 'Magento_UrlRewriteGraphQl':
Module 'Magento_AsynchronousOperations':
Module 'Magento_Usps':
Module 'Magento_ReviewSampleData':
Module 'Klarna_Kp':
Module 'Magento_Version':
Module 'Magento_Swagger':
Module 'Magento_WebapiAsync':
Module 'Magento_WebapiSecurity':
Module 'Magento_SalesRuleSampleData':
Module 'Magento_WeeeGraphQl':
Module 'Magento_CatalogWidget':
Module 'Magento_WidgetSampleData':
Module 'Dotdigitalgroup_Email':
Module 'Magento_WishlistAnalytics':
Module 'Magento_WishlistSampleData':
Module 'Temando_Shipping':
Module 'Vertex_Tax':
Schema post-updates:
Module 'Magento_Store':
Module 'Magento_Directory':
Module 'Magento_Theme':
Module 'Cream_Backend':
Module 'Magento_Backend':
Module 'Magento_Variable':
Module 'Magento_Eav':
Module 'Magento_AdminNotification':
Module 'Magento_Indexer':
Running schema recurring...

Module 'Magento_Customer':
Module 'MSP_ReCaptcha':
Module 'Magento_Config':
Module 'Magento_Cms':
Module 'Magento_AdvancedPricingImportExport':
Module 'Magento_Search':
Module 'Magento_Amqp':
Running schema recurring...
Module 'Magento_Security':
Module 'Magento_User':
Module 'Magento_Authorization':
Module 'Magento_Quote':
Module 'Magento_Robots':
Module 'Magento_Backup':
Module 'Magento_Rule':
Module 'Magento_Catalog':
Running schema recurring...

Module 'Magento_GraphQl':
Module 'Magento_BundleImportExport':
Module 'Magento_CatalogRule':
Module 'Magento_CacheInvalidate':
Module 'Magento_SalesSequence':
Running schema recurring...

Module 'Magento_Payment':
Module 'Magento_Integration':
Running schema recurring...

Module 'Magento_EavGraphQl':
Module 'Magento_CatalogImportExport':
Module 'Magento_CatalogInventory':
Running schema recurring...

Module 'Magento_CatalogInventoryGraphQl':
Module 'Magento_Cron':
Running schema recurring...

Module 'Magento_Msrp':
Module 'Magento_CatalogSearch':
Module 'Magento_UrlRewrite':
Module 'Magento_Sales':
Module 'Magento_CatalogUrlRewrite':
Running schema recurring...

Module 'Magento_CatalogUrlRewriteGraphQl':
Module 'Magento_Widget':
Module 'Magento_Ui':
Module 'Magento_Checkout':
Module 'Magento_Vault':
Module 'Magento_CmsGraphQl':
Module 'Magento_Tax':
Module 'Magento_CmsUrlRewrite':
Module 'Magento_CmsUrlRewriteGraphQl':
Module 'MSP_TwoFactorAuth':
Module 'Magento_ConfigurableImportExport':
Module 'Magento_Downloadable':
Module 'Magento_StoreGraphQl':
Module 'Magento_ConfigurableProduct':
Running schema recurring...

Module 'Magento_SampleData':
Module 'Magento_Contact':
Module 'Magento_Cookie':
Module 'Magento_Email':
Module 'Magento_CurrencySymbol':
Module 'Magento_Paypal':
Module 'Magento_Analytics':
Module 'Magento_CustomerGraphQl':
Module 'Magento_CustomerImportExport':
Module 'Magento_CustomerSampleData':
Module 'Magento_Deploy':
Module 'Magento_Developer':
Module 'Magento_Dhl':
Module 'Magento_GroupedProduct':
Module 'Magento_Weee':
Running schema recurring...

Module 'Magento_CatalogGraphQl':
Module 'Magento_ImportExport':
Module 'Magento_Bundle':
Running schema recurring...

Module 'Klarna_Core':
Module 'Magento_BundleGraphQl':
Module 'Magento_AdvancedSearch':
Module 'Magento_Newsletter':
Module 'Magento_EncryptionKey':
Module 'Magento_Fedex':
Module 'Magento_GiftMessage':
Module 'Magento_GoogleAdwords':
Module 'Magento_GoogleAnalytics':
Module 'Magento_GoogleOptimizer':
Module 'Magento_ConfigurableProductGraphQl':
Module 'Magento_GroupedImportExport':
Module 'Magento_CatalogSampleData':
Module 'Magento_GroupedProductGraphQl':
Module 'Magento_CatalogRuleConfigurable':
Module 'Magento_DownloadableImportExport':
Module 'Amazon_Core':
Module 'Magento_InstantPurchase':
Module 'Magento_CatalogAnalytics':
Module 'Magento_Inventory':
Module 'Magento_InventoryAdminUi':
Module 'Magento_InventoryApi':
Module 'Magento_InventoryBundleProduct':
Module 'Magento_InventoryBundleProductAdminUi':
Module 'Magento_InventoryCatalog':
Module 'Magento_InventorySales':
Module 'Magento_InventoryCatalogAdminUi':
Module 'Magento_InventoryCatalogApi':
Module 'Magento_InventoryCatalogSearch':
Module 'Magento_InventoryConfigurableProduct':
Module 'Magento_InventoryConfigurableProductAdminUi':
Module 'Magento_InventoryConfigurableProductIndexer':
Module 'Magento_InventoryConfiguration':
Module 'Magento_InventoryConfigurationApi':
Module 'Magento_InventoryGroupedProduct':
Module 'Magento_InventoryGroupedProductAdminUi':
Module 'Magento_InventoryGroupedProductIndexer':
Module 'Magento_InventoryImportExport':
Module 'Magento_InventoryIndexer':
Module 'Magento_InventoryLowQuantityNotification':
Module 'Magento_InventoryLowQuantityNotificationAdminUi':
Module 'Magento_InventoryLowQuantityNotificationApi':
Module 'Magento_InventoryMultiDimensionalIndexerApi':
Module 'Magento_InventoryProductAlert':
Module 'Magento_InventoryReservations':
Module 'Magento_InventoryReservationsApi':
Module 'Magento_InventoryCache':
Module 'Magento_InventorySalesAdminUi':
Module 'Magento_InventorySalesApi':
Module 'Magento_InventorySalesFrontendUi':
Module 'Magento_InventoryShipping':
Module 'Magento_Shipping':
Module 'Magento_InventorySourceDeductionApi':
Module 'Magento_InventorySourceSelection':
Module 'Magento_InventorySourceSelectionApi':
Module 'Magento_LayeredNavigation':
Module 'Magento_Marketplace':
Module 'Magento_MediaStorage':
Module 'Magento_MessageQueue':
Module 'Magento_GroupedProductSampleData':
Module 'Magento_MsrpSampleData':
Module 'Magento_Multishipping':
Module 'Magento_MysqlMq':
Running schema recurring...

Module 'Magento_NewRelicReporting':
Module 'Magento_ProductAlert':
Running schema recurring...

Module 'Magento_OfflinePayments':
Module 'Magento_SalesRule':
Module 'Magento_OfflineShipping':
Module 'Magento_PageCache':
Module 'Magento_Authorizenet':
Module 'Magento_Braintree':
Module 'Magento_Persistent':
Module 'Magento_Reports':
Running schema recurring...

Module 'Magento_BundleSampleData':
Module 'Magento_ProductVideo':
Module 'Klarna_Ordermanagement':
Module 'Magento_QuoteAnalytics':
Module 'Magento_QuoteGraphQl':
Module 'Magento_ReleaseNotification':
Module 'Magento_Review':
Module 'Magento_RequireJs':
Module 'Magento_SendFriend':
Module 'Magento_ReviewAnalytics':
Module 'Magento_DownloadableSampleData':
Module 'Magento_Sitemap':
Module 'Magento_Rss':
Module 'Magento_Captcha':
Module 'Amazon_Login':
Module 'Magento_SalesAnalytics':
Module 'Magento_SalesInventory':
Module 'Magento_OfflineShippingSampleData':
Module 'Magento_ConfigurableSampleData':
Module 'Magento_TaxSampleData':
Module 'Amazon_Payment':
Module 'Magento_ThemeSampleData':
Module 'Magento_Elasticsearch':
Module 'Magento_CustomerAnalytics':
Module 'Magento_Wishlist':
Running schema recurring...

Module 'Cream_DutchShop':
Module 'Magento_Signifyd':
Module 'Cream_BaseShop':
Module 'Magento_ConfigurableProductSales':
Module 'Magento_DownloadableGraphQl':
Module 'Magento_Webapi':
Module 'Magento_SwaggerWebapi':
Module 'Magento_SwaggerWebapiAsync':
Module 'Magento_Swatches':
Module 'Magento_SwatchesGraphQl':
Module 'Magento_SwatchesLayeredNavigation':
Module 'Magento_SwatchesSampleData':
Module 'Magento_CatalogRuleSampleData':
Module 'Magento_TaxGraphQl':
Module 'Magento_TaxImportExport':
Module 'Magento_SalesSampleData':
Module 'Magento_CheckoutAgreements':
Module 'Magento_CmsSampleData':
Module 'Magento_Tinymce3':
Module 'Magento_Translation':
Module 'Magento_InventoryShippingAdminUi':
Module 'Magento_Ups':
Module 'Magento_ProductLinksSampleData':
Module 'Magento_UrlRewriteGraphQl':
Module 'Magento_AsynchronousOperations':
Module 'Magento_Usps':
Module 'Magento_ReviewSampleData':
Module 'Klarna_Kp':
Module 'Magento_Version':
Module 'Magento_Swagger':
Module 'Magento_WebapiAsync':
Module 'Magento_WebapiSecurity':
Module 'Magento_SalesRuleSampleData':
Module 'Magento_WeeeGraphQl':
Module 'Magento_CatalogWidget':
Module 'Magento_WidgetSampleData':
Module 'Dotdigitalgroup_Email':
Running schema recurring...

Module 'Magento_WishlistAnalytics':
Module 'Magento_WishlistSampleData':
Module 'Temando_Shipping':
Module 'Vertex_Tax':
Nothing to import.
[root@web01 current]# bin/magento setup:db:status
Declarative Schema is not up to date
Run 'setup:upgrade' to update your DB schema and data.

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 15
Comments: 64 (39 by maintainers)

Commits related to this issue

Magento 2: DB Configuration for Zero Downtime Deployments (https://github.com/magento/magento2/issues/19597#issuecomment-573510633) — committed to theCapypara/riptide-repo by theCapypara 4 years ago
Include notice regarding `sysvar_explicit_defaults_for_timestamps` As per https://github.com/magento/magento2/issues/19597#issuecomment-573510633 If this setting is not enabled, `setup:db:status` ... — committed to cmacdonald-au/devdocs by cmacdonald-au 4 years ago
Added note regarding use of explicit_defaults_for_timestamps=on Per https://github.com/magento/magento2/issues/19597#issuecomment-573510633 and https://github.com/magento/magento2/issues/19597#issuec... — committed to davidalger/devdocs by davidalger 4 years ago

Most upvoted comments

@engcom-backlog-nazar Believe me, I know the right channels for getting EE support. That’s not the issue here. The issue here is in the OpenSource core, not the EE modules. They just happen to trigger the bug in the core level schema comparison. Basically what it sounds like you’re saying is: find a module on the marketplace that triggers the bug or do something that doesn’t trigger the pat response when anyone mentions enterprise (which honestly, I could basically just find which EE module that is doing this and create my own module that sets up the same schema and the issue would still be there). Can we please stop with the EE channels crap and acknowledge when there are real issues in the OpenSource core that sits underneath and affects EE? Nobody is trying to get EE support through the wrong channels here, but I am trying to see the Open Source product fixed.

+38

davidalger on Dec 10, 2018

Why is this marked as done? Which Magento version has this corrected?

+16

giacmir on Nov 19, 2020

Please note that this feature is being used to determine if you can do a zero-downtime deployment. This bug breaking the setup:db:status command is breaking the ability to do zero-downtime deployments!

+13

dverkade on Dec 6, 2018

I was debugging this for an old instance (2.3.1, upgrade coming next month 🤞 ) and was writing a tool to help understand it better.

I wrote the below script which gives this kind of output, I see keys matching the ones discussed above by @php4umagento so I’m going to leave debugging further until the upgrade works.

I’m thinking of adding my script to the CI pipeline though (after i prettify it and make it a bit more magento-y, this was a quick hack locally!) so that if something like this reoccurs again (in magento core or a custom module) i can at least know about it before it breaks 0 downtime deploys on production.

This debug script probably won’t cover everything off as its only been run against one instance, but a good start. And even if I have missed a case it is still a non-zero exit code which will flag in my pipeline.

drop_reference                FK_12C2962523CC7EF2EA73AD4121B5DBFF               
drop_reference                MAGENTO_BANNER_CATRULE_RULE_ID_SEQUENCE_CATRULE_SEQUENCE_VAL
drop_reference                MAGENTO_BANNER_SALESRULE_RULE_ID_SEQUENCE_SALESRULE_SEQUENCE_VAL
drop_reference                MAGENTO_TARGETRULE_PRD_PRD_ID_SEQUENCE_PRD_SEQUENCE_VAL
drop_reference                MAGENTO_GIFTREGISTRY_ITEM_PRD_ID_SEQUENCE_PRD_SEQUENCE_VAL
drop_reference                MAGENTO_CATEVENT_EVENT_CTGR_ID_SEQUENCE_CAT_CTGR_SEQUENCE_VAL
drop_reference                FK_81ACD87B0CB5AB8CC9B8D0B05F30FBB8               
drop_reference                MAGENTO_SALESRULE_FILTER_RULE_ID_SEQUENCE_SALESRULE_SEQUENCE_VAL
drop_reference                VISUAL_MERCHANDISER_RULE_CTGR_ID_SEQUENCE_CAT_CTGR_SEQUENCE_VAL
drop_reference                MAGENTO_CATPERMISSIONS_CTGR_ID_SEQUENCE_CAT_CTGR_SEQUENCE_VAL
add_complex_element           MAGENTO_VERSIONSCMS_HIERARCHY_NODE_PAGE_ID_CMS_PAGE_PAGE_ID
add_complex_element           MAGENTO_BANNER_CATALOGRULE_RULE_ID_CATALOGRULE_RULE_ID
add_complex_element           MAGENTO_BANNER_SALESRULE_RULE_ID_SALESRULE_RULE_ID
add_complex_element           MAGENTO_TARGETRULE_PRD_PRD_ID_CAT_PRD_ENTT_ENTT_ID
add_complex_element           MAGENTO_GIFTREGISTRY_ITEM_PRD_ID_CAT_PRD_ENTT_ENTT_ID
add_complex_element           MAGENTO_CATEVENT_EVENT_CTGR_ID_CAT_CTGR_ENTT_ENTT_ID
add_complex_element           MAGENTO_REMINDER_RULE_SALESRULE_ID_SALESRULE_RULE_ID
add_complex_element           MAGENTO_SALESRULE_FILTER_RULE_ID_SALESRULE_RULE_ID
modify_column                 magento_sales_creditmemo_grid_archive adjustment_positive
modify_column                 magento_sales_creditmemo_grid_archive adjustment_negative
add_complex_element           VISUAL_MERCHANDISER_RULE_CTGR_ID_CAT_CTGR_ENTT_ENTT_ID
add_complex_element           MAGENTO_CATPERMISSIONS_CTGR_ID_CAT_CTGR_ENTT_ENTT_ID

<?php
use Magento\Framework\App\Bootstrap;
require __DIR__ . '/app/bootstrap.php';

$bootstrap = Bootstrap::create(BP, $_SERVER);
$obj = $bootstrap->getObjectManager();
$state = $obj->get(Magento\Framework\App\State::class);
$state->setAreaCode('adminhtml');

$diff   = $obj->get(Magento\Framework\Setup\Declaration\Schema\Diff\SchemaDiff::class);
$config = $obj->get(Magento\Framework\Setup\Declaration\Schema\SchemaConfigInterface::class);

/**
 * @see Magento\Framework\Setup\Declaration\Schema\UpToDateDeclarativeSchema::isUpToDate()
 */
$declarativeSchema = $config->getDeclarationConfig();
$dbSchema = $config->getDbConfig();
$differences = $diff->diff($declarativeSchema, $dbSchema)->getAll();
if (empty($differences)) {
    exit(0);
}

foreach ($differences as $differenceGroups) {
    foreach ($differenceGroups as $differenceGroupTitle => $differences) {
        /** @var \Magento\Framework\Setup\Declaration\Schema\ElementHistory $difference */
        foreach ($differences as $difference) {
            $old = $difference->getOld();
            $new = $difference->getNew();

            echo str_pad($differenceGroupTitle, 30);

            switch ($differenceGroupTitle) {
                case 'add_complex_element':
                    if ($old !== null) {
                        throw new \Exception("Complex object with an old version?");
                    }
                    echo str_pad($new->getName(), 50);
                    break;
                case 'modify_column':
                    if ($old->getName() != $new->getName()) {
                        throw new \Exception("Name mismatch with " . $old->getName() . " and " . $new->getName());
                    }
                    echo str_pad($new->getTable()->getName() . ' ' . $new->getName(), 50);
                    break;
                case 'add_column':
                    if ($old !== null) {
                        throw new \Exception("Can't add a column with an old one?");
                    }
                    echo str_pad($new->getTable()->getName() . ' ' . $new->getName(), 50);
                    break;
                default:
                    if ($old->getName() != $new->getName()) {
                        throw new \Exception("Name mismatch with " . $old->getName() . " and " . $new->getName());
                    }
                    echo str_pad($old->getName(), 50);
            }
            echo PHP_EOL;
        }
    }
}

exit(1);

+12

convenient on Aug 13, 2020

Hi,

I have done some extra debugging into this. My setup is on MariaDB version 10.2 and this gives issues. Please have a look at the following screenshots:

Table declaration on MySQL 5.7

Table declaration on MariaDB 10.2

As you can see MariaDB changes CURRENT_TIMESTAMP into current_timestamp(). Due to this change the class Magento\Framework\Setup\Declaration\Schema\Comparer will compare the DB schema to the declared schema, which is not the same so this will returning false:

# Declared array in compare function
Array
(
    [type] => timestamp
    [default] => CURRENT_TIMESTAMP
    [onUpdate] => 
    [comment] => Bulk start time
)
# DB schema array in compare function
Array
(
    [type] => timestamp
    [default] => current_timestamp()
    [onUpdate] => 
    [comment] => Bulk start time
)

There’s also an issue with NULL fields:

# Declared array in compare function
Array
(
    [type] => varbinary
    [nullable] => 1
    [default] => 
    [length] => 39
    [comment] => Related Bulk UUID
)
# DB schema array in compare function
Array
(
    [type] => varbinary
    [nullable] => 1
    [default] => NULL
    [length] => 39
    [comment] => Related Bulk UUID
)

And with defaults on integer fields, where MariaDB will set the default to “0”

# Declared array in compare function
Array
(
    [type] => int
    [nullable] => 1
    [padding] => 11
    [unsigned] => 
    [identity] => 
    [default] => 
    [comment] => Which type of user
)
# DB schema array in compare function
Array
(
    [type] => int
    [nullable] => 1
    [padding] => 11
    [unsigned] => 
    [identity] => 
    [default] => 0
    [comment] => Which type of user
)

According to the Magento 2.3 documentation it’s compatible with MariaDB, but unfortunately it’s not: https://devdocs.magento.com/guides/v2.3/install-gde/system-requirements-tech.html

+12

dverkade on Dec 7, 2018

Hi @davidalger You can report EE related issue in one of two ways You can use the Support portal associated with your account or If you are a Partner reporting on behalf of a merchant, use the Partner portal.

GitHub is intended for Magento Open Source users to report on issues related to Open Source only. There are no account management services associated with GitHub.

For Open Source it’s now only reproducible on versions of database

ghost on Dec 10, 2018

@cmacdonald-au I had missed that note, thanks! Just checked with the following in my docker file:

    image: mariadb:10.3
    command:
      - mysqld
      - --explicit_defaults_for_timestamp=on

Success! 🎉

$ bin/magento setup:db:status
All modules are up to date.

Seems to be ok with and without sample data as long as explicit_defaults_for_timestamp=on is there. MariaDB 10.3 vs MySQL 5.7 doesn’t make a difference (i.e. it is required on both). I’ve got a production PerconaDB 5.7 system running Commerce which I’ll get to find out on relatively soon (we’re working on getting that baby upgraded from 2.2.x to 2.3.4 as I write this). Here’s to hoping 🤞

davidalger on Mar 24, 2020

I have debugged this extensively today. the problem is that some core modules use db_schema.xml to define a foreign key contraint, but then also have schema-recurring scripts that changes the constraints again. so each time a db-schema update is done, constraints are created, then removed, and other constraints created again.

as an example the module magento/module-bundle:

the db-schema declares the foreign key constraint CAT_PRD_BNDL_SELECTION_PRD_ID_CAT_PRD_ENTT_ENTT_ID to the table catalog_product_bundle_selection
vendor/magento/module-bundle/Setup/Recurring.php uses ExternalFKSetup to add an “additional” foreign key MAGENTO_SALESRULE_FILTER_RULE_ID_SEQUENCE_SALESRULE_SEQUENCE_VAL to the same table.
ExternalFKSetup removes all old FK when it creates the new one

therefore, setup:db:status can never be clean. it always detects the removed constraint, and while setup:db-schema:upgrade recreates it, it is instantly removed again during the recurring phase.

in one of our projects, I identified this behavior for the following modules:

magento/module-bundle
magento/module-target-rule
magento/advanced-sales-rule there might be more cases tho.

heldchen on May 21, 2019

Any updates on this issue please?

csdougliss on Feb 11, 2019

I can also verify that this is an issue in Magento 2.3.4. The issue is within the new Magento_AdobeIms module with the field access_token_expires_at. It will always try to run ALTER TABLE adobe_user_profile MODIFY COLUMN access_token_expires_at timestamp NOT NULL COMMENT "Access Token Expires At"

If you dont use the Magento_AdobeIms module, just disabling it is a workaround, but doesn’t fix the actual issue.

kurtinge on Jan 30, 2020

Has that issue with v2.3.6 Running the script @convenient provided produces this output: modify_column adobe_user_profile access_token_expires_at

-> fixed by disabling modules: Magento_AdobeStockImageAdminUi Magento_AdobeIms

kanevbg on Mar 9, 2021

Two things from this side;

One of the modules we brought in did not appropriately specify the default & on_update values for a timestamp column. We needed to fix that && set explicit_defaults_for_timestamp = on within my.cnf to prevent the database from applying schema settings that were not defined in db_schema.xml (running percona 5.7 here) - this would consistently report differences in setup:db:status (ref: https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_explicit_defaults_for_timestamp) I don’t believe this is mentioned in the Magento system setup docs anywhere…
For those of you in EE land, the sales archive module synchronises its tables against those in the base sales module - so the schema may drift unless you manually reconcile them yourself in schema/patches.

Once we worked those out - the patches as specified above by @heldchen & @php4umagento resolved the problem for us

cmacdonald-au on Jan 13, 2020

Does not appear to be resolved. Spun up https://github.com/davidalger/m2demo/ using ./start.sh --no-sampledata (making this an easy reproducible test case for Open Source) and re-built images with 2.3.4 codebase and got the following:

www-data@m2demo-php-fpm:01:54 PM:/var/www/html$ bin/magento --version
Magento CLI 2.3.4
www-data@m2demo-php-fpm:01:54 PM:/var/www/html$ bin/magento setup:db:status
Declarative Schema is not up to date
Run 'setup:upgrade' to update your DB schema and data.
www-data@m2demo-php-fpm:01:54 PM:/var/www/html$ php -v
PHP 7.3.13 (cli) (built: Dec 19 2019 03:21:53) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.13, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.3.13, Copyright (c) 1999-2018, by Zend Technologies
www-data@m2demo-php-fpm:01:54 PM:/var/www/html$ mysql -hdb -umagento -pmagento -e "show variables like 'version'"
+---------------+----------------------------------------+
| Variable_name | Value                                  |
+---------------+----------------------------------------+
| version       | 10.3.21-MariaDB-1:10.3.21+maria~bionic |
+---------------+----------------------------------------+

Same exact results if I install on mariadb:10.2 image. In fact, if I deploy on mysql:5.7 image same story…reports incorrectly:

www-data@m2demo-php-fpm:02:11 PM:/var/www/html$ bin/magento setup:db:status
Declarative Schema is not up to date
Run 'setup:upgrade' to update your DB schema and data.
www-data@m2demo-php-fpm:02:12 PM:/var/www/html$ mysql -hdb -umagento -pmagento -e "show variables like 'version'"
+---------------+--------+
| Variable_name | Value  |
+---------------+--------+
| version       | 5.7.29 |
+---------------+--------+
www-data@m2demo-php-fpm:02:12 PM:/var/www/html$ bin/magento --version
Magento CLI 2.3.4

This is with 2.3.4 Open Source images freely available here: https://hub.docker.com/r/davidalger/magento

davidalger on Jan 29, 2020

It looks like a fix was added to the core yesterday: https://github.com/magento/magento2/commit/edec48ae150b0173f2f967058777295e6c615dc3

Not sure if it will fix all reports in this thread though, but I think it should fix the ones @dverkade mentioned at least.

hostep on Nov 2, 2019

@cmacdonald-au I opened a PR to get a note added: https://github.com/magento/devdocs/pull/6916

davidalger on Mar 24, 2020

@kurtinge @davidalger @Parakoopa Any of you feel like / capable of taking this one on? ref: https://github.com/magento/devdocs/pull/6855#issuecomment-599291562 - that PR draft will likely get canned as I have SFA chance of getting the CLA processed within the next few months…

pleeease 😃

cmacdonald-au on Mar 22, 2020

@rganin It looks like you’re the one who committed the code (or at least merged the PR) that multiple people have reported as not fixing the issue reported here. It looks like you’re not subscribed to this issue, so I’m pinging you here so you’re aware of the comments above.

erikhansen on Dec 22, 2019

@0x15f : Main issue with EE is, that there are some modifications after Schema-Update, like entity_id to row_id. This is executed after SchemaDiff, but setup:db:status checks only, if SchemaDiff is empty.

jwundrak on Dec 17, 2019

Noting here that I implemented edec48a as a patch on my 2.3.2-p2 local environment and after setup:upgrade I’m still seeing setup:db:status report that Declarative Schema is not up to date

Same for me on 2.3.3.

alexgayshunewave on Dec 15, 2019

Confirmed with Magento 2.3.1 on MySQL 5.6.

marciomaciel on Aug 14, 2019

Ok I got it working - by applying @heldchen two patches and adding 3 of mine:

+++ a/vendor/magento/module-cms-staging/etc/db_schema.xml	(date 1562853317000)
@@ -46,6 +46,9 @@
         <constraint xsi:type="foreign" referenceId="CMS_PAGE_PAGE_ID_SEQUENCE_CMS_PAGE_SEQUENCE_VALUE" table="cms_page"
                     column="page_id" referenceTable="sequence_cms_page" referenceColumn="sequence_value"
                     onDelete="CASCADE"/>
+        <constraint xsi:type="foreign" referenceId="FK_12C2962523CC7EF2EA73AD4121B5DBFF" table="cms_page"
+                    column="page_id" referenceTable="sequence_cms_page" referenceColumn="sequence_value"
+                    onDelete="CASCADE"/>
         <index referenceId="CMS_PAGE_CREATED_IN" indexType="btree">
             <column name="created_in"/>
         </index>

--- a/vendor/magento/module-reminder/etc/db_schema.xml	(date 1562853202000)
+++ a/vendor/magento/module-reminder/etc/db_schema.xml	(date 1562853202000)
@@ -31,6 +31,9 @@
         <constraint xsi:type="foreign" referenceId="MAGENTO_REMINDER_RULE_SALESRULE_ID_SALESRULE_RULE_ID"
                     table="magento_reminder_rule" column="salesrule_id" referenceTable="salesrule"
                     referenceColumn="rule_id" onDelete="SET NULL"/>
+        <constraint xsi:type="foreign" referenceId="FK_81ACD87B0CB5AB8CC9B8D0B05F30FBB8"
+                    table="magento_reminder_rule" column="salesrule_id" referenceTable="sequence_salesrule"
+                    referenceColumn="sequence_value" onDelete="SET NULL"/>
         <index referenceId="MAGENTO_REMINDER_RULE_SALESRULE_ID" indexType="btree">
             <column name="salesrule_id"/>
         </index>

--- a/vendor/magento/module-versions-cms/etc/db_schema.xml	(date 1562853050000)
+++ a/vendor/magento/module-versions-cms/etc/db_schema.xml	(date 1562853050000)
@@ -60,6 +60,9 @@
             <column name="scope"/>
             <column name="scope_id"/>
         </constraint>
+        <constraint xsi:type="foreign" referenceId="FK_12C2962523CC7EF2EA73AD4121B5DBFF"
+                    table="magento_versionscms_hierarchy_node" column="page_id"
+                    referenceTable="sequence_cms_page" referenceColumn="sequence_value" onDelete="CASCADE"/>
         <index referenceId="MAGENTO_VERSIONSCMS_HIERARCHY_NODE_PARENT_NODE_ID" indexType="btree">
             <column name="parent_node_id"/>
         </index>

php4umagento on Jul 11, 2019

ExternalFKSetup should only create missing foreign keys without deleting unrelated ones. here’s a patch that does just that:

diff --git a/Setup/ExternalFKSetup.php b/Setup/ExternalFKSetup.php
index 4247b7b1a..42642f747 100644
--- a/Setup/ExternalFKSetup.php
+++ b/Setup/ExternalFKSetup.php
@@ -82,8 +82,54 @@ class ExternalFKSetup
             $this->setup->getTable($this->entityTable)
         );
         if (!$entityTableInfo[$this->entityColumn]['PRIMARY']) {
-            $this->dropOldForeignKey();
-            $this->addForeignKeys();
+
+            // find the foreign keys on the entityTable
+
+            $entityForeignKeys = $this->setup->getConnection()->getForeignKeys(
+                $this->setup->getTable($this->entityTable)
+            );
+            $entityForeignKeys = array_filter(
+                $entityForeignKeys,
+                function ($key) {
+                    return $key['COLUMN_NAME'] == $this->entityColumn;
+                }
+            );
+
+            $targetForeignKeys = $this->setup->getConnection()->getForeignKeys(
+                $this->setup->getTable($this->externalTable)
+            );
+
+            // for each found key, create one on the targetTable
+
+            foreach ($entityForeignKeys as $entityForeignKey) {
+
+                $targetForeignKeyName = $this->setup->getFkName(
+                    $this->externalTable,
+                    $this->externalColumn,
+                    $this->setup->getTablePlaceholder($entityForeignKey['REF_TABLE_NAME']),
+                    $entityForeignKey['REF_COLUMN_NAME']
+                );
+
+                // check if the foreign key on the targetTable already exists
+
+                foreach ($targetForeignKeys as $targetForeignKey) {
+                    if ($targetForeignKey['FK_NAME'] === $targetForeignKeyName) {
+                        // skip adding
+                        continue 2;
+                    }
+                }
+
+                // add foreign key on targetTable based on the entityTable FK
+
+                $this->setup->getConnection()->addForeignKey(
+                    $targetForeignKeyName,
+                    $this->setup->getTable($this->externalTable),
+                    $this->externalColumn,
+                    $entityForeignKey['REF_TABLE_NAME'],
+                    $entityForeignKey['REF_COLUMN_NAME'],
+                    $this->onDelete
+                );
+            }
         } else {
             $this->addDefaultForeignKey();
         }

this patch is not sufficient to get setup:db:status working ok, more changes are required as the ExternalFKSetup foreign keys are being removed by the db_schema installer on the next run.

for my case, all the offending modules created FKs with xxxx_SEQUENCE_VALS names, so I just have the TableDiff class ignore those:

diff --git a/Setup/Declaration/Schema/Diff/TableDiff.php b/Setup/Declaration/Schema/Diff/TableDiff.php
index 87045df86..6814ce627 100644
--- a/Setup/Declaration/Schema/Diff/TableDiff.php
+++ b/Setup/Declaration/Schema/Diff/TableDiff.php
@@ -203,6 +203,14 @@ class TableDiff
             }
 
             //Elements that should be removed
+            if ($elementType == self::CONSTRAINT_DIFF_TYPE) {
+                foreach (array_keys($generatedElements) as $elementName) {
+                    if (preg_match('/_SEQUENCE_VAL$/', $elementName)) {
+                        // remove auto-generated extra constraints
+                        unset($generatedElements[$elementName]);
+                    }
+                }
+            }
             if ($this->diffManager->shouldBeRemoved($generatedElements)) {
                 $diff = $this->diffManager->registerRemoval($diff, $generatedElements);
             }

use together with @yanncharlou’s code at your own risk

heldchen on May 21, 2019