magento2: HEAD request returns 404

Hi @kolucciy. Thank you for your report. The issue has been fixed in magento/magento2#21378 by @mattijv in 2.3-develop branch Related commit(s):

• 46191d82d04e2392cb58a2c03121987a14052043
• d8eb1608e5a5377bdf2fb723413402220c7b9965
• 72630cde9e3041efeac4f798dbeba0f3db586ab9
• 4f7db3a20848947424f1013d7102c443278d692d
• e198914f06b97dee9ed6fdc0e761f0f347500438
• 22b9e56dbb429cf2877a41779e23528c7fce8f0d
• 69bd2ddad2929226d1568ea646f156050aa6f0f7
• ce2e4d3fee94a4d0c18caeef43bf3e1b220d1218
• 56d4cc2f94376d201e6c2a12a8f10ee553ea0f1e
• 2b587896186370614f8aaf1b53f3fd6d3bd3a8c1
• 5b79a3083130aa2e1bd1277b9bddca8cf7015f84
• 6bac20887d203260a4429d46304b4d3fee3d38ee
• 880404cf816de072921e60c71d0adf21146fd3ff
• bc3ef7ad15b23869face298c51b2b3bd301326f8
• be7712026b24690d5e6a7e35d814874909716c7f
• fecce53f4cc2ba2edb6f227c271be49ce179e753
• 3d19c097479e8d7dcb9ee4c6390dac45d576955b
• 2a337f94624a584b7a678da2a9dae28f505d7e47
• b5683e86bf6ca0554ca63e3fe9b93ab73558370f
• 740e4bd80021fbfc8a781c3137c4b9c5cc0b176c

The fix will be available with the upcoming 2.3.2 release.

This is an issue with the Magento built-in full page cache. Here’s a quick fix for the issue while waiting for Magento to fix the root cause in the core:

Add this plugin in a di.xml somewhere

<type name="Magento\Framework\App\PageCache\Kernel">
    <plugin name="prevent_caching_404_head_results" type="VENDOR\MODULE\Plugin\PreventCachingHead404ResultsPlugin" />
</type>

And create the following class in the path defined by the di.xml file

<?php

namespace VENDOR\MODULE\Plugin;

class PreventCachingHead404ResultsPlugin
{
    /**
     * @var \Magento\Framework\App\Request\Http
     */
    protected $request;

    /**
     * PreventCachingHeadRequestsPlugin constructor.
     * @param \Magento\Framework\App\Request\Http $request
     */
    public function __construct(\Magento\Framework\App\Request\Http $request)
    {
        $this->request = $request;
    }

    /**
     * Prevent caching HEAD requests that return a 404 result
     * @param \Magento\Framework\App\PageCache\Kernel $subject
     * @param callable $proceed
     * @param \Magento\Framework\App\Response\Http $response
     * @return void
     */
    public function aroundProcess(
        \Magento\Framework\App\PageCache\Kernel $subject,
        callable $proceed,
        \Magento\Framework\App\Response\Http $response
    ) {
        if ($this->request->isHead() && $response->getHttpResponseCode() == 404) {
            return;
        }
        return $proceed($response);
    }
}

Alternatively you can disable the fullpage cache, but that is not advisable since it affects performance. Better solution is to put the store behind a Varnish cache, since that seems to be unaffected by this.

Got it. I only patched app/etc/di.xml, but now I understood that I (also?) have to patch the file vendor/magento/magento2-base/app/etc/di.xml. Thanks @mattijv!

@SteveEdson While waiting for the next release you can do a couple interim fixes:

1. Setup a Varnish cache infront of your Magento installation and use that instead of the Magento built-in full page cache. Running a Varnish cache is in any case beneficial.

and/or

1. Apply the change in this commit to the app/etc/di.xml file. That is the most important change to prevent HEAD requests from 404’ing your controllers (the rest is just fluff about handling the HEAD requests up to spec).

Have a production mode 2.3 site with Nginx that sometimes serves homepage as a 404 think this might be why. Suspect head requests sent by a “down notifier” service and then the 404 is cached and served up by full page cache.

Confirm 404 when doing Head request, flushing FPC cleans it out.

jQuery.ajax({ type: "HEAD", async: true, url: '/', }).done(function(message,text,jqXHR){ console.log(message, text, jqXHR.status) });

Someone made a plugin module to work around this in the meantime, same as mattijv’s earlier post. https://github.com/magic42/module-prevent-head-404-cache

HEAD requests should be supported automatically (on the framework level) for all GET requests, just w/o body. Caching should take this into account (GET should have separate cache to avoid empty body).

Looks like also supported by @Vinai