magento2: session_start(): Failed to read session data: user (path: /var/lib/php/sessions)

@AykutCevik I believe this is indeed due to the max_concurrency value in env.php under the session => redis section. Magento is failing to save/retrieve sessions from the redis back-end based on the value of this config. However the error that’s being reported here is stating that having failed to access Redis, the session also couldn’t be accessed from the native PHP session directory (it may not exist on the server in the php.ini specified location, have a check on your server).

@engcom-backlog-nickolas, to consistently replicate the issue please try the following steps:

Preconditions

• Redis enabled for sessions
• Sessions Redis env.php config as per defaults in M2 devdocs (Max_concurency => 6)

Steps to reproduce

• Using Google Chrome, login to the admin area
• Open 10 or more browser tabs with the orders grid page
• Click the left most tab
• Hold shift and click the right most tab (this selects all tabs)
• Right click the right most tab and select “reload”
• Check the tabs after they have all reloaded, only ~6 will successfully load, the rest will exhibit this error *
• Check the var/report folder for the start_session error report relating to the tabs that failed.
• Now modify the max_concurrency to a value greater than the number of tabs, reload them all again and none will fail.

* if you aren’t seeing this error, check in your PHP sessions directory. It should be empty (as Redis is the session back-end of course), however if there are session files building up in the directory, you are seeing a symptom of the same problem here, although it hides the error because PHP handles the session as a fallback. @AykutCevik try a php -i | grep session and check your session save path is available and has the correct permissions. Once corrected you’ll likely see PHP handling the sessions rather than your error.

This is all happening because of contention for accessing session data from single threaded Redis. The max_concurency config value defines the size of the queue that can form to access Redis and retrieve the session. In the event of lots of traffic hitting the server at once, some users will receive this behavior because the queue is full, so the request to access the Redis session back-end is denied. PHP session handling kicks in here and if configured correctly will start creating sessions (this is still an issue as Redis is meant to be doing this for a reason, PHP will be saving to local disk so sessions wont be available across multiple web servers etc).

This same error will happen also on the front-end of Magento2. Try it by opening 10 or more tabs of different pages; clear the FPC (or just disable full page cache to emulate hitting un-cached pages); reload all the tabs at once as above and again some will fail. This is particularly bad when you have items in your cart as the local storage is not synched, resulting in the “I have no items in my cart, but there are items in my mini-cart” scenario I’m sure you’ve come across.

I would suggest that “6” as a default for max_concurrency is no good for most sites and that a better explanation of how to derive the correct value for the config should be made available. The Devdocs state “at least 10% of the number of PHP processes” but not why. I would assume that with 10% covered, in the unlikely(ish) event all your users hit pages at the same time (think Flash sale, Black Friday etc), 90% would receive this behavior still. I’m guessing the 10% suggestion is a stab in the dark at the likely percentage of requests that might all hit at the same time. This value should probably be derived from the max_children / max_requests values of the server’s PHP-FPM pools, looking to accommodate as many requests as might come in concurrently.

Would be great to get a view from Magento on this!

Thanks

@AykutCevik, we are closing this issue due to inactivity. If you’d like to update it, please reopen the issue.

We are still getting this issue even after increasing the max_concurrency to 18. We have set it now to 54 and are observing if this issue still occurs.

However - I still do not quite get why the native session fallback does not work. If I disable Redis for sessions, sessions with the native session storage work fine. But why does the fallback not work, when using Redis as the session storage?

After reading through this thread multiple times and looking into the relevant Magento code, I think that the following is going on:

• Magento can fall back to file based session handling, when Redis is not available at all (see \Magento\Framework\Session\SaveHandler::callSafely), but cannot handle the situation when the session is already created in Redis, but it can’t be read from, due to a concurrency issue. Looking at \Magento\Framework\Session\SaveHandler\Redis::read, we can see that on ConcurrentConnectionsExceededException Magento immediately redirects to a 503 page. It doesn’t even try to fall back to the file based session handling.
• The error that we get (Warning: session_start(): Failed to read session data: user) is not related to permission issues. When configuring Magento save the sessions to files, instead of redis, it can perfectly read and write from/to the same location.

Considering all this, I ended up subclassing \Magento\Framework\Session\SaveHandler\Redis, and I added some logic to retry the read on ConcurrentConnectionsExceededException, with exponential backoff. This is what I came up with: https://gist.github.com/pkarsai/0dcc35294870c057b9be9803cba891b6. The tests look promising so far, but it’s not yet in production. BTW, max_concurrency was set to 30 and were still having this problem, that’s why I ended up implementing this custom solution.

Just came here to see that the issue is not resolved for over 3 years. 😔

This happens for me sometimes on the frontend product page. Magento 2.4.3 with redis-server 6.2.5

'session' => [
        'save' => 'redis',
        'redis' => [
            'host' => '127.0.0.1',
            'port' => '6379',
            'password' => '',
            'timeout' => '2.5',
            'persistent_identifier' => '',
            'database' => '2',
            'compression_threshold' => '2048',
            'compression_library' => 'gzip',
            'log_level' => '4',
            'max_concurrency' => '6',
            'break_after_frontend' => '5',
            'break_after_adminhtml' => '30',
            'first_lifetime' => '600',
            'bot_first_lifetime' => '60',
            'bot_lifetime' => '7200',
            'disable_locking' => '0',
            'min_lifetime' => '60',
            'max_lifetime' => '2592000',
            'sentinel_master' => '',
            'sentinel_servers' => '',
            'sentinel_connect_retries' => '5',
            'sentinel_verify_master' => '0'
        ]
    ],

Error:
Warning: session_start(): Failed to read session data: user (path: /var/lib/php/sessions) in vendor/magento/framework/Session/SessionManager.php on line 204

Trace:
#1 session_start() called at [vendor/magento/framework/Session/SessionManager.php:204]
#2 Magento\Framework\Session\SessionManager->start() called at [generated/code/Magento/Customer/Model/Session/Interceptor.php:23]
#3 Magento\Customer\Model\Session\Interceptor->start() called at [vendor/magento/framework/Session/SessionManager.php:141]
#4 Magento\Framework\Session\SessionManager->__construct() called at [vendor/magento/module-customer/Model/Session.php:180]
#5 Magento\Customer\Model\Session->__construct() called at [generated/code/Magento/Customer/Model/Session/Interceptor.php:14]
#6 Magento\Customer\Model\Session\Interceptor->__construct() called at [vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php:121]
#7 Magento\Framework\ObjectManager\Factory\AbstractFactory->createObject() called at [vendor/magento/framework/ObjectManager/Factory/Compiled.php:108]
#8 Magento\Framework\ObjectManager\Factory\Compiled->create() called at [vendor/magento/framework/ObjectManager/ObjectManager.php:56]
#9 Magento\Framework\ObjectManager\ObjectManager->create() called at [generated/code/Magento/Customer/Model/SessionFactory.php:43]
#10 Magento\Customer\Model\SessionFactory->create() called at [vendor/amasty/mostviewed/Model/Customer/GroupValidator.php:36]
#11 Amasty\Mostviewed\Model\Customer\GroupValidator->validate() called at [vendor/amasty/mostviewed/Model/Repository/GroupRepository.php:192]
#12 Amasty\Mostviewed\Model\Repository\GroupRepository->validateGroup() called at [vendor/amasty/mostviewed/Model/Repository/GroupRepository.php:136]
#13 Amasty\Mostviewed\Model\Repository\GroupRepository->getGroupByIdAndPosition() called at [vendor/amasty/mostviewed/Model/ProductProvider.php:167]
#14 Amasty\Mostviewed\Model\ProductProvider->modifyCollection() called at [vendor/amasty/mostviewed/Plugin/Community/AbstractProduct.php:74]
#15 Amasty\Mostviewed\Plugin\Community\AbstractProduct->prepareCollection() called at [vendor/amasty/mostviewed/Plugin/Community/Related.php:24]
#16 Amasty\Mostviewed\Plugin\Community\Related->afterGetItems() called at [vendor/magento/framework/Interception/Interceptor.php:146]
#17 Magento\Catalog\Block\Product\ProductList\Related\Interceptor->Magento\Framework\Interception\{closure}() called at [vendor/magento/framework/Interception/Interceptor.php:153]
#18 Magento\Catalog\Block\Product\ProductList\Related\Interceptor->___callPlugins() called at [generated/code/Magento/Catalog/Block/Product/ProductList/Related/Interceptor.php:23]
#19 Magento\Catalog\Block\Product\ProductList\Related\Interceptor->getItems() called at [vendor/magento/module-catalog/Block/Product/ProductList/Related.php:147]
#20 Magento\Catalog\Block\Product\ProductList\Related->getIdentities() called at [vendor/magento/module-page-cache/Model/Layout/LayoutPlugin.php:96]
#21 Magento\PageCache\Model\Layout\LayoutPlugin->afterGetOutput() called at [vendor/magento/framework/Interception/Interceptor.php:146]
#22 Magento\Framework\View\Layout\Interceptor->Magento\Framework\Interception\{closure}() called at [vendor/magento/framework/Interception/Interceptor.php:153]
#23 Magento\Framework\View\Layout\Interceptor->___callPlugins() called at [generated/code/Magento/Framework/View/Layout/Interceptor.php:41]
#24 Magento\Framework\View\Layout\Interceptor->getOutput() called at [vendor/magento/framework/View/Result/Page.php:260]
#25 Magento\Framework\View\Result\Page->render() called at [vendor/magento/framework/View/Result/Layout.php:171]
#26 Magento\Framework\View\Result\Layout->renderResult() called at [vendor/magento/framework/Interception/Interceptor.php:58]
#27 Magento\Framework\View\Result\Page\Interceptor->___callParent() called at [vendor/magento/framework/Interception/Interceptor.php:138]
#28 Magento\Framework\View\Result\Page\Interceptor->Magento\Framework\Interception\{closure}() called at [vendor/magezon/module-core/Plugin/View/Result/Layout.php:24]
#29 Magezon\Core\Plugin\View\Result\Layout->aroundRenderResult() called at [vendor/magento/framework/Interception/Interceptor.php:135]
#30 Magento\Framework\View\Result\Page\Interceptor->Magento\Framework\Interception\{closure}() called at [vendor/magento/framework/Interception/Interceptor.php:153]
#31 Magento\Framework\View\Result\Page\Interceptor->___callPlugins() called at [generated/code/Magento/Framework/View/Result/Page/Interceptor.php:23]
#32 Magento\Framework\View\Result\Page\Interceptor->renderResult() called at [vendor/magento/framework/App/Http.php:120]
#33 Magento\Framework\App\Http->launch() called at [vendor/magento/framework/App/Bootstrap.php:264]
#34 Magento\Framework\App\Bootstrap->run() called at [pub/index.php:29]

Any help?

This issue still happening in Magento 2.3.4 @magento-engcom-team please give us your opinion.

HI @AykutCevik Thank you for you report, the fix for this issue has already available in 2.2-develop and 2.3-develop branch, fixed by this pr -> https://github.com/magento/magento2/pull/17608

Why this is P4 ? this issue is easy reproduce in any page just pressing multiple times F5 button and then we got

Hi,

Due to this issue, we faced a few issues in the custom extensions that were resulting in the partially broken product page (custom options and add to cart blocks) were missing AND then cached in Varnish.

How it happened - in the view file we had the code $block->isLoggedIn(), that inside was creating customer session and checking if the customer is logged in.

In the custom extension we had code similar to this:

        if ($this->customerSessionFactory->create()->isLoggedIn()) {
            $customerGroup = $this->customerSessionFactory->create()->getCustomer()->getGroupId();
        }

I strongly believe we should change severity & priority to S1 P2. @sidolov @gabrieldagama what do you think?

---

The investigation showed that basically when you’re just creating any session object - it tries to start the session and as result tries to connect to Redis.

In the constructor, it just executes the start method: https://github.com/magento/magento2/blob/caefb4e63f0c726c996118c2707afaf63ed3c665/lib/internal/Magento/Framework/Session/SessionManager.php#L117-L142

This method runs session_start. https://github.com/magento/magento2/blob/caefb4e63f0c726c996118c2707afaf63ed3c665/lib/internal/Magento/Framework/Session/SessionManager.php#L179-L222

It means example code from the extension that had just 2 lines were doing session_start 2 times.

---

Unfortunately, I found a similar code in many places in Magento. https://github.com/magento/magento2/blob/a31f4a35c018ad09654e5bd5871086b73fbd3d2d/app/code/Magento/Persistent/Observer/RenewCookieObserver.php#L83-L87 https://github.com/magento/magento2/blob/a31f4a35c018ad09654e5bd5871086b73fbd3d2d/app/code/Magento/Persistent/Observer/ClearExpiredCronJobObserver.php#L54

---

Seems like the correct fix for this issue would be one of those options, or maybe even both:

1. Implement the removal of session auto-start during the session object creation
2. Implementing optimistic locks for sessions

@ihor-sviziev thanks for investigation and detailed report! I changed priority to P2

I confirm, this issue exists in Magento 2.3.5-p1.

A Magento core dev may have a look since we can’t provide any updates.

Agree with @AykutCevik we have this appearing in var/report every now and then, its very difficult to track down and/or replicate but is happening. We are also using Redis for session saves.

I was able to replicate using the view order > view order grid as @AykutCevik suggested, for a split second I saw the error on the Order view page, before being taken to the order grid.

Here is the stack trace generated:

{"0":"Warning: session_start(): Failed to read session data: user (path: /var/session/) in /vendor/magento/framework/Session/SessionManager.php on line 189","1":"
0 [internal function]: Magento\\Framework\\App\\ErrorHandler->handler(2, 'session_start()...', '/var/www/vhosts...', 189, Array)\n
1 /vendor/magento/framework/Session/SessionManager.php(189): session_start()\n
2 /generated/code/Magento/Backend/Model/Session/Interceptor.php(24): Magento\\Framework\\Session\\SessionManager->start()\n
3 /vendor/magento/framework/Session/SessionManager.php(130): Magento\\Backend\\Model\\Session\\Interceptor->start()\n
4 /generated/code/Magento/Backend/Model/Session/Interceptor.php(14): Magento\\Framework\\Session\\SessionManager->__construct(Object(Magento\\Framework\\App\\Request\\Http), Object(Magento\\Framework\\Session\\SidResolver\\Proxy), Object(Magento\\Backend\\Model\\Session\\AdminConfig), Object(Magento\\Framework\\Session\\SaveHandler), Object(Magento\\Framework\\Session\\Validator), Object(Magento\\Framework\\Session\\Storage), Object(Magento\\Framework\\Stdlib\\Cookie\\PhpCookieManager), Object(Magento\\Framework\\Stdlib\\Cookie\\CookieMetadataFactory), Object(Magento\\Framework\\App\\State))\n
5 /vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php(111): Magento\\Backend\\Model\\Session\\Interceptor->__construct(Object(Magento\\Framework\\App\\Request\\Http), Object(Magento\\Framework\\Session\\SidResolver\\Proxy), Object(Magento\\Backend\\Model\\Session\\AdminConfig), Object(Magento\\Framework\\Session\\SaveHandler), Object(Magento\\Framework\\Session\\Validator), Object(Magento\\Framework\\Session\\Storage), Object(Magento\\Framework\\Stdlib\\Cookie\\PhpCookieManager), Object(Magento\\Framework\\Stdlib\\Cookie\\CookieMetadataFactory), Object(Magento\\Framework\\App\\State))\n
6 /vendor/magento/framework/ObjectManager/Factory/Compiled.php(108): Magento\\Framework\\ObjectManager\\Factory\\AbstractFactory->createObject('Magento\\\\Backend...', Array)\n
7 /vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\\Framework\\ObjectManager\\Factory\\Compiled->create('Magento\\\\Backend...')\n
8 /vendor/magento/framework/ObjectManager/Factory/Compiled.php(79): Magento\\Framework\\ObjectManager\\Factory\\Compiled->get('Magento\\\\Backend...')\n
9 /vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\\Framework\\ObjectManager\\Factory\\Compiled->create('Magento\\\\Store\\\\A...')\n
10 /vendor/magento/framework/ObjectManager/Factory/Compiled.php(79): Magento\\Framework\\ObjectManager\\Factory\\Compiled->get('Magento\\\\Store\\\\A...')\n
11 /vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\\Framework\\ObjectManager\\Factory\\Compiled->create('Magento\\\\Backend...')\n
12 /vendor/magento/framework/ObjectManager/Factory/Compiled.php(79): Magento\\Framework\\ObjectManager\\Factory\\Compiled->get('Magento\\\\Backend...')\n
13 /vendor/magento/framework/ObjectManager/ObjectManager.php(56): Magento\\Framework\\ObjectManager\\Factory\\Compiled->create('Magento\\\\Ui\\\\Cont...', Array)\n
14 /vendor/magento/framework/App/ActionFactory.php(40): Magento\\Framework\\ObjectManager\\ObjectManager->create('Magento\\\\Ui\\\\Cont...')\n
15 /vendor/magento/framework/App/Router/Base.php(297): Magento\\Framework\\App\\ActionFactory->create('Magento\\\\Ui\\\\Cont...')\n
16 /vendor/magento/framework/App/Router/Base.php(158): Magento\\Framework\\App\\Router\\Base->matchAction(Object(Magento\\Framework\\App\\Request\\Http), Array)\n
17 /vendor/magento/framework/App/FrontController.php(50): Magento\\Framework\\App\\Router\\Base->match(Object(Magento\\Framework\\App\\Request\\Http))\n
18 /vendor/magento/framework/Interception/Interceptor.php(58): Magento\\Framework\\App\\FrontController->dispatch(Object(Magento\\Framework\\App\\Request\\Http))\n
19 /vendor/magento/framework/Interception/Interceptor.php(138): Magento\\Framework\\App\\FrontController\\Interceptor->___callParent('dispatch', Array)\n
20 /vendor/magento/framework/Interception/Interceptor.php(153): Magento\\Framework\\App\\FrontController\\Interceptor->Magento\\Framework\\Interception\\{closure}(Object(Magento\\Framework\\App\\Request\\Http))\n
21 /generated/code/Magento/Framework/App/FrontController/Interceptor.php(26): Magento\\Framework\\App\\FrontController\\Interceptor->___callPlugins('dispatch', Array, Array)\n
22 /vendor/magento/framework/App/Http.php(135): Magento\\Framework\\App\\FrontController\\Interceptor->dispatch(Object(Magento\\Framework\\App\\Request\\Http))\n
23 /vendor/magento/framework/App/Bootstrap.php(256): Magento\\Framework\\App\\Http->launch()\n
24 /pub/index.php(37): Magento\\Framework\\App\\Bootstrap->run(Object(Magento\\Framework\\App\\Http\\Interceptor))\n
25 {main}","url":"/admin/mui/bookmark/save/key/0194bbb75ffe47f654ca4e5343628982d1264a99d6a680a341326e9111d89296/?isAjax=true","script_name":"/index.php"}

From searching online, it appears to be either a) a red herring message and some session read data is being returned as NULL or FALSE instead of ‘’, b) or Magento is falling back to the native session handler in \Magento\Framework\Session\SaveHandlerFactory::create c) or something else 😃

Please create pull request with your suggestion, will definitely review it!

Unfortunately I still know very little about Magento’s session handling implementation at this point, so it might take some time 😉.

Hello everyone and @engcom-backlog-nazar

I got some help from senior engineers at the company I host with.

They agreed and changed max_concurrency from 6 to 20.

Since then, I haven’t seen the problem again - fingers crossed!

Apologies for not replying sooner.

@chris-pook This seems to have gone away after increasing max_concurrency but it is very worrying

Update on the above comment

Having looked further in to this, I think potentially my above comment may not be completely accurate. The key being the code implementation of the function below:

\Cm\RedisSession\Handler::read()

This function implements the locking mechanism and appears to hold the lock per $sessionId. So this issue is not a problem globally across multiple sessions after all. I.e on a flash sale day, it would not be the case that the first 6 concurrent requests globally would get through and the rest bounce. Instead you would see all requests served but for each individual user’s session only 6 concurrent requests allowed through.

To test this, I have repeated the front-end mass concurrent reload steps from before (with max_concurrency set to 6 still) only this time using ab:

ab -c 100 -n 100 <M2-instance-domain>

Concurrency Level:      100
Time taken for tests:   6.037 seconds
Complete requests:      100
Failed requests:        0

Here 100 concurrent requests are served no problem with no failures despite sessions being started.

Having discovered this I am happy that the 10% of PHP processes (php-fpm max_children) suggestion is probably quite sound. You don’t want a single user session hogging the Redis instance making everyone else wait, so better to serve them 503 errors beyond their first X concurrent requests.

The likelihood is that this limit will only ever be exceeded by admin users trying to do too much at once, or abuse of the front-end.

It would still be great to get a line from Magento here to confirm. Apologies if this is going back over old ground that may have been covered elsewhere.

Thanks

May this be related to max_concurrency where the fallback is the default PHP session handling? 🤔 (https://devdocs.magento.com/guides/v2.2/config-guide/redis/redis-session.html)