orm: Significant performance degradation with `enable_lazy_ghost_objects: true`

Here is a screenshot of a compare callgraph in Tideways Profiler, it shows the relevant parts that are slow:

• createLazyGhost static factory
• The callback returned from ProxyFactory::getProxyFactory

The overall slowdown is roughly in line what @rrehbein reported.

I will take a deeper look when i have more time.

I looked at createLazyGhost factory method and found that it acts both as an initializer when invoked the first time, and then as a factory for lazy ghosts. It seems when the initializer code is run every time as well, this causes a lot of wasted CPU and allocated memory. A simple attempt to split both methods up cut 50% of the slowdown:

I hardcoded it into my doctrine/orm and symfony/var_exporter components of @javer testproject. So no nice diff, the changed lines are here: https://gist.github.com/beberlei/97899ea7ce4261502a6df1300ab9f181

This would require var-exporter to add these new methods, so I am looking to find a simpler fix that cuts the time and memory in createLazyGhost already.

Below is a better patch for the ORM side. Together with this one https://github.com/symfony/symfony/pull/54253, I get the following stats on my machine:

• lazy-ghosts disabled: 2.94 s
• lazy-ghosts patched: 4.02 s
• lazy-ghosts unpatched: 7.83 s

Locally, I also have a native proxy implementation for php-src (I need to write an RFC about it), and with it, I get 3.27s with the current API and implementation. That’s good food for thoughts on my side to move this RFC forward 😃

Here is the Blackfire.io comparison between disabled and patched lazy-ghosts: https://blackfire.io/profiles/compare/4561cec4-97af-4c40-a180-ee5745ca8119/graph

--- a/src/Proxy/ProxyFactory.php
+++ b/src/Proxy/ProxyFactory.php
@@ -360,11 +360,11 @@ EOPHP;
      */
     private function createLazyInitializer(ClassMetadata $classMetadata, EntityPersister $entityPersister, IdentifierFlattener $identifierFlattener): Closure
     {
-        return static function (InternalProxy $proxy, InternalProxy $original) use ($entityPersister, $classMetadata, $identifierFlattener): void {
-            $identifier = $classMetadata->getIdentifierValues($original);
-            $entity     = $entityPersister->loadById($identifier, $original);
+        return static function (InternalProxy $proxy) use ($entityPersister, $classMetadata, $identifierFlattener): void {
+            $identifier = $classMetadata->getIdentifierValues($proxy);
+            $original   = $entityPersister->loadById($identifier);
 
-            if ($entity === null) {
+            if ($original === null) {
                 throw EntityNotFoundException::fromClassNameAndIdentifier(
                     $classMetadata->getName(),
                     $identifierFlattener->flattenIdentifier($classMetadata, $identifier)
@@ -382,7 +382,7 @@ EOPHP;
                     continue;
                 }
 
-                $property->setValue($proxy, $property->getValue($entity));
+                $property->setValue($proxy, $property->getValue($original));
             }
         };
     }
@@ -468,9 +468,7 @@ EOPHP;
         $identifierFields = array_intersect_key($class->getReflectionProperties(), $identifiers);
 
         $proxyFactory = Closure::bind(static function (array $identifier) use ($initializer, $skippedProperties, $identifierFields, $className): InternalProxy {
-            $proxy = self::createLazyGhost(static function (InternalProxy $object) use ($initializer, &$proxy): void {
-                $initializer($object, $proxy);
-            }, $skippedProperties);
+            $proxy = self::createLazyGhost($initializer, $skippedProperties);
 
             foreach ($identifierFields as $idField => $reflector) {
                 if (! isset($identifier[$idField])) {

This is good enough to me. […] The legacy proxy system has many serious limitations.

I don’t doubt that the new setup is better, and I’m sure eventually it will catch up, but right now we’re seeing the same results as you outline in https://github.com/doctrine/orm/issues/11087#issuecomment-1994002826, so losing half the performance (from 17s to 29s for setting up a subset of test fixtures).

That’s many times better than before your patch, of course, but still a fairly high cost… I hope the issue keeps getting some attention until we’re in a closer performance range.

EDIT: turns out I was comparing on 2.19.2 which lacks the patch; using 2.19.3 seems to fix the issue entirely for our use-case (at least with synthetic loads, we’ll have to compare for real load).

I can confirm that after upgrading to doctrine/orm 2.19.3 turning on enable_lazy_ghost_objects makes the initial test suite slower only by negligible 5%.

Thank you very much @beberlei and @nicolas-grekas !

turns out I was comparing on 2.19.2 which lacks the patch; using 2.19.3 seems to fix the issue entirely for our use-case (at least with synthetic loads, we’ll have to compare for real load).

For what it’s worth, we also see no performance difference at all after deploying to our production:

So I can only agree with the decision to close it 👍