pdo-event-store: Running projections causes constant full table scans due incorrect forced index

We’re developing a pretty new app using prooph, with only ~20k events in a single stream. We have ~30 projections, and when running these all at once the database gets completely destroyed.

I dived in with a debugger and figured out this is because we are doing 30x full table scans per 100ms, as every loop of each projection is doing a full table scan.

Here’s an example query:

SELECT * FROM `_4228e4a00331b5d5e751db0481828e22a2c3c8ef` USE INDEX(ix_query_aggregate)
WHERE `no` >= 19067
ORDER BY `no` ASC
LIMIT 1000;

image

Note the USE INDEX(ix_query_aggregate). This forces mysql to not use the the primary index on no.

Removing the USE INDEX means that the index is used properly, and the query is orders of magnitude faster.

image

Apologies if this is a misconfiguration from our end, but I think that the ix_query_aggregate index should only be using when doing queries based on metadata, not when scanning the event stream.

Running: prooph/pdo-event-store: v1.12.0 prooph/event-store: v7.5.6

Thanks in advanced!

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 15 (9 by maintainers)

Most upvoted comments

No worries, i’ll test shortly 😃

Can you test this please and let me know? If this works for both cases I would definitely merge a pull request.

On Wed, Apr 15, 2020, 19:21 atymic notifications@github.com wrote:

Ahh, interesting. @codeliner https://github.com/codeliner is this a reasonable solution to accept into the library?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/prooph/pdo-event-store/issues/224#issuecomment-614327647, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADAJPEB5WADMCTRWGTLHQDRMY6RHANCNFSM4MHLNSIQ .

The following index hint solved this problem for us, in https://github.com/prooph/pdo-event-store/blob/master/src/PersistenceStrategy/MySqlSingleStreamStrategy.php#L95

public function indexName(): string
{
    return 'ix_query_aggregate, PRIMARY';
}

It works for both, running projections and loading aggregates. Might be worth trying before adding an additional event store configuration.

@ahmed-alaa I think this is important for your set up as well ☝️