You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@pekko.apache.org by "wjohnsonbynder (via GitHub)" <gi...@apache.org> on 2024/02/08 16:48:20 UTC
[PR] Rework filter clause before join on eventsByTag [incubator-pekko-persistence-jdbc]
wjohnsonbynder opened a new pull request, #106:
URL: https://github.com/apache/incubator-pekko-persistence-jdbc/pull/106
Original issue
https://github.com/apache/incubator-pekko-persistence-jdbc/issues/105
The event-journal join to event-tag is happening before the offset/ordering where clause.
The subquery is therefore returning the entire event-journal when getting eventsByTag.
The filter for ordering only pertains to the event-journal. By moving that into the subquery where clause it will not return the entire event-journal each time eventsByTag is called.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] Rework filter clause before join on eventsByTag [incubator-pekko-persistence-jdbc]
Posted by "wjohnsonbynder (via GitHub)" <gi...@apache.org>.
wjohnsonbynder commented on PR #106:
URL: https://github.com/apache/incubator-pekko-persistence-jdbc/pull/106#issuecomment-1934908566
I think some dbs handle Orderby in a subquery by optimizing them out. Moved the orderby to the main query.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] Rework filter clause before join on eventsByTag [incubator-pekko-persistence-jdbc]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on PR #106:
URL: https://github.com/apache/incubator-pekko-persistence-jdbc/pull/106#issuecomment-1936029324
@wjohnsonbynder I had a look at the postgres integration times from the CI build for your PR and the time taken by the "show the configured performance characteristics" test takes a similar amount of time as for runs without your change.
Would you be able to have a look at the "show the configured performance characteristics" test to see if it is a useful test for your change? If it is, we could increase the iterations to see if makes the perf changes more obvious.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] Rework filter clause before join on eventsByTag [incubator-pekko-persistence-jdbc]
Posted by "wjohnsonbynder (via GitHub)" <gi...@apache.org>.
wjohnsonbynder commented on PR #106:
URL: https://github.com/apache/incubator-pekko-persistence-jdbc/pull/106#issuecomment-1938712794
Looks I wasted a your time, sorry for that. I did some digging. Postgres is great DB and the query plan for either optimize to the best version. I think my original timing ran with the order by in the subquery. Orderby on a subquery is removed so the "increased performance" was the query not running the order by at all.
We have a large event journal and the order by on this query isn't fast. I need to dig into the query some more and see what the parameters are and see if we can optimize the size of the result set or how often our projections make the query.
Closing this out and the issue.
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] Rework filter clause before join on eventsByTag [incubator-pekko-persistence-jdbc]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on PR #106:
URL: https://github.com/apache/incubator-pekko-persistence-jdbc/pull/106#issuecomment-1935969690
@wjohnsonbynder would you be able to try to get some performance numbers?
https://github.com/apache/incubator-pekko-persistence-jdbc/blob/main/core/src/test/scala/org/apache/pekko/persistence/jdbc/query/EventsByTagTest.scala#L396
"show the configured performance characteristics" looks like a test could demo a perf improvement.
We have integration tests and this CI script gives you an idea of how to run them with postgres but there are other files for other databases if you prefer another database.
https://github.com/apache/incubator-pekko-persistence-jdbc/blob/main/.github/workflows/postgres-tests.yml
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] Rework filter clause before join on eventsByTag [incubator-pekko-persistence-jdbc]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
mdedetrich commented on PR #106:
URL: https://github.com/apache/incubator-pekko-persistence-jdbc/pull/106#issuecomment-1934977314
@pjfanning I would set this for 1.1.x since its a behaviour change, wdyt?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] Rework filter clause before join on eventsByTag [incubator-pekko-persistence-jdbc]
Posted by "wjohnsonbynder (via GitHub)" <gi...@apache.org>.
wjohnsonbynder closed pull request #106: Rework filter clause before join on eventsByTag
URL: https://github.com/apache/incubator-pekko-persistence-jdbc/pull/106
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org