[D] How practical is to use this for a OLTP DB engine? [datafusion]

["mamcx (via GitHub)" <gi...@apache.org>]

GitHub user mamcx created a discussion: How practical is to use this for a OLTP DB engine?

The idea of reusing building blocks is appealing, unfortunately most I see is only from OLAP-like workloads.

I have in mind an engine that is a mix of both, but biased to OLTP for use in normal erp-like apps.

Is there any experiment related to this? how much of a penalty is there?

P.D: I well aware of the distinctions in workloads and have worked as part of a commercial DB engine, so what I asking is not about the basics, instead, is (assuming all other things are made right), how much of a penalty is there.

If for example the answer is like "median 4x" but the speed could in theory make possible go faster than PG, then it could be a good idea.

GitHub link: https://github.com/apache/datafusion/discussions/24041

----
This is an automatically sent email for github@datafusion.apache.org.
To unsubscribe, please send an email to: github-unsubscribe@datafusion.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [D] How practical is to use this for a OLTP DB engine? [datafusion]

["alamb (via GitHub)" <gi...@apache.org>]

GitHub user alamb added a comment to the discussion: How practical is to use this for a OLTP DB engine?

I don't have any good measurements, but many people have made some sort of hybrid data processing engine using DataFusion

DataFusion does have the classic OLAP architecture (vectorized columnar execution) so for single row queries (that process a single row, or 10s of rows) there is more overhead than a row at a time model

I don't know of any measurements however -- it would be interesting to make some and post your findings

GitHub link: https://github.com/apache/datafusion/discussions/24041#discussioncomment-17931955

----
This is an automatically sent email for github@datafusion.apache.org.
To unsubscribe, please send an email to: github-unsubscribe@datafusion.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org