[DataFusion] MVCC

[Olo Sawyerr <os...@outlook.com>]

Hi there.

Does DataFusion have any MVCC capabilities?

Regards,
Olo

Sent from Outlook for iOS<https://aka.ms/o0ukef>

Re: [DataFusion] MVCC

[Andy Grove <an...@gmail.com>]

There is an integration between DataFusion and Delta Lake that provides
some level of transaction support. I have no idea if it qualifies as MVCC
or not (I have not spent time looking at this project in detail yet).

https://github.com/delta-io/delta-rs#features

It would be good to know more about your requirements though. Are you
looking to build an OLTP database on top of DataFusion for example?

Hope that is helpful.

Andy.


On Wed, Jan 11, 2023 at 1:10 AM Andrew Lamb <al...@influxdata.com> wrote:

> Hi Olo,
>
> DataFusion is based on the Arrow model and thus all its data is read only;
> In order to update data a new copy must be made.
>
> I guess you could call this a version of MVCC.
>
> People often use DataFusion to operate on data in files stored in formats
> such as Parquet which are similarly read optimize, or their own data
> sources  which provides data in record batches
>
> Hope that helps,
> Andrew
>
> On Tue, Jan 10, 2023 at 11:08 PM Olo Sawyerr <os...@outlook.com> wrote:
>
>> Hi there.
>>
>> Does DataFusion have any MVCC capabilities?
>>
>> Regards,
>> Olo
>>
>> Sent from Outlook for iOS <https://aka.ms/o0ukef>
>>
>

Re: [DataFusion] MVCC

[Andrew Lamb <al...@influxdata.com>]

Hi Olo,

DataFusion is based on the Arrow model and thus all its data is read only;
In order to update data a new copy must be made.

I guess you could call this a version of MVCC.

People often use DataFusion to operate on data in files stored in formats
such as Parquet which are similarly read optimize, or their own data
sources  which provides data in record batches

Hope that helps,
Andrew

On Tue, Jan 10, 2023 at 11:08 PM Olo Sawyerr <os...@outlook.com> wrote:

> Hi there.
>
> Does DataFusion have any MVCC capabilities?
>
> Regards,
> Olo
>
> Sent from Outlook for iOS <https://aka.ms/o0ukef>
>