ACID transaction over ORC

[Nirav Patel <ni...@gmail.com>]

Hi,

Are there any companies using hive ACID capabilities ? I am trying to
understand how stable/reliable is ACID transaction on ORC in terms of
ingesting real time data over 10 mins window and writing those data
potentially on same ORC partition (merge into).

Nirav

Re: ACID transaction over ORC

[Nirav Patel <ni...@gmail.com>]

Thats really helpful. Yes I prefer open table format like iceberg but we
are unfortunately locked into google and dataflow/beam having people
transitioning over to something like iceberg and spark instead of beam
seems like an overhaul at the moment.

On Thu, Mar 23, 2023 at 2:22 AM Attila Turoczy <at...@cloudera.com>
wrote:

> Hi Nirav,
>
> Yes, there are several big fortune 500 companies that are using Hive Acid
> without any major issue. The latest version is pretty stable. Depending on
> your use case you can choose Hive ACID or Hive + Iceberg (4.0). if your use
> case is more on frequent delete / update the hive acid + orc is a better
> choice. If you just occasionally delete / update the rows, Iceberg could be
> a better solution. (I personally would prefer Iceberg) but it depends on
> your use case. Anyhow both will be an excellent choice.
>
> -Attila
>
> On Thu, Mar 23, 2023 at 1:31 AM Nirav Patel <ni...@gmail.com> wrote:
>
>> Hi,
>>
>> Are there any companies using hive ACID capabilities ? I am trying to
>> understand how stable/reliable is ACID transaction on ORC in terms of
>> ingesting real time data over 10 mins window and writing those data
>> potentially on same ORC partition (merge into).
>>
>> Nirav
>>
>

Re: ACID transaction over ORC

[Attila Turoczy <at...@cloudera.com>]

Hi Nirav,

Yes, there are several big fortune 500 companies that are using Hive Acid
without any major issue. The latest version is pretty stable. Depending on
your use case you can choose Hive ACID or Hive + Iceberg (4.0). if your use
case is more on frequent delete / update the hive acid + orc is a better
choice. If you just occasionally delete / update the rows, Iceberg could be
a better solution. (I personally would prefer Iceberg) but it depends on
your use case. Anyhow both will be an excellent choice.

-Attila

On Thu, Mar 23, 2023 at 1:31 AM Nirav Patel <ni...@gmail.com> wrote:

> Hi,
>
> Are there any companies using hive ACID capabilities ? I am trying to
> understand how stable/reliable is ACID transaction on ORC in terms of
> ingesting real time data over 10 mins window and writing those data
> potentially on same ORC partition (merge into).
>
> Nirav
>