You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hudi.apache.org by Marina <pp...@yahoo.com.INVALID> on 2019/11/24 16:58:07 UTC

Hudi vs AresDB?

Hi, I was reading Uber engineering blogs about your Big Data backend architecture - very interesting problems you guys are solving and very interesting architecture approaches.I noticed that there are two open-source systems created/maintained by Uber - Hudi and AresDB (Introducing AresDB: Uber's GPU-Powered Open Source, Real-time Analytics Engine

| 
| 
| 
| 
 | 
 |

 |

 |
| 
| 
 | 
Introducing AresDB: Uber's GPU-Powered Open Source, Real-time Analytics ...

AresDB, Uber's open source real-time analytics engine, leverages GPUs to enable real-time computation and data p...
 |

 |

 |



).I'm a bit confused as to what are the differences/goals/needs for both systems... 
Both seem to be addressing the issue of ingesting/storing large quantities of data and providing real-time (and other types) access to query the data....
Thank you!Marina

Re: Hudi vs AresDB?

Posted by Marina <pp...@yahoo.com.INVALID>.

 Thanks, Vinoth, makes sense,Marina

    On Monday, November 25, 2019, 2:59:14 PM EST, Vinoth Chandar <vi...@apache.org> wrote:  
 
 Hi Marina,

Thanks for reaching out. Hudi is now part of the Apache Software Foundation
actually. :)

Cutting to the chase, although you can build real-time dashboards using
both, both systems are pretty different and provide different tradeoffs.

For e.g:

- AresDB primarily keeps data in memory (so queries could be faster, but
data size could be limited)
- While Hudi works with a shared storage model writing data out
persistentlyto S3/HDFS etc, so you can scale to very large datasets
- Query performance on Hudi is left upto to the engine you choose. (you
could cache data infront of S3/HDFS, but again could still be slower than
in-memory)

Hope that helps

Thanks,
Vinoth

On Sun, Nov 24, 2019 at 8:58 AM Marina <pp...@yahoo.com.invalid> wrote:

> Hi, I was reading Uber engineering blogs about your Big Data backend
> architecture - very interesting problems you guys are solving and very
> interesting architecture approaches.I noticed that there are two
> open-source systems created/maintained by Uber - Hudi and AresDB
> (Introducing AresDB: Uber's GPU-Powered Open Source, Real-time Analytics
> Engine
>
> |
> |
> |
> |
>  |
>  |
>
>  |
>
>  |
> |
> |
>  |
> Introducing AresDB: Uber's GPU-Powered Open Source, Real-time Analytics ...
>
> AresDB, Uber's open source real-time analytics engine, leverages GPUs to
> enable real-time computation and data p...
>  |
>
>  |
>
>  |
>
>
>
> ).I'm a bit confused as to what are the differences/goals/needs for both
> systems...
> Both seem to be addressing the issue of ingesting/storing large quantities
> of data and providing real-time (and other types) access to query the
> data....
> Thank you!Marina
>
>

Re: Hudi vs AresDB?

Posted by Vinoth Chandar <vi...@apache.org>.

Hi Marina,

Thanks for reaching out. Hudi is now part of the Apache Software Foundation
actually. :)

Cutting to the chase, although you can build real-time dashboards using
both, both systems are pretty different and provide different tradeoffs.

For e.g:

- AresDB primarily keeps data in memory (so queries could be faster, but
data size could be limited)
- While Hudi works with a shared storage model writing data out
persistentlyto S3/HDFS etc, so you can scale to very large datasets
- Query performance on Hudi is left upto to the engine you choose. (you
could cache data infront of S3/HDFS, but again could still be slower than
in-memory)

Hope that helps

Thanks,
Vinoth

On Sun, Nov 24, 2019 at 8:58 AM Marina <pp...@yahoo.com.invalid> wrote:

> Hi, I was reading Uber engineering blogs about your Big Data backend
> architecture - very interesting problems you guys are solving and very
> interesting architecture approaches.I noticed that there are two
> open-source systems created/maintained by Uber - Hudi and AresDB
> (Introducing AresDB: Uber's GPU-Powered Open Source, Real-time Analytics
> Engine
>
> |
> |
> |
> |
>  |
>  |
>
>  |
>
>  |
> |
> |
>  |
> Introducing AresDB: Uber's GPU-Powered Open Source, Real-time Analytics ...
>
> AresDB, Uber's open source real-time analytics engine, leverages GPUs to
> enable real-time computation and data p...
>  |
>
>  |
>
>  |
>
>
>
> ).I'm a bit confused as to what are the differences/goals/needs for both
> systems...
> Both seem to be addressing the issue of ingesting/storing large quantities
> of data and providing real-time (and other types) access to query the
> data....
> Thank you!Marina
>
>