Streaming on different store types

[Flavio Pompermaier <po...@okkam.it>]

Hi everybody,

I have a scenario where I would like to stream data to different
persistency types (i.e. sql db, graphdb ,hdfs, etc) and perform some
filtering and trasformation as the the data comes in.
The problem is to maintain consistency between all datastores (maybe some
operation could fail) and managing the U and D operations of CRUD...is
there any suggested solution for that?

Best,
FP

Re: Streaming on different store types

[Jörn Franke <jo...@gmail.com>]

Hallo,

I fear you have to write your own transaction logic for it (coordination,e.
.g. via Zookeeper, transaction log, depending on your requirements raft
/paxos etc.).

However, before you embark on this journey question yourself if your
application really needs it and what data load you expect.

Best regards
Le 30 juil. 2014 16:46, "Flavio Pompermaier" <po...@okkam.it> a écrit
:

> Hi everybody,
>
> I have a scenario where I would like to stream data to different
> persistency types (i.e. sql db, graphdb ,hdfs, etc) and perform some
> filtering and trasformation as the the data comes in.
> The problem is to maintain consistency between all datastores (maybe some
> operation could fail) and managing the U and D operations of CRUD...is
> there any suggested solution for that?
>
> Best,
> FP
>
>
>