You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by syepes <sy...@gmail.com> on 2015/12/22 14:36:25 UTC

Sink - Cassandra

Hello,

I have just started testing out Flink and would like to migrate one of my
Spark Streaming jobs (Kafka->Spark->C*) to Flink.

- Is there anyone using with Flink with Cassandra?
- Does there exist a working Cassandra Sink, the only thing I have found is:
https://github.com/rzvoncek/flink/tree/2b281120f4206c4fd66bec22090e0b6d62ebb8ad/flink-staging/flink-cassandra

Regards,
Sebastian




--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Sink-Cassandra-tp4107.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Sink - Cassandra

Posted by nragon <nu...@wedotechnologies.com>.

Nick,

Can you send some of your examples using phoenix?

Thanks



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Sink-Cassandra-tp4107p15197.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Sink - Cassandra

Posted by Nick Dimiduk <nd...@gmail.com>.

Yes the approach works fine for my use-case; has been in "production" for
quite some time. My implementation has some untested scenarios around job
restarting and failures, so of course your mileage may vary.

On Mon, May 15, 2017 at 5:58 AM, nragon <nuno.goncalves@wedotechnologies.com
> wrote:

> Hi Nick,
>
> I'm trying to integrate Hbase with streaming. Did you accomplish good
> results(writes/s) with phoenix?
> Can you share your code.
>
> Thanks,
> Nuno
>
>
>
> --
> View this message in context: http://apache-flink-user-
> mailing-list-archive.2336050.n4.nabble.com/Sink-Cassandra-
> tp4107p13138.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>

Re: Sink - Cassandra

Posted by nragon <nu...@wedotechnologies.com>.

Hi Nick,

I'm trying to integrate Hbase with streaming. Did you accomplish good
results(writes/s) with phoenix?
Can you share your code.

Thanks,
Nuno



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Sink-Cassandra-tp4107p13138.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Sink - Cassandra

Posted by Nick Dimiduk <nd...@apache.org>.

Hi Sebastian,

I've had preliminary success with a steaming job that is Kafka -> Flink ->
HBase (actually, Phoenix) using the Hadoop OutputFormat adapter. A little
glue was required but it seems to work okay. My guess it it would be the
same for Cassandra. Maybe that can get you started?

Good luck,
-Nick

On Tuesday, January 5, 2016, Aljoscha Krettek <al...@apache.org> wrote:

> Hi Sebastian,
> I’m afraid the people working on Flink don’t have much experience with
> Cassandra. Maybe you could look into the Elasticsearch sink and adapt it to
> write to Cassandra instead. That could be a valuable addition to Flink.
>
> Cheers,
> Aljoscha
> > On 22 Dec 2015, at 14:36, syepes <syepes@gmail.com <javascript:;>>
> wrote:
> >
> > Hello,
> >
> > I have just started testing out Flink and would like to migrate one of my
> > Spark Streaming jobs (Kafka->Spark->C*) to Flink.
> >
> > - Is there anyone using with Flink with Cassandra?
> > - Does there exist a working Cassandra Sink, the only thing I have found
> is:
> >
> https://github.com/rzvoncek/flink/tree/2b281120f4206c4fd66bec22090e0b6d62ebb8ad/flink-staging/flink-cassandra
> >
> > Regards,
> > Sebastian
> >
> >
> >
> >
> > --
> > View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Sink-Cassandra-tp4107.html
> > Sent from the Apache Flink User Mailing List archive. mailing list
> archive at Nabble.com.
>
>

Re: Sink - Cassandra

Posted by Aljoscha Krettek <al...@apache.org>.

Hi Sebastian,
I’m afraid the people working on Flink don’t have much experience with Cassandra. Maybe you could look into the Elasticsearch sink and adapt it to write to Cassandra instead. That could be a valuable addition to Flink.

Cheers,
Aljoscha
> On 22 Dec 2015, at 14:36, syepes <sy...@gmail.com> wrote:
> 
> Hello,
> 
> I have just started testing out Flink and would like to migrate one of my
> Spark Streaming jobs (Kafka->Spark->C*) to Flink.
> 
> - Is there anyone using with Flink with Cassandra?
> - Does there exist a working Cassandra Sink, the only thing I have found is:
> https://github.com/rzvoncek/flink/tree/2b281120f4206c4fd66bec22090e0b6d62ebb8ad/flink-staging/flink-cassandra
> 
> Regards,
> Sebastian
> 
> 
> 
> 
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Sink-Cassandra-tp4107.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.