Publishing from cassandra

[Alexander Orr <me...@gmail.com>]

Hi,

I'm wondering if someone could help me, I'd like to use cassandra to store
data and publish this on dowstream to another database (kdb if anyone is
interested). Essentially I'd like to be able to run a function or operation
on cassandra from an upstream process that would insert to table and
publish the data on downstream.

I can't see anything in the docs, but I'm relatively new to cassandra.
Assuming there's not something simple already in place what would be the
best way to impliment this kind of mechanism? I have some java that will
allow me to talk to the db I want to, but I'm not sure of the  best way to
integrate this with cassandra.

UDFs seem to have ponential, but I don't think it's possible to use
external libraries/classes within UDFs. All I can think of at the minute is
either having a process which controls cassandra, publishes to it and also
the downstream system directly or cloning the git repo and seeing if I can
hack in some extra functionality.

Any suggestions welcome.

Thanks

Alex

Re: Publishing from cassandra

[Clint Martin <cl...@coolfiretechnologies.com>]

I would probably use a messaging layer to perform this operation. Kafka
works very well, but depending on your throughput requirements almost
anything should work.

The idea is to publish your mutation requests to the messaging layer and
allow multiple consumers to process those mutation requests into each of
your db layers (Cassandra and kdb)

I'm certain you can get something to work in the way you originally
intended, possibly using Cassandra triggers.. Or maybe using batch
processing through spark. However just about anything you implement will
incur performance overhead for Cassandra and may impede your ability to
scale.

I tend to believe in using the right tool for the job, and I don't believe
that replication from Cassandra is the right way to go here.

Separating your replication mechanism from your persistence mechanism
allows you to decouple your system and manage or replace reach part of it
independently.

Clint

On Apr 24, 2016 10:51 AM, "Alexander Orr" <me...@gmail.com> wrote:

> Hi,
>
> I'm wondering if someone could help me, I'd like to use cassandra to store
> data and publish this on dowstream to another database (kdb if anyone is
> interested). Essentially I'd like to be able to run a function or operation
> on cassandra from an upstream process that would insert to table and
> publish the data on downstream.
>
> I can't see anything in the docs, but I'm relatively new to cassandra.
> Assuming there's not something simple already in place what would be the
> best way to impliment this kind of mechanism? I have some java that will
> allow me to talk to the db I want to, but I'm not sure of the  best way to
> integrate this with cassandra.
>
> UDFs seem to have ponential, but I don't think it's possible to use
> external libraries/classes within UDFs. All I can think of at the minute is
> either having a process which controls cassandra, publishes to it and also
> the downstream system directly or cloning the git repo and seeing if I can
> hack in some extra functionality.
>
> Any suggestions welcome.
>
> Thanks
>
> Alex
>

RE: Publishing from cassandra

[Ashic Mahtab <as...@live.com>]

Hi Alex,Cassandra Triggers may help. They come with their drawbacks, but they can be an option for something like this:
https://docs.datastax.com/en/cql/3.1/cql/cql_reference/trigger_r.html
-Ashic.

Date: Sun, 24 Apr 2016 12:05:31 -0400
Subject: Re: Publishing from cassandra
From: michael.laing@nytimes.com
To: user@cassandra.apache.org

You could take a look at, or follow: https://issues.apache.org/jira/browse/CASSANDRA-8844
On Sun, Apr 24, 2016 at 10:51 AM, Alexander Orr <me...@gmail.com> wrote:
Hi,
I'm wondering if someone could help me, I'd like to use cassandra to store data and publish this on dowstream to another database (kdb if anyone is interested). Essentially I'd like to be able to run a function or operation on cassandra from an upstream process that would insert to table and publish the data on downstream.
I can't see anything in the docs, but I'm relatively new to cassandra. Assuming there's not something simple already in place what would be the best way to impliment this kind of mechanism? I have some java that will allow me to talk to the db I want to, but I'm not sure of the  best way to integrate this with cassandra. 
UDFs seem to have ponential, but I don't think it's possible to use external libraries/classes within UDFs. All I can think of at the minute is either having a process which controls cassandra, publishes to it and also the downstream system directly or cloning the git repo and seeing if I can hack in some extra functionality.
Any suggestions welcome.
Thanks 
Alex

 		 	   		  

Re: Publishing from cassandra

["Laing, Michael" <mi...@nytimes.com>]

You could take a look at, or follow:
https://issues.apache.org/jira/browse/CASSANDRA-8844

On Sun, Apr 24, 2016 at 10:51 AM, Alexander Orr <me...@gmail.com> wrote:

> Hi,
>
> I'm wondering if someone could help me, I'd like to use cassandra to store
> data and publish this on dowstream to another database (kdb if anyone is
> interested). Essentially I'd like to be able to run a function or operation
> on cassandra from an upstream process that would insert to table and
> publish the data on downstream.
>
> I can't see anything in the docs, but I'm relatively new to cassandra.
> Assuming there's not something simple already in place what would be the
> best way to impliment this kind of mechanism? I have some java that will
> allow me to talk to the db I want to, but I'm not sure of the  best way to
> integrate this with cassandra.
>
> UDFs seem to have ponential, but I don't think it's possible to use
> external libraries/classes within UDFs. All I can think of at the minute is
> either having a process which controls cassandra, publishes to it and also
> the downstream system directly or cloning the git repo and seeing if I can
> hack in some extra functionality.
>
> Any suggestions welcome.
>
> Thanks
>
> Alex
>