You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by sat <sa...@gmail.com> on 2016/11/11 17:50:47 UTC
ITrigger - Help
Hi,
We are planning to use ITrigger to notify changes, when we execute scripts
or run commands in cqlsh prompt. If the operation is performed through our
application CRUD API, we are planning to handle notification in our CRUD
API itself, however if user performs some operation(like write operation in
cqlsh prompt) we want to handle those changes and update modules that are
listening to those changes.
Could you please let us know whether it is possible to differentiate
updates done through cqlsh prompt and through application.
We also thought about creating multiple users in cassandra and using
different user for cqlsh and for the application. If we go with this
approach, do we get the user who modified the table in ITrigger
implementation (ie., augment method)
Basically we are trying to limit/restrict usage of ITrigger just for cqlsh
prompt as it is little complex and risky (came to know it will impact
cassandra running in that node).
Thanks and Regards
A.SathishKumar
Re: ITrigger - Help
Posted by siddharth verma <si...@gmail.com>.
Hi Sathish,
You could look into, Change Data Capture (CDC) ( https://issues.apache.org/
jira/browse/CASSANDRA-8844 .
It might help you for some of your requirements.
Regards
Siddharth Verma
On Fri, Nov 11, 2016 at 11:34 PM, Jonathan Haddad <jo...@jonhaddad.com> wrote:
> cqlsh uses the Python driver, I don't see how there would be any way to
> differentiate where the request came from unless you stuck an extra field
> in the table that you always write when you're not in cqlsh, or you
> modified cqlsh to include that field whenever it did an insert.
>
> Checking iTrigger source, all you get is a reference to the ColumnFamily
> and some metadata. At a glance of trunk, it doesn't look like you get the
> user that initiated the query.
>
> To be honest, I wouldn't do any of this, it feels like it's going to
> become an error prone mess. Your best bet is to layer something on top of
> the driver yourself. The cleanest way I think think of, long term, is to
> submit a JIRA / patch to enable some class loading & listener hooks in
> cqlsh itself. Without a patch and a really good use case I don't know who
> would want to maintain that though, as it would lock the team into using
> Python for cqlsh.
>
> Jon
>
> On Fri, Nov 11, 2016 at 9:52 AM sat <sa...@gmail.com> wrote:
>
>> Hi,
>>
>> We are planning to use ITrigger to notify changes, when we execute
>> scripts or run commands in cqlsh prompt. If the operation is performed
>> through our application CRUD API, we are planning to handle notification in
>> our CRUD API itself, however if user performs some operation(like write
>> operation in cqlsh prompt) we want to handle those changes and update
>> modules that are listening to those changes.
>>
>> Could you please let us know whether it is possible to differentiate
>> updates done through cqlsh prompt and through application.
>>
>> We also thought about creating multiple users in cassandra and using
>> different user for cqlsh and for the application. If we go with this
>> approach, do we get the user who modified the table in ITrigger
>> implementation (ie., augment method)
>>
>>
>> Basically we are trying to limit/restrict usage of ITrigger just for
>> cqlsh prompt as it is little complex and risky (came to know it will impact
>> cassandra running in that node).
>>
>> Thanks and Regards
>> A.SathishKumar
>>
>>
--
Siddharth Verma
(Visit https://github.com/siddv29/cfs for a high speed cassandra full table
scan)
Re: ITrigger - Help
Posted by siddharth verma <si...@gmail.com>.
I haven't tried CDC either.
Read about it at an abstract level.
Suggested it as an option for exploration.
We too use trigger on production to indicate which primary key has been
acted(update/insert/delete) upon.
Regards
On Sat, Nov 12, 2016 at 12:08 AM, Jonathan Haddad <jo...@jonhaddad.com> wrote:
> Using CDC is going to be... difficult. First off (to my knowledge) all
> you get is a CommitLogReader. If you take a look at the Mutation class
> (everything is serialized and deserialized there), there's no user
> reference. You only get a keyspace, key, and a PartitionUpdate, which
> don't include any user information.
>
> Next, you may need to dedupe your messages, since you will get RF messages
> for every mutation. CDC is per-node, vs triggers which are executed at the
> coordinator level. This may not apply to you as you only want queries that
> came through cqlsh, but I don't see a reasonable way to differentiate all
> the mutations anyway so I think this is a bust.
>
> I haven't spent a lot of time in this code, happy to be corrected if I'm
> wrong.
>
> Jon
>
>
> On Fri, Nov 11, 2016 at 10:14 AM siddharth verma <
> sidd.verma29.list@gmail.com> wrote:
>
>> Hi Sathish,
>> You could look into, Change Data Capture (CDC) (
>> https://issues.apache.org/jira/browse/CASSANDRA-8844 .
>> It might help you for some of your requirements.
>>
>> Regards
>> Siddharth Verma
>>
>> On Fri, Nov 11, 2016 at 11:34 PM, Jonathan Haddad <jo...@jonhaddad.com>
>> wrote:
>>
>> cqlsh uses the Python driver, I don't see how there would be any way to
>> differentiate where the request came from unless you stuck an extra field
>> in the table that you always write when you're not in cqlsh, or you
>> modified cqlsh to include that field whenever it did an insert.
>>
>> Checking iTrigger source, all you get is a reference to the ColumnFamily
>> and some metadata. At a glance of trunk, it doesn't look like you get the
>> user that initiated the query.
>>
>> To be honest, I wouldn't do any of this, it feels like it's going to
>> become an error prone mess. Your best bet is to layer something on top of
>> the driver yourself. The cleanest way I think think of, long term, is to
>> submit a JIRA / patch to enable some class loading & listener hooks in
>> cqlsh itself. Without a patch and a really good use case I don't know who
>> would want to maintain that though, as it would lock the team into using
>> Python for cqlsh.
>>
>> Jon
>>
>> On Fri, Nov 11, 2016 at 9:52 AM sat <sa...@gmail.com> wrote:
>>
>> Hi,
>>
>> We are planning to use ITrigger to notify changes, when we execute
>> scripts or run commands in cqlsh prompt. If the operation is performed
>> through our application CRUD API, we are planning to handle notification in
>> our CRUD API itself, however if user performs some operation(like write
>> operation in cqlsh prompt) we want to handle those changes and update
>> modules that are listening to those changes.
>>
>> Could you please let us know whether it is possible to differentiate
>> updates done through cqlsh prompt and through application.
>>
>> We also thought about creating multiple users in cassandra and using
>> different user for cqlsh and for the application. If we go with this
>> approach, do we get the user who modified the table in ITrigger
>> implementation (ie., augment method)
>>
>>
>> Basically we are trying to limit/restrict usage of ITrigger just for
>> cqlsh prompt as it is little complex and risky (came to know it will impact
>> cassandra running in that node).
>>
>> Thanks and Regards
>> A.SathishKumar
>>
>>
>>
>>
>> --
>> Siddharth Verma
>> (Visit https://github.com/siddv29/cfs for a high speed cassandra full
>> table scan)
>>
>
--
Siddharth Verma
(Visit https://github.com/siddv29/cfs for a high speed cassandra full table
scan)
Re: ITrigger - Help
Posted by Jonathan Haddad <jo...@jonhaddad.com>.
Using CDC is going to be... difficult. First off (to my knowledge) all you
get is a CommitLogReader. If you take a look at the Mutation class
(everything is serialized and deserialized there), there's no user
reference. You only get a keyspace, key, and a PartitionUpdate, which
don't include any user information.
Next, you may need to dedupe your messages, since you will get RF messages
for every mutation. CDC is per-node, vs triggers which are executed at the
coordinator level. This may not apply to you as you only want queries that
came through cqlsh, but I don't see a reasonable way to differentiate all
the mutations anyway so I think this is a bust.
I haven't spent a lot of time in this code, happy to be corrected if I'm
wrong.
Jon
On Fri, Nov 11, 2016 at 10:14 AM siddharth verma <
sidd.verma29.list@gmail.com> wrote:
> Hi Sathish,
> You could look into, Change Data Capture (CDC) (
> https://issues.apache.org/jira/browse/CASSANDRA-8844 .
> It might help you for some of your requirements.
>
> Regards
> Siddharth Verma
>
> On Fri, Nov 11, 2016 at 11:34 PM, Jonathan Haddad <jo...@jonhaddad.com>
> wrote:
>
> cqlsh uses the Python driver, I don't see how there would be any way to
> differentiate where the request came from unless you stuck an extra field
> in the table that you always write when you're not in cqlsh, or you
> modified cqlsh to include that field whenever it did an insert.
>
> Checking iTrigger source, all you get is a reference to the ColumnFamily
> and some metadata. At a glance of trunk, it doesn't look like you get the
> user that initiated the query.
>
> To be honest, I wouldn't do any of this, it feels like it's going to
> become an error prone mess. Your best bet is to layer something on top of
> the driver yourself. The cleanest way I think think of, long term, is to
> submit a JIRA / patch to enable some class loading & listener hooks in
> cqlsh itself. Without a patch and a really good use case I don't know who
> would want to maintain that though, as it would lock the team into using
> Python for cqlsh.
>
> Jon
>
> On Fri, Nov 11, 2016 at 9:52 AM sat <sa...@gmail.com> wrote:
>
> Hi,
>
> We are planning to use ITrigger to notify changes, when we execute scripts
> or run commands in cqlsh prompt. If the operation is performed through our
> application CRUD API, we are planning to handle notification in our CRUD
> API itself, however if user performs some operation(like write operation in
> cqlsh prompt) we want to handle those changes and update modules that are
> listening to those changes.
>
> Could you please let us know whether it is possible to differentiate
> updates done through cqlsh prompt and through application.
>
> We also thought about creating multiple users in cassandra and using
> different user for cqlsh and for the application. If we go with this
> approach, do we get the user who modified the table in ITrigger
> implementation (ie., augment method)
>
>
> Basically we are trying to limit/restrict usage of ITrigger just for cqlsh
> prompt as it is little complex and risky (came to know it will impact
> cassandra running in that node).
>
> Thanks and Regards
> A.SathishKumar
>
>
>
>
> --
> Siddharth Verma
> (Visit https://github.com/siddv29/cfs for a high speed cassandra full
> table scan)
>
Re: ITrigger - Help
Posted by sat <sa...@gmail.com>.
Hi Siddharth Verma,
We explored this option, it seems it outputs the change only to a log file
and we cannot get notified to a listener class. Could you please provide us
what kind of information is pushed in the commit log and when we should
read commitlog. Do we need to instantiate CommitLogReader.java and read it
for every few seconds. Could you please provide us detailed
example/tutorial of how to use this.
Thanks and Regards
A.SathishKumar
On Fri, Nov 11, 2016 at 10:13 AM, siddharth verma <
sidd.verma29.list@gmail.com> wrote:
> Hi Sathish,
> You could look into, Change Data Capture (CDC) (
> https://issues.apache.org/jira/browse/CASSANDRA-8844 .
> It might help you for some of your requirements.
>
> Regards
> Siddharth Verma
>
> On Fri, Nov 11, 2016 at 11:34 PM, Jonathan Haddad <jo...@jonhaddad.com>
> wrote:
>
>> cqlsh uses the Python driver, I don't see how there would be any way to
>> differentiate where the request came from unless you stuck an extra field
>> in the table that you always write when you're not in cqlsh, or you
>> modified cqlsh to include that field whenever it did an insert.
>>
>> Checking iTrigger source, all you get is a reference to the ColumnFamily
>> and some metadata. At a glance of trunk, it doesn't look like you get the
>> user that initiated the query.
>>
>> To be honest, I wouldn't do any of this, it feels like it's going to
>> become an error prone mess. Your best bet is to layer something on top of
>> the driver yourself. The cleanest way I think think of, long term, is to
>> submit a JIRA / patch to enable some class loading & listener hooks in
>> cqlsh itself. Without a patch and a really good use case I don't know who
>> would want to maintain that though, as it would lock the team into using
>> Python for cqlsh.
>>
>> Jon
>>
>> On Fri, Nov 11, 2016 at 9:52 AM sat <sa...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> We are planning to use ITrigger to notify changes, when we execute
>>> scripts or run commands in cqlsh prompt. If the operation is performed
>>> through our application CRUD API, we are planning to handle notification in
>>> our CRUD API itself, however if user performs some operation(like write
>>> operation in cqlsh prompt) we want to handle those changes and update
>>> modules that are listening to those changes.
>>>
>>> Could you please let us know whether it is possible to differentiate
>>> updates done through cqlsh prompt and through application.
>>>
>>> We also thought about creating multiple users in cassandra and using
>>> different user for cqlsh and for the application. If we go with this
>>> approach, do we get the user who modified the table in ITrigger
>>> implementation (ie., augment method)
>>>
>>>
>>> Basically we are trying to limit/restrict usage of ITrigger just for
>>> cqlsh prompt as it is little complex and risky (came to know it will impact
>>> cassandra running in that node).
>>>
>>> Thanks and Regards
>>> A.SathishKumar
>>>
>>>
>
>
> --
> Siddharth Verma
> (Visit https://github.com/siddv29/cfs for a high speed cassandra full
> table scan)
>
--
A.SathishKumar
044-24735023
Re: ITrigger - Help
Posted by siddharth verma <si...@gmail.com>.
Hi Sathish,
You could look into, Change Data Capture (CDC) (
https://issues.apache.org/jira/browse/CASSANDRA-8844 .
It might help you for some of your requirements.
Regards
Siddharth Verma
On Fri, Nov 11, 2016 at 11:34 PM, Jonathan Haddad <jo...@jonhaddad.com> wrote:
> cqlsh uses the Python driver, I don't see how there would be any way to
> differentiate where the request came from unless you stuck an extra field
> in the table that you always write when you're not in cqlsh, or you
> modified cqlsh to include that field whenever it did an insert.
>
> Checking iTrigger source, all you get is a reference to the ColumnFamily
> and some metadata. At a glance of trunk, it doesn't look like you get the
> user that initiated the query.
>
> To be honest, I wouldn't do any of this, it feels like it's going to
> become an error prone mess. Your best bet is to layer something on top of
> the driver yourself. The cleanest way I think think of, long term, is to
> submit a JIRA / patch to enable some class loading & listener hooks in
> cqlsh itself. Without a patch and a really good use case I don't know who
> would want to maintain that though, as it would lock the team into using
> Python for cqlsh.
>
> Jon
>
> On Fri, Nov 11, 2016 at 9:52 AM sat <sa...@gmail.com> wrote:
>
>> Hi,
>>
>> We are planning to use ITrigger to notify changes, when we execute
>> scripts or run commands in cqlsh prompt. If the operation is performed
>> through our application CRUD API, we are planning to handle notification in
>> our CRUD API itself, however if user performs some operation(like write
>> operation in cqlsh prompt) we want to handle those changes and update
>> modules that are listening to those changes.
>>
>> Could you please let us know whether it is possible to differentiate
>> updates done through cqlsh prompt and through application.
>>
>> We also thought about creating multiple users in cassandra and using
>> different user for cqlsh and for the application. If we go with this
>> approach, do we get the user who modified the table in ITrigger
>> implementation (ie., augment method)
>>
>>
>> Basically we are trying to limit/restrict usage of ITrigger just for
>> cqlsh prompt as it is little complex and risky (came to know it will impact
>> cassandra running in that node).
>>
>> Thanks and Regards
>> A.SathishKumar
>>
>>
--
Siddharth Verma
(Visit https://github.com/siddv29/cfs for a high speed cassandra full table
scan)
Re: ITrigger - Help
Posted by sat <sa...@gmail.com>.
Hi Jon,
Thanks for your prompt answer.
Thanks
A.SathishKumar
On Fri, Nov 11, 2016 at 10:04 AM, Jonathan Haddad <jo...@jonhaddad.com> wrote:
> cqlsh uses the Python driver, I don't see how there would be any way to
> differentiate where the request came from unless you stuck an extra field
> in the table that you always write when you're not in cqlsh, or you
> modified cqlsh to include that field whenever it did an insert.
>
> Checking iTrigger source, all you get is a reference to the ColumnFamily
> and some metadata. At a glance of trunk, it doesn't look like you get the
> user that initiated the query.
>
> To be honest, I wouldn't do any of this, it feels like it's going to
> become an error prone mess. Your best bet is to layer something on top of
> the driver yourself. The cleanest way I think think of, long term, is to
> submit a JIRA / patch to enable some class loading & listener hooks in
> cqlsh itself. Without a patch and a really good use case I don't know who
> would want to maintain that though, as it would lock the team into using
> Python for cqlsh.
>
> Jon
>
> On Fri, Nov 11, 2016 at 9:52 AM sat <sa...@gmail.com> wrote:
>
>> Hi,
>>
>> We are planning to use ITrigger to notify changes, when we execute
>> scripts or run commands in cqlsh prompt. If the operation is performed
>> through our application CRUD API, we are planning to handle notification in
>> our CRUD API itself, however if user performs some operation(like write
>> operation in cqlsh prompt) we want to handle those changes and update
>> modules that are listening to those changes.
>>
>> Could you please let us know whether it is possible to differentiate
>> updates done through cqlsh prompt and through application.
>>
>> We also thought about creating multiple users in cassandra and using
>> different user for cqlsh and for the application. If we go with this
>> approach, do we get the user who modified the table in ITrigger
>> implementation (ie., augment method)
>>
>>
>> Basically we are trying to limit/restrict usage of ITrigger just for
>> cqlsh prompt as it is little complex and risky (came to know it will impact
>> cassandra running in that node).
>>
>> Thanks and Regards
>> A.SathishKumar
>>
>>
--
A.SathishKumar
044-24735023
Re: ITrigger - Help
Posted by siddharth verma <si...@gmail.com>.
Hi Sathish,
You could look into, Change Data Capture (CDC) (
https://issues.apache.org/jira/browse/CASSANDRA-8844 .
It might help you for some of your requirements.
Regards
Siddharth Verma
On Fri, Nov 11, 2016 at 11:34 PM, Jonathan Haddad <jo...@jonhaddad.com> wrote:
> cqlsh uses the Python driver, I don't see how there would be any way to
> differentiate where the request came from unless you stuck an extra field
> in the table that you always write when you're not in cqlsh, or you
> modified cqlsh to include that field whenever it did an insert.
>
> Checking iTrigger source, all you get is a reference to the ColumnFamily
> and some metadata. At a glance of trunk, it doesn't look like you get the
> user that initiated the query.
>
> To be honest, I wouldn't do any of this, it feels like it's going to
> become an error prone mess. Your best bet is to layer something on top of
> the driver yourself. The cleanest way I think think of, long term, is to
> submit a JIRA / patch to enable some class loading & listener hooks in
> cqlsh itself. Without a patch and a really good use case I don't know who
> would want to maintain that though, as it would lock the team into using
> Python for cqlsh.
>
> Jon
>
> On Fri, Nov 11, 2016 at 9:52 AM sat <sa...@gmail.com> wrote:
>
>> Hi,
>>
>> We are planning to use ITrigger to notify changes, when we execute
>> scripts or run commands in cqlsh prompt. If the operation is performed
>> through our application CRUD API, we are planning to handle notification in
>> our CRUD API itself, however if user performs some operation(like write
>> operation in cqlsh prompt) we want to handle those changes and update
>> modules that are listening to those changes.
>>
>> Could you please let us know whether it is possible to differentiate
>> updates done through cqlsh prompt and through application.
>>
>> We also thought about creating multiple users in cassandra and using
>> different user for cqlsh and for the application. If we go with this
>> approach, do we get the user who modified the table in ITrigger
>> implementation (ie., augment method)
>>
>>
>> Basically we are trying to limit/restrict usage of ITrigger just for
>> cqlsh prompt as it is little complex and risky (came to know it will impact
>> cassandra running in that node).
>>
>> Thanks and Regards
>> A.SathishKumar
>>
>>
--
Siddharth Verma
(Visit https://github.com/siddv29/cfs for a high speed cassandra full table
scan)
Re: ITrigger - Help
Posted by Jonathan Haddad <jo...@jonhaddad.com>.
cqlsh uses the Python driver, I don't see how there would be any way to
differentiate where the request came from unless you stuck an extra field
in the table that you always write when you're not in cqlsh, or you
modified cqlsh to include that field whenever it did an insert.
Checking iTrigger source, all you get is a reference to the ColumnFamily
and some metadata. At a glance of trunk, it doesn't look like you get the
user that initiated the query.
To be honest, I wouldn't do any of this, it feels like it's going to become
an error prone mess. Your best bet is to layer something on top of the
driver yourself. The cleanest way I think think of, long term, is to
submit a JIRA / patch to enable some class loading & listener hooks in
cqlsh itself. Without a patch and a really good use case I don't know who
would want to maintain that though, as it would lock the team into using
Python for cqlsh.
Jon
On Fri, Nov 11, 2016 at 9:52 AM sat <sa...@gmail.com> wrote:
> Hi,
>
> We are planning to use ITrigger to notify changes, when we execute scripts
> or run commands in cqlsh prompt. If the operation is performed through our
> application CRUD API, we are planning to handle notification in our CRUD
> API itself, however if user performs some operation(like write operation in
> cqlsh prompt) we want to handle those changes and update modules that are
> listening to those changes.
>
> Could you please let us know whether it is possible to differentiate
> updates done through cqlsh prompt and through application.
>
> We also thought about creating multiple users in cassandra and using
> different user for cqlsh and for the application. If we go with this
> approach, do we get the user who modified the table in ITrigger
> implementation (ie., augment method)
>
>
> Basically we are trying to limit/restrict usage of ITrigger just for cqlsh
> prompt as it is little complex and risky (came to know it will impact
> cassandra running in that node).
>
> Thanks and Regards
> A.SathishKumar
>
>