You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Stefan Miklosovic <st...@instaclustr.com> on 2021/07/19 13:32:05 UTC
[DISCUSS] Diagnostic events in virtual tables
Hi,
I wonder if people would be interested in having diagnostic events in
virtual tables?
I put something together here (heavy wip) (1) but that is the idea I have.
I know that Stefan Podkowinski did a spectacular job under
CASSANDRA-12944 and the possibility to persist locally was elaborated
on in (2) where the conclusion was made that maybe it is more suitable
to put it into chronicle queues via BinLog path and so on.
The benefits of bin log is that we have events persisted and they can
be inspected further when node is offline.
While data in virtual tables cease to exist after nodes are down, one
nice benefit of having it in virtual tables is that we can query it
comfortably via CQL and I think that this solution is more suitable to
have on an every day basis from operator's point of view. There is
still a way to dump it somewhere else anyway if one is really
interested in doing so.
Do you think that the bin log solution is overall superior to the
virtual tables approach and we should just forget about having it in
virtual tables?
If that is the case, what I would like to see is to have some
pluggability here so I might implement this on my own and configure a
node to put it to virtual tables for me, it does not necessarily have
to be the part of Cassandra code base if we are strongly against that.
(1) https://github.com/instaclustr/cassandra/commit/0dd60dc0a847619fdb704b700154e624b21a0c35
(2) https://issues.apache.org/jira/browse/CASSANDRA-13460
Regards
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Diagnostic events in virtual tables
Posted by Mick Semb Wever <mc...@apache.org>.
>
> If one subscribes via JMX, this subscription is not propagated to the
>
underlying CQL table. So she might subscribe to 5 events but there
> would be none in vtable.
There isn't any need for any underlying CQL table here.
The enabled state, and subscriptions, is in DatabaseDescriptor
and DiagnosticEventService. Both interfaces: JMX and vtable; would update
these values. And it is intentional that any changes made at runtime are
reset when a node restarts. If the events are implemented to be persisted
in a chronicle queue, then they will be available after a restart (readable
via either jmx or vtable).
Let's continue this discussion on CASSANDRA-13460.
Re: [DISCUSS] Diagnostic events in virtual tables
Posted by Chris Lohfink <cl...@gmail.com>.
Perhaps re-add the settings virtual table mutability. That way the same
place can be used to update settings at runtime for multiple things instead
of creating a new virtual table per service we want to make hot props for.
Might be kinda nice to allow REGISTER and EVENT CQL events to be created
with virtual tables as well for some extended functionality.
Chris
On Fri, Sep 10, 2021 at 2:36 AM Stefan Miklosovic <
stefan.miklosovic@instaclustr.com> wrote:
> Hi Mick,
>
> I returned to this after some time and here are my questions about this.
>
> I am waiting for 16806 to be merged which introduces abstract mutable
> vtables (1) on top of which I want to build what you have proposed.
> I do not think we need a non-virtual table for this and this is
> actually super handy in this case because we can react on updates /
> inserts / deletes and
> subscribe / unsubscribe an event to DiagnosticsService while modifying
> that vtable. Otherwise, I do not see an easy and straightforward way
> to react
> to our modifications to that table (maybe via QueryHandler but that is
> quite cumbersome and not too performance friendly).
>
> The question I have is rather semantic one. If we enable / disable
> events via this table, a user will suddenly have two ways to subscribe
> - via JMX and by CQL. Is this ok?
> If one subscribes via JMX, this subscription is not propagated to the
> underlying CQL table. So she might subscribe to 5 events but there
> would be none in vtable. On the other hand,
> if we subscribe via CQL, that will populate some maps in
> DiagnosticsService / DiagnosticsPersistence. Hence, my concern is
> about having this discrepancy between what we see
> in vtable and what is enabled via JMX path. How would you address this?
>
> Regards
>
> (1) https://github.com/apache/cassandra/pull/1117/files
>
> On Sat, 24 Jul 2021 at 18:59, Mick Semb Wever <mc...@apache.org> wrote:
> >
> > >
> > > I am not sure yet how the implementation in case of virtual tables
> > > fits into the overall picture of "pluggability".
> >
> >
> >
> > Yeah, it was a goal of the design to make writing new types as easy as
> > possible, so having to wire up a new vtable for each new event type works
> > against that.
> >
> > I'd be inclined to start with just two tables: one "diagnostic_events"
> that
> > lists all events, and another "diagnostic_service" which lists what's
> > enabled (and allows you to enable them live). With the design of
> > Diagnostic Events as subscribe and pull, it would make sense (to me) if
> > enabling an event class in the diagnostic_service table ('update
> > diagnostic_service set enabled = true where class = '<event_name>') then
> > also does a subscribesAll with a noop consumer.
> >
> > Extending/configuring the limit makes sense as part of the binlog
> > implementation. I'm hesitant about allowing users to increase the
> in-memory
> > store.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] Diagnostic events in virtual tables
Posted by Stefan Miklosovic <st...@instaclustr.com>.
Hi Mick,
I returned to this after some time and here are my questions about this.
I am waiting for 16806 to be merged which introduces abstract mutable
vtables (1) on top of which I want to build what you have proposed.
I do not think we need a non-virtual table for this and this is
actually super handy in this case because we can react on updates /
inserts / deletes and
subscribe / unsubscribe an event to DiagnosticsService while modifying
that vtable. Otherwise, I do not see an easy and straightforward way
to react
to our modifications to that table (maybe via QueryHandler but that is
quite cumbersome and not too performance friendly).
The question I have is rather semantic one. If we enable / disable
events via this table, a user will suddenly have two ways to subscribe
- via JMX and by CQL. Is this ok?
If one subscribes via JMX, this subscription is not propagated to the
underlying CQL table. So she might subscribe to 5 events but there
would be none in vtable. On the other hand,
if we subscribe via CQL, that will populate some maps in
DiagnosticsService / DiagnosticsPersistence. Hence, my concern is
about having this discrepancy between what we see
in vtable and what is enabled via JMX path. How would you address this?
Regards
(1) https://github.com/apache/cassandra/pull/1117/files
On Sat, 24 Jul 2021 at 18:59, Mick Semb Wever <mc...@apache.org> wrote:
>
> >
> > I am not sure yet how the implementation in case of virtual tables
> > fits into the overall picture of "pluggability".
>
>
>
> Yeah, it was a goal of the design to make writing new types as easy as
> possible, so having to wire up a new vtable for each new event type works
> against that.
>
> I'd be inclined to start with just two tables: one "diagnostic_events" that
> lists all events, and another "diagnostic_service" which lists what's
> enabled (and allows you to enable them live). With the design of
> Diagnostic Events as subscribe and pull, it would make sense (to me) if
> enabling an event class in the diagnostic_service table ('update
> diagnostic_service set enabled = true where class = '<event_name>') then
> also does a subscribesAll with a noop consumer.
>
> Extending/configuring the limit makes sense as part of the binlog
> implementation. I'm hesitant about allowing users to increase the in-memory
> store.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Diagnostic events in virtual tables
Posted by Mick Semb Wever <mc...@apache.org>.
>
> I am not sure yet how the implementation in case of virtual tables
> fits into the overall picture of "pluggability".
Yeah, it was a goal of the design to make writing new types as easy as
possible, so having to wire up a new vtable for each new event type works
against that.
I'd be inclined to start with just two tables: one "diagnostic_events" that
lists all events, and another "diagnostic_service" which lists what's
enabled (and allows you to enable them live). With the design of
Diagnostic Events as subscribe and pull, it would make sense (to me) if
enabling an event class in the diagnostic_service table ('update
diagnostic_service set enabled = true where class = '<event_name>') then
also does a subscribesAll with a noop consumer.
Extending/configuring the limit makes sense as part of the binlog
implementation. I'm hesitant about allowing users to increase the in-memory
store.
Re: [DISCUSS] Diagnostic events in virtual tables
Posted by Stefan Miklosovic <st...@instaclustr.com>.
Yes, limits are good, in case of a solution for virtual tables. I can
imagine that it will be use just for the sake of diagnostics what a
node is doing and so on and even we lift the limit to something like
1000, that is already a plenty to show a respective operator (as a
person) what a node does in the particular moment or some time ago.
For a more "forensics" approach it might be ideal to put it to
chronicle queues to have it persistent and an offline debugging but
that is something the other (pluggable) implementation would cover.
It should be also possible to tune this in runtime, e.g. the limit,
not sure how hard this is to implement in the run time when
diagnostics "framework" is started, however we should be at least able
to turn it off completely on demand.
I am not sure yet how the implementation in case of virtual tables
fits into the overall picture of "pluggability". I would say that is a
completely standalone approach. Virtual tables are quite unique in
this regard. However, I might somehow "force it" to implement some
extension point / interface just for the sake of having it all on the
same approach.
On Sat, 24 Jul 2021 at 10:34, Mick Semb Wever <mc...@apache.org> wrote:
>
> Some input…
>
> We don't need a CEP for this: Diagnostic Events already exists, as do
> Virtual Tables. But if you want to take it to a CEP, that's to be
> encouraged :)
>
> Virtual Tables should remain virtual, i.e. they shouldn't be backed by
> custom system_* tables solely for the sake of persisting the virtual table
> data. Furthermore, I'd avoid a) the write amplification this would cause,
> and b) putting queue data into a system table. (Let's not repeat the hints
> table).
>
> A big +1 to adding a virtual table that reads `DiagnosticEventPersistence.
> instance().getEvents(..)`
> That is, adding a virtual table for diagnostic events that honours the
> already existing backend store (whatever it is memory or binlog). This will
> be an effective and elegant solution to CASSANDRA-13472 (thank you for
> bringing this to light!)
>
> And, a big +1 to adding local persistence of diagnostic events using binlog
> (ChronicleQueue).
>
>
> > My concern is that there would be "too much data" in them and it might grow
> and grow as a node is live, …
>
>
> It is in the same category as auditing and fql. The limit is there as it is
> expected that a client is collecting these events, and that the store
> (memory or binlog) is not the permanent archive. Check out how Reaper
> collects and archives diagnostic events:
> https://github.com/thelastpickle/cassandra-reaper/blob/master/src/server/src/main/java/io/cassandrareaper/service/DiagEventPoller.java
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Diagnostic events in virtual tables
Posted by Mick Semb Wever <mc...@apache.org>.
Some input…
We don't need a CEP for this: Diagnostic Events already exists, as do
Virtual Tables. But if you want to take it to a CEP, that's to be
encouraged :)
Virtual Tables should remain virtual, i.e. they shouldn't be backed by
custom system_* tables solely for the sake of persisting the virtual table
data. Furthermore, I'd avoid a) the write amplification this would cause,
and b) putting queue data into a system table. (Let's not repeat the hints
table).
A big +1 to adding a virtual table that reads `DiagnosticEventPersistence.
instance().getEvents(..)`
That is, adding a virtual table for diagnostic events that honours the
already existing backend store (whatever it is memory or binlog). This will
be an effective and elegant solution to CASSANDRA-13472 (thank you for
bringing this to light!)
And, a big +1 to adding local persistence of diagnostic events using binlog
(ChronicleQueue).
> My concern is that there would be "too much data" in them and it might grow
and grow as a node is live, …
It is in the same category as auditing and fql. The limit is there as it is
expected that a client is collecting these events, and that the store
(memory or binlog) is not the permanent archive. Check out how Reaper
collects and archives diagnostic events:
https://github.com/thelastpickle/cassandra-reaper/blob/master/src/server/src/main/java/io/cassandrareaper/service/DiagEventPoller.java
Re: [DISCUSS] Diagnostic events in virtual tables
Posted by Stefan Miklosovic <st...@instaclustr.com>.
Thank you both for the answers, really appreciate it.
My concern is that there would be "too much data" in them and it might
grow and grow as a node is live, however, that being said, I think
there is currently a hardcap like 200 events per event type in the
current implementation (1). Hence I see that it is somehow limited
even for now which might be just fine. Nevertheless that limit might
be at least configurable (at runtime).
Ideally, there should be some kind of way to persist them anywhere,
not only to some in-memory structure so virtual tables and bin logs
are clearly two candidates to do this. So when I extend this idea, it
should be just pluggable or some kind of extension point should be
there.
I will give it a second thought how that might be generalised and I'll
try to come up with CEP covering this.
Thank you once again for the response.
(1) https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/diag/store/DiagnosticEventMemoryStore.java#L39
On Wed, 21 Jul 2021 at 18:22, Jeremiah D Jordan
<je...@gmail.com> wrote:
>
> Yes, I think it would make sense to have the events available in a virtual table, especially if we are trying to move our operational management in that direction.
>
> But, why does it need to be bin log or virtual tables? Why not both? The virtual tables could even return the data stored in the bin log making them persistent if wanted.
>
> -Jeremiah
>
> > On Jul 21, 2021, at 7:45 AM, Paulo Motta <pa...@gmail.com> wrote:
> >
> > I'm not very familiar with diagnostic events, but I think there's great
> > value in providing a Virtual Table implementation to diagnostic events,
> > since this will help in the long term objective of providing a unified
> > interface to node monitoring, so +1 from my side. I think it would
> > definitely help to write a CEP to detail the proposal and cast a vote if
> > there are no objections.
> >
> > In case you are not aware, Apache has the concept of lazy consensus <
> > https://community.apache.org/committers/lazyConsensus.html>, so as long as
> > nobody opposes your proposal you're good to move forward with it, so people
> > not caring to even dropping an e-mail can actually be a good thing. ;)
> >
> > Em qua., 21 de jul. de 2021 às 03:39, Stefan Miklosovic <
> > stefan.miklosovic@instaclustr.com> escreveu:
> >
> >> Hi,
> >>
> >> should I create CEP first or people just absolutely do not care to
> >> even drop an email and it does not make sense to them?
> >>
> >> Regards
> >>
> >> On Mon, 19 Jul 2021 at 15:32, Stefan Miklosovic
> >> <st...@instaclustr.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I wonder if people would be interested in having diagnostic events in
> >>> virtual tables?
> >>>
> >>> I put something together here (heavy wip) (1) but that is the idea I
> >> have.
> >>>
> >>> I know that Stefan Podkowinski did a spectacular job under
> >>> CASSANDRA-12944 and the possibility to persist locally was elaborated
> >>> on in (2) where the conclusion was made that maybe it is more suitable
> >>> to put it into chronicle queues via BinLog path and so on.
> >>>
> >>> The benefits of bin log is that we have events persisted and they can
> >>> be inspected further when node is offline.
> >>>
> >>> While data in virtual tables cease to exist after nodes are down, one
> >>> nice benefit of having it in virtual tables is that we can query it
> >>> comfortably via CQL and I think that this solution is more suitable to
> >>> have on an every day basis from operator's point of view. There is
> >>> still a way to dump it somewhere else anyway if one is really
> >>> interested in doing so.
> >>>
> >>> Do you think that the bin log solution is overall superior to the
> >>> virtual tables approach and we should just forget about having it in
> >>> virtual tables?
> >>>
> >>> If that is the case, what I would like to see is to have some
> >>> pluggability here so I might implement this on my own and configure a
> >>> node to put it to virtual tables for me, it does not necessarily have
> >>> to be the part of Cassandra code base if we are strongly against that.
> >>>
> >>> (1)
> >> https://github.com/instaclustr/cassandra/commit/0dd60dc0a847619fdb704b700154e624b21a0c35
> >>> (2) https://issues.apache.org/jira/browse/CASSANDRA-13460
> >>>
> >>> Regards
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Diagnostic events in virtual tables
Posted by Jeremiah D Jordan <je...@gmail.com>.
Yes, I think it would make sense to have the events available in a virtual table, especially if we are trying to move our operational management in that direction.
But, why does it need to be bin log or virtual tables? Why not both? The virtual tables could even return the data stored in the bin log making them persistent if wanted.
-Jeremiah
> On Jul 21, 2021, at 7:45 AM, Paulo Motta <pa...@gmail.com> wrote:
>
> I'm not very familiar with diagnostic events, but I think there's great
> value in providing a Virtual Table implementation to diagnostic events,
> since this will help in the long term objective of providing a unified
> interface to node monitoring, so +1 from my side. I think it would
> definitely help to write a CEP to detail the proposal and cast a vote if
> there are no objections.
>
> In case you are not aware, Apache has the concept of lazy consensus <
> https://community.apache.org/committers/lazyConsensus.html>, so as long as
> nobody opposes your proposal you're good to move forward with it, so people
> not caring to even dropping an e-mail can actually be a good thing. ;)
>
> Em qua., 21 de jul. de 2021 às 03:39, Stefan Miklosovic <
> stefan.miklosovic@instaclustr.com> escreveu:
>
>> Hi,
>>
>> should I create CEP first or people just absolutely do not care to
>> even drop an email and it does not make sense to them?
>>
>> Regards
>>
>> On Mon, 19 Jul 2021 at 15:32, Stefan Miklosovic
>> <st...@instaclustr.com> wrote:
>>>
>>> Hi,
>>>
>>> I wonder if people would be interested in having diagnostic events in
>>> virtual tables?
>>>
>>> I put something together here (heavy wip) (1) but that is the idea I
>> have.
>>>
>>> I know that Stefan Podkowinski did a spectacular job under
>>> CASSANDRA-12944 and the possibility to persist locally was elaborated
>>> on in (2) where the conclusion was made that maybe it is more suitable
>>> to put it into chronicle queues via BinLog path and so on.
>>>
>>> The benefits of bin log is that we have events persisted and they can
>>> be inspected further when node is offline.
>>>
>>> While data in virtual tables cease to exist after nodes are down, one
>>> nice benefit of having it in virtual tables is that we can query it
>>> comfortably via CQL and I think that this solution is more suitable to
>>> have on an every day basis from operator's point of view. There is
>>> still a way to dump it somewhere else anyway if one is really
>>> interested in doing so.
>>>
>>> Do you think that the bin log solution is overall superior to the
>>> virtual tables approach and we should just forget about having it in
>>> virtual tables?
>>>
>>> If that is the case, what I would like to see is to have some
>>> pluggability here so I might implement this on my own and configure a
>>> node to put it to virtual tables for me, it does not necessarily have
>>> to be the part of Cassandra code base if we are strongly against that.
>>>
>>> (1)
>> https://github.com/instaclustr/cassandra/commit/0dd60dc0a847619fdb704b700154e624b21a0c35
>>> (2) https://issues.apache.org/jira/browse/CASSANDRA-13460
>>>
>>> Regards
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Diagnostic events in virtual tables
Posted by Paulo Motta <pa...@gmail.com>.
I'm not very familiar with diagnostic events, but I think there's great
value in providing a Virtual Table implementation to diagnostic events,
since this will help in the long term objective of providing a unified
interface to node monitoring, so +1 from my side. I think it would
definitely help to write a CEP to detail the proposal and cast a vote if
there are no objections.
In case you are not aware, Apache has the concept of lazy consensus <
https://community.apache.org/committers/lazyConsensus.html>, so as long as
nobody opposes your proposal you're good to move forward with it, so people
not caring to even dropping an e-mail can actually be a good thing. ;)
Em qua., 21 de jul. de 2021 às 03:39, Stefan Miklosovic <
stefan.miklosovic@instaclustr.com> escreveu:
> Hi,
>
> should I create CEP first or people just absolutely do not care to
> even drop an email and it does not make sense to them?
>
> Regards
>
> On Mon, 19 Jul 2021 at 15:32, Stefan Miklosovic
> <st...@instaclustr.com> wrote:
> >
> > Hi,
> >
> > I wonder if people would be interested in having diagnostic events in
> > virtual tables?
> >
> > I put something together here (heavy wip) (1) but that is the idea I
> have.
> >
> > I know that Stefan Podkowinski did a spectacular job under
> > CASSANDRA-12944 and the possibility to persist locally was elaborated
> > on in (2) where the conclusion was made that maybe it is more suitable
> > to put it into chronicle queues via BinLog path and so on.
> >
> > The benefits of bin log is that we have events persisted and they can
> > be inspected further when node is offline.
> >
> > While data in virtual tables cease to exist after nodes are down, one
> > nice benefit of having it in virtual tables is that we can query it
> > comfortably via CQL and I think that this solution is more suitable to
> > have on an every day basis from operator's point of view. There is
> > still a way to dump it somewhere else anyway if one is really
> > interested in doing so.
> >
> > Do you think that the bin log solution is overall superior to the
> > virtual tables approach and we should just forget about having it in
> > virtual tables?
> >
> > If that is the case, what I would like to see is to have some
> > pluggability here so I might implement this on my own and configure a
> > node to put it to virtual tables for me, it does not necessarily have
> > to be the part of Cassandra code base if we are strongly against that.
> >
> > (1)
> https://github.com/instaclustr/cassandra/commit/0dd60dc0a847619fdb704b700154e624b21a0c35
> > (2) https://issues.apache.org/jira/browse/CASSANDRA-13460
> >
> > Regards
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] Diagnostic events in virtual tables
Posted by Stefan Miklosovic <st...@instaclustr.com>.
Hi,
should I create CEP first or people just absolutely do not care to
even drop an email and it does not make sense to them?
Regards
On Mon, 19 Jul 2021 at 15:32, Stefan Miklosovic
<st...@instaclustr.com> wrote:
>
> Hi,
>
> I wonder if people would be interested in having diagnostic events in
> virtual tables?
>
> I put something together here (heavy wip) (1) but that is the idea I have.
>
> I know that Stefan Podkowinski did a spectacular job under
> CASSANDRA-12944 and the possibility to persist locally was elaborated
> on in (2) where the conclusion was made that maybe it is more suitable
> to put it into chronicle queues via BinLog path and so on.
>
> The benefits of bin log is that we have events persisted and they can
> be inspected further when node is offline.
>
> While data in virtual tables cease to exist after nodes are down, one
> nice benefit of having it in virtual tables is that we can query it
> comfortably via CQL and I think that this solution is more suitable to
> have on an every day basis from operator's point of view. There is
> still a way to dump it somewhere else anyway if one is really
> interested in doing so.
>
> Do you think that the bin log solution is overall superior to the
> virtual tables approach and we should just forget about having it in
> virtual tables?
>
> If that is the case, what I would like to see is to have some
> pluggability here so I might implement this on my own and configure a
> node to put it to virtual tables for me, it does not necessarily have
> to be the part of Cassandra code base if we are strongly against that.
>
> (1) https://github.com/instaclustr/cassandra/commit/0dd60dc0a847619fdb704b700154e624b21a0c35
> (2) https://issues.apache.org/jira/browse/CASSANDRA-13460
>
> Regards
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org