Re: [DISCUSS] Diagnostic events in virtual tables

[Stefan Miklosovic <st...@instaclustr.com>]

Hi Mick,

I returned to this after some time and here are my questions about this.

I am waiting for 16806 to be merged which introduces abstract mutable
vtables (1) on top of which I want to build what you have proposed.
I do not think we need a non-virtual table for this and this is
actually super handy in this case because we can react on updates /
inserts / deletes and
subscribe / unsubscribe an event to DiagnosticsService while modifying
that vtable. Otherwise, I do not see an easy and straightforward way
to react
to our modifications to that table (maybe via QueryHandler but that is
quite cumbersome and not too performance friendly).

The question I have is rather semantic one. If we enable / disable
events via this table, a user will suddenly have two ways to subscribe
- via JMX and by CQL. Is this ok?
If one subscribes via JMX, this subscription is not propagated to the
underlying CQL table. So she might subscribe to 5 events but there
would be none in vtable. On the other hand,
if we subscribe via CQL, that will populate some maps in
DiagnosticsService / DiagnosticsPersistence. Hence, my concern is
about having this discrepancy between what we see
in vtable and what is enabled via JMX path. How would you address this?

Regards

(1) https://github.com/apache/cassandra/pull/1117/files

On Sat, 24 Jul 2021 at 18:59, Mick Semb Wever <mc...@apache.org> wrote:
>
> >
> > I am not sure yet how the implementation in case of virtual tables
> > fits into the overall picture of "pluggability".
>
>
>
> Yeah, it was a goal of the design to make writing new types as easy as
> possible, so having to wire up a new vtable for each new event type works
> against that.
>
> I'd be inclined to start with just two tables: one "diagnostic_events" that
> lists all events, and another "diagnostic_service" which lists what's
> enabled (and allows you to enable them live).  With the design of
> Diagnostic Events as subscribe and pull, it would make sense (to me) if
> enabling an event class in the diagnostic_service table ('update
> diagnostic_service set enabled = true where class = '<event_name>') then
> also does a subscribesAll with a noop consumer.
>
> Extending/configuring the limit makes sense as part of the binlog
> implementation. I'm hesitant about allowing users to increase the in-memory
> store.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org

Re: [DISCUSS] Diagnostic events in virtual tables

[Mick Semb Wever <mc...@apache.org>]

>
> If one subscribes via JMX, this subscription is not propagated to the
>
underlying CQL table. So she might subscribe to 5 events but there
> would be none in vtable.



There isn't any need for any underlying CQL table here.
The enabled state, and subscriptions, is in DatabaseDescriptor
and DiagnosticEventService. Both interfaces: JMX and vtable; would update
these values. And it is intentional that any changes made at runtime are
reset when a node restarts. If the events are implemented to be persisted
in a chronicle queue, then they will be available after a restart (readable
via either jmx or vtable).

Let's continue this discussion on CASSANDRA-13460.

Re: [DISCUSS] Diagnostic events in virtual tables

[Chris Lohfink <cl...@gmail.com>]

Perhaps re-add the settings virtual table mutability. That way the same
place can be used to update settings at runtime for multiple things instead
of creating a new virtual table per service we want to make hot props for.

Might be kinda nice to allow REGISTER and EVENT CQL events to be created
with virtual tables as well for some extended functionality.

Chris

On Fri, Sep 10, 2021 at 2:36 AM Stefan Miklosovic <
stefan.miklosovic@instaclustr.com> wrote:

> Hi Mick,
>
> I returned to this after some time and here are my questions about this.
>
> I am waiting for 16806 to be merged which introduces abstract mutable
> vtables (1) on top of which I want to build what you have proposed.
> I do not think we need a non-virtual table for this and this is
> actually super handy in this case because we can react on updates /
> inserts / deletes and
> subscribe / unsubscribe an event to DiagnosticsService while modifying
> that vtable. Otherwise, I do not see an easy and straightforward way
> to react
> to our modifications to that table (maybe via QueryHandler but that is
> quite cumbersome and not too performance friendly).
>
> The question I have is rather semantic one. If we enable / disable
> events via this table, a user will suddenly have two ways to subscribe
> - via JMX and by CQL. Is this ok?
> If one subscribes via JMX, this subscription is not propagated to the
> underlying CQL table. So she might subscribe to 5 events but there
> would be none in vtable. On the other hand,
> if we subscribe via CQL, that will populate some maps in
> DiagnosticsService / DiagnosticsPersistence. Hence, my concern is
> about having this discrepancy between what we see
> in vtable and what is enabled via JMX path. How would you address this?
>
> Regards
>
> (1) https://github.com/apache/cassandra/pull/1117/files
>
> On Sat, 24 Jul 2021 at 18:59, Mick Semb Wever <mc...@apache.org> wrote:
> >
> > >
> > > I am not sure yet how the implementation in case of virtual tables
> > > fits into the overall picture of "pluggability".
> >
> >
> >
> > Yeah, it was a goal of the design to make writing new types as easy as
> > possible, so having to wire up a new vtable for each new event type works
> > against that.
> >
> > I'd be inclined to start with just two tables: one "diagnostic_events"
> that
> > lists all events, and another "diagnostic_service" which lists what's
> > enabled (and allows you to enable them live).  With the design of
> > Diagnostic Events as subscribe and pull, it would make sense (to me) if
> > enabling an event class in the diagnostic_service table ('update
> > diagnostic_service set enabled = true where class = '<event_name>') then
> > also does a subscribesAll with a noop consumer.
> >
> > Extending/configuring the limit makes sense as part of the binlog
> > implementation. I'm hesitant about allowing users to increase the
> in-memory
> > store.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>