You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by "D. Salvatore" <dd...@gmail.com> on 2018/03/12 16:43:35 UTC
Anomaly detection
Hello everyone,
Do you know if exist a Cassandra tool that performs anomaly detection?
Thank you in advance
Salvatore
Re: Anomaly detection
Posted by Alain RODRIGUEZ <ar...@gmail.com>.
Hello Salvatore,
I used SPM Sematext that had this feature already 3 or 4 years ago I
believe. SPM offers some out of the box dashboards for Cassandra. See
https://sematext.com/spm/
More recently we built some templates for Cassandra in Data Dog that I
believe should show the most important metrics and hopefully in a
usable/efficient way for alerting and troubleshooting. If this is of
interest, check https://www.datadoghq.com/blog/tlp-cassandra-dashboards/.
Datadog provides anomaly detection for a while too:
https://www.datadoghq.com/blog/introducing-anomaly-detection-datadog/.
If you choose Datadog, adding anomaly detection to the overview dashboard
charts is probably a good starting point.
The tools mentioned above are commercial solutions. I am not sure about
which open source stacks could provide this feature.
That being said, looking from time to time to the dashboards (at least) and
setting alerts using the standard 'thresholds' was often enough. I did not
make real use of anomaly detections, even though there are probably
interesting things that can be done on alerting with this feature.
Hope that helps,
C*heers,
-----------------------
Alain Rodriguez - @arodream - alain@thelastpickle.com
France / Spain
The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com
2018-03-13 10:07 GMT+00:00 Rahul Singh <ra...@gmail.com>:
> I’ve used OpsCenter, New Relic, Splunk, and ELK and all of them have ways
> to visualize what’s going on. Eventually I just forked a cfstats2csv python
> program and started making formatted excel files which made it easy to spot
> anomalies and filter keyspaces / tables across nodes. I have some basic
> anomaly detection based on std. deviation but it’s only a static snapshot
> based detection. I think you want something that may be looking at the time
> series of the whole dataset.
>
> Regardless cfstats , tpstats are good places to see
> What’s going on and then determine what you need to monitor via other
> tools.
>
> --
> Rahul Singh
> rahul.singh@anant.us
>
> Anant Corporation
>
> On Mar 12, 2018, 10:02 PM -0400, Fernando Ipar <ip...@gmail.com>,
> wrote:
>
> Hello Salvatore,
>
> On Mon, Mar 12, 2018 at 2:12 PM, D. Salvatore <dd...@gmail.com>
> wrote:
>
>> Hi Rahul,
>> I was mainly thinking about performance anomaly detection but I am also
>> interested in other types such as fault detection, data or queries
>> anomalies.
>>
>
> I know VividCortex (http://vividcortex.com) supports Cassandra (2.1 or
> greater) and I also know it does automatic (they call it adaptive) fault
> detection for MySQL. I took a quick look at their website and could not
> find an explicit list of features they support for Cassandra but it's
> possible that fault detection is one of them too, so if SaaS is an option
> I'd recommend you take a look at them.
>
> Regards,
>
> Fernando Ipar
>
> http://fernandoipar.com
>
>
Re: Anomaly detection
Posted by Rahul Singh <ra...@gmail.com>.
I’ve used OpsCenter, New Relic, Splunk, and ELK and all of them have ways to visualize what’s going on. Eventually I just forked a cfstats2csv python program and started making formatted excel files which made it easy to spot anomalies and filter keyspaces / tables across nodes. I have some basic anomaly detection based on std. deviation but it’s only a static snapshot based detection. I think you want something that may be looking at the time series of the whole dataset.
Regardless cfstats , tpstats are good places to see
What’s going on and then determine what you need to monitor via other tools.
--
Rahul Singh
rahul.singh@anant.us
Anant Corporation
On Mar 12, 2018, 10:02 PM -0400, Fernando Ipar <ip...@gmail.com>, wrote:
> Hello Salvatore,
>
> > On Mon, Mar 12, 2018 at 2:12 PM, D. Salvatore <dd...@gmail.com> wrote:
> > > Hi Rahul,
> > > I was mainly thinking about performance anomaly detection but I am also interested in other types such as fault detection, data or queries anomalies.
> >
> > I know VividCortex (http://vividcortex.com) supports Cassandra (2.1 or greater) and I also know it does automatic (they call it adaptive) fault detection for MySQL. I took a quick look at their website and could not find an explicit list of features they support for Cassandra but it's possible that fault detection is one of them too, so if SaaS is an option I'd recommend you take a look at them.
> >
> > Regards,
> > Fernando Ipar
> > http://fernandoipar.com
Re: Anomaly detection
Posted by Fernando Ipar <ip...@gmail.com>.
Hello Salvatore,
On Mon, Mar 12, 2018 at 2:12 PM, D. Salvatore <dd...@gmail.com>
wrote:
> Hi Rahul,
> I was mainly thinking about performance anomaly detection but I am also
> interested in other types such as fault detection, data or queries
> anomalies.
>
I know VividCortex (http://vividcortex.com) supports Cassandra (2.1 or
greater) and I also know it does automatic (they call it adaptive) fault
detection for MySQL. I took a quick look at their website and could not
find an explicit list of features they support for Cassandra but it's
possible that fault detection is one of them too, so if SaaS is an option
I'd recommend you take a look at them.
Regards,
Fernando Ipar
http://fernandoipar.com
Re: Anomaly detection
Posted by "D. Salvatore" <dd...@gmail.com>.
Hi Rahul,
I was mainly thinking about performance anomaly detection but I am also
interested in other types such as fault detection, data or queries
anomalies.
Thanks
2018-03-12 16:52 GMT+00:00 Rahul Singh <ra...@gmail.com>:
> Anomaly detection of what? The data inside Cassandra or Casandra metrics?
>
> --
> Rahul Singh
> rahul.singh@anant.us
>
> Anant Corporation
>
> On Mar 12, 2018, 12:44 PM -0400, D. Salvatore <dd...@gmail.com>,
> wrote:
>
> Hello everyone,
> Do you know if exist a Cassandra tool that performs anomaly detection?
>
> Thank you in advance
> Salvatore
>
>
Re: Anomaly detection
Posted by Rahul Singh <ra...@gmail.com>.
Anomaly detection of what? The data inside Cassandra or Casandra metrics?
--
Rahul Singh
rahul.singh@anant.us
Anant Corporation
On Mar 12, 2018, 12:44 PM -0400, D. Salvatore <dd...@gmail.com>, wrote:
> Hello everyone,
> Do you know if exist a Cassandra tool that performs anomaly detection?
>
> Thank you in advance
> Salvatore