You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Bill Walters <bi...@gmail.com> on 2017/10/01 16:14:42 UTC
Hi All,
I need some help with deploying a monitoring and alerting system for our
new Cassandra 3.0.4 cluster that we are setting up in AWS East region.
I have a good experience with Cassandra as we are running some 2.0.16
clusters in production on our on-prem servers. We use Nagios tool to
monitor and alert our on-call people if the any of the nodes in our on-prem
servers go down. (Nagios is the default monitoring and alerting system used
by our company)
Since, our leadership started a plan to migrate our infrastructure to
cloud, we have chosen AWS as our public cloud.
We are planning to use same old Nagios as our monitoring and alerting
system even for our cloud servers.
But not sure if this is the ideal approach, I have seen uses cases where Yelp
used Sensu
<https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwiv55qr58_WAhWIsVQKHZx0AJkQFggoMAA&url=https%3A%2F%2Fengineeringblog.yelp.com%2F2016%2F06%2Fmonitoring-cassandra-at-scale.html&usg=AOvVaw1dSqH468Tl1nUuxLBb4YA5>
and Netflix wrote their own tool
<https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=4&cad=rja&uact=8&ved=0ahUKEwj5hK2_58_WAhVmzFQKHUSiA4YQtwIIVTAD&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dw3WbVMavy2I&usg=AOvVaw25xUCY2Jc7fGh7ngQF9ozt>
for
monitoring their cloud Cassandra clusters.
Please let me know if there are any cloud native monitoring systems that
work well with Cassandra, we will review it for our setup.
Thank You,
Bill Walters.
Re:
Posted by Lutaya Shafiq Holmes <lu...@gmail.com>.
AWS is a good choice, you will have to use Amazon Web Services EC2 .
Regards
On Sunday, October 1, 2017, Bill Walters <bi...@gmail.com> wrote:
> Hi All,
>
> I need some help with deploying a monitoring and alerting system for our
> new Cassandra 3.0.4 cluster that we are setting up in AWS East region.
> I have a good experience with Cassandra as we are running some 2.0.16
> clusters in production on our on-prem servers. We use Nagios tool to
> monitor and alert our on-call people if the any of the nodes in our on-prem
> servers go down. (Nagios is the default monitoring and alerting system used
> by our company)
> Since, our leadership started a plan to migrate our infrastructure to
> cloud, we have chosen AWS as our public cloud.
> We are planning to use same old Nagios as our monitoring and alerting
> system even for our cloud servers.
> But not sure if this is the ideal approach, I have seen uses cases where Yelp
> used Sensu
> <https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwiv55qr58_WAhWIsVQKHZx0AJkQFggoMAA&url=https%3A%2F%2Fengineeringblog.yelp.com%2F2016%2F06%2Fmonitoring-cassandra-at-scale.html&usg=AOvVaw1dSqH468Tl1nUuxLBb4YA5>
> and Netflix wrote their own tool
> <https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=4&cad=rja&uact=8&ved=0ahUKEwj5hK2_58_WAhVmzFQKHUSiA4YQtwIIVTAD&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dw3WbVMavy2I&usg=AOvVaw25xUCY2Jc7fGh7ngQF9ozt> for
> monitoring their cloud Cassandra clusters.
>
> Please let me know if there are any cloud native monitoring systems that
> work well with Cassandra, we will review it for our setup.
>
>
>
> Thank You,
> Bill Walters.
>
--
Lutaaya Shafiq
Web: www.ronzag.com | info@ronzag.com
Mobile: +256702772721 | +256783564130
Twitter: @lutayashafiq
Skype: lutaya5
Blog: lutayashafiq.com
http://www.fourcornersalliancegroup.com/?a=shafiqholmes
"The most beautiful people we have known are those who have known defeat,
known suffering, known struggle, known loss and have found their way out of
the depths. These persons have an appreciation, a sensitivity and an
understanding of life that fills them with compassion, gentleness and a
deep loving concern. Beautiful people do not just happen." - *Elisabeth
Kubler-Ross*
Re:
Posted by Oleksandr Shulgin <ol...@zalando.de>.
On Sun, Oct 1, 2017 at 6:14 PM, Bill Walters <bi...@gmail.com>
wrote:
> Hi All,
>
> I need some help with deploying a monitoring and alerting system for our
> new Cassandra 3.0.4 cluster that we are setting up in AWS East region.
> I have a good experience with Cassandra as we are running some 2.0.16
> clusters in production on our on-prem servers. We use Nagios tool to
> monitor and alert our on-call people if the any of the nodes in our on-prem
> servers go down. (Nagios is the default monitoring and alerting system used
> by our company)
> Since, our leadership started a plan to migrate our infrastructure to
> cloud, we have chosen AWS as our public cloud.
> We are planning to use same old Nagios as our monitoring and alerting
> system even for our cloud servers.
> But not sure if this is the ideal approach, I have seen uses cases where Yelp
> used Sensu
> <https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwiv55qr58_WAhWIsVQKHZx0AJkQFggoMAA&url=https%3A%2F%2Fengineeringblog.yelp.com%2F2016%2F06%2Fmonitoring-cassandra-at-scale.html&usg=AOvVaw1dSqH468Tl1nUuxLBb4YA5>
> and Netflix wrote their own tool
> <https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=4&cad=rja&uact=8&ved=0ahUKEwj5hK2_58_WAhVmzFQKHUSiA4YQtwIIVTAD&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dw3WbVMavy2I&usg=AOvVaw25xUCY2Jc7fGh7ngQF9ozt> for
> monitoring their cloud Cassandra clusters.
>
> Please let me know if there are any cloud native monitoring systems that
> work well with Cassandra, we will review it for our setup.
>
Hi,
We've written our own deployment and management tool as well (monitoring
not included): https://github.com/zalando/planb-cassandra/
This setup relies on using persistent EBS volumes for data + EC2 instance
auto-recovery feature. We've seen a number of successful recovery events
requiring 0 manual effort to recover the service during the past ~1.5 years.
This cannot work, however, with EC2 ephemeral storage (the SSD/nvme storage
attached directly to instances of t types m3, r3 and i3).
Regards,
--
Alex
Re:
Posted by daemeon reiydelle <da...@gmail.com>.
What specifically are you looking to monitor? As per above, Datadog has
superb components for monitoring, and no need do develop and support
anything, for a price of course. I have found management sometimes sees
devops resources as pretty low cost (pay for 40, get 70 hours work per
week). Depends on how big your clusters are, whether they are Hadoop MR,
add Hive, add Spark, add Ignite, etc.
Same sort of questions apply to your etl/ingest: Kafka/NiFi, Streaming, etc.
We like to say that we don’t get to choose our parents, that they were
given by chance – yet, we can truly choose whose children we wish to be. -
Seneca the Younger
*Daemeon C.M. ReiydelleSan Francisco 1.415.501.0198London 44 020 8144 9872*
On Sun, Oct 1, 2017 at 9:57 AM, Jeff Jirsa <jj...@gmail.com> wrote:
> I've seen successful AWS deployments in the past with Datadog and
> Graphite+Seyren
>
>
>
> On Sun, Oct 1, 2017 at 9:14 AM, Bill Walters <bi...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> I need some help with deploying a monitoring and alerting system for our
>> new Cassandra 3.0.4 cluster that we are setting up in AWS East region.
>> I have a good experience with Cassandra as we are running some 2.0.16
>> clusters in production on our on-prem servers. We use Nagios tool to
>> monitor and alert our on-call people if the any of the nodes in our on-prem
>> servers go down. (Nagios is the default monitoring and alerting system used
>> by our company)
>> Since, our leadership started a plan to migrate our infrastructure to
>> cloud, we have chosen AWS as our public cloud.
>> We are planning to use same old Nagios as our monitoring and alerting
>> system even for our cloud servers.
>> But not sure if this is the ideal approach, I have seen uses cases where Yelp
>> used Sensu
>> <https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwiv55qr58_WAhWIsVQKHZx0AJkQFggoMAA&url=https%3A%2F%2Fengineeringblog.yelp.com%2F2016%2F06%2Fmonitoring-cassandra-at-scale.html&usg=AOvVaw1dSqH468Tl1nUuxLBb4YA5>
>> and Netflix wrote their own tool
>> <https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=4&cad=rja&uact=8&ved=0ahUKEwj5hK2_58_WAhVmzFQKHUSiA4YQtwIIVTAD&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dw3WbVMavy2I&usg=AOvVaw25xUCY2Jc7fGh7ngQF9ozt> for
>> monitoring their cloud Cassandra clusters.
>>
>> Please let me know if there are any cloud native monitoring systems that
>> work well with Cassandra, we will review it for our setup.
>>
>>
>>
>> Thank You,
>> Bill Walters.
>>
>
>
Re:
Posted by Jeff Jirsa <jj...@gmail.com>.
I've seen successful AWS deployments in the past with Datadog and
Graphite+Seyren
On Sun, Oct 1, 2017 at 9:14 AM, Bill Walters <bi...@gmail.com>
wrote:
> Hi All,
>
> I need some help with deploying a monitoring and alerting system for our
> new Cassandra 3.0.4 cluster that we are setting up in AWS East region.
> I have a good experience with Cassandra as we are running some 2.0.16
> clusters in production on our on-prem servers. We use Nagios tool to
> monitor and alert our on-call people if the any of the nodes in our on-prem
> servers go down. (Nagios is the default monitoring and alerting system used
> by our company)
> Since, our leadership started a plan to migrate our infrastructure to
> cloud, we have chosen AWS as our public cloud.
> We are planning to use same old Nagios as our monitoring and alerting
> system even for our cloud servers.
> But not sure if this is the ideal approach, I have seen uses cases where Yelp
> used Sensu
> <https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwiv55qr58_WAhWIsVQKHZx0AJkQFggoMAA&url=https%3A%2F%2Fengineeringblog.yelp.com%2F2016%2F06%2Fmonitoring-cassandra-at-scale.html&usg=AOvVaw1dSqH468Tl1nUuxLBb4YA5>
> and Netflix wrote their own tool
> <https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=4&cad=rja&uact=8&ved=0ahUKEwj5hK2_58_WAhVmzFQKHUSiA4YQtwIIVTAD&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dw3WbVMavy2I&usg=AOvVaw25xUCY2Jc7fGh7ngQF9ozt> for
> monitoring their cloud Cassandra clusters.
>
> Please let me know if there are any cloud native monitoring systems that
> work well with Cassandra, we will review it for our setup.
>
>
>
> Thank You,
> Bill Walters.
>