MiNiFi PeriodicStatusReporter and AWS CloudWatch

[Jeff Zemerick <jz...@apache.org>]

Hi everyone,

I'm looking for ways to monitor a distributed fleet of MiNiFi (Java)
installations. I'd like essentially a dashboard that shows the overall
health of the installations. I can leverage AWS CloudWatch for the
dashboard capability with the added benefit of CloudWatch alarms. The
PeriodicStatusReporter looks like a good way to implement it. Before I give
it a go, I just wanted to make sure I'm not reinventing the wheel anywhere
and that the community might find this useful instead of just me or if
there are any gotchas that I might be missing.

Thanks!
Jeff

Re: MiNiFi PeriodicStatusReporter and AWS CloudWatch

[Jeff Zemerick <jz...@apache.org>]

Thanks! I will take a look at those.

Jeff

On Wed, May 23, 2018 at 9:20 AM Aldrin Piri <al...@gmail.com> wrote:

> Hey Jeff,

> I don't think there is anything that maps directly to the functionality
you
> are trying to cover.  From a broader, ecosystem standpoint there is a
> Processor in NiFi that transmits data to Cloudwatch as part of the AWS
> NAR.  It might be helpful to explore what functionality is there
especially
> with auth scenarios as I do remember scoping out a few PRs related to such
> matters.

> In terms of execution, I might be more inclined to opt for a reporting
> task.   I think the status reporter's original scope was more of a "local"
> and interactive experience with the instance.  For this kind of regular,
> hands off approach, you could get the same type of information in the
> format specifically needed via the context exposed to the reporting task.

> --aldrin

> On Wed, May 23, 2018 at 8:41 AM, Jeff Zemerick <jz...@apache.org>
wrote:

> > Hi everyone,
> >
> > I'm looking for ways to monitor a distributed fleet of MiNiFi (Java)
> > installations. I'd like essentially a dashboard that shows the overall
> > health of the installations. I can leverage AWS CloudWatch for the
> > dashboard capability with the added benefit of CloudWatch alarms. The
> > PeriodicStatusReporter looks like a good way to implement it. Before I
give
> > it a go, I just wanted to make sure I'm not reinventing the wheel
anywhere
> > and that the community might find this useful instead of just me or if
> > there are any gotchas that I might be missing.
> >
> > Thanks!
> > Jeff
> >

Re: MiNiFi PeriodicStatusReporter and AWS CloudWatch

[Aldrin Piri <al...@gmail.com>]

Hey Jeff,

I don't think there is anything that maps directly to the functionality you
are trying to cover.  From a broader, ecosystem standpoint there is a
Processor in NiFi that transmits data to Cloudwatch as part of the AWS
NAR.  It might be helpful to explore what functionality is there especially
with auth scenarios as I do remember scoping out a few PRs related to such
matters.

In terms of execution, I might be more inclined to opt for a reporting
task.   I think the status reporter's original scope was more of a "local"
and interactive experience with the instance.  For this kind of regular,
hands off approach, you could get the same type of information in the
format specifically needed via the context exposed to the reporting task.

--aldrin

On Wed, May 23, 2018 at 8:41 AM, Jeff Zemerick <jz...@apache.org> wrote:

> Hi everyone,
>
> I'm looking for ways to monitor a distributed fleet of MiNiFi (Java)
> installations. I'd like essentially a dashboard that shows the overall
> health of the installations. I can leverage AWS CloudWatch for the
> dashboard capability with the added benefit of CloudWatch alarms. The
> PeriodicStatusReporter looks like a good way to implement it. Before I give
> it a go, I just wanted to make sure I'm not reinventing the wheel anywhere
> and that the community might find this useful instead of just me or if
> there are any gotchas that I might be missing.
>
> Thanks!
> Jeff
>