CloudWatch Metrics Reporter

[Rafi Aroch <ra...@gmail.com>]

Hi,

In my team we use CloudWatch as our monitoring & alerting system.
I noticed that CloudWatch does not appear in the list of supported
Reporters.
I was wondering why is that. Was there no demand from the community? Is it
related to licensing issue with AWS? Was it a technical concern?

Would you accept this contribution into Flink?

Thanks,
Rafi

Re: CloudWatch Metrics Reporter

[Bowen Li <bo...@gmail.com>]

To @Chesnay 's question on AWS library: yes, it would require using
aws-java-sdk-core
<https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-core>and
aws-java-sdk-cloudwatch
<https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-cloudwatch>.
Both
are of Apache License 2.0. This is different than
flink-connector-kinesis's Kinesis
Producer/Consumer Libraries which use Amazon Software License. Thus Flink
can build a cloudwatch reporter module and publish to mvn.

However, I'm neutral to developing a cloudwatch reporter module, the thing
itself. According to my own experience of using cloudwatch, it's ok for
metrics of other AWS services, but it's neither handy nor friendly for
metrics of external services, because of its design limitations. I didn't
use cloudwatch even when I ran Flink on AWS EMR. Not sure how popular it is
in Flink community. Besides, it takes future commitment of maintaining it.
Thus, we can give it more thoughts if we decide to build a cloudwatch
reporter module.

Thanks, Bowen



On Wed, May 16, 2018 at 1:41 AM, Dyana Rose <dy...@salecycle.com>
wrote:

> I've written a cloud watch reporter for our own use. It's not pretty to
> crack out the metrics correctly for cloudwatch as the current metrics don't
> all set the metric names in a good hierarchy and then they aren't all added
> to the metric variables either.
>
> If someone opens the Jira I can see about getting our code up as an example
> branch of what I had to do. Unless I missed something, I think the current
> metrics need a bit of a brush up.
>
> Dyana
>
> On 16 May 2018 at 09:23, Chesnay Schepler <ch...@apache.org> wrote:
>
> > Hello,
> >
> > there was no demand for a CloudWatch reporter so far.
> >
> > I only quickly skimmed the API docs, but it appears that the data is
> > inserted via REST.
> > Would the reporter require the usage of any aws library, or could be use
> > an arbitrary http client?
> > If it is the latter there shouldn't be a licensing issue as i understand
> > it.
> >
> > Please open a JIRA, let's move the discussion there.
> >
> >
> > On 16.05.2018 10:12, Rafi Aroch wrote:
> >
> >> Hi,
> >>
> >> In my team we use CloudWatch as our monitoring & alerting system.
> >> I noticed that CloudWatch does not appear in the list of supported
> >> Reporters.
> >> I was wondering why is that. Was there no demand from the community? Is
> it
> >> related to licensing issue with AWS? Was it a technical concern?
> >>
> >> Would you accept this contribution into Flink?
> >>
> >> Thanks,
> >> Rafi
> >>
> >>
> >
>
>
> --
>
> Dyana Rose
> Software Engineer
>
>
> W: www.salecycle.com <http://www.salecycle.com/>
> [image: Marketing Permissions Service]
> <https://t.xink.io/Tracking/Index/9LwBAKNtAAAwphkA0>
>

Re: CloudWatch Metrics Reporter

[Dyana Rose <dy...@salecycle.com>]

I've written a cloud watch reporter for our own use. It's not pretty to
crack out the metrics correctly for cloudwatch as the current metrics don't
all set the metric names in a good hierarchy and then they aren't all added
to the metric variables either.

If someone opens the Jira I can see about getting our code up as an example
branch of what I had to do. Unless I missed something, I think the current
metrics need a bit of a brush up.

Dyana

On 16 May 2018 at 09:23, Chesnay Schepler <ch...@apache.org> wrote:

> Hello,
>
> there was no demand for a CloudWatch reporter so far.
>
> I only quickly skimmed the API docs, but it appears that the data is
> inserted via REST.
> Would the reporter require the usage of any aws library, or could be use
> an arbitrary http client?
> If it is the latter there shouldn't be a licensing issue as i understand
> it.
>
> Please open a JIRA, let's move the discussion there.
>
>
> On 16.05.2018 10:12, Rafi Aroch wrote:
>
>> Hi,
>>
>> In my team we use CloudWatch as our monitoring & alerting system.
>> I noticed that CloudWatch does not appear in the list of supported
>> Reporters.
>> I was wondering why is that. Was there no demand from the community? Is it
>> related to licensing issue with AWS? Was it a technical concern?
>>
>> Would you accept this contribution into Flink?
>>
>> Thanks,
>> Rafi
>>
>>
>


-- 

Dyana Rose
Software Engineer


W: www.salecycle.com <http://www.salecycle.com/>
[image: Marketing Permissions Service]
<https://t.xink.io/Tracking/Index/9LwBAKNtAAAwphkA0>

Re: CloudWatch Metrics Reporter

[Chesnay Schepler <ch...@apache.org>]

Hello,

there was no demand for a CloudWatch reporter so far.

I only quickly skimmed the API docs, but it appears that the data is 
inserted via REST.
Would the reporter require the usage of any aws library, or could be use 
an arbitrary http client?
If it is the latter there shouldn't be a licensing issue as i understand it.

Please open a JIRA, let's move the discussion there.

On 16.05.2018 10:12, Rafi Aroch wrote:
> Hi,
>
> In my team we use CloudWatch as our monitoring & alerting system.
> I noticed that CloudWatch does not appear in the list of supported
> Reporters.
> I was wondering why is that. Was there no demand from the community? Is it
> related to licensing issue with AWS? Was it a technical concern?
>
> Would you accept this contribution into Flink?
>
> Thanks,
> Rafi
>