Re: Service discovery for flink-metrics-prometheus

[Dongwon Kim <ea...@gmail.com>]

Hi all,

I also suffer from the lack of service discovery for flink-metrics-prometheus while using YARN for deployment, Prometheus for instrumentation, and Flink for stream processing.
I just upload a Python script for the purpose here: https://github.com/eastcirclek/flink-service-discovery <https://github.com/eastcirclek/flink-service-discovery>
Hope it can be helpful for your use case.

Best,

- Dongwon


https://github.com/eastcirclek/flink-service-discovery <https://github.com/eastcirclek/flink-service-discovery>

> 2018. 1. 8. 오후 7:27, Chesnay Schepler <ch...@apache.org> 작성:
> 
> Yes, the logs are the only way to find out which port the reporter is bound to. 
> 
> We may be able to display this information in the web-UI, but it isn't very high on my list and will probably require 
> modifications to the reporter interface. 
> 
> On 06.01.2018 04:24, Kien Truong wrote:
>> Hi,
>> 
>> We are using YARN for deployment, so the combination of host&port for the Prometheus reporters can be really random depending on how the containers are co-located.  
>> 
>> One option we thought of was scrapping the log for this information, but it can be really messy in the long run.
>> 
>> Regards,
>> Kien
>> 
>> Sent from TypeApp <http://www.typeapp.com/r?b=11678>
>> On Jan 5, 2018, at 03:53, Stephan Ewen <sewen@apache.org <ma...@apache.org>> wrote:
>> How are you running deploying your Flink processes? For Service Discovery for Prometheus on Kubernetes, there are a few articles out there...
>> 
>> On Thu, Jan 4, 2018 at 3:52 PM, Aljoscha Krettek <aljoscha@apache.org <ma...@apache.org>> wrote: 
>> I'm not aware of how this is typically done but maybe Chesnay (cc'ed) has an idea. 
>> 
>> > On 14. Dec 2017, at 16:55, Kien Truong < duckientruong@gmail.com <ma...@gmail.com>> wrote: 
>> > 
>> > Hi, 
>> > 
>> > Does anyone have recommendations about integrating flink-metrics-prometheus with some SD mechanism 
>> > 
>> > so that Prometheus can pick up the Task Manager's location dynamically ? 
>> > 
>> > Best regards, 
>> > 
>> > Kien 
>> > 
>> 
>> 
>