You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Sreejith S <sr...@gmail.com> on 2016/08/29 10:09:46 UTC
Flink JMX
Hi All,
I am using Flink-1.1.1 and i enabled JMX metrics in configuration file. In
the task manger log i can see JMX is running.
Is this metrics exposed only through Flink Metrics API's ?
I tried to connect to flink JMX URL using normal javax , but connections
getting refused.
Thanks,
--
*Sreejith.S*
https://github.com/srijiths/
http://srijiths.wordpress.com/
tweet2sree@twitter <http://tweet2Sree>
Re: Flink JMX
Posted by Sreejith S <sr...@gmail.com>.
Thank you very much Chesnay, your pointer on "java.rmi.server.hostname"
solved the issue. Now i am able to get flink metrics in 1.1.1
"host" setting in JMX Reporter was because i thought of not exposing JMX
metrics as public. Since my flink cluster is in AWS cloud. So i tried to
bind it to private ip and private to pulic ip mapping for JMX port is
disabled.Not sure this make sense :)
Thank You @Robert for the comments.
Regards,
Srijith
On Mon, Aug 29, 2016 at 8:16 PM, Chesnay Schepler <ch...@apache.org>
wrote:
> Hello,
>
> That you can't access JMX in 1.0.3 even though you set all the JVM JMX
> options is unrelated to Flink. As such your JMX setup in general is broken.
> Note that in order to remotely access JMX you usually have to set
> "java.rmi.server.hostname" system-property on the host as well.
>
> Regarding the reporter:
>
> - When you manually set all the JVM JMX properties you don't have to
> specify a port for the JMXReporter.
> - There is no "host" setting for the JMX reporter. Out of curiosity,
> what do you think setting it would do?
>
> Please keep us updated about your progress.
>
> Regards,
> Chesnay
>
>
> On 29.08.2016 15:41, Robert Metzger wrote:
>
> Hi,
>
> I think in Flink 1.1.1 JMX will be started on port 8080, 8081 or 8082 (on
> the JM, 8081 is probably occupied by the web interface).
>
> On Mon, Aug 29, 2016 at 1:25 PM, Sreejith S <sr...@gmail.com> wrote:
>
>> Hi Chesnay,
>>
>> I added the below configuration in flink-conf in each taskmanagers.
>> (flink 1.0.3 version )
>>
>> # Enable JMX
>>
>> env.java.opts: -Dcom.sun.management.jmxremote
>> -Dcom.sun.management.jmxremote.port=9999
>> -Dcom.sun.management.jmxremote.authenticate=false
>> -Dcom.sun.management.jmxremote.ssl=false
>> -Dcom.sun.management.jmxremote.host=XX.XX.XX.XX
>>
>> Then tried to access it via service:jmx:rmi:///jndi/rmi://XX.XX.XX.XX:9999/jmxrmi
>> from a normal java program using javax.management.remote.JMXConnector
>>
>> But connection refusing. I checked my port, its opened.
>>
>> Then i added below configs in Flink 1.1.1
>>
>> # JMX Metrics
>> metrics.reporters : jmx_reporter
>> metrics.reporter.jmx_reporter.class: org.apache.flink.metrics.jmx.J
>> MXReporter
>> metrics.reporter.jmx_reporter.host: XX.XX.XX.XX
>> metrics.reporter.jmx_reporter.port: 8080-8082
>>
>> But no hope. Am i miss anything ?
>>
>> Thank You
>>
>>
>> On Mon, Aug 29, 2016 at 4:36 PM, Chesnay Schepler < <ch...@apache.org>
>> chesnay@apache.org> wrote:
>>
>>> Hello,
>>>
>>> can you post the jmx config entries and give us more details on how you
>>> want to access it?
>>>
>>> Regards,
>>> Chesnay
>>>
>>>
>>> On 29.08.2016 12:09, Sreejith S wrote:
>>>
>>> Hi All,
>>>
>>> I am using Flink-1.1.1 and i enabled JMX metrics in configuration file.
>>> In the task manger log i can see JMX is running.
>>>
>>> Is this metrics exposed only through Flink Metrics API's ?
>>>
>>> I tried to connect to flink JMX URL using normal javax , but connections
>>> getting refused.
>>>
>>> Thanks,
>>>
>>> --
>>>
>>>
>>> *Sreejith.S*
>>> https://github.com/srijiths/
>>> <http://srijiths.wordpress.com/>http://srijiths.wordpress.com/
>>> tweet2sree@twitter <http://tweet2Sree>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>>
>>
>> *Sreejith.S*
>> https://github.com/srijiths/
>> http://srijiths.wordpress.com/
>> tweet2sree@twitter <http://tweet2Sree>
>>
>>
>>
>>
>>
>>
>>
>
>
--
*Sreejith.S*
https://github.com/srijiths/
http://srijiths.wordpress.com/
tweet2sree@twitter <http://tweet2Sree>
Re: Flink JMX
Posted by Chesnay Schepler <ch...@apache.org>.
Hello,
That you can't access JMX in 1.0.3 even though you set all the JVM JMX
options is unrelated to Flink. As such your JMX setup in general is broken.
Note that in order to remotely access JMX you usually have to set
"java.rmi.server.hostname" system-property on the host as well.
Regarding the reporter:
* When you manually set all the JVM JMX properties you don't have to
specify a port for the JMXReporter.
* There is no "host" setting for the JMX reporter. Out of curiosity,
what do you think setting it would do?
Please keep us updated about your progress.
Regards,
Chesnay
On 29.08.2016 15:41, Robert Metzger wrote:
> Hi,
>
> I think in Flink 1.1.1 JMX will be started on port 8080, 8081 or 8082
> (on the JM, 8081 is probably occupied by the web interface).
>
> On Mon, Aug 29, 2016 at 1:25 PM, Sreejith S <srssreejith@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi Chesnay,
>
> I added the below configuration in flink-conf in each
> taskmanagers. (flink 1.0.3 version )
>
> # Enable JMX
>
> env.java.opts: -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.port=9999
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.host=XX.XX.XX.XX
>
> Then tried to access it
> via service:jmx:rmi:///jndi/rmi://XX.XX.XX.XX:9999/jmxrmi from a
> normal java program using javax.management.remote.JMXConnector
>
> But connection refusing. I checked my port, its opened.
>
> Then i added below configs in Flink 1.1.1
>
> # JMX Metrics
> metrics.reporters : jmx_reporter
> metrics.reporter.jmx_reporter.class:
> org.apache.flink.metrics.jmx.JMXReporter
> metrics.reporter.jmx_reporter.host: XX.XX.XX.XX
> metrics.reporter.jmx_reporter.port: 8080-8082
>
> But no hope. Am i miss anything ?
>
> Thank You
>
>
> On Mon, Aug 29, 2016 at 4:36 PM, Chesnay Schepler
> <chesnay@apache.org <ma...@apache.org>> wrote:
>
> Hello,
>
> can you post the jmx config entries and give us more details
> on how you want to access it?
>
> Regards,
> Chesnay
>
>
> On 29.08.2016 12:09, Sreejith S wrote:
>> Hi All,
>>
>> I am using Flink-1.1.1 and i enabled JMX metrics in
>> configuration file. In the task manger log i can see JMX is
>> running.
>>
>> Is this metrics exposed only through Flink Metrics API's ?
>>
>> I tried to connect to flink JMX URL using normal javax , but
>> connections getting refused.
>>
>> Thanks,
>>
>> --
>>
>>
>> *Sreejith.S*
>> https://github.com/srijiths/
>> http://srijiths.wordpress.com/
>> **tweet2sree@twitter <http://tweet2Sree>
>>
>>
>>
>>
>>
>>
>>
>
>
>
>
> --
>
>
> *Sreejith.S*
> https://github.com/srijiths/
> http://srijiths.wordpress.com/
> **tweet2sree@twitter <http://tweet2Sree>
>
>
>
>
>
>
>
>
Re: Flink JMX
Posted by Robert Metzger <rm...@apache.org>.
Hi,
I think in Flink 1.1.1 JMX will be started on port 8080, 8081 or 8082 (on
the JM, 8081 is probably occupied by the web interface).
On Mon, Aug 29, 2016 at 1:25 PM, Sreejith S <sr...@gmail.com> wrote:
> Hi Chesnay,
>
> I added the below configuration in flink-conf in each taskmanagers. (flink
> 1.0.3 version )
>
> # Enable JMX
>
> env.java.opts: -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.port=9999
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.host=XX.XX.XX.XX
>
> Then tried to access it via service:jmx:rmi:///jndi/rmi://XX.XX.XX.XX:9999/jmxrmi
> from a normal java program using javax.management.remote.JMXConnector
>
> But connection refusing. I checked my port, its opened.
>
> Then i added below configs in Flink 1.1.1
>
> # JMX Metrics
> metrics.reporters : jmx_reporter
> metrics.reporter.jmx_reporter.class: org.apache.flink.metrics.jmx.
> JMXReporter
> metrics.reporter.jmx_reporter.host: XX.XX.XX.XX
> metrics.reporter.jmx_reporter.port: 8080-8082
>
> But no hope. Am i miss anything ?
>
> Thank You
>
>
> On Mon, Aug 29, 2016 at 4:36 PM, Chesnay Schepler <ch...@apache.org>
> wrote:
>
>> Hello,
>>
>> can you post the jmx config entries and give us more details on how you
>> want to access it?
>>
>> Regards,
>> Chesnay
>>
>>
>> On 29.08.2016 12:09, Sreejith S wrote:
>>
>> Hi All,
>>
>> I am using Flink-1.1.1 and i enabled JMX metrics in configuration file.
>> In the task manger log i can see JMX is running.
>>
>> Is this metrics exposed only through Flink Metrics API's ?
>>
>> I tried to connect to flink JMX URL using normal javax , but connections
>> getting refused.
>>
>> Thanks,
>>
>> --
>>
>>
>> *Sreejith.S*
>> https://github.com/srijiths/
>> http://srijiths.wordpress.com/
>> tweet2sree@twitter <http://tweet2Sree>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> --
>
>
> *Sreejith.S*
> https://github.com/srijiths/
> http://srijiths.wordpress.com/
> tweet2sree@twitter <http://tweet2Sree>
>
>
>
>
>
>
>
Re: Flink JMX
Posted by Sreejith S <sr...@gmail.com>.
Hi Chesnay,
I added the below configuration in flink-conf in each taskmanagers. (flink
1.0.3 version )
# Enable JMX
env.java.opts: -Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9999
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.host=XX.XX.XX.XX
Then tried to access it
via service:jmx:rmi:///jndi/rmi://XX.XX.XX.XX:9999/jmxrmi from a normal
java program using javax.management.remote.JMXConnector
But connection refusing. I checked my port, its opened.
Then i added below configs in Flink 1.1.1
# JMX Metrics
metrics.reporters : jmx_reporter
metrics.reporter.jmx_reporter.class:
org.apache.flink.metrics.jmx.JMXReporter
metrics.reporter.jmx_reporter.host: XX.XX.XX.XX
metrics.reporter.jmx_reporter.port: 8080-8082
But no hope. Am i miss anything ?
Thank You
On Mon, Aug 29, 2016 at 4:36 PM, Chesnay Schepler <ch...@apache.org>
wrote:
> Hello,
>
> can you post the jmx config entries and give us more details on how you
> want to access it?
>
> Regards,
> Chesnay
>
>
> On 29.08.2016 12:09, Sreejith S wrote:
>
> Hi All,
>
> I am using Flink-1.1.1 and i enabled JMX metrics in configuration file. In
> the task manger log i can see JMX is running.
>
> Is this metrics exposed only through Flink Metrics API's ?
>
> I tried to connect to flink JMX URL using normal javax , but connections
> getting refused.
>
> Thanks,
>
> --
>
>
> *Sreejith.S*
> https://github.com/srijiths/
> http://srijiths.wordpress.com/
> tweet2sree@twitter <http://tweet2Sree>
>
>
>
>
>
>
>
>
--
*Sreejith.S*
https://github.com/srijiths/
http://srijiths.wordpress.com/
tweet2sree@twitter <http://tweet2Sree>
Re: Flink JMX
Posted by Chesnay Schepler <ch...@apache.org>.
Hello,
can you post the jmx config entries and give us more details on how you
want to access it?
Regards,
Chesnay
On 29.08.2016 12:09, Sreejith S wrote:
> Hi All,
>
> I am using Flink-1.1.1 and i enabled JMX metrics in configuration
> file. In the task manger log i can see JMX is running.
>
> Is this metrics exposed only through Flink Metrics API's ?
>
> I tried to connect to flink JMX URL using normal javax , but
> connections getting refused.
>
> Thanks,
>
> --
>
>
> *Sreejith.S*
> https://github.com/srijiths/
> http://srijiths.wordpress.com/
> **tweet2sree@twitter <http://tweet2Sree>
>
>
>
>
>
>
>