You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Vladimir Tretyakov <vl...@sematext.com> on 2014/09/11 13:30:14 UTC
JMXSink for YARN deployment
Hello, we are in Sematext (https://apps.sematext.com/) are writing
Monitoring tool for Spark and we came across one question:
How to enable JMX metrics for YARN deployment?
We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink"
to file $SPARK_HOME/conf/metrics.properties but it doesn't work.
Everything works in Standalone mode, but not in YARN mode.
Can somebody help?
Thx!
PS: I've found also
https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112
without answer.
Re: JMXSink for YARN deployment
Posted by Kyle Lin <ky...@gmail.com>.
Hello guys
I got a solution.
I set -Dcom.sun.management.jmxremote.port=0 to let system assign a unused
port.
Kyle
2016-01-12 16:54 GMT+08:00 Kyle Lin <ky...@gmail.com>:
> Hello there
>
>
> I run both driver and master on the same node, so I got "Port already in
> use" exception.
>
> Is there any solution to set different port for each component?
>
> Kyle
>
>
> 2015-12-05 5:54 GMT+08:00 spearson23 <sp...@gmail.com>:
>
>> Run "spark-submit --help" to see all available options.
>>
>> To get JMX to work you need to:
>>
>> spark-submit --driver-java-options "-Dcom.sun.management.jmxremote
>> -Dcom.sun.management.jmxremote.authenticate=false
>> -Dcom.sun.management.jmxremote.ssl=false
>> -Dcom.sun.management.jmxremote.port=JMX_PORT" --conf
>> spark.metrics.conf=metrics.properties --class 'CLASS_NAME' --master
>> yarn-cluster --files /PATH/TO/metrics.properties /PATH/TO/JAR.FILE
>>
>>
>> This will run JMX on the driver node on or "JMX_PORT". Note that the
>> driver
>> node and the YARN master node are not the same, you'll have to look where
>> spark put the driver node and then connect there.
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/JMXSink-for-YARN-deployment-tp13958p25572.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>
Re: JMXSink for YARN deployment
Posted by Kyle Lin <ky...@gmail.com>.
Hello there
I run both driver and master on the same node, so I got "Port already in
use" exception.
Is there any solution to set different port for each component?
Kyle
2015-12-05 5:54 GMT+08:00 spearson23 <sp...@gmail.com>:
> Run "spark-submit --help" to see all available options.
>
> To get JMX to work you need to:
>
> spark-submit --driver-java-options "-Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.port=JMX_PORT" --conf
> spark.metrics.conf=metrics.properties --class 'CLASS_NAME' --master
> yarn-cluster --files /PATH/TO/metrics.properties /PATH/TO/JAR.FILE
>
>
> This will run JMX on the driver node on or "JMX_PORT". Note that the
> driver
> node and the YARN master node are not the same, you'll have to look where
> spark put the driver node and then connect there.
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/JMXSink-for-YARN-deployment-tp13958p25572.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
Re: JMXSink for YARN deployment
Posted by spearson23 <sp...@gmail.com>.
Run "spark-submit --help" to see all available options.
To get JMX to work you need to:
spark-submit --driver-java-options "-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.port=JMX_PORT" --conf
spark.metrics.conf=metrics.properties --class 'CLASS_NAME' --master
yarn-cluster --files /PATH/TO/metrics.properties /PATH/TO/JAR.FILE
This will run JMX on the driver node on or "JMX_PORT". Note that the driver
node and the YARN master node are not the same, you'll have to look where
spark put the driver node and then connect there.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/JMXSink-for-YARN-deployment-tp13958p25572.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: JMXSink for YARN deployment
Posted by Vladimir Tretyakov <vl...@sematext.com>.
Hello again, thx for doc, tried different ways with '--files', no luck.
Can somebody who already has Spark on YARN try enable JMX metrics sink?
Maybe problem in my hands:)
PS, I've also tried to play with 'yarn.nodemanager.local-dirs', no results.
On Thu, Sep 11, 2014 at 8:23 PM, Kousuke Saruta <sa...@oss.nttdata.co.jp>
wrote:
> Hi, Vladimir
>
> You can see about --files option at
> https://spark.apache.org/docs/latest/running-on-yarn.html
>
> If you use --files option like "--files /path-to-metrics.properties", the
> file is distributed to each executor/driver's working directory, so they
> can load the file.
>
> - Kousuke
>
>
>
> (2014/09/12 2:16), Vladimir Tretyakov wrote:
>
> Hi, Kousuke,
>
> Can you please explain a bit detailed what do you mean, I am new in
> Spark, looked at
> https://spark.apache.org/docs/latest/submitting-applications.html seems
> there is no '--files' option.
>
> I just have to add '--files /path-to-metrics.properties' ? Undocumented
> ability?
>
> Thx for answer.
>
>
>
> On Thu, Sep 11, 2014 at 5:55 PM, Kousuke Saruta <sarutak@oss.nttdata.co.jp
> > wrote:
>
>> Hi Vladimir
>>
>> How about use --files option with spark-submit?
>>
>> - Kousuke
>>
>>
>> (2014/09/11 23:43), Vladimir Tretyakov wrote:
>>
>> Hi again, yeah , I've tried to use ” spark.metrics.conf” before my
>> question in ML, had no luck:(
>> Any other ideas from somebody?
>> Seems nobody use metrics in YARN deployment mode.
>> How about Mesos? I didn't try but maybe Spark has the same difficulties
>> on Mesos?
>>
>> PS: Spark is great thing in general, will be nice to see metrics in
>> YARN/Mesos mode, not only in Standalone:)
>>
>>
>> On Thu, Sep 11, 2014 at 5:25 PM, Shao, Saisai <sa...@intel.com>
>> wrote:
>>
>>> I think you can try to use ” spark.metrics.conf” to manually specify
>>> the path of metrics.properties, but the prerequisite is that each container
>>> should find this file in their local FS because this file is loaded locally.
>>>
>>>
>>>
>>> Besides I think this might be a kind of workaround, a better solution is
>>> to fix this by some other solutions.
>>>
>>>
>>>
>>> Thanks
>>>
>>> Jerry
>>>
>>>
>>>
>>> *From:* Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com]
>>> *Sent:* Thursday, September 11, 2014 10:08 PM
>>> *Cc:* user@spark.apache.org
>>> *Subject:* Re: JMXSink for YARN deployment
>>>
>>>
>>>
>>> Hi Shao, thx for explanation, any ideas how to fix it? Where should I
>>> put metrics.properties file?
>>>
>>>
>>>
>>> On Thu, Sep 11, 2014 at 4:18 PM, Shao, Saisai <sa...@intel.com>
>>> wrote:
>>>
>>> Hi，
>>>
>>>
>>>
>>> I’m guessing the problem is that driver or executor cannot get the
>>> metrics.properties configuration file in the yarn container, so metrics
>>> system cannot load the right sinks.
>>>
>>>
>>>
>>> Thanks
>>>
>>> Jerry
>>>
>>>
>>>
>>> *From:* Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com]
>>> *Sent:* Thursday, September 11, 2014 7:30 PM
>>> *To:* user@spark.apache.org
>>> *Subject:* JMXSink for YARN deployment
>>>
>>>
>>>
>>> Hello, we are in Sematext (https://apps.sematext.com/) are writing
>>> Monitoring tool for Spark and we came across one question:
>>>
>>>
>>>
>>> How to enable JMX metrics for YARN deployment?
>>>
>>>
>>>
>>> We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink"
>>>
>>> to file $SPARK_HOME/conf/metrics.properties but it doesn't work.
>>>
>>>
>>>
>>> Everything works in Standalone mode, but not in YARN mode.
>>>
>>>
>>>
>>> Can somebody help?
>>>
>>>
>>>
>>> Thx!
>>>
>>>
>>>
>>> PS: I've found also
>>> https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112
>>> without answer.
>>>
>>>
>>>
>>
>>
>>
>
>
Re: JMXSink for YARN deployment
Posted by Kousuke Saruta <sa...@oss.nttdata.co.jp>.
Hi, Vladimir
You can see about --files option at
https://spark.apache.org/docs/latest/running-on-yarn.html
If you use --files option like "--files /path-to-metrics.properties",
the file is distributed to each executor/driver's working directory, so
they can load the file.
- Kousuke
(2014/09/12 2:16), Vladimir Tretyakov wrote:
> Hi, Kousuke,
>
> Can you please explain a bit detailed what do you mean, I am new in
> Spark, looked at
> https://spark.apache.org/docs/latest/submitting-applications.html
> seems there is no '--files' option.
>
> I just have to add '--files /path-to-metrics.properties' ?
> Undocumented ability?
>
> Thx for answer.
>
>
>
> On Thu, Sep 11, 2014 at 5:55 PM, Kousuke Saruta
> <sarutak@oss.nttdata.co.jp <ma...@oss.nttdata.co.jp>> wrote:
>
> Hi Vladimir
>
> How about use --files option with spark-submit?
>
> - Kousuke
>
>
> (2014/09/11 23:43), Vladimir Tretyakov wrote:
>> Hi again, yeah , I've tried to use ” spark.metrics.conf” before
>> my question in ML, had no luck:(
>> Any other ideas from somebody?
>> Seems nobody use metrics in YARN deployment mode.
>> How about Mesos? I didn't try but maybe Spark has the same
>> difficulties on Mesos?
>>
>> PS: Spark is great thing in general, will be nice to see metrics
>> in YARN/Mesos mode, not only in Standalone:)
>>
>>
>> On Thu, Sep 11, 2014 at 5:25 PM, Shao, Saisai
>> <saisai.shao@intel.com <ma...@intel.com>> wrote:
>>
>> I think you can try to use ”spark.metrics.conf” to manually
>> specify the path of metrics.properties, but the prerequisite
>> is that each container should find this file in their local
>> FS because this file is loaded locally.
>>
>> Besides I think this might be a kind of workaround, a better
>> solution is to fix this by some other solutions.
>>
>> Thanks
>>
>> Jerry
>>
>> *From:*Vladimir Tretyakov
>> [mailto:vladimir.tretyakov@sematext.com
>> <ma...@sematext.com>]
>> *Sent:* Thursday, September 11, 2014 10:08 PM
>> *Cc:* user@spark.apache.org <ma...@spark.apache.org>
>> *Subject:* Re: JMXSink for YARN deployment
>>
>> Hi Shao, thx for explanation, any ideas how to fix it? Where
>> should I put metrics.properties file?
>>
>> On Thu, Sep 11, 2014 at 4:18 PM, Shao, Saisai
>> <saisai.shao@intel.com <ma...@intel.com>> wrote:
>>
>> Hi，
>>
>> I’m guessing the problem is that driver or executor cannot
>> get the metrics.properties configuration file in the yarn
>> container, so metrics system cannot load the right sinks.
>>
>> Thanks
>>
>> Jerry
>>
>> *From:*Vladimir Tretyakov
>> [mailto:vladimir.tretyakov@sematext.com
>> <ma...@sematext.com>]
>> *Sent:* Thursday, September 11, 2014 7:30 PM
>> *To:* user@spark.apache.org <ma...@spark.apache.org>
>> *Subject:* JMXSink for YARN deployment
>>
>> Hello, we are in Sematext (https://apps.sematext.com/) are
>> writing Monitoring tool for Spark and we came across one
>> question:
>>
>> How to enable JMX metrics for YARN deployment?
>>
>> We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink"
>>
>> to file $SPARK_HOME/conf/metrics.properties but it doesn't work.
>>
>> Everything works in Standalone mode, but not in YARN mode.
>>
>> Can somebody help?
>>
>> Thx!
>>
>> PS: I've found also
>> https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112
>> without answer.
>>
>>
>
>
Re: JMXSink for YARN deployment
Posted by Vladimir Tretyakov <vl...@sematext.com>.
Hi, Kousuke,
Can you please explain a bit detailed what do you mean, I am new in Spark,
looked at https://spark.apache.org/docs/latest/submitting-applications.html
seems there is no '--files' option.
I just have to add '--files /path-to-metrics.properties' ? Undocumented
ability?
Thx for answer.
On Thu, Sep 11, 2014 at 5:55 PM, Kousuke Saruta <sa...@oss.nttdata.co.jp>
wrote:
> Hi Vladimir
>
> How about use --files option with spark-submit?
>
> - Kousuke
>
>
> (2014/09/11 23:43), Vladimir Tretyakov wrote:
>
> Hi again, yeah , I've tried to use ” spark.metrics.conf” before my
> question in ML, had no luck:(
> Any other ideas from somebody?
> Seems nobody use metrics in YARN deployment mode.
> How about Mesos? I didn't try but maybe Spark has the same difficulties on
> Mesos?
>
> PS: Spark is great thing in general, will be nice to see metrics in
> YARN/Mesos mode, not only in Standalone:)
>
>
> On Thu, Sep 11, 2014 at 5:25 PM, Shao, Saisai <sa...@intel.com>
> wrote:
>
>> I think you can try to use ” spark.metrics.conf” to manually specify
>> the path of metrics.properties, but the prerequisite is that each container
>> should find this file in their local FS because this file is loaded locally.
>>
>>
>>
>> Besides I think this might be a kind of workaround, a better solution is
>> to fix this by some other solutions.
>>
>>
>>
>> Thanks
>>
>> Jerry
>>
>>
>>
>> *From:* Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com]
>> *Sent:* Thursday, September 11, 2014 10:08 PM
>> *Cc:* user@spark.apache.org
>> *Subject:* Re: JMXSink for YARN deployment
>>
>>
>>
>> Hi Shao, thx for explanation, any ideas how to fix it? Where should I put
>> metrics.properties file?
>>
>>
>>
>> On Thu, Sep 11, 2014 at 4:18 PM, Shao, Saisai <sa...@intel.com>
>> wrote:
>>
>> Hi，
>>
>>
>>
>> I’m guessing the problem is that driver or executor cannot get the
>> metrics.properties configuration file in the yarn container, so metrics
>> system cannot load the right sinks.
>>
>>
>>
>> Thanks
>>
>> Jerry
>>
>>
>>
>> *From:* Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com]
>> *Sent:* Thursday, September 11, 2014 7:30 PM
>> *To:* user@spark.apache.org
>> *Subject:* JMXSink for YARN deployment
>>
>>
>>
>> Hello, we are in Sematext (https://apps.sematext.com/) are writing
>> Monitoring tool for Spark and we came across one question:
>>
>>
>>
>> How to enable JMX metrics for YARN deployment?
>>
>>
>>
>> We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink"
>>
>> to file $SPARK_HOME/conf/metrics.properties but it doesn't work.
>>
>>
>>
>> Everything works in Standalone mode, but not in YARN mode.
>>
>>
>>
>> Can somebody help?
>>
>>
>>
>> Thx!
>>
>>
>>
>> PS: I've found also
>> https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112
>> without answer.
>>
>>
>>
>
>
>
Re: JMXSink for YARN deployment
Posted by Kousuke Saruta <sa...@oss.nttdata.co.jp>.
Hi Vladimir
How about use --files option with spark-submit?
- Kousuke
(2014/09/11 23:43), Vladimir Tretyakov wrote:
> Hi again, yeah , I've tried to use ” spark.metrics.conf” before my
> question in ML, had no luck:(
> Any other ideas from somebody?
> Seems nobody use metrics in YARN deployment mode.
> How about Mesos? I didn't try but maybe Spark has the same
> difficulties on Mesos?
>
> PS: Spark is great thing in general, will be nice to see metrics in
> YARN/Mesos mode, not only in Standalone:)
>
>
> On Thu, Sep 11, 2014 at 5:25 PM, Shao, Saisai <saisai.shao@intel.com
> <ma...@intel.com>> wrote:
>
> I think you can try to use ”spark.metrics.conf” to manually
> specify the path of metrics.properties, but the prerequisite is
> that each container should find this file in their local FS
> because this file is loaded locally.
>
> Besides I think this might be a kind of workaround, a better
> solution is to fix this by some other solutions.
>
> Thanks
>
> Jerry
>
> *From:*Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com
> <ma...@sematext.com>]
> *Sent:* Thursday, September 11, 2014 10:08 PM
> *Cc:* user@spark.apache.org <ma...@spark.apache.org>
> *Subject:* Re: JMXSink for YARN deployment
>
> Hi Shao, thx for explanation, any ideas how to fix it? Where
> should I put metrics.properties file?
>
> On Thu, Sep 11, 2014 at 4:18 PM, Shao, Saisai
> <saisai.shao@intel.com <ma...@intel.com>> wrote:
>
> Hi，
>
> I’m guessing the problem is that driver or executor cannot get the
> metrics.properties configuration file in the yarn container, so
> metrics system cannot load the right sinks.
>
> Thanks
>
> Jerry
>
> *From:*Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com
> <ma...@sematext.com>]
> *Sent:* Thursday, September 11, 2014 7:30 PM
> *To:* user@spark.apache.org <ma...@spark.apache.org>
> *Subject:* JMXSink for YARN deployment
>
> Hello, we are in Sematext (https://apps.sematext.com/) are writing
> Monitoring tool for Spark and we came across one question:
>
> How to enable JMX metrics for YARN deployment?
>
> We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink"
>
> to file $SPARK_HOME/conf/metrics.properties but it doesn't work.
>
> Everything works in Standalone mode, but not in YARN mode.
>
> Can somebody help?
>
> Thx!
>
> PS: I've found also
> https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112
> without answer.
>
>
Re: JMXSink for YARN deployment
Posted by Vladimir Tretyakov <vl...@sematext.com>.
Hi again, yeah , I've tried to use ” spark.metrics.conf” before my question
in ML, had no luck:(
Any other ideas from somebody?
Seems nobody use metrics in YARN deployment mode.
How about Mesos? I didn't try but maybe Spark has the same difficulties on
Mesos?
PS: Spark is great thing in general, will be nice to see metrics in
YARN/Mesos mode, not only in Standalone:)
On Thu, Sep 11, 2014 at 5:25 PM, Shao, Saisai <sa...@intel.com> wrote:
> I think you can try to use ” spark.metrics.conf” to manually specify the
> path of metrics.properties, but the prerequisite is that each container
> should find this file in their local FS because this file is loaded locally.
>
>
>
> Besides I think this might be a kind of workaround, a better solution is
> to fix this by some other solutions.
>
>
>
> Thanks
>
> Jerry
>
>
>
> *From:* Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com]
> *Sent:* Thursday, September 11, 2014 10:08 PM
> *Cc:* user@spark.apache.org
> *Subject:* Re: JMXSink for YARN deployment
>
>
>
> Hi Shao, thx for explanation, any ideas how to fix it? Where should I put
> metrics.properties file?
>
>
>
> On Thu, Sep 11, 2014 at 4:18 PM, Shao, Saisai <sa...@intel.com>
> wrote:
>
> Hi，
>
>
>
> I’m guessing the problem is that driver or executor cannot get the
> metrics.properties configuration file in the yarn container, so metrics
> system cannot load the right sinks.
>
>
>
> Thanks
>
> Jerry
>
>
>
> *From:* Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com]
> *Sent:* Thursday, September 11, 2014 7:30 PM
> *To:* user@spark.apache.org
> *Subject:* JMXSink for YARN deployment
>
>
>
> Hello, we are in Sematext (https://apps.sematext.com/) are writing
> Monitoring tool for Spark and we came across one question:
>
>
>
> How to enable JMX metrics for YARN deployment?
>
>
>
> We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink"
>
> to file $SPARK_HOME/conf/metrics.properties but it doesn't work.
>
>
>
> Everything works in Standalone mode, but not in YARN mode.
>
>
>
> Can somebody help?
>
>
>
> Thx!
>
>
>
> PS: I've found also
> https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112
> without answer.
>
>
>
RE: JMXSink for YARN deployment
Posted by "Shao, Saisai" <sa...@intel.com>.
I think you can try to use ” spark.metrics.conf” to manually specify the path of metrics.properties, but the prerequisite is that each container should find this file in their local FS because this file is loaded locally.
Besides I think this might be a kind of workaround, a better solution is to fix this by some other solutions.
Thanks
Jerry
From: Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com]
Sent: Thursday, September 11, 2014 10:08 PM
Cc: user@spark.apache.org
Subject: Re: JMXSink for YARN deployment
Hi Shao, thx for explanation, any ideas how to fix it? Where should I put metrics.properties file?
On Thu, Sep 11, 2014 at 4:18 PM, Shao, Saisai <sa...@intel.com>> wrote:
Hi，
I’m guessing the problem is that driver or executor cannot get the metrics.properties configuration file in the yarn container, so metrics system cannot load the right sinks.
Thanks
Jerry
From: Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com<ma...@sematext.com>]
Sent: Thursday, September 11, 2014 7:30 PM
To: user@spark.apache.org<ma...@spark.apache.org>
Subject: JMXSink for YARN deployment
Hello, we are in Sematext (https://apps.sematext.com/) are writing Monitoring tool for Spark and we came across one question:
How to enable JMX metrics for YARN deployment?
We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink"
to file $SPARK_HOME/conf/metrics.properties but it doesn't work.
Everything works in Standalone mode, but not in YARN mode.
Can somebody help?
Thx!
PS: I've found also https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112 without answer.
Re: JMXSink for YARN deployment
Posted by Vladimir Tretyakov <vl...@sematext.com>.
Hi Shao, thx for explanation, any ideas how to fix it? Where should I put
metrics.properties file?
On Thu, Sep 11, 2014 at 4:18 PM, Shao, Saisai <sa...@intel.com> wrote:
> Hi，
>
>
>
> I’m guessing the problem is that driver or executor cannot get the
> metrics.properties configuration file in the yarn container, so metrics
> system cannot load the right sinks.
>
>
>
> Thanks
>
> Jerry
>
>
>
> *From:* Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com]
> *Sent:* Thursday, September 11, 2014 7:30 PM
> *To:* user@spark.apache.org
> *Subject:* JMXSink for YARN deployment
>
>
>
> Hello, we are in Sematext (https://apps.sematext.com/) are writing
> Monitoring tool for Spark and we came across one question:
>
>
>
> How to enable JMX metrics for YARN deployment?
>
>
>
> We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink"
>
> to file $SPARK_HOME/conf/metrics.properties but it doesn't work.
>
>
>
> Everything works in Standalone mode, but not in YARN mode.
>
>
>
> Can somebody help?
>
>
>
> Thx!
>
>
>
> PS: I've found also
> https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112
> without answer.
>
Re: JMXSink for YARN deployment
Posted by spearson23 <sp...@gmail.com>.
We use a metrics.property file on YARN by submitting applications like this:
spark-submit --conf spark.metrics.conf=metrics.properties --class CLASS_NAME
--master yarn-cluster --files /PATH/TO/metrics.properties /PATH/TO/CODE.JAR
/PATH/TO/CONFIG.FILE APP_NAME
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/JMXSink-for-YARN-deployment-tp13958p25570.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: JMXSink for YARN deployment
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi,
Jerry said "I'm guessing", so maybe the thing to try is to check if his
guess is correct.
What about running sudo lsof | grep metrics.properties ? I imagine you
should be able to see it if the file was found and read. If Jerry is
right, then I think you will NOT see it.
Next, how about trying some bogus value in metrics.properties, like *.sink.
jmx.class=org.apache.spark.metrics.sink.*BUGUSSink*? If the file is being
read then specifying such bogus value should make something log an error or
throw exception at start, I assume. If you don't see this then maybe this
file is not being read at all.
Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/
On Thu, Sep 11, 2014 at 9:18 AM, Shao, Saisai <sa...@intel.com> wrote:
> Hi，
>
>
>
> I’m guessing the problem is that driver or executor cannot get the
> metrics.properties configuration file in the yarn container, so metrics
> system cannot load the right sinks.
>
>
>
> Thanks
>
> Jerry
>
>
>
> *From:* Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com]
> *Sent:* Thursday, September 11, 2014 7:30 PM
> *To:* user@spark.apache.org
> *Subject:* JMXSink for YARN deployment
>
>
>
> Hello, we are in Sematext (https://apps.sematext.com/) are writing
> Monitoring tool for Spark and we came across one question:
>
>
>
> How to enable JMX metrics for YARN deployment?
>
>
>
> We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink"
>
> to file $SPARK_HOME/conf/metrics.properties but it doesn't work.
>
>
>
> Everything works in Standalone mode, but not in YARN mode.
>
>
>
> Can somebody help?
>
>
>
> Thx!
>
>
>
> PS: I've found also
> https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112
> without answer.
>
RE: JMXSink for YARN deployment
Posted by "Shao, Saisai" <sa...@intel.com>.
Hi，
I’m guessing the problem is that driver or executor cannot get the metrics.properties configuration file in the yarn container, so metrics system cannot load the right sinks.
Thanks
Jerry
From: Vladimir Tretyakov [mailto:vladimir.tretyakov@sematext.com]
Sent: Thursday, September 11, 2014 7:30 PM
To: user@spark.apache.org
Subject: JMXSink for YARN deployment
Hello, we are in Sematext (https://apps.sematext.com/) are writing Monitoring tool for Spark and we came across one question:
How to enable JMX metrics for YARN deployment?
We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink"
to file $SPARK_HOME/conf/metrics.properties but it doesn't work.
Everything works in Standalone mode, but not in YARN mode.
Can somebody help?
Thx!
PS: I've found also https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112 without answer.