You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by Robert Metzger <rm...@apache.org> on 2015/07/07 16:36:00 UTC
Unable to use storm-slider
Hi,
I used this tutorial
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.2.0/YARN_RM_v22/running_applications_on_slider/index.html#Item1.1
to start Storm on YARN using Slider.
Everything went well in the tutorial, until I wanted to execute:
storm-slider --app <yarn-app-name> quicklinks
I got the following output:
$ storm-slider --app storm-test1 quicklinks
2015-07-07 14:15:22,766 [main] INFO impl.TimelineClientImpl - Timeline
service address:
http://robert-streaming-w-1.c.XXX.internal:8188/ws/v1/timeline/
2015-07-07 14:15:23,422 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot
be loaded.
2015-07-07 14:15:23,431 [main] INFO client.RMProxy - Connecting to
ResourceManager at robert-streaming-w-1.c.XXX.internal/10.XXX.220:8050
2015-07-07 14:15:23,571 [main] INFO imps.CuratorFrameworkImpl - Starting
2015-07-07 14:15:23,616 [main-EventThread] INFO
state.ConnectionStateManager - State change: CONNECTED
2015-07-07 14:15:23,616 [ConnectionStateManager-0] WARN
state.ConnectionStateManager - There are no ConnectionStateListeners
registered.
2015-07-07 14:15:24,892 [main] INFO client.SliderClient - {}
java.io.FileNotFoundException: Unknown configuration storm-site
at
org.apache.slider.core.registry.retrieve.RegistryRetriever.retrieveConfiguration(RegistryRetriever.java:282)
at
org.apache.slider.client.SliderClient.actionRegistryGetConfig(SliderClient.java:3119)
at
org.apache.slider.client.SliderClient.actionRegistry(SliderClient.java:2610)
at org.apache.slider.client.SliderClient.exec(SliderClient.java:404)
at org.apache.slider.client.SliderClient.runService(SliderClient.java:348)
at
org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:188)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:475)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:403)
at
org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.java:630)
at org.apache.slider.Slider.main(Slider.java:49)
2015-07-07 14:15:24,896 [main] INFO util.ExitUtil - Exiting with status 44
Failed to read slider deployed storm config
Submitting a job with:
storm-slider --app storm-test1 jar storm-jobs-0.1-SNAPSHOT.jar
also fails with the same error.
There was another user asking a similar question on this list. The
suggestion there was to enable the YARN application registry.
In my YARN configuration, I have:
<property><name>hadoop.registry.rm.enabled</name><value>true</value><source>java.io.BufferedInputStream@41ccf24c
</source></property>
and
<property><name>hadoop.registry.zk.quorum</name><value>robert-streaming-m.c.XXXX.internal:2181,robert-streaming-w-0.c.XXXX.internal:2181,robert-streaming-w-1.c.XXX.internal:2181</value><source>java.io.BufferedInputStream@41ccf24c
</source></property>
I'm not sure whether I set the "site.global.app_user" correctly. I put
"robert" in there. I used that user as well to deploy the slider app.
How can I resolve this issue?
Re: Unable to use storm-slider
Posted by Gour Saha <gs...@hortonworks.com>.
Is the Storm app running? Can you check the logs?
If this is not a secured cluster, you need to set app_user to yarn and
submit the job as yarn user as well.
-Gour
On 7/7/15, 7:36 AM, "Robert Metzger" <rm...@apache.org> wrote:
>Hi,
>
>I used this tutorial
>http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.2.0/YARN_RM_v22/runnin
>g_applications_on_slider/index.html#Item1.1
>
>to start Storm on YARN using Slider.
>
>Everything went well in the tutorial, until I wanted to execute:
>
>storm-slider --app <yarn-app-name> quicklinks
>
>I got the following output:
>
>$ storm-slider --app storm-test1 quicklinks
>2015-07-07 14:15:22,766 [main] INFO impl.TimelineClientImpl - Timeline
>service address:
>http://robert-streaming-w-1.c.XXX.internal:8188/ws/v1/timeline/
>2015-07-07 14:15:23,422 [main] WARN shortcircuit.DomainSocketFactory -
>The
>short-circuit local reads feature cannot be used because libhadoop cannot
>be loaded.
>2015-07-07 14:15:23,431 [main] INFO client.RMProxy - Connecting to
>ResourceManager at robert-streaming-w-1.c.XXX.internal/10.XXX.220:8050
>2015-07-07 14:15:23,571 [main] INFO imps.CuratorFrameworkImpl - Starting
>2015-07-07 14:15:23,616 [main-EventThread] INFO
> state.ConnectionStateManager - State change: CONNECTED
>2015-07-07 14:15:23,616 [ConnectionStateManager-0] WARN
> state.ConnectionStateManager - There are no ConnectionStateListeners
>registered.
>2015-07-07 14:15:24,892 [main] INFO client.SliderClient - {}
>java.io.FileNotFoundException: Unknown configuration storm-site
>at
>org.apache.slider.core.registry.retrieve.RegistryRetriever.retrieveConfigu
>ration(RegistryRetriever.java:282)
>at
>org.apache.slider.client.SliderClient.actionRegistryGetConfig(SliderClient
>.java:3119)
>at
>org.apache.slider.client.SliderClient.actionRegistry(SliderClient.java:261
>0)
>at org.apache.slider.client.SliderClient.exec(SliderClient.java:404)
>at org.apache.slider.client.SliderClient.runService(SliderClient.java:348)
>at
>org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.
>java:188)
>at
>org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceL
>auncher.java:475)
>at
>org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLa
>uncher.java:403)
>at
>org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.ja
>va:630)
>at org.apache.slider.Slider.main(Slider.java:49)
>2015-07-07 14:15:24,896 [main] INFO util.ExitUtil - Exiting with status
>44
>Failed to read slider deployed storm config
>
>
>Submitting a job with:
>storm-slider --app storm-test1 jar storm-jobs-0.1-SNAPSHOT.jar
>
>also fails with the same error.
>
>There was another user asking a similar question on this list. The
>suggestion there was to enable the YARN application registry.
>In my YARN configuration, I have:
>
><property><name>hadoop.registry.rm.enabled</name><value>true</value><sourc
>e>java.io.BufferedInputStream@41ccf24c
></source></property>
>
>and
>
><property><name>hadoop.registry.zk.quorum</name><value>robert-streaming-m.
>c.XXXX.internal:2181,robert-streaming-w-0.c.XXXX.internal:2181,robert-stre
>aming-w-1.c.XXX.internal:2181</value><source>java.io.BufferedInputStream@4
>1ccf24c
></source></property>
>
>I'm not sure whether I set the "site.global.app_user" correctly. I put
>"robert" in there. I used that user as well to deploy the slider app.
>
>How can I resolve this issue?