You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by Foolish Ewe <fo...@hotmail.com> on 2017/04/19 01:35:44 UTC
Trying to get he memcached example to run for a new build
Hello All:
I'm trying to run the memcached example locally but it fails pretty quick and claims "protobuf-java-2.5.0.jar does not exist".
Consider the following script (test-slider.sh):
#!/bin/bash
# this does not work when run during docker-build, we need to do this once we are logged in
echo "memcached built, registering memcached with slider and then running it"
/usr/lib/slider/bin/slider install-package --package /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip --name jmemcached --debug
echo "Listing applications - before starting jmemcached"
/usr/lib/slider/bin/slider list --manager localhost:8032
# startup memcached properly, yarn task name will be mymemcached
/usr/lib/slider/bin/slider create mymemcached --template /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json --resources /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json --manager localhost:8032 --debug
echo "Listing applications - after starting jmemcached"
/usr/lib/slider/bin/slider list --manager localhost:8032
echo "Finished $0"
I am having difficulty in locating the logs after the failure, any idea where they would be? yarn logs cannot find them, and I'm not having luck with the name node. I did modify the yarn-site.xml read at startup to include the lines:
<!-- Begin modifications for debugging slider -->
<!-- 60 minutes after a failure to see what is left in the directory-->
<property>
<name>yarn.nodemanager.delete.debug-delay-sec</name>
<value>3600</value>
</property>
<!--time before the process gets a -9 (Should it be 30 seconds?)-->
<property>
<name>yarn.nodemanager.sleep-delay-before-sigkill.ms</name>
<value>3600000</value>
</property>
<!-- End modifications for debugging slider -->
Regarding the error, I'm seeing, the slider install-package appears to work correctly from what I can tell, but the slider create encounters run time errors when launched, it seems to be looking for protobuf-java-2.5.0.jar. I tried making fat jars for the memcached application and the various slider packages, to no avail. Any ideas how to resolve these errors reported in the resource manager:
Application application_1492557996590_0002 failed 2 times due to AM Container for appattempt_1492557996590_0002_000002 exited with exitCode: -1000
For more detailed output, check application tracking page:http://quickstart.cloudera:8088/proxy/application_1492557996590_0002/Then, click on links to logs of each attempt.
Diagnostics: File file:/root/.slider/cluster/mymemcached/tmp/application_1492557996590_0002/am/lib/protobuf-java-2.5.0.jar does not exist
java.io.FileNotFoundException: File file:/root/.slider/cluster/mymemcached/tmp/application_1492557996590_0002/am/lib/protobuf-java-2.5.0.jar does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:542)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:755)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:532)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:425)
at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Failing this attempt. Failing the application.
The test-slider.sh script's output is as follows:
# bash -x /tmp/mybuild/test_slider.sh
+ echo 'memcached built, registering memcached with slider and then running it'
memcached built, registering memcached with slider and then running it
+ /usr/lib/slider/bin/slider install-package --package /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip --name jmemcached --debug
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-19 01:10:44,080 [main] INFO client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032
2017-04-19 01:10:44,243 [main] ERROR main.ServiceLauncher - Unable to access supplied pkg file at /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip
2017-04-19 01:10:44,247 [main] INFO util.ExitUtil - Exiting with status 40
+ echo 'Listing applications - before starting jmemcached'
Listing applications - before starting jmemcached
+ /usr/lib/slider/bin/slider list --manager localhost:8032
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-19 01:10:46,066 [main] INFO client.RMProxy - Connecting to ResourceManager at localhost/127.0.0.1:8032
2017-04-19 01:10:46,217 [main] INFO util.ExitUtil - Exiting with status 0
+ /usr/lib/slider/bin/slider create mymemcached --template /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json --resources /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json --manager localhost:8032 --debug
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-19 01:10:48,070 [main] INFO client.RMProxy - Connecting to ResourceManager at localhost/127.0.0.1:8032
2017-04-19 01:10:48,647 [main] INFO agent.AgentClientProvider - Validating app definition /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-19 01:10:48,648 [main] INFO agent.AgentUtils - Reading metainfo at /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-19 01:10:48,675 [main] INFO tools.SliderUtils - Reading metainfo.xml of size 2202
2017-04-19 01:10:48,893 [main] INFO client.SliderClient - No credentials requested
2017-04-19 01:10:48,938 [main] INFO agent.AgentUtils - Reading metainfo at /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-19 01:10:48,939 [main] INFO tools.SliderUtils - Reading metainfo.xml of size 2202
2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log include patterns:
2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log exclude patterns:
2017-04-19 01:10:49,861 [main] INFO slideram.SliderAMClientProvider - Loading all dependencies for AM.
2017-04-19 01:10:49,862 [main] INFO tools.SliderUtils - Loading all dependencies from /usr/lib/slider/lib
2017-04-19 01:10:51,324 [main] INFO agent.AgentClientProvider - Automatically uploading the agent tarball at file:/root/.slider/cluster/mymemcached/tmp/application_1492557996590_0003/agent
2017-04-19 01:10:51,361 [main] INFO agent.AgentClientProvider - Validating app definition /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-19 01:10:51,361 [main] INFO agent.AgentUtils - Reading metainfo at /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-19 01:10:51,364 [main] INFO tools.SliderUtils - Reading metainfo.xml of size 2202
2017-04-19 01:10:51,429 [main] INFO Configuration.deprecation - slider.registry.path is deprecated. Instead, use hadoop.registry.zk.root
2017-04-19 01:10:51,436 [main] INFO launch.AppMasterLauncher - Submitting application to Resource Manager
2017-04-19 01:10:51,480 [main] INFO impl.YarnClientImpl - Submitted application application_1492557996590_0003
2017-04-19 01:10:51,484 [main] INFO util.ExitUtil - Exiting with status 0
+ echo 'Listing applications - after starting jmemcached'
Listing applications - after starting jmemcached
+ /usr/lib/slider/bin/slider list --manager localhost:8032
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-19 01:10:53,893 [main] INFO client.RMProxy - Connecting to ResourceManager at localhost/127.0.0.1:8032
mymemcached ACCEPTED application_1492557996590_0003
2017-04-19 01:10:54,186 [main] INFO util.ExitUtil - Exiting with status 0
+ echo 'Finished /tmp/mybuild/test_slider.sh'
Finished /tmp/mybuild/test_slider.sh
Thanks:
Bill
Re: Trying to get he memcached example to run for a new build
Posted by Foolish Ewe <fo...@hotmail.com>.
Hi All:
Thanks to Billie Rinaldi's help and some code diving, I've been able to get slider to advance the jmemcached task to the running state. However, I don't think jmemcached is actually running (e.g. the port 11211 is not found by netstat or lsof).
I suspect something has gone awry during the launching of the container in slider, and I could use a bit of guidance regarding how to track this down.
If I understand correctly, slider is launching 2 containers, an application manager and the one that is supposed to run jmemcached. I think the jmemcached container is failing and relaunching frequently under a different container ID, and that the container id suffix is climbing (e.g. initially I think it was container_1493146167513_0001_01_000008 when I first checked, now the container name suffix ends in 0155. In any case, I'm trying to understand what is happening, looking at:
http://quickstart.cloudera:8042/node/containerlogs/container_1493146167513_0001_01_000008/root/slider-agent.log/?start=0, I see that the container names
INFO 2017-04-25 19:31:51,440 main.py:85 - loglevel=logging.INFO
INFO 2017-04-25 19:31:51,440 main.py:96 - Newloglevel=logging.DEBUG
INFO 2017-04-25 19:31:51,440 main.py:242 - Using AGENT_WORK_ROOT = /var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/root/appcache/application_1493146167513_0001/container_1493146167513_0001_01_000008
INFO 2017-04-25 19:31:51,441 main.py:243 - Using AGENT_LOG_ROOT = /var/log/hadoop-yarn/containers/application_1493146167513_0001/container_1493146167513_0001_01_000008
[Omitting logs for brevity, Note that the container name below might be getting corrupted to now read container_1493146167513_0001_01_000008___MEMCACHED]
INFO 2017-04-25 19:31:51,478 NetUtil.py:67 - DEBUG: Trying to connect to the server at https://quickstart.cloudera:34593/ws/v1/slider/agents/
INFO 2017-04-25 19:31:51,478 NetUtil.py:38 - Connecting to the following url https://quickstart.cloudera:34593/ws/v1/slider/agents/
INFO 2017-04-25 19:31:51,536 NetUtil.py:45 - Calling url received 200
DEBUG 2017-04-25 19:31:51,537 Controller.py:63 - Initializing Controller RPC thread.
INFO 2017-04-25 19:31:51,539 Controller.py:137 - Registering with the server at https://quickstart.cloudera:40959/ws/v1/slider/agents/container_1493146167513_0001_01_000008___MEMCACHED/register with data '{"actualState": 0, "logFolders": {}, "agentVersion": "1", "allocatedPorts": {}, "timestamp": 1493148711538, "expectedState": 0, "tags": "", "responseId": -1, "publicHostname": "quickstart.cloudera", "label": "container_1493146167513_0001_01_000008___MEMCACHED"}'
INFO 2017-04-25 19:31:51,539 security.py:89 - SSL Connect being called.. connecting to the server
INFO 2017-04-25 19:31:51,604 security.py:51 - SSL connection established. Two-way SSL authentication is turned off on the server.
INFO 2017-04-25 19:31:51,607 Controller.py:180 - Unable to connect to: https://quickstart.cloudera:40959/ws/v1/slider/agents/container_1493146167513_0001_01_000008___MEMCACHED/register
Traceback (most recent call last):
File "/var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/root/appcache/application_1493146167513_0001/filecache/21/slider-agent.tar.gz/slider-agent/agent/Controller.py", line 139, in registerWithServer
regResp = json.loads(response)
File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Any idea how I can better debug this and analyze this?
With best regards:
Bill
________________________________
From: Foolish Ewe <fo...@hotmail.com>
Sent: Wednesday, April 19, 2017 1:35 AM
To: dev@slider.incubator.apache.org
Subject: Trying to get he memcached example to run for a new build
Hello All:
I'm trying to run the memcached example locally but it fails pretty quick and claims "protobuf-java-2.5.0.jar does not exist".
Consider the following script (test-slider.sh):
#!/bin/bash
# this does not work when run during docker-build, we need to do this once we are logged in
echo "memcached built, registering memcached with slider and then running it"
/usr/lib/slider/bin/slider install-package --package /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip --name jmemcached --debug
echo "Listing applications - before starting jmemcached"
/usr/lib/slider/bin/slider list --manager localhost:8032
# startup memcached properly, yarn task name will be mymemcached
/usr/lib/slider/bin/slider create mymemcached --template /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json --resources /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json --manager localhost:8032 --debug
echo "Listing applications - after starting jmemcached"
/usr/lib/slider/bin/slider list --manager localhost:8032
echo "Finished $0"
I am having difficulty in locating the logs after the failure, any idea where they would be? yarn logs cannot find them, and I'm not having luck with the name node. I did modify the yarn-site.xml read at startup to include the lines:
<!-- Begin modifications for debugging slider -->
<!-- 60 minutes after a failure to see what is left in the directory-->
<property>
<name>yarn.nodemanager.delete.debug-delay-sec</name>
<value>3600</value>
</property>
<!--time before the process gets a -9 (Should it be 30 seconds?)-->
<property>
<name>yarn.nodemanager.sleep-delay-before-sigkill.ms</name>
<value>3600000</value>
</property>
<!-- End modifications for debugging slider -->
Regarding the error, I'm seeing, the slider install-package appears to work correctly from what I can tell, but the slider create encounters run time errors when launched, it seems to be looking for protobuf-java-2.5.0.jar. I tried making fat jars for the memcached application and the various slider packages, to no avail. Any ideas how to resolve these errors reported in the resource manager:
Application application_1492557996590_0002 failed 2 times due to AM Container for appattempt_1492557996590_0002_000002 exited with exitCode: -1000
For more detailed output, check application tracking page:http://quickstart.cloudera:8088/proxy/application_1492557996590_0002/Then, click on links to logs of each attempt.
Diagnostics: File file:/root/.slider/cluster/mymemcached/tmp/application_1492557996590_0002/am/lib/protobuf-java-2.5.0.jar does not exist
java.io.FileNotFoundException: File file:/root/.slider/cluster/mymemcached/tmp/application_1492557996590_0002/am/lib/protobuf-java-2.5.0.jar does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:542)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:755)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:532)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:425)
at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Failing this attempt. Failing the application.
The test-slider.sh script's output is as follows:
# bash -x /tmp/mybuild/test_slider.sh
+ echo 'memcached built, registering memcached with slider and then running it'
memcached built, registering memcached with slider and then running it
+ /usr/lib/slider/bin/slider install-package --package /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip --name jmemcached --debug
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
www.slf4j.org
SLF4J warning or error messages and their meanings No SLF4J providers were found. This warning, i.e. not an error, message is reported when no SLF4J ...
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-19 01:10:44,080 [main] INFO client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032
2017-04-19 01:10:44,243 [main] ERROR main.ServiceLauncher - Unable to access supplied pkg file at /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip
2017-04-19 01:10:44,247 [main] INFO util.ExitUtil - Exiting with status 40
+ echo 'Listing applications - before starting jmemcached'
Listing applications - before starting jmemcached
+ /usr/lib/slider/bin/slider list --manager localhost:8032
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
www.slf4j.org
SLF4J warning or error messages and their meanings No SLF4J providers were found. This warning, i.e. not an error, message is reported when no SLF4J ...
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-19 01:10:46,066 [main] INFO client.RMProxy - Connecting to ResourceManager at localhost/127.0.0.1:8032
2017-04-19 01:10:46,217 [main] INFO util.ExitUtil - Exiting with status 0
+ /usr/lib/slider/bin/slider create mymemcached --template /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json --resources /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json --manager localhost:8032 --debug
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
www.slf4j.org
SLF4J warning or error messages and their meanings No SLF4J providers were found. This warning, i.e. not an error, message is reported when no SLF4J ...
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-19 01:10:48,070 [main] INFO client.RMProxy - Connecting to ResourceManager at localhost/127.0.0.1:8032
2017-04-19 01:10:48,647 [main] INFO agent.AgentClientProvider - Validating app definition /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-19 01:10:48,648 [main] INFO agent.AgentUtils - Reading metainfo at /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-19 01:10:48,675 [main] INFO tools.SliderUtils - Reading metainfo.xml of size 2202
2017-04-19 01:10:48,893 [main] INFO client.SliderClient - No credentials requested
2017-04-19 01:10:48,938 [main] INFO agent.AgentUtils - Reading metainfo at /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-19 01:10:48,939 [main] INFO tools.SliderUtils - Reading metainfo.xml of size 2202
2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log include patterns:
2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log exclude patterns:
2017-04-19 01:10:49,861 [main] INFO slideram.SliderAMClientProvider - Loading all dependencies for AM.
2017-04-19 01:10:49,862 [main] INFO tools.SliderUtils - Loading all dependencies from /usr/lib/slider/lib
2017-04-19 01:10:51,324 [main] INFO agent.AgentClientProvider - Automatically uploading the agent tarball at file:/root/.slider/cluster/mymemcached/tmp/application_1492557996590_0003/agent
2017-04-19 01:10:51,361 [main] INFO agent.AgentClientProvider - Validating app definition /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-19 01:10:51,361 [main] INFO agent.AgentUtils - Reading metainfo at /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-19 01:10:51,364 [main] INFO tools.SliderUtils - Reading metainfo.xml of size 2202
2017-04-19 01:10:51,429 [main] INFO Configuration.deprecation - slider.registry.path is deprecated. Instead, use hadoop.registry.zk.root
2017-04-19 01:10:51,436 [main] INFO launch.AppMasterLauncher - Submitting application to Resource Manager
2017-04-19 01:10:51,480 [main] INFO impl.YarnClientImpl - Submitted application application_1492557996590_0003
2017-04-19 01:10:51,484 [main] INFO util.ExitUtil - Exiting with status 0
+ echo 'Listing applications - after starting jmemcached'
Listing applications - after starting jmemcached
+ /usr/lib/slider/bin/slider list --manager localhost:8032
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
www.slf4j.org
SLF4J warning or error messages and their meanings No SLF4J providers were found. This warning, i.e. not an error, message is reported when no SLF4J ...
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-19 01:10:53,893 [main] INFO client.RMProxy - Connecting to ResourceManager at localhost/127.0.0.1:8032
mymemcached ACCEPTED application_1492557996590_0003
2017-04-19 01:10:54,186 [main] INFO util.ExitUtil - Exiting with status 0
+ echo 'Finished /tmp/mybuild/test_slider.sh'
Finished /tmp/mybuild/test_slider.sh
Thanks:
Bill
Re: Trying to get he memcached example to run for a new build
Posted by Billie Rinaldi <bi...@gmail.com>.
The install package command copies a local file to HDFS. After that you
will reference the file's location in HDFS. Your application.def should be
set to ".slider/package/jmemcached/jmemcached-1.0.1.zip" (jmemcached is the
--name you provided). You only need to install the package once, not every
time you run an application.
To get Slider to pick up Hadoop configs, add "export
HADOOP_CONF_DIR=/etc/hadoop/conf" (or whatever the correct path is for your
system) to the slider-env.sh file.
On Fri, Apr 21, 2017 at 12:39 PM, Foolish Ewe <fo...@hotmail.com>
wrote:
> Thanks Billie for the quick and thoughtful reply.
>
>
> You are correct, I'm testing in a single node docker container and have
> slider built from source and installed during the docker build action.
>
>
> Regarding the file systems (in the docker container) I want to be precise
> and make sure that I understand what needs to be in the hdfs and what needs
> to be on the local linux (to the docker container) file system.
>
> I have not had much luck in getting slider to pickup code or configuration
> data from HDFS, is there a setting I need to apply to do this?
>
> Also does slider write any logs? Where would they be?
>
>
> Regarding the protobuf-java-2.5.0.jar, I see it in the local (not HDFS)
> directory: /usr/lib/slider/lib/
>
> # ls /usr/lib/slider/lib/protobuf-java-2.5.0.jar
>
> /usr/lib/slider/lib/protobuf-java-2.5.0.jar
>
> Is this the correct location? Should I install these jars in HDFS, if so
> what is the appropriate hdfs path?
>
>
> Regarding jmemcached-1.0.0.zip, you are correct, I forgot to update the
> jmemcached file name in the script (I updated to 1.0.1 since I built
> memcached for source to create an uber jar using the maven assembly
> plugin), I should have caught that.
>
> The revised test script was updated to reflect this the install-package
> command now has the correct version number in the zip name and now reads:
>
>
> So now the script reads (the commented out lines using hdfs did not work,
> not sure why, even when I change appConfig to use hdfs url paths):
>
> #!/bin/bash
>
> # this does not work when run during docker-build, we need to do this once
> we are logged in
>
> echo "setting up the enviornment variables"
>
> env
>
> export JAVA_HOME="/usr/lib/jvm/java"
>
> export HADOOP_CONF="/etc/hadoop/conf"
>
> env
>
> echo "memcached built, registering memcached with slider and then running
> it"
>
> /usr/lib/slider/bin/slider install-package --package
> /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
> --name jmemcached --debug
>
> #echo "using hdfs version, posting artifacts to hdfs"
>
> #hadoop fs -put ./incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
> ./incubator-slider/app-packages/memcached/*.json
> /user/root/artifacts/memcached
>
> #/usr/lib/slider/bin/slider install-package --package
> hdfs://quickstart.cloudera/user/root/artifacts/memcached/jmemcached-1.0.1.zip
> --name jmemcached --debug
>
> echo "Listing applications - before starting jmemcached"
>
> /usr/lib/slider/bin/slider list --manager localhost:8032
>
> # startup memcached properly, yarn task name will be mymemcached
>
> /usr/lib/slider/bin/slider create mymemcached --template
> /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json
> --resources /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json
> --manager localhost:8032 --debug
>
> #/usr/lib/slider/bin/slider create mymemcached --template
> hdfs://quickstart.cloudera/user/root/artifacts/memcached/appConfig.json
> --resources hdfs://quickstart.cloudera/user/root/artifacts/memcached/resources-default.json
> --manager localhost:8032 --debug
>
> echo "Listing applications - after starting jmemcached"
>
> /usr/lib/slider/bin/slider list --manager localhost:8032
>
> echo "Finished $0"
>
>
> appConfig.json reads:
>
> {
>
> "schema": "http://example.org/specification/v2.0.0",
>
> "metadata": {
>
> },
>
> "global": {
>
> "agent.conf": "/tmp/mybuild/agent/conf/agent.ini",
>
> "application.def": "/tmp/mybuild/incubator-
> slider/app-packages/memcached/jmemcached-1.0.1.zip",
>
> "java_home": "/usr/lib/jvm/java/",
>
> "site.global.additional_cp": "/tmp/mybuild/incubator-
> slider/slider-assembly/target/lib/",
>
> "site.global.xmx_val": "256m",
>
> "site.global.xms_val": "128m",
>
> "site.global.memory_val": "200M",
>
> "site.global.listen_port": "${MEMCACHED.ALLOCATED_PORT}{
> PER_CONTAINER}"
>
> },
>
> "components": {
>
> "slider-appmaster": {
>
> "jvm.heapsize": "256M"
>
> }
>
> }
>
> }
>
>
> /tmp/mybuild/agent/conf/agent.ini reads:
>
>
> app_task_dir=.
>
> app_log_dir=.
>
> app_tmp_dir=app/tmp
>
>
> log_dir=.
>
> run_dir=infra/run
>
> version_file=infra/version
>
>
> ;log_level=INFO
>
> log_level=DEBUG
>
>
> [python]
>
>
> [command]
>
> max_retries=2
>
> sleep_between_retries=1
>
> auto_restart=5,5
>
>
> [security]
>
>
> [heartbeat]
>
> state_interval=6
>
> log_lines_count=300
>
>
>
> And then running the corrected script I see:
>
> + bash -x ./test_slider.sh
>
> + echo 'setting up the enviornment variables'
>
> setting up the enviornment variables
>
> + env
>
> HOSTNAME=quickstart.cloudera
>
> TERM=xterm
>
> LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=
> 01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=
> 37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:
> *.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:
> *.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;
> 31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;
> 31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*
> .lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*
> .tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:
> *.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:
> *.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;
> 31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=
> 01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=
> 01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.
> svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:
> *.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=
> 01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.
> vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.
> rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:
> *.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:
> *.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:
> *.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:
> *.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=
> 01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=
> 01;36:*.spx=01;36:*.xspf=01;36:
>
> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
>
> PWD=/tmp/mybuild
>
> LANG=en_US.UTF-8
>
> SHLVL=3
>
> HOME=/root
>
> no_proxy=*.local, 169.254/16
>
> LESSOPEN=||/usr/bin/lesspipe.sh %s
>
> container=docker
>
> _=/usr/bin/env
>
> + export JAVA_HOME=/usr/lib/jvm/java
>
> + JAVA_HOME=/usr/lib/jvm/java
>
> + export HADOOP_CONF=/etc/hadoop/conf
>
> + HADOOP_CONF=/etc/hadoop/conf
>
> + env
>
> HOSTNAME=quickstart.cloudera
>
> TERM=xterm
>
> HADOOP_CONF=/etc/hadoop/conf
>
> LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=
> 01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=
> 37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:
> *.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:
> *.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;
> 31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;
> 31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*
> .lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*
> .tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:
> *.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:
> *.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;
> 31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=
> 01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=
> 01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.
> svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:
> *.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=
> 01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.
> vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.
> rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:
> *.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:
> *.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:
> *.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:
> *.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=
> 01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=
> 01;36:*.spx=01;36:*.xspf=01;36:
>
> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
>
> PWD=/tmp/mybuild
>
> JAVA_HOME=/usr/lib/jvm/java
>
> LANG=en_US.UTF-8
>
> SHLVL=3
>
> HOME=/root
>
> no_proxy=*.local, 169.254/16
>
> LESSOPEN=||/usr/bin/lesspipe.sh %s
>
> container=docker
>
> _=/usr/bin/env
>
> + echo 'memcached built, registering memcached with slider and then
> running it'
>
> memcached built, registering memcached with slider and then running it
>
> + /usr/lib/slider/bin/slider install-package --package
> /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
> --name jmemcached --debug --replacepkg
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-21 19:22:12,948 [main] INFO client.RMProxy - Connecting to
> ResourceManager at /0.0.0.0:8032
>
> 2017-04-21 19:22:13,093 [main] INFO client.SliderClient - Installing
> package file:/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
> at file:/root/.slider/package/jmemcached/jmemcached-1.0.1.zip and
> overwrite is true.
>
> 2017-04-21 19:22:13,109 [main] INFO util.ExitUtil - Exiting with status 0
>
> + echo 'Listing applications - before starting jmemcached'
>
> Listing applications - before starting jmemcached
>
> + /usr/lib/slider/bin/slider list --manager localhost:8032
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-21 19:22:15,070 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
>
> 2017-04-21 19:22:15,222 [main] INFO util.ExitUtil - Exiting with status 0
>
> + /usr/lib/slider/bin/slider create mymemcached --template
> /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json
> --resources /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json
> --manager localhost:8032 --debug
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-21 19:22:17,183 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
>
> 2017-04-21 19:22:17,661 [main] INFO agent.AgentClientProvider -
> Validating app definition /tmp/mybuild/incubator-slider/
> app-packages/memcached/jmemcached-1.0.1.zip
>
> 2017-04-21 19:22:17,662 [main] INFO agent.AgentUtils - Reading metainfo
> at /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.1.zip
>
> 2017-04-21 19:22:17,685 [main] INFO tools.SliderUtils - Reading
> metainfo.xml of size 2202
>
> 2017-04-21 19:22:17,909 [main] INFO client.SliderClient - No credentials
> requested
>
> 2017-04-21 19:22:18,108 [main] INFO agent.AgentUtils - Reading metainfo
> at /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.1.zip
>
> 2017-04-21 19:22:18,116 [main] INFO tools.SliderUtils - Reading
> metainfo.xml of size 2202
>
> 2017-04-21 19:22:18,177 [main] INFO launch.AbstractLauncher - Log include
> patterns:
>
> 2017-04-21 19:22:18,177 [main] INFO launch.AbstractLauncher - Log exclude
> patterns:
>
> 2017-04-21 19:22:18,938 [main] INFO slideram.SliderAMClientProvider -
> Loading all dependencies for AM.
>
> 2017-04-21 19:22:18,939 [main] INFO tools.SliderUtils - Loading all
> dependencies from /usr/lib/slider/lib
>
> 2017-04-21 19:22:20,687 [main] INFO agent.AgentClientProvider -
> Automatically uploading the agent tarball at file:/root/.slider/cluster/
> mymemcached/tmp/application_1492799085220_0003/agent
>
> 2017-04-21 19:22:20,727 [main] INFO agent.AgentClientProvider -
> Validating app definition /tmp/mybuild/incubator-slider/
> app-packages/memcached/jmemcached-1.0.1.zip
>
> 2017-04-21 19:22:20,727 [main] INFO agent.AgentUtils - Reading metainfo
> at /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.1.zip
>
> 2017-04-21 19:22:20,735 [main] INFO tools.SliderUtils - Reading
> metainfo.xml of size 2202
>
> 2017-04-21 19:22:20,807 [main] INFO Configuration.deprecation -
> slider.registry.path is deprecated. Instead, use hadoop.registry.zk.root
>
> 2017-04-21 19:22:20,813 [main] INFO launch.AppMasterLauncher - Submitting
> application to Resource Manager
>
> 2017-04-21 19:22:20,838 [main] INFO impl.YarnClientImpl - Submitted
> application application_1492799085220_0003
>
> 2017-04-21 19:22:20,845 [main] INFO util.ExitUtil - Exiting with status 0
>
> + echo 'Listing applications - after starting jmemcached'
>
> Listing applications - after starting jmemcached
>
> + /usr/lib/slider/bin/slider list --manager localhost:8032
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-21 19:22:22,857 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
>
> mymemcached ACCEPTED application_1492799085220_0003
>
> 2017-04-21 19:22:23,142 [main] INFO util.ExitUtil - Exiting with status 0
>
> + echo 'Finished ./test_slider.sh'
>
> Finished ./test_slider.sh
>
> I am still getting the dreaded failure as reported in the yarn resource
> manager:
> Application application_1492799085220_0003 failed 2 times due to AM
> Container for appattempt_1492799085220_0003_000002 exited with exitCode:
> -1000
> For more detailed output, check application tracking page:
> http://quickstart.cloudera:8088/proxy/application_1492799085220_0003/Then,
> click on links to logs of each attempt.
> Diagnostics: File file:/root/.slider/cluster/mymemcached/tmp/application_
> 1492799085220_0003/am/lib/protobuf-java-2.5.0.jar does not exist
> java.io.FileNotFoundException: File file:/root/.slider/cluster/
> mymemcached/tmp/application_1492799085220_0003/am/lib/protobuf-java-2.5.0.jar
> does not exist
> at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(
> RawLocalFileSystem.java:542)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(
> RawLocalFileSystem.java:755)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(
> RawLocalFileSystem.java:532)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(
> FilterFileSystem.java:425)
> at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251)
> at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1693)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Failing this attempt. Failing the application.
>
>
>
> With best regards:
>
> Bill
>
> ________________________________
> From: Billie Rinaldi <bi...@gmail.com>
> Sent: Wednesday, April 19, 2017 5:36 PM
> To: dev@slider.incubator.apache.org
> Subject: Re: Trying to get he memcached example to run for a new build
>
> The protobuf-java-2.5.0.jar should be in the Slider lib directory. It will
> get pushed out to the AM lib directory from there.
>
> There are some other issues with the script output. The install package
> command has not succeeded, and says "ERROR main.ServiceLauncher - Unable to
> access supplied pkg file at /tmp/mybuild/incubator-slider/
> app-packages/memcached/jmemcached-1.0.0.zip"
>
> Also the line INFO agent.AgentClientProvider - Validating app definition
> /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
> indicates that it is reading the app definition from the local filesystem,
> when it should be reading it from HDFS. Maybe since this is a single VM
> setup it is able to continue despite these errors, but if you were running
> on a real cluster you'd have to make sure install package was working and
> fix the app definition path in the appConfig.
>
> On Tue, Apr 18, 2017 at 6:35 PM, Foolish Ewe <fo...@hotmail.com>
> wrote:
>
> > Hello All:
> >
> >
> > I'm trying to run the memcached example locally but it fails pretty quick
> > and claims "protobuf-java-2.5.0.jar does not exist".
> >
> >
> > Consider the following script (test-slider.sh):
> >
> > #!/bin/bash
> >
> > # this does not work when run during docker-build, we need to do this
> once
> > we are logged in
> >
> > echo "memcached built, registering memcached with slider and then running
> > it"
> >
> > /usr/lib/slider/bin/slider install-package --package
> > /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.0.zip
> > --name jmemcached --debug
> >
> > echo "Listing applications - before starting jmemcached"
> >
> > /usr/lib/slider/bin/slider list --manager localhost:8032
> >
> > # startup memcached properly, yarn task name will be mymemcached
> >
> > /usr/lib/slider/bin/slider create mymemcached --template
> > /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json
> > --resources /tmp/mybuild/incubator-slider/app-packages/memcached/
> resources-default.json
> > --manager localhost:8032 --debug
> >
> > echo "Listing applications - after starting jmemcached"
> >
> > /usr/lib/slider/bin/slider list --manager localhost:8032
> >
> > echo "Finished $0"
> >
> > I am having difficulty in locating the logs after the failure, any idea
> > where they would be? yarn logs cannot find them, and I'm not having luck
> > with the name node. I did modify the yarn-site.xml read at startup to
> > include the lines:
> >
> > <!-- Begin modifications for debugging slider -->
> >
> > <!-- 60 minutes after a failure to see what is left in the directory-->
> >
> > <property>
> >
> > <name>yarn.nodemanager.delete.debug-delay-sec</name>
> >
> > <value>3600</value>
> >
> > </property>
> >
> >
> > <!--time before the process gets a -9 (Should it be 30 seconds?)-->
> >
> > <property>
> >
> > <name>yarn.nodemanager.sleep-delay-before-sigkill.ms</name>
> >
> > <value>3600000</value>
> >
> > </property>
> >
> > <!-- End modifications for debugging slider -->
> >
> >
> > Regarding the error, I'm seeing, the slider install-package appears to
> > work correctly from what I can tell, but the slider create encounters run
> > time errors when launched, it seems to be looking for
> > protobuf-java-2.5.0.jar. I tried making fat jars for the memcached
> > application and the various slider packages, to no avail. Any ideas how
> to
> > resolve these errors reported in the resource manager:
> > Application application_1492557996590_0002 failed 2 times due to AM
> > Container for appattempt_1492557996590_0002_000002 exited with exitCode:
> > -1000
> > For more detailed output, check application tracking page:
> > http://quickstart.cloudera:8088/proxy/application_
> 1492557996590_0002/Then,
> > click on links to logs of each attempt.
> > Diagnostics: File file:/root/.slider/cluster/
> mymemcached/tmp/application_
> > 1492557996590_0002/am/lib/protobuf-java-2.5.0.jar does not exist
> > java.io.FileNotFoundException: File file:/root/.slider/cluster/
> > mymemcached/tmp/application_1492557996590_0002/am/lib/
> protobuf-java-2.5.0.jar
> > does not exist
> > at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(
> > RawLocalFileSystem.java:542)
> > at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(
> > RawLocalFileSystem.java:755)
> > at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(
> > RawLocalFileSystem.java:532)
> > at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(
> > FilterFileSystem.java:425)
> > at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251)
> > at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61)
> > at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
> > at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:422)
> > at org.apache.hadoop.security.UserGroupInformation.doAs(
> > UserGroupInformation.java:1693)
> > at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
> > at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > ThreadPoolExecutor.java:1142)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> > Failing this attempt. Failing the application.
> >
> >
> > The test-slider.sh script's output is as follows:
> >
> > # bash -x /tmp/mybuild/test_slider.sh
> >
> > + echo 'memcached built, registering memcached with slider and then
> > running it'
> >
> > memcached built, registering memcached with slider and then running it
> >
> > + /usr/lib/slider/bin/slider install-package --package
> > /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.0.zip
> > --name jmemcached --debug
> >
> > SLF4J: Class path contains multiple SLF4J bindings.
> >
> > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> > slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> > StaticLoggerBinder.class]
> >
> > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> > slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> >
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
> www.slf4j.org
> SLF4J warning or error messages and their meanings No SLF4J providers were
> found. This warning, i.e. not an error, message is reported when no SLF4J
> ...
>
>
>
> > explanation.
> >
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> >
> > 2017-04-19 01:10:44,080 [main] INFO client.RMProxy - Connecting to
> > ResourceManager at /0.0.0.0:8032
> >
> > 2017-04-19 01:10:44,243 [main] ERROR main.ServiceLauncher - Unable to
> > access supplied pkg file at /tmp/mybuild/incubator-slider/
> > app-packages/memcached/jmemcached-1.0.0.zip
> >
> > 2017-04-19 01:10:44,247 [main] INFO util.ExitUtil - Exiting with status
> 40
> >
> > + echo 'Listing applications - before starting jmemcached'
> >
> > Listing applications - before starting jmemcached
> >
> > + /usr/lib/slider/bin/slider list --manager localhost:8032
> >
> > SLF4J: Class path contains multiple SLF4J bindings.
> >
> > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> > slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> > StaticLoggerBinder.class]
> >
> > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> > slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> >
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
> www.slf4j.org
> SLF4J warning or error messages and their meanings No SLF4J providers were
> found. This warning, i.e. not an error, message is reported when no SLF4J
> ...
>
>
>
> > explanation.
> >
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> >
> > 2017-04-19 01:10:46,066 [main] INFO client.RMProxy - Connecting to
> > ResourceManager at localhost/127.0.0.1:8032
> >
> > 2017-04-19 01:10:46,217 [main] INFO util.ExitUtil - Exiting with status
> 0
> >
> > + /usr/lib/slider/bin/slider create mymemcached --template
> > /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json
> > --resources /tmp/mybuild/incubator-slider/app-packages/memcached/
> resources-default.json
> > --manager localhost:8032 --debug
> >
> > SLF4J: Class path contains multiple SLF4J bindings.
> >
> > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> > slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> > StaticLoggerBinder.class]
> >
> > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> > slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> >
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
> www.slf4j.org
> SLF4J warning or error messages and their meanings No SLF4J providers were
> found. This warning, i.e. not an error, message is reported when no SLF4J
> ...
>
>
>
> > explanation.
> >
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> >
> > 2017-04-19 01:10:48,070 [main] INFO client.RMProxy - Connecting to
> > ResourceManager at localhost/127.0.0.1:8032
> >
> > 2017-04-19 01:10:48,647 [main] INFO agent.AgentClientProvider -
> > Validating app definition /tmp/mybuild/incubator-slider/
> > app-packages/memcached/jmemcached-1.0.1.zip
> >
> > 2017-04-19 01:10:48,648 [main] INFO agent.AgentUtils - Reading metainfo
> > at /tmp/mybuild/incubator-slider/app-packages/memcached/
> > jmemcached-1.0.1.zip
> >
> > 2017-04-19 01:10:48,675 [main] INFO tools.SliderUtils - Reading
> > metainfo.xml of size 2202
> >
> > 2017-04-19 01:10:48,893 [main] INFO client.SliderClient - No credentials
> > requested
> >
> > 2017-04-19 01:10:48,938 [main] INFO agent.AgentUtils - Reading metainfo
> > at /tmp/mybuild/incubator-slider/app-packages/memcached/
> > jmemcached-1.0.1.zip
> >
> > 2017-04-19 01:10:48,939 [main] INFO tools.SliderUtils - Reading
> > metainfo.xml of size 2202
> >
> > 2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log
> include
> > patterns:
> >
> > 2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log
> exclude
> > patterns:
> >
> > 2017-04-19 01:10:49,861 [main] INFO slideram.SliderAMClientProvider -
> > Loading all dependencies for AM.
> >
> > 2017-04-19 01:10:49,862 [main] INFO tools.SliderUtils - Loading all
> > dependencies from /usr/lib/slider/lib
> >
> > 2017-04-19 01:10:51,324 [main] INFO agent.AgentClientProvider -
> > Automatically uploading the agent tarball at file:/root/.slider/cluster/
> > mymemcached/tmp/application_1492557996590_0003/agent
> >
> > 2017-04-19 01:10:51,361 [main] INFO agent.AgentClientProvider -
> > Validating app definition /tmp/mybuild/incubator-slider/
> > app-packages/memcached/jmemcached-1.0.1.zip
> >
> > 2017-04-19 01:10:51,361 [main] INFO agent.AgentUtils - Reading metainfo
> > at /tmp/mybuild/incubator-slider/app-packages/memcached/
> > jmemcached-1.0.1.zip
> >
> > 2017-04-19 01:10:51,364 [main] INFO tools.SliderUtils - Reading
> > metainfo.xml of size 2202
> >
> > 2017-04-19 01:10:51,429 [main] INFO Configuration.deprecation -
> > slider.registry.path is deprecated. Instead, use hadoop.registry.zk.root
> >
> > 2017-04-19 01:10:51,436 [main] INFO launch.AppMasterLauncher -
> Submitting
> > application to Resource Manager
> >
> > 2017-04-19 01:10:51,480 [main] INFO impl.YarnClientImpl - Submitted
> > application application_1492557996590_0003
> >
> > 2017-04-19 01:10:51,484 [main] INFO util.ExitUtil - Exiting with status
> 0
> >
> > + echo 'Listing applications - after starting jmemcached'
> >
> > Listing applications - after starting jmemcached
> >
> > + /usr/lib/slider/bin/slider list --manager localhost:8032
> >
> > SLF4J: Class path contains multiple SLF4J bindings.
> >
> > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> > slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> > StaticLoggerBinder.class]
> >
> > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> > slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> >
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
> www.slf4j.org
> SLF4J warning or error messages and their meanings No SLF4J providers were
> found. This warning, i.e. not an error, message is reported when no SLF4J
> ...
>
>
>
> > explanation.
> >
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> >
> > 2017-04-19 01:10:53,893 [main] INFO client.RMProxy - Connecting to
> > ResourceManager at localhost/127.0.0.1:8032
> >
> > mymemcached ACCEPTED
> application_1492557996590_0003
> >
> > 2017-04-19 01:10:54,186 [main] INFO util.ExitUtil - Exiting with status
> 0
> >
> > + echo 'Finished /tmp/mybuild/test_slider.sh'
> >
> > Finished /tmp/mybuild/test_slider.sh
> >
> >
> > Thanks:
> >
> >
> > Bill
> >
>
Re: Trying to get he memcached example to run for a new build
Posted by Foolish Ewe <fo...@hotmail.com>.
Thanks Billie for the quick and thoughtful reply.
You are correct, I'm testing in a single node docker container and have slider built from source and installed during the docker build action.
Regarding the file systems (in the docker container) I want to be precise and make sure that I understand what needs to be in the hdfs and what needs to be on the local linux (to the docker container) file system.
I have not had much luck in getting slider to pickup code or configuration data from HDFS, is there a setting I need to apply to do this?
Also does slider write any logs? Where would they be?
Regarding the protobuf-java-2.5.0.jar, I see it in the local (not HDFS) directory: /usr/lib/slider/lib/
# ls /usr/lib/slider/lib/protobuf-java-2.5.0.jar
/usr/lib/slider/lib/protobuf-java-2.5.0.jar
Is this the correct location? Should I install these jars in HDFS, if so what is the appropriate hdfs path?
Regarding jmemcached-1.0.0.zip, you are correct, I forgot to update the jmemcached file name in the script (I updated to 1.0.1 since I built memcached for source to create an uber jar using the maven assembly plugin), I should have caught that.
The revised test script was updated to reflect this the install-package command now has the correct version number in the zip name and now reads:
So now the script reads (the commented out lines using hdfs did not work, not sure why, even when I change appConfig to use hdfs url paths):
#!/bin/bash
# this does not work when run during docker-build, we need to do this once we are logged in
echo "setting up the enviornment variables"
env
export JAVA_HOME="/usr/lib/jvm/java"
export HADOOP_CONF="/etc/hadoop/conf"
env
echo "memcached built, registering memcached with slider and then running it"
/usr/lib/slider/bin/slider install-package --package /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip --name jmemcached --debug
#echo "using hdfs version, posting artifacts to hdfs"
#hadoop fs -put ./incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip ./incubator-slider/app-packages/memcached/*.json /user/root/artifacts/memcached
#/usr/lib/slider/bin/slider install-package --package hdfs://quickstart.cloudera/user/root/artifacts/memcached/jmemcached-1.0.1.zip --name jmemcached --debug
echo "Listing applications - before starting jmemcached"
/usr/lib/slider/bin/slider list --manager localhost:8032
# startup memcached properly, yarn task name will be mymemcached
/usr/lib/slider/bin/slider create mymemcached --template /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json --resources /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json --manager localhost:8032 --debug
#/usr/lib/slider/bin/slider create mymemcached --template hdfs://quickstart.cloudera/user/root/artifacts/memcached/appConfig.json --resources hdfs://quickstart.cloudera/user/root/artifacts/memcached/resources-default.json --manager localhost:8032 --debug
echo "Listing applications - after starting jmemcached"
/usr/lib/slider/bin/slider list --manager localhost:8032
echo "Finished $0"
appConfig.json reads:
{
"schema": "http://example.org/specification/v2.0.0",
"metadata": {
},
"global": {
"agent.conf": "/tmp/mybuild/agent/conf/agent.ini",
"application.def": "/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip",
"java_home": "/usr/lib/jvm/java/",
"site.global.additional_cp": "/tmp/mybuild/incubator-slider/slider-assembly/target/lib/",
"site.global.xmx_val": "256m",
"site.global.xms_val": "128m",
"site.global.memory_val": "200M",
"site.global.listen_port": "${MEMCACHED.ALLOCATED_PORT}{PER_CONTAINER}"
},
"components": {
"slider-appmaster": {
"jvm.heapsize": "256M"
}
}
}
/tmp/mybuild/agent/conf/agent.ini reads:
app_task_dir=.
app_log_dir=.
app_tmp_dir=app/tmp
log_dir=.
run_dir=infra/run
version_file=infra/version
;log_level=INFO
log_level=DEBUG
[python]
[command]
max_retries=2
sleep_between_retries=1
auto_restart=5,5
[security]
[heartbeat]
state_interval=6
log_lines_count=300
And then running the corrected script I see:
+ bash -x ./test_slider.sh
+ echo 'setting up the enviornment variables'
setting up the enviornment variables
+ env
HOSTNAME=quickstart.cloudera
TERM=xterm
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/tmp/mybuild
LANG=en_US.UTF-8
SHLVL=3
HOME=/root
no_proxy=*.local, 169.254/16
LESSOPEN=||/usr/bin/lesspipe.sh %s
container=docker
_=/usr/bin/env
+ export JAVA_HOME=/usr/lib/jvm/java
+ JAVA_HOME=/usr/lib/jvm/java
+ export HADOOP_CONF=/etc/hadoop/conf
+ HADOOP_CONF=/etc/hadoop/conf
+ env
HOSTNAME=quickstart.cloudera
TERM=xterm
HADOOP_CONF=/etc/hadoop/conf
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/tmp/mybuild
JAVA_HOME=/usr/lib/jvm/java
LANG=en_US.UTF-8
SHLVL=3
HOME=/root
no_proxy=*.local, 169.254/16
LESSOPEN=||/usr/bin/lesspipe.sh %s
container=docker
_=/usr/bin/env
+ echo 'memcached built, registering memcached with slider and then running it'
memcached built, registering memcached with slider and then running it
+ /usr/lib/slider/bin/slider install-package --package /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip --name jmemcached --debug --replacepkg
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-21 19:22:12,948 [main] INFO client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032
2017-04-21 19:22:13,093 [main] INFO client.SliderClient - Installing package file:/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip at file:/root/.slider/package/jmemcached/jmemcached-1.0.1.zip and overwrite is true.
2017-04-21 19:22:13,109 [main] INFO util.ExitUtil - Exiting with status 0
+ echo 'Listing applications - before starting jmemcached'
Listing applications - before starting jmemcached
+ /usr/lib/slider/bin/slider list --manager localhost:8032
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-21 19:22:15,070 [main] INFO client.RMProxy - Connecting to ResourceManager at localhost/127.0.0.1:8032
2017-04-21 19:22:15,222 [main] INFO util.ExitUtil - Exiting with status 0
+ /usr/lib/slider/bin/slider create mymemcached --template /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json --resources /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json --manager localhost:8032 --debug
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-21 19:22:17,183 [main] INFO client.RMProxy - Connecting to ResourceManager at localhost/127.0.0.1:8032
2017-04-21 19:22:17,661 [main] INFO agent.AgentClientProvider - Validating app definition /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-21 19:22:17,662 [main] INFO agent.AgentUtils - Reading metainfo at /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-21 19:22:17,685 [main] INFO tools.SliderUtils - Reading metainfo.xml of size 2202
2017-04-21 19:22:17,909 [main] INFO client.SliderClient - No credentials requested
2017-04-21 19:22:18,108 [main] INFO agent.AgentUtils - Reading metainfo at /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-21 19:22:18,116 [main] INFO tools.SliderUtils - Reading metainfo.xml of size 2202
2017-04-21 19:22:18,177 [main] INFO launch.AbstractLauncher - Log include patterns:
2017-04-21 19:22:18,177 [main] INFO launch.AbstractLauncher - Log exclude patterns:
2017-04-21 19:22:18,938 [main] INFO slideram.SliderAMClientProvider - Loading all dependencies for AM.
2017-04-21 19:22:18,939 [main] INFO tools.SliderUtils - Loading all dependencies from /usr/lib/slider/lib
2017-04-21 19:22:20,687 [main] INFO agent.AgentClientProvider - Automatically uploading the agent tarball at file:/root/.slider/cluster/mymemcached/tmp/application_1492799085220_0003/agent
2017-04-21 19:22:20,727 [main] INFO agent.AgentClientProvider - Validating app definition /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-21 19:22:20,727 [main] INFO agent.AgentUtils - Reading metainfo at /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
2017-04-21 19:22:20,735 [main] INFO tools.SliderUtils - Reading metainfo.xml of size 2202
2017-04-21 19:22:20,807 [main] INFO Configuration.deprecation - slider.registry.path is deprecated. Instead, use hadoop.registry.zk.root
2017-04-21 19:22:20,813 [main] INFO launch.AppMasterLauncher - Submitting application to Resource Manager
2017-04-21 19:22:20,838 [main] INFO impl.YarnClientImpl - Submitted application application_1492799085220_0003
2017-04-21 19:22:20,845 [main] INFO util.ExitUtil - Exiting with status 0
+ echo 'Listing applications - after starting jmemcached'
Listing applications - after starting jmemcached
+ /usr/lib/slider/bin/slider list --manager localhost:8032
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slider-core-0.60.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-04-21 19:22:22,857 [main] INFO client.RMProxy - Connecting to ResourceManager at localhost/127.0.0.1:8032
mymemcached ACCEPTED application_1492799085220_0003
2017-04-21 19:22:23,142 [main] INFO util.ExitUtil - Exiting with status 0
+ echo 'Finished ./test_slider.sh'
Finished ./test_slider.sh
I am still getting the dreaded failure as reported in the yarn resource manager:
Application application_1492799085220_0003 failed 2 times due to AM Container for appattempt_1492799085220_0003_000002 exited with exitCode: -1000
For more detailed output, check application tracking page:http://quickstart.cloudera:8088/proxy/application_1492799085220_0003/Then, click on links to logs of each attempt.
Diagnostics: File file:/root/.slider/cluster/mymemcached/tmp/application_1492799085220_0003/am/lib/protobuf-java-2.5.0.jar does not exist
java.io.FileNotFoundException: File file:/root/.slider/cluster/mymemcached/tmp/application_1492799085220_0003/am/lib/protobuf-java-2.5.0.jar does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:542)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:755)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:532)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:425)
at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Failing this attempt. Failing the application.
With best regards:
Bill
________________________________
From: Billie Rinaldi <bi...@gmail.com>
Sent: Wednesday, April 19, 2017 5:36 PM
To: dev@slider.incubator.apache.org
Subject: Re: Trying to get he memcached example to run for a new build
The protobuf-java-2.5.0.jar should be in the Slider lib directory. It will
get pushed out to the AM lib directory from there.
There are some other issues with the script output. The install package
command has not succeeded, and says "ERROR main.ServiceLauncher - Unable to
access supplied pkg file at /tmp/mybuild/incubator-slider/
app-packages/memcached/jmemcached-1.0.0.zip"
Also the line INFO agent.AgentClientProvider - Validating app definition
/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
indicates that it is reading the app definition from the local filesystem,
when it should be reading it from HDFS. Maybe since this is a single VM
setup it is able to continue despite these errors, but if you were running
on a real cluster you'd have to make sure install package was working and
fix the app definition path in the appConfig.
On Tue, Apr 18, 2017 at 6:35 PM, Foolish Ewe <fo...@hotmail.com> wrote:
> Hello All:
>
>
> I'm trying to run the memcached example locally but it fails pretty quick
> and claims "protobuf-java-2.5.0.jar does not exist".
>
>
> Consider the following script (test-slider.sh):
>
> #!/bin/bash
>
> # this does not work when run during docker-build, we need to do this once
> we are logged in
>
> echo "memcached built, registering memcached with slider and then running
> it"
>
> /usr/lib/slider/bin/slider install-package --package
> /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip
> --name jmemcached --debug
>
> echo "Listing applications - before starting jmemcached"
>
> /usr/lib/slider/bin/slider list --manager localhost:8032
>
> # startup memcached properly, yarn task name will be mymemcached
>
> /usr/lib/slider/bin/slider create mymemcached --template
> /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json
> --resources /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json
> --manager localhost:8032 --debug
>
> echo "Listing applications - after starting jmemcached"
>
> /usr/lib/slider/bin/slider list --manager localhost:8032
>
> echo "Finished $0"
>
> I am having difficulty in locating the logs after the failure, any idea
> where they would be? yarn logs cannot find them, and I'm not having luck
> with the name node. I did modify the yarn-site.xml read at startup to
> include the lines:
>
> <!-- Begin modifications for debugging slider -->
>
> <!-- 60 minutes after a failure to see what is left in the directory-->
>
> <property>
>
> <name>yarn.nodemanager.delete.debug-delay-sec</name>
>
> <value>3600</value>
>
> </property>
>
>
> <!--time before the process gets a -9 (Should it be 30 seconds?)-->
>
> <property>
>
> <name>yarn.nodemanager.sleep-delay-before-sigkill.ms</name>
>
> <value>3600000</value>
>
> </property>
>
> <!-- End modifications for debugging slider -->
>
>
> Regarding the error, I'm seeing, the slider install-package appears to
> work correctly from what I can tell, but the slider create encounters run
> time errors when launched, it seems to be looking for
> protobuf-java-2.5.0.jar. I tried making fat jars for the memcached
> application and the various slider packages, to no avail. Any ideas how to
> resolve these errors reported in the resource manager:
> Application application_1492557996590_0002 failed 2 times due to AM
> Container for appattempt_1492557996590_0002_000002 exited with exitCode:
> -1000
> For more detailed output, check application tracking page:
> http://quickstart.cloudera:8088/proxy/application_1492557996590_0002/Then,
> click on links to logs of each attempt.
> Diagnostics: File file:/root/.slider/cluster/mymemcached/tmp/application_
> 1492557996590_0002/am/lib/protobuf-java-2.5.0.jar does not exist
> java.io.FileNotFoundException: File file:/root/.slider/cluster/
> mymemcached/tmp/application_1492557996590_0002/am/lib/protobuf-java-2.5.0.jar
> does not exist
> at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(
> RawLocalFileSystem.java:542)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(
> RawLocalFileSystem.java:755)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(
> RawLocalFileSystem.java:532)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(
> FilterFileSystem.java:425)
> at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251)
> at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1693)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Failing this attempt. Failing the application.
>
>
> The test-slider.sh script's output is as follows:
>
> # bash -x /tmp/mybuild/test_slider.sh
>
> + echo 'memcached built, registering memcached with slider and then
> running it'
>
> memcached built, registering memcached with slider and then running it
>
> + /usr/lib/slider/bin/slider install-package --package
> /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip
> --name jmemcached --debug
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
www.slf4j.org
SLF4J warning or error messages and their meanings No SLF4J providers were found. This warning, i.e. not an error, message is reported when no SLF4J ...
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-19 01:10:44,080 [main] INFO client.RMProxy - Connecting to
> ResourceManager at /0.0.0.0:8032
>
> 2017-04-19 01:10:44,243 [main] ERROR main.ServiceLauncher - Unable to
> access supplied pkg file at /tmp/mybuild/incubator-slider/
> app-packages/memcached/jmemcached-1.0.0.zip
>
> 2017-04-19 01:10:44,247 [main] INFO util.ExitUtil - Exiting with status 40
>
> + echo 'Listing applications - before starting jmemcached'
>
> Listing applications - before starting jmemcached
>
> + /usr/lib/slider/bin/slider list --manager localhost:8032
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
www.slf4j.org
SLF4J warning or error messages and their meanings No SLF4J providers were found. This warning, i.e. not an error, message is reported when no SLF4J ...
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-19 01:10:46,066 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
>
> 2017-04-19 01:10:46,217 [main] INFO util.ExitUtil - Exiting with status 0
>
> + /usr/lib/slider/bin/slider create mymemcached --template
> /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json
> --resources /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json
> --manager localhost:8032 --debug
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
www.slf4j.org
SLF4J warning or error messages and their meanings No SLF4J providers were found. This warning, i.e. not an error, message is reported when no SLF4J ...
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-19 01:10:48,070 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
>
> 2017-04-19 01:10:48,647 [main] INFO agent.AgentClientProvider -
> Validating app definition /tmp/mybuild/incubator-slider/
> app-packages/memcached/jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:48,648 [main] INFO agent.AgentUtils - Reading metainfo
> at /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:48,675 [main] INFO tools.SliderUtils - Reading
> metainfo.xml of size 2202
>
> 2017-04-19 01:10:48,893 [main] INFO client.SliderClient - No credentials
> requested
>
> 2017-04-19 01:10:48,938 [main] INFO agent.AgentUtils - Reading metainfo
> at /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:48,939 [main] INFO tools.SliderUtils - Reading
> metainfo.xml of size 2202
>
> 2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log include
> patterns:
>
> 2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log exclude
> patterns:
>
> 2017-04-19 01:10:49,861 [main] INFO slideram.SliderAMClientProvider -
> Loading all dependencies for AM.
>
> 2017-04-19 01:10:49,862 [main] INFO tools.SliderUtils - Loading all
> dependencies from /usr/lib/slider/lib
>
> 2017-04-19 01:10:51,324 [main] INFO agent.AgentClientProvider -
> Automatically uploading the agent tarball at file:/root/.slider/cluster/
> mymemcached/tmp/application_1492557996590_0003/agent
>
> 2017-04-19 01:10:51,361 [main] INFO agent.AgentClientProvider -
> Validating app definition /tmp/mybuild/incubator-slider/
> app-packages/memcached/jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:51,361 [main] INFO agent.AgentUtils - Reading metainfo
> at /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:51,364 [main] INFO tools.SliderUtils - Reading
> metainfo.xml of size 2202
>
> 2017-04-19 01:10:51,429 [main] INFO Configuration.deprecation -
> slider.registry.path is deprecated. Instead, use hadoop.registry.zk.root
>
> 2017-04-19 01:10:51,436 [main] INFO launch.AppMasterLauncher - Submitting
> application to Resource Manager
>
> 2017-04-19 01:10:51,480 [main] INFO impl.YarnClientImpl - Submitted
> application application_1492557996590_0003
>
> 2017-04-19 01:10:51,484 [main] INFO util.ExitUtil - Exiting with status 0
>
> + echo 'Listing applications - after starting jmemcached'
>
> Listing applications - after starting jmemcached
>
> + /usr/lib/slider/bin/slider list --manager localhost:8032
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
SLF4J Error Codes<http://www.slf4j.org/codes.html#multiple_bindings>
www.slf4j.org
SLF4J warning or error messages and their meanings No SLF4J providers were found. This warning, i.e. not an error, message is reported when no SLF4J ...
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-19 01:10:53,893 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
>
> mymemcached ACCEPTED application_1492557996590_0003
>
> 2017-04-19 01:10:54,186 [main] INFO util.ExitUtil - Exiting with status 0
>
> + echo 'Finished /tmp/mybuild/test_slider.sh'
>
> Finished /tmp/mybuild/test_slider.sh
>
>
> Thanks:
>
>
> Bill
>
Re: Trying to get he memcached example to run for a new build
Posted by Billie Rinaldi <bi...@gmail.com>.
The protobuf-java-2.5.0.jar should be in the Slider lib directory. It will
get pushed out to the AM lib directory from there.
There are some other issues with the script output. The install package
command has not succeeded, and says "ERROR main.ServiceLauncher - Unable to
access supplied pkg file at /tmp/mybuild/incubator-slider/
app-packages/memcached/jmemcached-1.0.0.zip"
Also the line INFO agent.AgentClientProvider - Validating app definition
/tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip
indicates that it is reading the app definition from the local filesystem,
when it should be reading it from HDFS. Maybe since this is a single VM
setup it is able to continue despite these errors, but if you were running
on a real cluster you'd have to make sure install package was working and
fix the app definition path in the appConfig.
On Tue, Apr 18, 2017 at 6:35 PM, Foolish Ewe <fo...@hotmail.com> wrote:
> Hello All:
>
>
> I'm trying to run the memcached example locally but it fails pretty quick
> and claims "protobuf-java-2.5.0.jar does not exist".
>
>
> Consider the following script (test-slider.sh):
>
> #!/bin/bash
>
> # this does not work when run during docker-build, we need to do this once
> we are logged in
>
> echo "memcached built, registering memcached with slider and then running
> it"
>
> /usr/lib/slider/bin/slider install-package --package
> /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip
> --name jmemcached --debug
>
> echo "Listing applications - before starting jmemcached"
>
> /usr/lib/slider/bin/slider list --manager localhost:8032
>
> # startup memcached properly, yarn task name will be mymemcached
>
> /usr/lib/slider/bin/slider create mymemcached --template
> /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json
> --resources /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json
> --manager localhost:8032 --debug
>
> echo "Listing applications - after starting jmemcached"
>
> /usr/lib/slider/bin/slider list --manager localhost:8032
>
> echo "Finished $0"
>
> I am having difficulty in locating the logs after the failure, any idea
> where they would be? yarn logs cannot find them, and I'm not having luck
> with the name node. I did modify the yarn-site.xml read at startup to
> include the lines:
>
> <!-- Begin modifications for debugging slider -->
>
> <!-- 60 minutes after a failure to see what is left in the directory-->
>
> <property>
>
> <name>yarn.nodemanager.delete.debug-delay-sec</name>
>
> <value>3600</value>
>
> </property>
>
>
> <!--time before the process gets a -9 (Should it be 30 seconds?)-->
>
> <property>
>
> <name>yarn.nodemanager.sleep-delay-before-sigkill.ms</name>
>
> <value>3600000</value>
>
> </property>
>
> <!-- End modifications for debugging slider -->
>
>
> Regarding the error, I'm seeing, the slider install-package appears to
> work correctly from what I can tell, but the slider create encounters run
> time errors when launched, it seems to be looking for
> protobuf-java-2.5.0.jar. I tried making fat jars for the memcached
> application and the various slider packages, to no avail. Any ideas how to
> resolve these errors reported in the resource manager:
> Application application_1492557996590_0002 failed 2 times due to AM
> Container for appattempt_1492557996590_0002_000002 exited with exitCode:
> -1000
> For more detailed output, check application tracking page:
> http://quickstart.cloudera:8088/proxy/application_1492557996590_0002/Then,
> click on links to logs of each attempt.
> Diagnostics: File file:/root/.slider/cluster/mymemcached/tmp/application_
> 1492557996590_0002/am/lib/protobuf-java-2.5.0.jar does not exist
> java.io.FileNotFoundException: File file:/root/.slider/cluster/
> mymemcached/tmp/application_1492557996590_0002/am/lib/protobuf-java-2.5.0.jar
> does not exist
> at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(
> RawLocalFileSystem.java:542)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(
> RawLocalFileSystem.java:755)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(
> RawLocalFileSystem.java:532)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(
> FilterFileSystem.java:425)
> at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251)
> at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1693)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Failing this attempt. Failing the application.
>
>
> The test-slider.sh script's output is as follows:
>
> # bash -x /tmp/mybuild/test_slider.sh
>
> + echo 'memcached built, registering memcached with slider and then
> running it'
>
> memcached built, registering memcached with slider and then running it
>
> + /usr/lib/slider/bin/slider install-package --package
> /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip
> --name jmemcached --debug
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-19 01:10:44,080 [main] INFO client.RMProxy - Connecting to
> ResourceManager at /0.0.0.0:8032
>
> 2017-04-19 01:10:44,243 [main] ERROR main.ServiceLauncher - Unable to
> access supplied pkg file at /tmp/mybuild/incubator-slider/
> app-packages/memcached/jmemcached-1.0.0.zip
>
> 2017-04-19 01:10:44,247 [main] INFO util.ExitUtil - Exiting with status 40
>
> + echo 'Listing applications - before starting jmemcached'
>
> Listing applications - before starting jmemcached
>
> + /usr/lib/slider/bin/slider list --manager localhost:8032
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-19 01:10:46,066 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
>
> 2017-04-19 01:10:46,217 [main] INFO util.ExitUtil - Exiting with status 0
>
> + /usr/lib/slider/bin/slider create mymemcached --template
> /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json
> --resources /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json
> --manager localhost:8032 --debug
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-19 01:10:48,070 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
>
> 2017-04-19 01:10:48,647 [main] INFO agent.AgentClientProvider -
> Validating app definition /tmp/mybuild/incubator-slider/
> app-packages/memcached/jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:48,648 [main] INFO agent.AgentUtils - Reading metainfo
> at /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:48,675 [main] INFO tools.SliderUtils - Reading
> metainfo.xml of size 2202
>
> 2017-04-19 01:10:48,893 [main] INFO client.SliderClient - No credentials
> requested
>
> 2017-04-19 01:10:48,938 [main] INFO agent.AgentUtils - Reading metainfo
> at /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:48,939 [main] INFO tools.SliderUtils - Reading
> metainfo.xml of size 2202
>
> 2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log include
> patterns:
>
> 2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log exclude
> patterns:
>
> 2017-04-19 01:10:49,861 [main] INFO slideram.SliderAMClientProvider -
> Loading all dependencies for AM.
>
> 2017-04-19 01:10:49,862 [main] INFO tools.SliderUtils - Loading all
> dependencies from /usr/lib/slider/lib
>
> 2017-04-19 01:10:51,324 [main] INFO agent.AgentClientProvider -
> Automatically uploading the agent tarball at file:/root/.slider/cluster/
> mymemcached/tmp/application_1492557996590_0003/agent
>
> 2017-04-19 01:10:51,361 [main] INFO agent.AgentClientProvider -
> Validating app definition /tmp/mybuild/incubator-slider/
> app-packages/memcached/jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:51,361 [main] INFO agent.AgentUtils - Reading metainfo
> at /tmp/mybuild/incubator-slider/app-packages/memcached/
> jmemcached-1.0.1.zip
>
> 2017-04-19 01:10:51,364 [main] INFO tools.SliderUtils - Reading
> metainfo.xml of size 2202
>
> 2017-04-19 01:10:51,429 [main] INFO Configuration.deprecation -
> slider.registry.path is deprecated. Instead, use hadoop.registry.zk.root
>
> 2017-04-19 01:10:51,436 [main] INFO launch.AppMasterLauncher - Submitting
> application to Resource Manager
>
> 2017-04-19 01:10:51,480 [main] INFO impl.YarnClientImpl - Submitted
> application application_1492557996590_0003
>
> 2017-04-19 01:10:51,484 [main] INFO util.ExitUtil - Exiting with status 0
>
> + echo 'Listing applications - after starting jmemcached'
>
> Listing applications - after starting jmemcached
>
> + /usr/lib/slider/bin/slider list --manager localhost:8032
>
> SLF4J: Class path contains multiple SLF4J bindings.
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slider-core-0.60.0-incubating.jar!/org/slf4j/impl/
> StaticLoggerBinder.class]
>
> SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/
> slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
>
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> 2017-04-19 01:10:53,893 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
>
> mymemcached ACCEPTED application_1492557996590_0003
>
> 2017-04-19 01:10:54,186 [main] INFO util.ExitUtil - Exiting with status 0
>
> + echo 'Finished /tmp/mybuild/test_slider.sh'
>
> Finished /tmp/mybuild/test_slider.sh
>
>
> Thanks:
>
>
> Bill
>