You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by "Francis.Hu" <fr...@reachjunction.com> on 2014/05/05 12:06:10 UTC
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Hi,All
We run a spark cluster with three workers.
created a spark streaming application,
then run the spark project using below command:
shell> sbt run spark://192.168.219.129:7077 tcp://192.168.20.118:5556 foo
we looked at the webui of workers, jobs failed without any error or info,
but FileNotFoundException occurred in workers' log file as below:
Is this an existent issue of spark?
-----------------------------in workers'
logs/spark-francis-org.apache.spark.deploy.worker.Worker-1-ubuntu-4.out-----
-----------------------
14/05/05 02:39:39 WARN AbstractHttpConnection:
/logPage/?appId=app-20140505053550-0000&executorId=2&logType=stdout
java.io.FileNotFoundException:
/test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or
directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.s
cala:52)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.s
cala:52)
at
org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java
:1040)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:
976)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135
)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:1
16)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpCo
nnection.java:483)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpC
onnection.java:920)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplet
e(AbstractHttpConnection.java:982)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java
:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.
java:628)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.j
ava:52)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:
608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:5
43)
at java.lang.Thread.run(Thread.java:722)
14/05/05 02:39:41 WARN AbstractHttpConnection:
/logPage/?appId=app-20140505053550-0000&executorId=9&logType=stderr
java.io.FileNotFoundException:
/test/spark-0.9.1/work/app-20140505053550-0000/9/stderr (No such file or
directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.s
cala:52)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.s
cala:52)
at
org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java
:1040)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:
976)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135
)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:1
16)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpCo
nnection.java:483)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpC
onnection.java:920)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplet
e(AbstractHttpConnection.java:982)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java
:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.
java:628)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.j
ava:52)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:
608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:5
43)
at java.lang.Thread.run(Thread.java:722)
Thanks,
Francis.Hu
答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Posted by "Francis.Hu" <fr...@reachjunction.com>.
I have just the problem resolved via running master and work daemons individually on where they are.
if I execute the shell: sbin/start-all.sh , the problem always exist.
发件人: Francis.Hu [mailto:francis.hu@reachjunction.com]
发送时间: Tuesday, May 06, 2014 10:31
收件人: user@spark.apache.org
主题: 答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
i looked into the log again, all exceptions are about FileNotFoundException . In the Webui, no anymore info I can check except for the basic description of job.
Attached the log file, could you help to take a look ? Thanks.
Francis.Hu
发件人: Tathagata Das [mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 10:16
收件人: user@spark.apache.org
主题: Re: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Can you check the Spark worker logs on that machine. Either from the web ui, or directly. Should be /test/spark-XXX/logs/ See if that has any error.
If there is not permission issue, I am not why stdout and stderr is not being generated.
TD
On Mon, May 5, 2014 at 7:13 PM, Francis.Hu <fr...@reachjunction.com> wrote:
The file does not exist in fact and no permission issue.
francis@ubuntu-4:/test/spark-0.9.1$ ll work/app-20140505053550-0000/
total 24
drwxrwxr-x 6 francis francis 4096 May 5 05:35 ./
drwxrwxr-x 11 francis francis 4096 May 5 06:18 ../
drwxrwxr-x 2 francis francis 4096 May 5 05:35 2/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 4/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 7/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 9/
Francis
发件人: Tathagata Das [mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 3:45
收件人: user@spark.apache.org
主题: Re: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Do those file actually exist? Those stdout/stderr should have the output of the spark's executors running in the workers, and its weird that they dont exist. Could be permission issue - maybe the directories/files are not being generated because it cannot?
TD
On Mon, May 5, 2014 at 3:06 AM, Francis.Hu <fr...@reachjunction.com> wrote:
Hi,All
We run a spark cluster with three workers.
created a spark streaming application,
then run the spark project using below command:
shell> sbt run spark://192.168.219.129:7077 tcp://192.168.20.118:5556 foo
we looked at the webui of workers, jobs failed without any error or info, but FileNotFoundException occurred in workers' log file as below:
Is this an existent issue of spark?
-----------------------------in workers' logs/spark-francis-org.apache.spark.deploy.worker.Worker-1-ubuntu-4.out----------------------------
14/05/05 02:39:39 WARN AbstractHttpConnection: /logPage/?appId=app-20140505053550-0000&executorId=2&logType=stdout
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)
14/05/05 02:39:41 WARN AbstractHttpConnection: /logPage/?appId=app-20140505053550-0000&executorId=9&logType=stderr
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/9/stderr (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)
Thanks,
Francis.Hu
答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Posted by "Francis.Hu" <fr...@reachjunction.com>.
i looked into the log again, all exceptions are about FileNotFoundException . In the Webui, no anymore info I can check except for the basic description of job.
Attached the log file, could you help to take a look ? Thanks.
Francis.Hu
发件人: Tathagata Das [mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 10:16
收件人: user@spark.apache.org
主题: Re: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Can you check the Spark worker logs on that machine. Either from the web ui, or directly. Should be /test/spark-XXX/logs/ See if that has any error.
If there is not permission issue, I am not why stdout and stderr is not being generated.
TD
On Mon, May 5, 2014 at 7:13 PM, Francis.Hu <fr...@reachjunction.com> wrote:
The file does not exist in fact and no permission issue.
francis@ubuntu-4:/test/spark-0.9.1$ ll work/app-20140505053550-0000/
total 24
drwxrwxr-x 6 francis francis 4096 May 5 05:35 ./
drwxrwxr-x 11 francis francis 4096 May 5 06:18 ../
drwxrwxr-x 2 francis francis 4096 May 5 05:35 2/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 4/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 7/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 9/
Francis
发件人: Tathagata Das [mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 3:45
收件人: user@spark.apache.org
主题: Re: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Do those file actually exist? Those stdout/stderr should have the output of the spark's executors running in the workers, and its weird that they dont exist. Could be permission issue - maybe the directories/files are not being generated because it cannot?
TD
On Mon, May 5, 2014 at 3:06 AM, Francis.Hu <fr...@reachjunction.com> wrote:
Hi,All
We run a spark cluster with three workers.
created a spark streaming application,
then run the spark project using below command:
shell> sbt run spark://192.168.219.129:7077 tcp://192.168.20.118:5556 foo
we looked at the webui of workers, jobs failed without any error or info, but FileNotFoundException occurred in workers' log file as below:
Is this an existent issue of spark?
-----------------------------in workers' logs/spark-francis-org.apache.spark.deploy.worker.Worker-1-ubuntu-4.out----------------------------
14/05/05 02:39:39 WARN AbstractHttpConnection: /logPage/?appId=app-20140505053550-0000&executorId=2&logType=stdout
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)
14/05/05 02:39:41 WARN AbstractHttpConnection: /logPage/?appId=app-20140505053550-0000&executorId=9&logType=stderr
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/9/stderr (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)
Thanks,
Francis.Hu
RE: 答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Posted by Andrew Lee <al...@hotmail.com>.
Sorry to ask a blunt question, are the local files all available on all nodes?
From: francis.hu@reachjunction.com
To: user@spark.apache.org
Subject: RE: 答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Date: Tue, 6 May 2014 10:06:17 +0000
We are using local files for testing, the uri of HDFS looks like : hdfs://xxxx:xx/path/to/file
From: alee526@hotmail.com
To: user@spark.apache.org
Subject: RE: 答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Date: Tue, 6 May 2014 00:29:26 -0700
Hi Francis,
Are you trying to access files on local machines or on HDFS? Any examples for the URI how it looks like?
From: francis.hu@reachjunction.com
To: user@spark.apache.org
Subject: 答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Date: Tue, 6 May 2014 12:12:24 +0800
I looked into the source code of WorkerWebUI.scala, seems there is
no file existent check
Francis
发件人: Tathagata Das
[mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 10:16
收件人: user@spark.apache.org
主题: Re: 答复: java.io.FileNotFoundException:
/test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or
directory)
Can you check the Spark worker logs on that
machine. Either from the web ui, or directly. Should be /test/spark-XXX/logs/
See if that has any error.
If there is not permission issue, I am not
why stdout and stderr is not being generated.
TD
On Mon, May 5, 2014 at 7:13 PM, Francis.Hu
<fr...@reachjunction.com>
wrote:
The file does not exist in fact and no permission issue.
francis@ubuntu-4:/test/spark-0.9.1$ ll work/app-20140505053550-0000/
total 24
drwxrwxr-x 6 francis francis 4096 May 5 05:35 ./
drwxrwxr-x 11 francis francis 4096 May 5 06:18 ../
drwxrwxr-x 2 francis francis 4096 May 5 05:35 2/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 4/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 7/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 9/
Francis
发件人: Tathagata Das [mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 3:45
收件人: user@spark.apache.org
主题: Re: java.io.FileNotFoundException:
/test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or
directory)
Do those file actually exist? Those stdout/stderr should have the
output of the spark's executors running in the workers, and its weird that they
dont exist. Could be permission issue - maybe the directories/files are not
being generated because it cannot?
TD
On Mon, May 5, 2014 at 3:06 AM, Francis.Hu <fr...@reachjunction.com>
wrote:
Hi,All
We run a spark cluster with three workers.
created a spark streaming application,
then run the spark project using below command:
shell> sbt run spark://192.168.219.129:7077 tcp://192.168.20.118:5556 foo
we looked at the webui of workers, jobs failed without any error or
info, but FileNotFoundException occurred in workers' log file as below:
Is this an existent issue of spark?
-----------------------------in workers'
logs/spark-francis-org.apache.spark.deploy.worker.Worker-1-ubuntu-4.out----------------------------
14/05/05 02:39:39 WARN AbstractHttpConnection:
/logPage/?appId=app-20140505053550-0000&executorId=2&logType=stdout
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout
(No such file or directory)
at
java.io.FileInputStream.open(Native Method)
at
java.io.FileInputStream.<init>(FileInputStream.java:138)
at
org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at
org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at
org.eclipse.jetty.server.Server.handle(Server.java:363)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at
java.lang.Thread.run(Thread.java:722)
14/05/05 02:39:41 WARN AbstractHttpConnection:
/logPage/?appId=app-20140505053550-0000&executorId=9&logType=stderr
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/9/stderr
(No such file or directory)
at
java.io.FileInputStream.open(Native Method)
at
java.io.FileInputStream.<init>(FileInputStream.java:138)
at
org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at
org.eclipse.jetty.server.Server.handle(Server.java:363)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at
java.lang.Thread.run(Thread.java:722)
Thanks,
Francis.Hu
RE: 答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Posted by Francis Hu <fr...@reachjunction.com>.
We are using local files for testing, the uri of HDFS looks like : hdfs://xxxx:xx/path/to/file
From: alee526@hotmail.com
To: user@spark.apache.org
Subject: RE: 答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Date: Tue, 6 May 2014 00:29:26 -0700
Hi Francis,
Are you trying to access files on local machines or on HDFS? Any examples for the URI how it looks like?
From: francis.hu@reachjunction.com
To: user@spark.apache.org
Subject: 答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Date: Tue, 6 May 2014 12:12:24 +0800
I looked into the source code of WorkerWebUI.scala, seems there is
no file existent check
Francis
发件人: Tathagata Das
[mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 10:16
收件人: user@spark.apache.org
主题: Re: 答复: java.io.FileNotFoundException:
/test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or
directory)
Can you check the Spark worker logs on that
machine. Either from the web ui, or directly. Should be /test/spark-XXX/logs/
See if that has any error.
If there is not permission issue, I am not
why stdout and stderr is not being generated.
TD
On Mon, May 5, 2014 at 7:13 PM, Francis.Hu
<fr...@reachjunction.com>
wrote:
The file does not exist in fact and no permission issue.
francis@ubuntu-4:/test/spark-0.9.1$ ll work/app-20140505053550-0000/
total 24
drwxrwxr-x 6 francis francis 4096 May 5 05:35 ./
drwxrwxr-x 11 francis francis 4096 May 5 06:18 ../
drwxrwxr-x 2 francis francis 4096 May 5 05:35 2/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 4/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 7/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 9/
Francis
发件人: Tathagata Das [mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 3:45
收件人: user@spark.apache.org
主题: Re: java.io.FileNotFoundException:
/test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or
directory)
Do those file actually exist? Those stdout/stderr should have the
output of the spark's executors running in the workers, and its weird that they
dont exist. Could be permission issue - maybe the directories/files are not
being generated because it cannot?
TD
On Mon, May 5, 2014 at 3:06 AM, Francis.Hu <fr...@reachjunction.com>
wrote:
Hi,All
We run a spark cluster with three workers.
created a spark streaming application,
then run the spark project using below command:
shell> sbt run spark://192.168.219.129:7077 tcp://192.168.20.118:5556 foo
we looked at the webui of workers, jobs failed without any error or
info, but FileNotFoundException occurred in workers' log file as below:
Is this an existent issue of spark?
-----------------------------in workers'
logs/spark-francis-org.apache.spark.deploy.worker.Worker-1-ubuntu-4.out----------------------------
14/05/05 02:39:39 WARN AbstractHttpConnection:
/logPage/?appId=app-20140505053550-0000&executorId=2&logType=stdout
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout
(No such file or directory)
at
java.io.FileInputStream.open(Native Method)
at
java.io.FileInputStream.<init>(FileInputStream.java:138)
at
org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at
org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at
org.eclipse.jetty.server.Server.handle(Server.java:363)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at
java.lang.Thread.run(Thread.java:722)
14/05/05 02:39:41 WARN AbstractHttpConnection:
/logPage/?appId=app-20140505053550-0000&executorId=9&logType=stderr
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/9/stderr
(No such file or directory)
at
java.io.FileInputStream.open(Native Method)
at
java.io.FileInputStream.<init>(FileInputStream.java:138)
at
org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at
org.eclipse.jetty.server.Server.handle(Server.java:363)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at
java.lang.Thread.run(Thread.java:722)
Thanks,
Francis.Hu
RE: 答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Posted by Andrew Lee <al...@hotmail.com>.
Hi Francis,
Are you trying to access files on local machines or on HDFS? Any examples for the URI how it looks like?
From: francis.hu@reachjunction.com
To: user@spark.apache.org
Subject: 答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Date: Tue, 6 May 2014 12:12:24 +0800
I looked into the source code of WorkerWebUI.scala, seems there is
no file existent check
Francis
发件人: Tathagata Das
[mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 10:16
收件人: user@spark.apache.org
主题: Re: 答复: java.io.FileNotFoundException:
/test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or
directory)
Can you check the Spark worker logs on that
machine. Either from the web ui, or directly. Should be /test/spark-XXX/logs/
See if that has any error.
If there is not permission issue, I am not
why stdout and stderr is not being generated.
TD
On Mon, May 5, 2014 at 7:13 PM, Francis.Hu
<fr...@reachjunction.com>
wrote:
The file does not exist in fact and no permission issue.
francis@ubuntu-4:/test/spark-0.9.1$ ll work/app-20140505053550-0000/
total 24
drwxrwxr-x 6 francis francis 4096 May 5 05:35 ./
drwxrwxr-x 11 francis francis 4096 May 5 06:18 ../
drwxrwxr-x 2 francis francis 4096 May 5 05:35 2/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 4/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 7/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 9/
Francis
发件人: Tathagata Das [mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 3:45
收件人: user@spark.apache.org
主题: Re: java.io.FileNotFoundException:
/test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or
directory)
Do those file actually exist? Those stdout/stderr should have the
output of the spark's executors running in the workers, and its weird that they
dont exist. Could be permission issue - maybe the directories/files are not
being generated because it cannot?
TD
On Mon, May 5, 2014 at 3:06 AM, Francis.Hu <fr...@reachjunction.com>
wrote:
Hi,All
We run a spark cluster with three workers.
created a spark streaming application,
then run the spark project using below command:
shell> sbt run spark://192.168.219.129:7077 tcp://192.168.20.118:5556 foo
we looked at the webui of workers, jobs failed without any error or
info, but FileNotFoundException occurred in workers' log file as below:
Is this an existent issue of spark?
-----------------------------in workers'
logs/spark-francis-org.apache.spark.deploy.worker.Worker-1-ubuntu-4.out----------------------------
14/05/05 02:39:39 WARN AbstractHttpConnection:
/logPage/?appId=app-20140505053550-0000&executorId=2&logType=stdout
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout
(No such file or directory)
at
java.io.FileInputStream.open(Native Method)
at
java.io.FileInputStream.<init>(FileInputStream.java:138)
at
org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at
org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at
org.eclipse.jetty.server.Server.handle(Server.java:363)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at
java.lang.Thread.run(Thread.java:722)
14/05/05 02:39:41 WARN AbstractHttpConnection:
/logPage/?appId=app-20140505053550-0000&executorId=9&logType=stderr
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/9/stderr
(No such file or directory)
at
java.io.FileInputStream.open(Native Method)
at
java.io.FileInputStream.<init>(FileInputStream.java:138)
at
org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at
org.eclipse.jetty.server.Server.handle(Server.java:363)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at
java.lang.Thread.run(Thread.java:722)
Thanks,
Francis.Hu
答复: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Posted by "Francis.Hu" <fr...@reachjunction.com>.
I looked into the source code of WorkerWebUI.scala, seems there is no file existent check
Francis
发件人: Tathagata Das [mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 10:16
收件人: user@spark.apache.org
主题: Re: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Can you check the Spark worker logs on that machine. Either from the web ui, or directly. Should be /test/spark-XXX/logs/ See if that has any error.
If there is not permission issue, I am not why stdout and stderr is not being generated.
TD
On Mon, May 5, 2014 at 7:13 PM, Francis.Hu <fr...@reachjunction.com> wrote:
The file does not exist in fact and no permission issue.
francis@ubuntu-4:/test/spark-0.9.1$ ll work/app-20140505053550-0000/
total 24
drwxrwxr-x 6 francis francis 4096 May 5 05:35 ./
drwxrwxr-x 11 francis francis 4096 May 5 06:18 ../
drwxrwxr-x 2 francis francis 4096 May 5 05:35 2/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 4/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 7/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 9/
Francis
发件人: Tathagata Das [mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 3:45
收件人: user@spark.apache.org
主题: Re: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Do those file actually exist? Those stdout/stderr should have the output of the spark's executors running in the workers, and its weird that they dont exist. Could be permission issue - maybe the directories/files are not being generated because it cannot?
TD
On Mon, May 5, 2014 at 3:06 AM, Francis.Hu <fr...@reachjunction.com> wrote:
Hi,All
We run a spark cluster with three workers.
created a spark streaming application,
then run the spark project using below command:
shell> sbt run spark://192.168.219.129:7077 tcp://192.168.20.118:5556 foo
we looked at the webui of workers, jobs failed without any error or info, but FileNotFoundException occurred in workers' log file as below:
Is this an existent issue of spark?
-----------------------------in workers' logs/spark-francis-org.apache.spark.deploy.worker.Worker-1-ubuntu-4.out----------------------------
14/05/05 02:39:39 WARN AbstractHttpConnection: /logPage/?appId=app-20140505053550-0000&executorId=2&logType=stdout
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)
14/05/05 02:39:41 WARN AbstractHttpConnection: /logPage/?appId=app-20140505053550-0000&executorId=9&logType=stderr
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/9/stderr (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)
Thanks,
Francis.Hu
Re: 答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Posted by Tathagata Das <ta...@gmail.com>.
Can you check the Spark worker logs on that machine. Either from the web
ui, or directly. Should be /test/spark-XXX/logs/ See if that has any error.
If there is not permission issue, I am not why stdout and stderr is not
being generated.
TD
On Mon, May 5, 2014 at 7:13 PM, Francis.Hu <fr...@reachjunction.com>wrote:
> The file does not exist in fact and no permission issue.
>
>
>
> francis@ubuntu-4:/test/spark-0.9.1$ ll work/app-20140505053550-0000/
>
> total 24
>
> drwxrwxr-x 6 francis francis 4096 May 5 05:35 ./
>
> drwxrwxr-x 11 francis francis 4096 May 5 06:18 ../
>
> drwxrwxr-x 2 francis francis 4096 May 5 05:35 2/
>
> drwxrwxr-x 2 francis francis 4096 May 5 05:35 4/
>
> drwxrwxr-x 2 francis francis 4096 May 5 05:35 7/
>
> drwxrwxr-x 2 francis francis 4096 May 5 05:35 9/
>
>
>
> Francis
>
>
>
> *发件人:* Tathagata Das [mailto:tathagata.das1565@gmail.com]
> *发送时间:* Tuesday, May 06, 2014 3:45
> *收件人:* user@spark.apache.org
> *主题:* Re: java.io.FileNotFoundException:
> /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or
> directory)
>
>
>
> Do those file actually exist? Those stdout/stderr should have the output
> of the spark's executors running in the workers, and its weird that they
> dont exist. Could be permission issue - maybe the directories/files are not
> being generated because it cannot?
>
>
>
> TD
>
>
>
> On Mon, May 5, 2014 at 3:06 AM, Francis.Hu <fr...@reachjunction.com>
> wrote:
>
> Hi,All
>
>
>
>
>
> We run a spark cluster with three workers.
>
> created a spark streaming application,
>
> then run the spark project using below command:
>
>
>
> shell> sbt run spark://192.168.219.129:7077 tcp://192.168.20.118:5556 foo
>
>
>
> we looked at the webui of workers, jobs failed without any error or info,
> but FileNotFoundException occurred in workers' log file as below:
>
> Is this an existent issue of spark?
>
>
>
>
>
> -----------------------------in workers'
> logs/spark-francis-org.apache.spark.deploy.worker.Worker-1-ubuntu-4.out----------------------------
>
>
>
> 14/05/05 02:39:39 WARN AbstractHttpConnection:
> /logPage/?appId=app-20140505053550-0000&executorId=2&logType=stdout
>
> java.io.FileNotFoundException:
> /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or
> directory)
>
> at java.io.FileInputStream.open(Native Method)
>
> at java.io.FileInputStream.<init>(FileInputStream.java:138)
>
> at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
>
> at
> org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
>
> at
> org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
>
> at
> org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
>
> at
> org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
>
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>
> at
> org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
>
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>
> at org.eclipse.jetty.server.Server.handle(Server.java:363)
>
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
>
> at
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
>
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
>
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
>
> at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
>
> at
> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
>
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
>
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>
> at java.lang.Thread.run(Thread.java:722)
>
> 14/05/05 02:39:41 WARN AbstractHttpConnection:
> /logPage/?appId=app-20140505053550-0000&executorId=9&logType=stderr
>
> java.io.FileNotFoundException:
> /test/spark-0.9.1/work/app-20140505053550-0000/9/stderr (No such file or
> directory)
>
> at java.io.FileInputStream.open(Native Method)
>
> at java.io.FileInputStream.<init>(FileInputStream.java:138)
>
> at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
>
> at
> org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
>
> at
> org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
>
> at
> org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
>
> at
> org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
>
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>
> at
> org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
>
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>
> at org.eclipse.jetty.server.Server.handle(Server.java:363)
>
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
>
> at
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
>
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
>
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
>
> at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
>
> at
> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
>
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
>
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>
> at java.lang.Thread.run(Thread.java:722)
>
>
>
>
>
>
>
> Thanks,
>
> Francis.Hu
>
>
>
答复: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Posted by "Francis.Hu" <fr...@reachjunction.com>.
The file does not exist in fact and no permission issue.
francis@ubuntu-4:/test/spark-0.9.1$ ll work/app-20140505053550-0000/
total 24
drwxrwxr-x 6 francis francis 4096 May 5 05:35 ./
drwxrwxr-x 11 francis francis 4096 May 5 06:18 ../
drwxrwxr-x 2 francis francis 4096 May 5 05:35 2/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 4/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 7/
drwxrwxr-x 2 francis francis 4096 May 5 05:35 9/
Francis
发件人: Tathagata Das [mailto:tathagata.das1565@gmail.com]
发送时间: Tuesday, May 06, 2014 3:45
收件人: user@spark.apache.org
主题: Re: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
Do those file actually exist? Those stdout/stderr should have the output of the spark's executors running in the workers, and its weird that they dont exist. Could be permission issue - maybe the directories/files are not being generated because it cannot?
TD
On Mon, May 5, 2014 at 3:06 AM, Francis.Hu <fr...@reachjunction.com> wrote:
Hi,All
We run a spark cluster with three workers.
created a spark streaming application,
then run the spark project using below command:
shell> sbt run spark://192.168.219.129:7077 tcp://192.168.20.118:5556 foo
we looked at the webui of workers, jobs failed without any error or info, but FileNotFoundException occurred in workers' log file as below:
Is this an existent issue of spark?
-----------------------------in workers' logs/spark-francis-org.apache.spark.deploy.worker.Worker-1-ubuntu-4.out----------------------------
14/05/05 02:39:39 WARN AbstractHttpConnection: /logPage/?appId=app-20140505053550-0000&executorId=2&logType=stdout
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)
14/05/05 02:39:41 WARN AbstractHttpConnection: /logPage/?appId=app-20140505053550-0000&executorId=9&logType=stderr
java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/9/stderr (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
at org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)
Thanks,
Francis.Hu
Re: java.io.FileNotFoundException: /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout
(No such file or directory)
Posted by Tathagata Das <ta...@gmail.com>.
Do those file actually exist? Those stdout/stderr should have the output of
the spark's executors running in the workers, and its weird that they dont
exist. Could be permission issue - maybe the directories/files are not
being generated because it cannot?
TD
On Mon, May 5, 2014 at 3:06 AM, Francis.Hu <fr...@reachjunction.com>wrote:
> Hi,All
>
>
>
>
>
> We run a spark cluster with three workers.
>
> created a spark streaming application,
>
> then run the spark project using below command:
>
>
>
> shell> sbt run spark://192.168.219.129:7077 tcp://192.168.20.118:5556 foo
>
>
>
> we looked at the webui of workers, jobs failed without any error or info,
> but FileNotFoundException occurred in workers' log file as below:
>
> Is this an existent issue of spark?
>
>
>
>
>
> -----------------------------in workers'
> logs/spark-francis-org.apache.spark.deploy.worker.Worker-1-ubuntu-4.out----------------------------
>
>
>
> 14/05/05 02:39:39 WARN AbstractHttpConnection:
> /logPage/?appId=app-20140505053550-0000&executorId=2&logType=stdout
>
> java.io.FileNotFoundException:
> /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or
> directory)
>
> at java.io.FileInputStream.open(Native Method)
>
> at java.io.FileInputStream.<init>(FileInputStream.java:138)
>
> at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
>
> at
> org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
>
> at
> org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
>
> at
> org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
>
> at
> org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
>
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>
> at
> org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
>
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>
> at org.eclipse.jetty.server.Server.handle(Server.java:363)
>
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
>
> at
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
>
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
>
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
>
> at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
>
> at
> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
>
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
>
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>
> at java.lang.Thread.run(Thread.java:722)
>
> 14/05/05 02:39:41 WARN AbstractHttpConnection:
> /logPage/?appId=app-20140505053550-0000&executorId=9&logType=stderr
>
> java.io.FileNotFoundException:
> /test/spark-0.9.1/work/app-20140505053550-0000/9/stderr (No such file or
> directory)
>
> at java.io.FileInputStream.open(Native Method)
>
> at java.io.FileInputStream.<init>(FileInputStream.java:138)
>
> at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
>
> at
> org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
>
> at
> org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
>
> at
> org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52)
>
> at
> org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976)
>
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>
> at
> org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
>
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>
> at org.eclipse.jetty.server.Server.handle(Server.java:363)
>
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
>
> at
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
>
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
>
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
>
> at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
>
> at
> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
>
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
>
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>
> at java.lang.Thread.run(Thread.java:722)
>
>
>
>
>
>
>
> Thanks,
>
> Francis.Hu
>