You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Richard Huang (JIRA)" <ji...@apache.org> on 2015/05/13 06:46:59 UTC
[jira] [Commented] (TEZ-494) Investigate: Exception while
submitting a dag to a TezSession
[ https://issues.apache.org/jira/browse/TEZ-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541312#comment-14541312 ]
Richard Huang commented on TEZ-494:
-----------------------------------
[~hiteshahuja]I had this issue when we have a lot of files to process (let's say around 2400 files, each is about 700M). when I pick up less partitions to process, it goes fine. I use pig on tez, here is the configuration:
MapR 2.4.1 hadoop
pig 0.14.0
hive 0.13
tez 0.5.3 (build with the mapr lib)
The job runs fine with the regular MR job though.
here is the tez output:
2015-05-13 04:40:02,999 [PigTezLauncher-0] ERROR org.apache.pig.backend.hadoop.executionengine.tez.TezJob - Cannot submit DAG - Application id: application_1430152270521_27554
org.apache.tez.dag.api.TezException: com.google.protobuf.ServiceException: java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "xxxxxxxxxx"; destination host is: "xxxxxxxxx":51956;
at org.apache.tez.client.TezClient.submitDAGSession(TezClient.java:415)
at org.apache.tez.client.TezClient.submitDAG(TezClient.java:351)
at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:162)
at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:167)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.google.protobuf.ServiceException: java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "xxxxxxxx"; destination host is: "xxxxxxxxx":51956;
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:216)
at com.sun.proxy.$Proxy13.submitDAG(Unknown Source)
at org.apache.tez.client.TezClient.submitDAGSession(TezClient.java:408)
... 8 more
Caused by: java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "xxxxxx"; destination host is: "xxxxxxxx":51956;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
at org.apache.hadoop.ipc.Client.call(Client.java:1416)
at org.apache.hadoop.ipc.Client.call(Client.java:1365)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
... 10 more
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:392)
at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1056)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:951)
2015-05-13 04:40:03,528 [main] INFO org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics:
HadoopVersion: 2.4.1-mapr-1408
PigVersion: 0.14.0-mapr-1502
TezVersion: 0.5.3
UserId: prod
FileName: xxxx.pig
StartedAt: 2015-05-13 04:35:25
FinishedAt: 2015-05-13 04:40:03
Features: GROUP_BY,FILTER,UNION
> Investigate: Exception while submitting a dag to a TezSession
> -------------------------------------------------------------
>
> Key: TEZ-494
> URL: https://issues.apache.org/jira/browse/TEZ-494
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Siddharth Seth
> Assignee: Rekha Joshi
>
> {code}
> destination host is: "local":54446;
> at org.apache.tez.client.TezSession.submitDAG(TezSession.java:139)
> at org.apache.tez.mapreduce.examples.TestBroadcast.main(TestBroadcast.java:155)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
> at org.apache.tez.mapreduce.examples.ExampleDriver.main(ExampleDriver.java:78)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> Caused by: com.google.protobuf.ServiceException: java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "local/10.11.2.219"; destination host is: "local":54446;
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:216)
> at $Proxy12.submitDAG(Unknown Source)
> at org.apache.tez.client.TezSession.submitDAG(TezSession.java:137)
> ... 13 more
> Caused by: java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "local/10.11.2.219"; destination host is: "local":54446;
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
> at org.apache.hadoop.ipc.Client.call(Client.java:1408)
> at org.apache.hadoop.ipc.Client.call(Client.java:1357)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> ... 15 more
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1049)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:944)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)