You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Hitesh Shah (JIRA)" <ji...@apache.org> on 2014/02/27 01:11:20 UTC
[jira] [Commented] (TEZ-896) AM Split generation broken on secure
clusters when using mapred API
[ https://issues.apache.org/jira/browse/TEZ-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13913782#comment-13913782 ]
Hitesh Shah commented on TEZ-896:
---------------------------------
{code}
+ if (LOG.isDebugEnabled()) {
+ StringBuilder sb = new StringBuilder();
+ sb.append("#dagTokens=").append(dagCredentials.numberOfTokens());
+ if (dagCredentials.numberOfTokens() > 0) {
+ sb.append(", Services: ");
+ for (Token<?> t : dagCredentials.getAllTokens()) {
+ sb.append(t.getService()).append(",");
+ }
+ }
+ LOG.debug(sb.toString());
{code}
- utility log function? Same code is in 3 places.
{code}
- LOG.info("Grouping mapred api input splits");
+ LOG.info("Grouping mapred api input splits");
{code}
- Spurious change? Adding whitespace I think.
Rest of the patch looks fine.
> AM Split generation broken on secure clusters when using mapred API
> -------------------------------------------------------------------
>
> Key: TEZ-896
> URL: https://issues.apache.org/jira/browse/TEZ-896
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.3.0
> Reporter: Deepesh Khandelwal
> Assignee: Siddharth Seth
> Attachments: TEZ-896.1.txt
>
>
> On secure Hive on Tez, running a simple query fails with the following error:
> {noformat}
> hive> select count(*) from studenttab10k;
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (application id: application_1393316649566_0156)
> Map 1: -/- Reducer 2: 0/1
> Status: Failed
> Vertex failed, vertexName=Map 1, vertexId=vertex_1393316649566_0156_1_01, diagnostics=[Vertex Input: studenttab10k initializer failed., Delegation Token can be issued only with kerberos or web authentication
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getDelegationToken(FSNamesystem.java:6194)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getDelegationToken(NameNodeRpcServer.java:457)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getDelegationToken(ClientNamenodeProtocolServerSideTranslatorPB.java:886)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1973)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1969)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1967)
> ]
> Vertex killed, vertexName=Reducer 2, vertexId=vertex_1393316649566_0156_1_00, diagnostics=[Vertex received Kill in INITED state.]
> DAG failed due to vertex failure. failedVertices:1 killedVertices:1
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)