You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Li Lu (JIRA)" <ji...@apache.org> on 2016/05/09 23:32:13 UTC
[jira] [Commented] (HADOOP-12563) Updated utility to create/modify
token files
[ https://issues.apache.org/jira/browse/HADOOP-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15277321#comment-15277321 ]
Li Lu commented on HADOOP-12563:
--------------------------------
I noticed the following errors when running tez on our trunk code:
{code}
2016-05-09 16:12:18,733 [ERROR] [main] |app.DAGAppMaster|: Error starting DAGAppMaster
java.io.IOException: Exception reading /tmp/hadoop-llu/nm-local-dir/usercache/llu/appcache/application_1462833671675_0001/container_e03_1462833671675_0001_01_000001/container_tokens
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:197)
at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:789)
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:748)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:621)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2345)
Caused by: java.io.IOException: Unknown version 1 in token storage.
at org.apache.hadoop.security.Credentials.readTokenStorageStream(Credentials.java:215)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:194)
... 4 more
{code}
Both Tez and Hadoop are from the latest master/trunk.
Looks like we introduced some incompatible changes in this JIRA? I marked this JIRA as an incompatible change for now.
> Updated utility to create/modify token files
> --------------------------------------------
>
> Key: HADOOP-12563
> URL: https://issues.apache.org/jira/browse/HADOOP-12563
> Project: Hadoop Common
> Issue Type: New Feature
> Affects Versions: 3.0.0
> Reporter: Allen Wittenauer
> Assignee: Matthew Paduano
> Fix For: 3.0.0
>
> Attachments: HADOOP-12563.01.patch, HADOOP-12563.02.patch, HADOOP-12563.03.patch, HADOOP-12563.04.patch, HADOOP-12563.05.patch, HADOOP-12563.06.patch, HADOOP-12563.07.patch, HADOOP-12563.07.patch, HADOOP-12563.08.patch, HADOOP-12563.09.patch, HADOOP-12563.10.patch, HADOOP-12563.11.patch, HADOOP-12563.12.patch, HADOOP-12563.13.patch, HADOOP-12563.14.patch, HADOOP-12563.15.patch, HADOOP-12563.16.patch, dtutil-test-out, example_dtutil_commands_and_output.txt, generalized_token_case.pdf
>
>
> hdfs fetchdt is missing some critical features and is geared almost exclusively towards HDFS operations. Additionally, the token files that are created use Java serializations which are hard/impossible to deal with in other languages. It should be replaced with a better utility in common that can read/write protobuf-based token files, has enough flexibility to be used with other services, and offers key functionality such as append and rename. The old version file format should still be supported for backward compatibility, but will be effectively deprecated.
> A follow-on JIRA will deprecrate fetchdt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org