You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Alex Wang <he...@gmail.com> on 2019/10/17 13:05:35 UTC
Exception in thread "main" org.codehaus.jackson.map.exc.UnrecognizedPropertyException:
Unrecognized field "Token" (Class org.apache.hadoop.yarn.api.records.timeline.TimelineDelegationTokenResponse),
not marked as ignorable
Hello everyone:
Our hadoop , hbase cluster has Kerberos authentication enabled.
The hadoop version is 2.7.3 and the hbase version is 1.3.5.
1. Kinit initializes the ticket.
Ticket cache: FILE:/tmp/krb5cc_1111
Default principal: myuser@XXXXXX.COM
Valid starting Expires Service principal
10/17/2019 18:00:38 10/18/2019 18:00:38 krbtgt/XXXXXX.COM@XXXXXX.COM
Renew until 10/24/2019 18:00:38
2. hbase org.apache.hadoop.hbase.mapreduce.ImportTsv
-Dimporttsv.separator=, -Dimporttsv.columns="HBASE_ROW_KEY,
id,temp:in,temp:out,vibration,pressure:in,pressure:out" sensor /
Tmp/hbase.csv
The error is as follows. Can someone give me some advice?
Exception in thread "main"
org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
field "Token" (Class
org.apache.hadoop.yarn.api.records.timeline.TimelineDelegationTokenResponse),
not marked as ignorable
At [Source: N/A; line: -1, column: -1] (through reference chain:
org.apache.hadoop.yarn.api.records.timeline.TimelineDelegationTokenResponse["Token"])
At
org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
At
org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
At
org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
At
org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
At
org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
At
org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
At
org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
At
org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
At org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
At org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1999)
At
org.apache.hadoop.yarn.client.api.impl.TimelineAuthenticator.validateAndParseResponse(TimelineAuthenticator.java:222)
At
org.apache.hadoop.yarn.client.api.impl.TimelineAuthenticator.getDelegationToken(TimelineAuthenticator.java:114)
At
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.getDelegationToken(TimelineClientImpl.java:167)
At
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.addTimelineDelegationToken(YarnClientImpl.java:275)
At
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:221)
At
org.apache.hadoop.mapred.ResourceMgrDelegate.submitApplication(ResourceMgrDelegate.java:282)
At org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:289)
At
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)
At org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
At org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
At java.security.AccessController.doPrivileged(Native Method)
At javax.security.auth.Subject.doAs(Subject.java:422)
At
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
At org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
At org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
At org.apache.hadoop.hbase.mapreduce.ImportTsv.run(ImportTsv.java:782)
At org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
At org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
At org.apache.hadoop.hbase.mapreduce.ImportTsv.main(ImportTsv.java:794)
--
Best
Re: Exception in thread "main" org.codehaus.jackson.map.exc.UnrecognizedPropertyException:
Unrecognized field "Token" (Class org.apache.hadoop.yarn.api.records.timeline.TimelineDelegationTokenResponse),
not marked as ignorable
Posted by Alex Wang <he...@gmail.com>.
Hi Prabhu:
Thank you for your reply. I checked the environment of our cluster.
I looked at the "Hadoop version support matrix" and found that
1.3.x supports Hadoop-2.7.1+
Reference: http://hbase.apache.org/book.html#hadoop
Then I looked at the pom file of the hbase source. The hadoop
version of hbase 1.3.5 dependency is 2.5.1.
See https://github.com/apache/hbase/blob/rel%2F1.3.5/pom.xml#L1246
BTW:
In the error message, at
org.apache.hadoop.yarn.client.api.impl.TimelineAuthenticator.validateAndParseResponse(TimelineAuthenticator.java:222)
I found that the TimelineAuthenticator.java file was removed since
hadoop2.6.
Reference: https://jira.apache.org/jira/secure/attachment/12675639/YARN-2676.5.patch
Thank you for your correct answer. Thanks again.
Prabhu Josephraj <pj...@cloudera.com> 于2019年10月17日周四 下午10:25写道：
>
> Suspect the TimelineClient and ApplicationHistoryServer are using different hadoop libraries. Can you make sure the client uses the same hadoop jars and
> dependency jars as the ApplicationHistoryServer process. Simple workaround is to disable timeline service for this job.
>
> hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dyarn.timeline-service.enabled=false -Dimporttsv.separator=, -Dimporttsv.columns="HBASE_ROW_KEY, id,temp:in,temp:out,vibration,pressure:in,pressure:out" sensor / Tmp/hbase.csv
>
>
> On Thu, Oct 17, 2019 at 6:36 PM Alex Wang <he...@gmail.com> wrote:
>>
>> Hello everyone:
>> Our hadoop , hbase cluster has Kerberos authentication enabled.
>> The hadoop version is 2.7.3 and the hbase version is 1.3.5.
>>
>> 1. Kinit initializes the ticket.
>> Ticket cache: FILE:/tmp/krb5cc_1111
>> Default principal: myuser@XXXXXX.COM
>>
>> Valid starting Expires Service principal
>> 10/17/2019 18:00:38 10/18/2019 18:00:38 krbtgt/XXXXXX.COM@XXXXXX.COM
>> Renew until 10/24/2019 18:00:38
>>
>> 2. hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator=, -Dimporttsv.columns="HBASE_ROW_KEY, id,temp:in,temp:out,vibration,pressure:in,pressure:out" sensor / Tmp/hbase.csv
>>
>> The error is as follows. Can someone give me some advice?
>>
>> Exception in thread "main" org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "Token" (Class org.apache.hadoop.yarn.api.records.timeline.TimelineDelegationTokenResponse), not marked as ignorable
>> At [Source: N/A; line: -1, column: -1] (through reference chain: org.apache.hadoop.yarn.api.records.timeline.TimelineDelegationTokenResponse["Token"])
>> At org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
>> At org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
>> At org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
>> At org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
>> At org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
>> At org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
>> At org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
>> At org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>> At org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
>> At org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1999)
>> At org.apache.hadoop.yarn.client.api.impl.TimelineAuthenticator.validateAndParseResponse(TimelineAuthenticator.java:222)
>> At org.apache.hadoop.yarn.client.api.impl.TimelineAuthenticator.getDelegationToken(TimelineAuthenticator.java:114)
>> At org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.getDelegationToken(TimelineClientImpl.java:167)
>> At org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.addTimelineDelegationToken(YarnClientImpl.java:275)
>> At org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:221)
>> At org.apache.hadoop.mapred.ResourceMgrDelegate.submitApplication(ResourceMgrDelegate.java:282)
>> At org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:289)
>> At org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)
>> At org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
>> At org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
>> At java.security.AccessController.doPrivileged(Native Method)
>> At javax.security.auth.Subject.doAs(Subject.java:422)
>> At org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
>> At org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
>> At org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
>> At org.apache.hadoop.hbase.mapreduce.ImportTsv.run(ImportTsv.java:782)
>> At org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> At org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> At org.apache.hadoop.hbase.mapreduce.ImportTsv.main(ImportTsv.java:794)
>>
>>
>> --
>> Best
--
Best
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Exception in thread "main" org.codehaus.jackson.map.exc.UnrecognizedPropertyException:
Unrecognized field "Token" (Class org.apache.hadoop.yarn.api.records.timeline.TimelineDelegationTokenResponse),
not marked as ignorable
Posted by Prabhu Josephraj <pj...@cloudera.com.INVALID>.
Suspect the TimelineClient and ApplicationHistoryServer are using different
hadoop libraries. Can you make sure the client uses the same hadoop jars
and
dependency jars as the ApplicationHistoryServer process. Simple workaround
is to disable timeline service for this job.
hbase org.apache.hadoop.hbase.mapreduce.ImportTsv
*-Dyarn.timeline-service.enabled=false* -Dimporttsv.separator=,
-Dimporttsv.columns="HBASE_ROW_KEY,
id,temp:in,temp:out,vibration,pressure:in,pressure:out"
sensor / Tmp/hbase.csv
On Thu, Oct 17, 2019 at 6:36 PM Alex Wang <he...@gmail.com> wrote:
> Hello everyone:
> Our hadoop , hbase cluster has Kerberos authentication enabled.
> The hadoop version is 2.7.3 and the hbase version is 1.3.5.
>
> 1. Kinit initializes the ticket.
> Ticket cache: FILE:/tmp/krb5cc_1111
> Default principal: myuser@XXXXXX.COM
>
> Valid starting Expires Service principal
> 10/17/2019 18:00:38 10/18/2019 18:00:38 krbtgt/XXXXXX.COM@XXXXXX.COM
> Renew until 10/24/2019 18:00:38
>
> 2. hbase org.apache.hadoop.hbase.mapreduce.ImportTsv
> -Dimporttsv.separator=, -Dimporttsv.columns="HBASE_ROW_KEY,
> id,temp:in,temp:out,vibration,pressure:in,pressure:out" sensor /
> Tmp/hbase.csv
>
> The error is as follows. Can someone give me some advice?
>
> Exception in thread "main"
> org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
> field "Token" (Class
> org.apache.hadoop.yarn.api.records.timeline.TimelineDelegationTokenResponse),
> not marked as ignorable
> At [Source: N/A; line: -1, column: -1] (through reference chain:
> org.apache.hadoop.yarn.api.records.timeline.TimelineDelegationTokenResponse["Token"])
> At
> org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
> At
> org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
> At
> org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
> At
> org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
> At
> org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
> At
> org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
> At
> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
> At
> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
> At org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
> At org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1999)
> At
> org.apache.hadoop.yarn.client.api.impl.TimelineAuthenticator.validateAndParseResponse(TimelineAuthenticator.java:222)
> At
> org.apache.hadoop.yarn.client.api.impl.TimelineAuthenticator.getDelegationToken(TimelineAuthenticator.java:114)
> At
> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.getDelegationToken(TimelineClientImpl.java:167)
> At
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.addTimelineDelegationToken(YarnClientImpl.java:275)
> At
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:221)
> At
> org.apache.hadoop.mapred.ResourceMgrDelegate.submitApplication(ResourceMgrDelegate.java:282)
> At org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:289)
> At
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)
> At org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> At org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> At java.security.AccessController.doPrivileged(Native Method)
> At javax.security.auth.Subject.doAs(Subject.java:422)
> At
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
> At org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> At org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
> At org.apache.hadoop.hbase.mapreduce.ImportTsv.run(ImportTsv.java:782)
> At org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> At org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> At org.apache.hadoop.hbase.mapreduce.ImportTsv.main(ImportTsv.java:794)
>
>
> --
> Best
>