You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by ZongtianHou <zo...@icloud.com.INVALID> on 2018/07/07 01:09:34 UTC
Libhdfs question
Hi, everyone:
I used a libhdfs3 client to access the secure hdfs. When I get data from it, occasionally there will be an error, below is the log. It seems the the rpc use the token 775 which has been canceled to access the hdfs, but I already create a token 776 and use it to do the hdfs filesystem connect. Why will this happen, or can someone give me some clue about the reason.
In the libhdfs3 part
RpcChannel.cpp: 848: HdfsRpcServerException: org.apache.hadoop.security.token.SecretManager$InvalidToken: token (HDFS_DELEGATION_TOKEN token 775 for kousouda) can't be found in cache
@ Hdfs::Internal::RpcChannelImpl::readOneResponse(bool)
@ Hdfs::Internal::RpcChannelImpl::setupSaslConnection()
@ Hdfs::Internal::RpcChannelImpl::connect()
@ Hdfs",,,,,,,,"SysLoggerMain","syslogger.c",518,
2018-07-07 00:45:47.172357 CST,,,p73607,th-1339755584,,,,0,,,seg-10000,,,,,"LOG","00000","3rd party error log:
::Internal::RpcChannelImpl::invokeInternal(std::__1::shared_ptr<Hdfs::Internal::RpcRemoteCall>)
@ Hdfs::Internal::RpcChannelImpl::invoke(Hdfs::Internal::RpcCall const&)
@ Hdfs::Internal::NamenodeImpl::invoke(Hdfs::Internal::RpcCall const&)
@ Hdfs::Internal::NamenodeImpl::getBlockLocations(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, long long, long long, Hdfs::Internal::LocatedBlocks&)
@ Hdfs::Internal::NamenodeProxy::getBlockLocations(std::__1::basic_str",,,,,,,,"SysLoggerMain","syslogger.c",518,
2018-07-07 00:45:47.172411 CST,,,p73607,th-1339755584,,,,0,,,seg-10000,,,,,"LOG","00000","3rd party error log:
ing<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, long long, long long, Hdfs::Internal::LocatedBlocks&)
@ Hdfs::Internal::FileSystemImpl::getBlockLocations(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, long long, long long, Hdfs::Internal::LocatedBlocks&)
@ Hdfs::Internal::InputStreamImpl::updateBlockInfos()
The namenode part
2018-07-07 00:44:55,572 INFO SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for kousouda@CW.COM (auth:KERBEROS) for protocol=interface org.apache.hadoop.hdfs.protocol.ClientProtocol
2018-07-07 00:44:55,572 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Token cancellation requested for identifier: HDFS_DELEGATION_TOKEN token 775 for kousouda
2018-07-07 00:44:55,573 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2804 Total time for transactions(ms): 478 Number of transactions batched in Syncs: 89 Number of syncs: 2142 SyncTimes(ms): 1063
2018-07-07 00:45:11,112 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for kousouda@CW.COM (auth:KERBEROS)
2018-07-07 00:45:11,114 INFO SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for kousouda@CW.COM (auth:KERBEROS) for protocol=interface org.apache.hadoop.hdfs.protocol.ClientProtocol 2018-07-07 00:45:11,132 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Creating password for identifier: HDFS_DELEGATION_TOKEN token 776 for kousouda, currentKey: 14 2018-07-07 00:45:43,430 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for kousouda@CW.COM (auth:TOKEN)
2018-07-07 00:45:43,431 INFO SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for kousouda@CW.COM (auth:TOKEN) for protocol=interface org.apache.hadoop.hdfs.protocol.ClientProtocol
2018-07-07 00:45:44,222 WARN SecurityLogger.org.apache.hadoop.ipc.Server: Auth failed for 127.0.0.1:62651:null (DIGEST-MD5: IO error acquiring password) 2018-07-07 00:45:44,222 INFO org.apache.hadoop.ipc.Server: Socket Reader #1 for port 8020: readAndProcess from client 127.0.0.1 threw exception [org.apache.hadoop.security.token.SecretManager$InvalidToken: token (HDFS_DELEGATION_TOKEN token 775 for kousouda) can't be found in cache]
2018-07-07 00:45:45,213 WARN SecurityLogger.org.apache.hadoop.ipc.Server: Auth failed for 127.0.0.1:62652:null (DIGEST-MD5: IO error acquiring password)
2018-07-07 00:45:45,213 INFO org.apache.hadoop.ipc.Server: Socket Reader #1 for port 8020: readAndProcess from client 127.0.0.1 threw exception [org.apache.hadoop.security.token.SecretManager$InvalidToken: token (HDFS_DELEGATION_TOKEN token 775 for kousouda) can't be found in cache]
2018-07-07 00:45:45,994 WARN SecurityLogger.org.apache.hadoop.ipc.Server: Auth failed for 127.0.0.1:62653:null (DIGEST-MD5: IO error acquiring password)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org