You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "He Xiaoqiao (JIRA)" <ji...@apache.org> on 2018/11/19 09:32:00 UTC
[jira] [Commented] (HDFS-12862) CacheDirective may invalidata,when
NN restart or make a transition to Active.
[ https://issues.apache.org/jira/browse/HDFS-12862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691458#comment-16691458 ]
He Xiaoqiao commented on HDFS-12862:
------------------------------------
LGTM for [^HDFS-12862-trunk.003.patch] , ping [~daryn],[~jojochuang] Do you mind another review?
> CacheDirective may invalidata,when NN restart or make a transition to Active.
> -----------------------------------------------------------------------------
>
> Key: HDFS-12862
> URL: https://issues.apache.org/jira/browse/HDFS-12862
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: caching, hdfs
> Affects Versions: 2.7.1
> Environment:
> Reporter: Wang XL
> Priority: Major
> Labels: patch
> Attachments: HDFS-12862-branch-2.7.1.001.patch, HDFS-12862-trunk.002.patch, HDFS-12862-trunk.003.patch
>
>
> The logic in FSNDNCacheOp#modifyCacheDirective is not correct. when modify cacheDirective,the expiration in directive may be a relative expiryTime, and EditLog will serial a relative expiry time.
> {code:java}
> // Some comments here
> static void modifyCacheDirective(
> FSNamesystem fsn, CacheManager cacheManager, CacheDirectiveInfo directive,
> EnumSet<CacheFlag> flags, boolean logRetryCache) throws IOException {
> final FSPermissionChecker pc = getFsPermissionChecker(fsn);
> cacheManager.modifyDirective(directive, pc, flags);
> fsn.getEditLog().logModifyCacheDirectiveInfo(directive, logRetryCache);
> }
> {code}
> But when SBN replay the log ,it will invoke FSImageSerialization#readCacheDirectiveInfo as a absolute expiryTime.It will result in the inconsistency .
> {code:java}
> public static CacheDirectiveInfo readCacheDirectiveInfo(DataInput in)
> throws IOException {
> CacheDirectiveInfo.Builder builder =
> new CacheDirectiveInfo.Builder();
> builder.setId(readLong(in));
> int flags = in.readInt();
> if ((flags & 0x1) != 0) {
> builder.setPath(new Path(readString(in)));
> }
> if ((flags & 0x2) != 0) {
> builder.setReplication(readShort(in));
> }
> if ((flags & 0x4) != 0) {
> builder.setPool(readString(in));
> }
> if ((flags & 0x8) != 0) {
> builder.setExpiration(
> CacheDirectiveInfo.Expiration.newAbsolute(readLong(in)));
> }
> if ((flags & ~0xF) != 0) {
> throw new IOException("unknown flags set in " +
> "ModifyCacheDirectiveInfoOp: " + flags);
> }
> return builder.build();
> }
> {code}
> In other words， fsn.getEditLog().logModifyCacheDirectiveInfo(directive, logRetryCache) may serial a relative expiry time,But builder.setExpiration(CacheDirectiveInfo.Expiration.newAbsolute(readLong(in))) read it as a absolute expiryTime.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org