You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Eric Badger (Jira)" <ji...@apache.org> on 2020/04/02 19:46:00 UTC
[jira] [Updated] (HDFS-14986) ReplicaCachingGetSpaceUsed throws
ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HDFS-14986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Badger updated HDFS-14986:
-------------------------------
Fix Version/s: (was: 3.2.2)
[~weichiu], cherry-picking this patch to branch-3.2 broke compilation. I have reverted it from branch-3.2. I see that you cherry-picked several other patches to branch-3.2 after this one. Please compile to make sure that you don't unintentionally break the build and cause other developers to spend time fixing it. [~cliang], you also committed a patch to branch-3.2 (albeit in YARN, not HDFS) after this patch had broken compilation. I know it's annoying to compile every little change, but it's pretty frustrating having to track down the patch that broke compilation, revert it, and update the relevant JIRAs.
> ReplicaCachingGetSpaceUsed throws ConcurrentModificationException
> ------------------------------------------------------------------
>
> Key: HDFS-14986
> URL: https://issues.apache.org/jira/browse/HDFS-14986
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode, performance
> Affects Versions: 2.10.0
> Reporter: Ryan Wu
> Assignee: Aiphago
> Priority: Major
> Fix For: 3.3.0, 3.1.4, 2.10.1
>
> Attachments: HDFS-14986.001.patch, HDFS-14986.002.patch, HDFS-14986.003.patch, HDFS-14986.004.patch, HDFS-14986.005.patch, HDFS-14986.006.patch
>
>
> Running DU across lots of disks is very expensive . We applied the patch HDFS-14313 to get used space from ReplicaInfo in memory.However, new du threads throw the exception
> {code:java}
> // 2019-11-08 18:07:13,858 ERROR [refreshUsed-/home/vipshop/hard_disk/7/dfs/dn/current/BP-1203969992-XXXX-1450855658517] org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed: ReplicaCachingGetSpaceUsed refresh error
> java.util.ConcurrentModificationException: Tree has been modified outside of iterator
> at org.apache.hadoop.hdfs.util.FoldedTreeSet$TreeSetIterator.checkForModification(FoldedTreeSet.java:311)
> at org.apache.hadoop.hdfs.util.FoldedTreeSet$TreeSetIterator.hasNext(FoldedTreeSet.java:256)
> at java.util.AbstractCollection.addAll(AbstractCollection.java:343)
> at java.util.HashSet.<init>(HashSet.java:120)
> at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.deepCopyReplica(FsDatasetImpl.java:1052)
> at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed.refresh(ReplicaCachingGetSpaceUsed.java:73)
> at org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:178)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org