You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Dhananjay Badaya (Jira)" <ji...@apache.org> on 2021/11/10 07:16:00 UTC
[jira] [Commented] (HADOOP-13500) Concurrency issues when using
Configuration iterator
[ https://issues.apache.org/jira/browse/HADOOP-13500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17441534#comment-17441534 ]
Dhananjay Badaya commented on HADOOP-13500:
-------------------------------------------
Encountered this with Hive
{code:java}
java.util.ConcurrentModificationException at java.util.Hashtable$Enumerator.next(Hashtable.java:1387) at org.apache.hadoop.conf.Configuration.iterator(Configuration.java:2625) at org.apache.hadoop.hive.conf.HiveConf.getProperties(HiveConf.java:4047) at org.apache.hadoop.hive.conf.HiveConf.getAllProperties(HiveConf.java:4043) at org.apache.hadoop.hive.conf.HiveConf.getChangedProperties(HiveConf.java:4362) at org.apache.hadoop.hive.ql.exec.Utilities.isDefaultNameNode(Utilities.java:3572) at org.apache.hadoop.hive.ql.plan.CreateTableDesc.toTable(CreateTableDesc.java:795) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4321) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:354) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100){code}
As [~jlowe] mentioned earlier the fix could be to wrap this [this|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L2964] for loop in a *synchronized (properties)* block.
If this solution looks acceptable then I will be happy to pick this up.
> Concurrency issues when using Configuration iterator
> ----------------------------------------------------
>
> Key: HADOOP-13500
> URL: https://issues.apache.org/jira/browse/HADOOP-13500
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Jason Darrell Lowe
> Priority: Major
>
> It is possible to encounter a ConcurrentModificationException while trying to iterate a Configuration object. The iterator method tries to walk the underlying Property object without proper synchronization, so another thread simultaneously calling the set method can trigger it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org