[ https://issues.apache.org/jira/browse/IMPALA-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sai Hemanth Gantasala resolved IMPALA-13126.
--------------------------------------------
Fix Version/s: Impala 4.5.0
Resolution: Fixed
> ReloadEvent.isOlderEvent() should hold the table read lock
> ----------------------------------------------------------
>
> Key: IMPALA-13126
> URL: https://issues.apache.org/jira/browse/IMPALA-13126
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Sai Hemanth Gantasala
> Priority: Critical
> Labels: catalog-2024
> Fix For: Impala 4.5.0
>
>
> Saw an exception like this:
> {noformat}
> E0601 09:11:25.275251 246 MetastoreEventsProcessor.java:990] Unexpected exception received while processing event
> Java exception follows:
> java.util.ConcurrentModificationException
> at java.util.HashMap$HashIterator.nextNode(HashMap.java:1469)
> at java.util.HashMap$ValueIterator.next(HashMap.java:1498)
> at org.apache.impala.catalog.FeFsTable$Utils.getPartitionFromThriftPartitionSpec(FeFsTable.java:616)
> at org.apache.impala.catalog.HdfsTable.getPartitionFromThriftPartitionSpec(HdfsTable.java:597)
> at org.apache.impala.catalog.Catalog.getHdfsPartition(Catalog.java:511)
> at org.apache.impala.catalog.Catalog.getHdfsPartition(Catalog.java:489)
> at org.apache.impala.catalog.CatalogServiceCatalog.isPartitionLoadedAfterEvent(CatalogServiceCatalog.java:4024)
> at org.apache.impala.catalog.events.MetastoreEvents$ReloadEvent.isOlderEvent(MetastoreEvents.java:2754)
> at org.apache.impala.catalog.events.MetastoreEvents$ReloadEvent.processTableEvent(MetastoreEvents.java:2729)
> at org.apache.impala.catalog.events.MetastoreEvents$MetastoreTableEvent.process(MetastoreEvents.java:1107)
> at org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.processIfEnabled(MetastoreEvents.java:531)
> at org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:1164)
> at org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:972)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750) {noformat}
> For a partition-level RELOAD event, ReloadEvent.isOlderEvent() needs to check whether the corresponding partition is reloaded after the event. This should be done after holding the table read lock. Otherwise, EventProcessor could hit the error above when there are concurrent DDLs/DMLs modifying the partition list.
> CC [~VenuReddy]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org