You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Joe McDonnell (Jira)" <ji...@apache.org> on 2026/07/23 19:52:00 UTC
[jira] [Commented] (IMPALA-14967) Event processor hits NullPointerException for CatalogOpExecutor.removeTableIfNotAddedLater() and goes into ERROR state
[ https://issues.apache.org/jira/browse/IMPALA-14967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18098585#comment-18098585 ]
Joe McDonnell commented on IMPALA-14967:
----------------------------------------
Saw this again on an exhaustive job. Several tests fail during the end-to-end parallel tests.
{noformat}
E20260721 05:39:41.380749 3275089 MetastoreEvents.java:980] EventId: 58870 EventType: DROP_TABLE Target: test_convert_table_36ca0746.special_chars. Failed to handle event processing failure for table: test_convert_table_36ca0746.special_chars
Java exception follows:
java.lang.NullPointerException: Cannot invoke "org.apache.impala.catalog.Table.setCatalogVersion(long)" because "removedTbl" is null
at org.apache.impala.service.CatalogOpExecutor.removeTableIfNotAddedLater(CatalogOpExecutor.java:874)
at org.apache.impala.catalog.events.RenameTableBarrierEvent.process(RenameTableBarrierEvent.java:209)
at org.apache.impala.catalog.events.RenameTableBarrierEvent.processIfEnabled(RenameTableBarrierEvent.java:192)
at org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.processEvent(TableEventExecutor.java:320)
at org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.process(TableEventExecutor.java:347)
at org.apache.impala.catalog.events.TableEventExecutor.process(TableEventExecutor.java:499)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
E20260721 05:39:41.382032 3275089 TableEventExecutor.java:502] Exception occurred for executor: DbEventExecutor-1.TableEventExecutor-3
E20260721 05:39:41.382186 3275089 MetastoreEventsProcessor.java:1407] Unexpected exception received while processing event
Java exception follows:
java.lang.NullPointerException: Cannot invoke "org.apache.impala.catalog.Table.setCatalogVersion(long)" because "removedTbl" is null
at org.apache.impala.service.CatalogOpExecutor.removeTableIfNotAddedLater(CatalogOpExecutor.java:874)
at org.apache.impala.catalog.events.RenameTableBarrierEvent.process(RenameTableBarrierEvent.java:209)
at org.apache.impala.catalog.events.RenameTableBarrierEvent.processIfEnabled(RenameTableBarrierEvent.java:192)
at org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.processEvent(TableEventExecutor.java:320)
at org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.process(TableEventExecutor.java:347)
at org.apache.impala.catalog.events.TableEventExecutor.process(TableEventExecutor.java:499)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
E20260721 05:39:41.382831 3275089 MetastoreEventsProcessor.java:1783] Event id: 58870
Event Type: DROP_TABLE
Event time: 0
Database name: test_convert_table_36ca0746
Table name: special_chars
Event message: null
... lots of time ...
E20260721 06:17:51.285123 3275924 CatalogServiceCatalog.java:4733] 344d1e32ecef548b:7f1a377400000000] Current state of HMS event processor is ERROR{noformat}
The test failures are all after this point, and have a variety of symptoms (tables missing, event processor in error state, etc).
> Event processor hits NullPointerException for CatalogOpExecutor.removeTableIfNotAddedLater() and goes into ERROR state
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-14967
> URL: https://issues.apache.org/jira/browse/IMPALA-14967
> Project: IMPALA
> Issue Type: Bug
> Reporter: Pranav Yogi Lodha
> Priority: Critical
> Labels: broken-build, flaky
>
> h3. Error Message
> Exception: Event processor is not working. Status: ERROR. Error msg: 2026-05-01T06:48:43.942372438 Unexpected exception received while processing event java.lang.NullPointerException: Cannot invoke "org.apache.impala.catalog.Table.setCatalogVersion(long)" because "removedTbl" is null at org.apache.impala.service.CatalogOpExecutor.removeTableIfNotAddedLater(CatalogOpExecutor.java:874) at org.apache.impala.catalog.events.RenameTableBarrierEvent.process(RenameTableBarrierEvent.java:209) at org.apache.impala.catalog.events.RenameTableBarrierEvent.processIfEnabled(RenameTableBarrierEvent.java:192) at org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.processEvent(TableEventExecutor.java:320) at org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.process(TableEventExecutor.java:347) at org.apache.impala.catalog.events.TableEventExecutor.process(TableEventExecutor.java:499) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840) Event id: 46381 Event Type: DROP_TABLE Event time: 0 Database name: test_count_star_optimization_in_complex_query_a19cc8f9 Table name: expected_merge_exclude_columns Event message: null
> h3. Stacktrace
> /data/jenkins/workspace/impala-cdh_main_staging-core-ubsan-arm/repos/Impala/tests/metadata/test_testcase_builder.py:38: in test_query_without_from self._test_export_and_import(0, 0, 0, "SELECT 5 * 20") self = <tests.metadata.test_testcase_builder.TestTestcaseBuilder object at 0xffff92224d50> /data/jenkins/workspace/impala-cdh_main_staging-core-ubsan-arm/repos/Impala/tests/metadata/test_testcase_builder.py:102: in _test_export_and_import EventProcessorUtils.wait_for_event_processing(self) hdfs_path = '/tmp/impala-testcase-data-d388835f-83ef-4ee3-afd3-34950744b114' index = 23 num_dbs = 0 num_tbls = 0 num_views = 0 original_plan = ['Max Per-Host Resource Reservation: Memory=0B Threads=1', 'Per-Host Resource Estimates: Memory=10MB', 'Codegen disabl...ner', 'Analyzed query: SELECT CAST(100 AS SMALLINT)', '', 'F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1', ...] query = 'SELECT 5 * 20' result = <tests.common.impala_connection.ImpylaHS2ResultSet object at 0xffff921c2310> self = <tests.metadata.test_testcase_builder.TestTestcaseBuilder object at 0xffff92224d50> tbls = () testcase_generate_query = "COPY TESTCASE TO '/tmp' SELECT 5 * 20" testcase_path = "'hdfs://localhost:20500/tmp/impala-testcase-data-d388835f-83ef-4ee3-afd3-34950744b114'" tmp_path = '/tmp' views = () util/event_processor_utils.py:92: in wait_for_event_processing EventProcessorUtils.wait_for_event_processing_impl(test_suite.hive_client, error_status_possible = False impala_cluster = <tests.common.impala_cluster.ImpalaCluster object at 0xffff91d948d0> test_suite = <tests.metadata.test_testcase_builder.TestTestcaseBuilder object at 0xffff92224d50> timeout = 10 util/event_processor_utils.py:106: in wait_for_event_processing_impl EventProcessorUtils.wait_for_synced_event_id(timeout, error_status_possible, current_event_id = 49314 error_status_possible = False hive_client = <impala_thrift_gen.hive_metastore.ThriftHiveMetastore.Client object at 0xffff91b49310> impala_cluster = <tests.common.impala_cluster.ImpalaCluster object at 0xffff91d948d0> timeout = 10 util/event_processor_utils.py:67: in wait_for_synced_event_id raise Exception("Event processor is not working. Status: \{0}. Error msg: \{1}" E Exception: Event processor is not working. Status: ERROR. Error msg: 2026-05-01T06:48:43.942372438 E Unexpected exception received while processing event E java.lang.NullPointerException: Cannot invoke "org.apache.impala.catalog.Table.setCatalogVersion(long)" because "removedTbl" is null E at org.apache.impala.service.CatalogOpExecutor.removeTableIfNotAddedLater(CatalogOpExecutor.java:874) E at org.apache.impala.catalog.events.RenameTableBarrierEvent.process(RenameTableBarrierEvent.java:209) E at org.apache.impala.catalog.events.RenameTableBarrierEvent.processIfEnabled(RenameTableBarrierEvent.java:192) E at org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.processEvent(TableEventExecutor.java:320) E at org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.process(TableEventExecutor.java:347) E at org.apache.impala.catalog.events.TableEventExecutor.process(TableEventExecutor.java:499) E at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) E at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) E at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) E at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) E at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) E at java.base/java.lang.Thread.run(Thread.java:840) E E Event id: 46381 E Event Type: DROP_TABLE E Event time: 0 E Database name: test_count_star_optimization_in_complex_query_a19cc8f9 E Table name: expected_merge_exclude_columns E Event message: null TIMEOUT_MULTIPLIER_IF_THERE_IS_PROGRESS = 10 current_synced_id = 46380 end_time = 1777644392.8926485 error_msg = '2026-05-01T06:48:43.942372438\nUnexpected exception received while processing event\njava.lang.NullPointerException: ...est_count_star_optimization_in_complex_query_a19cc8f9\nTable name: expected_merge_exclude_columns\nEvent message: null' error_status_possible = False last_synced_id = 46380 last_synced_time = 1777644292.89848 status = 'ERROR' status_list = ['ACTIVE', 'PAUSED'] t = 1777644292.8984802 target_event_id = 49314 timeout = 10 total_timeot = 100
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org