You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Mihaly Szjatinya (Jira)" <ji...@apache.org> on 2026/08/11 11:14:00 UTC
[jira] [Updated] (IMPALA-15254) test_enc_dec_gen_compress in test_charcodec.py fails with AnalysisException when HMS event processor is slow
[ https://issues.apache.org/jira/browse/IMPALA-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mihaly Szjatinya updated IMPALA-15254:
--------------------------------------
Description:
`prepare_encoded_test_table_compress()` in `tests/query_test/test_charcodec.py` creates a table and loads data via Hive (`run_stmt_in_hive`) but never calls `INVALIDATE METADATA` or `REFRESH` afterward. Normally Impala picks up Hive-created tables automatically via the HMS event processor. However, when the event processor is lagging or has crashed (e.g. during long exhaustive runs on resource-constrained environments), the asynchronous propagation may not complete before `_compare_tables` queries the table, causing:
{code:java}
AnalysisException: Could not resolve table reference: '<db>.<charset>_gen_None'
{code}
*Fix:* Add {{INVALIDATE METADATA
{db}
.
{encoded_table}
}} (or {{{}REFRESH{}}}) at the end of `prepare_encoded_test_table_compress`, consistent with the other helpers.
was:
`prepare_encoded_test_table_compress()` in `tests/query_test/test_charcodec.py` creates a table and loads data via Hive (`run_stmt_in_hive`) but never calls `INVALIDATE METADATA` or `REFRESH` afterward. Normally Impala picks up Hive-created tables automatically via the HMS event processor. However, when the event processor is lagging or has crashed (e.g. during long exhaustive runs on resource-constrained environments), the asynchronous propagation may not complete before `_compare_tables` queries the table, causing:
{code}
AnalysisException: Could not resolve table reference: '<db>.<charset>_gen_None'
{code}
All other {{prepare_*}} helpers in the same file explicitly sync the catalog after Hive writes:
- {{prepare_utf8_test_table}} -- calls REFRESH (line 117)
- {{prepare_encoded_test_table}} -- calls REFRESH (line 129)
- {{prepare_utf8_test_table_partitions}} -- calls REFRESH (line 223)
- {{prepare_encoded_test_table_partitions}} -- calls REFRESH (line 240)
{{prepare_encoded_test_table_compress}} (line 173) is the only one that omits this step. This is a test-reliability bug, not a product bug.
*Fix:* Add {{INVALIDATE METADATA {db}.{encoded_table}}} (or {{REFRESH}}) at the end of `prepare_encoded_test_table_compress`, consistent with the other helpers.
> test_enc_dec_gen_compress in test_charcodec.py fails with AnalysisException when HMS event processor is slow
> ------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-15254
> URL: https://issues.apache.org/jira/browse/IMPALA-15254
> Project: IMPALA
> Issue Type: Bug
> Reporter: Mihaly Szjatinya
> Assignee: Mihaly Szjatinya
> Priority: Major
> Labels: test-failure
>
> `prepare_encoded_test_table_compress()` in `tests/query_test/test_charcodec.py` creates a table and loads data via Hive (`run_stmt_in_hive`) but never calls `INVALIDATE METADATA` or `REFRESH` afterward. Normally Impala picks up Hive-created tables automatically via the HMS event processor. However, when the event processor is lagging or has crashed (e.g. during long exhaustive runs on resource-constrained environments), the asynchronous propagation may not complete before `_compare_tables` queries the table, causing:
> {code:java}
> AnalysisException: Could not resolve table reference: '<db>.<charset>_gen_None'
> {code}
> *Fix:* Add {{INVALIDATE METADATA
> {db}
> .
> {encoded_table}
> }} (or {{{}REFRESH{}}}) at the end of `prepare_encoded_test_table_compress`, consistent with the other helpers.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org