You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by la...@apache.org on 2025/09/12 16:38:34 UTC
(impala) branch master updated (7fabd2709 -> 968559da2)
This is an automated email from the ASF dual-hosted git repository.
laszlog pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
from 7fabd2709 IMPALA-14411: enable_workload_mgmt should work with V2 profile
new 492b2b7f4 IMPALA-14406: Fix test_column_case_sensitivity for newer Iceberg versions
new 968559da2 IMPALA-14398: Deflake test_load_table_with_primary_key_attr
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../QueryTest/iceberg-column-case-sensitivity-issue.test | 7 ++++---
tests/comparison/db_connection.py | 15 ++++++++++-----
2 files changed, 14 insertions(+), 8 deletions(-)
(impala) 02/02: IMPALA-14398: Deflake test_load_table_with_primary_key_attr
Posted by la...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
laszlog pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 968559da26bbea5a54987c6c2fafb62a03fbaabe
Author: Riza Suminto <ri...@cloudera.com>
AuthorDate: Wed Sep 10 09:51:51 2025 -0700
IMPALA-14398: Deflake test_load_table_with_primary_key_attr
test_load_table_with_primary_key_attr has been intermittently fail with
following error message since Python3 switch in IMPALA-14333.
dbm.error: db type could not be determined
Error happen inside load_unique_col_metadata. But both
load_unique_col_metadata and persist_unique_col_metadata are seemingly
unused anymore.
While the root cause is still unknown, commenting
load_unique_col_metadata in db_connection.py can make the tests pass.
Tests are not affected because the they do not assert against any unique
columns. This patch implement that workaround until we can revisit the
issue with query generator infrastructure.
Fix few flake8 warnings in db_connection.py.
Testing:
Loop and pass test_show_create_table 10 times using downstream jenkins.
Change-Id: I2caf6b9cc5a4f06237323cb32bbd47afef576fa1
Reviewed-on: http://gerrit.cloudera.org:8080/23412
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
tests/comparison/db_connection.py | 15 ++++++++++-----
1 file changed, 10 insertions(+), 5 deletions(-)
diff --git a/tests/comparison/db_connection.py b/tests/comparison/db_connection.py
index 8705024eb..0c2b60b8d 100644
--- a/tests/comparison/db_connection.py
+++ b/tests/comparison/db_connection.py
@@ -423,7 +423,12 @@ class DbCursor(object):
'''Return a Table with table and col names always in lowercase.'''
describe_rows = self.execute_and_fetchall(self.make_describe_table_sql(table_name))
table = self.create_table_from_describe(table_name, describe_rows)
- self.load_unique_col_metadata(table)
+ # IMPALA-14398: Commenting this because, testing with Python 3.8, it hit
+ # dbm.error: db type could not be determined
+ # Commenting it does not affect any existing infra tests.
+ # Need to revisit if persist_unique_col_metadata and load_unique_col_metadata
+ # is still needed.
+ # self.load_unique_col_metadata(table)
return table
def make_describe_table_sql(self, table_name):
@@ -685,7 +690,7 @@ class DbConnection(with_metaclass(ABCMeta, object)):
class ImpalaCursor(DbCursor):
- PK_SEARCH_PATTERN = re.compile('PRIMARY KEY \((?P<keys>.*?)\)')
+ PK_SEARCH_PATTERN = re.compile(r'PRIMARY KEY \((?P<keys>.*?)\)')
STORAGE_FORMATS_WITH_PRIMARY_KEYS = ('KUDU',)
@classmethod
@@ -1212,12 +1217,12 @@ class OracleConnection(DbConnection):
def _connect(self):
try:
import cx_Oracle
- except:
+ except Exception:
print('Error importing cx_Oracle. Please make sure it is installed. '
'See the README for details.')
raise
- self._conn = cx_Oracle.connect('%(user)s/%(password)s@%(host)s:%(port)s/%(service)s'
- % (self._user_name, self._password, self._host_name, self._port, self._service))
+ self._conn = cx_Oracle.connect('{}/{}@{}:{}/{}'.format(
+ self._user_name, self._password, self._host_name, self._port, self._service))
def cursor(self):
cursor = super(OracleConnection, self).cursor()
(impala) 01/02: IMPALA-14406: Fix test_column_case_sensitivity for newer Iceberg versions
Posted by la...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
laszlog pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 492b2b7f46af8a2b0e0a868f3947916bb822e2a7
Author: Daniel Vanko <dv...@cloudera.com>
AuthorDate: Mon Sep 8 11:50:22 2025 +0200
IMPALA-14406: Fix test_column_case_sensitivity for newer Iceberg versions
New test introduced in IMPALA-14290 depends on the Iceberg versions,
because newer ones (e.g. 1.5.2) will show
{"start_time_month":"646","end_time_day":"19916"}
instead of
{"start_time_day":null,"end_time_month":null,"start_time_month":"646","end_time_day":"19916"}
The test now accepts both cases.
Testing:
* ran query_test/test_iceberg.py with both Iceberg 1.3.1 and 1.5.2
Change-Id: I17e368ac043d1fbf80a78dcac6ab1be5a297b6ea
Reviewed-on: http://gerrit.cloudera.org:8080/23389
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
.../queries/QueryTest/iceberg-column-case-sensitivity-issue.test | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/testdata/workloads/functional-query/queries/QueryTest/iceberg-column-case-sensitivity-issue.test b/testdata/workloads/functional-query/queries/QueryTest/iceberg-column-case-sensitivity-issue.test
index 3e0dd0d00..3059afd8c 100644
--- a/testdata/workloads/functional-query/queries/QueryTest/iceberg-column-case-sensitivity-issue.test
+++ b/testdata/workloads/functional-query/queries/QueryTest/iceberg-column-case-sensitivity-issue.test
@@ -46,9 +46,10 @@ SHOW PARTITIONS partition_case_insensitive;
---- LABELS
Partition, Number Of Rows, Number Of Files
---- RESULTS
-'{"start_time_day":null,"end_time_month":null,"start_time_month":"646","end_time_day":"19916"}',1,1
-'{"start_time_day":null,"end_time_month":null,"start_time_month":"647","end_time_day":"19916"}',1,1
-'{"start_time_day":null,"end_time_month":null,"start_time_month":"647","end_time_day":"19947"}',2,2
+# IMPALA-14406: in newer Iceberg versions (e.g. 1.5.2), partition values with null are not shown
+regex:'{("start_time_day":null,"end_time_month":null,)?"start_time_month":"646","end_time_day":"19916"}',1,1
+regex:'{("start_time_day":null,"end_time_month":null,)?"start_time_month":"647","end_time_day":"19916"}',1,1
+regex:'{("start_time_day":null,"end_time_month":null,)?"start_time_month":"647","end_time_day":"19947"}',2,2
---- TYPES
STRING, BIGINT, BIGINT
====