You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2026/10/16 01:20:00 UTC
[jira] [Resolved] (IMPALA-15290) test_execute_remove_orphan_files fails in file existence check
[ https://issues.apache.org/jira/browse/IMPALA-15290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Quanlong Huang resolved IMPALA-15290.
-------------------------------------
Fix Version/s: Impala 5.0.0
Resolution: Fixed
> test_execute_remove_orphan_files fails in file existence check
> --------------------------------------------------------------
>
> Key: IMPALA-15290
> URL: https://issues.apache.org/jira/browse/IMPALA-15290
> Project: IMPALA
> Issue Type: Bug
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> Stacktrace
> {noformat}
> query_test/test_iceberg.py:502: in test_execute_remove_orphan_files
> assert self.filesystem_client.exists(os.path.join(DATA_PATH, file_parq1))
> E AssertionError: assert False
> E + where False = <bound method HadoopFsCommandLineClient.exists of <tests.util.hdfs_util.HadoopFsCommandLineClient object at 0x7fd3a9a06d10>>('/test-warehouse/test_execute_remove_orphan_files_1e8edd5.db/tbl_with_orphan_files/data/00000-0-data-gfurnstahl_20220906113044_157fc172-f5d3-4c70-8653-fff150b6136a-job_16619542960420_0002-1-00001.parquet')
> E + where <bound method HadoopFsCommandLineClient.exists of <tests.util.hdfs_util.HadoopFsCommandLineClient object at 0x7fd3a9a06d10>> = <tests.util.hdfs_util.HadoopFsCommandLineClient object at 0x7fd3a9a06d10>.exists
> E + where <tests.util.hdfs_util.HadoopFsCommandLineClient object at 0x7fd3a9a06d10> = <test_iceberg.TestIcebergTable object at 0x7fd3a98e8950>.filesystem_client
> E + and '/test-warehouse/test_execute_remove_orphan_files_1e8edd5.db/tbl_with_orphan_files/data/00000-0-data-gfurnstahl_20220906113044_157fc172-f5d3-4c70-8653-fff150b6136a-job_16619542960420_0002-1-00001.parquet' = <function join at 0x7fd3b6e34f40>('/test-warehouse/test_execute_remove_orphan_files_1e8edd5.db/tbl_with_orphan_files/data', '00000-0-data-gfurnstahl_20220906113044_157fc172-f5d3-4c70-8653-fff150b6136a-job_16619542960420_0002-1-00001.parquet')
> E + where <function join at 0x7fd3b6e34f40> = <module 'posixpath' (frozen)>.join
> E + where <module 'posixpath' (frozen)> = os.path
> DATA_PATH = '/test-warehouse/test_execute_remove_orphan_files_1e8edd5.db/tbl_with_orphan_files/data'
> METADATA_PATH = '/test-warehouse/test_execute_remove_orphan_files_1e8edd5.db/tbl_with_orphan_files/metadata'
> SRC_DIR = '/data/jenkins/workspace/impala-asf-master-core-s3-data-cache/repos/Impala/testdata/data/iceberg_test/iceberg_mixed_file_format_test/{0}/{1}'
> TABLE_PATH = '/test-warehouse/test_execute_remove_orphan_files_1e8edd5.db/tbl_with_orphan_files'
> db_tbl = 'test_execute_remove_orphan_files_1e8edd5.tbl_with_orphan_files'
> file_avro1 = '055baf62-de6d-4583-bf21-f187f9482343-m0.avro'
> file_parq1 = '00000-0-data-gfurnstahl_20220906113044_157fc172-f5d3-4c70-8653-fff150b6136a-job_16619542960420_0002-1-00001.parquet'
> impalad_client = <tests.common.impala_connection.ImpylaHS2Connection object at 0x7fd3a9aa06d0>
> insert_q = 'insert into {0} values ({1})'
> result = <tests.common.impala_connection.ImpylaHS2ResultSet object at 0x7fd3a9aa3590>
> self = <test_iceberg.TestIcebergTable object at 0x7fd3a98e8950>
> tbl_name = 'tbl_with_orphan_files'
> unique_database = 'test_execute_remove_orphan_files_1e8edd5'{noformat}
> The code is:
> {code:python}
> 494 # Copy first set of junk files.
> 495 file_parq1 = "00000-0-data-gfurnstahl_20220906113044_157fc172-f5d3-4c70-8653-" \
> 496 "fff150b6136a-job_16619542960420_0002-1-00001.parquet"
> 497 file_avro1 = "055baf62-de6d-4583-bf21-f187f9482343-m0.avro"
> 498 self.filesystem_client.copy_from_local(
> 499 SRC_DIR.format('data', file_parq1), DATA_PATH)
> 500 self.filesystem_client.copy_from_local(
> 501 SRC_DIR.format('metadata', file_avro1), METADATA_PATH)
> 502 assert self.filesystem_client.exists(os.path.join(DATA_PATH, file_parq1))
> 503 assert self.filesystem_client.exists(os.path.join(METADATA_PATH, file_avro1)){code}
> https://github.com/apache/impala/blob/2157bc8278dd9db8dcb7dc77d982919a9d61f4b0/tests/query_test/test_iceberg.py#L502
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org