You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2026/10/16 01:20:00 UTC

[jira] [Resolved] (IMPALA-15290) test_execute_remove_orphan_files fails in file existence check

     [ https://issues.apache.org/jira/browse/IMPALA-15290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Quanlong Huang resolved IMPALA-15290.
-------------------------------------
    Fix Version/s: Impala 5.0.0
       Resolution: Fixed

> test_execute_remove_orphan_files fails in file existence check
> --------------------------------------------------------------
>
>                 Key: IMPALA-15290
>                 URL: https://issues.apache.org/jira/browse/IMPALA-15290
>             Project: IMPALA
>          Issue Type: Bug
>            Reporter: Quanlong Huang
>            Assignee: Quanlong Huang
>            Priority: Major
>             Fix For: Impala 5.0.0
>
>
> Stacktrace
> {noformat}
> query_test/test_iceberg.py:502: in test_execute_remove_orphan_files
>     assert self.filesystem_client.exists(os.path.join(DATA_PATH, file_parq1))
> E   AssertionError: assert False
> E    +  where False = <bound method HadoopFsCommandLineClient.exists of <tests.util.hdfs_util.HadoopFsCommandLineClient object at 0x7fd3a9a06d10>>('/test-warehouse/test_execute_remove_orphan_files_1e8edd5.db/tbl_with_orphan_files/data/00000-0-data-gfurnstahl_20220906113044_157fc172-f5d3-4c70-8653-fff150b6136a-job_16619542960420_0002-1-00001.parquet')
> E    +    where <bound method HadoopFsCommandLineClient.exists of <tests.util.hdfs_util.HadoopFsCommandLineClient object at 0x7fd3a9a06d10>> = <tests.util.hdfs_util.HadoopFsCommandLineClient object at 0x7fd3a9a06d10>.exists
> E    +      where <tests.util.hdfs_util.HadoopFsCommandLineClient object at 0x7fd3a9a06d10> = <test_iceberg.TestIcebergTable object at 0x7fd3a98e8950>.filesystem_client
> E    +    and   '/test-warehouse/test_execute_remove_orphan_files_1e8edd5.db/tbl_with_orphan_files/data/00000-0-data-gfurnstahl_20220906113044_157fc172-f5d3-4c70-8653-fff150b6136a-job_16619542960420_0002-1-00001.parquet' = <function join at 0x7fd3b6e34f40>('/test-warehouse/test_execute_remove_orphan_files_1e8edd5.db/tbl_with_orphan_files/data', '00000-0-data-gfurnstahl_20220906113044_157fc172-f5d3-4c70-8653-fff150b6136a-job_16619542960420_0002-1-00001.parquet')
> E    +      where <function join at 0x7fd3b6e34f40> = <module 'posixpath' (frozen)>.join
> E    +        where <module 'posixpath' (frozen)> = os.path
>         DATA_PATH  = '/test-warehouse/test_execute_remove_orphan_files_1e8edd5.db/tbl_with_orphan_files/data'
>         METADATA_PATH = '/test-warehouse/test_execute_remove_orphan_files_1e8edd5.db/tbl_with_orphan_files/metadata'
>         SRC_DIR    = '/data/jenkins/workspace/impala-asf-master-core-s3-data-cache/repos/Impala/testdata/data/iceberg_test/iceberg_mixed_file_format_test/{0}/{1}'
>         TABLE_PATH = '/test-warehouse/test_execute_remove_orphan_files_1e8edd5.db/tbl_with_orphan_files'
>         db_tbl     = 'test_execute_remove_orphan_files_1e8edd5.tbl_with_orphan_files'
>         file_avro1 = '055baf62-de6d-4583-bf21-f187f9482343-m0.avro'
>         file_parq1 = '00000-0-data-gfurnstahl_20220906113044_157fc172-f5d3-4c70-8653-fff150b6136a-job_16619542960420_0002-1-00001.parquet'
>         impalad_client = <tests.common.impala_connection.ImpylaHS2Connection object at 0x7fd3a9aa06d0>
>         insert_q   = 'insert into {0} values ({1})'
>         result     = <tests.common.impala_connection.ImpylaHS2ResultSet object at 0x7fd3a9aa3590>
>         self       = <test_iceberg.TestIcebergTable object at 0x7fd3a98e8950>
>         tbl_name   = 'tbl_with_orphan_files'
>         unique_database = 'test_execute_remove_orphan_files_1e8edd5'{noformat}
> The code is:
> {code:python}
>  494       # Copy first set of junk files.
>  495       file_parq1 = "00000-0-data-gfurnstahl_20220906113044_157fc172-f5d3-4c70-8653-" \
>  496           "fff150b6136a-job_16619542960420_0002-1-00001.parquet"
>  497       file_avro1 = "055baf62-de6d-4583-bf21-f187f9482343-m0.avro"
>  498       self.filesystem_client.copy_from_local(
>  499           SRC_DIR.format('data', file_parq1), DATA_PATH)
>  500       self.filesystem_client.copy_from_local(
>  501           SRC_DIR.format('metadata', file_avro1), METADATA_PATH)
>  502       assert self.filesystem_client.exists(os.path.join(DATA_PATH, file_parq1))
>  503       assert self.filesystem_client.exists(os.path.join(METADATA_PATH, file_avro1)){code}
> https://github.com/apache/impala/blob/2157bc8278dd9db8dcb7dc77d982919a9d61f4b0/tests/query_test/test_iceberg.py#L502



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org