[jira] [Resolved] (IMPALA-10062) TestCompressedNonText.test_insensitivity_to_extension can fail due to wrong filename

["Joe McDonnell (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-10062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joe McDonnell resolved IMPALA-10062.
------------------------------------
    Fix Version/s: Impala 4.0
       Resolution: Fixed

> TestCompressedNonText.test_insensitivity_to_extension can fail due to wrong filename
> ------------------------------------------------------------------------------------
>
>                 Key: IMPALA-10062
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10062
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>    Affects Versions: Impala 4.0
>            Reporter: Joe McDonnell
>            Priority: Critical
>              Labels: broken-build, flaky
>             Fix For: Impala 4.0
>
>
> The fix for IMPALA-10005 added a new TestCompressedNonText test. It relies on Hive generating specific file names when writing these compressed tables (i.e. it expects a file named 000000_0). It looks like that is not guaranteed by dataload, which can lead to failures like this:
> {noformat}
> query_test/test_compressed_formats.py:142: in test_insensitivity_to_extension
>     unique_database, 'tinytable', db_suffix, '000000_0', src_extension, ext)
> query_test/test_compressed_formats.py:86: in _copy_and_query_compressed_file
>     self.filesystem_client.copy(src_file, dest_file, overwrite=True)
> util/hdfs_util.py:79: in copy
>     self.hdfs_filesystem_client.copy(src, dst, overwrite)
> util/hdfs_util.py:241: in copy
>     '{0} copy failed: '.format(self.filesystem_type) + stderr + "; " + stdout
> E   AssertionError: HDFS copy failed: cp: `/test-warehouse/tinytable_avro_snap/000000_0': No such file or directory
> E   ;{noformat}
> The file list shows that the filename is actually "/test-warehouse/tinytable_avro_snap/000000_1"
> We should update the test to tolerate this. The actual base filename doesn't matter for this test.
> I have seen this exactly once so far.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)