You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Joe McDonnell (Jira)" <ji...@apache.org> on 2020/12/04 19:15:00 UTC
[jira] [Resolved] (IMPALA-9004) TestCompressedFormats is broken for
text files
[ https://issues.apache.org/jira/browse/IMPALA-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joe McDonnell resolved IMPALA-9004.
-----------------------------------
Fix Version/s: Impala 4.0
Resolution: Fixed
This was fixed as part of IMPALA-10005.
> TestCompressedFormats is broken for text files
> ----------------------------------------------
>
> Key: IMPALA-9004
> URL: https://issues.apache.org/jira/browse/IMPALA-9004
> Project: IMPALA
> Issue Type: Test
> Reporter: Sahil Takiar
> Priority: Major
> Fix For: Impala 4.0
>
>
> While working on IMPALA-8950, we made a fix to {{TestCompressedFormats}} so that it actually checks the exit status of the {{hdfs dfs -cp}} command, turns out that this command has been silently failing whenever {{test_compressed_formats}} runs with {{file_format}} = {{text}}.
> For some reason, data load writes compressed text files with their corresponding file compression suffix, but for compressed seq/rc files, it does not:
> {code:java}
> hdfs dfs -ls /test-warehouse/tinytable_seq_*
> Found 1 items
> -rwxr-xr-x 3 systest supergroup 325 2019-08-22 14:32 /test-warehouse/tinytable_seq_bzip/000000_0
> Found 1 items
> -rwxr-xr-x 3 systest supergroup 215 2019-08-22 14:32 /test-warehouse/tinytable_seq_def/000000_0
> Found 1 items
> -rwxr-xr-x 3 systest supergroup 260 2019-08-22 14:32 /test-warehouse/tinytable_seq_gzip/000000_0
> Found 1 items
> -rwxr-xr-x 3 systest supergroup 301 2019-08-22 14:32 /test-warehouse/tinytable_seq_record_bzip/000000_0
> Found 1 items
> -rwxr-xr-x 3 systest supergroup 209 2019-08-22 14:32 /test-warehouse/tinytable_seq_record_def/000000_0
> Found 1 items
> -rwxr-xr-x 3 systest supergroup 242 2019-08-22 14:32 /test-warehouse/tinytable_seq_record_gzip/000000_0
> Found 1 items
> -rwxr-xr-x 3 systest supergroup 233 2019-08-22 14:32 /test-warehouse/tinytable_seq_record_snap/000000_0
> Found 2 items
> -rwxr-xr-x 3 systest supergroup 243 2019-08-22 14:32 /test-warehouse/tinytable_seq_snap/000000_0
> hdfs dfs -ls /test-warehouse/tinytable_text_*
> Found 1 items
> -rwxr-xr-x 3 systest supergroup 59 2019-08-22 14:32 /test-warehouse/tinytable_text_bzip/000000_0.bz2
> Found 1 items
> -rwxr-xr-x 3 systest supergroup 28 2019-08-22 14:32 /test-warehouse/tinytable_text_def/000000_0.deflate
> Found 1 items
> -rwxr-xr-x 3 systest supergroup 40 2019-08-22 14:32 /test-warehouse/tinytable_text_gzip/000000_0.gz
> Found 2 items
> -rwxr-xr-x 3 systest supergroup 87 2019-08-22 14:32 /test-warehouse/tinytable_text_lzo/000000_0.lzo
> -rw-r--r-- 3 systest supergroup 8 2019-08-22 14:42 /test-warehouse/tinytable_text_lzo/000000_0.lzo.index
> Found 1 items
> -rwxr-xr-x 3 systest supergroup 41 2019-08-22 14:32 /test-warehouse/tinytable_text_snap/000000_0.snappy{code}
> Not sure if that is by design or not, but it is causing the tests to fail for all text files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)