[jira] [Resolved] (IMPALA-5287) Add a test for skip.header.line.count on compressed files

["Lars Volker (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-5287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Lars Volker resolved IMPALA-5287.
---------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.9.0


IMPALA-5287: Test skip.header.line.count on gzip

This change fixed IMPALA-4873 by adding the capability to supply a dict
'test_file_vars' to run_test_case(). Keys in this dict will be replaced
with their values inside test queries before they are executed.

Change-Id: Ie3f3c29a42501cfb2751f7ad0af166eb88f63b70
Reviewed-on: http://gerrit.cloudera.org:8080/6817
Reviewed-by: Michael Brown <mi...@cloudera.com>
Tested-by: Impala Public Jenkins

> Add a test for skip.header.line.count on compressed files
> ---------------------------------------------------------
>
>                 Key: IMPALA-5287
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5287
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 2.9.0
>            Reporter: Lars Volker
>            Assignee: Lars Volker
>            Priority: Critical
>             Fix For: Impala 2.9.0
>
>
> Before the fix for IMPALA-3905 was merged, the HDFS text scanner initialized the decompressor after finding the first row. This was wrong, but not an issue for normal compressed tables, since for those we only issue a single scan range, ant therefore can skip searching for the first newline character.
> However, this broke skipping header lines at the beginning of compressed files. We should add a test for skip.header.line.count on compressed files to prevent a regression in the future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)