[jira] [Created] (DRILL-5106) Refactor SkipRecordsInspector to exclude check for predefined file formats

["Arina Ielchiieva (JIRA)" <ji...@apache.org>]

Arina Ielchiieva created DRILL-5106:
---------------------------------------

             Summary: Refactor SkipRecordsInspector to exclude check for predefined file formats
                 Key: DRILL-5106
                 URL: https://issues.apache.org/jira/browse/DRILL-5106
             Project: Apache Drill
          Issue Type: Improvement
          Components: Storage - Hive
    Affects Versions: 1.9.0
            Reporter: Arina Ielchiieva
            Priority: Minor


After changes introduced in DRILL-4982, SkipRecordInspector is used only for predefined formats (using hasHeaderFooter: false / true). But SkipRecordInspector has its own check for formats where skip strategy can be applied. Acceptable file formats are stored in private final Set<Object> fileFormats and initialized in constructor, currently it contains only one format - TextInputFormat. Now this check is redundant and may lead to ignoring hasHeaderFooter setting true for any other format except of Text.

To do:
1. remove private final Set<Object> fileFormats
2. remove if block from SkipRecordsInspector.retrievePositiveIntProperty:
{code}
 if (!fileFormats.contains(tableProperties.get(hive_metastoreConstants.FILE_INPUT_FORMAT))) {
return propertyIntValue;
}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)