[jira] [Created] (ORC-1059) Align findColumns behaviour between 1.6 and 1.7 release

["Panagiotis Garefalakis (Jira)" <ji...@apache.org>]

Panagiotis Garefalakis created ORC-1059:
-------------------------------------------

             Summary: Align findColumns behaviour between 1.6 and 1.7 release
                 Key: ORC-1059
                 URL: https://issues.apache.org/jira/browse/ORC-1059
             Project: ORC
          Issue Type: Bug
            Reporter: Panagiotis Garefalakis
            Assignee: Panagiotis Garefalakis


ORC-741 changed the behaviour of column resolution failing with an IllegalArgumentException when a column is not found in the read schema.

Column resolution is also using by mapSargColumnsToOrcInternalColId when pushing down filters and throws an IllegalArgumentException exception when a filter column is not part of the schema instead of ignoring it (as it was the case in 1.6 and back).

1.6 and back
{code:java}
try {
  TypeDescription readerColumn = evolution.getReaderBaseSchema().findSubtype(
      columnName, evolution.isSchemaEvolutionCaseAware);
  TypeDescription fileColumn = evolution.getFileType(readerColumn);
  return fileColumn == null ? -1 : fileColumn.getId();
} catch (IllegalArgumentException e) {
  if (LOG.isDebugEnabled()){
    LOG.debug("{}", e.getMessage());
  }
  return -1;
} {code}


1.7 and main
{code:java}
try {
  TypeDescription readerColumn = evolution.getReaderBaseSchema().findSubtype(
    columnName, evolution.isSchemaEvolutionCaseAware);
  return evolution.getFileType(readerColumn);
} catch (IllegalArgumentException e) {
  throw new IllegalArgumentException("Filter could not find column with name: " +
                                     columnName + " on " + evolution.getReaderBaseSchema(),
                                     e);
} {code}


This makes harder to upgrade to 1.7 on downstream consumers like Hive https://issues.apache.org/jira/browse/HIVE-25497



--
This message was sent by Atlassian Jira
(v8.20.1#820001)