[jira] [Commented] (HIVE-14800) Handle off by 3 in ORC split generation based on split strategy used

["Gopal V (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-14800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15507454#comment-15507454 ] 

Gopal V commented on HIVE-14800:
--------------------------------

They are both valid splits, so unlikely to need changes there - the only issue is with external consumption of those details, like using those values as hashcode inputs.

We could implement a hashcode directly out of ORC though.

> Handle off by 3 in ORC split generation based on split strategy used
> --------------------------------------------------------------------
>
>                 Key: HIVE-14800
>                 URL: https://issues.apache.org/jira/browse/HIVE-14800
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Siddharth Seth
>
> BI will apparently generate splits starting at offset 0.
> ETL will skip the ORC header and generate a split starting at offset 3.
> There's a workaround in the HiveSplitGenreator to handle this for consistent splits. Ideally, Orc split generation should take care of this.
> cc [~prasanth_j], [~gopalv]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)