[jira] [Commented] (HIVE-9076) incompatFileSet in AbstractFileMergeOperator should be marked to skip task id check

["Prasanth Jayachandran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242044#comment-14242044 ] 

Prasanth Jayachandran commented on HIVE-9076:
---------------------------------------------

What is the resultant file set after merging?
I would expect either
{code}
000000_0 (v12) (all v12 files merged together)
000000_1 (v11)
000000_1_copy_1 (v11)
000000_1_copy_2 (v11)
{code}
or
{code}
000000_0 (v11) (all v11 files merged together)
000000_0_copy_1 (v12) (this is actual 000000_0 file but renamed to 000000_0_copy_1 as merge output file has same name)
000000_0_copy_2 (v12) (this is actual 000000_0_copy_1 file but renamed to 000000_0_copy_2 as 000000_0_copy_1 exists)
000000_2 (v12)
{code}

Different order will also be possible as incompatFileSet iteration order might be different.

> incompatFileSet in AbstractFileMergeOperator should be marked to skip task id check
> -----------------------------------------------------------------------------------
>
>                 Key: HIVE-9076
>                 URL: https://issues.apache.org/jira/browse/HIVE-9076
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>            Reporter: Navis
>            Assignee: Navis
>            Priority: Minor
>
> In some file composition, AbstractFileMergeOperator removes incompatible files. For example,
> {noformat}
> 000000_0 (v12)
> 000000_0_copy_1 (v12)
> 000000_1 (v11)
> 000000_1_copy_1 (v11)
> 000000_1_copy_2 (v11)
> 000000_2 (v12)
> {noformat}
> 000000_1 (v11) will be removed because 000000 is assigned to new merged file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)