[jira] [Resolved] (HIVE-16038) MM tables: fix (or disable) inferring buckets

["Sergey Shelukhin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-16038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sergey Shelukhin resolved HIVE-16038.
-------------------------------------
       Resolution: Fixed
    Fix Version/s: hive-14535

Would be very easy to fix for a particular MM ID, but there's no guarantee that other MM IDs would conform to the inferred buckets, so I added comments and warnings and let it continue to fail (by discarding the inferred data, as it does already when the job doesn't produce the requisite number of files for a partition, see _dyn_part test).
I suspect similar issues may affect ACID tables and any other nested directory cases (and some overwrites?).

If somebody cares about this feature it should be easy to fix based on the comment added in the patch.

> MM tables: fix (or disable) inferring buckets
> ---------------------------------------------
>
>                 Key: HIVE-16038
>                 URL: https://issues.apache.org/jira/browse/HIVE-16038
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>             Fix For: hive-14535
>
>
> The following tests on minimr produce diffs if all tables are changed to MM:
> {noformat}
> infer_bucket_sort_dyn_part
> infer_bucket_sort_num_buckets
> infer_bucket_sort_merge
> infer_bucket_sort_reducers_power_two
> {noformat}
> Some of these disable strict checks for bucketing load, which wouldn't work by design; the rest should work. Either that, or we should disable this for MM tables - seems like an obscure feature.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)