You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Navis (JIRA)" <ji...@apache.org> on 2012/07/18 11:59:33 UTC

[jira] [Created] (HIVE-3270) SMBJoin should be applied atomically

Navis created HIVE-3270:
---------------------------

             Summary: SMBJoin should be applied atomically 
                 Key: HIVE-3270
                 URL: https://issues.apache.org/jira/browse/HIVE-3270
             Project: Hive
          Issue Type: Improvement
          Components: Query Processor
            Reporter: Navis
            Priority: Minor


For using smbjoin, mapjoin hint(/*+MAPJOIN(..)*/) + bucket mapjoin configuration(hive.optimize.bucketmapjoin) also should be preceded.

But when BucketMapJoinOptimizer or SortedMergeBucketMapJoinOptimizer fails to convert mapjoin to smbjoin by some reason, it will be executed as a (bucket) mapjoin, which possibly cause OOM in map task.

I think there should be a hint for SMBJoin, which tries to convert common-join to smbjoin and when it fails, simply fallbacks to common-join.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-3270) SMBJoin should be applied atomically

Posted by "Navis (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Navis resolved HIVE-3270.
-------------------------

    Resolution: Duplicate

This is duplicated by HIVE-3403.
                
> SMBJoin should be applied atomically 
> -------------------------------------
>
>                 Key: HIVE-3270
>                 URL: https://issues.apache.org/jira/browse/HIVE-3270
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Processor
>            Reporter: Navis
>            Priority: Minor
>
> For using smbjoin, mapjoin hint(/*+MAPJOIN(..)*/) + bucket mapjoin configuration(hive.optimize.bucketmapjoin) also should be preceded.
> But when BucketMapJoinOptimizer or SortedMergeBucketMapJoinOptimizer fails to convert mapjoin to smbjoin by some reason, it will be executed as a (bucket) mapjoin, which possibly cause OOM in map task.
> I think there should be a hint for SMBJoin, which tries to convert common-join to smbjoin and when it fails, simply fallbacks to common-join.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira