[jira] [Commented] (HIVE-1065) When data is large,a reducer's memory can not hold the big data by using list to store the data, and get an error "out of memory"

["ransom.hezhiqiang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276691#comment-13276691 ] 

ransom.hezhiqiang commented on HIVE-1065:
-----------------------------------------

MapJoin,bucket mapJoin,sorted bucket MapJoin.
and also can used Stream table to steam the big table. 
if two table's data is also very large .
try skew join. change the tasknum.
                
> When data is large,a reducer's memory can not hold the big data by using list to store the data, and get an error "out of memory"
> ---------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-1065
>                 URL: https://issues.apache.org/jira/browse/HIVE-1065
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.3.0
>         Environment: redhat enterprise 4.0
>            Reporter: fuyangchang
>            Priority: Critical
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> When data is large,I do a "join" operation ,a reducer's memory can not hold the big data by using list to store the data, and get an error "out of memory"

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira