You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Namit Jain (Created) (JIRA)" <ji...@apache.org> on 2011/12/09 21:08:39 UTC

[jira] [Created] (HIVE-2641) SkewJoin should report progress while writing skew keys to HDFS

SkewJoin should report progress while writing skew keys to HDFS
---------------------------------------------------------------

                 Key: HIVE-2641
                 URL: https://issues.apache.org/jira/browse/HIVE-2641
             Project: Hive
          Issue Type: Improvement
            Reporter: Namit Jain


RowContainer.copyToDFSDirectory is used to write the skew keys to HDFS. But this can take a long time, and while it is happening, the MR framework does not see progress from the reducer. The code should do the copy in smaller unit and report progress to the MR framework

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2641) SkewJoin should report progress while writing skew keys to HDFS

Posted by "Namit Jain (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/HIVE-2641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Namit Jain reassigned HIVE-2641:
--------------------------------

    Assignee: Namit Jain
    
> SkewJoin should report progress while writing skew keys to HDFS
> ---------------------------------------------------------------
>
>                 Key: HIVE-2641
>                 URL: https://issues.apache.org/jira/browse/HIVE-2641
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Namit Jain
>            Assignee: Namit Jain
>
> RowContainer.copyToDFSDirectory is used to write the skew keys to HDFS. But this can take a long time, and while it is happening, the MR framework does not see progress from the reducer. The code should do the copy in smaller unit and report progress to the MR framework

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira