You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ted Yu (JIRA)" <ji...@apache.org> on 2010/04/24 04:49:50 UTC

[jira] Created: (HIVE-1324) support more than 1 reducer in local test mode

support more than 1 reducer in local test mode
----------------------------------------------

                 Key: HIVE-1324
                 URL: https://issues.apache.org/jira/browse/HIVE-1324
             Project: Hadoop Hive
          Issue Type: Improvement
          Components: Testing Infrastructure
    Affects Versions: 0.5.0
            Reporter: Ted Yu


Currently only 1 reducer is supported in local test mode.
In order to write unit tests that simulate real production environment, it would be desirable to support more than 1 reducer.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1324) support more than 1 reducer in local test mode

Posted by "John Sichi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-1324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860561#action_12860561 ] 

John Sichi commented on HIVE-1324:
----------------------------------

We actually have something called minimr mode where it runs with a mini DFS cluster and can run multiple reducers.  I recently checked in an automated unit test for the HBase bulk load which uses this:  HIVE-1295.  I created the test file with the suffix .m instead of .q.

There's a JIRA open for doing this for ql tests (HIVE-117), so I think this is a dup, unless you actually want to add infrastructure for simulating map/reduce, which seems like too much work.

We need to undo the ql/build.xml change from HIVE-1312, but make the classpath change only apply for Hadoop 0.20.


> support more than 1 reducer in local test mode
> ----------------------------------------------
>
>                 Key: HIVE-1324
>                 URL: https://issues.apache.org/jira/browse/HIVE-1324
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Testing Infrastructure
>    Affects Versions: 0.5.0
>            Reporter: Ted Yu
>
> Currently only 1 reducer is supported in local test mode.
> In order to write unit tests that simulate real production environment, it would be desirable to support more than 1 reducer.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.