[jira] Commented: (HIVE-204) Provide option to run hadoop via TestMiniMR

["Zheng Shao (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705201#action_12705201 ] 

Zheng Shao commented on HIVE-204:
---------------------------------

+1 but it failed some 0.17.0 tests.

Seems like class path problem.

{code}
ant -Dhadoop.version=0.17.0 package test -Dtestcase=TestCliDriver -Dqfile=input16_cc.q

    [junit] java.lang.ClassNotFoundException: org.apache.hadoop.hive.serde2.TestSerDe
    [junit] Continuing ...
    [junit] org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.serde2.TestSerDe
    [junit]     at org.apache.hadoop.hive.ql.exec.MapOperator.initialize(MapOperator.java:153)
    [junit]     at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:57)
    [junit]     at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
    [junit]     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
    [junit]     at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:157)
    [junit] Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.serde2.TestSerDe
    [junit]     at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    [junit]     at java.security.AccessController.doPrivileged(Native Method)
    [junit]     at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    [junit]     at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    [junit]     at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    [junit]     at org.apache.hadoop.hive.ql.exec.MapOperator.initialize(MapOperator.java:108)
    [junit]     ... 4 more
{code}

For this patch, we need to do both:
{code}
ant clean test
ant -Dhadoop.version=0.17.0 clean test
{code}

If the tests are passed, I am OK with this committed. But I am gone for the next 2 weeks, so maybe somebody else can help commit it.


> Provide option to run hadoop via TestMiniMR
> -------------------------------------------
>
>                 Key: HIVE-204
>                 URL: https://issues.apache.org/jira/browse/HIVE-204
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Testing Infrastructure
>            Reporter: Raghotham Murthy
>            Assignee: Namit Jain
>         Attachments: hive.204.1.patch, hive.204.2.patch, hive.204.3.patch
>
>
> Right now MapRedTask does an exec on the hadoop command line. This prevents us from stepping through query execution code. If there were an option to run hadoop via MiniMR, this will create tasks within the same VM and so allows stepping into the execution code. See, src/test/org/apache/hadoop/mapred/TestMiniMRWithDFS.java in the hadoop source code for an example.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.