[jira] [Commented] (HIVE-7651) Investigate why union two RDDs generated from two MapTrans does not get the right result

["Na Yang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-7651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093083#comment-14093083 ] 

Na Yang commented on HIVE-7651:
-------------------------------

This issue was caused by a single jobConf instance are used by multiple MapTrans. The fix is included in the patch of HIVE-7541.  

> Investigate why union two RDDs generated from two MapTrans does not get the right result
> ----------------------------------------------------------------------------------------
>
>                 Key: HIVE-7651
>                 URL: https://issues.apache.org/jira/browse/HIVE-7651
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>            Reporter: Na Yang
>
> If the SparkWork has two map works as root, then use the current generate(basework) API to generate two mapTran. union the RDDs processed by the two mapTrans does not generate the correct result. 
> If two input RDDs come from different data tables, then the union result is empty.
> if two input RDDs come from the same data table, then the union result is not correct. The same row of data happen 4 times in the union result.
> Need to investigate why this happen and how to fix it.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)