You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Gopal Nagar (JIRA)" <ji...@apache.org> on 2017/01/19 11:03:26 UTC
[jira] [Updated] (SPARK-19286) Spark/Hive insert Into has
unexpected results
[ https://issues.apache.org/jira/browse/SPARK-19286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gopal Nagar updated SPARK-19286:
--------------------------------
Description:
Hi Team,
We are using Java API to call Spark job for inserting the data in Hive table. When we perform Append operation, Spark inserts data in Hive considering that Hive columns are in Alphabetical order. We don’t see this issue if we using Pyspark. We are using Spark 1.6.1 & Hive 1.0 in EMR 4.6.0.
Further researching on the issue, I found that similar issue has been reported in Spark 2.2.0. Could you please advise if this issue present in Spark 1.6.1 as well ?
https://issues.apache.org/jira/browse/SPARK-14543
As per our understanding PySpark & Java called Spark both will use same Spark APIs in backend for similar operation. Please advise if this is not the case.
was:
Hi Team,
We are using Java API to call Spark job for inserting the data in Hive table. When we perform Append operation, Spark inserts data in Hive considering that Hive columns are in Alphabetical order. We don’t see this issue if we using Pyspark. We are using Spark 1.6.1 & Hive 1.0 in EMR 4.6.0.
Further researching on the issue, I found that similar issue has been reported in Spark 2.2.0. Could you please advise if this issue present in Spark 1.6.1 as well ?
https://issues.apache.org/jira/browse/SPARK-14543
> Spark/Hive insert Into has unexpected results
> ---------------------------------------------
>
> Key: SPARK-19286
> URL: https://issues.apache.org/jira/browse/SPARK-19286
> Project: Spark
> Issue Type: Bug
> Components: Java API
> Affects Versions: 1.6.1
> Environment: Dev
> Reporter: Gopal Nagar
>
> Hi Team,
> We are using Java API to call Spark job for inserting the data in Hive table. When we perform Append operation, Spark inserts data in Hive considering that Hive columns are in Alphabetical order. We don’t see this issue if we using Pyspark. We are using Spark 1.6.1 & Hive 1.0 in EMR 4.6.0.
> Further researching on the issue, I found that similar issue has been reported in Spark 2.2.0. Could you please advise if this issue present in Spark 1.6.1 as well ?
> https://issues.apache.org/jira/browse/SPARK-14543
> As per our understanding PySpark & Java called Spark both will use same Spark APIs in backend for similar operation. Please advise if this is not the case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org