You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jianshi Huang (JIRA)" <ji...@apache.org> on 2014/12/07 05:30:12 UTC
[jira] [Updated] (SPARK-4781) Column values become all NULL after
doing ALTER TABLE CHANGE for renaming column names (Parquet external table
in HiveContext)
[ https://issues.apache.org/jira/browse/SPARK-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jianshi Huang updated SPARK-4781:
---------------------------------
Description:
I have a table say created like follows:
CREATE EXTERNAL TABLE pmt (
`sorted::cre_ts` string
)
STORED AS PARQUET
LOCATION '...'
And I renamed the column from sorted::cre_ts to cre_ts by doing:
ALTER TABLE pmt CHANGE `sorted::cre_ts` cre_ts string
After renaming the column, the values in the column become all NULLs.
Before renaming:
scala> sql("select `sorted::cre_ts` from pmt limit 1").collect
res12: Array[org.apache.spark.sql.Row] = Array([12/02/2014 07:38:54])
Execute renaming:
scala> sql("alter table pmt change `sorted::cre_ts` cre_ts string")
res13: org.apache.spark.sql.SchemaRDD =
SchemaRDD[972] at RDD at SchemaRDD.scala:108
== Query Plan ==
<Native command: executed by Hive>
After renaming:
scala> sql("select cre_ts from pmt limit 1").collect
res16: Array[org.apache.spark.sql.Row] = Array([null])
Jianshi
was:
I have a table say created like follows:
CREATE EXTERNAL TABLE pmt {
`sorted::cre_ts` string
}
And I renamed the column from sorted::cre_ts to cre_ts by doing:
ALTER TABLE pmt CHANGE `sorted::cre_ts` cre_ts string
After renaming the column, the values in the column become all NULLs.
Before renaming:
scala> sql("select `sorted::cre_ts` from pmt limit 1").collect
res12: Array[org.apache.spark.sql.Row] = Array([12/02/2014 07:38:54])
Execute renaming:
scala> sql("alter table pmt change `sorted::cre_ts` cre_ts string")
res13: org.apache.spark.sql.SchemaRDD =
SchemaRDD[972] at RDD at SchemaRDD.scala:108
== Query Plan ==
<Native command: executed by Hive>
After renaming:
scala> sql("select cre_ts from pmt limit 1").collect
res16: Array[org.apache.spark.sql.Row] = Array([null])
Jianshi
> Column values become all NULL after doing ALTER TABLE CHANGE for renaming column names (Parquet external table in HiveContext)
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-4781
> URL: https://issues.apache.org/jira/browse/SPARK-4781
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 1.2.0, 1.3.0, 1.2.1
> Reporter: Jianshi Huang
>
> I have a table say created like follows:
> CREATE EXTERNAL TABLE pmt (
> `sorted::cre_ts` string
> )
> STORED AS PARQUET
> LOCATION '...'
> And I renamed the column from sorted::cre_ts to cre_ts by doing:
> ALTER TABLE pmt CHANGE `sorted::cre_ts` cre_ts string
> After renaming the column, the values in the column become all NULLs.
> Before renaming:
> scala> sql("select `sorted::cre_ts` from pmt limit 1").collect
> res12: Array[org.apache.spark.sql.Row] = Array([12/02/2014 07:38:54])
> Execute renaming:
> scala> sql("alter table pmt change `sorted::cre_ts` cre_ts string")
> res13: org.apache.spark.sql.SchemaRDD =
> SchemaRDD[972] at RDD at SchemaRDD.scala:108
> == Query Plan ==
> <Native command: executed by Hive>
> After renaming:
> scala> sql("select cre_ts from pmt limit 1").collect
> res16: Array[org.apache.spark.sql.Row] = Array([null])
> Jianshi
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org