You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Emanuele Bardelli (JIRA)" <ji...@apache.org> on 2019/06/03 12:45:01 UTC
[jira] [Commented] (SPARK-26388) No support for "alter table ..
replace columns" to drop columns
[ https://issues.apache.org/jira/browse/SPARK-26388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854557#comment-16854557 ]
Emanuele Bardelli commented on SPARK-26388:
-------------------------------------------
Just submitted a PR for this, reviews and feedback welcome!
> No support for "alter table .. replace columns" to drop columns
> ---------------------------------------------------------------
>
> Key: SPARK-26388
> URL: https://issues.apache.org/jira/browse/SPARK-26388
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.1, 2.3.1, 2.3.2
> Reporter: nirav patel
> Priority: Major
>
> Looks like hive {{replace columns}} is not working with spark 2.2.1 and 2.3.1
>
> create table myschema.mytable(a int, b int, c int)
> alter table myschema.mytable replace columns (a int,b int,d int)
>
> *Expected Behavior*
> it should drop column c and add column d.
> alter table... replace columns.. should work just as it works in hive.
> It replaces existing columns with new ones. It delete if column is not mentioned.
>
> here's the snippet of hive cli:
> hive> desc mytable;
> OK
> a int
> b int
> c int
> Time taken: 0.05 seconds, Fetched: 3 row(s)
> hive> alter table mytable replace columns(a int, b int, d int);
> OK
> Time taken: 0.078 seconds
> hive> desc mytable;
> OK
> a int
> b int
> d int
> Time taken: 0.03 seconds, Fetched: 3 row(s)
>
> *Actual Result*
> Exception in thread "main" org.apache.spark.sql.catalyst.parser.ParseException: Operation not allowed: alter table replace columns
> {{ADD COLUMNS}} works which seemed to previously reported and fixed as well:
> https://issues.apache.org/jira/browse/SPARK-18893
>
> Replace columns should be supported as well. afaik, that's the only way to delete hive columns.
>
>
> It supposed to work according to this docs:
> [https://docs.databricks.com/spark/latest/spark-sql/language-manual/alter-table-or-view.html#replace-columns]
> [https://spark.apache.org/docs/2.2.0/sql-programming-guide.html#supported-hive-features]
>
> but it's throwing error for me on 2 different versions.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org