You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Song Jun (JIRA)" <ji...@apache.org> on 2016/12/29 06:43:58 UTC
[jira] [Commented] (SPARK-18930) Inserting in partitioned table -
partitioned field should be last in select statement.
[ https://issues.apache.org/jira/browse/SPARK-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15784678#comment-15784678 ]
Song Jun commented on SPARK-18930:
----------------------------------
from hive document, https://cwiki.apache.org/confluence/display/Hive/Tutorial#Tutorial-Dynamic-PartitionInsert
Note that the dynamic partition values are selected by ordering, not name, and taken as the last columns from the select clause.
and test it on hive also have the same logic as your description .
I think we can close this jira?
> Inserting in partitioned table - partitioned field should be last in select statement.
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-18930
> URL: https://issues.apache.org/jira/browse/SPARK-18930
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.2
> Reporter: Egor Pahomov
>
> CREATE TABLE temp.test_partitioning_4 (
> num string
> )
> PARTITIONED BY (
> day string)
> stored as parquet
> INSERT INTO TABLE temp.test_partitioning_4 PARTITION (day)
> select day, count(*) as num from
> hss.session where year=2016 and month=4
> group by day
> Resulted schema on HDFS: /temp.db/test_partitioning_3/day=62456298, emp.db/test_partitioning_3/day=69094345....
> As you can imagine these numbers are num of records. But! When I do select * from temp.test_partitioning_4 data is correct.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org