You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Mohit Jaggi (JIRA)" <ji...@apache.org> on 2015/07/20 08:07:04 UTC
[jira] [Commented] (SPARK-9181) round bracket used in naming
aggregations is not allowed by parquet writer
[ https://issues.apache.org/jira/browse/SPARK-9181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633080#comment-14633080 ]
Mohit Jaggi commented on SPARK-9181:
------------------------------------
example --
scala>sdf.groupBy("col0_0").agg(avg("col2_2")).show()
+-----------+-----------+
| col0_0|AVG(col2_2)|
+-----------+-----------+
|[B@4b9bb066| 23.2|
|[B@5a5a1f51| 900.7|
|[B@52be2d2e| 123.3|
|[B@43885172| 2.987|
|[B@2a9b3601| 21.9|
|[B@33f69181| 3.678|
| [B@cac6ce| 1.897|
| [B@c1feca| 90.2|
+-----------+-----------+
scala> sdf.groupBy("col0_0").agg(avg("col2_2")).write.parquet("/tmp/x1")
15/07/19 23:03:40 ERROR InsertIntoHadoopFsRelation: Aborting job.
java.lang.RuntimeException: Attribute name "AVG(col2_2)" contains invalid character(s) among " ,;{}() =". Please use alias to rename it.
> round bracket used in naming aggregations is not allowed by parquet writer
> --------------------------------------------------------------------------
>
> Key: SPARK-9181
> URL: https://issues.apache.org/jira/browse/SPARK-9181
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.4.0
> Reporter: Mohit Jaggi
> Priority: Minor
>
> round bracket used in naming aggregations is not allowed by parquet writer. so one has to rename the aggregate columns before writing to parquet.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org