You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2017/07/26 09:17:00 UTC
[jira] [Commented] (SPARK-21536) Remove the workaroud to allow dots
in field names in R's createDataFame
[ https://issues.apache.org/jira/browse/SPARK-21536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101410#comment-16101410 ]
Hyukjin Kwon commented on SPARK-21536:
--------------------------------------
I just realised what I found while fixing this is related with SPARK-12191.
> Remove the workaroud to allow dots in field names in R's createDataFame
> -----------------------------------------------------------------------
>
> Key: SPARK-21536
> URL: https://issues.apache.org/jira/browse/SPARK-21536
> Project: Spark
> Issue Type: Improvement
> Components: SparkR
> Affects Versions: 2.3.0
> Reporter: Hyukjin Kwon
> Priority: Minor
>
> We are currently converting dots to underscore in some cases as below in SparkR
> {code}
> > createDataFrame(list(list(1)), "a.a")
> SparkDataFrame[a_a:double]
> ...
> In FUN(X[[i]], ...) : Use a_a instead of a.a as column name
> {code}
> {code}
> > createDataFrame(list(list(a.a = 1)))
> SparkDataFrame[a_a:double]
> ...
> In FUN(X[[i]], ...) : Use a_a instead of a.a as column name
> {code}
> This looks introduced in the first place due to SPARK-2775 but now it is fixed in SPARK-6898.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org