You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2022/03/18 03:00:02 UTC
[jira] [Commented] (SPARK-38592) Column name contains back tick `
[ https://issues.apache.org/jira/browse/SPARK-38592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17508536#comment-17508536 ]
Hyukjin Kwon commented on SPARK-38592:
--------------------------------------
[~JuzDDM] mind showing the fully self-contained reproducer?
> Column name contains back tick `
> --------------------------------
>
> Key: SPARK-38592
> URL: https://issues.apache.org/jira/browse/SPARK-38592
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.1
> Reporter: Dennis Du
> Priority: Major
>
> Try to modify the data frame to ensure column names have no special characters.
> {code:java}
> df.columns.map { columnName =>
> df.col(surroundingBackTickedName(columnName)).as(normalizeName(columnName))
> }
> {code}
> *surroundingBackTickedName()* will enclose column name with backticks.
> However, col() kept having issue with column name contains back tick because {*}parseAttributeName{*}() only takes backticks that appear in pair. I am wondering if there is a workaround
> {code:java}
> org.apache.spark.sql.AnalysisException: Cannot resolve column name "`COLNAME`2`" among (COLID, COLNAME!4, COLNAME#6, COLNAME$7, COLNAME%8, COLNAME'25, COLNAME(11, COLNAME)12, COLNAME*10, COLNAME+16, COLNAME,26, COLNAME-13, COLNAME/30, COLNAME:24, COLNAME;23, COLNAME<27, COLNAME=15, COLNAME>29, COLNAME?31, COLNAME@5, COLNAME`2){code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org