You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jarno Rajala (Jira)" <ji...@apache.org> on 2025/04/08 09:11:00 UTC
[jira] [Updated] (SPARK-51744) Columns with spaces cannot be aliased or used in expressions when quotedRegexColumnNames=true
[ https://issues.apache.org/jira/browse/SPARK-51744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jarno Rajala updated SPARK-51744:
---------------------------------
Description:
When {{{}spark.sql.parser.quotedRegexColumnNames=true{}}}, the following query fails:
{{select `X Y` as xy from (select 1 as `X Y`)}}
giving the error message
{{[INVALID_USAGE_OF_STAR_OR_REGEX] Invalid usage of regular expression 'X Y' in expression `alias`. SQLSTATE: 42000}}
Similar errors occur with other expressions like {{{}`X Y`/2{}}}. This severely restricts use of columns with spaces when {{{}quotedRegexColumnNames=true{}}}.
There are many ways that this could be fixed.
I would suggest at least that backticked columns not be treated as regexes if they do not contain any regex tokens.
This issue is related to SPARK-36698. However, that issue is a feature request and not specifically about column names with spaces. However, the proposed PR would presumably fix this issue as well.
was:
When {{{}spark.sql.parser.quotedRegexColumnNames=true{}}}, the following query fails:
{{select `X Y` as xy from (select 1 as `X Y`)}}
giving the error message
{{[INVALID_USAGE_OF_STAR_OR_REGEX] Invalid usage of regular expression 'X Y' in expression `alias`. SQLSTATE: 42000}}
Similar errors occur with other expressions like {{{}`X Y`/2{}}}. This severely restricts use of columns with spaces when {{{}quotedRegexColumnNames=true{}}}.
There are many ways that this could be fixed.
I would suggest at least that backticked columns not be treated as regexes if they do not contain any regex tokens.
This issue is related to SPARK-36698. However, that issue is a feature request and not specifically about column names with spaces.
> Columns with spaces cannot be aliased or used in expressions when quotedRegexColumnNames=true
> ---------------------------------------------------------------------------------------------
>
> Key: SPARK-51744
> URL: https://issues.apache.org/jira/browse/SPARK-51744
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.3.2, 3.5.0
> Reporter: Jarno Rajala
> Priority: Minor
>
> When {{{}spark.sql.parser.quotedRegexColumnNames=true{}}}, the following query fails:
> {{select `X Y` as xy from (select 1 as `X Y`)}}
> giving the error message
> {{[INVALID_USAGE_OF_STAR_OR_REGEX] Invalid usage of regular expression 'X Y' in expression `alias`. SQLSTATE: 42000}}
> Similar errors occur with other expressions like {{{}`X Y`/2{}}}. This severely restricts use of columns with spaces when {{{}quotedRegexColumnNames=true{}}}.
> There are many ways that this could be fixed.
> I would suggest at least that backticked columns not be treated as regexes if they do not contain any regex tokens.
> This issue is related to SPARK-36698. However, that issue is a feature request and not specifically about column names with spaces. However, the proposed PR would presumably fix this issue as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org