You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "MaxGekk (via GitHub)" <gi...@apache.org> on 2023/09/21 20:18:34 UTC
[GitHub] [spark] MaxGekk opened a new pull request, #43037: [WIP][DOCS] Improve examples for `LIKE`
MaxGekk opened a new pull request, #43037:
URL: https://github.com/apache/spark/pull/43037
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
8. If you want to add or modify an error type or message, please read the guideline first in
'core/src/main/resources/error/README.md'.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
The description of `LIKE` says:
```
... in order to match "\abc", the pattern should be "\\abc"
```
but in Spark SQL shell:
```sql
spark-sql (default)> SELECT c FROM t;
\abc
spark-sql (default)> SELECT c LIKE "\\abc" FROM t;
[INVALID_FORMAT.ESC_IN_THE_MIDDLE] The format is invalid: '\\abc'. The escape character is not allowed to precede 'a'.
spark-sql (default)> SELECT c LIKE "\\\\abc" FROM t;
true
```
So, the description might confuse users since the pattern must contain 4 slashes.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
By running new and modified tests:
```
$ build/sbt "test:testOnly *.StringFunctionsSuite"
$ build/sbt "sql/test:testOnly org.apache.spark.sql.expressions.ExpressionInfoSuite"
```
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "beliefer (via GitHub)" <gi...@apache.org>.
beliefer commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1333950726
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
`LIKE` and Java both handle `\` and only `LIKE` handles `/`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1333918575
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
> Do we have a doc to explain what the string value we get eventually by writing string literals in SQL?
There is the doc about string literals: https://spark.apache.org/docs/latest/sql-ref-literals.html#string-literal. It explains something implemented in https://github.com/apache/spark/blob/5f643ee1444d41ca78ef397d9675efac550a6967/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkParserUtils.scala#L34-L54
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45262][SQL][TESTS][DOCS] Improve examples for regexp parameters [spark]
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on PR #43037:
URL: https://github.com/apache/spark/pull/43037#issuecomment-1751143739
@srowen The failure of `ExpressionsSchemaSuite` is related to my changes. I have regenerated `sql-expression-schema.md` to fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45262][SQL][TESTS][DOCS] Improve examples for regexp parameters [spark]
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on PR #43037:
URL: https://github.com/apache/spark/pull/43037#issuecomment-1751313881
Merging to master. Thank you, @srowen @cloud-fan @beliefer @dongjoon-hyun for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1333690369
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
This is the example you want to fix, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45262][SQL][TESTS][DOCS] Improve examples for regexp parameters [spark]
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1348736128
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -108,13 +108,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
Review Comment:
Added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "beliefer (via GitHub)" <gi...@apache.org>.
beliefer commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1333914632
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
> https://github.com/apache/spark/blob/fdedec1de54fa7619e1d1b59eb6f15a49e727428/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala#L95-L96
I guess the "\abc" means the fact value but the literal. Of course, it make users confused. I agree to update the doc here to give more explanation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45262][SQL][TESTS][DOCS] Improve examples for regexp parameters [spark]
Posted by "srowen (via GitHub)" <gi...@apache.org>.
srowen commented on PR #43037:
URL: https://github.com/apache/spark/pull/43037#issuecomment-1751113205
Just rerun tests, I guess, though doesn't look related. I'm surprised at the number of build issues we've had lately
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1333691609
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
Do we need to change this part?
https://github.com/apache/spark/blob/fdedec1de54fa7619e1d1b59eb6f15a49e727428/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala#L95-L96
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1333976698
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
> Use \ to escape special characters
So for `\a` that doesn't escape anything, `\` is treated as a normal char.
I think the problem here is the LIKE doc describes the behavior with the actual string value of the pattern. Can we reference the string literal doc here? And give examples about how to match `\abc`: `r'\\abc'` or `'\\\\abc'`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1333938363
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
> This is the example you want to fix, right?
> SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
@dongjoon-hyun I would change this to don't mix 2 different things: how the SQL parser handles regular string literals and how `LIKE` handles patterns. Not obvious why those 2 escapes `/` and `\` lead to 2 different patterns:
```sql
spark-sql (default)> SELECT R'%SystemDrive%\Users\John' LIKE '\%SystemDrive\%\\\\Users%' ESCAPE R'\';
true
spark-sql (default)> SELECT R'%SystemDrive%/Users/John' LIKE '/%SystemDrive/%//Users%' ESCAPE R'/';
true
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1333918575
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
> Do we have a doc to explain what the string value we get eventually by writing string literals in SQL?
There is the doc about string literals: https://spark.apache.org/docs/latest/sql-ref-literals.html#string-literal. It explain something implemented in https://github.com/apache/spark/blob/5f643ee1444d41ca78ef397d9675efac550a6967/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkParserUtils.scala#L34-L54
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45262][SQL][TESTS][DOCS] Improve examples for regexp parameters [spark]
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk closed pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for regexp parameters
URL: https://github.com/apache/spark/pull/43037
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "beliefer (via GitHub)" <gi...@apache.org>.
beliefer commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1333950726
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
LIKE and Java both handle `\` and only LIKE handles `/`.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
`LIKE` and Java both handle `\` and only LIKE handles `/`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1333772840
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
I'm seeing some weird behaviors:
```
scala> sql(""" select "a\b" """).show
+---+
|a\b|
+---+
|a\b|
+---+
scala> sql(""" select "a\\b" """).show
+---+
|a\b|
+---+
|a\b|
+---+
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE` [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1348189324
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -108,13 +108,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
Review Comment:
shall we improve the examples for other like functions as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1339026155
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
I have updated the description of string literals in https://github.com/apache/spark/pull/43152, so, we can refer to the doc from here and other expression descriptions. Please, take a look at the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1333758923
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
it looks to me that we follow the java string semantic: if you write `"\\abc"`, you actually get `\abc`. Is it also true in simple `SELECT` not `LIKE`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a diff in pull request #43037: [SPARK-45262][SQL][TESTS][DOCS] Improve examples for `LIKE`
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #43037:
URL: https://github.com/apache/spark/pull/43037#discussion_r1333772437
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -105,13 +105,15 @@ abstract class StringRegexExpression extends BinaryExpression
Examples:
> SELECT _FUNC_('Spark', '_park');
true
+ > SELECT '\\abc' AS S, S _FUNC_ R'\\abc', S _FUNC_ '\\\\abc';
+ \abc true true
> SET spark.sql.parser.escapedStringLiterals=true;
spark.sql.parser.escapedStringLiterals true
> SELECT '%SystemDrive%\Users\John' _FUNC_ '\%SystemDrive\%\\Users%';
true
> SET spark.sql.parser.escapedStringLiterals=false;
spark.sql.parser.escapedStringLiterals false
- > SELECT '%SystemDrive%\\Users\\John' _FUNC_ '\%SystemDrive\%\\\\Users%';
Review Comment:
Do we have a doc to explain what the string value we get eventually by writing string literals in SQL?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org