You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by viirya <gi...@git.apache.org> on 2017/05/07 05:11:17 UTC
[GitHub] spark pull request #17887: [SPARK-20399][SQL][WIP] Add a config to fallback ...
GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/17887
[SPARK-20399][SQL][WIP] Add a config to fallback string literal parsing consistent with old sql parser behavior
## What changes were proposed in this pull request?
Follow the discussion in #17736, this patch adds a config to fallback to 1.6 string literal parsing.
## How was this patch tested?
Jenkins tests.
Please review http://spark.apache.org/contributing.html before opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 add-config-fallback-string-parsing
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17887.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17887
----
commit d0b2c2278ec7d10cc1ab998be489e6553a8dc193
Author: Liang-Chi Hsieh <vi...@gmail.com>
Date: 2017-04-19T01:49:47Z
Add a config to fallback string literal parsing consistent with old sql parser behavior.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115801045
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +151,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if the `str` parameter is "abc\td", the `regexp` can match it is:
+ "^abc\\\\td$".
+
+ Examples:
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\\Users.*'
+ true
+
+ There is a SQL config 'spark.sql.parser.escapedStringLiterals' can be used to fallback
+ to Spark 1.6 behavior regarding string literal parsing. For example, if the config is
+ enabled, the `regexp` can match "abc\td" is "^abc\\t$".
--- End diff --
`can match` -> `that can match`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115908703
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +151,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if the `str` parameter is "abc\td", the `regexp` can match it is:
+ "^abc\\\\td$".
+
+ Examples:
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\\Users.*'
+ true
--- End diff --
Ok.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76611/testReport)** for PR 17887 at commit [`04a9fd3`](https://github.com/apache/spark/commit/04a9fd34c7489079da2b02a8f3a5ca84d87b0017).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76722/testReport)** for PR 17887 at commit [`9ce7eb0`](https://github.com/apache/spark/commit/9ce7eb0450249fdc25e19adf6bcfe35b274dd086).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76559 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76559/testReport)** for PR 17887 at commit [`8ae0747`](https://github.com/apache/spark/commit/8ae074784d145dc2298fa81c0c9097aad48fc349).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/17887
@gatorsmile OK. Let me update it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115731376
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +144,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if the `str` parameter is "abc\td", the `regexp` can match it is:
+ "^abc\\\\td$".
+
+ Examples:
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\\Users.*'
+ true
+
+ There is a SQL config 'spark.sql.parser.escapedStringLiterals' can be used to fallback
+ to Spark 1.6 behavior regarding string literal parsing. For example, if the config is
+ enabled, the `regexp` can match "abc\td" is "^abc\\t$".
+
+ Examples (spark.sql.parser.escapedStringLiterals is enabled):
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\Users.*'
+ true
+
+ See also:
+ Use LIKE to match with simple string pattern.
--- End diff --
I've updated the doc of `Like`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76611 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76611/testReport)** for PR 17887 at commit [`04a9fd3`](https://github.com/apache/spark/commit/04a9fd34c7489079da2b02a8f3a5ca84d87b0017).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r116060915
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +151,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if to match "\abc", a regular expression for `regexp` can be "^\\abc$".
--- End diff --
`if to match` has a grammar issue. You need to change it to `to match`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/17887
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76559/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/17887
Could you update the involved function description? For example, `RLike`? I believe not only @dbtsai 's team hit this issue. It should be documented in the function description. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76722/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76786/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76540/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76814 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76814/testReport)** for PR 17887 at commit [`8ecb2ea`](https://github.com/apache/spark/commit/8ecb2eacaa2bec7837dd68e8dd0563a15204c3d4).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115207039
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala ---
@@ -447,6 +462,44 @@ class ExpressionParserSuite extends PlanTest {
assertEqual("'\\u0057\\u006F\\u0072\\u006C\\u0064\\u0020\\u003A\\u0029'", "World :)")
}
+ test("strings with ESCAPED_STRING_LITERALS = true") {
--- End diff --
we have a very similar test case `strings`, can we merge them?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115660739
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -196,6 +196,14 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val ESCAPED_STRING_LITERALS = buildConf("spark.sql.parser.escapedStringLiterals")
+ .internal()
+ .doc("When true, string literals (including regex patterns) remains escaped in our SQL " +
--- End diff --
Nit: `remains` -> `remain`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76560/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115717329
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +144,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if the `str` parameter is "abc\td", the `regexp` can match it is:
+ "^abc\\\\td$".
+
+ Examples:
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\\Users.*'
+ true
+
+ There is a SQL config 'spark.sql.parser.escapedStringLiterals' can be used to fallback
+ to Spark 1.6 behavior regarding string literal parsing. For example, if the config is
+ enabled, the `regexp` can match "abc\td" is "^abc\\t$".
+
+ Examples (spark.sql.parser.escapedStringLiterals is enabled):
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\Users.*'
+ true
+
+ See also:
+ Use LIKE to match with simple string pattern.
--- End diff --
shall we also update the document of LIKE?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17887
thanks, merging to master/2.2!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/17887
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76559 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76559/testReport)** for PR 17887 at commit [`8ae0747`](https://github.com/apache/spark/commit/8ae074784d145dc2298fa81c0c9097aad48fc349).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76814/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76735/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76782 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76782/testReport)** for PR 17887 at commit [`d8cd670`](https://github.com/apache/spark/commit/d8cd670aaf0050c8ebe96302709eadef671c615b).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76786 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76786/testReport)** for PR 17887 at commit [`d8cd670`](https://github.com/apache/spark/commit/d8cd670aaf0050c8ebe96302709eadef671c615b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115800867
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +151,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if the `str` parameter is "abc\td", the `regexp` can match it is:
+ "^abc\\\\td$".
+
+ Examples:
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\\Users.*'
+ true
+
+ There is a SQL config 'spark.sql.parser.escapedStringLiterals' can be used to fallback
+ to Spark 1.6 behavior regarding string literal parsing. For example, if the config is
--- End diff --
`Spark 1.6 behavior` -> `the Spark 1.6 behavior`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/17887
Thanks @cloud-fan @gatorsmile
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/17887
LGTM cc @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/17887
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115730446
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +144,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if the `str` parameter is "abc\td", the `regexp` can match it is:
+ "^abc\\\\td$".
+
+ Examples:
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\\Users.*'
+ true
+
+ There is a SQL config 'spark.sql.parser.escapedStringLiterals' can be used to fallback
+ to Spark 1.6 behavior regarding string literal parsing. For example, if the config is
+ enabled, the `regexp` can match "abc\td" is "^abc\\t$".
+
+ Examples (spark.sql.parser.escapedStringLiterals is enabled):
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\Users.*'
+ true
+
+ See also:
+ Use LIKE to match with simple string pattern.
--- End diff --
Ah. I think we don't need to update the doc of `Like`. The two special symbols `%` and `_` are parsed in the same way as 1.6 parser.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL][WIP] Add a config to fallback ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115162139
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala ---
@@ -68,6 +68,11 @@ object ParserUtils {
/** Convert a string node into a string. */
def string(node: TerminalNode): String = unescapeSQLString(node.getText)
+ /** Convert a string node into a string without unescaping. */
+ def stringWithoutUnescape(node: TerminalNode): String = {
+ node.getText.slice(1, node.getText.size - 1)
--- End diff --
The string rule in SqlBase.g4 forces that the input has always quotes at the starting and ending. I may add a comment here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76729/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115728334
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +144,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if the `str` parameter is "abc\td", the `regexp` can match it is:
+ "^abc\\\\td$".
+
+ Examples:
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\\Users.*'
+ true
+
+ There is a SQL config 'spark.sql.parser.escapedStringLiterals' can be used to fallback
+ to Spark 1.6 behavior regarding string literal parsing. For example, if the config is
+ enabled, the `regexp` can match "abc\td" is "^abc\\t$".
+
+ Examples (spark.sql.parser.escapedStringLiterals is enabled):
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\Users.*'
+ true
+
+ See also:
+ Use LIKE to match with simple string pattern.
--- End diff --
I was afraid of duplication info there. But OK, let me add few lines into `Like` too.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115800730
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +151,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if the `str` parameter is "abc\td", the `regexp` can match it is:
+ "^abc\\\\td$".
+
+ Examples:
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\\Users.*'
+ true
+
+ There is a SQL config 'spark.sql.parser.escapedStringLiterals' can be used to fallback
--- End diff --
`can be used` -> `that can be used`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76560/testReport)** for PR 17887 at commit [`ab77de7`](https://github.com/apache/spark/commit/ab77de7d057ed284160fc051bfbfd6afb0a3a923).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115995239
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +151,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if to match "abc\td", a regular expression for `regexp` can be "^abc\\\\td$".
--- End diff --
OK. Let me change to SQL shell string.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL][WIP] Add a config to fallback ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115162152
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -196,6 +196,14 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val NO_UNESCAPED_SQL_STRING = buildConf("spark.sql.noUnescapedStringLiteral")
+ .internal()
+ .doc("Since Spark 2.0, we use unescaped SQL string for string literals including regex. " +
+ "It is different than 1.6 behavior. Enabling this config can use no unescaped SQL string " +
+ "literals and mitigate migration problem.")
--- End diff --
Sure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76542/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r116130809
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +151,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if to match "\abc", a regular expression for `regexp` can be "^\\abc$".
--- End diff --
Sure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115801833
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +151,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if the `str` parameter is "abc\td", the `regexp` can match it is:
+ "^abc\\\\td$".
+
+ Examples:
+ > SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\\Users.*'
+ true
--- End diff --
This is when `spark.sql.parser.escapedStringLiterals` is set to false.
How about moving these two examples in the same place? Then, we can clearly explain the behavior differences caused by `spark.sql.parser.escapedStringLiterals`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/17887
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115959442
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +151,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if to match "abc\td", a regular expression for `regexp` can be "^abc\\\\td$".
--- End diff --
I think the example should be based on SQL shell instead of java string literal, here should be `"^abc\\td$"`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17887
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115406428
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala ---
@@ -1168,6 +1169,18 @@ class DatasetSuite extends QueryTest with SharedSQLContext {
val ds = Seq(WithMapInOption(Some(Map(1 -> 1)))).toDS()
checkDataset(ds, WithMapInOption(Some(Map(1 -> 1))))
}
+
+ test("do not unescaped regex pattern string") {
--- End diff --
add jira id and when we should not unescape
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/17887
OK. I also think about it too after reading the doc of RLike.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76782/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL][WIP] Add a config to fallback ...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115155941
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala ---
@@ -68,6 +68,11 @@ object ParserUtils {
/** Convert a string node into a string. */
def string(node: TerminalNode): String = unescapeSQLString(node.getText)
+ /** Convert a string node into a string without unescaping. */
+ def stringWithoutUnescape(node: TerminalNode): String = {
+ node.getText.slice(1, node.getText.size - 1)
--- End diff --
For safety, do we still need to check whether the starting and ending characters are quotes?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76750 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76750/testReport)** for PR 17887 at commit [`e854b10`](https://github.com/apache/spark/commit/e854b104e61681724b4e90fc2e480ccebfd75ae4).
* This patch **fails from timeout after a configured wait of \`250m\`**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115406775
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala ---
@@ -413,38 +428,102 @@ class ExpressionParserSuite extends PlanTest {
}
test("strings") {
--- End diff --
how about something like
```
Seq(true, false).foreach { escape =>
val conf = new SQLConf()
conf.setConfString(SQLConf.ESCAPED_STRING_LITERALS.key, "true")
val parser = new CatalystSqlParser(conf)
// tests that have same result whatever the conf is
assertEqual("\"hello\"", "hello")
...
// tests that have different result regarding the conf
if (escape) {
assert(...)
...
} else {
assert(...)
...
}
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17887
LGTM except the document change like @gatorsmile suggested
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/17887
Thanks @viirya We'll backport it and test it out soon. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76563 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76563/testReport)** for PR 17887 at commit [`ab77de7`](https://github.com/apache/spark/commit/ab77de7d057ed284160fc051bfbfd6afb0a3a923).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/17887
Generally, it looks reasonable to me. Also cc @jodersky who hit this issue before.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76563/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76729 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76729/testReport)** for PR 17887 at commit [`3241b88`](https://github.com/apache/spark/commit/3241b88c37478652c78b1d8d4809385b47410c51).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76722/testReport)** for PR 17887 at commit [`9ce7eb0`](https://github.com/apache/spark/commit/9ce7eb0450249fdc25e19adf6bcfe35b274dd086).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76837 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76837/testReport)** for PR 17887 at commit [`375eb9c`](https://github.com/apache/spark/commit/375eb9cd747cc75d2f51da1dabe824dbbce52790).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115800474
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ---
@@ -144,7 +151,31 @@ case class Like(left: Expression, right: Expression) extends StringRegexExpressi
}
@ExpressionDescription(
- usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.")
+ usage = "str _FUNC_ regexp - Returns true if `str` matches `regexp`, or false otherwise.",
+ extended = """
+ Arguments:
+ str - a string expression
+ regexp - a string expression. The pattern string should be a Java regular expression.
+
+ Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL parser.
+ For example, if the `str` parameter is "abc\td", the `regexp` can match it is:
--- End diff --
> For example, if the `str` parameter is "abc\td", the `regexp` can match it is: "^abc\\\\td$".
->
> For example, to match "abc\td", a regular expression for `regexp` can be "^abc\\\\td$".
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76542/testReport)** for PR 17887 at commit [`d0b2c22`](https://github.com/apache/spark/commit/d0b2c2278ec7d10cc1ab998be489e6553a8dc193).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115205264
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala ---
@@ -160,6 +166,15 @@ class ExpressionParserSuite extends PlanTest {
assertEqual("a not regexp 'pattern%'", !('a rlike "pattern%"))
}
+ test("like expressions with ESCAPED_STRING_LITERALS = true") {
+ val conf = new SQLConf()
+ conf.setConfString("spark.sql.parser.escapedStringLiterals", "true")
--- End diff --
use `SQLConf. ESCAPED_STRING_LITERALS.key`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL][WIP] Add a config to fallback ...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115155773
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -196,6 +196,14 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val NO_UNESCAPED_SQL_STRING = buildConf("spark.sql.noUnescapedStringLiteral")
--- End diff --
Double negatives are not encouraged in conf naming. This sounds the first parser conf.
How about `spark.sql.parser.escapeStringLiterals`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76542 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76542/testReport)** for PR 17887 at commit [`d0b2c22`](https://github.com/apache/spark/commit/d0b2c2278ec7d10cc1ab998be489e6553a8dc193).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class AstBuilder(conf: SQLConf) extends SqlBaseBaseVisitor[AnyRef] with Logging `
* `class CatalystSqlParser(conf: SQLConf) extends AbstractSqlParser `
* `class SparkSqlAstBuilder(conf: SQLConf) extends AstBuilder(conf) `
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL][WIP] Add a config to fallback ...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115156337
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -196,6 +196,14 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val NO_UNESCAPED_SQL_STRING = buildConf("spark.sql.noUnescapedStringLiteral")
+ .internal()
+ .doc("Since Spark 2.0, we use unescaped SQL string for string literals including regex. " +
+ "It is different than 1.6 behavior. Enabling this config can use no unescaped SQL string " +
+ "literals and mitigate migration problem.")
--- End diff --
How about
> When true, string literals (including regex patterns) remains escaped in our SQL parser. The default is false since Spark 2.0. Setting it to `true` can restore the behavior prior to Spark 2.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115649694
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala ---
@@ -413,38 +428,102 @@ class ExpressionParserSuite extends PlanTest {
}
test("strings") {
--- End diff --
Sure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76837/testReport)** for PR 17887 at commit [`375eb9c`](https://github.com/apache/spark/commit/375eb9cd747cc75d2f51da1dabe824dbbce52790).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/17887
cc @dbtsai @cloud-fan @hvanhovell
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76735/testReport)** for PR 17887 at commit [`c81f030`](https://github.com/apache/spark/commit/c81f030902d3a65298141ec6296d5d564fd68a66).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76837/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76540/testReport)** for PR 17887 at commit [`d0b2c22`](https://github.com/apache/spark/commit/d0b2c2278ec7d10cc1ab998be489e6553a8dc193).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76750 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76750/testReport)** for PR 17887 at commit [`e854b10`](https://github.com/apache/spark/commit/e854b104e61681724b4e90fc2e480ccebfd75ae4).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76786 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76786/testReport)** for PR 17887 at commit [`d8cd670`](https://github.com/apache/spark/commit/d8cd670aaf0050c8ebe96302709eadef671c615b).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL][WIP] Add a config to fallback string ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115661190
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -196,6 +196,14 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val ESCAPED_STRING_LITERALS = buildConf("spark.sql.parser.escapedStringLiterals")
+ .internal()
+ .doc("When true, string literals (including regex patterns) remains escaped in our SQL " +
--- End diff --
Sure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17887: [SPARK-20399][SQL][WIP] Add a config to fallback ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17887#discussion_r115162164
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -196,6 +196,14 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val NO_UNESCAPED_SQL_STRING = buildConf("spark.sql.noUnescapedStringLiteral")
--- End diff --
Sure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76750/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76814 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76814/testReport)** for PR 17887 at commit [`8ecb2ea`](https://github.com/apache/spark/commit/8ecb2eacaa2bec7837dd68e8dd0563a15204c3d4).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/17887
Please also add some examples in the function descriptions? It might help users understand how to correctly escape it. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76563 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76563/testReport)** for PR 17887 at commit [`ab77de7`](https://github.com/apache/spark/commit/ab77de7d057ed284160fc051bfbfd6afb0a3a923).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17887
**[Test build #76735 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76735/testReport)** for PR 17887 at commit [`c81f030`](https://github.com/apache/spark/commit/c81f030902d3a65298141ec6296d5d564fd68a66).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17887
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76611/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org