You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "miland-db (via GitHub)" <gi...@apache.org> on 2024/11/07 16:36:13 UTC
[PR] [SPARK-49913] Add check for unique label names in nested labeled scopes [spark]
miland-db opened a new pull request, #48795:
URL: https://github.com/apache/spark/pull/48795
### What changes were proposed in this pull request?
We are introducing checks for unique label names.
New rules for label names:
- Labels can't have the same name as some of the labels in scope surrounding them
- Labels can have the same name as other labels in the same scope
**Valid** code:
```
BEGIN
lbl: BEGIN
SELECT 1;
END;
lbl: BEGIN
SELECT 2;
END;
BEGIN
lbl: WHILE 1=1 DO
LEAVE lbl;
END WHILE;
END;
END
```
**Invalid** code:
```
BEGIN
lbl: BEGIN
lbl: BEGIN
SELECT 1;
END;
END;
END
```
### Why are the changes needed?
It will be needed in future when we release Local Scoped Variables for SQL Scripting so users can target variables from outer scopes if they are shadowed.
### How was this patch tested?
New unit tests in 'SqlScriptingParserSuite.scala'.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "miland-db (via GitHub)" <gi...@apache.org>.
miland-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1833346770
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingParserSuite.scala:
##########
@@ -1605,6 +1605,227 @@ class SqlScriptingParserSuite extends SparkFunSuite with SQLHelper {
assert(nestedLoopStmt.body.collection(1).asInstanceOf[IterateStatement].label == "lbl")
}
+ test("unique label names: nested begin-end blocks") {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1836176179
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,87 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
Review Comment:
No, each session has a parser instance in `SessionState`, that's why `AstBuilder` is stateless as it can be invoked by multiple threads at the same time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "dejankrak-db (via GitHub)" <gi...@apache.org>.
dejankrak-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1833112078
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingParserSuite.scala:
##########
@@ -1605,6 +1605,227 @@ class SqlScriptingParserSuite extends SparkFunSuite with SQLHelper {
assert(nestedLoopStmt.body.collection(1).asInstanceOf[IterateStatement].label == "lbl")
}
+ test("unique label names: nested begin-end blocks") {
Review Comment:
Consider perhaps also adding a positive unit test where one label name is a substring of the other one (e.g. lbl_1 and lbl_11), to check that this would not fail (this would expose bug if only first N characters up to the shorter label name length were considered as part of uniqueness check)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "davidm-db (via GitHub)" <gi...@apache.org>.
davidm-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1834586363
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,70 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
+ private val seenLabels: mutable.Set[String] = mutable.Set.empty
+
+ def init(): Unit = {
+ seenLabels.clear()
+ }
+
+ private def checkLabels(
+ beginLabelCtx: Option[BeginLabelContext],
+ endLabelCtx: Option[EndLabelContext]) : Unit = {
+ (beginLabelCtx, endLabelCtx) match {
+ case (Some(bl: BeginLabelContext), Some(el: EndLabelContext))
+ if bl.multipartIdentifier().getText.nonEmpty &&
Review Comment:
Removed my comment already, I wrote it before fully realizing what's going on :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #48795:
URL: https://github.com/apache/spark/pull/48795#issuecomment-2475202215
The linter failure is unrelated, thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1833522964
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,87 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
Review Comment:
Or we pass around a `LabelUtils` instance when parsing a single query.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "davidm-db (via GitHub)" <gi...@apache.org>.
davidm-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1834105742
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,70 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
+ private val seenLabels: mutable.Set[String] = mutable.Set.empty
+
+ def init(): Unit = {
+ seenLabels.clear()
+ }
+
+ private def checkLabels(
+ beginLabelCtx: Option[BeginLabelContext],
+ endLabelCtx: Option[EndLabelContext]) : Unit = {
+ (beginLabelCtx, endLabelCtx) match {
+ case (Some(bl: BeginLabelContext), Some(el: EndLabelContext))
+ if bl.multipartIdentifier().getText.nonEmpty &&
Review Comment:
I think you are right, but also doesn't hurt to have this check in case someone changes grammar for `multipartIdentifier` in the future for whatever reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "davidm-db (via GitHub)" <gi...@apache.org>.
davidm-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1834083008
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,87 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
Review Comment:
This crossed my mind yesterday, but we weren't sure. If that's the case, then you are right - I would say we just need to make `seenLabels` thread local because that's the only state required to track the labels.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan closed pull request #48795: [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes
URL: https://github.com/apache/spark/pull/48795
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "davidm-db (via GitHub)" <gi...@apache.org>.
davidm-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1834587169
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,70 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
+ private val seenLabels: mutable.Set[String] = mutable.Set.empty
+
+ def init(): Unit = {
+ seenLabels.clear()
+ }
+
+ private def checkLabels(
+ beginLabelCtx: Option[BeginLabelContext],
+ endLabelCtx: Option[EndLabelContext]) : Unit = {
+ (beginLabelCtx, endLabelCtx) match {
+ case (Some(bl: BeginLabelContext), Some(el: EndLabelContext))
+ if bl.multipartIdentifier().getText.nonEmpty &&
Review Comment:
In case you haven't seen my previous comment, I think this looks good at the moment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "miland-db (via GitHub)" <gi...@apache.org>.
miland-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1834265053
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,87 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
Review Comment:
@cloud-fan is `AstBuilder` instantiated every time when parser is invoked? Is it safe to keep some state inside `AstBuilder` class?
For `LabelUtils` for now is it enough to make `seenLabels` ThreadLocal? Can single parsing invocation be done from different threads, like threadpool with different threads that get appointed to do part of the parsing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "dejankrak-db (via GitHub)" <gi...@apache.org>.
dejankrak-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1833121469
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,70 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
+ private val seenLabels: mutable.Set[String] = mutable.Set.empty
+
+ def init(): Unit = {
+ seenLabels.clear()
+ }
+
+ private def checkLabels(
+ beginLabelCtx: Option[BeginLabelContext],
+ endLabelCtx: Option[EndLabelContext]) : Unit = {
+ (beginLabelCtx, endLabelCtx) match {
+ case (Some(bl: BeginLabelContext), Some(el: EndLabelContext))
+ if bl.multipartIdentifier().getText.nonEmpty &&
+ bl.multipartIdentifier().getText.toLowerCase(Locale.ROOT) !=
+ el.multipartIdentifier().getText.toLowerCase(Locale.ROOT) =>
+ withOrigin(bl) {
+ seenLabels.clear()
+ throw SqlScriptingErrors.labelsMismatch(
+ CurrentOrigin.get,
+ bl.multipartIdentifier().getText,
+ el.multipartIdentifier().getText)
+ }
+ case (None, Some(el: EndLabelContext)) =>
Review Comment:
Do you also need to explicitly handle case (Some(bl: BeginLabelContext), None), or this can fall into the default case _ below?
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,70 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
+ private val seenLabels: mutable.Set[String] = mutable.Set.empty
+
+ def init(): Unit = {
+ seenLabels.clear()
+ }
+
+ private def checkLabels(
+ beginLabelCtx: Option[BeginLabelContext],
+ endLabelCtx: Option[EndLabelContext]) : Unit = {
+ (beginLabelCtx, endLabelCtx) match {
+ case (Some(bl: BeginLabelContext), Some(el: EndLabelContext))
+ if bl.multipartIdentifier().getText.nonEmpty &&
Review Comment:
If begin label is empty, will the check always pass? Is this intended?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "miland-db (via GitHub)" <gi...@apache.org>.
miland-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1833358190
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,70 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
+ private val seenLabels: mutable.Set[String] = mutable.Set.empty
+
+ def init(): Unit = {
+ seenLabels.clear()
+ }
+
+ private def checkLabels(
+ beginLabelCtx: Option[BeginLabelContext],
+ endLabelCtx: Option[EndLabelContext]) : Unit = {
+ (beginLabelCtx, endLabelCtx) match {
+ case (Some(bl: BeginLabelContext), Some(el: EndLabelContext))
+ if bl.multipartIdentifier().getText.nonEmpty &&
Review Comment:
This checks that if both labels are defined (not None) and if they hold different string value it should fail as _BeginLabel_ and _EndLabel_ must match.
When I look at it now, I think we can remove `bl.multipartIdentifier().getText.nonEmpty` part from the check because we know that label is defined.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,70 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
+ private val seenLabels: mutable.Set[String] = mutable.Set.empty
+
+ def init(): Unit = {
+ seenLabels.clear()
+ }
+
+ private def checkLabels(
+ beginLabelCtx: Option[BeginLabelContext],
+ endLabelCtx: Option[EndLabelContext]) : Unit = {
+ (beginLabelCtx, endLabelCtx) match {
+ case (Some(bl: BeginLabelContext), Some(el: EndLabelContext))
+ if bl.multipartIdentifier().getText.nonEmpty &&
+ bl.multipartIdentifier().getText.toLowerCase(Locale.ROOT) !=
+ el.multipartIdentifier().getText.toLowerCase(Locale.ROOT) =>
+ withOrigin(bl) {
+ seenLabels.clear()
+ throw SqlScriptingErrors.labelsMismatch(
+ CurrentOrigin.get,
+ bl.multipartIdentifier().getText,
+ el.multipartIdentifier().getText)
+ }
+ case (None, Some(el: EndLabelContext)) =>
Review Comment:
This can fall into default `case _` because if _BeginLabel_ is defined and _EndLabel_ is not, that is always ok (except if BeginLabel is duplicate of some parent label but we are not concerned with that part here).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "miland-db (via GitHub)" <gi...@apache.org>.
miland-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1840071162
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,87 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
Review Comment:
I got it, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "miland-db (via GitHub)" <gi...@apache.org>.
miland-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1840068995
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala:
##########
@@ -142,17 +142,18 @@ class AstBuilder extends DataTypeAstBuilder
}
override def visitSingleCompoundStatement(ctx: SingleCompoundStatementContext): CompoundBody = {
- visit(ctx.beginEndCompoundBlock()).asInstanceOf[CompoundBody]
+ val seenLabels = Set[String]()
Review Comment:
@cloud-fan done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "davidm-db (via GitHub)" <gi...@apache.org>.
davidm-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1834105742
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,70 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
+ private val seenLabels: mutable.Set[String] = mutable.Set.empty
+
+ def init(): Unit = {
+ seenLabels.clear()
+ }
+
+ private def checkLabels(
+ beginLabelCtx: Option[BeginLabelContext],
+ endLabelCtx: Option[EndLabelContext]) : Unit = {
+ (beginLabelCtx, endLabelCtx) match {
+ case (Some(bl: BeginLabelContext), Some(el: EndLabelContext))
+ if bl.multipartIdentifier().getText.nonEmpty &&
Review Comment:
I think you are right, but also doesn't hurt to have this check in case someone changes grammar for `multipartIdentifier` in the future for whatever reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "miland-db (via GitHub)" <gi...@apache.org>.
miland-db commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1834376456
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,70 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
+ private val seenLabels: mutable.Set[String] = mutable.Set.empty
+
+ def init(): Unit = {
+ seenLabels.clear()
+ }
+
+ private def checkLabels(
+ beginLabelCtx: Option[BeginLabelContext],
+ endLabelCtx: Option[EndLabelContext]) : Unit = {
+ (beginLabelCtx, endLabelCtx) match {
+ case (Some(bl: BeginLabelContext), Some(el: EndLabelContext))
+ if bl.multipartIdentifier().getText.nonEmpty &&
Review Comment:
@davidm-db can you please confirm this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913] Add check for unique label names in nested labeled scopes [spark]
Posted by "miland-db (via GitHub)" <gi...@apache.org>.
miland-db commented on PR #48795:
URL: https://github.com/apache/spark/pull/48795#issuecomment-2462700789
Adding: @cloud-fan @srielau @davidm-db @dejankrak-db @dusantism-db
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1833275615
##########
common/utils/src/main/resources/error/error-conditions.json:
##########
@@ -1198,6 +1198,12 @@
],
"sqlState" : "23505"
},
+ "DUPLICATE_LABEL" : {
+ "message" : [
+ "Label <label> already exists."
Review Comment:
```suggestion
"The label <label> already exists. Choose another name or rename the existing label."
```
##########
common/utils/src/main/resources/error/error-conditions.json:
##########
@@ -1198,6 +1198,12 @@
],
"sqlState" : "23505"
},
+ "DUPLICATE_LABEL" : {
Review Comment:
There are a few `_EXISTS` conditions already (`FIELD_ALREADY_EXISTS`, `INDEX_ALREADY_EXISTS`, `LOCATION_ALREADY_EXISTS`, `PATH_ALREADY_EXISTS`) , and let's follow naming convention in `error-conditions.json`, and also make it consisitent with the error text:
```suggestion
"LABEL_ALREADY_EXISTS" : {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1833522692
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala:
##########
@@ -134,3 +139,87 @@ object ParserUtils extends SparkParserUtils {
sb.toString()
}
}
+
+object LabelUtils {
Review Comment:
is this class threadsafe? Users may invoke the parser in multiple threads. At least we should put states in ThreadLocal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49913][SQL] Add check for unique label names in nested labeled scopes [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #48795:
URL: https://github.com/apache/spark/pull/48795#discussion_r1839185018
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala:
##########
@@ -142,17 +142,18 @@ class AstBuilder extends DataTypeAstBuilder
}
override def visitSingleCompoundStatement(ctx: SingleCompoundStatementContext): CompoundBody = {
- visit(ctx.beginEndCompoundBlock()).asInstanceOf[CompoundBody]
+ val seenLabels = Set[String]()
Review Comment:
Yes this is the implementation I was expecting. Can we do a bit more encapsulation? e.g.
```
val labelCtx = new ScritpingLabelContext()
visitBeginEndCompoundBlockImpl...
...
class ScritpingLabelContext {
private val seenLabels = Set[String]()
def newLabel(label: String)...
...
}
``
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org