You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/05/03 16:27:06 UTC

[GitHub] [spark] neilagupta opened a new pull request, #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose due to nodePatterns being final.

neilagupta opened a new pull request, #36441:
URL: https://github.com/apache/spark/pull/36441

   <!--
   Thanks for sending a pull request!  Here are some tips for you:
     1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
     2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
     3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
     4. Be sure to keep the PR description updated to reflect all changes.
     5. Please write your PR title to summarize what this PR proposes.
     6. If possible, provide a concise example to reproduce the issue for a faster review.
     7. If you want to add a new configuration, please read the guideline first for naming configurations in
        'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
     8. If you want to add or modify an error type or message, please read the guideline first in
        'core/src/main/resources/error/README.md'.
   -->
   
   When creating classes that extend multiple `Expression` traits, they have an inheritance conflict due to `nodePatterns` being different values. `nodePatterns` is also final, which makes the inheritance conflict unavoidable.
   
   This error comes in after trying to override the field - `value nodePatterns in trait TimeZoneAwareExpression of type Seq[org.apache.spark.sql.catalyst.trees.TreePattern.TreePattern] cannot override final member.
   `
   
   ### What changes were proposed in this pull request?
   <!--
   Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue. 
   If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
     1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
     2. If you fix some SQL features, you can provide some references of other DBMSes.
     3. If there is design documentation, please add the link.
     4. If there is a discussion in the mailing list, please add the link.
   -->
   
   For now kept the traits changed to a minimum - removed `nodePatterns` final classifier from `TimeZoneAwareExpression`, `HigherOrderFunctions`, and `Generator`. But this change can be expanded to other traits.
   
   ### Why are the changes needed?
   <!--
   Please clarify why the changes are needed. For instance,
     1. If you propose a new API, clarify the use case for a new API.
     2. If you fix a bug, you can clarify why it is a bug.
   -->
   
   Enables multiple trait inheritance for certain `Expression` types.
   
   
   ### Does this PR introduce _any_ user-facing change?
   <!--
   Note that it means *any* user-facing change including all aspects such as the documentation fix.
   If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
   If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
   If no, write 'No'.
   -->
   No
   
   
   ### How was this patch tested?
   <!--
   If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
   If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
   If tests were not added, please describe why they were not added and/or why it was difficult to add.
   If benchmark tests were added, please run the benchmarks in GitHub Actions for the consistent environment, and the instructions could accord to: https://spark.apache.org/developer-tools.html#github-workflow-benchmarks.
   -->
   Added a test that creates a dummy class that inherits all three of `TimeZoneAwareExpression`, `HigherOrderFunctions`, and `Generator` - the test is only to see if this can compile now. Since it compiles and runs, this issue is resolved for those three traits.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] github-actions[bot] commented on pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36441:
URL: https://github.com/apache/spark/pull/36441#issuecomment-1449127359

   We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
   If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] hvanhovell commented on pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.

Posted by GitBox <gi...@apache.org>.
hvanhovell commented on PR #36441:
URL: https://github.com/apache/spark/pull/36441#issuecomment-1184865067

   @sigmod wanna take a look?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HuwCampbell commented on a diff in pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.

Posted by GitBox <gi...@apache.org>.
HuwCampbell commented on code in PR #36441:
URL: https://github.com/apache/spark/pull/36441#discussion_r973814236


##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HigherOrderFunctionsSuite.scala:
##########
@@ -839,6 +841,52 @@ class HigherOrderFunctionsSuite extends SparkFunSuite with ExpressionEvalHelper
       Seq(1d, 2d, Double.NaN, null))
   }
 
+  test("SPARK-39081: compatibility of combo of HigherOrderFunctions" +
+    " with other Expression subclasses") {
+    // Dummy example given in JIRA, this is to test a compile time issue only, has no real usage
+    case class MyExploder(
+                           arrays: Expression,    // Array[AnyDataType]
+                           asOfDate: Expression,  // LambdaFunction[AnyDataType -> TimestampType]
+                           extractor: Expression) // TimestampType
+      extends HigherOrderFunction with Generator with TimeZoneAwareExpression {
+
+      override def children: Seq[Expression] = Seq[Expression](arrays, asOfDate)
+
+      override def withNewChildrenInternal(newChildren: IndexedSeq[Expression]): Expression = null
+
+      override def eval(input: InternalRow): TraversableOnce[InternalRow] = null
+
+      override def nodePatternsInternal(): Seq[TreePattern] = Seq()
+
+      override def elementSchema: StructType = null
+
+      override def arguments: Seq[Expression] =
+        Seq(arrays, asOfDate)
+
+      override def argumentTypes: Seq[AbstractDataType] =
+        Seq(ArrayType, TimestampType)
+
+      override def doGenCode(ctx: CodegenContext, exp: ExprCode): ExprCode = null
+
+      override def functions: Seq[Expression] =
+        Seq(extractor)
+
+      override def functionTypes: Seq[TimestampType] =
+        Seq(TimestampType)
+
+      override def bind(f: (Expression,
+        Seq[(DataType, Boolean)]) => LambdaFunction): HigherOrderFunction = null
+
+      override def timeZoneId: Option[String] = None
+
+      override def withTimeZone(timeZoneId: String): TimeZoneAwareExpression = null
+    }
+
+    /* Should not get the error - value nodePatterns
+     ... cannot override final member, or conflict in nodePatterns between two types

Review Comment:
   Yeah I think you're pretty much dead on here, the correctness of the optimiser will be up to the user to enforce; as they will need to ensure that all node patterns are accounted for.
   
   But, and it's a big one, it makes it possible to compose these interfaces, even with the extra challenges... which it's just _not_ possible to do as of Spark 2.2.
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] JoshRosen commented on a diff in pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.

Posted by GitBox <gi...@apache.org>.
JoshRosen commented on code in PR #36441:
URL: https://github.com/apache/spark/pull/36441#discussion_r921495229


##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HigherOrderFunctionsSuite.scala:
##########
@@ -839,6 +841,52 @@ class HigherOrderFunctionsSuite extends SparkFunSuite with ExpressionEvalHelper
       Seq(1d, 2d, Double.NaN, null))
   }
 
+  test("SPARK-39081: compatibility of combo of HigherOrderFunctions" +
+    " with other Expression subclasses") {
+    // Dummy example given in JIRA, this is to test a compile time issue only, has no real usage
+    case class MyExploder(
+                           arrays: Expression,    // Array[AnyDataType]
+                           asOfDate: Expression,  // LambdaFunction[AnyDataType -> TimestampType]
+                           extractor: Expression) // TimestampType
+      extends HigherOrderFunction with Generator with TimeZoneAwareExpression {
+
+      override def children: Seq[Expression] = Seq[Expression](arrays, asOfDate)
+
+      override def withNewChildrenInternal(newChildren: IndexedSeq[Expression]): Expression = null
+
+      override def eval(input: InternalRow): TraversableOnce[InternalRow] = null
+
+      override def nodePatternsInternal(): Seq[TreePattern] = Seq()
+
+      override def elementSchema: StructType = null
+
+      override def arguments: Seq[Expression] =
+        Seq(arrays, asOfDate)
+
+      override def argumentTypes: Seq[AbstractDataType] =
+        Seq(ArrayType, TimestampType)
+
+      override def doGenCode(ctx: CodegenContext, exp: ExprCode): ExprCode = null
+
+      override def functions: Seq[Expression] =
+        Seq(extractor)
+
+      override def functionTypes: Seq[TimestampType] =
+        Seq(TimestampType)
+
+      override def bind(f: (Expression,
+        Seq[(DataType, Boolean)]) => LambdaFunction): HigherOrderFunction = null
+
+      override def timeZoneId: Option[String] = None
+
+      override def withTimeZone(timeZoneId: String): TimeZoneAwareExpression = null
+    }
+
+    /* Should not get the error - value nodePatterns
+     ... cannot override final member, or conflict in nodePatterns between two types

Review Comment:
   What is the final effective value of `nodePatterns` for this new expression?
   
   Ideally it would be the _union_ of its superclasses, but in practice I think it will pick up the value from the last trait mixed in and would only have the `TIME_ZONE_AWARE_EXPRESSION` pattern but not `GENERATOR` or `HIGH_ORDER_FUNCTION`.
   
   I think this will lead to optimizer correctness bugs because it violates an existing invariant: today, all subclasses of the expressions / traits with `final` node patterns are _guaranteed_ to have those patterns set, but that invariant no longer holds after your change. The optimizer uses these node patterns to prune tree traversals and skip optimizer rule invocation in cases where it knows that rules cannot apply because subtrees do not contain the node types used by the rule; see https://issues.apache.org/jira/browse/SPARK-35042 and https://docs.google.com/document/d/1SEUhkbo8X-0cYAJFYFDQhxUnKJBz4lLn3u4xR2qfWqk/edit#heading=h.4cp1yxp4c1fh for more background on this.
   
   For example, if you had a `HigherOrderFunction` subclass where the `HIGH_ORDER_FUNCTION` node pattern wasn't set then the `ResolveLambdaVariables` optimizer rule would end up [skipping that expression via pattern pruning](https://github.com/apache/spark/blob/35d00df9bba7238ad4f409999617fae4d04ddbfd/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/higherOrderFunctions.scala#L52-L58).
   
   (This is primarily a drive-by comment: I haven't done much recent work in the optimizer, so I'm not in the best position to suggest an alternative design or to lead the review of this PR. I just wanted to voice the top-of-my-head concern and explain (what I think is) the rationale for these variables having been final in the first place).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] JoshRosen commented on a diff in pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.

Posted by GitBox <gi...@apache.org>.
JoshRosen commented on code in PR #36441:
URL: https://github.com/apache/spark/pull/36441#discussion_r921495229


##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HigherOrderFunctionsSuite.scala:
##########
@@ -839,6 +841,52 @@ class HigherOrderFunctionsSuite extends SparkFunSuite with ExpressionEvalHelper
       Seq(1d, 2d, Double.NaN, null))
   }
 
+  test("SPARK-39081: compatibility of combo of HigherOrderFunctions" +
+    " with other Expression subclasses") {
+    // Dummy example given in JIRA, this is to test a compile time issue only, has no real usage
+    case class MyExploder(
+                           arrays: Expression,    // Array[AnyDataType]
+                           asOfDate: Expression,  // LambdaFunction[AnyDataType -> TimestampType]
+                           extractor: Expression) // TimestampType
+      extends HigherOrderFunction with Generator with TimeZoneAwareExpression {
+
+      override def children: Seq[Expression] = Seq[Expression](arrays, asOfDate)
+
+      override def withNewChildrenInternal(newChildren: IndexedSeq[Expression]): Expression = null
+
+      override def eval(input: InternalRow): TraversableOnce[InternalRow] = null
+
+      override def nodePatternsInternal(): Seq[TreePattern] = Seq()
+
+      override def elementSchema: StructType = null
+
+      override def arguments: Seq[Expression] =
+        Seq(arrays, asOfDate)
+
+      override def argumentTypes: Seq[AbstractDataType] =
+        Seq(ArrayType, TimestampType)
+
+      override def doGenCode(ctx: CodegenContext, exp: ExprCode): ExprCode = null
+
+      override def functions: Seq[Expression] =
+        Seq(extractor)
+
+      override def functionTypes: Seq[TimestampType] =
+        Seq(TimestampType)
+
+      override def bind(f: (Expression,
+        Seq[(DataType, Boolean)]) => LambdaFunction): HigherOrderFunction = null
+
+      override def timeZoneId: Option[String] = None
+
+      override def withTimeZone(timeZoneId: String): TimeZoneAwareExpression = null
+    }
+
+    /* Should not get the error - value nodePatterns
+     ... cannot override final member, or conflict in nodePatterns between two types

Review Comment:
   What is the final effective value of `nodePatterns` for this new expression?
   
   Ideally it would be the _union_ of its superclasses' `nodePatterns`, but in practice I think it will pick up the value from the last trait mixed in and would only have the `TIME_ZONE_AWARE_EXPRESSION` pattern but not `GENERATOR` or `HIGH_ORDER_FUNCTION`.
   
   I think this will lead to optimizer correctness bugs because it violates an existing invariant: today, all subclasses of the expressions / traits with `final` node patterns are _guaranteed_ to have those patterns set, but that invariant no longer holds after your change. The optimizer uses these node patterns to prune tree traversals and skip optimizer rule invocation in cases where it knows that rules cannot apply because subtrees do not contain the node types used by the rule; see https://issues.apache.org/jira/browse/SPARK-35042 and https://docs.google.com/document/d/1SEUhkbo8X-0cYAJFYFDQhxUnKJBz4lLn3u4xR2qfWqk/edit#heading=h.4cp1yxp4c1fh for more background on this.
   
   For example, if you had a `HigherOrderFunction` subclass where the `HIGH_ORDER_FUNCTION` node pattern wasn't set then the `ResolveLambdaVariables` optimizer rule would end up [skipping that expression via pattern pruning](https://github.com/apache/spark/blob/35d00df9bba7238ad4f409999617fae4d04ddbfd/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/higherOrderFunctions.scala#L52-L58).
   
   (This is primarily a drive-by comment: I haven't done much recent work in the optimizer, so I'm not in the best position to suggest an alternative design or to lead the review of this PR. I just wanted to voice the top-of-my-head concern and explain (what I think is) the rationale for these variables having been final in the first place).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HuwCampbell commented on pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.

Posted by GitBox <gi...@apache.org>.
HuwCampbell commented on PR #36441:
URL: https://github.com/apache/spark/pull/36441#issuecomment-1124519715

   Yes indeed, this would unblock me from upgrading to Spark 3.2 once there's a release.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] neilagupta commented on pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.

Posted by GitBox <gi...@apache.org>.
neilagupta commented on PR #36441:
URL: https://github.com/apache/spark/pull/36441#issuecomment-1129086391

   @AmplabJenkins  any chance I could get someone with write access to review this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HuwCampbell commented on a diff in pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.

Posted by GitBox <gi...@apache.org>.
HuwCampbell commented on code in PR #36441:
URL: https://github.com/apache/spark/pull/36441#discussion_r973814236


##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HigherOrderFunctionsSuite.scala:
##########
@@ -839,6 +841,52 @@ class HigherOrderFunctionsSuite extends SparkFunSuite with ExpressionEvalHelper
       Seq(1d, 2d, Double.NaN, null))
   }
 
+  test("SPARK-39081: compatibility of combo of HigherOrderFunctions" +
+    " with other Expression subclasses") {
+    // Dummy example given in JIRA, this is to test a compile time issue only, has no real usage
+    case class MyExploder(
+                           arrays: Expression,    // Array[AnyDataType]
+                           asOfDate: Expression,  // LambdaFunction[AnyDataType -> TimestampType]
+                           extractor: Expression) // TimestampType
+      extends HigherOrderFunction with Generator with TimeZoneAwareExpression {
+
+      override def children: Seq[Expression] = Seq[Expression](arrays, asOfDate)
+
+      override def withNewChildrenInternal(newChildren: IndexedSeq[Expression]): Expression = null
+
+      override def eval(input: InternalRow): TraversableOnce[InternalRow] = null
+
+      override def nodePatternsInternal(): Seq[TreePattern] = Seq()
+
+      override def elementSchema: StructType = null
+
+      override def arguments: Seq[Expression] =
+        Seq(arrays, asOfDate)
+
+      override def argumentTypes: Seq[AbstractDataType] =
+        Seq(ArrayType, TimestampType)
+
+      override def doGenCode(ctx: CodegenContext, exp: ExprCode): ExprCode = null
+
+      override def functions: Seq[Expression] =
+        Seq(extractor)
+
+      override def functionTypes: Seq[TimestampType] =
+        Seq(TimestampType)
+
+      override def bind(f: (Expression,
+        Seq[(DataType, Boolean)]) => LambdaFunction): HigherOrderFunction = null
+
+      override def timeZoneId: Option[String] = None
+
+      override def withTimeZone(timeZoneId: String): TimeZoneAwareExpression = null
+    }
+
+    /* Should not get the error - value nodePatterns
+     ... cannot override final member, or conflict in nodePatterns between two types

Review Comment:
   Yeah I think you're pretty much dead on here, the correctness of the optimiser will be up to the user to enforce; as they will need to ensure that all node patterns are accounted for.
   
   But, and it's a big one, it makes it possible to compose these interfaces, even with the extra challenges... which it's just _not_ possible to do as of Spark 3.2.
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] neilagupta commented on pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.

Posted by GitBox <gi...@apache.org>.
neilagupta commented on PR #36441:
URL: https://github.com/apache/spark/pull/36441#issuecomment-1124509681

   Hi @HuwCampbell that should be correct, I will update this PR with more modified subclasses of Expression to have their `nodePatterns` fields not final. Would that be in your interest?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HuwCampbell commented on pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.

Posted by GitBox <gi...@apache.org>.
HuwCampbell commented on PR #36441:
URL: https://github.com/apache/spark/pull/36441#issuecomment-1120675993

   This change would be valuable for me. I imagine that if manually overidden, one would have to specify all bit patterns used in order to get all appropriate rules applied.
   
   ```scala
   Seq(TIME_ZONE_AWARE_EXPRESSION, HIGHER_ORDER_FUNCTION, GENERATOR)
   ```
   
   is that correct?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on PR #36441:
URL: https://github.com/apache/spark/pull/36441#issuecomment-1116875710

   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] github-actions[bot] closed pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #36441: [SPARK-39091][SQL] Updating specific SQL Expression traits that don't compose when multiple are extended due to nodePatterns being final.
URL: https://github.com/apache/spark/pull/36441


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org