You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/11/09 05:18:08 UTC

[GitHub] [spark] linhongliu-db opened a new pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

linhongliu-db opened a new pull request #34528:
URL: https://github.com/apache/spark/pull/34528


   ### What changes were proposed in this pull request?
   This PR fixes an issue that the `injectedFunctions` in `SparkSessionExtensions` can't be
   resolved correctly if it's referred by a temporary view. This is because the injected functions
   are not `UserDefinedExpression`. So they will be treated as temporary functions but couldn't
   be captured as temporary functions during view creation. As a result, the function resolution
   will fail if it's reffered by a temp view.
   
   This PR adds a new concept `customBuiltinFunction`, so that the injected functions will be
   treated as builtin functions intead of tempoary ones. With with way, it's not needed to be
   captured by temp view anymore.
   
   ### Why are the changes needed?
   but fix
   
   
   ### Does this PR introduce _any_ user-facing change?
   After this PR, the `SessionCatalog.isTemporaryFunction` will return false if the function
   is a `injectedFunctions` in `SparkSessionExtensions`
   
   
   ### How was this patch tested?
   pending on test.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-965154674


   **[Test build #145048 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145048/testReport)** for PR 34528 at commit [`5547056`](https://github.com/apache/spark/commit/554705625594139de7ac8578bd2d776455ea9c17).
    * This patch **fails from timeout after a configured wait of `500m`**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-963842638


   **[Test build #145019 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145019/testReport)** for PR 34528 at commit [`889cb25`](https://github.com/apache/spark/commit/889cb25d5a33ec3c49681dea80b772fa2f0dfb91).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964873301


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49519/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964887729


   Kubernetes integration test unable to build dist.
   
   exiting with code: 1
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49521/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-965216986


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145048/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-965345144


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145050/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964815343


   **[Test build #145050 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145050/testReport)** for PR 34528 at commit [`48ffd81`](https://github.com/apache/spark/commit/48ffd8145621c6d5f5634d8978a4dd4b5841a052).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-963826534


   **[Test build #145017 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145017/testReport)** for PR 34528 at commit [`6221140`](https://github.com/apache/spark/commit/6221140f80baa156ed3f5d68cf21c4ee61040849).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964873301


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49519/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964887749






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-963888818


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49491/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-963994703


   **[Test build #145017 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145017/testReport)** for PR 34528 at commit [`6221140`](https://github.com/apache/spark/commit/6221140f80baa156ed3f5d68cf21c4ee61040849).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] linhongliu-db closed pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
linhongliu-db closed pull request #34528:
URL: https://github.com/apache/spark/pull/34528


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964815343


   **[Test build #145050 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145050/testReport)** for PR 34528 at commit [`48ffd81`](https://github.com/apache/spark/commit/48ffd8145621c6d5f5634d8978a4dd4b5841a052).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-963842638


   **[Test build #145019 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145019/testReport)** for PR 34528 at commit [`889cb25`](https://github.com/apache/spark/commit/889cb25d5a33ec3c49681dea80b772fa2f0dfb91).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] jerqi commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
jerqi commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964153404


   > but fix
   > 
   typo? but fix -> bug fix?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] linhongliu-db commented on a change in pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #34528:
URL: https://github.com/apache/spark/pull/34528#discussion_r745298702



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
##########
@@ -205,6 +222,22 @@ trait SimpleFunctionRegistryBase[T] extends FunctionRegistryBase[T] with Logging
     internalRegisterFunction(normalizedName, info, builder)
   }
 
+  override def registerCustomBuiltinFunction(
+      name: FunctionIdentifier,
+      info: ExpressionInfo,
+      builder: FunctionBuilder): Unit = {
+    // We didn't do any check when replacing a function even if it's a builtin one.
+    // So no need to do any extra check here.
+    if (name.database.isEmpty) {

Review comment:
       we actually shouldn't allow function with a database here. Not sure if it's necessary to add a check here because it may break existing use cases.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-963899138


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49491/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964795614


   **[Test build #145048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145048/testReport)** for PR 34528 at commit [`5547056`](https://github.com/apache/spark/commit/554705625594139de7ac8578bd2d776455ea9c17).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-965264094


   **[Test build #145050 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145050/testReport)** for PR 34528 at commit [`48ffd81`](https://github.com/apache/spark/commit/48ffd8145621c6d5f5634d8978a4dd4b5841a052).
    * This patch **fails from timeout after a configured wait of `500m`**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964029879


   **[Test build #145019 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145019/testReport)** for PR 34528 at commit [`889cb25`](https://github.com/apache/spark/commit/889cb25d5a33ec3c49681dea80b772fa2f0dfb91).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964039265


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145019/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-965345144


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145050/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] linhongliu-db commented on a change in pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #34528:
URL: https://github.com/apache/spark/pull/34528#discussion_r745302512



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/functions.scala
##########
@@ -171,6 +171,9 @@ case class DropFunctionCommand(
       if (FunctionRegistry.builtin.functionExists(FunctionIdentifier(functionName))) {
         throw QueryCompilationErrors.cannotDropNativeFuncError(functionName)
       }
+      if (catalog.isCustomBuiltinFunction(FunctionIdentifier(functionName))) {
+        throw QueryCompilationErrors.cannotDropCustomBuiltInFuncError(functionName)
+      }

Review comment:
       dropping a custom built-in function is fine. but it's weird if we can drop a built-in function.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-963861170


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49491/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-963996483


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145017/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964857462


   Kubernetes integration test unable to build dist.
   
   exiting with code: 1
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49519/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964795614


   **[Test build #145048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145048/testReport)** for PR 34528 at commit [`5547056`](https://github.com/apache/spark/commit/554705625594139de7ac8578bd2d776455ea9c17).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964887749


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49521/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] linhongliu-db commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-965164483


   close as I made another PR #34546 to fix this


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-963899138


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49491/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-963826534


   **[Test build #145017 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145017/testReport)** for PR 34528 at commit [`6221140`](https://github.com/apache/spark/commit/6221140f80baa156ed3f5d68cf21c4ee61040849).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-963996483


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145017/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #34528: [SPARK-37202][SQL] Treat `injectedFunctions` as custom built-in functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34528:
URL: https://github.com/apache/spark/pull/34528#issuecomment-964039265


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145019/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org