[spark] branch branch-3.4 updated: [SPARK-42631][CONNECT][FOLLOW-UP] Expose Column.expr to extensions

[hv...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

hvanhovell pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.4 by this push:
     new ce36692eeee [SPARK-42631][CONNECT][FOLLOW-UP] Expose Column.expr to extensions
ce36692eeee is described below

commit ce36692eeee7c2e459cc0a109112b29572cea52b
Author: Tom van Bussel <to...@databricks.com>
AuthorDate: Wed Mar 29 15:28:09 2023 -0400

    [SPARK-42631][CONNECT][FOLLOW-UP] Expose Column.expr to extensions
    
    ### What changes were proposed in this pull request?
    This PR is a follow-up to https://github.com/apache/spark/pull/40234, which makes it possible for extensions to create custom `Dataset`s and `Column`s. It exposes `Dataset.plan`, but unfortunately it does not expose `Column.expr`. This means that extensions cannot build custom `Column`s that provide a user provider `Column` as input.
    
    ### Why are the changes needed?
    See above.
    
    ### Does this PR introduce _any_ user-facing change?
    No. This only adds a change for a Developer API.
    
    ### How was this patch tested?
    Existing tests to make sure nothing breaks.
    
    Closes #40590 from tomvanbussel/SPARK-42631.
    
    Authored-by: Tom van Bussel <to...@databricks.com>
    Signed-off-by: Herman van Hovell <he...@databricks.com>
    (cherry picked from commit c3716c4ec68c2dea07e8cd896d79bd7175517a31)
    Signed-off-by: Herman van Hovell <he...@databricks.com>
---
 .../connect/client/jvm/src/main/scala/org/apache/spark/sql/Column.scala | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/Column.scala b/connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/Column.scala
index 4212747f57a..6a660a7482e 100644
--- a/connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/Column.scala
+++ b/connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/Column.scala
@@ -51,7 +51,7 @@ import org.apache.spark.sql.types._
  *
  * @since 3.4.0
  */
-class Column private[sql] (private[sql] val expr: proto.Expression) extends Logging {
+class Column private[sql] (@DeveloperApi val expr: proto.Expression) extends Logging {
 
   private[sql] def this(name: String, planId: Option[Long]) =
     this(Column.nameToExpression(name, planId))


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org