[GitHub] [spark] beliefer commented on a diff in pull request #41534: [SPARK-43937][CONNECT][PYTHON] Add ifnull,isnotnull,equal_null,nullif,nvl,nvl2 to Scala and Python

["beliefer (via GitHub)" <gi...@apache.org>]

beliefer commented on code in PR #41534:
URL: https://github.com/apache/spark/pull/41534#discussion_r1225072464


##########
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/functions.scala:
##########
@@ -5883,4 +5883,65 @@ object functions {
   }
   // scalastyle:off line.size.limit
 
+  //////////////////////////////////////////////////////////////////////////////////////////////
+  // Conditional Functions
+  //////////////////////////////////////////////////////////////////////////////////////////////
+
+  /**
+   * If `predicate` evaluates to true, then returns `trueValue`; otherwise returns `falseValue`.
+   *
+   * @group conditional_funcs
+   * @since 3.5.0
+   */
+  def `if`(predicate: Column, trueValue: Column, falseValue: Column): Column =
+    Column.fn("if", predicate, trueValue, falseValue)
+
+  /**
+   * Returns `col2` if `col1` is null, or `col1` otherwise.
+   *
+   * @group conditional_funcs
+   * @since 3.5.0
+   */
+  def ifnull(col1: Column, col2: Column): Column = Column.fn("ifnull", col1, col2)
+
+  /**
+   * Returns true if `col` is not null, or false otherwise.
+   *
+   * @group conditional_funcs
+   * @since 3.5.0
+   */
+  def isnotnull(col: Column): Column = Column.fn("isnotnull", col)
+
+  /**
+   * Returns same result as the EQUAL(=) operator for non-null operands, but returns true if both
+   * are null, false if one of the them is null.
+   *
+   * @group conditional_funcs
+   * @since 3.5.0
+   */
+  def equal_null(col1: Column, col2: Column): Column = Column.fn("equal_null", col1, col2)

Review Comment:
   I don't know the reason put the `EqualNull` into the group `misc_funcs`. It seems EqualNull should putted into group `predicate_funcs`. cc @cloud-fan @zhengruifeng 



##########
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/functions.scala:
##########
@@ -5883,4 +5883,65 @@ object functions {
   }
   // scalastyle:off line.size.limit
 
+  //////////////////////////////////////////////////////////////////////////////////////////////
+  // Conditional Functions
+  //////////////////////////////////////////////////////////////////////////////////////////////
+
+  /**
+   * If `predicate` evaluates to true, then returns `trueValue`; otherwise returns `falseValue`.
+   *
+   * @group conditional_funcs
+   * @since 3.5.0
+   */
+  def `if`(predicate: Column, trueValue: Column, falseValue: Column): Column =
+    Column.fn("if", predicate, trueValue, falseValue)
+
+  /**
+   * Returns `col2` if `col1` is null, or `col1` otherwise.
+   *
+   * @group conditional_funcs
+   * @since 3.5.0
+   */
+  def ifnull(col1: Column, col2: Column): Column = Column.fn("ifnull", col1, col2)
+
+  /**
+   * Returns true if `col` is not null, or false otherwise.
+   *
+   * @group conditional_funcs
+   * @since 3.5.0
+   */
+  def isnotnull(col: Column): Column = Column.fn("isnotnull", col)

Review Comment:
   `@group predicate_funcs`



##########
python/pyspark/sql/functions.py:
##########
@@ -11157,6 +11157,142 @@ def udf(
         return _create_py_udf(f=f, returnType=returnType, useArrow=useArrow)
 
 
+# ---------------  Conditional functions ------------------------
+

Review Comment:
   I encountered the same like problem. I want add the `any` API to Pyspark, but it is a Python keywords too.
   Personally, I use the name `_any` instead. How about `_if` ? cc @HyukjinKwon @zhengruifeng 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org