spark git commit: [MINOR] Update docs for functions.scala to make it clear not all the built-in functions are defined there

[li...@apache.org]

Repository: spark
Updated Branches:
  refs/heads/master 34ebcc6b5 -> 6424b146c


[MINOR] Update docs for functions.scala to make it clear not all the built-in functions are defined there

The title summarizes the change.

Author: Reynold Xin <rx...@databricks.com>

Closes #21318 from rxin/functions.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/6424b146
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/6424b146
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/6424b146

Branch: refs/heads/master
Commit: 6424b146c91fdca734a3ec972067e8e1f88e8b9e
Parents: 34ebcc6
Author: Reynold Xin <rx...@databricks.com>
Authored: Fri Jul 27 17:24:55 2018 -0700
Committer: Xiao Li <ga...@gmail.com>
Committed: Fri Jul 27 17:24:55 2018 -0700

----------------------------------------------------------------------
 .../main/scala/org/apache/spark/sql/functions.scala | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/6424b146/sql/core/src/main/scala/org/apache/spark/sql/functions.scala
----------------------------------------------------------------------
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/functions.scala b/sql/core/src/main/scala/org/apache/spark/sql/functions.scala
index bcd0c94..2772958 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/functions.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/functions.scala
@@ -39,7 +39,21 @@ import org.apache.spark.util.Utils
 
 
 /**
- * Functions available for DataFrame operations.
+ * Commonly used functions available for DataFrame operations. Using functions defined here provides
+ * a little bit more compile-time safety to make sure the function exists.
+ *
+ * Spark also includes more built-in functions that are less common and are not defined here.
+ * You can still access them (and all the functions defined here) using the `functions.expr()` API
+ * and calling them through a SQL expression string. You can find the entire list of functions for
+ * the latest version of Spark at https://spark.apache.org/docs/latest/api/sql/index.html.
+ *
+ * As an example, `isnan` is a function that is defined here. You can use `isnan(col("myCol"))`
+ * to invoke the `isnan` function. This way the programming language's compiler ensures `isnan`
+ * exists and is of the proper form. You can also use `expr("isnan(myCol)")` function to invoke the
+ * same function. In this case, Spark itself will ensure `isnan` exists when it analyzes the query.
+ *
+ * `regr_count` is an example of a function that is built-in but not defined here, because it is
+ * less commonly used. To invoke it, use `expr("regr_count(yCol, xCol)")`.
  *
  * @groupname udf_funcs UDF functions
  * @groupname agg_funcs Aggregate functions


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org