You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/10/03 15:44:07 UTC
[GitHub] [spark] rwpenney opened a new pull request #29938: Expose inverse hyperbolic trig functions through sql.functions API
rwpenney opened a new pull request #29938:
URL: https://github.com/apache/spark/pull/29938
This patch is a small extension to change-request SPARK-28133, which added inverse hyperbolic functions to the SQL interpreter, but did not include those methods within the Scala `sql.functions._` API. This patch makes `acosh`, `asinh` and `atanh` functions available through the Scala API.
Unit-tests have been added to `sql/core/src/test/scala/org/apache/spark/sql/MathFunctionsSuite.scala`. Manual testing has been done via `spark-shell`, using the following recipe:
```
val df = spark.range(0, 11)
.toDF("x")
.withColumn("x", ($"x" - 5) / 2.0)
val hyps = df.withColumn("tanh", tanh($"x"))
.withColumn("sinh", sinh($"x"))
.withColumn("cosh", cosh($"x"))
val invhyps = hyps.withColumn("atanh", atanh($"tanh"))
.withColumn("asinh", asinh($"sinh"))
.withColumn("acosh", acosh($"cosh"))
invhyps.show
```
which produces the following output:
```
+----+--------------------+-------------------+------------------+-------------------+-------------------+------------------+
| x| tanh| sinh| cosh| atanh| asinh| acosh|
+----+--------------------+-------------------+------------------+-------------------+-------------------+------------------+
|-2.5| -0.9866142981514303|-6.0502044810397875| 6.132289479663686| -2.500000000000001|-2.4999999999999956| 2.5|
|-2.0| -0.9640275800758169| -3.626860407847019|3.7621956910836314|-2.0000000000000004|-1.9999999999999991| 2.0|
|-1.5| -0.9051482536448664|-2.1292794550948173| 2.352409615243247|-1.4999999999999998|-1.4999999999999998| 1.5|
|-1.0| -0.7615941559557649|-1.1752011936438014| 1.543080634815244| -1.0| -1.0| 1.0|
|-0.5|-0.46211715726000974|-0.5210953054937474|1.1276259652063807| -0.5|-0.5000000000000002|0.4999999999999998|
| 0.0| 0.0| 0.0| 1.0| 0.0| 0.0| 0.0|
| 0.5| 0.46211715726000974| 0.5210953054937474|1.1276259652063807| 0.5| 0.5|0.4999999999999998|
| 1.0| 0.7615941559557649| 1.1752011936438014| 1.543080634815244| 1.0| 1.0| 1.0|
| 1.5| 0.9051482536448664| 2.1292794550948173| 2.352409615243247| 1.4999999999999998| 1.5| 1.5|
| 2.0| 0.9640275800758169| 3.626860407847019|3.7621956910836314| 2.0000000000000004| 2.0| 2.0|
| 2.5| 0.9866142981514303| 6.0502044810397875| 6.132289479663686| 2.500000000000001| 2.5| 2.5|
+----+--------------------+-------------------+------------------+-------------------+-------------------+------------------+
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703766640
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
zero323 commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703821605
> Still for consistency I would not mind adding them, to make languages consistent.
I agree with that, but I wonder if consistency requires `(String) => ...` variants.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703656833
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703766640
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703976139
Okay, I am fine with this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29938: Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703123437
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-708417235
Merged to master
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703656808
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34021/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703645801
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34021/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen closed pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
srowen closed pull request #29938:
URL: https://github.com/apache/spark/pull/29938
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703594669
I imagine these are quite rarely used - more in engineering than anything I can think of in data science. Still for consistency I would not mind adding them, to make languages consistent.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on pull request #29938: {SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
zero323 commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703132270
Sure. I asked there why we provide wrappers around certain functions and not other and @HyukjinKwon pointed out this:
https://github.com/apache/spark/blob/5af62a2ec74356ce1a97c1371321b3424b674289/sql/core/src/main/scala/org/apache/spark/sql/functions.scala#L45-L56
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703123620
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29938: {SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703123437
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] rwpenney commented on a change in pull request #29938: {SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
rwpenney commented on a change in pull request #29938:
URL: https://github.com/apache/spark/pull/29938#discussion_r499167700
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1427,6 +1427,20 @@ object functions {
*/
def acos(columnName: String): Column = acos(Column(columnName))
+ /**
+ * @return inverse hyperbolic cosine of `e`
+ *
+ * @group math_funcs
+ */
Review comment:
I wasn't sure whether this is something I should put in during the pull-request, or whether it gets added at a later stage.
Shall I presume that this is destined for Spark-3.1, or maybe 3.0.2?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
HyukjinKwon edited a comment on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703370465
There have been some discussions about which functions to add. Basically some expressions exist in SparkSQL core just for the sake of other DBMS compatibility. In other language APIs, some expressions make less sense.
It's better to make simplify the call and when to add so it ended up with writing as so at https://github.com/apache/spark/pull/29938#issuecomment-703132270. Let's avoid adding functions there just for the sake of matching.
How often are they used? I don't know enough about that. cc @WeichenXu123 or @srowen. Are they used commonly in data science?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #29938: {SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
zero323 commented on a change in pull request #29938:
URL: https://github.com/apache/spark/pull/29938#discussion_r499167847
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1427,6 +1427,20 @@ object functions {
*/
def acos(columnName: String): Column = acos(Column(columnName))
+ /**
+ * @return inverse hyperbolic cosine of `e`
+ *
+ * @group math_funcs
+ */
Review comment:
It would be 3.1, as new functions are not added in maintenance releases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on pull request #29938: Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
zero323 commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703124983
This might deserve a separate JIRA ticket. Also could you might want to add a ticket id and `[SQL]` tag to the title i.e. `[SPARK-xxxx][SQL]`.
I also suspect that omitting this in language API might be intentional, as pointed out [here](https://github.com/apache/spark/pull/28593#issuecomment-695861989).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29938: Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703123620
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] rwpenney commented on pull request #29938: {SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
rwpenney commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703129571
Thanks @zero323 - I've added tags to the title, as suggested.
I'm sorry, I'm not sure how your link to https://github.com/apache/spark/pull/28593 (which seems to be about timestamp processing) might explain why inverse hyperbolics weren't originally uncluded in `sql.functions._`. Could you clarify, thanks?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 commented on a change in pull request #29938: {SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
zero323 commented on a change in pull request #29938:
URL: https://github.com/apache/spark/pull/29938#discussion_r499166143
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1427,6 +1427,20 @@ object functions {
*/
def acos(columnName: String): Column = acos(Column(columnName))
+ /**
+ * @return inverse hyperbolic cosine of `e`
+ *
+ * @group math_funcs
+ */
Review comment:
We should probably have `@since` annotation here and for the remaining ones.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703370465
There have been some discussions about which functions to add. Basically some expressions exist in SparkSQL core just for the sake of other DBMS compatibility. In other language APIs, some expressions make less sense.
It's better to make simplify the call and when to add so it ended up with writing as so at https://github.com/apache/spark/pull/29938#issuecomment-703132270. Let's avoid adding functions there just for the sake of matching.
How often are they used? I don't know enough about that. cc @WeichenXu123 or @srowen. Are they used commonly in daily data science?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703765580
**[Test build #129414 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129414/testReport)** for PR 29938 at commit [`7aed453`](https://github.com/apache/spark/commit/7aed453983b6453940afea7b2c7f6535d69b1ffc).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703618357
ok to test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703619410
**[Test build #129414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129414/testReport)** for PR 29938 at commit [`7aed453`](https://github.com/apache/spark/commit/7aed453983b6453940afea7b2c7f6535d69b1ffc).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] rwpenney commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
rwpenney commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703730788
As per my previous comments - I'm not suggesting that these inverse-hypoerboic functions will be widely used, but if `sql.functions` makes ordinary trig-functions available with their inverses, it ought to do the same for hyperbolic trig-functions. If there are concerns about `sql.functions` becoming rather bloated, perhaps at some point we may need a `sql.math.functions` or `sql.functions.math`?
I have, personally, found use-cases where an inverse-sinh is quite handy in data-science applications, for compressing the dynamic range of signed quantities, in the same way `log` is often used for unsigned quantities. Hyperbolic-tangents appear sufficiently frequently in logistic regression models that having an inverse-tanh is also likely to be helpful.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zero323 edited a comment on pull request #29938: {SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
zero323 edited a comment on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703132270
Sure. I asked there why we provide wrappers around certain functions and not other and @HyukjinKwon pointed out this:
https://github.com/apache/spark/blob/5af62a2ec74356ce1a97c1371321b3424b674289/sql/core/src/main/scala/org/apache/spark/sql/functions.scala#L45-L56
I don't insist that it necessarily applies here (though arguably, average user doesn't need hyperbolic trig functions on daily basis), just giving some context :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] rwpenney commented on pull request #29938: {SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
rwpenney commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703135598
Thanks for the context.
I agree that hyperbolic trig functions aren't everyone's cup of tea, but I think it would be better to have symmetry: cos & acos, sin & asin, tan & atan are available but we currently only have cosh, sinh & tanh *without* their corresponding inverse functions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703656833
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29938: [SPARK-33061][SQL] Expose inverse hyperbolic trig functions through sql.functions API
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29938:
URL: https://github.com/apache/spark/pull/29938#issuecomment-703619410
**[Test build #129414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129414/testReport)** for PR 29938 at commit [`7aed453`](https://github.com/apache/spark/commit/7aed453983b6453940afea7b2c7f6535d69b1ffc).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org