You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by rxin <gi...@git.apache.org> on 2018/05/11 23:26:58 UTC

[GitHub] spark pull request #21309: [SPARK-23907] Removes regr_* functions in functio...

GitHub user rxin opened a pull request:

    https://github.com/apache/spark/pull/21309

    [SPARK-23907] Removes regr_* functions in functions.scala

    ## What changes were proposed in this pull request?
    This patch removes the various regr_* functions in functions.scala. They are so uncommon that I don't think they deserve real estate in functions.scala. We can consider adding them later if more users need them.
    
    ## How was this patch tested?
    Removed the associated test case as well.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/rxin/spark SPARK-23907

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/21309.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #21309
    
----
commit ce2c305169d90c4d7803338d85d2d4c92a8e1d3c
Author: Reynold Xin <rx...@...>
Date:   2018-05-11T23:24:15Z

    [SPARK-23907] Removes regr_ functions in functions.scala

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

Posted by rxin <gi...@git.apache.org>.

Github user rxin commented on the issue:

    https://github.com/apache/spark/pull/21309
  
    cc @gatorsmile @mgaido91 


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/21309
  
    Yup, then why not just deprecate other functions in other APIs for 3.0.0, and promote the usage of expr?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

Posted by rxin <gi...@git.apache.org>.

Github user rxin commented on the issue:

    https://github.com/apache/spark/pull/21309
  
    Better compile time error. Plus a lot of people are already using these.
    
    On Fri, May 11, 2018 at 7:35 PM Hyukjin Kwon <no...@github.com>
    wrote:
    
    > Yup, then why not just deprecate other functions in other APIs for 3.0.0,
    > and promote the usage of expr?
    >
    > —
    > You are receiving this because you were mentioned.
    > Reply to this email directly, view it on GitHub
    > <https://github.com/apache/spark/pull/21309#issuecomment-388524092>, or mute
    > the thread
    > <https://github.com/notifications/unsubscribe-auth/AATvPNbOEidl-IwkRFVW0kVpVjEPKoOgks5txkpdgaJpZM4T8LX4>
    > .
    >



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21309
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3164/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/21309
  
    I am asking this to use the same judgement for when to add it.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/21309
  
    Merged to master.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

Posted by rxin <gi...@git.apache.org>.

Github user rxin commented on the issue:

    https://github.com/apache/spark/pull/21309
  
    Btw it’s been always the case that the less commonly used functions are not
    part of this file. There is just a lot of overhead to maintaining all of
    them.
    
    I’m not even sure if the regr_* expressions should be added in the first
    place.
    
    On Fri, May 11, 2018 at 7:20 PM Hyukjin Kwon <no...@github.com>
    wrote:
    
    > @rxin <https://github.com/rxin>, how about splitting up this file by the
    > group or something, or deprecating all the functions that can be called via
    > expr for 3.0.0? To me, it looked a bit odd when some functions exist and
    > some did not. It was an actual use case and I had to check which function
    > exists or not every time.
    >
    > —
    > You are receiving this because you were mentioned.
    > Reply to this email directly, view it on GitHub
    > <https://github.com/apache/spark/pull/21309#issuecomment-388523458>, or mute
    > the thread
    > <https://github.com/notifications/unsubscribe-auth/AATvPKznGyNtcF57sol08PGgzbhth-4_ks5txkcKgaJpZM4T8LX4>
    > .
    >



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

Posted by mgaido91 <gi...@git.apache.org>.

Github user mgaido91 commented on the issue:

    https://github.com/apache/spark/pull/21309
  
    @rxin I also always had some problems in understanding which functions are available and which not. I think it is quite confusing for a user. I am wondering: what do you think if we automatically generate this file in order to make all them available to users? In this way probably we don't have the huge maintenance effort exposing all of them to users.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21309
  
    **[Test build #90537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90537/testReport)** for PR 21309 at commit [`ce2c305`](https://github.com/apache/spark/commit/ce2c305169d90c4d7803338d85d2d4c92a8e1d3c).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/21309
  
    Just had a short talk with Reynold. LGTM too. thanks for bearing with me.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21309
  
    **[Test build #90537 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90537/testReport)** for PR 21309 at commit [`ce2c305`](https://github.com/apache/spark/commit/ce2c305169d90c4d7803338d85d2d4c92a8e1d3c).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21309: [SPARK-23907] Removes regr_* functions in functions.scal...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/21309
  
    @rxin, how about splitting up this file by the group or something, or deprecating all the functions that can be called via expr for 3.0.0? To me, it looked a bit odd when some functions exist and some did not. It was an actual use case and I had to check which function exists or not every time.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org