[GitHub] spark pull request: SPARK-2372 [MLLIB] Grouped Optimization/Learni...

[kellrott <gi...@git.apache.org>]

Github user kellrott commented on the pull request:

    https://github.com/apache/spark/pull/1292#issuecomment-76562027
  
    This was originally written as a response to Spark didn't scale to having
    multiple jobs running at the same time (
    http://mail-archives.apache.org/mod_mbox/incubator-spark-user/201407.mbox/%3CCAKXMip2he3MoCuU09hf5dXrEQ7kgc3RvNUJtSU54w82baWnj6A@mail.gmail.com%3E
    ). One of the main reasons it wasn't written as an external package was
    that too much of the mllib code was marked as private and couldn't be used.
    
    Kyle
    
    
    On Fri, Feb 27, 2015 at 1:18 AM, Xiangrui Meng <no...@github.com>
    wrote:
    
    > @nchammas <https://github.com/nchammas> @kellrott
    > <https://github.com/kellrott> Sorry that I haven't got a time to review
    > this PR. The main issue is that this PR is not small. Though there are no
    > break changes, the added code is very similar to the ungrouped and hence we
    > have to maintain both in the future. I'm wondering how this compared to
    > letting Spark launching multiple jobs at the same time but using existing
    > code.
    >
    > Given the current backlog, I'd suggest maintaining it as a Spark package (
    > http://spark-packages.org/). So users can grab them easily and try.
    >
    > —
    > Reply to this email directly or view it on GitHub
    > <https://github.com/apache/spark/pull/1292#issuecomment-76360812>.
    >



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org