Spark-packages

[Stephen Boesch <ja...@gmail.com>]

What is the state of the spark-packages project(s) ?  When running a query
for machine learning algorithms the results are not encouraging.


https://spark-packages.org/?q=tags%3A%22Machine%20Learning%22

There are 62 packages. Only a few have actual releases - and even less with
dates in the past twelve months.

There are several from DataBricks among the chosen few that have recent
releases.

Here is one that actually seems to be in reasonable shape: the DB port of
Stanford coreNLP.

https://github.com/databricks/spark-corenlp

But .. one or two solid packages .. ?

It seems the  spark-packages approach seems not to have picked up  steam..
  Are there other places suggested to look for algorithms not included in
mllib itself ?

Re: Spark-packages

[Holden Karau <ho...@pigscanfly.ca>]

I think there is a bit more life in the connector side of things for
spark-packages, but there seem to be some outstanding issues with Python
support that are waiting on progress (see
https://github.com/databricks/sbt-spark-package/issues/26 ).
It's possible others are just distributing on maven central instead of
putting in the effort to publish to spark-packages, but I don't know if any
comprehensive index besides spark-packages currently.

On Sunday, November 6, 2016, Stephen Boesch <ja...@gmail.com> wrote:

>
> What is the state of the spark-packages project(s) ?  When running a query
> for machine learning algorithms the results are not encouraging.
>
>
> https://spark-packages.org/?q=tags%3A%22Machine%20Learning%22
>
> There are 62 packages. Only a few have actual releases - and even less
> with dates in the past twelve months.
>
> There are several from DataBricks among the chosen few that have recent
> releases.
>
> Here is one that actually seems to be in reasonable shape: the DB port of
> Stanford coreNLP.
>
> https://github.com/databricks/spark-corenlp
>
> But .. one or two solid packages .. ?
>
> It seems the  spark-packages approach seems not to have picked up  steam..
>   Are there other places suggested to look for algorithms not included in
> mllib itself ?
>
>
>
>

-- 
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau