You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by umargeek <um...@gmail.com> on 2018/05/23 05:04:36 UTC

Alternative for numpy in Spark Mlib

Hi Folks,

I am planning to rewrite one of my python module written for entropy
calculation using numpy into Spark Mlib so that it can be processed in
distributed manner.

Can you please advise on the possibilities of the same approach or any
alternatives.

Thanks,
Umar



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org

Re: Alternative for numpy in Spark Mlib

Posted by "Suzen, Mehmet" <su...@acm.org>.

You can  use Breeze, which is part of spark distribution:
https://github.com/scalanlp/breeze/wiki/Breeze-Linear-Algebra

Check out the modules under  import breeze._

On 23 May 2018 at 07:04, umargeek <um...@gmail.com> wrote:
> Hi Folks,
>
> I am planning to rewrite one of my python module written for entropy
> calculation using numpy into Spark Mlib so that it can be processed in
> distributed manner.
>
> Can you please advise on the possibilities of the same approach or any
> alternatives.
>
> Thanks,
> Umar
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org