Re: Export BLAS module on Spark MLlib

[DB Tsai <db...@dbtsai.com>]

I used reflection initially, but I found it's very slow especially in
a tight loop. Maybe caching the reflection can help which I never try.

Sincerely,

DB Tsai
----------------------------------------------------------
Web: https://www.dbtsai.com
PGP Key ID: 0xAF08DF8D


On Mon, Nov 30, 2015 at 2:15 PM, Burak Yavuz <br...@gmail.com> wrote:
> Or you could also use reflection like in this Spark Package:
> https://github.com/brkyvz/lazy-linalg/blob/master/src/main/scala/com/brkyvz/spark/linalg/BLASUtils.scala
>
> Best,
> Burak
>
> On Mon, Nov 30, 2015 at 12:48 PM, DB Tsai <db...@dbtsai.com> wrote:
>>
>> The workaround is have your code in the same package, or write some
>> utility wrapper in the same package so you can use them in your code.
>> Mostly we implement those BLAS for our own need, and we don't have
>> general use-case in mind. As a result, if we open them up prematurely,
>> it will add our api maintenance cost. Once it's getting mature, and
>> people are asking for them, we will gradually make them public.
>>
>> Thanks.
>>
>> Sincerely,
>>
>> DB Tsai
>> ----------------------------------------------------------
>> Web: https://www.dbtsai.com
>> PGP Key ID: 0xAF08DF8D
>>
>>
>> On Sat, Nov 28, 2015 at 5:20 AM, Sasaki Kai <le...@me.com> wrote:
>> > Hello
>> >
>> > I'm developing a Spark package that manipulates Vector and Matrix for
>> > machine learning.
>> > This package uses mllib.linalg.Vector and mllib.linalg.Matrix in order
>> > to
>> > achieve compatible interface to mllib itself. But mllib.linalg.BLAS
>> > module
>> > is private inside spark package. We cannot use BLAS from spark package.
>> > Due to this, there is no way to manipulate mllib.linalg.{Vector, Matrix}
>> > from spark package side.
>> >
>> > Is there any reason why BLAS module is not set public?
>> > If we cannot use BLAS, what is the reasonable option to manipulate
>> > Vector
>> > and Matrix from spark package?
>> >
>> > Regards
>> > Kai Sasaki(@Lewuathe)
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> For additional commands, e-mail: dev-help@spark.apache.org
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org