MLLib : Math on Vector and Matrix

[Thunder Stumpges <th...@gmail.com>]

I am upgrading from Spark 0.9.0 to 1.0 and I had a pretty good amount of
code working with internals of MLLib. One of the big changes was the move
from the old jblas.Matrix to the Vector/Matrix classes included in MLLib.

However I don't see how we're supposed to use them for ANYTHING other than
a container for passing data to the included APIs... how do we do any math
on them? Looking at the internal code, there are quite a number of
private[mllib] declarations including access to the Breeze representations
of the classes.

Was there a good reason this was not exposed? I could see maybe not wanting
to expose the 'toBreeze' function which would tie it to the breeze
implementation, however it would be nice to have the various mathematics
wrapped at least.

Right now I see no way to code any vector/matrix math without moving my
code namespaces into org.apache.spark.mllib or duplicating the code in
'toBreeze' in my own util functions. Not very appealing.

What are others doing?
thanks,
Thunder

Re: MLLib : Math on Vector and Matrix

[Xiangrui Meng <me...@gmail.com>]

Hi Thunder,

Please understand that both MLlib and breeze are in active
development. Before v1.0, we used jblas but in the public APIs we only
exposed Array[Double]. In v1.0, we introduced Vector that supports
both dense and sparse data and switched the backend to
breeze/netlib-java (except ALS). We only used few breeze methods in
our implementation and we benchmarked them one by one. It was hard to
foresee problems caused by including breeze at that time, for example,
https://issues.apache.org/jira/browse/SPARK-1520. Being conservative
in v1.0 was not a bad choice. We should benchmark breeze v0.8.1 for
v1.1 and consider make toBreeze a developer API. For now, if you are
migrating code from v0.9, you can use `Vector.toArray` to get the
value array. Sorry for the inconvenience!

Best,
Xiangrui

On Wed, Jul 2, 2014 at 2:42 PM, Dmitriy Lyubimov <dl...@gmail.com> wrote:
> in my humble opinion Spark should've supported linalg a-la [1] before it
> even started dumping methodologies into mllib.
>
> [1] http://mahout.apache.org/users/sparkbindings/home.html
>
>
> On Wed, Jul 2, 2014 at 2:16 PM, Thunder Stumpges
> <th...@gmail.com> wrote:
>>
>> Thanks. I always hate having to do stuff like this. It seems like they
>> went a bit overboard with all the "private[mllib]" declarations... possibly
>> all in the name of "thou shalt not change your public API". If you don't
>> make your public API usable, we end up having to work around it anyway...
>>
>> Oh well.
>>
>> Thunder
>>
>>
>>
>> On Wed, Jul 2, 2014 at 2:05 PM, Koert Kuipers <ko...@tresata.com> wrote:
>>>
>>> i did the second option: re-implemented .toBreeze as .breeze using pimp
>>> classes
>>>
>>>
>>> On Wed, Jul 2, 2014 at 5:00 PM, Thunder Stumpges
>>> <th...@gmail.com> wrote:
>>>>
>>>> I am upgrading from Spark 0.9.0 to 1.0 and I had a pretty good amount of
>>>> code working with internals of MLLib. One of the big changes was the move
>>>> from the old jblas.Matrix to the Vector/Matrix classes included in MLLib.
>>>>
>>>> However I don't see how we're supposed to use them for ANYTHING other
>>>> than a container for passing data to the included APIs... how do we do any
>>>> math on them? Looking at the internal code, there are quite a number of
>>>> private[mllib] declarations including access to the Breeze representations
>>>> of the classes.
>>>>
>>>> Was there a good reason this was not exposed? I could see maybe not
>>>> wanting to expose the 'toBreeze' function which would tie it to the breeze
>>>> implementation, however it would be nice to have the various mathematics
>>>> wrapped at least.
>>>>
>>>> Right now I see no way to code any vector/matrix math without moving my
>>>> code namespaces into org.apache.spark.mllib or duplicating the code in
>>>> 'toBreeze' in my own util functions. Not very appealing.
>>>>
>>>> What are others doing?
>>>> thanks,
>>>> Thunder
>>>>
>>>
>>
>

Re: MLLib : Math on Vector and Matrix

[Dmitriy Lyubimov <dl...@gmail.com>]

On Wed, Jul 2, 2014 at 11:40 PM, Xiangrui Meng <me...@gmail.com> wrote:

> Hi Dmitriy,
>
> It is sweet to have the bindings, but it is very easy to downgrade the
> performance with them. The BLAS/LAPACK APIs have been there for more
> than 20 years and they are still the top choice for high-performance
> linear algebra.


There's no such limitation there. In fact, LAPACK/jblas is very easy fruit
to have there.

 algebraic optimizer is not about so much about in-core block-on-block
techniques. It is about optimizing/simplification of algebraic expressions,
especially their distributed plans/side of things. Another side of the
story is consistent matrix representation for block-2-block in-memory
computations and passing stuff in and out. R-like look & feel.

It is true that in-core-only computations currently are not deferrably
optimized, nor do they have LAPack back but this is a low hanging fruit
there. the main idea is consistency of algebraic API/DSL, be it distributed
or in-core, and having algebraic optimizer, and pluggable backs (both
in-core backs or distributed engine backs as well).

It is so happened the only in-memory back right now is Mahout's Colt
derivation, but there's fairly little reason not to pick-plug
Breeze/Lapack, or say GPU -backed representations. In fact, that was my
first attempt a year ago (Breeze) but it unfortunately was not where it
needed to be (not sure about now).

As for LAPack, yes it is easy to integrate. But the only reason I
(personally) haven't integrated it yet is because my problems tend to be
sparse, not dense, and also fairly invasive in terms of custom matrix
traversals (probabilistic fitting, for the most part). So most specifically
tweaked methodologies are thus really more quasi-algebraic than purely
algebraic, unfortunately. Having LAPack blockwise operartors on dense
matrices would not help me terribly there.

But the architectural problem in terms of foundation, and, more
specifically, customization of processes IMO does exist here (in mllib).
This thread (and there was another one just like this a few threads below
this one) are read by me as the manifestation of such lack of algebraic
foundation apis/optimizers.

Re: MLLib : Math on Vector and Matrix

[Xiangrui Meng <me...@gmail.com>]

Hi Dmitriy,

It is sweet to have the bindings, but it is very easy to downgrade the
performance with them. The BLAS/LAPACK APIs have been there for more
than 20 years and they are still the top choice for high-performance
linear algebra. I'm thinking about whether it is possible to make the
evaluation lazy in bindings. For example,

y += a * x

can be translated to an AXPY call instead of creating a temporary
vector for a*x. There were some work in C++ but none achieved good
performance. I'm not sure whether this is a good direction to explore.

Best,
Xiangrui

Re: MLLib : Math on Vector and Matrix

[Dmitriy Lyubimov <dl...@gmail.com>]

in my humble opinion Spark should've supported linalg a-la [1] before it
even started dumping methodologies into mllib.

[1] http://mahout.apache.org/users/sparkbindings/home.html


On Wed, Jul 2, 2014 at 2:16 PM, Thunder Stumpges <thunder.stumpges@gmail.com
> wrote:

> Thanks. I always hate having to do stuff like this. It seems like they
> went a bit overboard with all the "private[mllib]" declarations... possibly
> all in the name of "thou shalt not change your public API". If you don't
> make your public API usable, we end up having to work around it anyway...
>
> Oh well.
>
> Thunder
>
>
>
> On Wed, Jul 2, 2014 at 2:05 PM, Koert Kuipers <ko...@tresata.com> wrote:
>
>> i did the second option: re-implemented .toBreeze as .breeze using pimp
>> classes
>>
>>
>> On Wed, Jul 2, 2014 at 5:00 PM, Thunder Stumpges <
>> thunder.stumpges@gmail.com> wrote:
>>
>>> I am upgrading from Spark 0.9.0 to 1.0 and I had a pretty good amount of
>>> code working with internals of MLLib. One of the big changes was the move
>>> from the old jblas.Matrix to the Vector/Matrix classes included in MLLib.
>>>
>>> However I don't see how we're supposed to use them for ANYTHING other
>>> than a container for passing data to the included APIs... how do we do any
>>> math on them? Looking at the internal code, there are quite a number of
>>> private[mllib] declarations including access to the Breeze representations
>>> of the classes.
>>>
>>> Was there a good reason this was not exposed? I could see maybe not
>>> wanting to expose the 'toBreeze' function which would tie it to the breeze
>>> implementation, however it would be nice to have the various mathematics
>>> wrapped at least.
>>>
>>> Right now I see no way to code any vector/matrix math without moving my
>>> code namespaces into org.apache.spark.mllib or duplicating the code in
>>> 'toBreeze' in my own util functions. Not very appealing.
>>>
>>> What are others doing?
>>> thanks,
>>> Thunder
>>>
>>>
>>
>

Re: MLLib : Math on Vector and Matrix

[Thunder Stumpges <th...@gmail.com>]

Thanks. I always hate having to do stuff like this. It seems like they went
a bit overboard with all the "private[mllib]" declarations... possibly all
in the name of "thou shalt not change your public API". If you don't make
your public API usable, we end up having to work around it anyway...

Oh well.

Thunder



On Wed, Jul 2, 2014 at 2:05 PM, Koert Kuipers <ko...@tresata.com> wrote:

> i did the second option: re-implemented .toBreeze as .breeze using pimp
> classes
>
>
> On Wed, Jul 2, 2014 at 5:00 PM, Thunder Stumpges <
> thunder.stumpges@gmail.com> wrote:
>
>> I am upgrading from Spark 0.9.0 to 1.0 and I had a pretty good amount of
>> code working with internals of MLLib. One of the big changes was the move
>> from the old jblas.Matrix to the Vector/Matrix classes included in MLLib.
>>
>> However I don't see how we're supposed to use them for ANYTHING other
>> than a container for passing data to the included APIs... how do we do any
>> math on them? Looking at the internal code, there are quite a number of
>> private[mllib] declarations including access to the Breeze representations
>> of the classes.
>>
>> Was there a good reason this was not exposed? I could see maybe not
>> wanting to expose the 'toBreeze' function which would tie it to the breeze
>> implementation, however it would be nice to have the various mathematics
>> wrapped at least.
>>
>> Right now I see no way to code any vector/matrix math without moving my
>> code namespaces into org.apache.spark.mllib or duplicating the code in
>> 'toBreeze' in my own util functions. Not very appealing.
>>
>> What are others doing?
>> thanks,
>> Thunder
>>
>>
>

Re: MLLib : Math on Vector and Matrix

[Koert Kuipers <ko...@tresata.com>]

i did the second option: re-implemented .toBreeze as .breeze using pimp
classes


On Wed, Jul 2, 2014 at 5:00 PM, Thunder Stumpges <thunder.stumpges@gmail.com
> wrote:

> I am upgrading from Spark 0.9.0 to 1.0 and I had a pretty good amount of
> code working with internals of MLLib. One of the big changes was the move
> from the old jblas.Matrix to the Vector/Matrix classes included in MLLib.
>
> However I don't see how we're supposed to use them for ANYTHING other than
> a container for passing data to the included APIs... how do we do any math
> on them? Looking at the internal code, there are quite a number of
> private[mllib] declarations including access to the Breeze representations
> of the classes.
>
> Was there a good reason this was not exposed? I could see maybe not
> wanting to expose the 'toBreeze' function which would tie it to the breeze
> implementation, however it would be nice to have the various mathematics
> wrapped at least.
>
> Right now I see no way to code any vector/matrix math without moving my
> code namespaces into org.apache.spark.mllib or duplicating the code in
> 'toBreeze' in my own util functions. Not very appealing.
>
> What are others doing?
> thanks,
> Thunder
>
>