Solve system of linear equations in Spark

[Cooper <ah...@gmail.com>]

I have a system of linear equations in the form of Ax = b to solve in Spark.

A is n by n

b is n by 1

I represent 'A' in the form of IndexedRowMatrix or RowMatrix and 'b' in the
form of DenseMatrix or DenseVector.

How can I solve this system to calculate the 'x' vector?

If the suggested solution is  Cholesky Decomposition
<https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/linalg/CholeskyDecomposition.scala> 
, would you please guide me through doing it as it is not part of the public
API ? For example if the original matrix A is:

1,2,3,4
2,1,5,6
3,5,1,7
4,6,7,1

and b is:

5,6,7,8

What is passed as argument to the "solve" method ?

Any other solution other than inversing 'A' would be very helpful.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Solve-system-of-linear-equations-in-Spark-tp27847.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org

Re: Solve system of linear equations in Spark

[Sean Owen <so...@cloudera.com>]

Do you not just want to use linear regression?
https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala

Of course it requires a DataFrame-like input but that may be more natural
to begin with.

If the data set is small, then putting it on the driver and solving locally
with a library is pretty easy.

The Cholesky decomposition above doesn't solve the linear system itself,
but helps solve AtAx = Atb, because AtA and Atb are small and so that part
can be done locally.


On Thu, Oct 6, 2016 at 6:49 AM Cooper <ah...@gmail.com> wrote:

> I have a system of linear equations in the form of Ax = b to solve in
> Spark.
>
> A is n by n
>
> b is n by 1
>
> I represent 'A' in the form of IndexedRowMatrix or RowMatrix and 'b' in the
> form of DenseMatrix or DenseVector.
>
> How can I solve this system to calculate the 'x' vector?
>
> If the suggested solution is  Cholesky Decomposition
> <
> https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/linalg/CholeskyDecomposition.scala
> >
> , would you please guide me through doing it as it is not part of the
> public
> API ? For example if the original matrix A is:
>
> 1,2,3,4
> 2,1,5,6
> 3,5,1,7
> 4,6,7,1
>
> and b is:
>
> 5,6,7,8
>
> What is passed as argument to the "solve" method ?
>
> Any other solution other than inversing 'A' would be very helpful.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Solve-system-of-linear-equations-in-Spark-tp27847.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>