You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Chandan Misra (JIRA)" <ji...@apache.org> on 2018/01/31 08:06:00 UTC
[jira] [Comment Edited] (SPARK-23266) Matrix Inversion on
BlockMatrix
[ https://issues.apache.org/jira/browse/SPARK-23266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346398#comment-16346398 ]
Chandan Misra edited comment on SPARK-23266 at 1/31/18 8:05 AM:
----------------------------------------------------------------
[Kriging|https://en.wikipedia.org/wiki/Kriging] is a geostatistical method for interpolating known attribute values of scattered data points to predict unknown attribute values at some other points. It is based on kriging weight calculation which has the equation of the form Cw=D. To get the weight vector, we need one matrix inversion and one matrix-vector multiplication. The size of C (covariance matrix), w vector and D vector are nxn, nx1 and nx1 respectively, where n is the number of interpolating points (input points) and interpolation is done for a single output point.
Now, when the output points change, we only require to change matrix D (vector for single point), and thus do not have to do the inverse again and again. We require matrix-vector multiplication on several nodes which is quick i.e. O(n^2) and easy.
was (Author: chandan-misra):
Kriging is a geostatistical method for interpolating known attribute values of scattered data points to predict unknown attribute values at some other points. It is based on kriging weight calculation which has the equation of the form Cw=D. To get the weight vector, we need one matrix inversion and one matrix-vector multiplication. The size of C (covariance matrix), w vector and D vector are nxn, nx1 and nx1 respectively, where n is the number of interpolating points (input points) and interpolation is done for a single output point.
Now, when the output points change, we only require to change matrix D (vector for single point), and thus do not have to do the inverse again and again. We require matrix-vector multiplication on several nodes which is quick i.e. O(n^2) and easy.
> Matrix Inversion on BlockMatrix
> -------------------------------
>
> Key: SPARK-23266
> URL: https://issues.apache.org/jira/browse/SPARK-23266
> Project: Spark
> Issue Type: New Feature
> Components: MLlib
> Affects Versions: 2.2.1
> Reporter: Chandan Misra
> Priority: Minor
>
> Matrix inversion is the basic building block for many other algorithms like regression, classification, geostatistical analysis using ordinary kriging etc. A simple Spark BlockMatrix based efficient distributed divide-and-conquer algorithm can be implemented using only *6* multiplications in each recursion level of the algorithm. The reference paper can be found in
> [https://arxiv.org/abs/1801.04723]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org