You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hama.apache.org by Apache Wiki <wi...@apache.org> on 2009/03/10 09:56:14 UTC

[Hama Wiki] Trivial Update of "Architecture" by udanax

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.

The following page has been changed by udanax:
http://wiki.apache.org/hama/Architecture

------------------------------------------------------------------------------
  ----
  = Overview =
  
- This is intended to explain and illustrate the concept of Hama. There are three main parts: 
+ This is intended to explain and illustrate the concept of Hama. There are two main parts: 
+  
+  * How to store the matrices?
+  * How to perform matrix operations using !MapReduce?
  
  == Building Block ==
  
  [http://wiki.apache.org/hama-data/attachments/Architecture/attachments/block.png]
  
+ == Store Dense/Sparse Matrices ==
+ 
  To store the matrices, Hama use a [http://hadoop.apache.org/hbase/ Hbase] -- Matrices are basically tables. They are ways of storing numbers and other things. Typical matrix has rows and columns. Actually called a 2-way matrix because it has two dimensions. For example, you might have respondents-by-attitudes. Of course, you might collect the same data on the same people at 5 points in time. In that case, you either have 5 different 2-way matrices, or you could think of it as a 3-way matrix, that is respondent-by-attitude-by-time.
  
  -- ''Just a thought, considering the depleted activity in HBase should we not explore ways to avoid HBase ? --Prasen ''
  
- ----
- 
- = Dense Matrix =
- For dense matrix computations, The block-partitioned algorithms used to minimize data movement and network cost. Dense Matrix and Blocked Dense Matrix are both stored in one table with other metadata. But, Blocked dense matrix can't be synchronized by dense matrix update.
- 
- {{{
-   // Generate matrix with random elements
-   DenseMatrix a = DenseMatrix.random(conf, 1000, 1000);
-   DenseMatrix b = DenseMatrix.random(conf, 1000, 1000);
-   
-   // The type of the Matrix to be blocked, must be dense.
-   a.blocking(2);
-   b.blocking(2);
- 
-   DenseMatrix c = a.mult(b);
- }}}
- 
- For example, The matrix multiplication of the original arrays can be transformed into matrix multiplication of blocks as describe below.
- 
- {{{
- C_block(1,1)=A_block(1,1)*B_block(1,1) + A_block(1,2)*B_block(2,1)
- 
- C                 A               B
- +-----+-----+     +-----+-----+   +-----+-----+
- | x x |     |     | --> | --> |   | | | |     |
- | x x |     |     | --> | --> |   | ↓ ↓ |     |
- +-----+-----+  =  +-----+-----+ * +-----+-----+
- |     |     |     |     |     |   | | | |     |
- |     |     |     |     |     |   | ↓ ↓ |     |
- +-----+-----+     +-----+-----+   +-----+-----+
- }}}
- 
-   -- To be statically sized blocks, What should we do? -- Edward
-