You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Edward J. Yoon (JIRA)" <ji...@apache.org> on 2009/02/11 10:54:59 UTC
[jira] Created: (HAMA-151) Add multiplication example of file
matrices
Add multiplication example of file matrices
-------------------------------------------
Key: HAMA-151
URL: https://issues.apache.org/jira/browse/HAMA-151
Project: Hama
Issue Type: New Feature
Components: documentation , examples
Affects Versions: 0.1.0
Reporter: Edward J. Yoon
Assignee: Edward J. Yoon
Fix For: 0.1.0
Using CollectBlocksMap (See also HAMA-150), We can collect the blocks to 'collectionTable' from HDFS files as describe below.
It would be nice if it added up the examples.
-----
public static class MyMapper extends CollectBlocksBase implements
CollectBlocksMap<IntWritable, MapWritable> {
private MapWritable value;
@Override
public void map(IntWritable key, MapWritable value,
OutputCollector<BlockID, VectorWritable> output, Reporter reporter)
throws IOException {
int startColumn;
int endColumn;
int blkRow = key.get() / mBlockRowSize;
this.value = value;
int i = 0;
do {
startColumn = i * mBlockColSize;
endColumn = startColumn + mBlockColSize - 1;
if (endColumn >= mColumns) // the last sub vector
endColumn = mColumns - 1;
output.collect(new BlockID(blkRow, i), new VectorWritable(key.get(),
subVector(startColumn, endColumn)));
i++;
} while (endColumn < (mColumns - 1));
}
private DenseVector subVector(int i0, int i1) {
DenseVector res = new DenseVector();
for (int i = i0; i <= i1; i++) {
res.set(i, ((DoubleWritable) this.value.get(new IntWritable(i))).get());
}
return res;
}
}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-151) Add multiplication example of file
matrices
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-151:
--------------------------------
Status: Patch Available (was: Open)
Submit patch to hudson.
> Add multiplication example of file matrices
> -------------------------------------------
>
> Key: HAMA-151
> URL: https://issues.apache.org/jira/browse/HAMA-151
> Project: Hama
> Issue Type: New Feature
> Components: documentation , examples
> Affects Versions: 0.1.0
> Reporter: Edward J. Yoon
> Assignee: Edward J. Yoon
> Fix For: 0.1.0
>
> Attachments: HAMA-151.patch
>
>
> Using CollectBlocksMap (See also HAMA-150), We can collect the blocks to 'collectionTable' from HDFS files as describe below.
> It would be nice if it added up the examples.
> -----
> public static class MyMapper extends CollectBlocksBase implements
> CollectBlocksMap<IntWritable, MapWritable> {
> private MapWritable value;
> @Override
> public void map(IntWritable key, MapWritable value,
> OutputCollector<BlockID, VectorWritable> output, Reporter reporter)
> throws IOException {
> int startColumn;
> int endColumn;
> int blkRow = key.get() / mBlockRowSize;
> this.value = value;
> int i = 0;
> do {
> startColumn = i * mBlockColSize;
> endColumn = startColumn + mBlockColSize - 1;
> if (endColumn >= mColumns) // the last sub vector
> endColumn = mColumns - 1;
> output.collect(new BlockID(blkRow, i), new VectorWritable(key.get(),
> subVector(startColumn, endColumn)));
> i++;
> } while (endColumn < (mColumns - 1));
> }
> private DenseVector subVector(int i0, int i1) {
> DenseVector res = new DenseVector();
> for (int i = i0; i <= i1; i++) {
> res.set(i, ((DoubleWritable) this.value.get(new IntWritable(i))).get());
> }
> return res;
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-151) Add multiplication example of file
matrices
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672863#action_12672863 ]
Hudson commented on HAMA-151:
-----------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12400073/HAMA-151.patch
against trunk revision 743189.
@author +1. The patch does not contain any @author tags.
tests included +1. The patch appears to include 4 new or modified tests.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hama-Patch/151/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hama-Patch/151/artifact/trunk/build/reports/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hama-Patch/151/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hama-Patch/151/console
This message is automatically generated.
> Add multiplication example of file matrices
> -------------------------------------------
>
> Key: HAMA-151
> URL: https://issues.apache.org/jira/browse/HAMA-151
> Project: Hama
> Issue Type: New Feature
> Components: documentation , examples
> Affects Versions: 0.1.0
> Reporter: Edward J. Yoon
> Assignee: Edward J. Yoon
> Fix For: 0.1.0
>
> Attachments: HAMA-151.patch
>
>
> Using CollectBlocksMap (See also HAMA-150), We can collect the blocks to 'collectionTable' from HDFS files as describe below.
> It would be nice if it added up the examples.
> -----
> public static class MyMapper extends CollectBlocksBase implements
> CollectBlocksMap<IntWritable, MapWritable> {
> private MapWritable value;
> @Override
> public void map(IntWritable key, MapWritable value,
> OutputCollector<BlockID, VectorWritable> output, Reporter reporter)
> throws IOException {
> int startColumn;
> int endColumn;
> int blkRow = key.get() / mBlockRowSize;
> this.value = value;
> int i = 0;
> do {
> startColumn = i * mBlockColSize;
> endColumn = startColumn + mBlockColSize - 1;
> if (endColumn >= mColumns) // the last sub vector
> endColumn = mColumns - 1;
> output.collect(new BlockID(blkRow, i), new VectorWritable(key.get(),
> subVector(startColumn, endColumn)));
> i++;
> } while (endColumn < (mColumns - 1));
> }
> private DenseVector subVector(int i0, int i1) {
> DenseVector res = new DenseVector();
> for (int i = i0; i <= i1; i++) {
> res.set(i, ((DoubleWritable) this.value.get(new IntWritable(i))).get());
> }
> return res;
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-151) Add multiplication example of file
matrices
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12673123#action_12673123 ]
Edward J. Yoon commented on HAMA-151:
-------------------------------------
Sangwon Seo, Thanks for your review.
Could you give me more comments?
> Add multiplication example of file matrices
> -------------------------------------------
>
> Key: HAMA-151
> URL: https://issues.apache.org/jira/browse/HAMA-151
> Project: Hama
> Issue Type: New Feature
> Components: documentation , examples
> Affects Versions: 0.1.0
> Reporter: Edward J. Yoon
> Assignee: Edward J. Yoon
> Fix For: 0.1.0
>
> Attachments: HAMA-151.patch
>
>
> Using CollectBlocksMap (See also HAMA-150), We can collect the blocks to 'collectionTable' from HDFS files as describe below.
> It would be nice if it added up the examples.
> -----
> public static class MyMapper extends CollectBlocksBase implements
> CollectBlocksMap<IntWritable, MapWritable> {
> private MapWritable value;
> @Override
> public void map(IntWritable key, MapWritable value,
> OutputCollector<BlockID, VectorWritable> output, Reporter reporter)
> throws IOException {
> int startColumn;
> int endColumn;
> int blkRow = key.get() / mBlockRowSize;
> this.value = value;
> int i = 0;
> do {
> startColumn = i * mBlockColSize;
> endColumn = startColumn + mBlockColSize - 1;
> if (endColumn >= mColumns) // the last sub vector
> endColumn = mColumns - 1;
> output.collect(new BlockID(blkRow, i), new VectorWritable(key.get(),
> subVector(startColumn, endColumn)));
> i++;
> } while (endColumn < (mColumns - 1));
> }
> private DenseVector subVector(int i0, int i1) {
> DenseVector res = new DenseVector();
> for (int i = i0; i <= i1; i++) {
> res.set(i, ((DoubleWritable) this.value.get(new IntWritable(i))).get());
> }
> return res;
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-151) Add multiplication example of file
matrices
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672572#action_12672572 ]
Edward J. Yoon commented on HAMA-151:
-------------------------------------
'CollectBlocksMapreduceBase' seems right name.
> Add multiplication example of file matrices
> -------------------------------------------
>
> Key: HAMA-151
> URL: https://issues.apache.org/jira/browse/HAMA-151
> Project: Hama
> Issue Type: New Feature
> Components: documentation , examples
> Affects Versions: 0.1.0
> Reporter: Edward J. Yoon
> Assignee: Edward J. Yoon
> Fix For: 0.1.0
>
>
> Using CollectBlocksMap (See also HAMA-150), We can collect the blocks to 'collectionTable' from HDFS files as describe below.
> It would be nice if it added up the examples.
> -----
> public static class MyMapper extends CollectBlocksBase implements
> CollectBlocksMap<IntWritable, MapWritable> {
> private MapWritable value;
> @Override
> public void map(IntWritable key, MapWritable value,
> OutputCollector<BlockID, VectorWritable> output, Reporter reporter)
> throws IOException {
> int startColumn;
> int endColumn;
> int blkRow = key.get() / mBlockRowSize;
> this.value = value;
> int i = 0;
> do {
> startColumn = i * mBlockColSize;
> endColumn = startColumn + mBlockColSize - 1;
> if (endColumn >= mColumns) // the last sub vector
> endColumn = mColumns - 1;
> output.collect(new BlockID(blkRow, i), new VectorWritable(key.get(),
> subVector(startColumn, endColumn)));
> i++;
> } while (endColumn < (mColumns - 1));
> }
> private DenseVector subVector(int i0, int i1) {
> DenseVector res = new DenseVector();
> for (int i = i0; i <= i1; i++) {
> res.set(i, ((DoubleWritable) this.value.get(new IntWritable(i))).get());
> }
> return res;
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-151) Add multiplication example of file
matrices
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-151:
--------------------------------
Attachment: HAMA-151.patch
It's a example of sequencefile matrix multiplication.
> Add multiplication example of file matrices
> -------------------------------------------
>
> Key: HAMA-151
> URL: https://issues.apache.org/jira/browse/HAMA-151
> Project: Hama
> Issue Type: New Feature
> Components: documentation , examples
> Affects Versions: 0.1.0
> Reporter: Edward J. Yoon
> Assignee: Edward J. Yoon
> Fix For: 0.1.0
>
> Attachments: HAMA-151.patch
>
>
> Using CollectBlocksMap (See also HAMA-150), We can collect the blocks to 'collectionTable' from HDFS files as describe below.
> It would be nice if it added up the examples.
> -----
> public static class MyMapper extends CollectBlocksBase implements
> CollectBlocksMap<IntWritable, MapWritable> {
> private MapWritable value;
> @Override
> public void map(IntWritable key, MapWritable value,
> OutputCollector<BlockID, VectorWritable> output, Reporter reporter)
> throws IOException {
> int startColumn;
> int endColumn;
> int blkRow = key.get() / mBlockRowSize;
> this.value = value;
> int i = 0;
> do {
> startColumn = i * mBlockColSize;
> endColumn = startColumn + mBlockColSize - 1;
> if (endColumn >= mColumns) // the last sub vector
> endColumn = mColumns - 1;
> output.collect(new BlockID(blkRow, i), new VectorWritable(key.get(),
> subVector(startColumn, endColumn)));
> i++;
> } while (endColumn < (mColumns - 1));
> }
> private DenseVector subVector(int i0, int i1) {
> DenseVector res = new DenseVector();
> for (int i = i0; i <= i1; i++) {
> res.set(i, ((DoubleWritable) this.value.get(new IntWritable(i))).get());
> }
> return res;
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-151) Add multiplication example of file
matrices
Posted by "Sangwon Seo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672878#action_12672878 ]
Sangwon Seo commented on HAMA-151:
----------------------------------
It's good example seemingly.
> Add multiplication example of file matrices
> -------------------------------------------
>
> Key: HAMA-151
> URL: https://issues.apache.org/jira/browse/HAMA-151
> Project: Hama
> Issue Type: New Feature
> Components: documentation , examples
> Affects Versions: 0.1.0
> Reporter: Edward J. Yoon
> Assignee: Edward J. Yoon
> Fix For: 0.1.0
>
> Attachments: HAMA-151.patch
>
>
> Using CollectBlocksMap (See also HAMA-150), We can collect the blocks to 'collectionTable' from HDFS files as describe below.
> It would be nice if it added up the examples.
> -----
> public static class MyMapper extends CollectBlocksBase implements
> CollectBlocksMap<IntWritable, MapWritable> {
> private MapWritable value;
> @Override
> public void map(IntWritable key, MapWritable value,
> OutputCollector<BlockID, VectorWritable> output, Reporter reporter)
> throws IOException {
> int startColumn;
> int endColumn;
> int blkRow = key.get() / mBlockRowSize;
> this.value = value;
> int i = 0;
> do {
> startColumn = i * mBlockColSize;
> endColumn = startColumn + mBlockColSize - 1;
> if (endColumn >= mColumns) // the last sub vector
> endColumn = mColumns - 1;
> output.collect(new BlockID(blkRow, i), new VectorWritable(key.get(),
> subVector(startColumn, endColumn)));
> i++;
> } while (endColumn < (mColumns - 1));
> }
> private DenseVector subVector(int i0, int i1) {
> DenseVector res = new DenseVector();
> for (int i = i0; i <= i1; i++) {
> res.set(i, ((DoubleWritable) this.value.get(new IntWritable(i))).get());
> }
> return res;
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-151) Add multiplication example of file
matrices
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-151:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this.
> Add multiplication example of file matrices
> -------------------------------------------
>
> Key: HAMA-151
> URL: https://issues.apache.org/jira/browse/HAMA-151
> Project: Hama
> Issue Type: New Feature
> Components: documentation , examples
> Affects Versions: 0.1.0
> Reporter: Edward J. Yoon
> Assignee: Edward J. Yoon
> Fix For: 0.1.0
>
> Attachments: HAMA-151.patch
>
>
> Using CollectBlocksMap (See also HAMA-150), We can collect the blocks to 'collectionTable' from HDFS files as describe below.
> It would be nice if it added up the examples.
> -----
> public static class MyMapper extends CollectBlocksBase implements
> CollectBlocksMap<IntWritable, MapWritable> {
> private MapWritable value;
> @Override
> public void map(IntWritable key, MapWritable value,
> OutputCollector<BlockID, VectorWritable> output, Reporter reporter)
> throws IOException {
> int startColumn;
> int endColumn;
> int blkRow = key.get() / mBlockRowSize;
> this.value = value;
> int i = 0;
> do {
> startColumn = i * mBlockColSize;
> endColumn = startColumn + mBlockColSize - 1;
> if (endColumn >= mColumns) // the last sub vector
> endColumn = mColumns - 1;
> output.collect(new BlockID(blkRow, i), new VectorWritable(key.get(),
> subVector(startColumn, endColumn)));
> i++;
> } while (endColumn < (mColumns - 1));
> }
> private DenseVector subVector(int i0, int i1) {
> DenseVector res = new DenseVector();
> for (int i = i0; i <= i1; i++) {
> res.set(i, ((DoubleWritable) this.value.get(new IntWritable(i))).get());
> }
> return res;
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.