You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Thomas Jungblut (JIRA)" <ji...@apache.org> on 2011/08/05 13:27:26 UTC
[jira] [Created] (WHIRR-355) [HAMA] Add Hama's PageRank example to
Whirr's examples
[HAMA] Add Hama's PageRank example to Whirr's examples
------------------------------------------------------
Key: WHIRR-355
URL: https://issues.apache.org/jira/browse/WHIRR-355
Project: Whirr
Issue Type: New Feature
Reporter: Thomas Jungblut
As already proposed on Hama's mailing list, we want to add an existing Hama example (Pagerank) to Whirr's examples.
[The link to the mailing list entry.|http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAJ-=ysnWXuUNat71SpQ135TOewpjP_7dpDmn418U-+Y1XBQA4Q@mail.gmail.com%3E]
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-355) [HAMA] Add Hama's PageRank example to
Whirr's examples
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated WHIRR-355:
----------------------------------
Attachment: WHIRR-355_1.patch
> [HAMA] Add Hama's PageRank example to Whirr's examples
> ------------------------------------------------------
>
> Key: WHIRR-355
> URL: https://issues.apache.org/jira/browse/WHIRR-355
> Project: Whirr
> Issue Type: New Feature
> Components: service/hama
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: WHIRR-355.patch, WHIRR-355_1.patch
>
>
> As already proposed on Hama's mailing list, we want to add an existing Hama example (Pagerank) to Whirr's examples.
> [The link to the mailing list entry.|http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAJ-=ysnWXuUNat71SpQ135TOewpjP_7dpDmn418U-+Y1XBQA4Q@mail.gmail.com%3E]
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-355) [HAMA] Add Hama's PageRank example
to Whirr's examples
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085862#comment-13085862 ]
Thomas Jungblut commented on WHIRR-355:
---------------------------------------
{quote}
I can help to test this application.
{quote}
I've finished with the patch, so it would be great if you can test this.
> [HAMA] Add Hama's PageRank example to Whirr's examples
> ------------------------------------------------------
>
> Key: WHIRR-355
> URL: https://issues.apache.org/jira/browse/WHIRR-355
> Project: Whirr
> Issue Type: New Feature
> Components: service/hama
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: WHIRR-355.patch, WHIRR-355_1.patch
>
>
> As already proposed on Hama's mailing list, we want to add an existing Hama example (Pagerank) to Whirr's examples.
> [The link to the mailing list entry.|http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAJ-=ysnWXuUNat71SpQ135TOewpjP_7dpDmn418U-+Y1XBQA4Q@mail.gmail.com%3E]
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-355) [HAMA] Add Hama's PageRank example
to Whirr's examples
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13079919#comment-13079919 ]
Thomas Jungblut commented on WHIRR-355:
---------------------------------------
Lets talk about it here:
https://issues.apache.org/jira/browse/HAMA-420
I would go for a pre-computed and downloadble sequence-file instead of creating it on-the-fly.
We can talk about it in HAMA-420.
> [HAMA] Add Hama's PageRank example to Whirr's examples
> ------------------------------------------------------
>
> Key: WHIRR-355
> URL: https://issues.apache.org/jira/browse/WHIRR-355
> Project: Whirr
> Issue Type: New Feature
> Reporter: Thomas Jungblut
>
> As already proposed on Hama's mailing list, we want to add an existing Hama example (Pagerank) to Whirr's examples.
> [The link to the mailing list entry.|http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAJ-=ysnWXuUNat71SpQ135TOewpjP_7dpDmn418U-+Y1XBQA4Q@mail.gmail.com%3E]
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-355) [HAMA] Add Hama's PageRank example
to Whirr's examples
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088177#comment-13088177 ]
Thomas Jungblut commented on WHIRR-355:
---------------------------------------
I've finished with HAMA-420, so let's push this here :)
> [HAMA] Add Hama's PageRank example to Whirr's examples
> ------------------------------------------------------
>
> Key: WHIRR-355
> URL: https://issues.apache.org/jira/browse/WHIRR-355
> Project: Whirr
> Issue Type: New Feature
> Components: service/hama
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: WHIRR-355.patch, WHIRR-355_1.patch
>
>
> As already proposed on Hama's mailing list, we want to add an existing Hama example (Pagerank) to Whirr's examples.
> [The link to the mailing list entry.|http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAJ-=ysnWXuUNat71SpQ135TOewpjP_7dpDmn418U-+Y1XBQA4Q@mail.gmail.com%3E]
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-355) [HAMA] Add Hama's PageRank example
to Whirr's examples
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13079917#comment-13079917 ]
Edward J. Yoon commented on WHIRR-355:
--------------------------------------
are you gonna add random data generator?
> [HAMA] Add Hama's PageRank example to Whirr's examples
> ------------------------------------------------------
>
> Key: WHIRR-355
> URL: https://issues.apache.org/jira/browse/WHIRR-355
> Project: Whirr
> Issue Type: New Feature
> Reporter: Thomas Jungblut
>
> As already proposed on Hama's mailing list, we want to add an existing Hama example (Pagerank) to Whirr's examples.
> [The link to the mailing list entry.|http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAJ-=ysnWXuUNat71SpQ135TOewpjP_7dpDmn418U-+Y1XBQA4Q@mail.gmail.com%3E]
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-355) [HAMA] Add Hama's PageRank example
to Whirr's examples
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084546#comment-13084546 ]
Andrei Savu commented on WHIRR-355:
-----------------------------------
Have you tried running it? The code looks good to me (just took a quick look over the patch).
> [HAMA] Add Hama's PageRank example to Whirr's examples
> ------------------------------------------------------
>
> Key: WHIRR-355
> URL: https://issues.apache.org/jira/browse/WHIRR-355
> Project: Whirr
> Issue Type: New Feature
> Components: service/hama
> Reporter: Thomas Jungblut
> Attachments: WHIRR-355.patch
>
>
> As already proposed on Hama's mailing list, we want to add an existing Hama example (Pagerank) to Whirr's examples.
> [The link to the mailing list entry.|http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAJ-=ysnWXuUNat71SpQ135TOewpjP_7dpDmn418U-+Y1XBQA4Q@mail.gmail.com%3E]
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-355) [HAMA] Add Hama's PageRank example
to Whirr's examples
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13079916#comment-13079916 ]
Thomas Jungblut commented on WHIRR-355:
---------------------------------------
So there was a [comment | http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAARMrHXB6zOSSBx3Q9pCC8Qjm64F1hV4mF=OiXMkKsVLZ7JVjA@mail.gmail.com%3E]from Adrian how we should do this.
Any suggestions?
For example/benchmark purposes we should generate a big random data file like it already exists in SSSP example [here|http://wiki.apache.org/hama/SSSP]. For this problem, I'll open an issue at Hama's Jira.
> [HAMA] Add Hama's PageRank example to Whirr's examples
> ------------------------------------------------------
>
> Key: WHIRR-355
> URL: https://issues.apache.org/jira/browse/WHIRR-355
> Project: Whirr
> Issue Type: New Feature
> Reporter: Thomas Jungblut
>
> As already proposed on Hama's mailing list, we want to add an existing Hama example (Pagerank) to Whirr's examples.
> [The link to the mailing list entry.|http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAJ-=ysnWXuUNat71SpQ135TOewpjP_7dpDmn418U-+Y1XBQA4Q@mail.gmail.com%3E]
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-355) [HAMA] Add Hama's PageRank example
to Whirr's examples
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084621#comment-13084621 ]
Thomas Jungblut commented on WHIRR-355:
---------------------------------------
No actually I didn't try it.
Anyways I have forgotten to add the example to the main driver.
Is there any way to wait on the number of grooms (to come up) available that were configured?
We don't have a ad-hoc task-scheduling at Hama, so the number of instances must be known when submitting.
This is going to be problematic, along with partitioning.
> [HAMA] Add Hama's PageRank example to Whirr's examples
> ------------------------------------------------------
>
> Key: WHIRR-355
> URL: https://issues.apache.org/jira/browse/WHIRR-355
> Project: Whirr
> Issue Type: New Feature
> Components: service/hama
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: WHIRR-355.patch
>
>
> As already proposed on Hama's mailing list, we want to add an existing Hama example (Pagerank) to Whirr's examples.
> [The link to the mailing list entry.|http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAJ-=ysnWXuUNat71SpQ135TOewpjP_7dpDmn418U-+Y1XBQA4Q@mail.gmail.com%3E]
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-355) [HAMA] Add Hama's PageRank example
to Whirr's examples
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085445#comment-13085445 ]
Edward J. Yoon commented on WHIRR-355:
--------------------------------------
I can help to test this application.
> [HAMA] Add Hama's PageRank example to Whirr's examples
> ------------------------------------------------------
>
> Key: WHIRR-355
> URL: https://issues.apache.org/jira/browse/WHIRR-355
> Project: Whirr
> Issue Type: New Feature
> Components: service/hama
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: WHIRR-355.patch, WHIRR-355_1.patch
>
>
> As already proposed on Hama's mailing list, we want to add an existing Hama example (Pagerank) to Whirr's examples.
> [The link to the mailing list entry.|http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAJ-=ysnWXuUNat71SpQ135TOewpjP_7dpDmn418U-+Y1XBQA4Q@mail.gmail.com%3E]
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-355) [HAMA] Add Hama's PageRank example to
Whirr's examples
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-355:
------------------------------
Component/s: service/hama
Assignee: Thomas Jungblut
> [HAMA] Add Hama's PageRank example to Whirr's examples
> ------------------------------------------------------
>
> Key: WHIRR-355
> URL: https://issues.apache.org/jira/browse/WHIRR-355
> Project: Whirr
> Issue Type: New Feature
> Components: service/hama
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: WHIRR-355.patch
>
>
> As already proposed on Hama's mailing list, we want to add an existing Hama example (Pagerank) to Whirr's examples.
> [The link to the mailing list entry.|http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAJ-=ysnWXuUNat71SpQ135TOewpjP_7dpDmn418U-+Y1XBQA4Q@mail.gmail.com%3E]
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-355) [HAMA] Add Hama's PageRank example to
Whirr's examples
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated WHIRR-355:
----------------------------------
Attachment: WHIRR-355.patch
First version.
There are a lot of haxx. And therefore I don't really know if it works. (probably not at all).
Same problem with the pom.xml, I've added Hama as it is stated in the own project's pom.xml.
> [HAMA] Add Hama's PageRank example to Whirr's examples
> ------------------------------------------------------
>
> Key: WHIRR-355
> URL: https://issues.apache.org/jira/browse/WHIRR-355
> Project: Whirr
> Issue Type: New Feature
> Reporter: Thomas Jungblut
> Attachments: WHIRR-355.patch
>
>
> As already proposed on Hama's mailing list, we want to add an existing Hama example (Pagerank) to Whirr's examples.
> [The link to the mailing list entry.|http://mail-archives.apache.org/mod_mbox/incubator-hama-dev/201107.mbox/%3CCAJ-=ysnWXuUNat71SpQ135TOewpjP_7dpDmn418U-+Y1XBQA4Q@mail.gmail.com%3E]
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira