You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Thomas Jungblut (Created) (JIRA)" <ji...@apache.org> on 2011/12/03 18:50:39 UTC
[jira] [Created] (HAMA-485) Fill Counters with useful information
Fill Counters with useful information
-------------------------------------
Key: HAMA-485
URL: https://issues.apache.org/jira/browse/HAMA-485
Project: Hama
Issue Type: Improvement
Affects Versions: 0.4.0
Reporter: Thomas Jungblut
Fix For: 0.5.0
We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-485:
---------------------------------
Comment: was deleted
(was: There were some hunk fails, but I resolved them accordingly. -> Resolved and thanks for the review Edward)
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch, HAMA-485_2.patch, HAMA-485_3.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut reassigned HAMA-485:
------------------------------------
Assignee: Thomas Jungblut
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225431#comment-13225431 ]
Thomas Jungblut commented on HAMA-485:
--------------------------------------
Oh I lost track of it. I'm going to commit this now.
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch, HAMA-485_2.patch, HAMA-485_3.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-485) Fill Counters with useful information
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222193#comment-13222193 ]
Edward J. Yoon commented on HAMA-485:
-------------------------------------
+1 Let's add this to TRUNK.
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch, HAMA-485_2.patch, HAMA-485_3.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225445#comment-13225445 ]
Thomas Jungblut commented on HAMA-485:
--------------------------------------
There were some hunk fails, but I resolved them accordingly. -> Resolved and thanks for the review Edward
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch, HAMA-485_2.patch, HAMA-485_3.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-485:
---------------------------------
Attachment: HAMA-485_2.patch
There it is, build is running well.
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch, HAMA-485_2.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-485:
---------------------------------
Status: Patch Available (was: Open)
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch, HAMA-485_2.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-485:
---------------------------------
Status: Open (was: Patch Available)
forgot the record reader..
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215546#comment-13215546 ]
Thomas Jungblut commented on HAMA-485:
--------------------------------------
{noformat}
12/02/24 11:26:00 INFO bsp.BSPJobClient: The total number of supersteps: 28
12/02/24 11:26:00 INFO bsp.BSPJobClient: Counters: 6
12/02/24 11:26:00 INFO bsp.BSPJobClient: org.apache.hama.bsp.JobInProgress$JobCounter
12/02/24 11:26:00 INFO bsp.BSPJobClient: LAUNCHED_TASKS=2
12/02/24 11:26:00 INFO bsp.BSPJobClient: org.apache.hama.bsp.BSPPeerImpl$PeerCounter
12/02/24 11:26:00 INFO bsp.BSPJobClient: SUPERSTEPS=28
12/02/24 11:26:00 INFO bsp.BSPJobClient: SUPERSTEP_SUM=60
12/02/24 11:26:00 INFO bsp.BSPJobClient: TIME_IN_SYNC_MS=0
12/02/24 11:26:00 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=120
12/02/24 11:26:00 INFO bsp.BSPJobClient: TASK_OUTPUT_RECORDS=120
{noformat}
Looks like this in the testcase.
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch, HAMA-485_2.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-485) Fill Counters with useful information
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225645#comment-13225645 ]
Hudson commented on HAMA-485:
-----------------------------
Integrated in Hama-Nightly #478 (See [https://builds.apache.org/job/Hama-Nightly/478/])
[HAMA-485]: Fill Counters with useful information (Revision 1298541)
Result = FAILURE
tjungblut :
Files :
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPPeer.java
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/bsp/LocalBSPRunner.java
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/AvroMessageManagerImpl.java
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/HadoopMessageManagerImpl.java
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/bsp/message/MessageManager.java
* /incubator/hama/trunk/core/src/test/java/org/apache/hama/bsp/message/TestAvroMessageManager.java
* /incubator/hama/trunk/core/src/test/java/org/apache/hama/bsp/message/TestHadoopMessageManager.java
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch, HAMA-485_2.patch, HAMA-485_3.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212896#comment-13212896 ]
Thomas Jungblut commented on HAMA-485:
--------------------------------------
TODO:
- record reader / writer wrapper like in hadoop which tracks the bytes read/written.
- Sync Service integration
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-485:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
There were some hunk fails, but I resolved them accordingly. -> Resolved and thanks for the review Edward.
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch, HAMA-485_2.patch, HAMA-485_3.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-485:
---------------------------------
Status: Patch Available (was: Open)
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-485:
---------------------------------
Attachment: HAMA-485.patch
really hacky work, but now we have a bit more useful counters. Message related things like bytes transferred and compressed are only available in the avro messenger, because the Hadoop RPC doesn't offer enough information.
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-485) Fill Counters with useful information
Posted by "Thomas Jungblut (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-485:
---------------------------------
Attachment: HAMA-485_3.patch
New patch which makes everything better.
Please review it.
> Fill Counters with useful information
> -------------------------------------
>
> Key: HAMA-485
> URL: https://issues.apache.org/jira/browse/HAMA-485
> Project: Hama
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Labels: bsp
> Fix For: 0.5.0
>
> Attachments: HAMA-485.patch, HAMA-485_2.patch, HAMA-485_3.patch
>
>
> We want to add message counters (bytes transfered, local messages sent, outward messages sent, time in sync etc.).
> Counters are already present, we should fill them accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira