You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2014/02/20 03:15:20 UTC
[jira] [Commented] (HAMA-870) Runtime message compression in Bundle
[ https://issues.apache.org/jira/browse/HAMA-870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906491#comment-13906491 ]
Hudson commented on HAMA-870:
-----------------------------
FAILURE: Integrated in Hama-trunk #277 (See [https://builds.apache.org/job/Hama-trunk/277/])
HAMA-870: Runtime message compression in Bundle (edwardyoon) (edwardyoon: rev 1570039)
* /hama/trunk/CHANGES.txt
* /hama/trunk/conf/hama-default.xml
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPMessageBundle.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/LocalBSPRunner.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/AbstractMessageManager.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/CompressableMessageManager.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/HamaMessageManager.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/HamaMessageManagerImpl.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/MessageManager.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/OutgoingMessageManager.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/OutgoingPOJOMessageBundle.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/compress/BSPMessageCompressor.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/compress/Bzip2Compressor.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/compress/SnappyCompressor.java
* /hama/trunk/core/src/test/java/org/apache/hama/bsp/TestCheckpoint.java
* /hama/trunk/core/src/test/java/org/apache/hama/bsp/message/compress/TestBSPMessageCompressor.java
* /hama/trunk/examples/src/main/java/org/apache/hama/examples/PiEstimator.java
* /hama/trunk/examples/src/main/java/org/apache/hama/examples/SemiClusterJobDriver.java
* /hama/trunk/examples/src/test/java/org/apache/hama/examples/SemiClusterMatchingTest.java
* /hama/trunk/graph/src/main/java/org/apache/hama/graph/OutgoingVertexMessagesManager.java
* /hama/trunk/graph/src/test/java/org/apache/hama/graph/TestSubmitGraphJob.java
> Runtime message compression in Bundle
> -------------------------------------
>
> Key: HAMA-870
> URL: https://issues.apache.org/jira/browse/HAMA-870
> Project: Hama
> Issue Type: Improvement
> Components: bsp core
> Affects Versions: 0.6.3
> Reporter: Edward J. Yoon
> Assignee: Edward J. Yoon
> Fix For: 0.7.0
>
> Attachments: runtimecompression.patch
>
>
> Original compression flow was like below:
> 1) Bundles messages in outgoing queue
> 2) Serializes bundle object to byte array
> 3) Compresses byte array to new byte array
> 4) Creates BSPCompressedBundle object for RPC communication
> So, it was extremely slow!
> Since BSPMessageBundle handles messages in byte array now, we can runtime compress the messages during bundling.
> We don't need BSPCompressedBundle class anymore.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)