You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Thomas Jungblut (JIRA)" <ji...@apache.org> on 2011/03/30 21:04:05 UTC
[jira] [Commented] (HAMA-362) Design a new data structure for tag
and message data of BSPMessage
[ https://issues.apache.org/jira/browse/HAMA-362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13013588#comment-13013588 ]
Thomas Jungblut commented on HAMA-362:
--------------------------------------
This issue sometimes annoys me. This is very similar to writing byte arrays in HBase, this annoys me too.
Do you have a concrete idea for that?
I've seen that the BSPMessage already defines the Writable interface, my idea would be to implement a "Messagable" interface.
You leave the concrete implementation to the user, but he must provide his own serialization and deserialization.
{code:title=Messagable.java|borderStyle=solid}
public interface Messagable {
// this will be a ByteArrayInputStream,
// or even a real stream if the data is coming from
// the network (don't know what to change behind the messages).
void read(InputStream in);
// ByteArrayOutputStream
void write(OutputStream out);
}
{code}
So you are keeping the underlying byte arrays to keep the data itself, but it is just a more convenient way to pass the arguments to the message.
The interfaces will be passed in both arguments of a constructor for tag and data. Similar to Hadoop's context.write(Writable key, Writable value)
Adding the constructor:
{code:title=BSPMessage.java|borderStyle=solid}
public BSPMessage(Messagable tag, Messagable data) {
// put the data into the byte arrays
}
{code}
> Design a new data structure for tag and message data of BSPMessage
> ------------------------------------------------------------------
>
> Key: HAMA-362
> URL: https://issues.apache.org/jira/browse/HAMA-362
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Fix For: 0.3.0
>
>
> The tag and message type are byte[]. However, I guess, the message of BSPMessage (almost) will be a {adjacent, vector, .., etc} *list*.
> By supporting generic datatypes, it will be able to represent any data structure and make easy-to-program.
> And, It should be compressible, writable, and flexible.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira