[jira] [Commented] (GIRAPH-909) support succinct representation of messages in messagestores

["Pavan Kumar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/GIRAPH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14019503#comment-14019503 ] 

Pavan Kumar commented on GIRAPH-909:
------------------------------------

#duplicate- GIRAPH-912 is the current ipsco-facto

> support succinct representation of messages in messagestores
> ------------------------------------------------------------
>
>                 Key: GIRAPH-909
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-909
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Pavan Kumar
>            Assignee: Pavan Kumar
>
> Currently we use ByteArrayVertexIdMessages data-structure to store vertex id & its messages. Even for ByteArrayOneToAllMessages data-structure, while storing in message-store we always convert it to ByteArrayVertexIdMessages, in this case if many vertices on a worker receive the same message, it is stored as many times. This uses up lot of memory, so if we have message stores that can avoid all this duplication in storing messages, we can bring down memory footprint by a large amount.
> Note: however, here the pre-req is that your graph has to be partitioned such that a vertex sends messages to vertices only in a few other workers (not all of them - as happens with HashPartitioning)
> This change depends heavily on GIRAPH-907 & GIRAPH-908
> I already have patch for it, need to tidy up few things, will put it up by end of this week.



--
This message was sent by Atlassian JIRA
(v6.2#6252)