[jira] [Resolved] (GIRAPH-1190) Improve memory usage of SendWorkerOneMessageToManyRequest

["Dionysios Logothetis (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GIRAPH-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dionysios Logothetis resolved GIRAPH-1190.
------------------------------------------
    Resolution: Won't Fix

> Improve memory usage of SendWorkerOneMessageToManyRequest 
> ----------------------------------------------------------
>
>                 Key: GIRAPH-1190
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-1190
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Dionysios Logothetis
>            Assignee: Dionysios Logothetis
>            Priority: Major
>
> The current implementation takes incoming messages stored as ByteArrayOneMessageToManyIds and prepares them as a map from partition id to a ByteArrayVertexIdMessages, which holds the messages for the corresponding partition. It then adds these to the message store.
> However, it is possible that these intermediate lists of message get big before they are added to the message store. If they reach the capacity of the underlying buffers, the job fails. This can be avoided if we push these lists to the message store before the get big. This is mostly beneficial when we use a combiner in which case the message store keeps only one value per vertex.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)