[jira] [Commented] (REEF-179) Solve the large memory footprint problem due to multiple serializations during call to an operator

["Dhruv Mahajan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/REEF-179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394801#comment-14394801 ] 

Dhruv Mahajan commented on REEF-179:
------------------------------------

We will introduce the concept of StreamingCodec. All the messages (User message, GroupCommunicationMessage and NSMessage) will have streaming codecs. The advantage is that it will delay writing the data till the very end when we will directly write it in to the stream. It will allow directly reading the data from the NetworkStream to the appropriate message at the user side.

> Solve the large memory footprint problem due to multiple serializations during call to an operator
> --------------------------------------------------------------------------------------------------
>
>                 Key: REEF-179
>                 URL: https://issues.apache.org/jira/browse/REEF-179
>             Project: REEF
>          Issue Type: Sub-task
>          Components: REEF-IO, REEF.NET, Wake
>            Reporter: Dhruv Mahajan
>   Original Estimate: 336h
>  Remaining Estimate: 336h
>
> Right now  during each communication call, there are three serialization steps happening where actual data/message to pass is encoded again and again leading to larger memory footprints. We would like to optimize it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)