[jira] [Commented] (IGNITE-4301) Hadoop: send only one shuffle ack at the very end of shuffle processing.

["Vladimir Ozerov (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15722325#comment-15722325 ] 

Vladimir Ozerov commented on IGNITE-4301:
-----------------------------------------

Design:

1) Send shuffle messages as usual.
2) When shuffle is finished on particular node ("last mapper" situation), send single "shuffle finished" message.
3) Remote node accepts this message and sends response when all shuffle tasks for original node is finished.

This way we will always send only 1 shuffle ack for N shuffle messages, as opposed to N acks now. Moreover, we will not have to store shuffle messages in-memory.

> Hadoop: send only one shuffle ack at the very end of shuffle processing.
> ------------------------------------------------------------------------
>
>                 Key: IGNITE-4301
>                 URL: https://issues.apache.org/jira/browse/IGNITE-4301
>             Project: Ignite
>          Issue Type: Sub-task
>          Components: general, hadoop
>    Affects Versions: 1.7, 1.8
>            Reporter: Vladimir Ozerov
>            Assignee: Vladimir Ozerov
>             Fix For: 2.0
>
>
> This shuffle acks will be processed ASAP, hence minimizing wait during completion.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)