You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by "Vladimir Ozerov (JIRA)" <ji...@apache.org> on 2016/11/23 08:02:58 UTC
[jira] [Created] (IGNITE-4270) Hadoop: optionally stripe mapper
output for every partition.
Vladimir Ozerov created IGNITE-4270:
---------------------------------------
Summary: Hadoop: optionally stripe mapper output for every partition.
Key: IGNITE-4270
URL: https://issues.apache.org/jira/browse/IGNITE-4270
Project: Ignite
Issue Type: Task
Components: hadoop
Affects Versions: 1.8
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
Fix For: 2.0
Currently we have R maps for M mappers, where R is number of reducers. For this reason many mappers writes to concurrent offheap data structure, loosing time on concurrency burden.
Let's add an option to create R * M maps, so that every mapper has dedicated map for every reducer. This will eliminate almost all concurrency overhead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)