You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by "Dhruv Mahajan (JIRA)" <ji...@apache.org> on 2015/08/04 09:36:04 UTC
[jira] [Updated] (REEF-554) Update IMRU APIs to enable writing of
REEF IMRU driver and client
[ https://issues.apache.org/jira/browse/REEF-554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dhruv Mahajan updated REEF-554:
-------------------------------
Description:
The aim of this jira is to modify IMRU APIs to enable implementation of REEF IMRU driver and client on top. Specifically,
a) APIs will be modified. Specifically,
- Instead of a single combined configuration supplied to the driver, it will get configuration separately for different components like Map function, Update function, Codecs, Reduce function. This is done because a) On the user side we in any case take them as separate ones, b) On driver not all evaluators need all the configurations, for example evaluator being assigned the Map function does not need Update function configuration and c) Merging all configurations together has issues of potential conflicts: for example TMapInput and TMapOutput using similar base codec for example VectorCodec.
- Introduce extra user specifications corresponding to PipelineDataConverters for TMapInput and TMapOutput in IMRUJobDefinition, Partitioned Dataset and number of mappers. The number of mappers is redundant since Partitioned Dataset will also have the information, but will be needed in Group Communication.
b) MapperCount example will be modified to be compatible with new API
c) InProcessIMRU will be modified to be compatible with the current API
was:
The aim of this jira is to modify IMRU APIs to enable implementation of REEF IMRU driver and client on top. Specifically,
a) APIS will be modified
b) MapperCount example will be modified to be compatible with new API
c) InProcessIMRU will be modified to be compatible with the current API
> Update IMRU APIs to enable writing of REEF IMRU driver and client
> -----------------------------------------------------------------
>
> Key: REEF-554
> URL: https://issues.apache.org/jira/browse/REEF-554
> Project: REEF
> Issue Type: Sub-task
> Components: REEF.NET
> Environment: C#
> Reporter: Dhruv Mahajan
> Assignee: Dhruv Mahajan
>
> The aim of this jira is to modify IMRU APIs to enable implementation of REEF IMRU driver and client on top. Specifically,
> a) APIs will be modified. Specifically,
> - Instead of a single combined configuration supplied to the driver, it will get configuration separately for different components like Map function, Update function, Codecs, Reduce function. This is done because a) On the user side we in any case take them as separate ones, b) On driver not all evaluators need all the configurations, for example evaluator being assigned the Map function does not need Update function configuration and c) Merging all configurations together has issues of potential conflicts: for example TMapInput and TMapOutput using similar base codec for example VectorCodec.
> - Introduce extra user specifications corresponding to PipelineDataConverters for TMapInput and TMapOutput in IMRUJobDefinition, Partitioned Dataset and number of mappers. The number of mappers is redundant since Partitioned Dataset will also have the information, but will be needed in Group Communication.
> b) MapperCount example will be modified to be compatible with new API
> c) InProcessIMRU will be modified to be compatible with the current API
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)