[jira] [Commented] (REEF-250) Support global and local Configurations

["Markus Weimer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/REEF-250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14509302#comment-14509302 ] 

Markus Weimer commented on REEF-250:
------------------------------------

[~beysims] and I had an offline conversation about this. We should also have one file for the JVM and CLR configuration each. This leaves us with four configurations:

 # CLR Global
 # CLR Local
 # JVM Global
 # JVM Local

We also considered the idea of using folders of configuration files instead of a single configuration file for each of the 4 configurations. This seemed appealing as it makes the interplay between different systems easier, but we ultimately concluded that it would be a bad idea: It would move the Configuration merge errors to a later stage of the application run. And getting errors as early as possible is one of the main value-adds of Tang and REEF we shouldn't loose.

> Support global and local Configurations
> ---------------------------------------
>
>                 Key: REEF-250
>                 URL: https://issues.apache.org/jira/browse/REEF-250
>             Project: REEF
>          Issue Type: Improvement
>          Components: REEF
>    Affects Versions: 0.10
>            Reporter: Markus Weimer
>            Priority: Minor
>
> We frequently have the need to mirror parameters from the Driver Configuration into all Evaluator configurations. Examples include the job name and network parameters ([REEF-246] and [REEF-247]).
> The current approach to do so is very cumbersome. We have the parameters injected in {{o.a.r.runtime.common.driver.evaluator.EvaluatorManager}} and then bind them to the Evaluator Configuration in {{o.a.r..runtime.common.driver.evaluator.AllocatedEvaluatorImpl#launch}}. This is error-prone and tedious.
> Instead, we could follow the approach we've taken for files: We support the notion of global and local files: Global files are made available on all containers (Driver and Evaluator) in the folder {{reef/global/}} in the working directory of the container. Local files are made available only on that specific container in the folder {{reef/local}}. 
> We should have global and local Configuration as well, each of which is available separately for JVM and CLR containers. We can standardize the file names for them in {{o.a.r.runtime.common.files.REEFFileNames}} to make sure we don't mistype them.
> What do you think?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)