[jira] [Comment Edited] (REEF-54) Creating skeleton of driver configuration files

["Julia (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/REEF-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237616#comment-14237616 ] 

Julia edited comment on REEF-54 at 12/8/14 5:46 PM:
----------------------------------------------------

Regarding required parameters – I agree we should not simply make it optional such as driver identifier. There are a few ways to handle it;
    a. Still make it optional but set default value for the named parameter. At Java side, this would ensure if the client doesn’t set it, it would always have a value. At .Net side, this would allow the .Net client to set its own value it we don’t set it in the skeleton.
    b. Leave it as required, introduce rebind in Tang. We might need to think about the impact if we allow a named parameter to be rebind. I guess at least we have to clone the existing bindings and create a new ConfigurationBuilder. This would allow C# client to bind the driver identifier again. 

Regarding HttpServer and handlers
There are two things that clients can bind for http. One is HttpServer, the other is HttpHandler. At Java side, we provided HttpServerImpl which starts JettyServer, we also provided HttpServerReefEventHandler. The configuration defined at Java side is for those two as they are written in java. C# clients can still define their own handlers and bind to the CLR configurations.  I guess the question is can C# clients define their own HttpServerImpl and bind it to the HttpServer interface instead of using the HttpServerImpl bound at java side? Well, we don’t support it today. May be later, we should provide a bridge for it. By that time, C# clients would be able to define their HTTP configuration entirely. 



was (Author: juliaw):
Regarding required parameters – I agree we should not simply make it optional such ad driver identifier. There are a few ways to handle it;
    a. Still make it optional but set default value for the named parameter. At Java side, this would ensure if the client doesn’t set it, it would always have a value. At .Net side, this would allow the .Net client to set its own value it we don’t set it in the skeleton.
    b. Leave it as required, introduce rebind in Tang. We might need to think about the impact if we allow a named parameter to be rebind. I guess at least we have to clone the existing bindings and create a new ConfigurationBuilder. This would allow C# client to bind the driver identifier again. 

Regarding HttpServer and handlers
There are two things that clients can bind for http. One is HttpServer, the other is HttpHandler. At Java side, we provided HttpServerImpl which starts JettyServer, we also provided HttpServerReefEventHandler. The configuration defined at Java side is for those two as they are written in java. C# clients can still define their own handlers and bind to the CLR configurations.  I guess the question is can C# clients define their own HttpServerImpl and bind it to the HttpServer interface instead of using the HttpServerImpl bound at java side? Well, we don’t support it today. May be later, we should provide a bridge for it. By that time, C# clients would be able to define their HTTP configuration entirely. 


> Creating skeleton of driver configuration files
> -----------------------------------------------
>
>                 Key: REEF-54
>                 URL: https://issues.apache.org/jira/browse/REEF-54
>             Project: REEF
>          Issue Type: Improvement
>            Reporter: Julia
>            Assignee: Julia
>
> Client can submit a Job through REST API. For that, they need a serialized driver configuration file. In .Net/Java hybrid driver (see REEF-51 for a description of that), today the driver config file is created in Java JobClient, that is inconvenient for the clients.
>  
> In order for clients to set their parameters at their side, we would like to generate a skeleton of the driver configurations which only contains java configuration settings. This would leave flexibility to the client so that they can deserialize the driver configuration, and set additional parameters and then serialize the result into driver config file again.  
> To make the configuration modularized, we would also like to separate HttpServer configuration and NameServer configuration from driver configuration file when creating the skeletons. They would become building blocks. Client can choose what they want in the driver and merge them as need. This would make Http Server and Name server become optional instead of build-in in a .Net/Java hybrid driver. 
> The driver skeleton configuration creation should be part of the project build process so that we can always get the latest config files on the fly. The config files generated should be also packed inside the jar file for clients to consume. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)