[jira] [Commented] (BEAM-137) Add implicit conf/pipeline-default.conf options file

["Nicholas Verbeck (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244950#comment-16244950 ] 

Nicholas Verbeck commented on BEAM-137:
---------------------------------------

So I'm looking at adding this support to make my life a little easier. What I've currently got in my head is what [~jbonofre] suggested but with a little expansion of logic to support [~lcwik] hints at wanting. As well as add support for multiple files. 

Current idea looks like this:
1) Add addFiles(String... files) to both PipelineOptions and PipelineOptions.Builder. Allowing for any number of files to be read in order of passed in values. fromArgs() being the final be all override of all values
2) The value of each file string would be a URI. Where the scheme will govern both the fetch location (Local, HTTP, etc) and format of data. Allowing for files to be stored where ever the user chooses but also support both the serialized JSON and prop files. As well as allow for more formats in the future. 

Example URI: prop+local:///spark/config.prop

> Add implicit conf/pipeline-default.conf options file
> ----------------------------------------------------
>
>                 Key: BEAM-137
>                 URL: https://issues.apache.org/jira/browse/BEAM-137
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-java-core
>            Reporter: Jean-Baptiste Onofré
>
> Right now, most of users provide the pipeline options via the main arguments.
> For instance, it's the classic way to provide pipeline input, etc.
> For convenience, it would be great that the pipeline looks for options in conf/[pipeline_name]-default.conf by default, and override the options using the main arguments.
> Thoughts ?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)