[jira] [Commented] (FLUME-1052) Core configuration component

["jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239224#comment-13239224 ] 

jiraposter@reviews.apache.org commented on FLUME-1052:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4502/
-----------------------------------------------------------

Review request for Flume.


Summary
-------

Main config component.


This addresses bug FLUME-1052.
    https://issues.apache.org/jira/browse/FLUME-1052


Diffs
-----

  flume-ng-configuration/pom.xml PRE-CREATION 
  flume-ng-configuration/src/main/java/org/apache/flume/conf/ComponentConfiguration.java PRE-CREATION 
  flume-ng-configuration/src/main/java/org/apache/flume/conf/ComponentConfigurationFactory.java PRE-CREATION 
  flume-ng-configuration/src/main/java/org/apache/flume/conf/ConfigurationException.java PRE-CREATION 
  flume-ng-configuration/src/main/java/org/apache/flume/conf/Context.java PRE-CREATION 
  flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfiguration.java PRE-CREATION 
  flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfigurationError.java PRE-CREATION 
  flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfigurationErrorType.java PRE-CREATION 
  flume-ng-core/pom.xml 37fb112 
  flume-ng-core/src/main/java/org/apache/flume/Context.java 5294e31 
  flume-ng-core/src/main/java/org/apache/flume/conf/ConfigurableComponent.java PRE-CREATION 
  flume-ng-core/src/main/java/org/apache/flume/conf/Configurables.java 84492e5 
  flume-ng-core/src/test/java/org/apache/flume/sink/TestFailoverSinkProcessor.java 3358cf4 
  flume-ng-core/src/test/java/org/apache/flume/sink/TestLoggerSink.java 92ff6fe 
  flume-ng-core/src/test/java/org/apache/flume/sink/TestRollingFileSink.java 7e26e2a 
  flume-ng-node/src/main/java/org/apache/flume/conf/properties/FlumeConfiguration.java d66f6d1 
  flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 50b9f0c 
  pom.xml c91222f 

Diff: https://reviews.apache.org/r/4502/diff


Testing
-------

Functional testing done.


Thanks,

Hari


                
> Core configuration component
> ----------------------------
>
>                 Key: FLUME-1052
>                 URL: https://issues.apache.org/jira/browse/FLUME-1052
>             Project: Flume
>          Issue Type: Sub-task
>    Affects Versions: v1.0.0
>            Reporter: Hari Shreedharan
>            Assignee: Hari Shreedharan
>             Fix For: v1.2.0
>
>
> Currently the configuration provider implementation encompasses all the syntactic and structural validation rules for loading the configuration. Externalizing this functionality to a library will allow external tools to easily operate on flume configuration files and be able to help parse and validate these files.
> Currently the configuration of each component sits deep inside the component themselves. There is no way to verify if a configuration is valid before run time. In most systems like Flume, it is likely that these confs will be deployed from one single host, on to the machines where flume agents are running. Only when each agent starts, invalid confs are identified because the Agents would terminate by throwing exceptions. This is a first attempt to make a component-aware configuration system which is independent of the Flume, and does not require the Flume jar to be installed. Each component needs a configuration manager, which configures the components. 
> Provide abstract Configuration stubs for each component type, sources, channels, sinks, selectors etc, which are in the new package, independent on ng-core. Now for each of the channels extend the abstract class and check the config properties for each of the required parameters for that component, for example: MultiplexingChannelSelectorConfigurator would look for default channel etc. If a particular component does not have a configuration class then let the current mechanism continue. 
> This will require implementation for each component, but it should not be too complex. One additional advantage we get from this is that we can separate out the config validation from the components into these stubs, but we will still need to read the values out of the Context as we do currently(else we end up making the configuration dependent on flume-ng-core itself which we wanted to avoid). 
> A problem with this is making a change to the configuration would require changes in the configuration classes and in the components also(where the configuration is read and the component is actually configured). I could not figure out a way of avoiding this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira