You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Brock Noland <br...@cloudera.com> on 2012/11/20 13:30:37 UTC
Re: Review Request: FLUME-1630: Flume configuration code could be improved

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7518/#review13622
-----------------------------------------------------------



flume-ng-node/src/test/java/org/apache/flume/node/TestAbstractConfigurationProvider.java
<https://reviews.apache.org/r/7518/#comment29203>

    whitespace



flume-ng-node/src/test/java/org/apache/flume/node/TestAbstractConfigurationProvider.java
<https://reviews.apache.org/r/7518/#comment29202>

    missing l on channel


- Brock Noland


On Oct. 11, 2012, 6:58 p.m., Brock Noland wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7518/
> -----------------------------------------------------------
> 
> (Updated Oct. 11, 2012, 6:58 p.m.)
> 
> 
> Review request for Flume.
> 
> 
> Description
> -------
> 
> Patch is not for commit, yet. It changes the configuration system into something extendable and maintainable. Additionally it changes the terminology from node to agent.  Once the patch is ready for review we should change the node package to agent to conform to the agent terminology.
> 
> Big ticket items:
> 
> 1) Abstract property file provider is changed to Abstract property provider. Two concrete implementations are provided, PropertyFileConfigurationProvider and PollingPropertyFileConfigurationProvider. There is an additional concrete implementation MemoryConfigurationProvider is in TestAbstractConfigurationProvider.
> 
> 2) Caching instances is removed from all factories. Instance caching is implemented in AbstractConfigurationProvider for channels *if* they have the Reusable annotation. MemoryChannel has this annotation.
> 
> 3) A layer of supervisors is removed. The application class now starts and stops the components when handleConfigurationEvent is called. This is called on startup if PropertyFileConfigurationProvider is used or whenever the configuration file changes if PollingPropertyFileConfigurationProvider is used. PollingPropertyFileConfigurationProvider uses EventBus (guava) to trigger the re-configuration.
> 
> 
> This addresses bug FLUME-1630.
>     https://issues.apache.org/jira/browse/FLUME-1630
> 
> 
> Diffs
> -----
> 
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java 6680a2c 
>   flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/JdbcChannel.java bca0c50 
>   flume-ng-channels/flume-recoverable-memory-channel/src/main/java/org/apache/flume/channel/recoverable/memory/RecoverableMemoryChannel.java 49e7cfd 
>   flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfiguration.java 9b209e8 
>   flume-ng-core/src/main/java/org/apache/flume/ChannelFactory.java e269909 
>   flume-ng-core/src/main/java/org/apache/flume/Constants.java 4c6992d 
>   flume-ng-core/src/main/java/org/apache/flume/SinkFactory.java d8fd0da 
>   flume-ng-core/src/main/java/org/apache/flume/SourceFactory.java 91cc866 
>   flume-ng-core/src/main/java/org/apache/flume/channel/DefaultChannelFactory.java ee32696 
>   flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java fc3a1e2 
>   flume-ng-core/src/main/java/org/apache/flume/sink/DefaultSinkFactory.java e71d44e 
>   flume-ng-core/src/main/java/org/apache/flume/source/DefaultSourceFactory.java 18533ae 
>   flume-ng-core/src/test/java/org/apache/flume/sink/TestDefaultSinkFactory.java ab3f447 
>   flume-ng-core/src/test/java/org/apache/flume/source/TestDefaultSourceFactory.java ff5b4d6 
>   flume-ng-node/pom.xml 5464bd3 
>   flume-ng-node/src/main/java/org/apache/flume/conf/file/AbstractFileConfigurationProvider.java a2c882b 
>   flume-ng-node/src/main/java/org/apache/flume/conf/file/SimpleNodeConfiguration.java 99b8bcc 
>   flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 8dbbe57 
>   flume-ng-node/src/main/java/org/apache/flume/node/AbstractConfigurationProvider.java PRE-CREATION 
>   flume-ng-node/src/main/java/org/apache/flume/node/Application.java 405afa3 
>   flume-ng-node/src/main/java/org/apache/flume/node/ConfigurationProvider.java ae732aa 
>   flume-ng-node/src/main/java/org/apache/flume/node/FlumeNode.java cee022d 
>   flume-ng-node/src/main/java/org/apache/flume/node/MaterializedConfiguration.java PRE-CREATION 
>   flume-ng-node/src/main/java/org/apache/flume/node/NodeConfiguration.java a24c939 
>   flume-ng-node/src/main/java/org/apache/flume/node/NodeManager.java 7457d87 
>   flume-ng-node/src/main/java/org/apache/flume/node/PollingPropertiesFileConfigurationProvider.java PRE-CREATION 
>   flume-ng-node/src/main/java/org/apache/flume/node/PropertiesFileConfigurationProvider.java PRE-CREATION 
>   flume-ng-node/src/main/java/org/apache/flume/node/SimpleMaterializedConfiguration.java PRE-CREATION 
>   flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/AbstractLogicalNodeManager.java 1fda07b 
>   flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java 60bfd5e 
>   flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/NodeConfigurationAware.java c20bf9b 
>   flume-ng-node/src/test/java/org/apache/flume/conf/properties/TestPropertiesFileConfigurationProvider.java d43aed6 
>   flume-ng-node/src/test/java/org/apache/flume/node/TestAbstractConfigurationProvider.java PRE-CREATION 
>   flume-ng-node/src/test/java/org/apache/flume/node/TestAbstractLogicalNodeManager.java 1cbc269 
>   flume-ng-node/src/test/java/org/apache/flume/node/TestApplication.java PRE-CREATION 
>   flume-ng-node/src/test/java/org/apache/flume/node/TestDefaultLogicalNodeManager.java 530b299 
>   flume-ng-node/src/test/java/org/apache/flume/node/TestFlumeNode.java f2dad6f 
>   flume-ng-node/src/test/java/org/apache/flume/node/TestFlumeNodeApplication.java f759af1 
>   flume-ng-node/src/test/java/org/apache/flume/node/TestPollingPropertiesFileConfigurationProvider.java PRE-CREATION 
>   flume-ng-node/src/test/java/org/apache/flume/node/TestPropertiesFileConfigurationProvider.java PRE-CREATION 
>   flume-ng-node/src/test/java/org/apache/flume/source/FlakeySequenceGeneratorSource.java 41e2f35 
>   flume-ng-node/src/test/java/org/apache/flume/source/TestNetcatSource.java 3c17d3d 
>   flume-ng-node/src/test/resources/flume-conf.properties 2b74d4c 
> 
> Diff: https://reviews.apache.org/r/7518/diff/
> 
> 
> Testing
> -------
> 
> Unit tests added for the new functionality.
> 
> 
> Thanks,
> 
> Brock Noland
> 
>