You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Ralph Goers (JIRA)" <ji...@apache.org> on 2012/08/22 09:03:38 UTC
[jira] [Commented] (FLUME-1502) Support for running simple
configurations embedded in host process
[ https://issues.apache.org/jira/browse/FLUME-1502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439320#comment-13439320 ]
Ralph Goers commented on FLUME-1502:
------------------------------------
I recently completed embedding Flume into Log4j 2 and can provide some of the pain points.
1. Log4j 2 uses XML or JSON while Flume cannot work without Properties. The solution I came up with was to use dynamically constructed properties.
2. Flume's configuration cannot be separate from Log4j. If Flume were to detect a change in its configuration it would shutdown everything and restart. This would cause the application interacting with the source to fail. I solved this by requiring that Flume be configured as
part of Log4j. If Log4j is reconfigured in such a way that the agent configuration changes Log4j will start a new agent, connect that with the new FlumeAppender instance and then cause Loggers to use the new configuration once it is fully instantiated. The previous agent will then be shut down.
3. I ended up replacing ConfigurationProvider with FlumeConfigurationBuilder. It exposes a load method that accepts the agent name, the Properties, and the NodeManager and returns a NodeConfiguration.
The source for this is at https://svn.apache.org/repos/asf/logging/log4j/log4j2/trunk/flume-ng/src/main/java/org/apache/logging/log4j/flume/appender/. In particular, FlumeEmbeddedManager.java and FlumeConfigurationBuilder.java do most of the work.
> Support for running simple configurations embedded in host process
> ------------------------------------------------------------------
>
> Key: FLUME-1502
> URL: https://issues.apache.org/jira/browse/FLUME-1502
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.2.0
> Reporter: Arvind Prabhakar
>
> Flume should provide a light-weight embeddable node manager that can be started in process where necessary. This will allow the users to embed light-weight agents within the host process where necessary.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira