You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Hari Shreedharan (JIRA)" <ji...@apache.org> on 2012/07/24 03:50:33 UTC
[jira] [Created] (FLUME-1392) Inactive channels get added to source
channels list causing NPE
Hari Shreedharan created FLUME-1392:
---------------------------------------
Summary: Inactive channels get added to source channels list causing NPE
Key: FLUME-1392
URL: https://issues.apache.org/jira/browse/FLUME-1392
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: v1.2.0
Reporter: Hari Shreedharan
Fix For: v1.3.0
Attachments: FLUME-1392.patch
Channels are not created, hence the references are null, causing NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1392) Inactive channels get added to
source channels list causing NPE
Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424013#comment-13424013 ]
Hari Shreedharan commented on FLUME-1392:
-----------------------------------------
Thanks Will.
Could we move these formatting issues to another jira. I have been planning to clean up the output from the configuration system for quite a while. I filed FLUME-1399 for this.
> Inactive channels get added to source channels list causing NPE
> ---------------------------------------------------------------
>
> Key: FLUME-1392
> URL: https://issues.apache.org/jira/browse/FLUME-1392
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1392-2.patch, FLUME-1392.patch
>
>
> Channels are not created, hence the references are null, causing NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1392) Inactive channels get added to
source channels list causing NPE
Posted by "Will McQueen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424241#comment-13424241 ]
Will McQueen commented on FLUME-1392:
-------------------------------------
+1 Ship it!
Sounds good, thanks Hari.
> Inactive channels get added to source channels list causing NPE
> ---------------------------------------------------------------
>
> Key: FLUME-1392
> URL: https://issues.apache.org/jira/browse/FLUME-1392
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1392-2.patch, FLUME-1392.patch
>
>
> Channels are not created, hence the references are null, causing NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1392) Inactive channels get added to source
channels list causing NPE
Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Shreedharan updated FLUME-1392:
------------------------------------
Attachment: FLUME-1392-2.patch
> Inactive channels get added to source channels list causing NPE
> ---------------------------------------------------------------
>
> Key: FLUME-1392
> URL: https://issues.apache.org/jira/browse/FLUME-1392
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1392-2.patch, FLUME-1392.patch
>
>
> Channels are not created, hence the references are null, causing NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1392) Inactive channels get added to
source channels list causing NPE
Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421626#comment-13421626 ]
Hari Shreedharan commented on FLUME-1392:
-----------------------------------------
Fixed both issues.
> Inactive channels get added to source channels list causing NPE
> ---------------------------------------------------------------
>
> Key: FLUME-1392
> URL: https://issues.apache.org/jira/browse/FLUME-1392
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1392-2.patch, FLUME-1392.patch
>
>
> Channels are not created, hence the references are null, causing NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1392) Inactive channels get added to source
channels list causing NPE
Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Shreedharan updated FLUME-1392:
------------------------------------
Attachment: FLUME-1392.patch
> Inactive channels get added to source channels list causing NPE
> ---------------------------------------------------------------
>
> Key: FLUME-1392
> URL: https://issues.apache.org/jira/browse/FLUME-1392
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1392.patch
>
>
> Channels are not created, hence the references are null, causing NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (FLUME-1392) Inactive channels get added to source
channels list causing NPE
Posted by "Will McQueen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Will McQueen closed FLUME-1392.
-------------------------------
> Inactive channels get added to source channels list causing NPE
> ---------------------------------------------------------------
>
> Key: FLUME-1392
> URL: https://issues.apache.org/jira/browse/FLUME-1392
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1392-2.patch, FLUME-1392.patch
>
>
> Channels are not created, hence the references are null, causing NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1392) Inactive channels get added to
source channels list causing NPE
Posted by "Will McQueen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424008#comment-13424008 ]
Will McQueen commented on FLUME-1392:
-------------------------------------
Fix confirmed. Just 3 niggles related to output:
#1:
("No Channels configured for " + sourceName)
=>
("No channels configured for source '" + sourceName +'" + ".")
#2:
("Channel " + sinkConf.getChannel() + " not in active set.")
=>
("Channel '" + sinkConf.getChannel() + "' not in active set.")
#3:
("Configuration for : " + sinkName + " has errors, and will be removed: ", e)
=>
("Configuration for sink '" + sinkName + "' has errors, and will be removed:", e)
Thank you.
Test # 1 results:
================
2012-07-27 09:47:25,384 (conf-file-poller-0) [WARN - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSources(FlumeConfiguration.java:571)] Removed r1 due to No Channels configured for r1
Test #2 results:
===============
2012-07-27 09:42:44,364 (conf-file-poller-0) [WARN - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:678)] Configuration for : k1 has errors, and will be removed:
org.apache.flume.conf.ConfigurationException: Channel c2 not in active set.
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:665)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:328)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.access$000(FlumeConfiguration.java:194)
at org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:108)
at org.apache.flume.conf.FlumeConfiguration.<init>(FlumeConfiguration.java:90)
at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:215)
at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
> Inactive channels get added to source channels list causing NPE
> ---------------------------------------------------------------
>
> Key: FLUME-1392
> URL: https://issues.apache.org/jira/browse/FLUME-1392
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1392-2.patch, FLUME-1392.patch
>
>
> Channels are not created, hence the references are null, causing NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1392) Inactive channels get added to
source channels list causing NPE
Posted by "Will McQueen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421159#comment-13421159 ]
Will McQueen commented on FLUME-1392:
-------------------------------------
Test 1
======
Tested with this config file:
agent.channels = c1
agent.sources = r1
agent.sinks = k1
agent.channels.c1.type = MEMORY
agent.channels.c2.type = MEMORY
agent.sources.r1.channels = c2
agent.sources.r1.type = NETCAT
agent.sources.r1.bind = 0.0.0.0
agent.sources.r1.port = 41414
agent.sinks.k1.channel = c1
agent.sinks.k1.type = LOGGER
...and got this:
[will@will-laptop apache-flume-1.3.0-SNAPSHOT]$ bin/flume-ng agent -c conf -f conf/flume.conf -n agent
+ exec /usr/java/latest/bin/java -Xmx20m -cp '/home/will/git/apache/apache-flume-ng/apache-flume-1.3.0-SNAPSHOT/conf:/home/will/git/apache/apache-flume-ng/apache-flume-1.3.0-SNAPSHOT/lib/*' -Djava.library.path= org.apache.flume.node.Application -f conf/flume.conf -n agent
2012-07-23 21:10:14,544 (main) [INFO - org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)] Starting lifecycle supervisor 1
2012-07-23 21:10:14,546 (main) [INFO - org.apache.flume.node.FlumeNode.start(FlumeNode.java:54)] Flume node starting - agent
2012-07-23 21:10:14,548 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:203)] Node manager starting
2012-07-23 21:10:14,549 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:67)] Configuration provider starting
2012-07-23 21:10:14,551 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)] Starting lifecycle supervisor 10
2012-07-23 21:10:14,552 (lifecycleSupervisor-1-0) [DEBUG - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:207)] Node manager started
2012-07-23 21:10:14,553 (lifecycleSupervisor-1-1) [DEBUG - org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:86)] Configuration provider started
2012-07-23 21:10:14,553 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)] Checking file:conf/flume.conf for changes
2012-07-23 21:10:14,553 (conf-file-poller-0) [INFO - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:195)] Reloading configuration file:conf/flume.conf
2012-07-23 21:10:14,557 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:904)] Added sinks: k1 Agent: agent
2012-07-23 21:10:14,557 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:990)] Processing:k1
2012-07-23 21:10:14,557 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:994)] Created context for k1: type
2012-07-23 21:10:14,557 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:990)] Processing:k1
2012-07-23 21:10:14,557 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:295)] Starting validation of configuration for agent: agent, initial-configuration: AgentConfiguration[agent]
SOURCES: {r1={ parameters:{port=41414, channels=c2, type=NETCAT, bind=0.0.0.0} }}
CHANNELS: {c1={ parameters:{type=MEMORY} }, c2={ parameters:{type=MEMORY} }}
SINKS: {k1={ parameters:{type=LOGGER, channel=c1} }}
2012-07-23 21:10:14,562 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:450)] Created channel c1
2012-07-23 21:10:14,574 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:651)] Creating sink: k1 using LOGGER
2012-07-23 21:10:14,577 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:353)] Post validation configuration for agent
AgentConfiguration created without Configuration stubs for which only basic syntactical validation was performed[agent]
SOURCES: {r1={ parameters:{port=41414, channels=null, type=NETCAT, bind=0.0.0.0} }}
CHANNELS: {c1={ parameters:{type=MEMORY} }}
AgentConfiguration created with Configuration stubs for which full validation was performed[agent]
SINKS: {k1=ComponentConfiguration[k1]
CONFIG:
CHANNEL:c1
}
2012-07-23 21:10:14,577 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:117)] Channels:c1
2012-07-23 21:10:14,578 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:118)] Sinks k1
2012-07-23 21:10:14,578 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:119)] Sources r1
2012-07-23 21:10:14,578 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:122)] Post-validation flume configuration contains configuration for agents: [agent]
2012-07-23 21:10:14,578 (conf-file-poller-0) [INFO - org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadChannels(PropertiesFileConfigurationProvider.java:249)] Creating channels
2012-07-23 21:10:14,578 (conf-file-poller-0) [DEBUG - org.apache.flume.channel.DefaultChannelFactory.create(DefaultChannelFactory.java:68)] Creating instance of channel c1 type MEMORY
2012-07-23 21:10:14,616 (conf-file-poller-0) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.<init>(MonitoredCounterGroup.java:68)] Monitoried counter group for type: CHANNEL, name: c1, registered successfully.
2012-07-23 21:10:14,616 (conf-file-poller-0) [INFO - org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadChannels(PropertiesFileConfigurationProvider.java:273)] created channel c1
2012-07-23 21:10:14,616 (conf-file-poller-0) [DEBUG - org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:74)] Creating instance of source r1, type NETCAT
2012-07-23 21:10:14,619 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:204)] Failed to load configuration data. Exception follows.
java.lang.NullPointerException: null value
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
at com.google.common.collect.ImmutableMap.entryOf(ImmutableMap.java:139)
at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:263)
at org.apache.flume.Context.getParameters(Context.java:53)
at org.apache.flume.conf.Configurables.ensureRequiredNonNull(Configurables.java:58)
at org.apache.flume.source.NetcatSource.configure(NetcatSource.java:135)
at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:323)
at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
The config failed, as desired. But 1 nit... could you please include the instance name in the error message (eg, c2 in this case) because seeing something like:
java.lang.NullPointerException: null value for c2
...would be more helpful than just:
java.lang.NullPointerException: null value
> Inactive channels get added to source channels list causing NPE
> ---------------------------------------------------------------
>
> Key: FLUME-1392
> URL: https://issues.apache.org/jira/browse/FLUME-1392
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1392.patch
>
>
> Channels are not created, hence the references are null, causing NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1392) Inactive channels get added to
source channels list causing NPE
Posted by "Will McQueen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421161#comment-13421161 ]
Will McQueen commented on FLUME-1392:
-------------------------------------
Test 2
======
Tested with this config file:
agent.channels = c1
agent.sources = r1
agent.sinks = k1
agent.channels.c1.type = MEMORY
agent.channels.c2.type = MEMORY
agent.sources.r1.channels = c1
agent.sources.r1.type = NETCAT
agent.sources.r1.bind = 0.0.0.0
agent.sources.r1.port = 41414
agent.sinks.k1.channel = c2
agent.sinks.k1.type = LOGGER
...and got this:
[will@will-laptop apache-flume-1.3.0-SNAPSHOT]$ bin/flume-ng agent -c conf -f conf/flume.conf -n agent
+ exec /usr/java/latest/bin/java -Xmx20m -cp '/home/will/git/apache/apache-flume-ng/apache-flume-1.3.0-SNAPSHOT/conf:/home/will/git/apache/apache-flume-ng/apache-flume-1.3.0-SNAPSHOT/lib/*' -Djava.library.path= org.apache.flume.node.Application -f conf/flume.conf -n agent
2012-07-23 21:15:05,370 (main) [INFO - org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)] Starting lifecycle supervisor 1
2012-07-23 21:15:05,372 (main) [INFO - org.apache.flume.node.FlumeNode.start(FlumeNode.java:54)] Flume node starting - agent
2012-07-23 21:15:05,374 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:203)] Node manager starting
2012-07-23 21:15:05,374 (lifecycleSupervisor-1-2) [INFO - org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:67)] Configuration provider starting
2012-07-23 21:15:05,377 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.lifecycle.LifecycleSupervisor.start(LifecycleSupervisor.java:67)] Starting lifecycle supervisor 10
2012-07-23 21:15:05,377 (lifecycleSupervisor-1-0) [DEBUG - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.start(DefaultLogicalNodeManager.java:207)] Node manager started
2012-07-23 21:15:05,378 (lifecycleSupervisor-1-2) [DEBUG - org.apache.flume.conf.file.AbstractFileConfigurationProvider.start(AbstractFileConfigurationProvider.java:86)] Configuration provider started
2012-07-23 21:15:05,378 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)] Checking file:conf/flume.conf for changes
2012-07-23 21:15:05,378 (conf-file-poller-0) [INFO - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:195)] Reloading configuration file:conf/flume.conf
2012-07-23 21:15:05,382 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:904)] Added sinks: k1 Agent: agent
2012-07-23 21:15:05,382 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:990)] Processing:k1
2012-07-23 21:15:05,382 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:994)] Created context for k1: type
2012-07-23 21:15:05,382 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:990)] Processing:k1
2012-07-23 21:15:05,383 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:295)] Starting validation of configuration for agent: agent, initial-configuration: AgentConfiguration[agent]
SOURCES: {r1={ parameters:{port=41414, channels=c1, type=NETCAT, bind=0.0.0.0} }}
CHANNELS: {c1={ parameters:{type=MEMORY} }, c2={ parameters:{type=MEMORY} }}
SINKS: {k1={ parameters:{type=LOGGER, channel=c2} }}
2012-07-23 21:15:05,387 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateChannels(FlumeConfiguration.java:450)] Created channel c1
2012-07-23 21:15:05,402 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:651)] Creating sink: k1 using LOGGER
2012-07-23 21:15:05,405 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:353)] Post validation configuration for agent
AgentConfiguration created without Configuration stubs for which only basic syntactical validation was performed[agent]
SOURCES: {r1={ parameters:{port=41414, channels=c1, type=NETCAT, bind=0.0.0.0} }}
CHANNELS: {c1={ parameters:{type=MEMORY} }}
AgentConfiguration created with Configuration stubs for which full validation was performed[agent]
SINKS: {k1=ComponentConfiguration[k1]
CONFIG:
CHANNEL:c2
}
2012-07-23 21:15:05,406 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:117)] Channels:c1
2012-07-23 21:15:05,406 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:118)] Sinks k1
2012-07-23 21:15:05,406 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:119)] Sources r1
2012-07-23 21:15:05,406 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:122)] Post-validation flume configuration contains configuration for agents: [agent]
2012-07-23 21:15:05,406 (conf-file-poller-0) [INFO - org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadChannels(PropertiesFileConfigurationProvider.java:249)] Creating channels
2012-07-23 21:15:05,406 (conf-file-poller-0) [DEBUG - org.apache.flume.channel.DefaultChannelFactory.create(DefaultChannelFactory.java:68)] Creating instance of channel c1 type MEMORY
2012-07-23 21:15:05,443 (conf-file-poller-0) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.<init>(MonitoredCounterGroup.java:68)] Monitoried counter group for type: CHANNEL, name: c1, registered successfully.
2012-07-23 21:15:05,443 (conf-file-poller-0) [INFO - org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadChannels(PropertiesFileConfigurationProvider.java:273)] created channel c1
2012-07-23 21:15:05,443 (conf-file-poller-0) [DEBUG - org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:74)] Creating instance of source r1, type NETCAT
2012-07-23 21:15:05,456 (conf-file-poller-0) [INFO - org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:70)] Creating instance of sink: k1, type: LOGGER
2012-07-23 21:15:05,458 (conf-file-poller-0) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:106)] Starting new configuration:{ sourceRunners:{r1=EventDrivenSourceRunner: { source:org.apache.flume.source.NetcatSource@5d5bdc50 }} sinkRunners:{k1=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@39dd3812 counterGroup:{ name:null counters:{} } }} channels:{c1=org.apache.flume.channel.MemoryChannel@6a8c436b} }
2012-07-23 21:15:05,458 (conf-file-poller-0) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:113)] Starting Channel c1
2012-07-23 21:15:05,459 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:82)] Component type: CHANNEL, name: c1 started
2012-07-23 21:15:05,459 (conf-file-poller-0) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:141)] Starting Sink k1
2012-07-23 21:15:05,459 (conf-file-poller-0) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:152)] Starting Source r1
2012-07-23 21:15:05,460 (lifecycleSupervisor-1-3) [INFO - org.apache.flume.source.NetcatSource.start(NetcatSource.java:147)] Source starting
2012-07-23 21:15:05,460 (lifecycleSupervisor-1-0) [ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:239)] Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@39dd3812 counterGroup:{ name:null counters:{} } } - Exception follows.
java.lang.IllegalStateException: No channel configured
at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.sink.AbstractSink.start(AbstractSink.java:42)
at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:237)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2012-07-23 21:15:05,468 (lifecycleSupervisor-1-3) [INFO - org.apache.flume.source.NetcatSource.start(NetcatSource.java:161)] Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/0:0:0:0:0:0:0:0:41414]
2012-07-23 21:15:05,468 (lifecycleSupervisor-1-3) [DEBUG - org.apache.flume.source.NetcatSource.start(NetcatSource.java:180)] Source started
2012-07-23 21:15:05,468 (Thread-1) [DEBUG - org.apache.flume.source.NetcatSource$AcceptHandler.run(NetcatSource.java:258)] Starting accept handler
2012-07-23 21:15:08,463 (lifecycleSupervisor-1-1) [ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:239)] Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@39dd3812 counterGroup:{ name:null counters:{} } } - Exception follows.
java.lang.IllegalStateException: No channel configured
at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.sink.AbstractSink.start(AbstractSink.java:42)
at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:237)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2012-07-23 21:15:11,464 (lifecycleSupervisor-1-3) [ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:239)] Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@39dd3812 counterGroup:{ name:null counters:{} } } - Exception follows.
java.lang.IllegalStateException: No channel configured
at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.sink.AbstractSink.start(AbstractSink.java:42)
at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:237)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
^C2012-07-23 21:15:12,742 (node-shutdownHook) [INFO - org.apache.flume.node.FlumeNode.stop(FlumeNode.java:67)] Flume node stopping - agent
2012-07-23 21:15:12,742 (node-shutdownHook) [INFO - org.apache.flume.lifecycle.LifecycleSupervisor.stop(LifecycleSupervisor.java:78)] Stopping lifecycle supervisor 9
2012-07-23 21:15:12,744 (node-shutdownHook) [INFO - org.apache.flume.conf.file.AbstractFileConfigurationProvider.stop(AbstractFileConfigurationProvider.java:91)] Configuration provider stopping
2012-07-23 21:15:12,744 (node-shutdownHook) [DEBUG - org.apache.flume.conf.file.AbstractFileConfigurationProvider.stop(AbstractFileConfigurationProvider.java:97)] Waiting for file watcher to terminate
2012-07-23 21:15:12,744 (node-shutdownHook) [DEBUG - org.apache.flume.conf.file.AbstractFileConfigurationProvider.stop(AbstractFileConfigurationProvider.java:107)] Configuration provider stopped
2012-07-23 21:15:12,744 (node-shutdownHook) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stop(DefaultLogicalNodeManager.java:215)] Node manager stopping
2012-07-23 21:15:12,745 (node-shutdownHook) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stopAllComponents(DefaultLogicalNodeManager.java:68)] Shutting down configuration: { sourceRunners:{r1=EventDrivenSourceRunner: { source:org.apache.flume.source.NetcatSource@5d5bdc50 }} sinkRunners:{k1=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@39dd3812 counterGroup:{ name:null counters:{} } }} channels:{c1=org.apache.flume.channel.MemoryChannel@6a8c436b} }
2012-07-23 21:15:12,745 (node-shutdownHook) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stopAllComponents(DefaultLogicalNodeManager.java:72)] Stopping Source r1
2012-07-23 21:15:12,745 (node-shutdownHook) [INFO - org.apache.flume.lifecycle.LifecycleSupervisor.unsupervise(LifecycleSupervisor.java:156)] Stopping component: EventDrivenSourceRunner: { source:org.apache.flume.source.NetcatSource@5d5bdc50 }
2012-07-23 21:15:12,745 (node-shutdownHook) [INFO - org.apache.flume.source.NetcatSource.stop(NetcatSource.java:186)] Source stopping
2012-07-23 21:15:12,745 (node-shutdownHook) [DEBUG - org.apache.flume.source.NetcatSource.stop(NetcatSource.java:191)] Stopping accept handler thread
2012-07-23 21:15:12,746 (node-shutdownHook) [DEBUG - org.apache.flume.source.NetcatSource.stop(NetcatSource.java:195)] Waiting for accept handler to finish
2012-07-23 21:15:12,746 (Thread-1) [DEBUG - org.apache.flume.source.NetcatSource$AcceptHandler.run(NetcatSource.java:281)] Accept handler exiting
2012-07-23 21:15:12,746 (node-shutdownHook) [DEBUG - org.apache.flume.source.NetcatSource.stop(NetcatSource.java:205)] Stopped accept handler thread
2012-07-23 21:15:12,746 (node-shutdownHook) [DEBUG - org.apache.flume.source.NetcatSource.stop(NetcatSource.java:220)] Waiting for handler service to stop
2012-07-23 21:15:12,747 (node-shutdownHook) [DEBUG - org.apache.flume.source.NetcatSource.stop(NetcatSource.java:235)] Handler service stopped
2012-07-23 21:15:12,747 (node-shutdownHook) [DEBUG - org.apache.flume.source.NetcatSource.stop(NetcatSource.java:238)] Source stopped. Event metrics:{ name:null counters:{open.attempts=1} }
2012-07-23 21:15:12,747 (node-shutdownHook) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stopAllComponents(DefaultLogicalNodeManager.java:82)] Stopping Sink k1
2012-07-23 21:15:12,747 (node-shutdownHook) [INFO - org.apache.flume.lifecycle.LifecycleSupervisor.unsupervise(LifecycleSupervisor.java:156)] Stopping component: SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@39dd3812 counterGroup:{ name:null counters:{} } }
2012-07-23 21:15:12,747 (node-shutdownHook) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stopAllComponents(DefaultLogicalNodeManager.java:92)] Stopping Channel c1
2012-07-23 21:15:12,747 (node-shutdownHook) [INFO - org.apache.flume.lifecycle.LifecycleSupervisor.unsupervise(LifecycleSupervisor.java:156)] Stopping component: org.apache.flume.channel.MemoryChannel@6a8c436b
2012-07-23 21:15:12,747 (node-shutdownHook) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.stop(MonitoredCounterGroup.java:87)] Component type: CHANNEL, name: c1 stopped
2012-07-23 21:15:12,747 (node-shutdownHook) [INFO - org.apache.flume.lifecycle.LifecycleSupervisor.stop(LifecycleSupervisor.java:78)] Stopping lifecycle supervisor 9
2012-07-23 21:15:12,748 (node-shutdownHook) [DEBUG - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stop(DefaultLogicalNodeManager.java:221)] Node manager stopped
....where IllegalStateException keeps getting displayed every 2 or 3 secs, so if you could please revise the patch to catch this case, that would be great. Thank you.
Cheers,
Will
> Inactive channels get added to source channels list causing NPE
> ---------------------------------------------------------------
>
> Key: FLUME-1392
> URL: https://issues.apache.org/jira/browse/FLUME-1392
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1392.patch
>
>
> Channels are not created, hence the references are null, causing NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira