You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by JR <ma...@gmail.com> on 2013/03/21 15:54:38 UTC
Channel not starting.
Hello,
Am a newbie, trying to set up acvro source, channel memory and hdfs
sink. Am encountering the following error: Not sure
what to do.
*Is there a way to 'search' the topics in the *
13/03/21 07:09:50 ERROR nodemanager.DefaultLogicalNodeManager: Error while
starting org.apache.flume.channel.MemoryChannel{name: ch1}
java.util.concurrent.RejectedExecutionException
at
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1779)
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:778)
at
java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:226)
at
java.util.concurrent.ScheduledThreadPoolExecutor.scheduleWithFixedDelay(ScheduledThreadPoolExecutor.java:454)
at
org.apache.flume.lifecycle.LifecycleSupervisor.supervise(LifecycleSupervisor.java:140)
at
org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:114)
at
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:225)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:452)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:328)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:161)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:109)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:191)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:215)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
at java.lang.Thread.run(Thread.java:738)
13/03/21 07:09:50 INFO nodemanager.DefaultLogicalNodeManager: Waiting for
channel: ch1 to start. Sleeping for 500 ms
13/03/21 07:09:50 INFO nodemanager.DefaultLogicalNodeManager: Waiting for
channel: ch1 to start. Sleeping for 500 ms
Re: Channel not starting.
Posted by Brock Noland <br...@cloudera.com>.
Reading the Sun JDK, that should only occur if the executor is shutting
down. Did you kill or ctrl-c the process? Could you send us the full log?
Brock
On Thu, Mar 21, 2013 at 9:07 AM, JR <ma...@gmail.com> wrote:
> Thanks for replying:
>
> OS: Linux SUSE 11.2
> Java(TM) SE Runtime Environment (build pxa6460sr12-20121025_01(SR12))
> J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64
> jvmxa6460sr12-20121024_126067 (JIT enabled, AOT enabled)
> J9VM - 20121024_126067
> JIT - r9_20120914_26057
> GC - 20120928_AA)
> JCL - 20121014_01
>
>
>
>
> #==============================================================================
> # Copied and Modified the config info in
> conf/flume-conf.properties.template
> # as below:
> #
> # =========================================================================
> agent1.sources = avroSource
> agent1.channels = ch1
> agent1.sinks = hdfsSink
> #
> agent1.sources.avroSource.type = avro
> agent1.sources.avroSource.bind = 0.0.0.0
> agent1.sources.avroSource.port = 41414
> agent1.sources.avroSource.channels = ch1
>
> # sink
> agent1.sinks.hdfsSink.type = hdfs
> agent1.sinks.hdfsSink.channel = ch1
> agent1.sinks.hdfsSink.hdfs.writeFormat = Text
> agent1.sinks.hdfsSink.hdfs.filePrefix = testing
> agent1.sinks.hdfsSink.hdfs.path = hdfs://
> hdtest111.svl.ibm.com:9000/flume/testingData # please change "
> hdtest111.svl.ibm.com" accordingly
>
> # channels
> agent1.channels.ch1.type = memory
> agent1.channels.ch1.capacity = 1000
> #
>
>
> On Thu, Mar 21, 2013 at 11:04 AM, Brock Noland <br...@cloudera.com> wrote:
>
>> Strange... Can you share 1) JVM 2) OS and 3) Flume config file?
>>
>>
>> On Thu, Mar 21, 2013 at 7:54 AM, JR <ma...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> Am a newbie, trying to set up acvro source, channel memory and hdfs
>>> sink. Am encountering the following error: Not sure
>>> what to do.
>>>
>>> *Is there a way to 'search' the topics in the *
>>>
>>> 13/03/21 07:09:50 ERROR nodemanager.DefaultLogicalNodeManager: Error
>>> while starting org.apache.flume.channel.MemoryChannel{name: ch1}
>>> java.util.concurrent.RejectedExecutionException
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1779)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:778)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:226)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor.scheduleWithFixedDelay(ScheduledThreadPoolExecutor.java:454)
>>> at
>>> org.apache.flume.lifecycle.LifecycleSupervisor.supervise(LifecycleSupervisor.java:140)
>>> at
>>> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:114)
>>> at
>>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:225)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:452)
>>> at
>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:328)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:161)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:109)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:191)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:215)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
>>> at java.lang.Thread.run(Thread.java:738)
>>> 13/03/21 07:09:50 INFO nodemanager.DefaultLogicalNodeManager: Waiting
>>> for channel: ch1 to start. Sleeping for 500 ms
>>> 13/03/21 07:09:50 INFO nodemanager.DefaultLogicalNodeManager: Waiting
>>> for channel: ch1 to start. Sleeping for 500 ms
>>>
>>
>>
>>
>> --
>> Apache MRUnit - Unit testing MapReduce -
>> http://incubator.apache.org/mrunit/
>>
>
>
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Re: Channel not starting.
Posted by JR <ma...@gmail.com>.
Thanks for replying:
OS: Linux SUSE 11.2
Java(TM) SE Runtime Environment (build pxa6460sr12-20121025_01(SR12))
J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64
jvmxa6460sr12-20121024_126067 (JIT enabled, AOT enabled)
J9VM - 20121024_126067
JIT - r9_20120914_26057
GC - 20120928_AA)
JCL - 20121014_01
#==============================================================================
# Copied and Modified the config info in
conf/flume-conf.properties.template
# as below:
#
# =========================================================================
agent1.sources = avroSource
agent1.channels = ch1
agent1.sinks = hdfsSink
#
agent1.sources.avroSource.type = avro
agent1.sources.avroSource.bind = 0.0.0.0
agent1.sources.avroSource.port = 41414
agent1.sources.avroSource.channels = ch1
# sink
agent1.sinks.hdfsSink.type = hdfs
agent1.sinks.hdfsSink.channel = ch1
agent1.sinks.hdfsSink.hdfs.writeFormat = Text
agent1.sinks.hdfsSink.hdfs.filePrefix = testing
agent1.sinks.hdfsSink.hdfs.path = hdfs://
hdtest111.svl.ibm.com:9000/flume/testingData # please change "
hdtest111.svl.ibm.com" accordingly
# channels
agent1.channels.ch1.type = memory
agent1.channels.ch1.capacity = 1000
#
On Thu, Mar 21, 2013 at 11:04 AM, Brock Noland <br...@cloudera.com> wrote:
> Strange... Can you share 1) JVM 2) OS and 3) Flume config file?
>
>
> On Thu, Mar 21, 2013 at 7:54 AM, JR <ma...@gmail.com> wrote:
>
>> Hello,
>>
>> Am a newbie, trying to set up acvro source, channel memory and hdfs
>> sink. Am encountering the following error: Not sure
>> what to do.
>>
>> *Is there a way to 'search' the topics in the *
>>
>> 13/03/21 07:09:50 ERROR nodemanager.DefaultLogicalNodeManager: Error
>> while starting org.apache.flume.channel.MemoryChannel{name: ch1}
>> java.util.concurrent.RejectedExecutionException
>> at
>> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1779)
>> at
>> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:778)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:226)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor.scheduleWithFixedDelay(ScheduledThreadPoolExecutor.java:454)
>> at
>> org.apache.flume.lifecycle.LifecycleSupervisor.supervise(LifecycleSupervisor.java:140)
>> at
>> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:114)
>> at
>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:225)
>> at
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
>> at
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
>> at
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:452)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:328)
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:161)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:109)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:191)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:215)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
>> at java.lang.Thread.run(Thread.java:738)
>> 13/03/21 07:09:50 INFO nodemanager.DefaultLogicalNodeManager: Waiting for
>> channel: ch1 to start. Sleeping for 500 ms
>> 13/03/21 07:09:50 INFO nodemanager.DefaultLogicalNodeManager: Waiting for
>> channel: ch1 to start. Sleeping for 500 ms
>>
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/
>
Re: Channel not starting.
Posted by Brock Noland <br...@cloudera.com>.
Strange... Can you share 1) JVM 2) OS and 3) Flume config file?
On Thu, Mar 21, 2013 at 7:54 AM, JR <ma...@gmail.com> wrote:
> Hello,
>
> Am a newbie, trying to set up acvro source, channel memory and hdfs
> sink. Am encountering the following error: Not sure
> what to do.
>
> *Is there a way to 'search' the topics in the *
>
> 13/03/21 07:09:50 ERROR nodemanager.DefaultLogicalNodeManager: Error while
> starting org.apache.flume.channel.MemoryChannel{name: ch1}
> java.util.concurrent.RejectedExecutionException
> at
> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1779)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:778)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:226)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.scheduleWithFixedDelay(ScheduledThreadPoolExecutor.java:454)
> at
> org.apache.flume.lifecycle.LifecycleSupervisor.supervise(LifecycleSupervisor.java:140)
> at
> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:114)
> at
> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:225)
> at
> org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
> at
> org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
> at
> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:452)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:328)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:161)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:109)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:191)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:215)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
> at java.lang.Thread.run(Thread.java:738)
> 13/03/21 07:09:50 INFO nodemanager.DefaultLogicalNodeManager: Waiting for
> channel: ch1 to start. Sleeping for 500 ms
> 13/03/21 07:09:50 INFO nodemanager.DefaultLogicalNodeManager: Waiting for
> channel: ch1 to start. Sleeping for 500 ms
>
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Re: Channel not starting.
Posted by Zhiwen Sun <pe...@gmail.com>.
Plz input your configuration file.
On Thursday, March 21, 2013, JR wrote:
> Hello,
>
> Am a newbie, trying to set up acvro source, channel memory and hdfs
> sink. Am encountering the following error: Not sure
> what to do.
>
> *Is there a way to 'search' the topics in the *
>
> 13/03/21 07:09:50 ERROR nodemanager.DefaultLogicalNodeManager: Error while
> starting org.apache.flume.channel.MemoryChannel{name: ch1}
> java.util.concurrent.RejectedExecutionException
> at
> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1779)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:778)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:226)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.scheduleWithFixedDelay(ScheduledThreadPoolExecutor.java:454)
> at
> org.apache.flume.lifecycle.LifecycleSupervisor.supervise(LifecycleSupervisor.java:140)
> at
> org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:114)
> at
> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:225)
> at
> org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
> at
> org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
> at
> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:452)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:328)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:161)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:109)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:191)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:215)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
> at java.lang.Thread.run(Thread.java:738)
> 13/03/21 07:09:50 INFO nodemanager.DefaultLogicalNodeManager: Waiting for
> channel: ch1 to start. Sleeping for 500 ms
> 13/03/21 07:09:50 INFO nodemanager.DefaultLogicalNodeManager: Waiting for
> channel: ch1 to start. Sleeping for 500 ms
>
--
Zhiwen Sun