You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Darshan Pandya <dp...@zaloni.com> on 2014/12/19 22:02:36 UTC

Flume IBM MQ - JMS Source

Hi Folks,
I am new to flume.
I wanted to check if anyone has connected an IBM MQ to the JMS Source in
Flume.
I quickly configured flume with an agent similar to the following, but it
throws me a class not found exception as shown below. I'm sure I am doing
something stupid.



a1.sources = r1
a1.channels = c1
a1.sinks = k1

#Source Definition
a1.sources.r1.type = jms
a1.sources.r1.initialContextFactory =
com.ibm.mq.jms.context.WMQInitialContextFactory
a1.sources.r1.connectionFactory = com.ibm.mq.jms.MQQueueConnectionFactory
a1.sources.r1.providerURL = tcp://<ipAddress>:<port>
a1.sources.r1.destinationName = QA.TEST.NAME
a1.sources.r1.destinationType = Topic

# Describe the sink
a1.sinks.k1.type = logger

# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100

# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1




14/12/19 11:52:31 INFO node.PollingPropertiesFileConfigurationProvider:
Configuration provider starting

14/12/19 11:52:31 INFO node.PollingPropertiesFileConfigurationProvider:
Reloading configuration file:./flume-conf/flume-test.conf

14/12/19 11:52:31 INFO conf.FlumeConfiguration: Added sinks: k1 Agent: a1

14/12/19 11:52:31 INFO conf.FlumeConfiguration: Processing:k1

14/12/19 11:52:31 INFO conf.FlumeConfiguration: Processing:k1

14/12/19 11:52:31 INFO conf.FlumeConfiguration: Post-validation flume
configuration contains configuration for agents: [a1]

14/12/19 11:52:31 INFO node.AbstractConfigurationProvider: Creating channels

14/12/19 11:52:31 INFO channel.DefaultChannelFactory: Creating instance of
channel c1 type memory

14/12/19 11:52:31 INFO node.AbstractConfigurationProvider: Created channel
c1

14/12/19 11:52:31 INFO source.DefaultSourceFactory: Creating instance of
source r1, type jms

14/12/19 11:52:31 ERROR node.PollingPropertiesFileConfigurationProvider:
Failed to start agent because dependencies were not found in classpath.
Error follows.

java.lang.NoClassDefFoundError: javax/jms/JMSException

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:190)

at
org.apache.flume.source.DefaultSourceFactory.getClass(DefaultSourceFactory.java:65)

at
org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:40)

at
org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:322)

at
org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:97)

at
org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.ClassNotFoundException: javax.jms.JMSException

at java.net.URLClassLoader$1.run(URLClassLoader.java:366)




-- 
Sincerely,
Darshan

Re: Flume IBM MQ - JMS Source

Posted by Ashish <pa...@gmail.com>.
Dependencies are still missing, need to add them to classpath too

On Tue, Dec 23, 2014 at 3:47 AM, Darshan Pandya <dp...@zaloni.com> wrote:
>
> Got beyond that point.
> Now I am trying to connect to IBM MQ
>
> Can anyone tell me what jars are needed to connect to IBM MQ.
> [image: Inline image 1]
>
> On Fri, Dec 19, 2014 at 4:21 PM, Jeff Lord <jl...@cloudera.com> wrote:
>
>> Do you have the jms class in your cp?
>> java.lang.NoClassDefFoundError: javax/jms/JMSException
>>
>> On Fri, Dec 19, 2014 at 1:02 PM, Darshan Pandya <dp...@zaloni.com>
>> wrote:
>>>
>>> Hi Folks,
>>> I am new to flume.
>>> I wanted to check if anyone has connected an IBM MQ to the JMS Source in
>>> Flume.
>>> I quickly configured flume with an agent similar to the following, but
>>> it throws me a class not found exception as shown below. I'm sure I am
>>> doing something stupid.
>>>
>>>
>>>
>>> a1.sources = r1
>>> a1.channels = c1
>>> a1.sinks = k1
>>>
>>> #Source Definition
>>> a1.sources.r1.type = jms
>>> a1.sources.r1.initialContextFactory =
>>> com.ibm.mq.jms.context.WMQInitialContextFactory
>>> a1.sources.r1.connectionFactory = com.ibm.mq.jms.MQQueueConnectionFactory
>>> a1.sources.r1.providerURL = tcp://<ipAddress>:<port>
>>> a1.sources.r1.destinationName = QA.TEST.NAME
>>> a1.sources.r1.destinationType = Topic
>>>
>>> # Describe the sink
>>> a1.sinks.k1.type = logger
>>>
>>> # Use a channel which buffers events in memory
>>> a1.channels.c1.type = memory
>>> a1.channels.c1.capacity = 1000
>>> a1.channels.c1.transactionCapacity = 100
>>>
>>> # Bind the source and sink to the channel
>>> a1.sources.r1.channels = c1
>>> a1.sinks.k1.channel = c1
>>>
>>>
>>>
>>>
>>> 14/12/19 11:52:31 INFO node.PollingPropertiesFileConfigurationProvider:
>>> Configuration provider starting
>>>
>>> 14/12/19 11:52:31 INFO node.PollingPropertiesFileConfigurationProvider:
>>> Reloading configuration file:./flume-conf/flume-test.conf
>>>
>>> 14/12/19 11:52:31 INFO conf.FlumeConfiguration: Added sinks: k1 Agent: a1
>>>
>>> 14/12/19 11:52:31 INFO conf.FlumeConfiguration: Processing:k1
>>>
>>> 14/12/19 11:52:31 INFO conf.FlumeConfiguration: Processing:k1
>>>
>>> 14/12/19 11:52:31 INFO conf.FlumeConfiguration: Post-validation flume
>>> configuration contains configuration for agents: [a1]
>>>
>>> 14/12/19 11:52:31 INFO node.AbstractConfigurationProvider: Creating
>>> channels
>>>
>>> 14/12/19 11:52:31 INFO channel.DefaultChannelFactory: Creating instance
>>> of channel c1 type memory
>>>
>>> 14/12/19 11:52:31 INFO node.AbstractConfigurationProvider: Created
>>> channel c1
>>>
>>> 14/12/19 11:52:31 INFO source.DefaultSourceFactory: Creating instance of
>>> source r1, type jms
>>>
>>> 14/12/19 11:52:31 ERROR node.PollingPropertiesFileConfigurationProvider:
>>> Failed to start agent because dependencies were not found in classpath.
>>> Error follows.
>>>
>>> java.lang.NoClassDefFoundError: javax/jms/JMSException
>>>
>>> at java.lang.Class.forName0(Native Method)
>>>
>>> at java.lang.Class.forName(Class.java:190)
>>>
>>> at
>>> org.apache.flume.source.DefaultSourceFactory.getClass(DefaultSourceFactory.java:65)
>>>
>>> at
>>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:40)
>>>
>>> at
>>> org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:322)
>>>
>>> at
>>> org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:97)
>>>
>>> at
>>> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
>>>
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>>>
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>>>
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>> Caused by: java.lang.ClassNotFoundException: javax.jms.JMSException
>>>
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>>>
>>>
>>>
>>>
>>> --
>>> Sincerely,
>>> Darshan
>>>
>>
>
>
> --
> Sincerely,
> Darshan
>


-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

Re: Flume IBM MQ - JMS Source

Posted by Darshan Pandya <dp...@zaloni.com>.
Got beyond that point.
Now I am trying to connect to IBM MQ

Can anyone tell me what jars are needed to connect to IBM MQ.
[image: Inline image 1]

On Fri, Dec 19, 2014 at 4:21 PM, Jeff Lord <jl...@cloudera.com> wrote:

> Do you have the jms class in your cp?
> java.lang.NoClassDefFoundError: javax/jms/JMSException
>
> On Fri, Dec 19, 2014 at 1:02 PM, Darshan Pandya <dp...@zaloni.com>
> wrote:
>>
>> Hi Folks,
>> I am new to flume.
>> I wanted to check if anyone has connected an IBM MQ to the JMS Source in
>> Flume.
>> I quickly configured flume with an agent similar to the following, but it
>> throws me a class not found exception as shown below. I'm sure I am doing
>> something stupid.
>>
>>
>>
>> a1.sources = r1
>> a1.channels = c1
>> a1.sinks = k1
>>
>> #Source Definition
>> a1.sources.r1.type = jms
>> a1.sources.r1.initialContextFactory =
>> com.ibm.mq.jms.context.WMQInitialContextFactory
>> a1.sources.r1.connectionFactory = com.ibm.mq.jms.MQQueueConnectionFactory
>> a1.sources.r1.providerURL = tcp://<ipAddress>:<port>
>> a1.sources.r1.destinationName = QA.TEST.NAME
>> a1.sources.r1.destinationType = Topic
>>
>> # Describe the sink
>> a1.sinks.k1.type = logger
>>
>> # Use a channel which buffers events in memory
>> a1.channels.c1.type = memory
>> a1.channels.c1.capacity = 1000
>> a1.channels.c1.transactionCapacity = 100
>>
>> # Bind the source and sink to the channel
>> a1.sources.r1.channels = c1
>> a1.sinks.k1.channel = c1
>>
>>
>>
>>
>> 14/12/19 11:52:31 INFO node.PollingPropertiesFileConfigurationProvider:
>> Configuration provider starting
>>
>> 14/12/19 11:52:31 INFO node.PollingPropertiesFileConfigurationProvider:
>> Reloading configuration file:./flume-conf/flume-test.conf
>>
>> 14/12/19 11:52:31 INFO conf.FlumeConfiguration: Added sinks: k1 Agent: a1
>>
>> 14/12/19 11:52:31 INFO conf.FlumeConfiguration: Processing:k1
>>
>> 14/12/19 11:52:31 INFO conf.FlumeConfiguration: Processing:k1
>>
>> 14/12/19 11:52:31 INFO conf.FlumeConfiguration: Post-validation flume
>> configuration contains configuration for agents: [a1]
>>
>> 14/12/19 11:52:31 INFO node.AbstractConfigurationProvider: Creating
>> channels
>>
>> 14/12/19 11:52:31 INFO channel.DefaultChannelFactory: Creating instance
>> of channel c1 type memory
>>
>> 14/12/19 11:52:31 INFO node.AbstractConfigurationProvider: Created
>> channel c1
>>
>> 14/12/19 11:52:31 INFO source.DefaultSourceFactory: Creating instance of
>> source r1, type jms
>>
>> 14/12/19 11:52:31 ERROR node.PollingPropertiesFileConfigurationProvider:
>> Failed to start agent because dependencies were not found in classpath.
>> Error follows.
>>
>> java.lang.NoClassDefFoundError: javax/jms/JMSException
>>
>> at java.lang.Class.forName0(Native Method)
>>
>> at java.lang.Class.forName(Class.java:190)
>>
>> at
>> org.apache.flume.source.DefaultSourceFactory.getClass(DefaultSourceFactory.java:65)
>>
>> at
>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:40)
>>
>> at
>> org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:322)
>>
>> at
>> org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:97)
>>
>> at
>> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
>>
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>>
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>>
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>
>> at java.lang.Thread.run(Thread.java:745)
>>
>> Caused by: java.lang.ClassNotFoundException: javax.jms.JMSException
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>>
>>
>>
>>
>> --
>> Sincerely,
>> Darshan
>>
>


-- 
Sincerely,
Darshan

Re: Flume IBM MQ - JMS Source

Posted by Jeff Lord <jl...@cloudera.com>.
Do you have the jms class in your cp?
java.lang.NoClassDefFoundError: javax/jms/JMSException

On Fri, Dec 19, 2014 at 1:02 PM, Darshan Pandya <dp...@zaloni.com> wrote:
>
> Hi Folks,
> I am new to flume.
> I wanted to check if anyone has connected an IBM MQ to the JMS Source in
> Flume.
> I quickly configured flume with an agent similar to the following, but it
> throws me a class not found exception as shown below. I'm sure I am doing
> something stupid.
>
>
>
> a1.sources = r1
> a1.channels = c1
> a1.sinks = k1
>
> #Source Definition
> a1.sources.r1.type = jms
> a1.sources.r1.initialContextFactory =
> com.ibm.mq.jms.context.WMQInitialContextFactory
> a1.sources.r1.connectionFactory = com.ibm.mq.jms.MQQueueConnectionFactory
> a1.sources.r1.providerURL = tcp://<ipAddress>:<port>
> a1.sources.r1.destinationName = QA.TEST.NAME
> a1.sources.r1.destinationType = Topic
>
> # Describe the sink
> a1.sinks.k1.type = logger
>
> # Use a channel which buffers events in memory
> a1.channels.c1.type = memory
> a1.channels.c1.capacity = 1000
> a1.channels.c1.transactionCapacity = 100
>
> # Bind the source and sink to the channel
> a1.sources.r1.channels = c1
> a1.sinks.k1.channel = c1
>
>
>
>
> 14/12/19 11:52:31 INFO node.PollingPropertiesFileConfigurationProvider:
> Configuration provider starting
>
> 14/12/19 11:52:31 INFO node.PollingPropertiesFileConfigurationProvider:
> Reloading configuration file:./flume-conf/flume-test.conf
>
> 14/12/19 11:52:31 INFO conf.FlumeConfiguration: Added sinks: k1 Agent: a1
>
> 14/12/19 11:52:31 INFO conf.FlumeConfiguration: Processing:k1
>
> 14/12/19 11:52:31 INFO conf.FlumeConfiguration: Processing:k1
>
> 14/12/19 11:52:31 INFO conf.FlumeConfiguration: Post-validation flume
> configuration contains configuration for agents: [a1]
>
> 14/12/19 11:52:31 INFO node.AbstractConfigurationProvider: Creating
> channels
>
> 14/12/19 11:52:31 INFO channel.DefaultChannelFactory: Creating instance of
> channel c1 type memory
>
> 14/12/19 11:52:31 INFO node.AbstractConfigurationProvider: Created channel
> c1
>
> 14/12/19 11:52:31 INFO source.DefaultSourceFactory: Creating instance of
> source r1, type jms
>
> 14/12/19 11:52:31 ERROR node.PollingPropertiesFileConfigurationProvider:
> Failed to start agent because dependencies were not found in classpath.
> Error follows.
>
> java.lang.NoClassDefFoundError: javax/jms/JMSException
>
> at java.lang.Class.forName0(Native Method)
>
> at java.lang.Class.forName(Class.java:190)
>
> at
> org.apache.flume.source.DefaultSourceFactory.getClass(DefaultSourceFactory.java:65)
>
> at
> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:40)
>
> at
> org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:322)
>
> at
> org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:97)
>
> at
> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Caused by: java.lang.ClassNotFoundException: javax.jms.JMSException
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>
>
>
>
> --
> Sincerely,
> Darshan
>