You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by 孙东亮 <da...@gmail.com> on 2013/01/14 04:25:25 UTC
Could not instantiate class org.apache.flume.clients.log4jappender
I'm a newbie for the Flume, and I just set up flume for a test.
I want to use the log4jappender to get log info from PIG scripts, the
log4j.properties is:
log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.Hostname = localhost
log4j.appender.flume.Port = 41414
log4j.logger.org.apache.pig=DEBUG,flume
And for the flume.config:
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = avro
a1.sources.r1.bind = localhost
a1.sources.r1.port = 41414
# Describe the sink
a1.sinks.k1.type = logger
# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
But I got the following error when run a pig script:
java.lang.ClassNotFoundException:
org.apache.flume.clients.log4jappender.Log4jAppender
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
at
org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
at
org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
at
org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
at
org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
at org.apache.pig.Main.configureLog4J(Main.java:678)
at org.apache.pig.Main.run(Main.java:337)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
log4j:ERROR Could not instantiate appender named "flume".
Could you please help me find what I miss or where is incorrect for the
configuration.
Thanks a lot!
Dongliang
RE: Could not instantiate class
org.apache.flume.clients.log4jappender
Posted by Chhaya Vishwakarma <Ch...@lntinfotech.com>.
Hi,
This error is coming because it needs some other jar from FLUME_HOME/lib/ I can’t tell you name of specific jar but I was facing same problem.Put jetty,Jackson,avro,slf4 and some more from FLUME_HOME/lib/ because every time you will run your script will give you different error like java.lang.NoClassDefFoundError
From: Dongliang Sun [mailto:davidsundl@gmail.com]
Sent: Monday, January 14, 2013 11:36 AM
To: cwoodson.dev@gmail.com; Chhaya Vishwakarma; user@flume.apache.org
Subject: Re: Could not instantiate class org.apache.flume.clients.log4jappender
REGISTER and -Dpig.additional.libraries both do not work, same error, could not find the class.
If put the particular jars in pig's lib, the following error:
Error before Pig is launched
----------------------------
ERROR 2998: Unhandled internal error. org/jboss/netty/channel/ChannelFactory
java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:168)
at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:128)
at org.apache.flume.clients.log4jappender.Log4jAppender.activateOptions(Log4jAppender.java:184)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
at org.apache.pig.Main.configureLog4J(Main.java:678)
at org.apache.pig.Main.run(Main.java:337)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ClassNotFoundException: org.jboss.netty.channel.ChannelFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 19 more
2013/1/14 Connor Woodson <cw...@gmail.com>>
You could try calling 'REGISTER <log4jappender jar>' at the start of your script, or on the command line by adding -Dpig.additional.libraries=<log4jappender jar> to your Pig call. I don't know if that will work or not though.
You could try placing it in pig's lib folder.
You also need to include the flume-ng-sdk jar the same way as the log4jappender one, as the sdk is a dependency of the appender.
On Sun, Jan 13, 2013 at 9:24 PM, Dongliang Sun <da...@gmail.com>> wrote:
Actually I have append the FLUME_HOME/lib/ to my CLASSPATH.
Whether there is some problem in my PIG log4j.properties?
Do you have any experience using log4jappender on PIG?
Thanks,
Dongliang
2013/1/14 Chhaya Vishwakarma <Ch...@lntinfotech.com>>
Put flume-log4j-appender<https://repository.cloudera.com/content/repositories/releases/com/cloudera/flume-log4j-appender/0.9.4-cdh3u1/flume-log4j-appender-0.9.4-cdh3u1.jar>.jar in CLASSPATH. Jar can be found in FLUME_HOME/lib/
From: 孙东亮 [mailto:davidsundl@gmail.com<ma...@gmail.com>]
Sent: Monday, January 14, 2013 8:55 AM
To: user@flume.apache.org<ma...@flume.apache.org>
Subject: Could not instantiate class org.apache.flume.clients.log4jappender
I'm a newbie for the Flume, and I just set up flume for a test.
I want to use the log4jappender to get log info from PIG scripts, the log4j.properties is:
log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.Hostname = localhost
log4j.appender.flume.Port = 41414
log4j.logger.org.apache.pig=DEBUG,flume
And for the flume.config:
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = avro
a1.sources.r1.bind = localhost
a1.sources.r1.port = 41414
# Describe the sink
a1.sinks.k1.type = logger
# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
But I got the following error when run a pig script:
java.lang.ClassNotFoundException: org.apache.flume.clients.log4jappender.Log4jAppender
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
at org.apache.pig.Main.configureLog4J(Main.java:678)
at org.apache.pig.Main.run(Main.java:337)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
log4j:ERROR Could not instantiate appender named "flume".
Could you please help me find what I miss or where is incorrect for the configuration.
Thanks a lot!
Dongliang
________________________________
The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"
RE: Could not instantiate class
org.apache.flume.clients.log4jappender
Posted by Chhaya Vishwakarma <Ch...@lntinfotech.com>.
Which version of slf4j you using ? for this error you need to put SLF4J 1.7.2 also in classpath as org.slf4j.spi.LocationAwareLogger.log is available in 1.7.2 of slf4j.
From: Dongliang Sun [mailto:davidsundl@gmail.com]
Sent: Monday, January 14, 2013 11:54 AM
To: Connor Woodson; Chhaya Vishwakarma; user@flume.apache.org
Subject: Re: Could not instantiate class org.apache.flume.clients.log4jappender
It does not work, -Dpig.additional.jars={REALPATH}/lib/*.
Also tried Chhaya said, I put all the libraries from FLUME_HOME/lib/* to PIG_HOME/lib/, got following error:
13/01/14 14:19:03 WARN api.NettyAvroRpcClient: Invalid value for batchSize: 0; Using default value.
Exception in thread "main" java.lang.NoSuchMethodError: org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
at org.apache.commons.logging.impl.SLF4JLocationAwareLog.error(SLF4JLocationAwareLog.java:211)
at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:125)
at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:79)
at org.apache.pig.Main.run(Main.java:589)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
2013/1/14 Connor Woodson <cw...@gmail.com>>
Alright so it is just before you weren't adding them properly to the classpath. I am not quite sure how to do that; a look at Pig's run script (bin/pig) might yield insight into an option to add them. Putting the files in pig's lib's folder almost works; the issue is those libraries have more dependencies that aren't in the classpath yet. Looking at their (log4jappender and sdk) poms could tell you exactly which libs you need to include, but let's not do that.
I mistyped the -D command from before. Try this:
-Dpig.additional.jars=${FLUME_HOME}/lib/*
FLUME_HOME obviously needs to be declared beforehand, or just replace that with the real path (and make sure the log4jappender and sdk jars, and all other jars, are in the libs folder)
On Sun, Jan 13, 2013 at 10:05 PM, Dongliang Sun <da...@gmail.com>> wrote:
REGISTER and -Dpig.additional.libraries both do not work, same error, could not find the class.
If put the particular jars in pig's lib, the following error:
Error before Pig is launched
----------------------------
ERROR 2998: Unhandled internal error. org/jboss/netty/channel/ChannelFactory
java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:168)
at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:128)
at org.apache.flume.clients.log4jappender.Log4jAppender.activateOptions(Log4jAppender.java:184)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
at org.apache.pig.Main.configureLog4J(Main.java:678)
at org.apache.pig.Main.run(Main.java:337)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ClassNotFoundException: org.jboss.netty.channel.ChannelFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 19 more
2013/1/14 Connor Woodson <cw...@gmail.com>>
You could try calling 'REGISTER <log4jappender jar>' at the start of your script, or on the command line by adding -Dpig.additional.libraries=<log4jappender jar> to your Pig call. I don't know if that will work or not though.
You could try placing it in pig's lib folder.
You also need to include the flume-ng-sdk jar the same way as the log4jappender one, as the sdk is a dependency of the appender.
On Sun, Jan 13, 2013 at 9:24 PM, Dongliang Sun <da...@gmail.com>> wrote:
Actually I have append the FLUME_HOME/lib/ to my CLASSPATH.
Whether there is some problem in my PIG log4j.properties?
Do you have any experience using log4jappender on PIG?
Thanks,
Dongliang
2013/1/14 Chhaya Vishwakarma <Ch...@lntinfotech.com>>
Put flume-log4j-appender<https://repository.cloudera.com/content/repositories/releases/com/cloudera/flume-log4j-appender/0.9.4-cdh3u1/flume-log4j-appender-0.9.4-cdh3u1.jar>.jar in CLASSPATH. Jar can be found in FLUME_HOME/lib/
From: 孙东亮 [mailto:davidsundl@gmail.com<ma...@gmail.com>]
Sent: Monday, January 14, 2013 8:55 AM
To: user@flume.apache.org<ma...@flume.apache.org>
Subject: Could not instantiate class org.apache.flume.clients.log4jappender
I'm a newbie for the Flume, and I just set up flume for a test.
I want to use the log4jappender to get log info from PIG scripts, the log4j.properties is:
log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.Hostname = localhost
log4j.appender.flume.Port = 41414
log4j.logger.org.apache.pig=DEBUG,flume
And for the flume.config:
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = avro
a1.sources.r1.bind = localhost
a1.sources.r1.port = 41414
# Describe the sink
a1.sinks.k1.type = logger
# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
But I got the following error when run a pig script:
java.lang.ClassNotFoundException: org.apache.flume.clients.log4jappender.Log4jAppender
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
at org.apache.pig.Main.configureLog4J(Main.java:678)
at org.apache.pig.Main.run(Main.java:337)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
log4j:ERROR Could not instantiate appender named "flume".
Could you please help me find what I miss or where is incorrect for the configuration.
Thanks a lot!
Dongliang
________________________________
The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Dongliang Sun <da...@gmail.com>.
It does not work, -Dpig.additional.jars={REALPATH}/lib/*.
Also tried Chhaya said, I put all the libraries from FLUME_HOME/lib/* to
PIG_HOME/lib/, got following error:
13/01/14 14:19:03 WARN api.NettyAvroRpcClient: Invalid value for batchSize:
0; Using default value.
Exception in thread "main" java.lang.NoSuchMethodError:
org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
at
org.apache.commons.logging.impl.SLF4JLocationAwareLog.error(SLF4JLocationAwareLog.java:211)
at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:125)
at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:79)
at org.apache.pig.Main.run(Main.java:589)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
2013/1/14 Connor Woodson <cw...@gmail.com>
> Alright so it is just before you weren't adding them properly to the
> classpath. I am not quite sure how to do that; a look at Pig's run script
> (bin/pig) might yield insight into an option to add them. Putting the files
> in pig's lib's folder almost works; the issue is those libraries have more
> dependencies that aren't in the classpath yet. Looking at their
> (log4jappender and sdk) poms could tell you exactly which libs you need to
> include, but let's not do that.
>
> I mistyped the -D command from before. Try this:
>
> -Dpig.additional.jars=${FLUME_HOME}/lib/*
>
> FLUME_HOME obviously needs to be declared beforehand, or just replace that
> with the real path (and make sure the log4jappender and sdk jars, and all
> other jars, are in the libs folder)
>
>
> On Sun, Jan 13, 2013 at 10:05 PM, Dongliang Sun <da...@gmail.com>wrote:
>
>> REGISTER and -Dpig.additional.libraries both do not work, same error,
>> could not find the class.
>>
>> If put the particular jars in pig's lib, the following error:
>> Error before Pig is launched
>> ----------------------------
>> ERROR 2998: Unhandled internal error.
>> org/jboss/netty/channel/ChannelFactory
>>
>> java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
>> at
>> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:168)
>> at
>> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:128)
>> at
>> org.apache.flume.clients.log4jappender.Log4jAppender.activateOptions(Log4jAppender.java:184)
>> at
>> org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
>> at
>> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
>> at
>> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
>> at
>> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689)
>>
>> at
>> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
>> at
>> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
>> at
>> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
>> at
>> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
>> at org.apache.pig.Main.configureLog4J(Main.java:678)
>> at org.apache.pig.Main.run(Main.java:337)
>> at org.apache.pig.Main.main(Main.java:111)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>> Caused by: java.lang.ClassNotFoundException:
>> org.jboss.netty.channel.ChannelFactory
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>> ... 19 more
>>
>>
>> 2013/1/14 Connor Woodson <cw...@gmail.com>
>>
>>> You could try calling 'REGISTER <log4jappender jar>' at the start of
>>> your script, or on the command line by adding
>>> -Dpig.additional.libraries=<log4jappender jar> to your Pig call. I don't
>>> know if that will work or not though.
>>>
>>> You could try placing it in pig's lib folder.
>>>
>>> You also need to include the flume-ng-sdk jar the same way as the
>>> log4jappender one, as the sdk is a dependency of the appender.
>>>
>>>
>>> On Sun, Jan 13, 2013 at 9:24 PM, Dongliang Sun <da...@gmail.com>wrote:
>>>
>>>> Actually I have append the FLUME_HOME/lib/ to my CLASSPATH.
>>>> Whether there is some problem in my PIG log4j.properties?
>>>> Do you have any experience using log4jappender on PIG?
>>>>
>>>> Thanks,
>>>> Dongliang
>>>>
>>>>
>>>>
>>>> 2013/1/14 Chhaya Vishwakarma <Ch...@lntinfotech.com>
>>>>
>>>>> Put flume-log4j-appender<https://repository.cloudera.com/content/repositories/releases/com/cloudera/flume-log4j-appender/0.9.4-cdh3u1/flume-log4j-appender-0.9.4-cdh3u1.jar>.jar
>>>>> in CLASSPATH. Jar can be found in FLUME_HOME/lib/****
>>>>>
>>>>> ** **
>>>>>
>>>>> *From:* 孙东亮 [mailto:davidsundl@gmail.com]
>>>>> *Sent:* Monday, January 14, 2013 8:55 AM
>>>>> *To:* user@flume.apache.org
>>>>> *Subject:* Could not instantiate class
>>>>> org.apache.flume.clients.log4jappender****
>>>>>
>>>>> ** **
>>>>>
>>>>> I'm a newbie for the Flume, and I just set up flume for a test.
>>>>> I want to use the log4jappender to get log info from PIG scripts, the
>>>>> log4j.properties is:****
>>>>>
>>>>> log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender ****
>>>>>
>>>>> log4j.appender.flume.Hostname = localhost ****
>>>>>
>>>>> log4j.appender.flume.Port = 41414 ****
>>>>>
>>>>> log4j.logger.org.apache.pig=DEBUG,flume****
>>>>>
>>>>> And for the flume.config: ****
>>>>>
>>>>> a1.sources = r1 ****
>>>>>
>>>>> a1.sinks = k1 ****
>>>>>
>>>>> a1.channels = c1 ****
>>>>>
>>>>> ** **
>>>>>
>>>>> # Describe/configure the source****
>>>>>
>>>>> a1.sources.r1.type = avro ****
>>>>>
>>>>> a1.sources.r1.bind = localhost ****
>>>>>
>>>>> a1.sources.r1.port = 41414 ****
>>>>>
>>>>> ** **
>>>>>
>>>>> # Describe the sink ****
>>>>>
>>>>> a1.sinks.k1.type = logger ****
>>>>>
>>>>> # Use a channel which buffers events in memory ****
>>>>>
>>>>> a1.channels.c1.type = memory ****
>>>>>
>>>>> a1.channels.c1.capacity = 1000 ****
>>>>>
>>>>> a1.channels.c1.transactionCapacity = 100 ****
>>>>>
>>>>> ** **
>>>>>
>>>>> # Bind the source and sink to the channel ****
>>>>>
>>>>> a1.sources.r1.channels = c1 ****
>>>>>
>>>>> a1.sinks.k1.channel = c1 ****
>>>>>
>>>>> But I got the following error when run a pig script:
>>>>> java.lang.ClassNotFoundException:
>>>>> org.apache.flume.clients.log4jappender.Log4jAppender
>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>>>> at java.lang.Class.forName0(Native Method)
>>>>> at java.lang.Class.forName(Class.java:169)
>>>>> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
>>>>> at
>>>>> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
>>>>> at
>>>>> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
>>>>> at
>>>>> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
>>>>> at
>>>>> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
>>>>> at
>>>>> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
>>>>> at
>>>>> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
>>>>> at
>>>>> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
>>>>> at org.apache.pig.Main.configureLog4J(Main.java:678)
>>>>> at org.apache.pig.Main.run(Main.java:337)
>>>>> at org.apache.pig.Main.main(Main.java:111)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>> log4j:ERROR Could not instantiate appender named "flume".****
>>>>>
>>>>> Could you please help me find what I miss or where is incorrect for
>>>>> the configuration.****
>>>>>
>>>>> Thanks a lot!
>>>>> Dongliang****
>>>>>
>>>>> ------------------------------
>>>>> The contents of this e-mail and any attachment(s) may contain
>>>>> confidential or privileged information for the intended recipient(s).
>>>>> Unintended recipients are prohibited from taking action on the basis of
>>>>> information in this e-mail and using or disseminating the information, and
>>>>> must notify the sender and delete it from their system. L&T Infotech will
>>>>> not accept responsibility or liability for the accuracy or completeness of,
>>>>> or the presence of any virus or disabling code in this e-mail"
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Dongliang Sun <da...@gmail.com>.
REGISTER and -Dpig.additional.libraries both do not work, same error, could
not find the class.
If put the particular jars in pig's lib, the following error:
Error before Pig is launched
----------------------------
ERROR 2998: Unhandled internal error. org/jboss/netty/channel/ChannelFactory
java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
at
org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:168)
at
org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:128)
at
org.apache.flume.clients.log4jappender.Log4jAppender.activateOptions(Log4jAppender.java:184)
at
org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
at
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
at
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
at
org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
at
org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
at
org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
at org.apache.pig.Main.configureLog4J(Main.java:678)
at org.apache.pig.Main.run(Main.java:337)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ClassNotFoundException:
org.jboss.netty.channel.ChannelFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 19 more
2013/1/14 Connor Woodson <cw...@gmail.com>
> You could try calling 'REGISTER <log4jappender jar>' at the start of your
> script, or on the command line by adding
> -Dpig.additional.libraries=<log4jappender jar> to your Pig call. I don't
> know if that will work or not though.
>
> You could try placing it in pig's lib folder.
>
> You also need to include the flume-ng-sdk jar the same way as the
> log4jappender one, as the sdk is a dependency of the appender.
>
>
> On Sun, Jan 13, 2013 at 9:24 PM, Dongliang Sun <da...@gmail.com>wrote:
>
>> Actually I have append the FLUME_HOME/lib/ to my CLASSPATH.
>> Whether there is some problem in my PIG log4j.properties?
>> Do you have any experience using log4jappender on PIG?
>>
>> Thanks,
>> Dongliang
>>
>>
>>
>> 2013/1/14 Chhaya Vishwakarma <Ch...@lntinfotech.com>
>>
>>> Put flume-log4j-appender<https://repository.cloudera.com/content/repositories/releases/com/cloudera/flume-log4j-appender/0.9.4-cdh3u1/flume-log4j-appender-0.9.4-cdh3u1.jar>.jar
>>> in CLASSPATH. Jar can be found in FLUME_HOME/lib/****
>>>
>>> ** **
>>>
>>> *From:* 孙东亮 [mailto:davidsundl@gmail.com]
>>> *Sent:* Monday, January 14, 2013 8:55 AM
>>> *To:* user@flume.apache.org
>>> *Subject:* Could not instantiate class
>>> org.apache.flume.clients.log4jappender****
>>>
>>> ** **
>>>
>>> I'm a newbie for the Flume, and I just set up flume for a test.
>>> I want to use the log4jappender to get log info from PIG scripts, the
>>> log4j.properties is:****
>>>
>>> log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender ****
>>>
>>> log4j.appender.flume.Hostname = localhost ****
>>>
>>> log4j.appender.flume.Port = 41414 ****
>>>
>>> log4j.logger.org.apache.pig=DEBUG,flume****
>>>
>>> And for the flume.config: ****
>>>
>>> a1.sources = r1 ****
>>>
>>> a1.sinks = k1 ****
>>>
>>> a1.channels = c1 ****
>>>
>>> ** **
>>>
>>> # Describe/configure the source****
>>>
>>> a1.sources.r1.type = avro ****
>>>
>>> a1.sources.r1.bind = localhost ****
>>>
>>> a1.sources.r1.port = 41414 ****
>>>
>>> ** **
>>>
>>> # Describe the sink ****
>>>
>>> a1.sinks.k1.type = logger ****
>>>
>>> # Use a channel which buffers events in memory ****
>>>
>>> a1.channels.c1.type = memory ****
>>>
>>> a1.channels.c1.capacity = 1000 ****
>>>
>>> a1.channels.c1.transactionCapacity = 100 ****
>>>
>>> ** **
>>>
>>> # Bind the source and sink to the channel ****
>>>
>>> a1.sources.r1.channels = c1 ****
>>>
>>> a1.sinks.k1.channel = c1 ****
>>>
>>> But I got the following error when run a pig script:
>>> java.lang.ClassNotFoundException:
>>> org.apache.flume.clients.log4jappender.Log4jAppender
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>> at java.lang.Class.forName0(Native Method)
>>> at java.lang.Class.forName(Class.java:169)
>>> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
>>> at
>>> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
>>> at
>>> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
>>> at
>>> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
>>> at
>>> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
>>> at
>>> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
>>> at
>>> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
>>> at
>>> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
>>> at org.apache.pig.Main.configureLog4J(Main.java:678)
>>> at org.apache.pig.Main.run(Main.java:337)
>>> at org.apache.pig.Main.main(Main.java:111)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>> log4j:ERROR Could not instantiate appender named "flume".****
>>>
>>> Could you please help me find what I miss or where is incorrect for the
>>> configuration.****
>>>
>>> Thanks a lot!
>>> Dongliang****
>>>
>>> ------------------------------
>>> The contents of this e-mail and any attachment(s) may contain
>>> confidential or privileged information for the intended recipient(s).
>>> Unintended recipients are prohibited from taking action on the basis of
>>> information in this e-mail and using or disseminating the information, and
>>> must notify the sender and delete it from their system. L&T Infotech will
>>> not accept responsibility or liability for the accuracy or completeness of,
>>> or the presence of any virus or disabling code in this e-mail"
>>>
>>
>>
>
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Connor Woodson <cw...@gmail.com>.
You could try calling 'REGISTER <log4jappender jar>' at the start of your
script, or on the command line by adding
-Dpig.additional.libraries=<log4jappender jar> to your Pig call. I don't
know if that will work or not though.
You could try placing it in pig's lib folder.
You also need to include the flume-ng-sdk jar the same way as the
log4jappender one, as the sdk is a dependency of the appender.
On Sun, Jan 13, 2013 at 9:24 PM, Dongliang Sun <da...@gmail.com> wrote:
> Actually I have append the FLUME_HOME/lib/ to my CLASSPATH.
> Whether there is some problem in my PIG log4j.properties?
> Do you have any experience using log4jappender on PIG?
>
> Thanks,
> Dongliang
>
>
>
> 2013/1/14 Chhaya Vishwakarma <Ch...@lntinfotech.com>
>
>> Put flume-log4j-appender<https://repository.cloudera.com/content/repositories/releases/com/cloudera/flume-log4j-appender/0.9.4-cdh3u1/flume-log4j-appender-0.9.4-cdh3u1.jar>.jar
>> in CLASSPATH. Jar can be found in FLUME_HOME/lib/****
>>
>> ** **
>>
>> *From:* 孙东亮 [mailto:davidsundl@gmail.com]
>> *Sent:* Monday, January 14, 2013 8:55 AM
>> *To:* user@flume.apache.org
>> *Subject:* Could not instantiate class
>> org.apache.flume.clients.log4jappender****
>>
>> ** **
>>
>> I'm a newbie for the Flume, and I just set up flume for a test.
>> I want to use the log4jappender to get log info from PIG scripts, the
>> log4j.properties is:****
>>
>> log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender ****
>>
>> log4j.appender.flume.Hostname = localhost ****
>>
>> log4j.appender.flume.Port = 41414 ****
>>
>> log4j.logger.org.apache.pig=DEBUG,flume****
>>
>> And for the flume.config: ****
>>
>> a1.sources = r1 ****
>>
>> a1.sinks = k1 ****
>>
>> a1.channels = c1 ****
>>
>> ** **
>>
>> # Describe/configure the source****
>>
>> a1.sources.r1.type = avro ****
>>
>> a1.sources.r1.bind = localhost ****
>>
>> a1.sources.r1.port = 41414 ****
>>
>> ** **
>>
>> # Describe the sink ****
>>
>> a1.sinks.k1.type = logger ****
>>
>> # Use a channel which buffers events in memory ****
>>
>> a1.channels.c1.type = memory ****
>>
>> a1.channels.c1.capacity = 1000 ****
>>
>> a1.channels.c1.transactionCapacity = 100 ****
>>
>> ** **
>>
>> # Bind the source and sink to the channel ****
>>
>> a1.sources.r1.channels = c1 ****
>>
>> a1.sinks.k1.channel = c1 ****
>>
>> But I got the following error when run a pig script:
>> java.lang.ClassNotFoundException:
>> org.apache.flume.clients.log4jappender.Log4jAppender
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:169)
>> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
>> at
>> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
>> at
>> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
>> at
>> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
>> at
>> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
>> at
>> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
>> at
>> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
>> at
>> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
>> at org.apache.pig.Main.configureLog4J(Main.java:678)
>> at org.apache.pig.Main.run(Main.java:337)
>> at org.apache.pig.Main.main(Main.java:111)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>> log4j:ERROR Could not instantiate appender named "flume".****
>>
>> Could you please help me find what I miss or where is incorrect for the
>> configuration.****
>>
>> Thanks a lot!
>> Dongliang****
>>
>> ------------------------------
>> The contents of this e-mail and any attachment(s) may contain
>> confidential or privileged information for the intended recipient(s).
>> Unintended recipients are prohibited from taking action on the basis of
>> information in this e-mail and using or disseminating the information, and
>> must notify the sender and delete it from their system. L&T Infotech will
>> not accept responsibility or liability for the accuracy or completeness of,
>> or the presence of any virus or disabling code in this e-mail"
>>
>
>
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Dongliang Sun <da...@gmail.com>.
Chhaya,
Thanks for your help. I have followed your instruction, put the particular
jar(flume-ng-log4jappender-1.3.1.jar) in my CLASSPATH also, but it still
does not work, same error.
2013/1/14 Chhaya Vishwakarma <Ch...@lntinfotech.com>
> Place that particular jar also.it will work****
>
> ** **
>
> *From:* Dongliang Sun [mailto:davidsundl@gmail.com]
> *Sent:* Monday, January 14, 2013 10:55 AM
> *To:* Chhaya Vishwakarma; user@flume.apache.org
> *Subject:* Re: Could not instantiate class
> org.apache.flume.clients.log4jappender****
>
> ** **
>
> Actually I have append the FLUME_HOME/lib/ to my CLASSPATH.****
>
> Whether there is some problem in my PIG log4j.properties?****
>
> Do you have any experience using log4jappender on PIG?****
>
> Thanks,****
>
> Dongliang****
>
> ** **
>
> 2013/1/14 Chhaya Vishwakarma <Ch...@lntinfotech.com>****
>
> Put flume-log4j-appender<https://repository.cloudera.com/content/repositories/releases/com/cloudera/flume-log4j-appender/0.9.4-cdh3u1/flume-log4j-appender-0.9.4-cdh3u1.jar>.jar
> in CLASSPATH. Jar can be found in FLUME_HOME/lib/****
>
> ****
>
> *From:* 孙东亮 [mailto:davidsundl@gmail.com]
> *Sent:* Monday, January 14, 2013 8:55 AM
> *To:* user@flume.apache.org
> *Subject:* Could not instantiate class
> org.apache.flume.clients.log4jappender****
>
> ****
>
> I'm a newbie for the Flume, and I just set up flume for a test.
> I want to use the log4jappender to get log info from PIG scripts, the
> log4j.properties is:****
>
> log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender ****
>
> log4j.appender.flume.Hostname = localhost ****
>
> log4j.appender.flume.Port = 41414 ****
>
> log4j.logger.org.apache.pig=DEBUG,flume****
>
> And for the flume.config: ****
>
> a1.sources = r1 ****
>
> a1.sinks = k1 ****
>
> a1.channels = c1 ****
>
> ****
>
> # Describe/configure the source****
>
> a1.sources.r1.type = avro ****
>
> a1.sources.r1.bind = localhost ****
>
> a1.sources.r1.port = 41414 ****
>
> ****
>
> # Describe the sink ****
>
> a1.sinks.k1.type = logger ****
>
> # Use a channel which buffers events in memory ****
>
> a1.channels.c1.type = memory ****
>
> a1.channels.c1.capacity = 1000 ****
>
> a1.channels.c1.transactionCapacity = 100 ****
>
> ****
>
> # Bind the source and sink to the channel ****
>
> a1.sources.r1.channels = c1 ****
>
> a1.sinks.k1.channel = c1 ****
>
> But I got the following error when run a pig script:
> java.lang.ClassNotFoundException:
> org.apache.flume.clients.log4jappender.Log4jAppender
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
> at
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
> at
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
> at
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
> at
> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
> at org.apache.pig.Main.configureLog4J(Main.java:678)
> at org.apache.pig.Main.run(Main.java:337)
> at org.apache.pig.Main.main(Main.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> log4j:ERROR Could not instantiate appender named "flume".****
>
> Could you please help me find what I miss or where is incorrect for the
> configuration.****
>
> Thanks a lot!
> Dongliang****
>
> ** **
> ------------------------------
>
> The contents of this e-mail and any attachment(s) may contain confidential
> or privileged information for the intended recipient(s). Unintended
> recipients are prohibited from taking action on the basis of information in
> this e-mail and using or disseminating the information, and must notify the
> sender and delete it from their system. L&T Infotech will not accept
> responsibility or liability for the accuracy or completeness of, or the
> presence of any virus or disabling code in this e-mail"****
>
> ** **
>
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Dongliang Sun <da...@gmail.com>.
Actually I have append the FLUME_HOME/lib/ to my CLASSPATH.
Whether there is some problem in my PIG log4j.properties?
Do you have any experience using log4jappender on PIG?
Thanks,
Dongliang
2013/1/14 Chhaya Vishwakarma <Ch...@lntinfotech.com>
> Put flume-log4j-appender<https://repository.cloudera.com/content/repositories/releases/com/cloudera/flume-log4j-appender/0.9.4-cdh3u1/flume-log4j-appender-0.9.4-cdh3u1.jar>.jar
> in CLASSPATH. Jar can be found in FLUME_HOME/lib/****
>
> ** **
>
> *From:* 孙东亮 [mailto:davidsundl@gmail.com]
> *Sent:* Monday, January 14, 2013 8:55 AM
> *To:* user@flume.apache.org
> *Subject:* Could not instantiate class
> org.apache.flume.clients.log4jappender****
>
> ** **
>
> I'm a newbie for the Flume, and I just set up flume for a test.
> I want to use the log4jappender to get log info from PIG scripts, the
> log4j.properties is:****
>
> log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender ****
>
> log4j.appender.flume.Hostname = localhost ****
>
> log4j.appender.flume.Port = 41414 ****
>
> log4j.logger.org.apache.pig=DEBUG,flume****
>
> And for the flume.config: ****
>
> a1.sources = r1 ****
>
> a1.sinks = k1 ****
>
> a1.channels = c1 ****
>
> ** **
>
> # Describe/configure the source****
>
> a1.sources.r1.type = avro ****
>
> a1.sources.r1.bind = localhost ****
>
> a1.sources.r1.port = 41414 ****
>
> ** **
>
> # Describe the sink ****
>
> a1.sinks.k1.type = logger ****
>
> # Use a channel which buffers events in memory ****
>
> a1.channels.c1.type = memory ****
>
> a1.channels.c1.capacity = 1000 ****
>
> a1.channels.c1.transactionCapacity = 100 ****
>
> ** **
>
> # Bind the source and sink to the channel ****
>
> a1.sources.r1.channels = c1 ****
>
> a1.sinks.k1.channel = c1 ****
>
> But I got the following error when run a pig script:
> java.lang.ClassNotFoundException:
> org.apache.flume.clients.log4jappender.Log4jAppender
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
> at
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
> at
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
> at
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
> at
> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
> at org.apache.pig.Main.configureLog4J(Main.java:678)
> at org.apache.pig.Main.run(Main.java:337)
> at org.apache.pig.Main.main(Main.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> log4j:ERROR Could not instantiate appender named "flume".****
>
> Could you please help me find what I miss or where is incorrect for the
> configuration.****
>
> Thanks a lot!
> Dongliang****
>
> ------------------------------
> The contents of this e-mail and any attachment(s) may contain confidential
> or privileged information for the intended recipient(s). Unintended
> recipients are prohibited from taking action on the basis of information in
> this e-mail and using or disseminating the information, and must notify the
> sender and delete it from their system. L&T Infotech will not accept
> responsibility or liability for the accuracy or completeness of, or the
> presence of any virus or disabling code in this e-mail"
>
RE: Could not instantiate class
org.apache.flume.clients.log4jappender
Posted by Chhaya Vishwakarma <Ch...@lntinfotech.com>.
Put flume-log4j-appender<https://repository.cloudera.com/content/repositories/releases/com/cloudera/flume-log4j-appender/0.9.4-cdh3u1/flume-log4j-appender-0.9.4-cdh3u1.jar>.jar in CLASSPATH. Jar can be found in FLUME_HOME/lib/
From: 孙东亮 [mailto:davidsundl@gmail.com]
Sent: Monday, January 14, 2013 8:55 AM
To: user@flume.apache.org
Subject: Could not instantiate class org.apache.flume.clients.log4jappender
I'm a newbie for the Flume, and I just set up flume for a test.
I want to use the log4jappender to get log info from PIG scripts, the log4j.properties is:
log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.Hostname = localhost
log4j.appender.flume.Port = 41414
log4j.logger.org.apache.pig=DEBUG,flume
And for the flume.config:
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = avro
a1.sources.r1.bind = localhost
a1.sources.r1.port = 41414
# Describe the sink
a1.sinks.k1.type = logger
# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
But I got the following error when run a pig script:
java.lang.ClassNotFoundException: org.apache.flume.clients.log4jappender.Log4jAppender
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
at org.apache.pig.Main.configureLog4J(Main.java:678)
at org.apache.pig.Main.run(Main.java:337)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
log4j:ERROR Could not instantiate appender named "flume".
Could you please help me find what I miss or where is incorrect for the configuration.
Thanks a lot!
Dongliang
________________________________
The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
May you ask on the Pig mailing list? This looks like not a Flume issue, and I think the pig knowledge is a bit limited here. The only option as I know so far has Connor described.
- Alex
On Jan 14, 2013, at 10:30 AM, Dongliang Sun <da...@gmail.com> wrote:
> I have followed Connor's instructions and build the pig from pig-trunk.
>
> But get the same error if I do not do anything else.
> log4j:ERROR Could not instantiate class
> [org.apache.flume.clients.log4jappender.Log4jAppender].
> java.lang.ClassNotFoundException:
> org.apache.flume.clients.log4jappender.Log4jAppender
>
> So what should I do next? Copy the jars from $FLUME_HOME/lib to
> pig-trunk/lib manually?
>
>
> 2013/1/14 Alexander Alten-Lorenz <wg...@gmail.com>
>
>> Thanks, thats correct.
>>
>> @Dongliang: Please follow the instructions Connor gave.
>>
>> - Alex
>>
>> On Jan 14, 2013, at 8:10 AM, Connor Woodson <cw...@gmail.com>
>> wrote:
>>
>>> He's running Pig, not Flume, but trying to get the Flume Log4jAppender
>>> working.
>>>
>>> If you want to try to compile with the log4jappender, download Pig from
>>> Github and add the following to ivy.xml, then 'ant build':
>>>
>>> <dependency org="org.apache.flume" name="flume-ng-clients" rev="1.3.1"/>
>>>
>>>
>>>
>>> On Sun, Jan 13, 2013 at 10:58 PM, Alexander Alten-Lorenz <
>>> wget.null@gmail.com> wrote:
>>>
>>>> Please post the flume startup line. I guess you have missed the --conf
>>>> switch.
>>>>
>>>> - Alex
>>>>
>>>> On Jan 14, 2013, at 4:25 AM, 孙东亮 <da...@gmail.com> wrote:
>>>>
>>>>> I'm a newbie for the Flume, and I just set up flume for a test.
>>>>> I want to use the log4jappender to get log info from PIG scripts, the
>>>>> log4j.properties is:
>>>>>
>>>>> log4j.appender.flume =
>>>> org.apache.flume.clients.log4jappender.Log4jAppender
>>>>> log4j.appender.flume.Hostname = localhost
>>>>> log4j.appender.flume.Port = 41414
>>>>> log4j.logger.org.apache.pig=DEBUG,flume
>>>>>
>>>>> And for the flume.config:
>>>>>
>>>>> a1.sources = r1
>>>>> a1.sinks = k1
>>>>> a1.channels = c1
>>>>>
>>>>> # Describe/configure the source
>>>>> a1.sources.r1.type = avro
>>>>> a1.sources.r1.bind = localhost
>>>>> a1.sources.r1.port = 41414
>>>>>
>>>>> # Describe the sink
>>>>> a1.sinks.k1.type = logger
>>>>> # Use a channel which buffers events in memory
>>>>> a1.channels.c1.type = memory
>>>>> a1.channels.c1.capacity = 1000
>>>>> a1.channels.c1.transactionCapacity = 100
>>>>>
>>>>> # Bind the source and sink to the channel
>>>>> a1.sources.r1.channels = c1
>>>>> a1.sinks.k1.channel = c1
>>>>>
>>>>> But I got the following error when run a pig script:
>>>>> java.lang.ClassNotFoundException:
>>>>> org.apache.flume.clients.log4jappender.Log4jAppender
>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>>>> at java.lang.Class.forName0(Native Method)
>>>>> at java.lang.Class.forName(Class.java:169)
>>>>> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
>>>>> at
>>>>>
>>>>
>> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
>>>>> at
>>>>>
>>>>
>> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
>>>>> at
>>>>>
>>>>
>> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
>>>>> at
>>>>>
>>>>
>> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
>>>>> at
>>>>>
>>>>
>> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
>>>>> at
>>>>>
>>>>
>> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
>>>>> at
>>>>>
>>>>
>> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
>>>>> at org.apache.pig.Main.configureLog4J(Main.java:678)
>>>>> at org.apache.pig.Main.run(Main.java:337)
>>>>> at org.apache.pig.Main.main(Main.java:111)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>>
>>>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>> at
>>>>>
>>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>> log4j:ERROR Could not instantiate appender named "flume".
>>>>>
>>>>> Could you please help me find what I miss or where is incorrect for the
>>>>> configuration.
>>>>>
>>>>> Thanks a lot!
>>>>> Dongliang
>>>>
>>>> --
>>>> Alexander Alten-Lorenz
>>>> http://mapredit.blogspot.com
>>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>>>
>>>>
>>
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>
>>
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Dongliang Sun <da...@gmail.com>.
I have followed Connor's instructions and build the pig from pig-trunk.
But get the same error if I do not do anything else.
log4j:ERROR Could not instantiate class
[org.apache.flume.clients.log4jappender.Log4jAppender].
java.lang.ClassNotFoundException:
org.apache.flume.clients.log4jappender.Log4jAppender
So what should I do next? Copy the jars from $FLUME_HOME/lib to
pig-trunk/lib manually?
2013/1/14 Alexander Alten-Lorenz <wg...@gmail.com>
> Thanks, thats correct.
>
> @Dongliang: Please follow the instructions Connor gave.
>
> - Alex
>
> On Jan 14, 2013, at 8:10 AM, Connor Woodson <cw...@gmail.com>
> wrote:
>
> > He's running Pig, not Flume, but trying to get the Flume Log4jAppender
> > working.
> >
> > If you want to try to compile with the log4jappender, download Pig from
> > Github and add the following to ivy.xml, then 'ant build':
> >
> > <dependency org="org.apache.flume" name="flume-ng-clients" rev="1.3.1"/>
> >
> >
> >
> > On Sun, Jan 13, 2013 at 10:58 PM, Alexander Alten-Lorenz <
> > wget.null@gmail.com> wrote:
> >
> >> Please post the flume startup line. I guess you have missed the --conf
> >> switch.
> >>
> >> - Alex
> >>
> >> On Jan 14, 2013, at 4:25 AM, 孙东亮 <da...@gmail.com> wrote:
> >>
> >>> I'm a newbie for the Flume, and I just set up flume for a test.
> >>> I want to use the log4jappender to get log info from PIG scripts, the
> >>> log4j.properties is:
> >>>
> >>> log4j.appender.flume =
> >> org.apache.flume.clients.log4jappender.Log4jAppender
> >>> log4j.appender.flume.Hostname = localhost
> >>> log4j.appender.flume.Port = 41414
> >>> log4j.logger.org.apache.pig=DEBUG,flume
> >>>
> >>> And for the flume.config:
> >>>
> >>> a1.sources = r1
> >>> a1.sinks = k1
> >>> a1.channels = c1
> >>>
> >>> # Describe/configure the source
> >>> a1.sources.r1.type = avro
> >>> a1.sources.r1.bind = localhost
> >>> a1.sources.r1.port = 41414
> >>>
> >>> # Describe the sink
> >>> a1.sinks.k1.type = logger
> >>> # Use a channel which buffers events in memory
> >>> a1.channels.c1.type = memory
> >>> a1.channels.c1.capacity = 1000
> >>> a1.channels.c1.transactionCapacity = 100
> >>>
> >>> # Bind the source and sink to the channel
> >>> a1.sources.r1.channels = c1
> >>> a1.sinks.k1.channel = c1
> >>>
> >>> But I got the following error when run a pig script:
> >>> java.lang.ClassNotFoundException:
> >>> org.apache.flume.clients.log4jappender.Log4jAppender
> >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> >>> at java.security.AccessController.doPrivileged(Native Method)
> >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> >>> at java.lang.Class.forName0(Native Method)
> >>> at java.lang.Class.forName(Class.java:169)
> >>> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
> >>> at
> >>>
> >>
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
> >>> at
> >>>
> >>
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
> >>> at
> >>>
> >>
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
> >>> at
> >>>
> >>
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
> >>> at
> >>>
> >>
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
> >>> at
> >>>
> >>
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
> >>> at
> >>>
> >>
> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
> >>> at org.apache.pig.Main.configureLog4J(Main.java:678)
> >>> at org.apache.pig.Main.run(Main.java:337)
> >>> at org.apache.pig.Main.main(Main.java:111)
> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>> at
> >>>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >>> at
> >>>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>> at java.lang.reflect.Method.invoke(Method.java:597)
> >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> >>> log4j:ERROR Could not instantiate appender named "flume".
> >>>
> >>> Could you please help me find what I miss or where is incorrect for the
> >>> configuration.
> >>>
> >>> Thanks a lot!
> >>> Dongliang
> >>
> >> --
> >> Alexander Alten-Lorenz
> >> http://mapredit.blogspot.com
> >> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
> >>
> >>
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
Thanks, thats correct.
@Dongliang: Please follow the instructions Connor gave.
- Alex
On Jan 14, 2013, at 8:10 AM, Connor Woodson <cw...@gmail.com> wrote:
> He's running Pig, not Flume, but trying to get the Flume Log4jAppender
> working.
>
> If you want to try to compile with the log4jappender, download Pig from
> Github and add the following to ivy.xml, then 'ant build':
>
> <dependency org="org.apache.flume" name="flume-ng-clients" rev="1.3.1"/>
>
>
>
> On Sun, Jan 13, 2013 at 10:58 PM, Alexander Alten-Lorenz <
> wget.null@gmail.com> wrote:
>
>> Please post the flume startup line. I guess you have missed the --conf
>> switch.
>>
>> - Alex
>>
>> On Jan 14, 2013, at 4:25 AM, 孙东亮 <da...@gmail.com> wrote:
>>
>>> I'm a newbie for the Flume, and I just set up flume for a test.
>>> I want to use the log4jappender to get log info from PIG scripts, the
>>> log4j.properties is:
>>>
>>> log4j.appender.flume =
>> org.apache.flume.clients.log4jappender.Log4jAppender
>>> log4j.appender.flume.Hostname = localhost
>>> log4j.appender.flume.Port = 41414
>>> log4j.logger.org.apache.pig=DEBUG,flume
>>>
>>> And for the flume.config:
>>>
>>> a1.sources = r1
>>> a1.sinks = k1
>>> a1.channels = c1
>>>
>>> # Describe/configure the source
>>> a1.sources.r1.type = avro
>>> a1.sources.r1.bind = localhost
>>> a1.sources.r1.port = 41414
>>>
>>> # Describe the sink
>>> a1.sinks.k1.type = logger
>>> # Use a channel which buffers events in memory
>>> a1.channels.c1.type = memory
>>> a1.channels.c1.capacity = 1000
>>> a1.channels.c1.transactionCapacity = 100
>>>
>>> # Bind the source and sink to the channel
>>> a1.sources.r1.channels = c1
>>> a1.sinks.k1.channel = c1
>>>
>>> But I got the following error when run a pig script:
>>> java.lang.ClassNotFoundException:
>>> org.apache.flume.clients.log4jappender.Log4jAppender
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>> at java.lang.Class.forName0(Native Method)
>>> at java.lang.Class.forName(Class.java:169)
>>> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
>>> at
>>>
>> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
>>> at
>>>
>> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
>>> at
>>>
>> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
>>> at
>>>
>> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
>>> at
>>>
>> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
>>> at
>>>
>> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
>>> at
>>>
>> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
>>> at org.apache.pig.Main.configureLog4J(Main.java:678)
>>> at org.apache.pig.Main.run(Main.java:337)
>>> at org.apache.pig.Main.main(Main.java:111)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at
>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>> log4j:ERROR Could not instantiate appender named "flume".
>>>
>>> Could you please help me find what I miss or where is incorrect for the
>>> configuration.
>>>
>>> Thanks a lot!
>>> Dongliang
>>
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>
>>
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Connor Woodson <cw...@gmail.com>.
He's running Pig, not Flume, but trying to get the Flume Log4jAppender
working.
If you want to try to compile with the log4jappender, download Pig from
Github and add the following to ivy.xml, then 'ant build':
<dependency org="org.apache.flume" name="flume-ng-clients" rev="1.3.1"/>
On Sun, Jan 13, 2013 at 10:58 PM, Alexander Alten-Lorenz <
wget.null@gmail.com> wrote:
> Please post the flume startup line. I guess you have missed the --conf
> switch.
>
> - Alex
>
> On Jan 14, 2013, at 4:25 AM, 孙东亮 <da...@gmail.com> wrote:
>
> > I'm a newbie for the Flume, and I just set up flume for a test.
> > I want to use the log4jappender to get log info from PIG scripts, the
> > log4j.properties is:
> >
> > log4j.appender.flume =
> org.apache.flume.clients.log4jappender.Log4jAppender
> > log4j.appender.flume.Hostname = localhost
> > log4j.appender.flume.Port = 41414
> > log4j.logger.org.apache.pig=DEBUG,flume
> >
> > And for the flume.config:
> >
> > a1.sources = r1
> > a1.sinks = k1
> > a1.channels = c1
> >
> > # Describe/configure the source
> > a1.sources.r1.type = avro
> > a1.sources.r1.bind = localhost
> > a1.sources.r1.port = 41414
> >
> > # Describe the sink
> > a1.sinks.k1.type = logger
> > # Use a channel which buffers events in memory
> > a1.channels.c1.type = memory
> > a1.channels.c1.capacity = 1000
> > a1.channels.c1.transactionCapacity = 100
> >
> > # Bind the source and sink to the channel
> > a1.sources.r1.channels = c1
> > a1.sinks.k1.channel = c1
> >
> > But I got the following error when run a pig script:
> > java.lang.ClassNotFoundException:
> > org.apache.flume.clients.log4jappender.Log4jAppender
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> > at java.lang.Class.forName0(Native Method)
> > at java.lang.Class.forName(Class.java:169)
> > at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
> > at
> >
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
> > at
> >
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
> > at
> >
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
> > at
> >
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
> > at
> >
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
> > at
> >
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
> > at
> >
> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
> > at org.apache.pig.Main.configureLog4J(Main.java:678)
> > at org.apache.pig.Main.run(Main.java:337)
> > at org.apache.pig.Main.main(Main.java:111)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > log4j:ERROR Could not instantiate appender named "flume".
> >
> > Could you please help me find what I miss or where is incorrect for the
> > configuration.
> >
> > Thanks a lot!
> > Dongliang
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Dongliang Sun <da...@gmail.com>.
The result from ps waux | grep flume :
hadoop 10160 2.8 3.2 186672 32960 pts/2 Sl+ 16:35 0:00
/usr/lib/jvm/jdk1.6.0_37/bin/java -Xmx20m -Dflume.root.logger=DEBUG,console
-cp
/home/hadoop/softwares/apache-flume-1.3.1-bin/conf:/home/hadoop/softwares/apache-flume-1.3.1-bin/lib/*:/home/hadoop/softwares/hadoop-1.1.1/libexec/../conf:/usr/lib/jvm/jdk1.6.0_37/lib/tools.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/..:/home/hadoop/softwares/hadoop-1.1.1/libexec/../hadoop-core-1.1.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/asm-3.2.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/aspectjrt-1.6.11.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/aspectjtools-1.6.11.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-beanutils-1.7.0.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-beanutils-core-1.8.0.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-cli-1.2.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-codec-1.4.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-collections-3.2.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-configuration-1.6.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-daemon-1.0.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-digester-1.8.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-el-1.0.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-httpclient-3.0.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-io-2.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-lang-2.4.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-logging-1.1.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-logging-api-1.0.4.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-math-2.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/commons-net-3.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/core-3.1.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/hadoop-capacity-scheduler-1.1.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/hadoop-fairscheduler-1.1.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/hadoop-thriftfs-1.1.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/hsqldb-1.8.0.10.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jackson-core-asl-1.8.8.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jasper-compiler-5.5.12.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jasper-runtime-5.5.12.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jdeb-0.8.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jersey-core-1.8.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jersey-json-1.8.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jersey-server-1.8.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jets3t-0.6.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jetty-6.1.26.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jetty-util-6.1.26.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jsch-0.1.42.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/junit-4.5.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/kfs-0.2.2.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/log4j-1.2.15.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/mockito-all-1.8.5.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/oro-2.0.8.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/servlet-api-2.5-20081211.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/xmlenc-0.52.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jsp-2.1/jsp-2.1.jar:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/jsp-2.1/jsp-api-2.1.jar
-Djava.library.path=:/home/hadoop/softwares/hadoop-1.1.1/libexec/../lib/native/Linux-i386-32
org.apache.flume.node.Application --conf-file ./conf/flume.conf --name
agent1
2013/1/14 Alexander Alten-Lorenz <wg...@gmail.com>
> Can you please post the running flume process (ps waux|grep flume)?
>
> - Alex
>
> On Jan 14, 2013, at 8:11 AM, Dongliang Sun <da...@gmail.com> wrote:
>
> > Alex,
> >
> > The flume startup no problem,
> > flume-ng agent --conf ./conf/ --conf-file ./conf/flume.conf --name agent1
> > -Dflume.root.logger=DEBUG,console
> >
> > I have test the following client,
> > bin/flume-ng avro-client -H localhost -p 41414 -F ~/test.txt
> >
> > It's OK, but if I can not make it work if I use the log4jappender as the
> > client.
> >
> > Thanks,
> > Dongliang
> >
> >
> > 2013/1/14 Alexander Alten-Lorenz <wg...@gmail.com>
> >
> >> Please post the flume startup line. I guess you have missed the --conf
> >> switch.
> >>
> >> - Alex
> >>
> >> On Jan 14, 2013, at 4:25 AM, 孙东亮 <da...@gmail.com> wrote:
> >>
> >>> I'm a newbie for the Flume, and I just set up flume for a test.
> >>> I want to use the log4jappender to get log info from PIG scripts, the
> >>> log4j.properties is:
> >>>
> >>> log4j.appender.flume =
> >> org.apache.flume.clients.log4jappender.Log4jAppender
> >>> log4j.appender.flume.Hostname = localhost
> >>> log4j.appender.flume.Port = 41414
> >>> log4j.logger.org.apache.pig=DEBUG,flume
> >>>
> >>> And for the flume.config:
> >>>
> >>> a1.sources = r1
> >>> a1.sinks = k1
> >>> a1.channels = c1
> >>>
> >>> # Describe/configure the source
> >>> a1.sources.r1.type = avro
> >>> a1.sources.r1.bind = localhost
> >>> a1.sources.r1.port = 41414
> >>>
> >>> # Describe the sink
> >>> a1.sinks.k1.type = logger
> >>> # Use a channel which buffers events in memory
> >>> a1.channels.c1.type = memory
> >>> a1.channels.c1.capacity = 1000
> >>> a1.channels.c1.transactionCapacity = 100
> >>>
> >>> # Bind the source and sink to the channel
> >>> a1.sources.r1.channels = c1
> >>> a1.sinks.k1.channel = c1
> >>>
> >>> But I got the following error when run a pig script:
> >>> java.lang.ClassNotFoundException:
> >>> org.apache.flume.clients.log4jappender.Log4jAppender
> >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> >>> at java.security.AccessController.doPrivileged(Native Method)
> >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> >>> at java.lang.Class.forName0(Native Method)
> >>> at java.lang.Class.forName(Class.java:169)
> >>> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
> >>> at
> >>>
> >>
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
> >>> at
> >>>
> >>
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
> >>> at
> >>>
> >>
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
> >>> at
> >>>
> >>
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
> >>> at
> >>>
> >>
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
> >>> at
> >>>
> >>
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
> >>> at
> >>>
> >>
> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
> >>> at org.apache.pig.Main.configureLog4J(Main.java:678)
> >>> at org.apache.pig.Main.run(Main.java:337)
> >>> at org.apache.pig.Main.main(Main.java:111)
> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>> at
> >>>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >>> at
> >>>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>> at java.lang.reflect.Method.invoke(Method.java:597)
> >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> >>> log4j:ERROR Could not instantiate appender named "flume".
> >>>
> >>> Could you please help me find what I miss or where is incorrect for the
> >>> configuration.
> >>>
> >>> Thanks a lot!
> >>> Dongliang
> >>
> >> --
> >> Alexander Alten-Lorenz
> >> http://mapredit.blogspot.com
> >> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
> >>
> >>
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
Can you please post the running flume process (ps waux|grep flume)?
- Alex
On Jan 14, 2013, at 8:11 AM, Dongliang Sun <da...@gmail.com> wrote:
> Alex,
>
> The flume startup no problem,
> flume-ng agent --conf ./conf/ --conf-file ./conf/flume.conf --name agent1
> -Dflume.root.logger=DEBUG,console
>
> I have test the following client,
> bin/flume-ng avro-client -H localhost -p 41414 -F ~/test.txt
>
> It's OK, but if I can not make it work if I use the log4jappender as the
> client.
>
> Thanks,
> Dongliang
>
>
> 2013/1/14 Alexander Alten-Lorenz <wg...@gmail.com>
>
>> Please post the flume startup line. I guess you have missed the --conf
>> switch.
>>
>> - Alex
>>
>> On Jan 14, 2013, at 4:25 AM, 孙东亮 <da...@gmail.com> wrote:
>>
>>> I'm a newbie for the Flume, and I just set up flume for a test.
>>> I want to use the log4jappender to get log info from PIG scripts, the
>>> log4j.properties is:
>>>
>>> log4j.appender.flume =
>> org.apache.flume.clients.log4jappender.Log4jAppender
>>> log4j.appender.flume.Hostname = localhost
>>> log4j.appender.flume.Port = 41414
>>> log4j.logger.org.apache.pig=DEBUG,flume
>>>
>>> And for the flume.config:
>>>
>>> a1.sources = r1
>>> a1.sinks = k1
>>> a1.channels = c1
>>>
>>> # Describe/configure the source
>>> a1.sources.r1.type = avro
>>> a1.sources.r1.bind = localhost
>>> a1.sources.r1.port = 41414
>>>
>>> # Describe the sink
>>> a1.sinks.k1.type = logger
>>> # Use a channel which buffers events in memory
>>> a1.channels.c1.type = memory
>>> a1.channels.c1.capacity = 1000
>>> a1.channels.c1.transactionCapacity = 100
>>>
>>> # Bind the source and sink to the channel
>>> a1.sources.r1.channels = c1
>>> a1.sinks.k1.channel = c1
>>>
>>> But I got the following error when run a pig script:
>>> java.lang.ClassNotFoundException:
>>> org.apache.flume.clients.log4jappender.Log4jAppender
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>> at java.lang.Class.forName0(Native Method)
>>> at java.lang.Class.forName(Class.java:169)
>>> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
>>> at
>>>
>> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
>>> at
>>>
>> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
>>> at
>>>
>> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
>>> at
>>>
>> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
>>> at
>>>
>> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
>>> at
>>>
>> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
>>> at
>>>
>> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
>>> at org.apache.pig.Main.configureLog4J(Main.java:678)
>>> at org.apache.pig.Main.run(Main.java:337)
>>> at org.apache.pig.Main.main(Main.java:111)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at
>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>> log4j:ERROR Could not instantiate appender named "flume".
>>>
>>> Could you please help me find what I miss or where is incorrect for the
>>> configuration.
>>>
>>> Thanks a lot!
>>> Dongliang
>>
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>
>>
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Dongliang Sun <da...@gmail.com>.
Alex,
The flume startup no problem,
flume-ng agent --conf ./conf/ --conf-file ./conf/flume.conf --name agent1
-Dflume.root.logger=DEBUG,console
I have test the following client,
bin/flume-ng avro-client -H localhost -p 41414 -F ~/test.txt
It's OK, but if I can not make it work if I use the log4jappender as the
client.
Thanks,
Dongliang
2013/1/14 Alexander Alten-Lorenz <wg...@gmail.com>
> Please post the flume startup line. I guess you have missed the --conf
> switch.
>
> - Alex
>
> On Jan 14, 2013, at 4:25 AM, 孙东亮 <da...@gmail.com> wrote:
>
> > I'm a newbie for the Flume, and I just set up flume for a test.
> > I want to use the log4jappender to get log info from PIG scripts, the
> > log4j.properties is:
> >
> > log4j.appender.flume =
> org.apache.flume.clients.log4jappender.Log4jAppender
> > log4j.appender.flume.Hostname = localhost
> > log4j.appender.flume.Port = 41414
> > log4j.logger.org.apache.pig=DEBUG,flume
> >
> > And for the flume.config:
> >
> > a1.sources = r1
> > a1.sinks = k1
> > a1.channels = c1
> >
> > # Describe/configure the source
> > a1.sources.r1.type = avro
> > a1.sources.r1.bind = localhost
> > a1.sources.r1.port = 41414
> >
> > # Describe the sink
> > a1.sinks.k1.type = logger
> > # Use a channel which buffers events in memory
> > a1.channels.c1.type = memory
> > a1.channels.c1.capacity = 1000
> > a1.channels.c1.transactionCapacity = 100
> >
> > # Bind the source and sink to the channel
> > a1.sources.r1.channels = c1
> > a1.sinks.k1.channel = c1
> >
> > But I got the following error when run a pig script:
> > java.lang.ClassNotFoundException:
> > org.apache.flume.clients.log4jappender.Log4jAppender
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> > at java.lang.Class.forName0(Native Method)
> > at java.lang.Class.forName(Class.java:169)
> > at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
> > at
> >
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
> > at
> >
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
> > at
> >
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
> > at
> >
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
> > at
> >
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
> > at
> >
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
> > at
> >
> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
> > at org.apache.pig.Main.configureLog4J(Main.java:678)
> > at org.apache.pig.Main.run(Main.java:337)
> > at org.apache.pig.Main.main(Main.java:111)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > log4j:ERROR Could not instantiate appender named "flume".
> >
> > Could you please help me find what I miss or where is incorrect for the
> > configuration.
> >
> > Thanks a lot!
> > Dongliang
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>
Re: Could not instantiate class org.apache.flume.clients.log4jappender
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
Please post the flume startup line. I guess you have missed the --conf switch.
- Alex
On Jan 14, 2013, at 4:25 AM, 孙东亮 <da...@gmail.com> wrote:
> I'm a newbie for the Flume, and I just set up flume for a test.
> I want to use the log4jappender to get log info from PIG scripts, the
> log4j.properties is:
>
> log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender
> log4j.appender.flume.Hostname = localhost
> log4j.appender.flume.Port = 41414
> log4j.logger.org.apache.pig=DEBUG,flume
>
> And for the flume.config:
>
> a1.sources = r1
> a1.sinks = k1
> a1.channels = c1
>
> # Describe/configure the source
> a1.sources.r1.type = avro
> a1.sources.r1.bind = localhost
> a1.sources.r1.port = 41414
>
> # Describe the sink
> a1.sinks.k1.type = logger
> # Use a channel which buffers events in memory
> a1.channels.c1.type = memory
> a1.channels.c1.capacity = 1000
> a1.channels.c1.transactionCapacity = 100
>
> # Bind the source and sink to the channel
> a1.sources.r1.channels = c1
> a1.sinks.k1.channel = c1
>
> But I got the following error when run a pig script:
> java.lang.ClassNotFoundException:
> org.apache.flume.clients.log4jappender.Log4jAppender
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121)
> at
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664)
> at
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
> at
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
> at
> org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
> at org.apache.pig.Main.configureLog4J(Main.java:678)
> at org.apache.pig.Main.run(Main.java:337)
> at org.apache.pig.Main.main(Main.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> log4j:ERROR Could not instantiate appender named "flume".
>
> Could you please help me find what I miss or where is incorrect for the
> configuration.
>
> Thanks a lot!
> Dongliang
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF