You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Alex Melville <am...@g.hmc.edu> on 2014/11/05 08:25:51 UTC
Location of Logging Files/How To Turn On Logging For Kafka Components
Background:
I have searched for a while online, and through the files located in the
kafka/logs directory, trying to find where kafka writes log output to in
order to debug the SimpleProducer I wrote. My producer is almost identical
to the simple producer located here
https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Example
except for I'm using Protobuffers instead of Strings to publish data to a
cluster. I'm receiving the following error when I try to run the
SimpleProducer
Exception in thread "main" kafka.common.FailedToSendMessageException:
Failed to send messages after 3 tries.
at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
at kafka.producer.Producer.send(Producer.scala:76)
at kafka.javaapi.producer.Producer.send(Producer.scala:33)
at stream.SimpleProducer.send(Unknown Source)
at stream.SimpleProducer.main(Unknown Source)
I know this isn't a network problem, because I ran the console-producer and
successfully published data to the same broker that my Simple Producer is
trying to publish to. I now want to try to debug this error.
Question:
Where would my Simple Producer write info about its startup and eventual
error, such that I can read it and try to reason as to why it failed? If it
produces no log data on its own, what is the best way to write this data to
a somewhere where I can use it to debug? I've noticed that log4j, which I
understand is a often-used library for logging in Java, came with my kafka
download. Am I supposed to use log4j for this info? I do not know very much
about log4j, so any info on how to get this setup would also be appreciated.
-Alex
Re: Location of Logging Files/How To Turn On Logging For Kafka Components
Posted by Jun Rao <ju...@gmail.com>.
You need to set the location of log4j.properties in a JVM variable. Take a
look at the kafka scripts in bin/
Thanks,
Jun
On Thu, Nov 13, 2014 at 10:53 PM, Alex Melville <am...@g.hmc.edu> wrote:
> Hi Jun,
>
> These are the two lines of log4j-related warnings I get when I try to run
> my producer:
>
> log4j:WARN No appenders could be found for logger
> (kafka.utils.VerifiableProperties).
>
> log4j:WARN Please initialize the log4j system properly.
>
>
> I have searched extensively online and have so far not found how to
> "initialize the log4j system" properly. All I want is to create debug
> logging so I can better find why my producer fails to send messages to the
> broker cluster.
>
>
>
> Alex
>
> On Thu, Nov 6, 2014 at 3:31 PM, Jun Rao <ju...@gmail.com> wrote:
>
> > The log4j entries before that error should tell you the cause of the
> error.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Nov 4, 2014 at 11:25 PM, Alex Melville <am...@g.hmc.edu>
> > wrote:
> >
> > > Background:
> > >
> > > I have searched for a while online, and through the files located in
> the
> > > kafka/logs directory, trying to find where kafka writes log output to
> in
> > > order to debug the SimpleProducer I wrote. My producer is almost
> > identical
> > > to the simple producer located here
> > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Example
> > >
> > > except for I'm using Protobuffers instead of Strings to publish data
> to a
> > > cluster. I'm receiving the following error when I try to run the
> > > SimpleProducer
> > >
> > > Exception in thread "main" kafka.common.FailedToSendMessageException:
> > > Failed to send messages after 3 tries.
> > >
> > > at
> > >
> > >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
> > >
> > > at kafka.producer.Producer.send(Producer.scala:76)
> > >
> > > at kafka.javaapi.producer.Producer.send(Producer.scala:33)
> > >
> > > at stream.SimpleProducer.send(Unknown Source)
> > >
> > > at stream.SimpleProducer.main(Unknown Source)
> > >
> > >
> > > I know this isn't a network problem, because I ran the console-producer
> > and
> > > successfully published data to the same broker that my Simple Producer
> is
> > > trying to publish to. I now want to try to debug this error.
> > >
> > >
> > >
> > > Question:
> > >
> > > Where would my Simple Producer write info about its startup and
> eventual
> > > error, such that I can read it and try to reason as to why it failed?
> If
> > it
> > > produces no log data on its own, what is the best way to write this
> data
> > to
> > > a somewhere where I can use it to debug? I've noticed that log4j,
> which I
> > > understand is a often-used library for logging in Java, came with my
> > kafka
> > > download. Am I supposed to use log4j for this info? I do not know very
> > much
> > > about log4j, so any info on how to get this setup would also be
> > > appreciated.
> > >
> > >
> > > -Alex
> > >
> >
>
Re: Location of Logging Files/How To Turn On Logging For Kafka Components
Posted by Alex Melville <am...@g.hmc.edu>.
Hi Jun,
These are the two lines of log4j-related warnings I get when I try to run
my producer:
log4j:WARN No appenders could be found for logger
(kafka.utils.VerifiableProperties).
log4j:WARN Please initialize the log4j system properly.
I have searched extensively online and have so far not found how to
"initialize the log4j system" properly. All I want is to create debug
logging so I can better find why my producer fails to send messages to the
broker cluster.
Alex
On Thu, Nov 6, 2014 at 3:31 PM, Jun Rao <ju...@gmail.com> wrote:
> The log4j entries before that error should tell you the cause of the error.
>
> Thanks,
>
> Jun
>
> On Tue, Nov 4, 2014 at 11:25 PM, Alex Melville <am...@g.hmc.edu>
> wrote:
>
> > Background:
> >
> > I have searched for a while online, and through the files located in the
> > kafka/logs directory, trying to find where kafka writes log output to in
> > order to debug the SimpleProducer I wrote. My producer is almost
> identical
> > to the simple producer located here
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Example
> >
> > except for I'm using Protobuffers instead of Strings to publish data to a
> > cluster. I'm receiving the following error when I try to run the
> > SimpleProducer
> >
> > Exception in thread "main" kafka.common.FailedToSendMessageException:
> > Failed to send messages after 3 tries.
> >
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
> >
> > at kafka.producer.Producer.send(Producer.scala:76)
> >
> > at kafka.javaapi.producer.Producer.send(Producer.scala:33)
> >
> > at stream.SimpleProducer.send(Unknown Source)
> >
> > at stream.SimpleProducer.main(Unknown Source)
> >
> >
> > I know this isn't a network problem, because I ran the console-producer
> and
> > successfully published data to the same broker that my Simple Producer is
> > trying to publish to. I now want to try to debug this error.
> >
> >
> >
> > Question:
> >
> > Where would my Simple Producer write info about its startup and eventual
> > error, such that I can read it and try to reason as to why it failed? If
> it
> > produces no log data on its own, what is the best way to write this data
> to
> > a somewhere where I can use it to debug? I've noticed that log4j, which I
> > understand is a often-used library for logging in Java, came with my
> kafka
> > download. Am I supposed to use log4j for this info? I do not know very
> much
> > about log4j, so any info on how to get this setup would also be
> > appreciated.
> >
> >
> > -Alex
> >
>
Re: Location of Logging Files/How To Turn On Logging For Kafka Components
Posted by Jun Rao <ju...@gmail.com>.
The log4j entries before that error should tell you the cause of the error.
Thanks,
Jun
On Tue, Nov 4, 2014 at 11:25 PM, Alex Melville <am...@g.hmc.edu> wrote:
> Background:
>
> I have searched for a while online, and through the files located in the
> kafka/logs directory, trying to find where kafka writes log output to in
> order to debug the SimpleProducer I wrote. My producer is almost identical
> to the simple producer located here
>
> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Example
>
> except for I'm using Protobuffers instead of Strings to publish data to a
> cluster. I'm receiving the following error when I try to run the
> SimpleProducer
>
> Exception in thread "main" kafka.common.FailedToSendMessageException:
> Failed to send messages after 3 tries.
>
> at
>
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
>
> at kafka.producer.Producer.send(Producer.scala:76)
>
> at kafka.javaapi.producer.Producer.send(Producer.scala:33)
>
> at stream.SimpleProducer.send(Unknown Source)
>
> at stream.SimpleProducer.main(Unknown Source)
>
>
> I know this isn't a network problem, because I ran the console-producer and
> successfully published data to the same broker that my Simple Producer is
> trying to publish to. I now want to try to debug this error.
>
>
>
> Question:
>
> Where would my Simple Producer write info about its startup and eventual
> error, such that I can read it and try to reason as to why it failed? If it
> produces no log data on its own, what is the best way to write this data to
> a somewhere where I can use it to debug? I've noticed that log4j, which I
> understand is a often-used library for logging in Java, came with my kafka
> download. Am I supposed to use log4j for this info? I do not know very much
> about log4j, so any info on how to get this setup would also be
> appreciated.
>
>
> -Alex
>