You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jay Kreps <ja...@gmail.com> on 2014/02/03 20:00:32 UTC
Logging in new clients
We probably need to add a small amount of logging in the new producer and
(soon) consumer clients. I wanted to have a quick discussion on logging
libraries before I start adding this in the producer.
Previously we have been pretty verbose loggers and I think we should stop
that. For clients you mostly don't need to log: if there is an error you
should throw it back not log it, so you don't need ERROR logging. Likewise
I think it is rude to pollute peoples logs with the details of client
initialization (establishing connections, etc), so you don't need INFO
logging. However perhaps there is an argument to be made for WARN and
DEBUG. I think it is perhaps useful to log a WARN when a server breaks a
connection or metadata initialization fails. It can sometimes also be
useful to be able to enable debug logging to see step by step processing in
the client, which is the case for DEBUG logging.
Here is my knowledge about the state of Java logging:
1. Most people still use log4j. The server is using log4j.
2. Second runner-up in slf4j. I personally consider slf4j pretty silly but
this is perhaps the more flexible choice since people can plug in different
stuff.
3. java.util.logging ships with the jdk, but for some reason no one uses it.
4. There is no critical mass around any other logging library.
The context for how to think about this is the following. We are not trying
to pick the "best" logging library. Fundamentally logging is pretty
straight-forward and for our simple use case it is inconceivable that any
particular library could be much better than any other in terms of feature
set. We want the most standard library. My goal is to minimize the
dependencies of the client and make our basic logging cases work for most
cases.
Is there a reason not to just the java.util.logging? It comes with the jdk
and supports pluggable appenders so people who have some other library can
plug that in right?
Basically my preference would be java.util.logging unless there is some
known problem with it, otherwise I guess slf4j, and if not that then log4j.
Thougts?
-Jay
Re: Logging in new clients
Posted by Steve Morin <st...@stevemorin.com>.
My vote would be with log4j, I don't have that much experience with log4j2
or a good feel for how much the industry is moving towards it.
On Mon, Feb 3, 2014 at 11:17 AM, Joel Koshy <jj...@gmail.com> wrote:
> We are already using other libraries in various parts of our code
> (e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
> other logging dependencies anyway. i.e., what do we gain by using jul?
> There may be a good reason why people don't use jul so I think we
> should fully understand that before going with jul. So it may be
> simpler to just stick with log4j for the client rewrites and
> investigate logging later.
>
> log4j2 is becoming more widespread and many users seem to be favorable
> toward logback. slf4j would cover all of these very easily. From what
> I understand jul does not make it very easy to plug in with these
> various options but I could be wrong.
>
> I completely agree on the need to fix our client logging as that will
> go a long way in usability for end-users unless we want to keep
> getting asked the "Why do I see this ERROR in my logs..?" questions.
>
> Joel
>
> On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> > >> Basically my preference would be java.util.logging unless there is
> some
> > known problem with it, otherwise I guess slf4j, and if not that then
> log4j.
> >
> > +1. My preference is to use java.util.logging to avoid adding an external
> > dependency,
> > but I'm not too sure about what's the "standard" out there, so open to
> > suggestions
> > on picking a different library.
> >
> >
> >
> > On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com> wrote:
> >
> > > We probably need to add a small amount of logging in the new producer
> and
> > > (soon) consumer clients. I wanted to have a quick discussion on logging
> > > libraries before I start adding this in the producer.
> > >
> > > Previously we have been pretty verbose loggers and I think we should
> stop
> > > that. For clients you mostly don't need to log: if there is an error
> you
> > > should throw it back not log it, so you don't need ERROR logging.
> Likewise
> > > I think it is rude to pollute peoples logs with the details of client
> > > initialization (establishing connections, etc), so you don't need INFO
> > > logging. However perhaps there is an argument to be made for WARN and
> > > DEBUG. I think it is perhaps useful to log a WARN when a server breaks
> a
> > > connection or metadata initialization fails. It can sometimes also be
> > > useful to be able to enable debug logging to see step by step
> processing in
> > > the client, which is the case for DEBUG logging.
> > >
> > > Here is my knowledge about the state of Java logging:
> > > 1. Most people still use log4j. The server is using log4j.
> > > 2. Second runner-up in slf4j. I personally consider slf4j pretty silly
> but
> > > this is perhaps the more flexible choice since people can plug in
> different
> > > stuff.
> > > 3. java.util.logging ships with the jdk, but for some reason no one
> uses
> > > it.
> > > 4. There is no critical mass around any other logging library.
> > >
> > > The context for how to think about this is the following. We are not
> trying
> > > to pick the "best" logging library. Fundamentally logging is pretty
> > > straight-forward and for our simple use case it is inconceivable that
> any
> > > particular library could be much better than any other in terms of
> feature
> > > set. We want the most standard library. My goal is to minimize the
> > > dependencies of the client and make our basic logging cases work for
> most
> > > cases.
> > >
> > > Is there a reason not to just the java.util.logging? It comes with the
> jdk
> > > and supports pluggable appenders so people who have some other library
> can
> > > plug that in right?
> > >
> > > Basically my preference would be java.util.logging unless there is some
> > > known problem with it, otherwise I guess slf4j, and if not that then
> log4j.
> > >
> > > Thougts?
> > >
> > > -Jay
> > >
>
>
Re: Logging in new clients
Posted by Jay Kreps <ja...@gmail.com>.
Alright, I'm sold. If no objections, slf4j it is.
-Jay
On Mon, Feb 3, 2014 at 1:11 PM, Joel Koshy <jj...@gmail.com> wrote:
> > I see server and client logging and dependencies are really different in
> my
> > mind. The server is a standalone process. Once we decouple the clients I
> > think we can actually be a lot more aggressive about using more libraries
> > on the server, why not? We have avoided that so far because they
> inevitably
> > leak into the clients since they arent separated.
> >
> > The problem on the client side is that if we choose anything used by our
> > clients and there is any incompatibility then they can't use kafka.
> >
> > From my point of view the improved logging libraries (log4j2, logback,
> etc)
> > are the worst option. They are new so likely will have api changes, and
> few
> > people use them so they don't get the benefit of sharing the same logging
> > library.
>
> Right - I was actually advocating sl4fj for this very reason (not a
> specific logging implementation).
>
> Clients would use the producer/consumer as part of some larger system
> that is more than likely to use something other than j.u.l (since no
> one seems to use it). With a layer like slf4j clients can choose
> whatever logging implementation they wish. If we go with j.u.l then
> we would force all clients to include a j.u.l configuration file just
> for their producer/consumer in addition to the configuration file for
> the rest of their system.
>
> Thanks,
>
> Joel
>
> >
> > On Mon, Feb 3, 2014 at 11:17 AM, Joel Koshy <jj...@gmail.com> wrote:
> >
> > > We are already using other libraries in various parts of our code
> > > (e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
> > > other logging dependencies anyway. i.e., what do we gain by using jul?
> > > There may be a good reason why people don't use jul so I think we
> > > should fully understand that before going with jul. So it may be
> > > simpler to just stick with log4j for the client rewrites and
> > > investigate logging later.
> > >
> > > log4j2 is becoming more widespread and many users seem to be favorable
> > > toward logback. slf4j would cover all of these very easily. From what
> > > I understand jul does not make it very easy to plug in with these
> > > various options but I could be wrong.
> > >
> > > I completely agree on the need to fix our client logging as that will
> > > go a long way in usability for end-users unless we want to keep
> > > getting asked the "Why do I see this ERROR in my logs..?" questions.
> > >
> > > Joel
> > >
> > > On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> > > > >> Basically my preference would be java.util.logging unless there is
> > > some
> > > > known problem with it, otherwise I guess slf4j, and if not that then
> > > log4j.
> > > >
> > > > +1. My preference is to use java.util.logging to avoid adding an
> external
> > > > dependency,
> > > > but I'm not too sure about what's the "standard" out there, so open
> to
> > > > suggestions
> > > > on picking a different library.
> > > >
> > > >
> > > >
> > > > On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com>
> wrote:
> > > >
> > > > > We probably need to add a small amount of logging in the new
> producer
> > > and
> > > > > (soon) consumer clients. I wanted to have a quick discussion on
> logging
> > > > > libraries before I start adding this in the producer.
> > > > >
> > > > > Previously we have been pretty verbose loggers and I think we
> should
> > > stop
> > > > > that. For clients you mostly don't need to log: if there is an
> error
> > > you
> > > > > should throw it back not log it, so you don't need ERROR logging.
> > > Likewise
> > > > > I think it is rude to pollute peoples logs with the details of
> client
> > > > > initialization (establishing connections, etc), so you don't need
> INFO
> > > > > logging. However perhaps there is an argument to be made for WARN
> and
> > > > > DEBUG. I think it is perhaps useful to log a WARN when a server
> breaks
> > > a
> > > > > connection or metadata initialization fails. It can sometimes also
> be
> > > > > useful to be able to enable debug logging to see step by step
> > > processing in
> > > > > the client, which is the case for DEBUG logging.
> > > > >
> > > > > Here is my knowledge about the state of Java logging:
> > > > > 1. Most people still use log4j. The server is using log4j.
> > > > > 2. Second runner-up in slf4j. I personally consider slf4j pretty
> silly
> > > but
> > > > > this is perhaps the more flexible choice since people can plug in
> > > different
> > > > > stuff.
> > > > > 3. java.util.logging ships with the jdk, but for some reason no one
> > > uses
> > > > > it.
> > > > > 4. There is no critical mass around any other logging library.
> > > > >
> > > > > The context for how to think about this is the following. We are
> not
> > > trying
> > > > > to pick the "best" logging library. Fundamentally logging is pretty
> > > > > straight-forward and for our simple use case it is inconceivable
> that
> > > any
> > > > > particular library could be much better than any other in terms of
> > > feature
> > > > > set. We want the most standard library. My goal is to minimize the
> > > > > dependencies of the client and make our basic logging cases work
> for
> > > most
> > > > > cases.
> > > > >
> > > > > Is there a reason not to just the java.util.logging? It comes with
> the
> > > jdk
> > > > > and supports pluggable appenders so people who have some other
> library
> > > can
> > > > > plug that in right?
> > > > >
> > > > > Basically my preference would be java.util.logging unless there is
> some
> > > > > known problem with it, otherwise I guess slf4j, and if not that
> then
> > > log4j.
> > > > >
> > > > > Thougts?
> > > > >
> > > > > -Jay
> > > > >
> > >
> > >
>
>
Re: Logging in new clients
Posted by Jay Kreps <ja...@gmail.com>.
Alright, I'm sold. If no objections, slf4j it is.
-Jay
On Mon, Feb 3, 2014 at 1:11 PM, Joel Koshy <jj...@gmail.com> wrote:
> > I see server and client logging and dependencies are really different in
> my
> > mind. The server is a standalone process. Once we decouple the clients I
> > think we can actually be a lot more aggressive about using more libraries
> > on the server, why not? We have avoided that so far because they
> inevitably
> > leak into the clients since they arent separated.
> >
> > The problem on the client side is that if we choose anything used by our
> > clients and there is any incompatibility then they can't use kafka.
> >
> > From my point of view the improved logging libraries (log4j2, logback,
> etc)
> > are the worst option. They are new so likely will have api changes, and
> few
> > people use them so they don't get the benefit of sharing the same logging
> > library.
>
> Right - I was actually advocating sl4fj for this very reason (not a
> specific logging implementation).
>
> Clients would use the producer/consumer as part of some larger system
> that is more than likely to use something other than j.u.l (since no
> one seems to use it). With a layer like slf4j clients can choose
> whatever logging implementation they wish. If we go with j.u.l then
> we would force all clients to include a j.u.l configuration file just
> for their producer/consumer in addition to the configuration file for
> the rest of their system.
>
> Thanks,
>
> Joel
>
> >
> > On Mon, Feb 3, 2014 at 11:17 AM, Joel Koshy <jj...@gmail.com> wrote:
> >
> > > We are already using other libraries in various parts of our code
> > > (e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
> > > other logging dependencies anyway. i.e., what do we gain by using jul?
> > > There may be a good reason why people don't use jul so I think we
> > > should fully understand that before going with jul. So it may be
> > > simpler to just stick with log4j for the client rewrites and
> > > investigate logging later.
> > >
> > > log4j2 is becoming more widespread and many users seem to be favorable
> > > toward logback. slf4j would cover all of these very easily. From what
> > > I understand jul does not make it very easy to plug in with these
> > > various options but I could be wrong.
> > >
> > > I completely agree on the need to fix our client logging as that will
> > > go a long way in usability for end-users unless we want to keep
> > > getting asked the "Why do I see this ERROR in my logs..?" questions.
> > >
> > > Joel
> > >
> > > On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> > > > >> Basically my preference would be java.util.logging unless there is
> > > some
> > > > known problem with it, otherwise I guess slf4j, and if not that then
> > > log4j.
> > > >
> > > > +1. My preference is to use java.util.logging to avoid adding an
> external
> > > > dependency,
> > > > but I'm not too sure about what's the "standard" out there, so open
> to
> > > > suggestions
> > > > on picking a different library.
> > > >
> > > >
> > > >
> > > > On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com>
> wrote:
> > > >
> > > > > We probably need to add a small amount of logging in the new
> producer
> > > and
> > > > > (soon) consumer clients. I wanted to have a quick discussion on
> logging
> > > > > libraries before I start adding this in the producer.
> > > > >
> > > > > Previously we have been pretty verbose loggers and I think we
> should
> > > stop
> > > > > that. For clients you mostly don't need to log: if there is an
> error
> > > you
> > > > > should throw it back not log it, so you don't need ERROR logging.
> > > Likewise
> > > > > I think it is rude to pollute peoples logs with the details of
> client
> > > > > initialization (establishing connections, etc), so you don't need
> INFO
> > > > > logging. However perhaps there is an argument to be made for WARN
> and
> > > > > DEBUG. I think it is perhaps useful to log a WARN when a server
> breaks
> > > a
> > > > > connection or metadata initialization fails. It can sometimes also
> be
> > > > > useful to be able to enable debug logging to see step by step
> > > processing in
> > > > > the client, which is the case for DEBUG logging.
> > > > >
> > > > > Here is my knowledge about the state of Java logging:
> > > > > 1. Most people still use log4j. The server is using log4j.
> > > > > 2. Second runner-up in slf4j. I personally consider slf4j pretty
> silly
> > > but
> > > > > this is perhaps the more flexible choice since people can plug in
> > > different
> > > > > stuff.
> > > > > 3. java.util.logging ships with the jdk, but for some reason no one
> > > uses
> > > > > it.
> > > > > 4. There is no critical mass around any other logging library.
> > > > >
> > > > > The context for how to think about this is the following. We are
> not
> > > trying
> > > > > to pick the "best" logging library. Fundamentally logging is pretty
> > > > > straight-forward and for our simple use case it is inconceivable
> that
> > > any
> > > > > particular library could be much better than any other in terms of
> > > feature
> > > > > set. We want the most standard library. My goal is to minimize the
> > > > > dependencies of the client and make our basic logging cases work
> for
> > > most
> > > > > cases.
> > > > >
> > > > > Is there a reason not to just the java.util.logging? It comes with
> the
> > > jdk
> > > > > and supports pluggable appenders so people who have some other
> library
> > > can
> > > > > plug that in right?
> > > > >
> > > > > Basically my preference would be java.util.logging unless there is
> some
> > > > > known problem with it, otherwise I guess slf4j, and if not that
> then
> > > log4j.
> > > > >
> > > > > Thougts?
> > > > >
> > > > > -Jay
> > > > >
> > >
> > >
>
>
Re: Logging in new clients
Posted by Joel Koshy <jj...@gmail.com>.
> I see server and client logging and dependencies are really different in my
> mind. The server is a standalone process. Once we decouple the clients I
> think we can actually be a lot more aggressive about using more libraries
> on the server, why not? We have avoided that so far because they inevitably
> leak into the clients since they arent separated.
>
> The problem on the client side is that if we choose anything used by our
> clients and there is any incompatibility then they can't use kafka.
>
> From my point of view the improved logging libraries (log4j2, logback, etc)
> are the worst option. They are new so likely will have api changes, and few
> people use them so they don't get the benefit of sharing the same logging
> library.
Right - I was actually advocating sl4fj for this very reason (not a
specific logging implementation).
Clients would use the producer/consumer as part of some larger system
that is more than likely to use something other than j.u.l (since no
one seems to use it). With a layer like slf4j clients can choose
whatever logging implementation they wish. If we go with j.u.l then
we would force all clients to include a j.u.l configuration file just
for their producer/consumer in addition to the configuration file for
the rest of their system.
Thanks,
Joel
>
> On Mon, Feb 3, 2014 at 11:17 AM, Joel Koshy <jj...@gmail.com> wrote:
>
> > We are already using other libraries in various parts of our code
> > (e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
> > other logging dependencies anyway. i.e., what do we gain by using jul?
> > There may be a good reason why people don't use jul so I think we
> > should fully understand that before going with jul. So it may be
> > simpler to just stick with log4j for the client rewrites and
> > investigate logging later.
> >
> > log4j2 is becoming more widespread and many users seem to be favorable
> > toward logback. slf4j would cover all of these very easily. From what
> > I understand jul does not make it very easy to plug in with these
> > various options but I could be wrong.
> >
> > I completely agree on the need to fix our client logging as that will
> > go a long way in usability for end-users unless we want to keep
> > getting asked the "Why do I see this ERROR in my logs..?" questions.
> >
> > Joel
> >
> > On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> > > >> Basically my preference would be java.util.logging unless there is
> > some
> > > known problem with it, otherwise I guess slf4j, and if not that then
> > log4j.
> > >
> > > +1. My preference is to use java.util.logging to avoid adding an external
> > > dependency,
> > > but I'm not too sure about what's the "standard" out there, so open to
> > > suggestions
> > > on picking a different library.
> > >
> > >
> > >
> > > On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com> wrote:
> > >
> > > > We probably need to add a small amount of logging in the new producer
> > and
> > > > (soon) consumer clients. I wanted to have a quick discussion on logging
> > > > libraries before I start adding this in the producer.
> > > >
> > > > Previously we have been pretty verbose loggers and I think we should
> > stop
> > > > that. For clients you mostly don't need to log: if there is an error
> > you
> > > > should throw it back not log it, so you don't need ERROR logging.
> > Likewise
> > > > I think it is rude to pollute peoples logs with the details of client
> > > > initialization (establishing connections, etc), so you don't need INFO
> > > > logging. However perhaps there is an argument to be made for WARN and
> > > > DEBUG. I think it is perhaps useful to log a WARN when a server breaks
> > a
> > > > connection or metadata initialization fails. It can sometimes also be
> > > > useful to be able to enable debug logging to see step by step
> > processing in
> > > > the client, which is the case for DEBUG logging.
> > > >
> > > > Here is my knowledge about the state of Java logging:
> > > > 1. Most people still use log4j. The server is using log4j.
> > > > 2. Second runner-up in slf4j. I personally consider slf4j pretty silly
> > but
> > > > this is perhaps the more flexible choice since people can plug in
> > different
> > > > stuff.
> > > > 3. java.util.logging ships with the jdk, but for some reason no one
> > uses
> > > > it.
> > > > 4. There is no critical mass around any other logging library.
> > > >
> > > > The context for how to think about this is the following. We are not
> > trying
> > > > to pick the "best" logging library. Fundamentally logging is pretty
> > > > straight-forward and for our simple use case it is inconceivable that
> > any
> > > > particular library could be much better than any other in terms of
> > feature
> > > > set. We want the most standard library. My goal is to minimize the
> > > > dependencies of the client and make our basic logging cases work for
> > most
> > > > cases.
> > > >
> > > > Is there a reason not to just the java.util.logging? It comes with the
> > jdk
> > > > and supports pluggable appenders so people who have some other library
> > can
> > > > plug that in right?
> > > >
> > > > Basically my preference would be java.util.logging unless there is some
> > > > known problem with it, otherwise I guess slf4j, and if not that then
> > log4j.
> > > >
> > > > Thougts?
> > > >
> > > > -Jay
> > > >
> >
> >
Re: Logging in new clients
Posted by Anatoly Fayngelerin <fa...@gmail.com>.
+1 for SLF4J. It is the only option that doesn't force a consumer to use a
particular style of configuration. It is common enough that I think it
would not cause any adoption friction. There's a relevant and lengthier
discussion here:
http://stackoverflow.com/questions/11359187/dont-get-it-why-not-jul
On Mon, Feb 3, 2014 at 3:00 PM, Jay Kreps <ja...@gmail.com> wrote:
> Hey Joel,
>
> I see server and client logging and dependencies are really different in my
> mind. The server is a standalone process. Once we decouple the clients I
> think we can actually be a lot more aggressive about using more libraries
> on the server, why not? We have avoided that so far because they inevitably
> leak into the clients since they arent separated.
>
> The problem on the client side is that if we choose anything used by our
> clients and there is any incompatibility then they can't use kafka.
>
> From my point of view the improved logging libraries (log4j2, logback, etc)
> are the worst option. They are new so likely will have api changes, and few
> people use them so they don't get the benefit of sharing the same logging
> library.
>
> -Jay
>
>
> On Mon, Feb 3, 2014 at 11:17 AM, Joel Koshy <jj...@gmail.com> wrote:
>
> > We are already using other libraries in various parts of our code
> > (e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
> > other logging dependencies anyway. i.e., what do we gain by using jul?
> > There may be a good reason why people don't use jul so I think we
> > should fully understand that before going with jul. So it may be
> > simpler to just stick with log4j for the client rewrites and
> > investigate logging later.
> >
> > log4j2 is becoming more widespread and many users seem to be favorable
> > toward logback. slf4j would cover all of these very easily. From what
> > I understand jul does not make it very easy to plug in with these
> > various options but I could be wrong.
> >
> > I completely agree on the need to fix our client logging as that will
> > go a long way in usability for end-users unless we want to keep
> > getting asked the "Why do I see this ERROR in my logs..?" questions.
> >
> > Joel
> >
> > On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> > > >> Basically my preference would be java.util.logging unless there is
> > some
> > > known problem with it, otherwise I guess slf4j, and if not that then
> > log4j.
> > >
> > > +1. My preference is to use java.util.logging to avoid adding an
> external
> > > dependency,
> > > but I'm not too sure about what's the "standard" out there, so open to
> > > suggestions
> > > on picking a different library.
> > >
> > >
> > >
> > > On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com>
> wrote:
> > >
> > > > We probably need to add a small amount of logging in the new producer
> > and
> > > > (soon) consumer clients. I wanted to have a quick discussion on
> logging
> > > > libraries before I start adding this in the producer.
> > > >
> > > > Previously we have been pretty verbose loggers and I think we should
> > stop
> > > > that. For clients you mostly don't need to log: if there is an error
> > you
> > > > should throw it back not log it, so you don't need ERROR logging.
> > Likewise
> > > > I think it is rude to pollute peoples logs with the details of client
> > > > initialization (establishing connections, etc), so you don't need
> INFO
> > > > logging. However perhaps there is an argument to be made for WARN and
> > > > DEBUG. I think it is perhaps useful to log a WARN when a server
> breaks
> > a
> > > > connection or metadata initialization fails. It can sometimes also be
> > > > useful to be able to enable debug logging to see step by step
> > processing in
> > > > the client, which is the case for DEBUG logging.
> > > >
> > > > Here is my knowledge about the state of Java logging:
> > > > 1. Most people still use log4j. The server is using log4j.
> > > > 2. Second runner-up in slf4j. I personally consider slf4j pretty
> silly
> > but
> > > > this is perhaps the more flexible choice since people can plug in
> > different
> > > > stuff.
> > > > 3. java.util.logging ships with the jdk, but for some reason no one
> > uses
> > > > it.
> > > > 4. There is no critical mass around any other logging library.
> > > >
> > > > The context for how to think about this is the following. We are not
> > trying
> > > > to pick the "best" logging library. Fundamentally logging is pretty
> > > > straight-forward and for our simple use case it is inconceivable that
> > any
> > > > particular library could be much better than any other in terms of
> > feature
> > > > set. We want the most standard library. My goal is to minimize the
> > > > dependencies of the client and make our basic logging cases work for
> > most
> > > > cases.
> > > >
> > > > Is there a reason not to just the java.util.logging? It comes with
> the
> > jdk
> > > > and supports pluggable appenders so people who have some other
> library
> > can
> > > > plug that in right?
> > > >
> > > > Basically my preference would be java.util.logging unless there is
> some
> > > > known problem with it, otherwise I guess slf4j, and if not that then
> > log4j.
> > > >
> > > > Thougts?
> > > >
> > > > -Jay
> > > >
> >
> >
>
Re: Logging in new clients
Posted by Joel Koshy <jj...@gmail.com>.
> I see server and client logging and dependencies are really different in my
> mind. The server is a standalone process. Once we decouple the clients I
> think we can actually be a lot more aggressive about using more libraries
> on the server, why not? We have avoided that so far because they inevitably
> leak into the clients since they arent separated.
>
> The problem on the client side is that if we choose anything used by our
> clients and there is any incompatibility then they can't use kafka.
>
> From my point of view the improved logging libraries (log4j2, logback, etc)
> are the worst option. They are new so likely will have api changes, and few
> people use them so they don't get the benefit of sharing the same logging
> library.
Right - I was actually advocating sl4fj for this very reason (not a
specific logging implementation).
Clients would use the producer/consumer as part of some larger system
that is more than likely to use something other than j.u.l (since no
one seems to use it). With a layer like slf4j clients can choose
whatever logging implementation they wish. If we go with j.u.l then
we would force all clients to include a j.u.l configuration file just
for their producer/consumer in addition to the configuration file for
the rest of their system.
Thanks,
Joel
>
> On Mon, Feb 3, 2014 at 11:17 AM, Joel Koshy <jj...@gmail.com> wrote:
>
> > We are already using other libraries in various parts of our code
> > (e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
> > other logging dependencies anyway. i.e., what do we gain by using jul?
> > There may be a good reason why people don't use jul so I think we
> > should fully understand that before going with jul. So it may be
> > simpler to just stick with log4j for the client rewrites and
> > investigate logging later.
> >
> > log4j2 is becoming more widespread and many users seem to be favorable
> > toward logback. slf4j would cover all of these very easily. From what
> > I understand jul does not make it very easy to plug in with these
> > various options but I could be wrong.
> >
> > I completely agree on the need to fix our client logging as that will
> > go a long way in usability for end-users unless we want to keep
> > getting asked the "Why do I see this ERROR in my logs..?" questions.
> >
> > Joel
> >
> > On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> > > >> Basically my preference would be java.util.logging unless there is
> > some
> > > known problem with it, otherwise I guess slf4j, and if not that then
> > log4j.
> > >
> > > +1. My preference is to use java.util.logging to avoid adding an external
> > > dependency,
> > > but I'm not too sure about what's the "standard" out there, so open to
> > > suggestions
> > > on picking a different library.
> > >
> > >
> > >
> > > On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com> wrote:
> > >
> > > > We probably need to add a small amount of logging in the new producer
> > and
> > > > (soon) consumer clients. I wanted to have a quick discussion on logging
> > > > libraries before I start adding this in the producer.
> > > >
> > > > Previously we have been pretty verbose loggers and I think we should
> > stop
> > > > that. For clients you mostly don't need to log: if there is an error
> > you
> > > > should throw it back not log it, so you don't need ERROR logging.
> > Likewise
> > > > I think it is rude to pollute peoples logs with the details of client
> > > > initialization (establishing connections, etc), so you don't need INFO
> > > > logging. However perhaps there is an argument to be made for WARN and
> > > > DEBUG. I think it is perhaps useful to log a WARN when a server breaks
> > a
> > > > connection or metadata initialization fails. It can sometimes also be
> > > > useful to be able to enable debug logging to see step by step
> > processing in
> > > > the client, which is the case for DEBUG logging.
> > > >
> > > > Here is my knowledge about the state of Java logging:
> > > > 1. Most people still use log4j. The server is using log4j.
> > > > 2. Second runner-up in slf4j. I personally consider slf4j pretty silly
> > but
> > > > this is perhaps the more flexible choice since people can plug in
> > different
> > > > stuff.
> > > > 3. java.util.logging ships with the jdk, but for some reason no one
> > uses
> > > > it.
> > > > 4. There is no critical mass around any other logging library.
> > > >
> > > > The context for how to think about this is the following. We are not
> > trying
> > > > to pick the "best" logging library. Fundamentally logging is pretty
> > > > straight-forward and for our simple use case it is inconceivable that
> > any
> > > > particular library could be much better than any other in terms of
> > feature
> > > > set. We want the most standard library. My goal is to minimize the
> > > > dependencies of the client and make our basic logging cases work for
> > most
> > > > cases.
> > > >
> > > > Is there a reason not to just the java.util.logging? It comes with the
> > jdk
> > > > and supports pluggable appenders so people who have some other library
> > can
> > > > plug that in right?
> > > >
> > > > Basically my preference would be java.util.logging unless there is some
> > > > known problem with it, otherwise I guess slf4j, and if not that then
> > log4j.
> > > >
> > > > Thougts?
> > > >
> > > > -Jay
> > > >
> >
> >
Re: Logging in new clients
Posted by Anatoly Fayngelerin <fa...@gmail.com>.
+1 for SLF4J. It is the only option that doesn't force a consumer to use a
particular style of configuration. It is common enough that I think it
would not cause any adoption friction. There's a relevant and lengthier
discussion here:
http://stackoverflow.com/questions/11359187/dont-get-it-why-not-jul
On Mon, Feb 3, 2014 at 3:00 PM, Jay Kreps <ja...@gmail.com> wrote:
> Hey Joel,
>
> I see server and client logging and dependencies are really different in my
> mind. The server is a standalone process. Once we decouple the clients I
> think we can actually be a lot more aggressive about using more libraries
> on the server, why not? We have avoided that so far because they inevitably
> leak into the clients since they arent separated.
>
> The problem on the client side is that if we choose anything used by our
> clients and there is any incompatibility then they can't use kafka.
>
> From my point of view the improved logging libraries (log4j2, logback, etc)
> are the worst option. They are new so likely will have api changes, and few
> people use them so they don't get the benefit of sharing the same logging
> library.
>
> -Jay
>
>
> On Mon, Feb 3, 2014 at 11:17 AM, Joel Koshy <jj...@gmail.com> wrote:
>
> > We are already using other libraries in various parts of our code
> > (e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
> > other logging dependencies anyway. i.e., what do we gain by using jul?
> > There may be a good reason why people don't use jul so I think we
> > should fully understand that before going with jul. So it may be
> > simpler to just stick with log4j for the client rewrites and
> > investigate logging later.
> >
> > log4j2 is becoming more widespread and many users seem to be favorable
> > toward logback. slf4j would cover all of these very easily. From what
> > I understand jul does not make it very easy to plug in with these
> > various options but I could be wrong.
> >
> > I completely agree on the need to fix our client logging as that will
> > go a long way in usability for end-users unless we want to keep
> > getting asked the "Why do I see this ERROR in my logs..?" questions.
> >
> > Joel
> >
> > On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> > > >> Basically my preference would be java.util.logging unless there is
> > some
> > > known problem with it, otherwise I guess slf4j, and if not that then
> > log4j.
> > >
> > > +1. My preference is to use java.util.logging to avoid adding an
> external
> > > dependency,
> > > but I'm not too sure about what's the "standard" out there, so open to
> > > suggestions
> > > on picking a different library.
> > >
> > >
> > >
> > > On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com>
> wrote:
> > >
> > > > We probably need to add a small amount of logging in the new producer
> > and
> > > > (soon) consumer clients. I wanted to have a quick discussion on
> logging
> > > > libraries before I start adding this in the producer.
> > > >
> > > > Previously we have been pretty verbose loggers and I think we should
> > stop
> > > > that. For clients you mostly don't need to log: if there is an error
> > you
> > > > should throw it back not log it, so you don't need ERROR logging.
> > Likewise
> > > > I think it is rude to pollute peoples logs with the details of client
> > > > initialization (establishing connections, etc), so you don't need
> INFO
> > > > logging. However perhaps there is an argument to be made for WARN and
> > > > DEBUG. I think it is perhaps useful to log a WARN when a server
> breaks
> > a
> > > > connection or metadata initialization fails. It can sometimes also be
> > > > useful to be able to enable debug logging to see step by step
> > processing in
> > > > the client, which is the case for DEBUG logging.
> > > >
> > > > Here is my knowledge about the state of Java logging:
> > > > 1. Most people still use log4j. The server is using log4j.
> > > > 2. Second runner-up in slf4j. I personally consider slf4j pretty
> silly
> > but
> > > > this is perhaps the more flexible choice since people can plug in
> > different
> > > > stuff.
> > > > 3. java.util.logging ships with the jdk, but for some reason no one
> > uses
> > > > it.
> > > > 4. There is no critical mass around any other logging library.
> > > >
> > > > The context for how to think about this is the following. We are not
> > trying
> > > > to pick the "best" logging library. Fundamentally logging is pretty
> > > > straight-forward and for our simple use case it is inconceivable that
> > any
> > > > particular library could be much better than any other in terms of
> > feature
> > > > set. We want the most standard library. My goal is to minimize the
> > > > dependencies of the client and make our basic logging cases work for
> > most
> > > > cases.
> > > >
> > > > Is there a reason not to just the java.util.logging? It comes with
> the
> > jdk
> > > > and supports pluggable appenders so people who have some other
> library
> > can
> > > > plug that in right?
> > > >
> > > > Basically my preference would be java.util.logging unless there is
> some
> > > > known problem with it, otherwise I guess slf4j, and if not that then
> > log4j.
> > > >
> > > > Thougts?
> > > >
> > > > -Jay
> > > >
> >
> >
>
Re: Logging in new clients
Posted by Jay Kreps <ja...@gmail.com>.
Hey Joel,
I see server and client logging and dependencies are really different in my
mind. The server is a standalone process. Once we decouple the clients I
think we can actually be a lot more aggressive about using more libraries
on the server, why not? We have avoided that so far because they inevitably
leak into the clients since they arent separated.
The problem on the client side is that if we choose anything used by our
clients and there is any incompatibility then they can't use kafka.
>From my point of view the improved logging libraries (log4j2, logback, etc)
are the worst option. They are new so likely will have api changes, and few
people use them so they don't get the benefit of sharing the same logging
library.
-Jay
On Mon, Feb 3, 2014 at 11:17 AM, Joel Koshy <jj...@gmail.com> wrote:
> We are already using other libraries in various parts of our code
> (e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
> other logging dependencies anyway. i.e., what do we gain by using jul?
> There may be a good reason why people don't use jul so I think we
> should fully understand that before going with jul. So it may be
> simpler to just stick with log4j for the client rewrites and
> investigate logging later.
>
> log4j2 is becoming more widespread and many users seem to be favorable
> toward logback. slf4j would cover all of these very easily. From what
> I understand jul does not make it very easy to plug in with these
> various options but I could be wrong.
>
> I completely agree on the need to fix our client logging as that will
> go a long way in usability for end-users unless we want to keep
> getting asked the "Why do I see this ERROR in my logs..?" questions.
>
> Joel
>
> On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> > >> Basically my preference would be java.util.logging unless there is
> some
> > known problem with it, otherwise I guess slf4j, and if not that then
> log4j.
> >
> > +1. My preference is to use java.util.logging to avoid adding an external
> > dependency,
> > but I'm not too sure about what's the "standard" out there, so open to
> > suggestions
> > on picking a different library.
> >
> >
> >
> > On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com> wrote:
> >
> > > We probably need to add a small amount of logging in the new producer
> and
> > > (soon) consumer clients. I wanted to have a quick discussion on logging
> > > libraries before I start adding this in the producer.
> > >
> > > Previously we have been pretty verbose loggers and I think we should
> stop
> > > that. For clients you mostly don't need to log: if there is an error
> you
> > > should throw it back not log it, so you don't need ERROR logging.
> Likewise
> > > I think it is rude to pollute peoples logs with the details of client
> > > initialization (establishing connections, etc), so you don't need INFO
> > > logging. However perhaps there is an argument to be made for WARN and
> > > DEBUG. I think it is perhaps useful to log a WARN when a server breaks
> a
> > > connection or metadata initialization fails. It can sometimes also be
> > > useful to be able to enable debug logging to see step by step
> processing in
> > > the client, which is the case for DEBUG logging.
> > >
> > > Here is my knowledge about the state of Java logging:
> > > 1. Most people still use log4j. The server is using log4j.
> > > 2. Second runner-up in slf4j. I personally consider slf4j pretty silly
> but
> > > this is perhaps the more flexible choice since people can plug in
> different
> > > stuff.
> > > 3. java.util.logging ships with the jdk, but for some reason no one
> uses
> > > it.
> > > 4. There is no critical mass around any other logging library.
> > >
> > > The context for how to think about this is the following. We are not
> trying
> > > to pick the "best" logging library. Fundamentally logging is pretty
> > > straight-forward and for our simple use case it is inconceivable that
> any
> > > particular library could be much better than any other in terms of
> feature
> > > set. We want the most standard library. My goal is to minimize the
> > > dependencies of the client and make our basic logging cases work for
> most
> > > cases.
> > >
> > > Is there a reason not to just the java.util.logging? It comes with the
> jdk
> > > and supports pluggable appenders so people who have some other library
> can
> > > plug that in right?
> > >
> > > Basically my preference would be java.util.logging unless there is some
> > > known problem with it, otherwise I guess slf4j, and if not that then
> log4j.
> > >
> > > Thougts?
> > >
> > > -Jay
> > >
>
>
Re: Logging in new clients
Posted by Jay Kreps <ja...@gmail.com>.
Hey Joel,
I see server and client logging and dependencies are really different in my
mind. The server is a standalone process. Once we decouple the clients I
think we can actually be a lot more aggressive about using more libraries
on the server, why not? We have avoided that so far because they inevitably
leak into the clients since they arent separated.
The problem on the client side is that if we choose anything used by our
clients and there is any incompatibility then they can't use kafka.
>From my point of view the improved logging libraries (log4j2, logback, etc)
are the worst option. They are new so likely will have api changes, and few
people use them so they don't get the benefit of sharing the same logging
library.
-Jay
On Mon, Feb 3, 2014 at 11:17 AM, Joel Koshy <jj...@gmail.com> wrote:
> We are already using other libraries in various parts of our code
> (e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
> other logging dependencies anyway. i.e., what do we gain by using jul?
> There may be a good reason why people don't use jul so I think we
> should fully understand that before going with jul. So it may be
> simpler to just stick with log4j for the client rewrites and
> investigate logging later.
>
> log4j2 is becoming more widespread and many users seem to be favorable
> toward logback. slf4j would cover all of these very easily. From what
> I understand jul does not make it very easy to plug in with these
> various options but I could be wrong.
>
> I completely agree on the need to fix our client logging as that will
> go a long way in usability for end-users unless we want to keep
> getting asked the "Why do I see this ERROR in my logs..?" questions.
>
> Joel
>
> On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> > >> Basically my preference would be java.util.logging unless there is
> some
> > known problem with it, otherwise I guess slf4j, and if not that then
> log4j.
> >
> > +1. My preference is to use java.util.logging to avoid adding an external
> > dependency,
> > but I'm not too sure about what's the "standard" out there, so open to
> > suggestions
> > on picking a different library.
> >
> >
> >
> > On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com> wrote:
> >
> > > We probably need to add a small amount of logging in the new producer
> and
> > > (soon) consumer clients. I wanted to have a quick discussion on logging
> > > libraries before I start adding this in the producer.
> > >
> > > Previously we have been pretty verbose loggers and I think we should
> stop
> > > that. For clients you mostly don't need to log: if there is an error
> you
> > > should throw it back not log it, so you don't need ERROR logging.
> Likewise
> > > I think it is rude to pollute peoples logs with the details of client
> > > initialization (establishing connections, etc), so you don't need INFO
> > > logging. However perhaps there is an argument to be made for WARN and
> > > DEBUG. I think it is perhaps useful to log a WARN when a server breaks
> a
> > > connection or metadata initialization fails. It can sometimes also be
> > > useful to be able to enable debug logging to see step by step
> processing in
> > > the client, which is the case for DEBUG logging.
> > >
> > > Here is my knowledge about the state of Java logging:
> > > 1. Most people still use log4j. The server is using log4j.
> > > 2. Second runner-up in slf4j. I personally consider slf4j pretty silly
> but
> > > this is perhaps the more flexible choice since people can plug in
> different
> > > stuff.
> > > 3. java.util.logging ships with the jdk, but for some reason no one
> uses
> > > it.
> > > 4. There is no critical mass around any other logging library.
> > >
> > > The context for how to think about this is the following. We are not
> trying
> > > to pick the "best" logging library. Fundamentally logging is pretty
> > > straight-forward and for our simple use case it is inconceivable that
> any
> > > particular library could be much better than any other in terms of
> feature
> > > set. We want the most standard library. My goal is to minimize the
> > > dependencies of the client and make our basic logging cases work for
> most
> > > cases.
> > >
> > > Is there a reason not to just the java.util.logging? It comes with the
> jdk
> > > and supports pluggable appenders so people who have some other library
> can
> > > plug that in right?
> > >
> > > Basically my preference would be java.util.logging unless there is some
> > > known problem with it, otherwise I guess slf4j, and if not that then
> log4j.
> > >
> > > Thougts?
> > >
> > > -Jay
> > >
>
>
Re: Logging in new clients
Posted by Steve Morin <st...@stevemorin.com>.
My vote would be with log4j, I don't have that much experience with log4j2
or a good feel for how much the industry is moving towards it.
On Mon, Feb 3, 2014 at 11:17 AM, Joel Koshy <jj...@gmail.com> wrote:
> We are already using other libraries in various parts of our code
> (e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
> other logging dependencies anyway. i.e., what do we gain by using jul?
> There may be a good reason why people don't use jul so I think we
> should fully understand that before going with jul. So it may be
> simpler to just stick with log4j for the client rewrites and
> investigate logging later.
>
> log4j2 is becoming more widespread and many users seem to be favorable
> toward logback. slf4j would cover all of these very easily. From what
> I understand jul does not make it very easy to plug in with these
> various options but I could be wrong.
>
> I completely agree on the need to fix our client logging as that will
> go a long way in usability for end-users unless we want to keep
> getting asked the "Why do I see this ERROR in my logs..?" questions.
>
> Joel
>
> On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> > >> Basically my preference would be java.util.logging unless there is
> some
> > known problem with it, otherwise I guess slf4j, and if not that then
> log4j.
> >
> > +1. My preference is to use java.util.logging to avoid adding an external
> > dependency,
> > but I'm not too sure about what's the "standard" out there, so open to
> > suggestions
> > on picking a different library.
> >
> >
> >
> > On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com> wrote:
> >
> > > We probably need to add a small amount of logging in the new producer
> and
> > > (soon) consumer clients. I wanted to have a quick discussion on logging
> > > libraries before I start adding this in the producer.
> > >
> > > Previously we have been pretty verbose loggers and I think we should
> stop
> > > that. For clients you mostly don't need to log: if there is an error
> you
> > > should throw it back not log it, so you don't need ERROR logging.
> Likewise
> > > I think it is rude to pollute peoples logs with the details of client
> > > initialization (establishing connections, etc), so you don't need INFO
> > > logging. However perhaps there is an argument to be made for WARN and
> > > DEBUG. I think it is perhaps useful to log a WARN when a server breaks
> a
> > > connection or metadata initialization fails. It can sometimes also be
> > > useful to be able to enable debug logging to see step by step
> processing in
> > > the client, which is the case for DEBUG logging.
> > >
> > > Here is my knowledge about the state of Java logging:
> > > 1. Most people still use log4j. The server is using log4j.
> > > 2. Second runner-up in slf4j. I personally consider slf4j pretty silly
> but
> > > this is perhaps the more flexible choice since people can plug in
> different
> > > stuff.
> > > 3. java.util.logging ships with the jdk, but for some reason no one
> uses
> > > it.
> > > 4. There is no critical mass around any other logging library.
> > >
> > > The context for how to think about this is the following. We are not
> trying
> > > to pick the "best" logging library. Fundamentally logging is pretty
> > > straight-forward and for our simple use case it is inconceivable that
> any
> > > particular library could be much better than any other in terms of
> feature
> > > set. We want the most standard library. My goal is to minimize the
> > > dependencies of the client and make our basic logging cases work for
> most
> > > cases.
> > >
> > > Is there a reason not to just the java.util.logging? It comes with the
> jdk
> > > and supports pluggable appenders so people who have some other library
> can
> > > plug that in right?
> > >
> > > Basically my preference would be java.util.logging unless there is some
> > > known problem with it, otherwise I guess slf4j, and if not that then
> log4j.
> > >
> > > Thougts?
> > >
> > > -Jay
> > >
>
>
Re: Logging in new clients
Posted by Joel Koshy <jj...@gmail.com>.
We are already using other libraries in various parts of our code
(e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
other logging dependencies anyway. i.e., what do we gain by using jul?
There may be a good reason why people don't use jul so I think we
should fully understand that before going with jul. So it may be
simpler to just stick with log4j for the client rewrites and
investigate logging later.
log4j2 is becoming more widespread and many users seem to be favorable
toward logback. slf4j would cover all of these very easily. From what
I understand jul does not make it very easy to plug in with these
various options but I could be wrong.
I completely agree on the need to fix our client logging as that will
go a long way in usability for end-users unless we want to keep
getting asked the "Why do I see this ERROR in my logs..?" questions.
Joel
On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> >> Basically my preference would be java.util.logging unless there is some
> known problem with it, otherwise I guess slf4j, and if not that then log4j.
>
> +1. My preference is to use java.util.logging to avoid adding an external
> dependency,
> but I'm not too sure about what's the "standard" out there, so open to
> suggestions
> on picking a different library.
>
>
>
> On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com> wrote:
>
> > We probably need to add a small amount of logging in the new producer and
> > (soon) consumer clients. I wanted to have a quick discussion on logging
> > libraries before I start adding this in the producer.
> >
> > Previously we have been pretty verbose loggers and I think we should stop
> > that. For clients you mostly don't need to log: if there is an error you
> > should throw it back not log it, so you don't need ERROR logging. Likewise
> > I think it is rude to pollute peoples logs with the details of client
> > initialization (establishing connections, etc), so you don't need INFO
> > logging. However perhaps there is an argument to be made for WARN and
> > DEBUG. I think it is perhaps useful to log a WARN when a server breaks a
> > connection or metadata initialization fails. It can sometimes also be
> > useful to be able to enable debug logging to see step by step processing in
> > the client, which is the case for DEBUG logging.
> >
> > Here is my knowledge about the state of Java logging:
> > 1. Most people still use log4j. The server is using log4j.
> > 2. Second runner-up in slf4j. I personally consider slf4j pretty silly but
> > this is perhaps the more flexible choice since people can plug in different
> > stuff.
> > 3. java.util.logging ships with the jdk, but for some reason no one uses
> > it.
> > 4. There is no critical mass around any other logging library.
> >
> > The context for how to think about this is the following. We are not trying
> > to pick the "best" logging library. Fundamentally logging is pretty
> > straight-forward and for our simple use case it is inconceivable that any
> > particular library could be much better than any other in terms of feature
> > set. We want the most standard library. My goal is to minimize the
> > dependencies of the client and make our basic logging cases work for most
> > cases.
> >
> > Is there a reason not to just the java.util.logging? It comes with the jdk
> > and supports pluggable appenders so people who have some other library can
> > plug that in right?
> >
> > Basically my preference would be java.util.logging unless there is some
> > known problem with it, otherwise I guess slf4j, and if not that then log4j.
> >
> > Thougts?
> >
> > -Jay
> >
Re: Logging in new clients
Posted by Joel Koshy <jj...@gmail.com>.
We are already using other libraries in various parts of our code
(e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
other logging dependencies anyway. i.e., what do we gain by using jul?
There may be a good reason why people don't use jul so I think we
should fully understand that before going with jul. So it may be
simpler to just stick with log4j for the client rewrites and
investigate logging later.
log4j2 is becoming more widespread and many users seem to be favorable
toward logback. slf4j would cover all of these very easily. From what
I understand jul does not make it very easy to plug in with these
various options but I could be wrong.
I completely agree on the need to fix our client logging as that will
go a long way in usability for end-users unless we want to keep
getting asked the "Why do I see this ERROR in my logs..?" questions.
Joel
On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> >> Basically my preference would be java.util.logging unless there is some
> known problem with it, otherwise I guess slf4j, and if not that then log4j.
>
> +1. My preference is to use java.util.logging to avoid adding an external
> dependency,
> but I'm not too sure about what's the "standard" out there, so open to
> suggestions
> on picking a different library.
>
>
>
> On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com> wrote:
>
> > We probably need to add a small amount of logging in the new producer and
> > (soon) consumer clients. I wanted to have a quick discussion on logging
> > libraries before I start adding this in the producer.
> >
> > Previously we have been pretty verbose loggers and I think we should stop
> > that. For clients you mostly don't need to log: if there is an error you
> > should throw it back not log it, so you don't need ERROR logging. Likewise
> > I think it is rude to pollute peoples logs with the details of client
> > initialization (establishing connections, etc), so you don't need INFO
> > logging. However perhaps there is an argument to be made for WARN and
> > DEBUG. I think it is perhaps useful to log a WARN when a server breaks a
> > connection or metadata initialization fails. It can sometimes also be
> > useful to be able to enable debug logging to see step by step processing in
> > the client, which is the case for DEBUG logging.
> >
> > Here is my knowledge about the state of Java logging:
> > 1. Most people still use log4j. The server is using log4j.
> > 2. Second runner-up in slf4j. I personally consider slf4j pretty silly but
> > this is perhaps the more flexible choice since people can plug in different
> > stuff.
> > 3. java.util.logging ships with the jdk, but for some reason no one uses
> > it.
> > 4. There is no critical mass around any other logging library.
> >
> > The context for how to think about this is the following. We are not trying
> > to pick the "best" logging library. Fundamentally logging is pretty
> > straight-forward and for our simple use case it is inconceivable that any
> > particular library could be much better than any other in terms of feature
> > set. We want the most standard library. My goal is to minimize the
> > dependencies of the client and make our basic logging cases work for most
> > cases.
> >
> > Is there a reason not to just the java.util.logging? It comes with the jdk
> > and supports pluggable appenders so people who have some other library can
> > plug that in right?
> >
> > Basically my preference would be java.util.logging unless there is some
> > known problem with it, otherwise I guess slf4j, and if not that then log4j.
> >
> > Thougts?
> >
> > -Jay
> >
Re: Logging in new clients
Posted by Xavier Stevens <xa...@gaikai.com>.
I'm all for less dependencies but I would personally vote for slf4j-api.
Just don't use any underlying implementations like logback or slf4j-log4j
bridge. Then people can hookup whatever they want.
On Mon, Feb 3, 2014 at 11:08 AM, Neha Narkhede <ne...@gmail.com>wrote:
> >> Basically my preference would be java.util.logging unless there is some
> known problem with it, otherwise I guess slf4j, and if not that then log4j.
>
> +1. My preference is to use java.util.logging to avoid adding an external
> dependency,
> but I'm not too sure about what's the "standard" out there, so open to
> suggestions
> on picking a different library.
>
>
>
> On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com> wrote:
>
> > We probably need to add a small amount of logging in the new producer and
> > (soon) consumer clients. I wanted to have a quick discussion on logging
> > libraries before I start adding this in the producer.
> >
> > Previously we have been pretty verbose loggers and I think we should stop
> > that. For clients you mostly don't need to log: if there is an error you
> > should throw it back not log it, so you don't need ERROR logging.
> Likewise
> > I think it is rude to pollute peoples logs with the details of client
> > initialization (establishing connections, etc), so you don't need INFO
> > logging. However perhaps there is an argument to be made for WARN and
> > DEBUG. I think it is perhaps useful to log a WARN when a server breaks a
> > connection or metadata initialization fails. It can sometimes also be
> > useful to be able to enable debug logging to see step by step processing
> in
> > the client, which is the case for DEBUG logging.
> >
> > Here is my knowledge about the state of Java logging:
> > 1. Most people still use log4j. The server is using log4j.
> > 2. Second runner-up in slf4j. I personally consider slf4j pretty silly
> but
> > this is perhaps the more flexible choice since people can plug in
> different
> > stuff.
> > 3. java.util.logging ships with the jdk, but for some reason no one uses
> > it.
> > 4. There is no critical mass around any other logging library.
> >
> > The context for how to think about this is the following. We are not
> trying
> > to pick the "best" logging library. Fundamentally logging is pretty
> > straight-forward and for our simple use case it is inconceivable that any
> > particular library could be much better than any other in terms of
> feature
> > set. We want the most standard library. My goal is to minimize the
> > dependencies of the client and make our basic logging cases work for most
> > cases.
> >
> > Is there a reason not to just the java.util.logging? It comes with the
> jdk
> > and supports pluggable appenders so people who have some other library
> can
> > plug that in right?
> >
> > Basically my preference would be java.util.logging unless there is some
> > known problem with it, otherwise I guess slf4j, and if not that then
> log4j.
> >
> > Thougts?
> >
> > -Jay
> >
>
Re: Logging in new clients
Posted by Xavier Stevens <xa...@gaikai.com>.
I'm all for less dependencies but I would personally vote for slf4j-api.
Just don't use any underlying implementations like logback or slf4j-log4j
bridge. Then people can hookup whatever they want.
On Mon, Feb 3, 2014 at 11:08 AM, Neha Narkhede <ne...@gmail.com>wrote:
> >> Basically my preference would be java.util.logging unless there is some
> known problem with it, otherwise I guess slf4j, and if not that then log4j.
>
> +1. My preference is to use java.util.logging to avoid adding an external
> dependency,
> but I'm not too sure about what's the "standard" out there, so open to
> suggestions
> on picking a different library.
>
>
>
> On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com> wrote:
>
> > We probably need to add a small amount of logging in the new producer and
> > (soon) consumer clients. I wanted to have a quick discussion on logging
> > libraries before I start adding this in the producer.
> >
> > Previously we have been pretty verbose loggers and I think we should stop
> > that. For clients you mostly don't need to log: if there is an error you
> > should throw it back not log it, so you don't need ERROR logging.
> Likewise
> > I think it is rude to pollute peoples logs with the details of client
> > initialization (establishing connections, etc), so you don't need INFO
> > logging. However perhaps there is an argument to be made for WARN and
> > DEBUG. I think it is perhaps useful to log a WARN when a server breaks a
> > connection or metadata initialization fails. It can sometimes also be
> > useful to be able to enable debug logging to see step by step processing
> in
> > the client, which is the case for DEBUG logging.
> >
> > Here is my knowledge about the state of Java logging:
> > 1. Most people still use log4j. The server is using log4j.
> > 2. Second runner-up in slf4j. I personally consider slf4j pretty silly
> but
> > this is perhaps the more flexible choice since people can plug in
> different
> > stuff.
> > 3. java.util.logging ships with the jdk, but for some reason no one uses
> > it.
> > 4. There is no critical mass around any other logging library.
> >
> > The context for how to think about this is the following. We are not
> trying
> > to pick the "best" logging library. Fundamentally logging is pretty
> > straight-forward and for our simple use case it is inconceivable that any
> > particular library could be much better than any other in terms of
> feature
> > set. We want the most standard library. My goal is to minimize the
> > dependencies of the client and make our basic logging cases work for most
> > cases.
> >
> > Is there a reason not to just the java.util.logging? It comes with the
> jdk
> > and supports pluggable appenders so people who have some other library
> can
> > plug that in right?
> >
> > Basically my preference would be java.util.logging unless there is some
> > known problem with it, otherwise I guess slf4j, and if not that then
> log4j.
> >
> > Thougts?
> >
> > -Jay
> >
>
Re: Logging in new clients
Posted by Neha Narkhede <ne...@gmail.com>.
>> Basically my preference would be java.util.logging unless there is some
known problem with it, otherwise I guess slf4j, and if not that then log4j.
+1. My preference is to use java.util.logging to avoid adding an external
dependency,
but I'm not too sure about what's the "standard" out there, so open to
suggestions
on picking a different library.
On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com> wrote:
> We probably need to add a small amount of logging in the new producer and
> (soon) consumer clients. I wanted to have a quick discussion on logging
> libraries before I start adding this in the producer.
>
> Previously we have been pretty verbose loggers and I think we should stop
> that. For clients you mostly don't need to log: if there is an error you
> should throw it back not log it, so you don't need ERROR logging. Likewise
> I think it is rude to pollute peoples logs with the details of client
> initialization (establishing connections, etc), so you don't need INFO
> logging. However perhaps there is an argument to be made for WARN and
> DEBUG. I think it is perhaps useful to log a WARN when a server breaks a
> connection or metadata initialization fails. It can sometimes also be
> useful to be able to enable debug logging to see step by step processing in
> the client, which is the case for DEBUG logging.
>
> Here is my knowledge about the state of Java logging:
> 1. Most people still use log4j. The server is using log4j.
> 2. Second runner-up in slf4j. I personally consider slf4j pretty silly but
> this is perhaps the more flexible choice since people can plug in different
> stuff.
> 3. java.util.logging ships with the jdk, but for some reason no one uses
> it.
> 4. There is no critical mass around any other logging library.
>
> The context for how to think about this is the following. We are not trying
> to pick the "best" logging library. Fundamentally logging is pretty
> straight-forward and for our simple use case it is inconceivable that any
> particular library could be much better than any other in terms of feature
> set. We want the most standard library. My goal is to minimize the
> dependencies of the client and make our basic logging cases work for most
> cases.
>
> Is there a reason not to just the java.util.logging? It comes with the jdk
> and supports pluggable appenders so people who have some other library can
> plug that in right?
>
> Basically my preference would be java.util.logging unless there is some
> known problem with it, otherwise I guess slf4j, and if not that then log4j.
>
> Thougts?
>
> -Jay
>
Re: Logging in new clients
Posted by Neha Narkhede <ne...@gmail.com>.
>> Basically my preference would be java.util.logging unless there is some
known problem with it, otherwise I guess slf4j, and if not that then log4j.
+1. My preference is to use java.util.logging to avoid adding an external
dependency,
but I'm not too sure about what's the "standard" out there, so open to
suggestions
on picking a different library.
On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <ja...@gmail.com> wrote:
> We probably need to add a small amount of logging in the new producer and
> (soon) consumer clients. I wanted to have a quick discussion on logging
> libraries before I start adding this in the producer.
>
> Previously we have been pretty verbose loggers and I think we should stop
> that. For clients you mostly don't need to log: if there is an error you
> should throw it back not log it, so you don't need ERROR logging. Likewise
> I think it is rude to pollute peoples logs with the details of client
> initialization (establishing connections, etc), so you don't need INFO
> logging. However perhaps there is an argument to be made for WARN and
> DEBUG. I think it is perhaps useful to log a WARN when a server breaks a
> connection or metadata initialization fails. It can sometimes also be
> useful to be able to enable debug logging to see step by step processing in
> the client, which is the case for DEBUG logging.
>
> Here is my knowledge about the state of Java logging:
> 1. Most people still use log4j. The server is using log4j.
> 2. Second runner-up in slf4j. I personally consider slf4j pretty silly but
> this is perhaps the more flexible choice since people can plug in different
> stuff.
> 3. java.util.logging ships with the jdk, but for some reason no one uses
> it.
> 4. There is no critical mass around any other logging library.
>
> The context for how to think about this is the following. We are not trying
> to pick the "best" logging library. Fundamentally logging is pretty
> straight-forward and for our simple use case it is inconceivable that any
> particular library could be much better than any other in terms of feature
> set. We want the most standard library. My goal is to minimize the
> dependencies of the client and make our basic logging cases work for most
> cases.
>
> Is there a reason not to just the java.util.logging? It comes with the jdk
> and supports pluggable appenders so people who have some other library can
> plug that in right?
>
> Basically my preference would be java.util.logging unless there is some
> known problem with it, otherwise I guess slf4j, and if not that then log4j.
>
> Thougts?
>
> -Jay
>