You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@logging.apache.org by Ralph Goers <ra...@dslextreme.com> on 2018/08/05 19:21:08 UTC
Logging and containers
Our performance page shows that logging to the console is extremely slow. Yet one of the “best practices” for containers is to have the applications log to STDOUT or STDERR. This leads me to two questions:
Is the performance of writing to STDOUT just as bad in a container? I have no reason to believe it wouldn’t be but have no evidence.
Assuming performance is poor what are the realistic alternatives? Is there something more Log4j needs to be doing to play well in a cloud environment?
Ralph
Re: Logging and containers
Posted by Matt Sicker <bo...@gmail.com>.
I had requested the HTTP appender a while back for potential use in writing
directly to Elasticsearch which helps reduce the moving parts involved in
any sort of log searching stack. There's also the option of hosting a
Graylog server and writing directly to it through a socket appender, though
I haven't tried that personally.
On Mon, 6 Aug 2018 at 19:06, Ralph Goers <ra...@dslextreme.com> wrote:
> Thanks for that. I see that the AWS SDK has a Java CloudWatch API that
> could be used to create an Appender. I also see
> https://docs.aws.amazon.com/lambda/latest/dg/java-logging.html <
> https://docs.aws.amazon.com/lambda/latest/dg/java-logging.html> where
> amazon provides an appender. The example shows 2.8 so hopefully it is
> compatible with our latest releases. This is good info to capture.
>
> Ralph
>
> > On Aug 6, 2018, at 3:53 PM, Vaid, Himali <hi...@ditech.com> wrote:
> >
> > We use docker containers running as ECS on AWS Ec2, I use log4net AWS
> appender to write directly to AWS cloudwatch.
> >
> > Sent from my iPhone
> >
> >> On Aug 6, 2018, at 6:38 PM, Remko Popma <re...@gmail.com> wrote:
> >>
> >> I have no experience with containers at all, sorry...
> >>
> >>> On Tue, Aug 7, 2018 at 1:13 AM, Matt Sicker <bo...@gmail.com> wrote:
> >>>
> >>> The way I've used Docker in the past has generally been to configure
> log4j2
> >>> to use a direct console appender (non-default option), async logging,
> and
> >>> then use a logging driver from Docker or Kubernetes or even some
> >>> cloud-specific log gathering service, which listen to standard out and
> >>> standard error.
> >>>
> >>> In some other Docker scenarios I've used a Kafka appender directly, but
> >>> nowadays I think it's easier to use the regular log drivers. I'd like
> to
> >>> explore more in this space, though.
> >>>
> >>> On Mon, 6 Aug 2018 at 10:57, Ralph Goers <ra...@dslextreme.com>
> >>> wrote:
> >>>
> >>>> Do you have any way of determining the performance difference of
> writing
> >>>> to a fie vs writing to stdout?
> >>>>
> >>>> Ralph
> >>>>
> >>>>> On Aug 6, 2018, at 8:47 AM, Rob Tompkins <ch...@gmail.com> wrote:
> >>>>>
> >>>>> I find myself writing to either standard out or a file. When I write
> to
> >>>> a file in docker I tend to “share” that file with the filesystem on
> the
> >>>> docker host. But, I prefer writing to standard our and appending that
> to
> >>> a
> >>>> file on the machine as it deals with less of the underlying filesystem
> >>>> networking (which is cumbersome).
> >>>>>
> >>>>> Don’t know if that helps.
> >>>>>
> >>>>> -Rob
> >>>>>
> >>>>>> On Aug 6, 2018, at 11:44 AM, Ralph Goers <
> ralph.goers@dslextreme.com>
> >>>> wrote:
> >>>>>>
> >>>>>> I don’t know. That is why I am asking if you guys have tried
> anything
> >>>> with Docker containers. Writing to stdout is a “best practice” so I am
> >>> just
> >>>> trying to validate whether that is good or bad advice or what needs
> to be
> >>>> done to make it work well. Or if Log4j should implement a Docker
> plugin
> >>> to
> >>>> write to, or something else.
> >>>>>>
> >>>>>>> On Aug 6, 2018, at 8:28 AM, Gary Gregory <ga...@gmail.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>> Can't you just configure the console appender with a large-ish
> buffer
> >>>> and
> >>>>>>> remove the bottleneck?
> >>>>>>>
> >>>>>>> Gary
> >>>>>>>
> >>>>>>> On Mon, Aug 6, 2018 at 8:55 AM Ralph Goers <
> >>> ralph.goers@dslextreme.com
> >>>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> So that begs the question, when logging to stdout in a container
> is
> >>> a
> >>>>>>>> console attached? i.e. can you normally view the output like you
> >>>> could in a
> >>>>>>>> regular VM or is it all redirected somewhere else? I haven’t
> worked
> >>>> much
> >>>>>>>> with Docker yet so I am afraid I don’t know the answer.
> >>>>>>>>
> >>>>>>>> Ralph
> >>>>>>>>
> >>>>>>>>> On Aug 6, 2018, at 6:40 AM, Remko Popma <re...@gmail.com>
> >>>> wrote:
> >>>>>>>>>
> >>>>>>>>> It may be to do with whether a tty is attached and how fast it
> is:
> >>>>>>>>
> >>>> https://stackoverflow.com/questions/3857052/why-is-
> >>> printing-to-stdout-so-slow-can-it-be-sped-up
> >>>>>>>>>
> >>>>>>>>> (Shameless plug) Every java main() method deserves
> >>>> http://picocli.info
> >>>>>>>>>
> >>>>>>>>>> On Aug 6, 2018, at 4:21, Ralph Goers <
> ralph.goers@dslextreme.com>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Our performance page shows that logging to the console is
> >>> extremely
> >>>>>>>> slow. Yet one of the “best practices” for containers is to have
> the
> >>>>>>>> applications log to STDOUT or STDERR. This leads me to two
> >>> questions:
> >>>>>>>>>> Is the performance of writing to STDOUT just as bad in a
> >>> container?
> >>>> I
> >>>>>>>> have no reason to believe it wouldn’t be but have no evidence.
> >>>>>>>>>> Assuming performance is poor what are the realistic
> alternatives?
> >>> Is
> >>>>>>>> there something more Log4j needs to be doing to play well in a
> cloud
> >>>>>>>> environment?
> >>>>>>>>>>
> >>>>>>>>>> Ralph
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>> --
> >>> Matt Sicker <bo...@gmail.com>
> >>>
>
>
--
Matt Sicker <bo...@gmail.com>
Re: Logging and containers
Posted by Ralph Goers <ra...@dslextreme.com>.
Thanks for that. I see that the AWS SDK has a Java CloudWatch API that could be used to create an Appender. I also see https://docs.aws.amazon.com/lambda/latest/dg/java-logging.html <https://docs.aws.amazon.com/lambda/latest/dg/java-logging.html> where amazon provides an appender. The example shows 2.8 so hopefully it is compatible with our latest releases. This is good info to capture.
Ralph
> On Aug 6, 2018, at 3:53 PM, Vaid, Himali <hi...@ditech.com> wrote:
>
> We use docker containers running as ECS on AWS Ec2, I use log4net AWS appender to write directly to AWS cloudwatch.
>
> Sent from my iPhone
>
>> On Aug 6, 2018, at 6:38 PM, Remko Popma <re...@gmail.com> wrote:
>>
>> I have no experience with containers at all, sorry...
>>
>>> On Tue, Aug 7, 2018 at 1:13 AM, Matt Sicker <bo...@gmail.com> wrote:
>>>
>>> The way I've used Docker in the past has generally been to configure log4j2
>>> to use a direct console appender (non-default option), async logging, and
>>> then use a logging driver from Docker or Kubernetes or even some
>>> cloud-specific log gathering service, which listen to standard out and
>>> standard error.
>>>
>>> In some other Docker scenarios I've used a Kafka appender directly, but
>>> nowadays I think it's easier to use the regular log drivers. I'd like to
>>> explore more in this space, though.
>>>
>>> On Mon, 6 Aug 2018 at 10:57, Ralph Goers <ra...@dslextreme.com>
>>> wrote:
>>>
>>>> Do you have any way of determining the performance difference of writing
>>>> to a fie vs writing to stdout?
>>>>
>>>> Ralph
>>>>
>>>>> On Aug 6, 2018, at 8:47 AM, Rob Tompkins <ch...@gmail.com> wrote:
>>>>>
>>>>> I find myself writing to either standard out or a file. When I write to
>>>> a file in docker I tend to “share” that file with the filesystem on the
>>>> docker host. But, I prefer writing to standard our and appending that to
>>> a
>>>> file on the machine as it deals with less of the underlying filesystem
>>>> networking (which is cumbersome).
>>>>>
>>>>> Don’t know if that helps.
>>>>>
>>>>> -Rob
>>>>>
>>>>>> On Aug 6, 2018, at 11:44 AM, Ralph Goers <ra...@dslextreme.com>
>>>> wrote:
>>>>>>
>>>>>> I don’t know. That is why I am asking if you guys have tried anything
>>>> with Docker containers. Writing to stdout is a “best practice” so I am
>>> just
>>>> trying to validate whether that is good or bad advice or what needs to be
>>>> done to make it work well. Or if Log4j should implement a Docker plugin
>>> to
>>>> write to, or something else.
>>>>>>
>>>>>>> On Aug 6, 2018, at 8:28 AM, Gary Gregory <ga...@gmail.com>
>>>> wrote:
>>>>>>>
>>>>>>> Can't you just configure the console appender with a large-ish buffer
>>>> and
>>>>>>> remove the bottleneck?
>>>>>>>
>>>>>>> Gary
>>>>>>>
>>>>>>> On Mon, Aug 6, 2018 at 8:55 AM Ralph Goers <
>>> ralph.goers@dslextreme.com
>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> So that begs the question, when logging to stdout in a container is
>>> a
>>>>>>>> console attached? i.e. can you normally view the output like you
>>>> could in a
>>>>>>>> regular VM or is it all redirected somewhere else? I haven’t worked
>>>> much
>>>>>>>> with Docker yet so I am afraid I don’t know the answer.
>>>>>>>>
>>>>>>>> Ralph
>>>>>>>>
>>>>>>>>> On Aug 6, 2018, at 6:40 AM, Remko Popma <re...@gmail.com>
>>>> wrote:
>>>>>>>>>
>>>>>>>>> It may be to do with whether a tty is attached and how fast it is:
>>>>>>>>
>>>> https://stackoverflow.com/questions/3857052/why-is-
>>> printing-to-stdout-so-slow-can-it-be-sped-up
>>>>>>>>>
>>>>>>>>> (Shameless plug) Every java main() method deserves
>>>> http://picocli.info
>>>>>>>>>
>>>>>>>>>> On Aug 6, 2018, at 4:21, Ralph Goers <ra...@dslextreme.com>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Our performance page shows that logging to the console is
>>> extremely
>>>>>>>> slow. Yet one of the “best practices” for containers is to have the
>>>>>>>> applications log to STDOUT or STDERR. This leads me to two
>>> questions:
>>>>>>>>>> Is the performance of writing to STDOUT just as bad in a
>>> container?
>>>> I
>>>>>>>> have no reason to believe it wouldn’t be but have no evidence.
>>>>>>>>>> Assuming performance is poor what are the realistic alternatives?
>>> Is
>>>>>>>> there something more Log4j needs to be doing to play well in a cloud
>>>>>>>> environment?
>>>>>>>>>>
>>>>>>>>>> Ralph
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>> Matt Sicker <bo...@gmail.com>
>>>
Re: Logging and containers
Posted by "Vaid, Himali" <hi...@ditech.com>.
We use docker containers running as ECS on AWS Ec2, I use log4net AWS appender to write directly to AWS cloudwatch.
Sent from my iPhone
> On Aug 6, 2018, at 6:38 PM, Remko Popma <re...@gmail.com> wrote:
>
> I have no experience with containers at all, sorry...
>
>> On Tue, Aug 7, 2018 at 1:13 AM, Matt Sicker <bo...@gmail.com> wrote:
>>
>> The way I've used Docker in the past has generally been to configure log4j2
>> to use a direct console appender (non-default option), async logging, and
>> then use a logging driver from Docker or Kubernetes or even some
>> cloud-specific log gathering service, which listen to standard out and
>> standard error.
>>
>> In some other Docker scenarios I've used a Kafka appender directly, but
>> nowadays I think it's easier to use the regular log drivers. I'd like to
>> explore more in this space, though.
>>
>> On Mon, 6 Aug 2018 at 10:57, Ralph Goers <ra...@dslextreme.com>
>> wrote:
>>
>>> Do you have any way of determining the performance difference of writing
>>> to a fie vs writing to stdout?
>>>
>>> Ralph
>>>
>>>> On Aug 6, 2018, at 8:47 AM, Rob Tompkins <ch...@gmail.com> wrote:
>>>>
>>>> I find myself writing to either standard out or a file. When I write to
>>> a file in docker I tend to “share” that file with the filesystem on the
>>> docker host. But, I prefer writing to standard our and appending that to
>> a
>>> file on the machine as it deals with less of the underlying filesystem
>>> networking (which is cumbersome).
>>>>
>>>> Don’t know if that helps.
>>>>
>>>> -Rob
>>>>
>>>>> On Aug 6, 2018, at 11:44 AM, Ralph Goers <ra...@dslextreme.com>
>>> wrote:
>>>>>
>>>>> I don’t know. That is why I am asking if you guys have tried anything
>>> with Docker containers. Writing to stdout is a “best practice” so I am
>> just
>>> trying to validate whether that is good or bad advice or what needs to be
>>> done to make it work well. Or if Log4j should implement a Docker plugin
>> to
>>> write to, or something else.
>>>>>
>>>>>> On Aug 6, 2018, at 8:28 AM, Gary Gregory <ga...@gmail.com>
>>> wrote:
>>>>>>
>>>>>> Can't you just configure the console appender with a large-ish buffer
>>> and
>>>>>> remove the bottleneck?
>>>>>>
>>>>>> Gary
>>>>>>
>>>>>> On Mon, Aug 6, 2018 at 8:55 AM Ralph Goers <
>> ralph.goers@dslextreme.com
>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> So that begs the question, when logging to stdout in a container is
>> a
>>>>>>> console attached? i.e. can you normally view the output like you
>>> could in a
>>>>>>> regular VM or is it all redirected somewhere else? I haven’t worked
>>> much
>>>>>>> with Docker yet so I am afraid I don’t know the answer.
>>>>>>>
>>>>>>> Ralph
>>>>>>>
>>>>>>>> On Aug 6, 2018, at 6:40 AM, Remko Popma <re...@gmail.com>
>>> wrote:
>>>>>>>>
>>>>>>>> It may be to do with whether a tty is attached and how fast it is:
>>>>>>>
>>> https://stackoverflow.com/questions/3857052/why-is-
>> printing-to-stdout-so-slow-can-it-be-sped-up
>>>>>>>>
>>>>>>>> (Shameless plug) Every java main() method deserves
>>> http://picocli.info
>>>>>>>>
>>>>>>>>> On Aug 6, 2018, at 4:21, Ralph Goers <ra...@dslextreme.com>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Our performance page shows that logging to the console is
>> extremely
>>>>>>> slow. Yet one of the “best practices” for containers is to have the
>>>>>>> applications log to STDOUT or STDERR. This leads me to two
>> questions:
>>>>>>>>> Is the performance of writing to STDOUT just as bad in a
>> container?
>>> I
>>>>>>> have no reason to believe it wouldn’t be but have no evidence.
>>>>>>>>> Assuming performance is poor what are the realistic alternatives?
>> Is
>>>>>>> there something more Log4j needs to be doing to play well in a cloud
>>>>>>> environment?
>>>>>>>>>
>>>>>>>>> Ralph
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>> --
>> Matt Sicker <bo...@gmail.com>
>>
Re: Logging and containers
Posted by Remko Popma <re...@gmail.com>.
I have no experience with containers at all, sorry...
On Tue, Aug 7, 2018 at 1:13 AM, Matt Sicker <bo...@gmail.com> wrote:
> The way I've used Docker in the past has generally been to configure log4j2
> to use a direct console appender (non-default option), async logging, and
> then use a logging driver from Docker or Kubernetes or even some
> cloud-specific log gathering service, which listen to standard out and
> standard error.
>
> In some other Docker scenarios I've used a Kafka appender directly, but
> nowadays I think it's easier to use the regular log drivers. I'd like to
> explore more in this space, though.
>
> On Mon, 6 Aug 2018 at 10:57, Ralph Goers <ra...@dslextreme.com>
> wrote:
>
> > Do you have any way of determining the performance difference of writing
> > to a fie vs writing to stdout?
> >
> > Ralph
> >
> > > On Aug 6, 2018, at 8:47 AM, Rob Tompkins <ch...@gmail.com> wrote:
> > >
> > > I find myself writing to either standard out or a file. When I write to
> > a file in docker I tend to “share” that file with the filesystem on the
> > docker host. But, I prefer writing to standard our and appending that to
> a
> > file on the machine as it deals with less of the underlying filesystem
> > networking (which is cumbersome).
> > >
> > > Don’t know if that helps.
> > >
> > > -Rob
> > >
> > >> On Aug 6, 2018, at 11:44 AM, Ralph Goers <ra...@dslextreme.com>
> > wrote:
> > >>
> > >> I don’t know. That is why I am asking if you guys have tried anything
> > with Docker containers. Writing to stdout is a “best practice” so I am
> just
> > trying to validate whether that is good or bad advice or what needs to be
> > done to make it work well. Or if Log4j should implement a Docker plugin
> to
> > write to, or something else.
> > >>
> > >>> On Aug 6, 2018, at 8:28 AM, Gary Gregory <ga...@gmail.com>
> > wrote:
> > >>>
> > >>> Can't you just configure the console appender with a large-ish buffer
> > and
> > >>> remove the bottleneck?
> > >>>
> > >>> Gary
> > >>>
> > >>> On Mon, Aug 6, 2018 at 8:55 AM Ralph Goers <
> ralph.goers@dslextreme.com
> > >
> > >>> wrote:
> > >>>
> > >>>> So that begs the question, when logging to stdout in a container is
> a
> > >>>> console attached? i.e. can you normally view the output like you
> > could in a
> > >>>> regular VM or is it all redirected somewhere else? I haven’t worked
> > much
> > >>>> with Docker yet so I am afraid I don’t know the answer.
> > >>>>
> > >>>> Ralph
> > >>>>
> > >>>>> On Aug 6, 2018, at 6:40 AM, Remko Popma <re...@gmail.com>
> > wrote:
> > >>>>>
> > >>>>> It may be to do with whether a tty is attached and how fast it is:
> > >>>>
> > https://stackoverflow.com/questions/3857052/why-is-
> printing-to-stdout-so-slow-can-it-be-sped-up
> > >>>>>
> > >>>>> (Shameless plug) Every java main() method deserves
> > http://picocli.info
> > >>>>>
> > >>>>>> On Aug 6, 2018, at 4:21, Ralph Goers <ra...@dslextreme.com>
> > >>>> wrote:
> > >>>>>>
> > >>>>>> Our performance page shows that logging to the console is
> extremely
> > >>>> slow. Yet one of the “best practices” for containers is to have the
> > >>>> applications log to STDOUT or STDERR. This leads me to two
> questions:
> > >>>>>> Is the performance of writing to STDOUT just as bad in a
> container?
> > I
> > >>>> have no reason to believe it wouldn’t be but have no evidence.
> > >>>>>> Assuming performance is poor what are the realistic alternatives?
> Is
> > >>>> there something more Log4j needs to be doing to play well in a cloud
> > >>>> environment?
> > >>>>>>
> > >>>>>> Ralph
> > >>>>
> > >>>>
> > >>>>
> > >>
> > >>
> > >
> > >
> >
> >
> >
>
> --
> Matt Sicker <bo...@gmail.com>
>
Re: Logging and containers
Posted by Matt Sicker <bo...@gmail.com>.
The way I've used Docker in the past has generally been to configure log4j2
to use a direct console appender (non-default option), async logging, and
then use a logging driver from Docker or Kubernetes or even some
cloud-specific log gathering service, which listen to standard out and
standard error.
In some other Docker scenarios I've used a Kafka appender directly, but
nowadays I think it's easier to use the regular log drivers. I'd like to
explore more in this space, though.
On Mon, 6 Aug 2018 at 10:57, Ralph Goers <ra...@dslextreme.com> wrote:
> Do you have any way of determining the performance difference of writing
> to a fie vs writing to stdout?
>
> Ralph
>
> > On Aug 6, 2018, at 8:47 AM, Rob Tompkins <ch...@gmail.com> wrote:
> >
> > I find myself writing to either standard out or a file. When I write to
> a file in docker I tend to “share” that file with the filesystem on the
> docker host. But, I prefer writing to standard our and appending that to a
> file on the machine as it deals with less of the underlying filesystem
> networking (which is cumbersome).
> >
> > Don’t know if that helps.
> >
> > -Rob
> >
> >> On Aug 6, 2018, at 11:44 AM, Ralph Goers <ra...@dslextreme.com>
> wrote:
> >>
> >> I don’t know. That is why I am asking if you guys have tried anything
> with Docker containers. Writing to stdout is a “best practice” so I am just
> trying to validate whether that is good or bad advice or what needs to be
> done to make it work well. Or if Log4j should implement a Docker plugin to
> write to, or something else.
> >>
> >>> On Aug 6, 2018, at 8:28 AM, Gary Gregory <ga...@gmail.com>
> wrote:
> >>>
> >>> Can't you just configure the console appender with a large-ish buffer
> and
> >>> remove the bottleneck?
> >>>
> >>> Gary
> >>>
> >>> On Mon, Aug 6, 2018 at 8:55 AM Ralph Goers <ralph.goers@dslextreme.com
> >
> >>> wrote:
> >>>
> >>>> So that begs the question, when logging to stdout in a container is a
> >>>> console attached? i.e. can you normally view the output like you
> could in a
> >>>> regular VM or is it all redirected somewhere else? I haven’t worked
> much
> >>>> with Docker yet so I am afraid I don’t know the answer.
> >>>>
> >>>> Ralph
> >>>>
> >>>>> On Aug 6, 2018, at 6:40 AM, Remko Popma <re...@gmail.com>
> wrote:
> >>>>>
> >>>>> It may be to do with whether a tty is attached and how fast it is:
> >>>>
> https://stackoverflow.com/questions/3857052/why-is-printing-to-stdout-so-slow-can-it-be-sped-up
> >>>>>
> >>>>> (Shameless plug) Every java main() method deserves
> http://picocli.info
> >>>>>
> >>>>>> On Aug 6, 2018, at 4:21, Ralph Goers <ra...@dslextreme.com>
> >>>> wrote:
> >>>>>>
> >>>>>> Our performance page shows that logging to the console is extremely
> >>>> slow. Yet one of the “best practices” for containers is to have the
> >>>> applications log to STDOUT or STDERR. This leads me to two questions:
> >>>>>> Is the performance of writing to STDOUT just as bad in a container?
> I
> >>>> have no reason to believe it wouldn’t be but have no evidence.
> >>>>>> Assuming performance is poor what are the realistic alternatives? Is
> >>>> there something more Log4j needs to be doing to play well in a cloud
> >>>> environment?
> >>>>>>
> >>>>>> Ralph
> >>>>
> >>>>
> >>>>
> >>
> >>
> >
> >
>
>
>
--
Matt Sicker <bo...@gmail.com>
Re: Logging and containers
Posted by Ralph Goers <ra...@dslextreme.com>.
Do you have any way of determining the performance difference of writing to a fie vs writing to stdout?
Ralph
> On Aug 6, 2018, at 8:47 AM, Rob Tompkins <ch...@gmail.com> wrote:
>
> I find myself writing to either standard out or a file. When I write to a file in docker I tend to “share” that file with the filesystem on the docker host. But, I prefer writing to standard our and appending that to a file on the machine as it deals with less of the underlying filesystem networking (which is cumbersome).
>
> Don’t know if that helps.
>
> -Rob
>
>> On Aug 6, 2018, at 11:44 AM, Ralph Goers <ra...@dslextreme.com> wrote:
>>
>> I don’t know. That is why I am asking if you guys have tried anything with Docker containers. Writing to stdout is a “best practice” so I am just trying to validate whether that is good or bad advice or what needs to be done to make it work well. Or if Log4j should implement a Docker plugin to write to, or something else.
>>
>>> On Aug 6, 2018, at 8:28 AM, Gary Gregory <ga...@gmail.com> wrote:
>>>
>>> Can't you just configure the console appender with a large-ish buffer and
>>> remove the bottleneck?
>>>
>>> Gary
>>>
>>> On Mon, Aug 6, 2018 at 8:55 AM Ralph Goers <ra...@dslextreme.com>
>>> wrote:
>>>
>>>> So that begs the question, when logging to stdout in a container is a
>>>> console attached? i.e. can you normally view the output like you could in a
>>>> regular VM or is it all redirected somewhere else? I haven’t worked much
>>>> with Docker yet so I am afraid I don’t know the answer.
>>>>
>>>> Ralph
>>>>
>>>>> On Aug 6, 2018, at 6:40 AM, Remko Popma <re...@gmail.com> wrote:
>>>>>
>>>>> It may be to do with whether a tty is attached and how fast it is:
>>>> https://stackoverflow.com/questions/3857052/why-is-printing-to-stdout-so-slow-can-it-be-sped-up
>>>>>
>>>>> (Shameless plug) Every java main() method deserves http://picocli.info
>>>>>
>>>>>> On Aug 6, 2018, at 4:21, Ralph Goers <ra...@dslextreme.com>
>>>> wrote:
>>>>>>
>>>>>> Our performance page shows that logging to the console is extremely
>>>> slow. Yet one of the “best practices” for containers is to have the
>>>> applications log to STDOUT or STDERR. This leads me to two questions:
>>>>>> Is the performance of writing to STDOUT just as bad in a container? I
>>>> have no reason to believe it wouldn’t be but have no evidence.
>>>>>> Assuming performance is poor what are the realistic alternatives? Is
>>>> there something more Log4j needs to be doing to play well in a cloud
>>>> environment?
>>>>>>
>>>>>> Ralph
>>>>
>>>>
>>>>
>>
>>
>
>
Re: Logging and containers
Posted by Rob Tompkins <ch...@gmail.com>.
I find myself writing to either standard out or a file. When I write to a file in docker I tend to “share” that file with the filesystem on the docker host. But, I prefer writing to standard our and appending that to a file on the machine as it deals with less of the underlying filesystem networking (which is cumbersome).
Don’t know if that helps.
-Rob
> On Aug 6, 2018, at 11:44 AM, Ralph Goers <ra...@dslextreme.com> wrote:
>
> I don’t know. That is why I am asking if you guys have tried anything with Docker containers. Writing to stdout is a “best practice” so I am just trying to validate whether that is good or bad advice or what needs to be done to make it work well. Or if Log4j should implement a Docker plugin to write to, or something else.
>
>> On Aug 6, 2018, at 8:28 AM, Gary Gregory <ga...@gmail.com> wrote:
>>
>> Can't you just configure the console appender with a large-ish buffer and
>> remove the bottleneck?
>>
>> Gary
>>
>> On Mon, Aug 6, 2018 at 8:55 AM Ralph Goers <ra...@dslextreme.com>
>> wrote:
>>
>>> So that begs the question, when logging to stdout in a container is a
>>> console attached? i.e. can you normally view the output like you could in a
>>> regular VM or is it all redirected somewhere else? I haven’t worked much
>>> with Docker yet so I am afraid I don’t know the answer.
>>>
>>> Ralph
>>>
>>>> On Aug 6, 2018, at 6:40 AM, Remko Popma <re...@gmail.com> wrote:
>>>>
>>>> It may be to do with whether a tty is attached and how fast it is:
>>> https://stackoverflow.com/questions/3857052/why-is-printing-to-stdout-so-slow-can-it-be-sped-up
>>>>
>>>> (Shameless plug) Every java main() method deserves http://picocli.info
>>>>
>>>>> On Aug 6, 2018, at 4:21, Ralph Goers <ra...@dslextreme.com>
>>> wrote:
>>>>>
>>>>> Our performance page shows that logging to the console is extremely
>>> slow. Yet one of the “best practices” for containers is to have the
>>> applications log to STDOUT or STDERR. This leads me to two questions:
>>>>> Is the performance of writing to STDOUT just as bad in a container? I
>>> have no reason to believe it wouldn’t be but have no evidence.
>>>>> Assuming performance is poor what are the realistic alternatives? Is
>>> there something more Log4j needs to be doing to play well in a cloud
>>> environment?
>>>>>
>>>>> Ralph
>>>
>>>
>>>
>
>
Re: Logging and containers
Posted by Ralph Goers <ra...@dslextreme.com>.
I don’t know. That is why I am asking if you guys have tried anything with Docker containers. Writing to stdout is a “best practice” so I am just trying to validate whether that is good or bad advice or what needs to be done to make it work well. Or if Log4j should implement a Docker plugin to write to, or something else.
> On Aug 6, 2018, at 8:28 AM, Gary Gregory <ga...@gmail.com> wrote:
>
> Can't you just configure the console appender with a large-ish buffer and
> remove the bottleneck?
>
> Gary
>
> On Mon, Aug 6, 2018 at 8:55 AM Ralph Goers <ra...@dslextreme.com>
> wrote:
>
>> So that begs the question, when logging to stdout in a container is a
>> console attached? i.e. can you normally view the output like you could in a
>> regular VM or is it all redirected somewhere else? I haven’t worked much
>> with Docker yet so I am afraid I don’t know the answer.
>>
>> Ralph
>>
>>> On Aug 6, 2018, at 6:40 AM, Remko Popma <re...@gmail.com> wrote:
>>>
>>> It may be to do with whether a tty is attached and how fast it is:
>> https://stackoverflow.com/questions/3857052/why-is-printing-to-stdout-so-slow-can-it-be-sped-up
>>>
>>> (Shameless plug) Every java main() method deserves http://picocli.info
>>>
>>>> On Aug 6, 2018, at 4:21, Ralph Goers <ra...@dslextreme.com>
>> wrote:
>>>>
>>>> Our performance page shows that logging to the console is extremely
>> slow. Yet one of the “best practices” for containers is to have the
>> applications log to STDOUT or STDERR. This leads me to two questions:
>>>> Is the performance of writing to STDOUT just as bad in a container? I
>> have no reason to believe it wouldn’t be but have no evidence.
>>>> Assuming performance is poor what are the realistic alternatives? Is
>> there something more Log4j needs to be doing to play well in a cloud
>> environment?
>>>>
>>>> Ralph
>>
>>
>>
Re: Logging and containers
Posted by Gary Gregory <ga...@gmail.com>.
Can't you just configure the console appender with a large-ish buffer and
remove the bottleneck?
Gary
On Mon, Aug 6, 2018 at 8:55 AM Ralph Goers <ra...@dslextreme.com>
wrote:
> So that begs the question, when logging to stdout in a container is a
> console attached? i.e. can you normally view the output like you could in a
> regular VM or is it all redirected somewhere else? I haven’t worked much
> with Docker yet so I am afraid I don’t know the answer.
>
> Ralph
>
> > On Aug 6, 2018, at 6:40 AM, Remko Popma <re...@gmail.com> wrote:
> >
> > It may be to do with whether a tty is attached and how fast it is:
> https://stackoverflow.com/questions/3857052/why-is-printing-to-stdout-so-slow-can-it-be-sped-up
> >
> > (Shameless plug) Every java main() method deserves http://picocli.info
> >
> >> On Aug 6, 2018, at 4:21, Ralph Goers <ra...@dslextreme.com>
> wrote:
> >>
> >> Our performance page shows that logging to the console is extremely
> slow. Yet one of the “best practices” for containers is to have the
> applications log to STDOUT or STDERR. This leads me to two questions:
> >> Is the performance of writing to STDOUT just as bad in a container? I
> have no reason to believe it wouldn’t be but have no evidence.
> >> Assuming performance is poor what are the realistic alternatives? Is
> there something more Log4j needs to be doing to play well in a cloud
> environment?
> >>
> >> Ralph
>
>
>
Re: Logging and containers
Posted by Ralph Goers <ra...@dslextreme.com>.
So that begs the question, when logging to stdout in a container is a console attached? i.e. can you normally view the output like you could in a regular VM or is it all redirected somewhere else? I haven’t worked much with Docker yet so I am afraid I don’t know the answer.
Ralph
> On Aug 6, 2018, at 6:40 AM, Remko Popma <re...@gmail.com> wrote:
>
> It may be to do with whether a tty is attached and how fast it is: https://stackoverflow.com/questions/3857052/why-is-printing-to-stdout-so-slow-can-it-be-sped-up
>
> (Shameless plug) Every java main() method deserves http://picocli.info
>
>> On Aug 6, 2018, at 4:21, Ralph Goers <ra...@dslextreme.com> wrote:
>>
>> Our performance page shows that logging to the console is extremely slow. Yet one of the “best practices” for containers is to have the applications log to STDOUT or STDERR. This leads me to two questions:
>> Is the performance of writing to STDOUT just as bad in a container? I have no reason to believe it wouldn’t be but have no evidence.
>> Assuming performance is poor what are the realistic alternatives? Is there something more Log4j needs to be doing to play well in a cloud environment?
>>
>> Ralph
Re: Logging and containers
Posted by Remko Popma <re...@gmail.com>.
It may be to do with whether a tty is attached and how fast it is: https://stackoverflow.com/questions/3857052/why-is-printing-to-stdout-so-slow-can-it-be-sped-up
(Shameless plug) Every java main() method deserves http://picocli.info
> On Aug 6, 2018, at 4:21, Ralph Goers <ra...@dslextreme.com> wrote:
>
> Our performance page shows that logging to the console is extremely slow. Yet one of the “best practices” for containers is to have the applications log to STDOUT or STDERR. This leads me to two questions:
> Is the performance of writing to STDOUT just as bad in a container? I have no reason to believe it wouldn’t be but have no evidence.
> Assuming performance is poor what are the realistic alternatives? Is there something more Log4j needs to be doing to play well in a cloud environment?
>
> Ralph