You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@logging.apache.org by Gaurav <ga...@gmail.com> on 2019/06/27 12:28:09 UTC
Do I need to clear programmatic loggers/appenders
Hi all,
My application creates millions of loggers and appenders.
I'm worried about the memory usage.
For that, I am doing following things.
1.Remove appender from LoggerConfig.
2.Stop the LoggerConfig.
3. Remove logger from Configuration.
But when I do the performance test, it prints the errors on console that "Attempted to append to non-started appender".
Do I need to clear them like this? Is there any better way to do it in a performance intensive application?
Please assist.
Re: Do I need to clear programmatic loggers/appenders
Posted by Ralph Goers <ra...@dslextreme.com>.
Why are you dynamically creating loggers? I know of no valid reason to do that.
Ralph
> On Jun 27, 2019, at 11:38 AM, Gaurav <ga...@gmail.com> wrote:
>
> I am creating millions of loggers. As, the logger is requested at runtime, I cannot store the configuration in the static log4j2.xml. So, I create a rolling file appender and attach it to a logger.
>
> On 2019/06/27 13:14:28, Ralph Goers <ra...@dslextreme.com> wrote:
>> You are creating millions of Loggers or millions of LoggerConfigs? What you are doing is incomplete. But why would you be dynamically creating millions of Loggers and Appenders? Whatever you are doing I am sure there is a better way to do it. Can you please describe your use case and why you think what you are doing solves it?
>>
>> Ralph
>>
>>> On Jun 27, 2019, at 5:28 AM, Gaurav <ga...@gmail.com> wrote:
>>>
>>> Hi all,
>>>
>>> My application creates millions of loggers and appenders.
>>>
>>> I'm worried about the memory usage.
>>>
>>> For that, I am doing following things.
>>> 1.Remove appender from LoggerConfig.
>>> 2.Stop the LoggerConfig.
>>> 3. Remove logger from Configuration.
>>>
>>> But when I do the performance test, it prints the errors on console that "Attempted to append to non-started appender".
>>>
>>> Do I need to clear them like this? Is there any better way to do it in a performance intensive application?
>>>
>>> Please assist.
>>>
>>
>>
>>
>
Re: Do I need to clear programmatic loggers/appenders
Posted by Matt Sicker <bo...@gmail.com>.
If you’re making loggers per user, for example, that might make more sense
as thread context data.
On Sat, Jun 29, 2019 at 06:11, Dominik Psenner <dp...@gmail.com> wrote:
> Creating this many appenders is troublesome also for any operating system
> because the number of open (file) handles is limited. A little bit old anf
> thus possibly outdated but still provides insights into windows specific
> details and highlights some corner stones for orientation:
>
>
> https://blogs.technet.microsoft.com/markrussinovich/2009/09/29/pushing-the-limits-of-windows-handles/
> --
> Dominik Psenner
>
> On Sat, Jun 29, 2019, 12:00 Jochen Wiedmann <jo...@gmail.com>
> wrote:
>
> > You are creating millions of loggers? Meaning either of
> >
> > - I have a million different logger Id's, and create a logger for
> > every single one, or
> > - I have a limited number of different logger Id's, but invoke
> > LoggerContext.getLogger(String), or
> > LogManager.getLogger(String), or something similar, with the same
> > id's, over and over again?
> >
> > Whatever, but whyever do you need millions of Appenders?
> >
> > Jochen
> >
> > On Thu, Jun 27, 2019 at 8:38 PM Gaurav <ga...@gmail.com> wrote:
> > >
> > > I am creating millions of loggers. As, the logger is requested at
> > runtime, I cannot store the configuration in the static log4j2.xml. So, I
> > create a rolling file appender and attach it to a logger.
> > >
> > > On 2019/06/27 13:14:28, Ralph Goers <ra...@dslextreme.com>
> wrote:
> > > > You are creating millions of Loggers or millions of LoggerConfigs?
> > What you are doing is incomplete. But why would you be dynamically
> creating
> > millions of Loggers and Appenders? Whatever you are doing I am sure there
> > is a better way to do it. Can you please describe your use case and why
> you
> > think what you are doing solves it?
> > > >
> > > > Ralph
> > > >
> > > > > On Jun 27, 2019, at 5:28 AM, Gaurav <ga...@gmail.com> wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > My application creates millions of loggers and appenders.
> > > > >
> > > > > I'm worried about the memory usage.
> > > > >
> > > > > For that, I am doing following things.
> > > > > 1.Remove appender from LoggerConfig.
> > > > > 2.Stop the LoggerConfig.
> > > > > 3. Remove logger from Configuration.
> > > > >
> > > > > But when I do the performance test, it prints the errors on console
> > that "Attempted to append to non-started appender".
> > > > >
> > > > > Do I need to clear them like this? Is there any better way to do it
> > in a performance intensive application?
> > > > >
> > > > > Please assist.
> > > > >
> > > >
> > > >
> > > >
> >
>
--
Matt Sicker <bo...@gmail.com>
Re: Do I need to clear programmatic loggers/appenders
Posted by Dominik Psenner <dp...@gmail.com>.
Creating this many appenders is troublesome also for any operating system
because the number of open (file) handles is limited. A little bit old anf
thus possibly outdated but still provides insights into windows specific
details and highlights some corner stones for orientation:
https://blogs.technet.microsoft.com/markrussinovich/2009/09/29/pushing-the-limits-of-windows-handles/
--
Dominik Psenner
On Sat, Jun 29, 2019, 12:00 Jochen Wiedmann <jo...@gmail.com>
wrote:
> You are creating millions of loggers? Meaning either of
>
> - I have a million different logger Id's, and create a logger for
> every single one, or
> - I have a limited number of different logger Id's, but invoke
> LoggerContext.getLogger(String), or
> LogManager.getLogger(String), or something similar, with the same
> id's, over and over again?
>
> Whatever, but whyever do you need millions of Appenders?
>
> Jochen
>
> On Thu, Jun 27, 2019 at 8:38 PM Gaurav <ga...@gmail.com> wrote:
> >
> > I am creating millions of loggers. As, the logger is requested at
> runtime, I cannot store the configuration in the static log4j2.xml. So, I
> create a rolling file appender and attach it to a logger.
> >
> > On 2019/06/27 13:14:28, Ralph Goers <ra...@dslextreme.com> wrote:
> > > You are creating millions of Loggers or millions of LoggerConfigs?
> What you are doing is incomplete. But why would you be dynamically creating
> millions of Loggers and Appenders? Whatever you are doing I am sure there
> is a better way to do it. Can you please describe your use case and why you
> think what you are doing solves it?
> > >
> > > Ralph
> > >
> > > > On Jun 27, 2019, at 5:28 AM, Gaurav <ga...@gmail.com> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > My application creates millions of loggers and appenders.
> > > >
> > > > I'm worried about the memory usage.
> > > >
> > > > For that, I am doing following things.
> > > > 1.Remove appender from LoggerConfig.
> > > > 2.Stop the LoggerConfig.
> > > > 3. Remove logger from Configuration.
> > > >
> > > > But when I do the performance test, it prints the errors on console
> that "Attempted to append to non-started appender".
> > > >
> > > > Do I need to clear them like this? Is there any better way to do it
> in a performance intensive application?
> > > >
> > > > Please assist.
> > > >
> > >
> > >
> > >
>
Re: Do I need to clear programmatic loggers/appenders
Posted by Jochen Wiedmann <jo...@gmail.com>.
You are creating millions of loggers? Meaning either of
- I have a million different logger Id's, and create a logger for
every single one, or
- I have a limited number of different logger Id's, but invoke
LoggerContext.getLogger(String), or
LogManager.getLogger(String), or something similar, with the same
id's, over and over again?
Whatever, but whyever do you need millions of Appenders?
Jochen
On Thu, Jun 27, 2019 at 8:38 PM Gaurav <ga...@gmail.com> wrote:
>
> I am creating millions of loggers. As, the logger is requested at runtime, I cannot store the configuration in the static log4j2.xml. So, I create a rolling file appender and attach it to a logger.
>
> On 2019/06/27 13:14:28, Ralph Goers <ra...@dslextreme.com> wrote:
> > You are creating millions of Loggers or millions of LoggerConfigs? What you are doing is incomplete. But why would you be dynamically creating millions of Loggers and Appenders? Whatever you are doing I am sure there is a better way to do it. Can you please describe your use case and why you think what you are doing solves it?
> >
> > Ralph
> >
> > > On Jun 27, 2019, at 5:28 AM, Gaurav <ga...@gmail.com> wrote:
> > >
> > > Hi all,
> > >
> > > My application creates millions of loggers and appenders.
> > >
> > > I'm worried about the memory usage.
> > >
> > > For that, I am doing following things.
> > > 1.Remove appender from LoggerConfig.
> > > 2.Stop the LoggerConfig.
> > > 3. Remove logger from Configuration.
> > >
> > > But when I do the performance test, it prints the errors on console that "Attempted to append to non-started appender".
> > >
> > > Do I need to clear them like this? Is there any better way to do it in a performance intensive application?
> > >
> > > Please assist.
> > >
> >
> >
> >
Re: Do I need to clear programmatic loggers/appenders
Posted by Gaurav <ga...@gmail.com>.
I am creating millions of loggers. As, the logger is requested at runtime, I cannot store the configuration in the static log4j2.xml. So, I create a rolling file appender and attach it to a logger.
On 2019/06/27 13:14:28, Ralph Goers <ra...@dslextreme.com> wrote:
> You are creating millions of Loggers or millions of LoggerConfigs? What you are doing is incomplete. But why would you be dynamically creating millions of Loggers and Appenders? Whatever you are doing I am sure there is a better way to do it. Can you please describe your use case and why you think what you are doing solves it?
>
> Ralph
>
> > On Jun 27, 2019, at 5:28 AM, Gaurav <ga...@gmail.com> wrote:
> >
> > Hi all,
> >
> > My application creates millions of loggers and appenders.
> >
> > I'm worried about the memory usage.
> >
> > For that, I am doing following things.
> > 1.Remove appender from LoggerConfig.
> > 2.Stop the LoggerConfig.
> > 3. Remove logger from Configuration.
> >
> > But when I do the performance test, it prints the errors on console that "Attempted to append to non-started appender".
> >
> > Do I need to clear them like this? Is there any better way to do it in a performance intensive application?
> >
> > Please assist.
> >
>
>
>
Re: Do I need to clear programmatic loggers/appenders
Posted by Ralph Goers <ra...@dslextreme.com>.
You are creating millions of Loggers or millions of LoggerConfigs? What you are doing is incomplete. But why would you be dynamically creating millions of Loggers and Appenders? Whatever you are doing I am sure there is a better way to do it. Can you please describe your use case and why you think what you are doing solves it?
Ralph
> On Jun 27, 2019, at 5:28 AM, Gaurav <ga...@gmail.com> wrote:
>
> Hi all,
>
> My application creates millions of loggers and appenders.
>
> I'm worried about the memory usage.
>
> For that, I am doing following things.
> 1.Remove appender from LoggerConfig.
> 2.Stop the LoggerConfig.
> 3. Remove logger from Configuration.
>
> But when I do the performance test, it prints the errors on console that "Attempted to append to non-started appender".
>
> Do I need to clear them like this? Is there any better way to do it in a performance intensive application?
>
> Please assist.
>