You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4cxx-user@logging.apache.org by "RUOFF, LARS (LARS)" <la...@alcatel-lucent.com> on 2014/01/15 15:37:20 UTC
Logging to memory mapped files
Hi,
I'm using log4cxx v0.9.7 (for historical reasons)
My question is whether it is possible (in this or any other version) to work with memory mapped files.
Objective is to have the application log both to a rolling i/o-buffered file on disk for the long-term log and an unbuffered memory mapped file for short-term crash analysis.
(Because in a buffered file, the most recent logs will probably be lost when the application crashes).
So is there any possibility to have an log4cxx appender logging to a memory mapped file? (primary target is Linux)
Does anybody have had experience with implementing that? How would you do it?
Best regards,
Lars Ruoff
Re: Logging to memory mapped files
Posted by Thorsten Schöning <ts...@am-soft.de>.
Guten Tag Rhys Ulerich,
am Mittwoch, 15. Januar 2014 um 16:11 schrieben Sie:
> Yeah. I was thinking that one would toss the file onto something
> RAMish. But you're right that it won't have the performance profile
> you want.
Why would writing to a RAM-disk without buffering suffer performance?
It may be slower than using memory mapped files with buffering, but
the only problem coming into my mind would be filesystem overhead of
the RAM-disk without buffering and that should be minimal because you
don't change the file structure much but only call fsync pretty often.
Depending on one's setup the persistant logging could be implemented
outside the application by simply rsyncing the RAM-disk contents to a
more persistent storage, which may even improve overall application
performance because it doesn't need to log to two different targets
anymore.
I think it's worth measuring your approach.
Mit freundlichen Grüßen,
Thorsten Schöning
--
Thorsten Schöning E-Mail:Thorsten.Schoening@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/
Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04
AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
RE: Logging to memory mapped files
Posted by "RUOFF, LARS (LARS)" <la...@alcatel-lucent.com>.
But then I would need another application for the receiving side.
With memory mapped files, it's the OS that handles this transparently and flushes the file to disk when the logging application crashes.
That seems like a more elegant solution to me.
Lars
From: chandpriyankara@gmail.com [mailto:chandpriyankara@gmail.com] On Behalf Of chand priyankara
Sent: jeudi 16 janvier 2014 03:04
To: Log4CXX User
Subject: Re: Logging to memory mapped files
Hi,
What about socket appenders? wont it be better than file appender, in your case. You can receive the data from other side. I think it'll be faster.
regards,
Chand Priyankara
|BSc(Eng) - Electrical & Information
|(094) 773-361-566
|chand@engineering.com
|http://chandpriyankara.blogspot.com
sent via internet
On Wed, Jan 15, 2014 at 8:41 PM, Rhys Ulerich <rh...@gmail.com> wrote:
>> Curiosity, turning off the buffering on FileAppender isn't close enough to what you want?
>
> Well, maybe.
> But won't turning off buffering severely decrease performance because of higher i/o activity?
Yeah. I was thinking that one would toss the file onto something
RAMish. But you're right that it won't have the performance profile
you want.
Sorry,
Rhys
Re: Logging to memory mapped files
Posted by chand priyankara <ch...@engineering.com>.
Hi,
What about socket appenders? wont it be better than file appender, in your
case. You can receive the data from other side. I think it'll be faster.
regards,
*Chand Priyankara* [image: Facebook]<http://www.facebook.com/chand.priyankara>
[image: LinkedIn] <http://lk.linkedin.com/in/chandpriyankara> [image:
Blogger] <http://chandpriyankara.blogspot.com/> [image: Google
Plus]<http://plus.google.com/104246340732624023499>
|BSc(Eng) - Electrical & Information
|(094) 773-361-566
|chand@engineering.com
|http://chandpriyankara.blogspot.com
<http://www.iucnredlist.org/amazing-species>sent via internet
On Wed, Jan 15, 2014 at 8:41 PM, Rhys Ulerich <rh...@gmail.com>wrote:
> >> Curiosity, turning off the buffering on FileAppender isn't close enough
> to what you want?
> >
> > Well, maybe.
> > But won't turning off buffering severely decrease performance because of
> higher i/o activity?
>
> Yeah. I was thinking that one would toss the file onto something
> RAMish. But you're right that it won't have the performance profile
> you want.
>
> Sorry,
> Rhys
>
Re: Logging to memory mapped files
Posted by Rhys Ulerich <rh...@gmail.com>.
>> Curiosity, turning off the buffering on FileAppender isn't close enough to what you want?
>
> Well, maybe.
> But won't turning off buffering severely decrease performance because of higher i/o activity?
Yeah. I was thinking that one would toss the file onto something
RAMish. But you're right that it won't have the performance profile
you want.
Sorry,
Rhys
RE: Logging to memory mapped files
Posted by "RUOFF, LARS (LARS)" <la...@alcatel-lucent.com>.
Thanks for your prompt answer, Rhys.
I don't understand all implications of it yet (I'm new to log4cxx), but i will work through it.
> Curiosity, turning off the buffering on FileAppender isn't close enough to what you want?
Well, maybe.
But won't turning off buffering severely decrease performance because of higher i/o activity?
It is a real-time application with high expected log throughput.
But anyway, your question is legitimate, i should probably profile-test this.
Lars R.
-----Original Message-----
From: Rhys Ulerich [mailto:rhys.ulerich@gmail.com]
Sent: mercredi 15 janvier 2014 15:55
To: Log4CXX User
Subject: Re: Logging to memory mapped files
> So is there any possibility to have an log4cxx appender logging to a
> memory mapped file? (primary target is Linux)
>
> Does anybody have had experience with implementing that? How would you
> do it?
No experience, no. A rough hack would be to call WriterAppender::setWriter on a ConsoleAppender instance passing a custom Writer implementation that groks a mmap-ed file. I'm unconvinced this would work, however, given the behavior of the private ConsoleAppender::target member mentioned in ConsoleAppender::setTarget. A better version would be, I think, to create a new WriterAppender subclass doing what you want.
Curiosity, turning off the buffering on FileAppender isn't close enough to what you want?
- Rhys
Re: Logging to memory mapped files
Posted by Rhys Ulerich <rh...@gmail.com>.
> So is there any possibility to have an log4cxx appender logging to a memory
> mapped file? (primary target is Linux)
>
> Does anybody have had experience with implementing that? How would you do
> it?
No experience, no. A rough hack would be to call
WriterAppender::setWriter on a ConsoleAppender instance passing a
custom Writer implementation that groks a mmap-ed file. I'm
unconvinced this would work, however, given the behavior of the
private ConsoleAppender::target member mentioned in
ConsoleAppender::setTarget. A better version would be, I think, to
create a new WriterAppender subclass doing what you want.
Curiosity, turning off the buffering on FileAppender isn't close
enough to what you want?
- Rhys