You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Perry Harrington <pe...@apsoft.com> on 2000/05/13 08:31:45 UTC

ANNOUNCE: BCLF - Binary Common Logging format

Hi all,

I've released some core code for a Binary ECLF equivelent.  It is
release .1, first release.

The advantages of the BCLF format are:

- compact; uses far less space per entry than ECLF
- machine readable; doens't require parsing to read logfile
- platform independent; all numerics are network byte order
- fast; since it's a binary file, and uses readv/writev, it's fast
- streamable; you could potentially stream the data to a socket, it's writeonly

I have seen a write rate of as many as 270,000 entries per second on a
466(66) Celeron, with a read rate of 80,000 (malloc) and 97,000 (static)
entries a second.

The reason the read rate is lower is because there are 2 reads vs the
single write during create.

I designed it as a format to greatly improve logfile data parsing and handling.

You can check out the project at:

	http://sourceforge.net/project/?group_id=5561

The download is there, it consists of a library of functions and a few test
programs.

There is no accompanying documentation, as the source is well commented.

--Perry

-- 
Perry Harrington       Linux rules all OSes.    APSoft      ()
email: perry@apsoft.com 			Think Blue. /\

Re: ANNOUNCE: BCLF - Binary Common Logging format

Posted by "Jeffrey W. Baker" <jw...@acm.org>.
On Mon, 15 May 2000, Bill Stoddard wrote:

> Does this have to be GPL'ed? Bleh...
> 
> Bill

I remember that the author specifically stated his intention to relicense
the code in an Apache-compatible way.  Right now that particular web page
has disappeared from sourceforge, but it was there a few days ago.  

-jwb

> 
> 
> > Hi all,
> > 
> > I've released some core code for a Binary ECLF equivelent.  It is
> > release .1, first release.
> > 
> > The advantages of the BCLF format are:
> > 
> > - compact; uses far less space per entry than ECLF
> > - machine readable; doens't require parsing to read logfile
> > - platform independent; all numerics are network byte order
> > - fast; since it's a binary file, and uses readv/writev, it's fast
> > - streamable; you could potentially stream the data to a socket, it's writeonly
> > 
> > I have seen a write rate of as many as 270,000 entries per second on a
> > 466(66) Celeron, with a read rate of 80,000 (malloc) and 97,000 (static)
> > entries a second.
> > 
> > The reason the read rate is lower is because there are 2 reads vs the
> > single write during create.
> > 
> > I designed it as a format to greatly improve logfile data parsing and handling.
> > 
> > You can check out the project at:
> > 
> > http://sourceforge.net/project/?group_id=5561
> > 
> > The download is there, it consists of a library of functions and a few test
> > programs.
> > 
> > There is no accompanying documentation, as the source is well commented.
> > 
> > --Perry
> > 
> > -- 
> > Perry Harrington       Linux rules all OSes.    APSoft      ()
> > email: perry@apsoft.com Think Blue. /\
> 
> 


Re: ANNOUNCE: BCLF - Binary Common Logging format

Posted by Bill Stoddard <st...@raleigh.ibm.com>.
Does this have to be GPL'ed? Bleh...

Bill


> Hi all,
> 
> I've released some core code for a Binary ECLF equivelent.  It is
> release .1, first release.
> 
> The advantages of the BCLF format are:
> 
> - compact; uses far less space per entry than ECLF
> - machine readable; doens't require parsing to read logfile
> - platform independent; all numerics are network byte order
> - fast; since it's a binary file, and uses readv/writev, it's fast
> - streamable; you could potentially stream the data to a socket, it's writeonly
> 
> I have seen a write rate of as many as 270,000 entries per second on a
> 466(66) Celeron, with a read rate of 80,000 (malloc) and 97,000 (static)
> entries a second.
> 
> The reason the read rate is lower is because there are 2 reads vs the
> single write during create.
> 
> I designed it as a format to greatly improve logfile data parsing and handling.
> 
> You can check out the project at:
> 
> http://sourceforge.net/project/?group_id=5561
> 
> The download is there, it consists of a library of functions and a few test
> programs.
> 
> There is no accompanying documentation, as the source is well commented.
> 
> --Perry
> 
> -- 
> Perry Harrington       Linux rules all OSes.    APSoft      ()
> email: perry@apsoft.com Think Blue. /\