LOG4CXX as a binary logger

[Alexander <ro...@mail.ru>]

Hello,

I want to do arbitrary binary data logging to the file.
After configuring a separate logger with config. given below
    <appender name="RF2" class="RollingFileAppender">
        <param name="File" value="c:/work/ptt/logs1/outputStrm.log" />
        <param name="MaxFileSize" value="1000KB" />
        <param name="MaxBackupIndex" value="100" />
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%m" />
        </layout>
    </appender>
    <logger name="outputStrm" additivity="false">
        <appender-ref ref="RF2" />
        <level value="DEBUG" />
    </logger>
I've done some test code:
(I use W2K and I won't use unicode for my current project)
    std::string s;
    s.resize(0xFF);
    for (BYTE i = 0;i<0xFF;++i)
    {
        s[i]=i;
    }
    OUTPUTSTREAM->forcedLog(::log4cxx::Level::INFO, s);
So, after that, an output file was 256 bytes instead 255!!!
I found, that it is due to "Streams were originally designed for text, so the 
default output mode is text. In text mode, the newline character (hexadecimal 
10) expands to a carriage return–linefeed (16-bit only). " (from MSDN).

The question is: how to do such logging, without changing the original LOG4CXX 
code?

Re: LOG4CXX as a binary logger

[Curt Arnold <ca...@apache.org>]

The message parameter on the logging calls is a string, that is a 
sequence of characters, not an arbitrary vector of bytes and log4cxx 
(and log4j and log4net for that matter) make no guarantees that 
arbitrary bytes will pass through the processing pipeline intact.  
Since the message is stated to be a string, log4cxx will make string 
like assumptions, that it is encoded in the current code page, that it 
can be safely converted to an unicode encoding and back, that it does 
not contain '\0', etc.

For example, your configuration could have just as easily been:

>     <appender name="RF2" class="RollingFileAppender">
>         <param name="File" value="c:/work/ptt/logs1/outputStrm.log" />
>         <param name="MaxFileSize" value="1000KB" />
>         <param name="MaxBackupIndex" value="100" />
            <param name="Encoding" value="UTF-16BE"/>
>         <layout class="org.apache.log4j.PatternLayout">
>             <param name="ConversionPattern" value="&#x0e59;%m" />
>         </layout>
>     </appender>

which would prefix the thai digit 9 to the message and output the file 
in the UTF-16BE encoding.  Since the configuration file could contain 
any unicode character and could specify a different encoding, log4cxx 
must have the freedom to convert everything internally to unicode, do 
the operations and then convert back to the specified or default output 
encoding.

I would suggest that you encode your messages as text strings.  If you 
really need to get a binary file back out, you could write an custom 
appender to decode the message and write the binary back out.  I'd 
suggest:

1. defining some byte encoding functions (signatures appropriate to how 
you store arbitrary byte sequences)

std::string encode(std::vector<unsigned char>&);
std::string encode(unsigned char*, size_t len);

2. Uses these methods in LOG4CXX_level macros

LOG4CXX_INFO(logger, encode(someBytes));

The LOG4CXX_level macros short-cut the evaluation of the message 
parameter, so if the message is not going to be output, the encode 
function will not be called.

This approach would allow you to still send the messages to the console 
or in emails where it could look like:

2005-04-26 16:33:32 somelogger INFO 0001020304...

and you could still have (if you need) an custom appender that would 
decode the message and write the desired binary output.