You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-user@logging.apache.org by Scott Harrington <sc...@sns-usa.com> on 2014/07/11 06:23:26 UTC
Make LogEvent implementations Externalizable
Ralph & co:
I hear you're gearing up for the release.
Last weekend I scratched an itch of mine relating to SocketAppender ->
SocketServer bandwidth, and was able to reduce a 500-character message
from around 1700 bytes to 700 bytes on the wire (it's easy to improve on
Java's default serialization).
I was going to submit an enhancement request with patch to JIRA but
instead I went on vacation for two weeks.
I made RingBufferLogEvent implement Externalizable, i.e. hand-coded
writeExternal / readExternal methods. I did NOT have time to make an
equivalent change to Log4jLogEvent, or to write up any performance tests
or regression tests.
Should I submit what I have for discussion and hopeful inclusion in 2.0?
Or will it have to wait for 2.1?
If we wait, then due to the necessary serialVersionUID change, v2.0
SocketAppender would not be able to talk to v2.1 SocketServer or vice
versa (unless ugly duplicate versions are maintained).
Below is what the added code looks like. I only tested in
RingBufferLogEvent but should be similarly usable in Log4jLogEvent, and
perhaps we should discuss a RingBufferLogEvent.readResolve that makes them
all become Log4jLogEvents on the SocketServer (receiving) end.
...
public void writeExternal(ObjectOutput out) throws IOException {
getThrownProxy();
out.writeByte(1); // wireFormat
int presenceMap = (loggerName == null ? 0 : 0x1) |
(marker == null ? 0 : 0x2) |
(fqcn == null ? 0 : 0x4) |
(level == null ? 0 : 0x8) |
(message == null ? 0 : (0x10 | (isSerializeAsString(message) ? 0 : 0x20))) |
(thrownProxy == null ? 0 : 0x40) |
(contextMap == null ? 0 : 0x80) |
(contextStack == null ? 0 : 0x100 | (contextStack.getDepth() == 0 ? 0 : 0x200)) |
(threadName == null ? 0 : 0x400) |
(location == null ? 0 : 0x800);
out.writeShort(presenceMap);
if (loggerName != null) {
out.writeUTF(loggerName);
}
if (marker != null) {
out.writeObject(marker);
}
if (fqcn != null) {
out.writeUTF(fqcn);
}
if (level != null) {
out.writeUTF(level.name());
}
if (message != null) {
if (isSerializeAsString(message)) {
out.writeUTF(message.getFormattedMessage());
}
else {
out.writeObject(message);
}
}
if (thrownProxy != null) {
out.writeObject(thrownProxy);
}
if (contextMap != null) {
writeString2StringMap(out, contextMap);
}
if (contextStack != null && contextStack.getDepth() != 0) {
out.writeObject(contextStack);
}
if (threadName != null) {
out.writeUTF(threadName);
}
if (location != null) {
out.writeUTF(location.getClassName());
out.writeUTF(location.getMethodName());
if ((presenceMap & 0x1000) != 0) {
out.writeUTF(location.getFileName());
}
out.writeInt(location.getLineNumber());
}
out.writeLong(currentTimeMillis);
out.writeBoolean(endOfBatch);
out.writeBoolean(includeLocation);
}
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
int wireFormat = in.readByte();
if (wireFormat == 1) {
int presenceMap = in.readShort();
loggerName = (presenceMap & 0x1) == 0 ? null :
in.readUTF();
marker = (presenceMap & 0x2) == 0 ? null :
(Marker) in.readObject();
fqcn = (presenceMap & 0x4) == 0 ? null :
in.readUTF();
level = (presenceMap & 0x8) == 0 ? null :
Level.valueOf(in.readUTF());
message = (presenceMap & 0x10) == 0 ? null :
(presenceMap & 0x20) == 0 ? new SimpleMessage(in.readUTF()) : (Message) in.readObject();
thrownProxy = (presenceMap & 0x40) == 0 ? null :
(ThrowableProxy) in.readObject();
contextMap = (presenceMap & 0x80) == 0 ? null :
readString2StringMap(in);
contextStack = (presenceMap & 0x100) == 0 ? null :
(presenceMap & 0x200) == 0 ? ThreadContext.EMPTY_STACK : (ContextStack) in.readObject();
threadName = (presenceMap & 0x400) == 0 ? null :
in.readUTF();
location = (presenceMap & 0x800) == 0 ? null :
new StackTraceElement(in.readUTF(), in.readUTF(), (presenceMap & 0x1000) == 0 ? null : in.readUTF(), in.readInt());
currentTimeMillis = in.readLong();
endOfBatch = in.readBoolean();
includeLocation = in.readBoolean();
}
else {
throw new StreamCorruptedException("Unsupported LogEvent wire format " + wireFormat);
}
}
private static boolean isSerializeAsString(Message message)
{
return message instanceof SimpleMessage || message instanceof ObjectMessage;
}
private void writeString2StringMap(ObjectOutput out, Map<String, String> map) throws IOException
{
out.writeInt(map.size());
for (Map.Entry<String, String> entry : map.entrySet()) {
out.writeUTF(entry.getKey());
out.writeUTF(entry.getValue());
}
}
private static Map<String, String> readString2StringMap(ObjectInput in) throws ClassNotFoundException, IOException
{
int size = in.readInt();
if (size == 0) {
return Collections.emptyMap();
}
Map<String, String> map = new HashMap<String, String>(size);
while (size-- > 0) {
map.put(in.readUTF(), in.readUTF());
}
return map;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: Make LogEvent implementations Externalizable
Posted by Ralph Goers <ra...@dslextreme.com>.
I’d be afraid of breaking compatibility even now. However, I think what you really want to do is to create an ExternalizedLayout and then just use that instead of the default SerializedLayout. If you want to supply that Layout as a patch to a Jira issue it could be added at any time.
Ralph
On Jul 10, 2014, at 9:23 PM, Scott Harrington <sc...@sns-usa.com> wrote:
> Ralph & co:
>
> I hear you're gearing up for the release.
>
> Last weekend I scratched an itch of mine relating to SocketAppender -> SocketServer bandwidth, and was able to reduce a 500-character message from around 1700 bytes to 700 bytes on the wire (it's easy to improve on Java's default serialization).
>
> I was going to submit an enhancement request with patch to JIRA but instead I went on vacation for two weeks.
>
> I made RingBufferLogEvent implement Externalizable, i.e. hand-coded writeExternal / readExternal methods. I did NOT have time to make an equivalent change to Log4jLogEvent, or to write up any performance tests or regression tests.
>
> Should I submit what I have for discussion and hopeful inclusion in 2.0?
>
> Or will it have to wait for 2.1?
>
> If we wait, then due to the necessary serialVersionUID change, v2.0 SocketAppender would not be able to talk to v2.1 SocketServer or vice versa (unless ugly duplicate versions are maintained).
>
> Below is what the added code looks like. I only tested in RingBufferLogEvent but should be similarly usable in Log4jLogEvent, and perhaps we should discuss a RingBufferLogEvent.readResolve that makes them all become Log4jLogEvents on the SocketServer (receiving) end.
>
> ...
>
> public void writeExternal(ObjectOutput out) throws IOException {
> getThrownProxy();
> out.writeByte(1); // wireFormat
> int presenceMap = (loggerName == null ? 0 : 0x1) |
> (marker == null ? 0 : 0x2) |
> (fqcn == null ? 0 : 0x4) |
> (level == null ? 0 : 0x8) |
> (message == null ? 0 : (0x10 | (isSerializeAsString(message) ? 0 : 0x20))) |
> (thrownProxy == null ? 0 : 0x40) |
> (contextMap == null ? 0 : 0x80) |
> (contextStack == null ? 0 : 0x100 | (contextStack.getDepth() == 0 ? 0 : 0x200)) |
> (threadName == null ? 0 : 0x400) |
> (location == null ? 0 : 0x800);
> out.writeShort(presenceMap);
> if (loggerName != null) {
> out.writeUTF(loggerName);
> }
> if (marker != null) {
> out.writeObject(marker);
> }
> if (fqcn != null) {
> out.writeUTF(fqcn);
> }
> if (level != null) {
> out.writeUTF(level.name());
> }
> if (message != null) {
> if (isSerializeAsString(message)) {
> out.writeUTF(message.getFormattedMessage());
> }
> else {
> out.writeObject(message);
> }
> }
> if (thrownProxy != null) {
> out.writeObject(thrownProxy);
> }
> if (contextMap != null) {
> writeString2StringMap(out, contextMap);
> }
> if (contextStack != null && contextStack.getDepth() != 0) {
> out.writeObject(contextStack);
> }
> if (threadName != null) {
> out.writeUTF(threadName);
> }
> if (location != null) {
> out.writeUTF(location.getClassName());
> out.writeUTF(location.getMethodName());
> if ((presenceMap & 0x1000) != 0) {
> out.writeUTF(location.getFileName());
> }
> out.writeInt(location.getLineNumber());
> }
> out.writeLong(currentTimeMillis);
> out.writeBoolean(endOfBatch);
> out.writeBoolean(includeLocation);
> }
>
> public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
> int wireFormat = in.readByte();
> if (wireFormat == 1) {
> int presenceMap = in.readShort();
> loggerName = (presenceMap & 0x1) == 0 ? null :
> in.readUTF();
> marker = (presenceMap & 0x2) == 0 ? null :
> (Marker) in.readObject();
> fqcn = (presenceMap & 0x4) == 0 ? null :
> in.readUTF();
> level = (presenceMap & 0x8) == 0 ? null :
> Level.valueOf(in.readUTF());
> message = (presenceMap & 0x10) == 0 ? null :
> (presenceMap & 0x20) == 0 ? new SimpleMessage(in.readUTF()) : (Message) in.readObject();
> thrownProxy = (presenceMap & 0x40) == 0 ? null :
> (ThrowableProxy) in.readObject();
> contextMap = (presenceMap & 0x80) == 0 ? null :
> readString2StringMap(in);
> contextStack = (presenceMap & 0x100) == 0 ? null :
> (presenceMap & 0x200) == 0 ? ThreadContext.EMPTY_STACK : (ContextStack) in.readObject();
> threadName = (presenceMap & 0x400) == 0 ? null :
> in.readUTF();
> location = (presenceMap & 0x800) == 0 ? null :
> new StackTraceElement(in.readUTF(), in.readUTF(), (presenceMap & 0x1000) == 0 ? null : in.readUTF(), in.readInt());
> currentTimeMillis = in.readLong();
> endOfBatch = in.readBoolean();
> includeLocation = in.readBoolean();
> }
> else {
> throw new StreamCorruptedException("Unsupported LogEvent wire format " + wireFormat);
> }
> }
>
> private static boolean isSerializeAsString(Message message)
> {
> return message instanceof SimpleMessage || message instanceof ObjectMessage;
> }
>
> private void writeString2StringMap(ObjectOutput out, Map<String, String> map) throws IOException
> {
> out.writeInt(map.size());
> for (Map.Entry<String, String> entry : map.entrySet()) {
> out.writeUTF(entry.getKey());
> out.writeUTF(entry.getValue());
> }
> }
>
> private static Map<String, String> readString2StringMap(ObjectInput in) throws ClassNotFoundException, IOException
> {
> int size = in.readInt();
> if (size == 0) {
> return Collections.emptyMap();
> }
> Map<String, String> map = new HashMap<String, String>(size);
> while (size-- > 0) {
> map.put(in.readUTF(), in.readUTF());
> }
> return map;
> }
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org