You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by eo...@apache.org on 2017/05/25 12:32:03 UTC
bookkeeper git commit: BOOKKEEPER-1075: BK LedgerMetadata: more
memory-efficient parsing of configs
Repository: bookkeeper
Updated Branches:
refs/heads/master 7b1eec470 -> 64bedc2f9
BOOKKEEPER-1075: BK LedgerMetadata: more memory-efficient parsing of configs
It is the contribution from Alex Yarmula
commit 9d9d7dd26235a9beda4421b7bed750fea1789076
Author: Alex Yarmula <aktwitter.com>
Date: Wed Sep 23 05:57:30 2015 -0700
BK LedgerMetadata: more memory-efficient parsing of configs
Looking at the most prevalent client-side memory allocations, I noticed that we allocate 4KB every time we open a ledger. This is caused by allocating a 4KB buffer (in TextFormat.toStringBuilder) to account for the maximum possible Protobufs message, which is unnecessary in our case: we know the exact size of the metadata ( << 500 B) and don't need to allocate more.
TextFormat.merge(Readable, Message.Builder) is the current method we use. This changes to use TextFormat.merge(CharSequence, Message.Builder), which avoids the extra 4K allocation conversion + an extra StringBuilder.
RB_ID=745700
Author: Alex Yarmula <ak...@twitter.com>
Author: Sijie Guo <si...@apache.org>
Reviewers: Enrico Olivelli, Sijie Guo
Closes #162 from sijie/bk_ledger_metadata_efficiency
Project: http://git-wip-us.apache.org/repos/asf/bookkeeper/repo
Commit: http://git-wip-us.apache.org/repos/asf/bookkeeper/commit/64bedc2f
Tree: http://git-wip-us.apache.org/repos/asf/bookkeeper/tree/64bedc2f
Diff: http://git-wip-us.apache.org/repos/asf/bookkeeper/diff/64bedc2f
Branch: refs/heads/master
Commit: 64bedc2f92d5bb17783c6ea2a2db7ea29170f264
Parents: 7b1eec4
Author: Alex Yarmula <ak...@twitter.com>
Authored: Thu May 25 14:31:57 2017 +0200
Committer: eolivelli <eo...@apache.org>
Committed: Thu May 25 14:31:57 2017 +0200
----------------------------------------------------------------------
.../java/org/apache/bookkeeper/client/LedgerMetadata.java | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/bookkeeper/blob/64bedc2f/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
----------------------------------------------------------------------
diff --git a/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java b/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
index f539186..b752148 100644
--- a/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
+++ b/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
@@ -29,6 +29,7 @@ import org.slf4j.LoggerFactory;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.StringReader;
+import java.nio.CharBuffer;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Iterator;
@@ -393,8 +394,15 @@ public class LedgerMetadata {
return parseVersion1Config(lc, reader);
}
+ // remaining size is total minus the length of the version line and '\n'
+ char[] configBuffer = new char[config.length() - (versionLine.length() + 1)];
+ if (configBuffer.length != reader.read(configBuffer, 0, configBuffer.length)) {
+ throw new IOException("Invalid metadata buffer");
+ }
+
LedgerMetadataFormat.Builder builder = LedgerMetadataFormat.newBuilder();
- TextFormat.merge(reader, builder);
+
+ TextFormat.merge((CharSequence) CharBuffer.wrap(configBuffer), builder);
LedgerMetadataFormat data = builder.build();
lc.writeQuorumSize = data.getQuorumSize();
if (data.hasCtime()) {