You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2018/02/01 04:28:29 UTC
[kudu-CR] jsonwriter: do some internal buffering in front of ostringstream
Hello Will Berkeley, Mike Percy,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/9181
to review the following change.
Change subject: jsonwriter: do some internal buffering in front of ostringstream
......................................................................
jsonwriter: do some internal buffering in front of ostringstream
JsonWriter outputs into a std::ostringstream, and the RapidJSON
interface is such that we always append one character at a time.
ostringstream::put(char) is not inlined, so this is very slow.
This patch adds a small buffer in front of the ostringstream. We flush
the buffer on the end of any JSON array or object.
The patch adds a simple benchmark to show the improvement:
Before:
I0131 20:26:09.437477 16201 jsonwriter-test.cc:187] Throughput: 87.0644MB/sec
After:
I0131 20:24:17.889117 15929 jsonwriter-test.cc:187] Throughput: 154.192MB/sec
The remaining perf issues seem to be due to our use of protobuf
reflection.
Change-Id: Ib45d58187cf80ecfe2f5a25e40d7042e4a27619d
---
M src/kudu/util/jsonwriter-test.cc
M src/kudu/util/jsonwriter.cc
2 files changed, 73 insertions(+), 21 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/81/9181/1
--
To view, visit http://gerrit.cloudera.org:8080/9181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib45d58187cf80ecfe2f5a25e40d7042e4a27619d
Gerrit-Change-Number: 9181
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] jsonwriter: do some internal buffering in front of ostringstream
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Will Berkeley, Mike Percy, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9181
to look at the new patch set (#2).
Change subject: jsonwriter: do some internal buffering in front of ostringstream
......................................................................
jsonwriter: do some internal buffering in front of ostringstream
JsonWriter outputs into a std::ostringstream, and the RapidJSON
interface is such that we always append one character at a time.
ostringstream::put(char) is not inlined, so this is very slow.
This patch adds a small buffer in front of the ostringstream. We flush
the buffer on the end of any JSON array or object.
The patch adds a simple benchmark to show the improvement:
Before:
I0131 20:26:09.437477 16201 jsonwriter-test.cc:187] Throughput: 87.0644MB/sec
After:
I0131 20:24:17.889117 15929 jsonwriter-test.cc:187] Throughput: 154.192MB/sec
The remaining perf issues seem to be due to our use of protobuf
reflection.
Change-Id: Ib45d58187cf80ecfe2f5a25e40d7042e4a27619d
---
M src/kudu/util/jsonwriter-test.cc
M src/kudu/util/jsonwriter.cc
2 files changed, 77 insertions(+), 21 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/81/9181/2
--
To view, visit http://gerrit.cloudera.org:8080/9181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib45d58187cf80ecfe2f5a25e40d7042e4a27619d
Gerrit-Change-Number: 9181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] jsonwriter: do some internal buffering in front of ostringstream
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/9181 )
Change subject: jsonwriter: do some internal buffering in front of ostringstream
......................................................................
jsonwriter: do some internal buffering in front of ostringstream
JsonWriter outputs into a std::ostringstream, and the RapidJSON
interface is such that we always append one character at a time.
ostringstream::put(char) is not inlined, so this is very slow.
This patch adds a small buffer in front of the ostringstream. We flush
the buffer on the end of any JSON array or object.
The patch adds a simple benchmark to show the improvement:
Before:
I0131 20:26:09.437477 16201 jsonwriter-test.cc:187] Throughput: 87.0644MB/sec
After:
I0131 20:24:17.889117 15929 jsonwriter-test.cc:187] Throughput: 154.192MB/sec
The remaining perf issues seem to be due to our use of protobuf
reflection.
Change-Id: Ib45d58187cf80ecfe2f5a25e40d7042e4a27619d
Reviewed-on: http://gerrit.cloudera.org:8080/9181
Tested-by: Kudu Jenkins
Reviewed-by: Will Berkeley <wd...@gmail.com>
---
M src/kudu/util/jsonwriter-test.cc
M src/kudu/util/jsonwriter.cc
2 files changed, 77 insertions(+), 21 deletions(-)
Approvals:
Kudu Jenkins: Verified
Will Berkeley: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/9181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib45d58187cf80ecfe2f5a25e40d7042e4a27619d
Gerrit-Change-Number: 9181
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] jsonwriter: do some internal buffering in front of ostringstream
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/9181 )
Change subject: jsonwriter: do some internal buffering in front of ostringstream
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/9181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib45d58187cf80ecfe2f5a25e40d7042e4a27619d
Gerrit-Change-Number: 9181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Fri, 02 Feb 2018 21:17:02 +0000
Gerrit-HasComments: No