You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2020/07/13 15:59:16 UTC
[kudu-CR] loadgen: use multiple KuduClient instances
Hello Alexey Serbin,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/16184
to review the following change.
Change subject: loadgen: use multiple KuduClient instances
......................................................................
loadgen: use multiple KuduClient instances
KuduClient is threadsafe but doesn't perform well under high
concurrency. This adds a flag (defaulted true) to 'kudu perf loadgen'
that lets it use a separate client for each inserter thread. This speeds
up the insertion substantially (about 30% in my quick local test, but
could be more depending on the machine and target cluster specs)
Change-Id: I722a0761e97894857e2ca3c0286b4467d7f27d47
---
M src/kudu/tools/tool_action_perf.cc
1 file changed, 22 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/84/16184/1
--
To view, visit http://gerrit.cloudera.org:8080/16184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I722a0761e97894857e2ca3c0286b4467d7f27d47
Gerrit-Change-Number: 16184
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
[kudu-CR] loadgen: use multiple KuduClient instances
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16184 )
Change subject: loadgen: use multiple KuduClient instances
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I722a0761e97894857e2ca3c0286b4467d7f27d47
Gerrit-Change-Number: 16184
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 16 Jul 2020 17:09:06 +0000
Gerrit-HasComments: No
[kudu-CR] loadgen: use multiple KuduClient instances
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16184 )
Change subject: loadgen: use multiple KuduClient instances
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I722a0761e97894857e2ca3c0286b4467d7f27d47
Gerrit-Change-Number: 16184
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Mon, 13 Jul 2020 16:12:51 +0000
Gerrit-HasComments: No
[kudu-CR] loadgen: use multiple KuduClient instances
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16184
to look at the new patch set (#2).
Change subject: loadgen: use multiple KuduClient instances
......................................................................
loadgen: use multiple KuduClient instances
KuduClient is threadsafe but doesn't perform well under high
concurrency. This adds a flag (defaulted true) to 'kudu perf loadgen'
that lets it use a separate client for each inserter thread. This speeds
up the insertion substantially (about 30% in my quick local test, but
could be more depending on the machine and target cluster specs)
Change-Id: I722a0761e97894857e2ca3c0286b4467d7f27d47
---
M src/kudu/tools/tool_action_perf.cc
1 file changed, 119 insertions(+), 98 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/84/16184/2
--
To view, visit http://gerrit.cloudera.org:8080/16184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I722a0761e97894857e2ca3c0286b4467d7f27d47
Gerrit-Change-Number: 16184
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] loadgen: use multiple KuduClient instances
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16184 )
Change subject: loadgen: use multiple KuduClient instances
......................................................................
loadgen: use multiple KuduClient instances
KuduClient is threadsafe but doesn't perform well under high
concurrency. This adds a flag (defaulted true) to 'kudu perf loadgen'
that lets it use a separate client for each inserter thread. This speeds
up the insertion substantially (about 30% in my quick local test, but
could be more depending on the machine and target cluster specs)
Change-Id: I722a0761e97894857e2ca3c0286b4467d7f27d47
Reviewed-on: http://gerrit.cloudera.org:8080/16184
Tested-by: Kudu Jenkins
Reviewed-by: Andrew Wong <aw...@cloudera.com>
---
M src/kudu/tools/tool_action_perf.cc
1 file changed, 119 insertions(+), 98 deletions(-)
Approvals:
Kudu Jenkins: Verified
Andrew Wong: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/16184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I722a0761e97894857e2ca3c0286b4467d7f27d47
Gerrit-Change-Number: 16184
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>