You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Adar Dembo (Code Review)" <ge...@cloudera.org> on 2019/04/05 22:07:20 UTC
[kudu-CR] tool: perf tablet scan action
Hello Mike Percy, Todd Lipcon,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/12945
to review the following change.
Change subject: tool: perf tablet_scan action
......................................................................
tool: perf tablet_scan action
I found this action useful for benchmarking raw tserver scan performance
(i.e. by removing the client from the picture). Not sure it's generally
useful, but if it is, burying it in the 'perf' mode makes sense.
Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
---
M src/kudu/tools/tool_action_perf.cc
1 file changed, 70 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/45/12945/1
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 1
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] tool: perf tablet scan action
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Andrew Wong, Grant Henke, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12945
to look at the new patch set (#3).
Change subject: tool: perf tablet_scan action
......................................................................
tool: perf tablet_scan action
I found this action useful for benchmarking raw tserver scan performance
(i.e. by removing the client from the picture). Not sure it's generally
useful, but if it is, burying it in the 'perf' mode makes sense.
Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_perf.cc
2 files changed, 144 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/45/12945/3
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 3
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] tool: perf tablet scan action
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Patch Set 4: Verified+1
Overriding Jenkins, known flake.
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 4
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 09 Apr 2019 22:46:21 +0000
Gerrit-HasComments: No
[kudu-CR] tool: perf tablet scan action
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Removed reviewer Kudu Jenkins with the following votes:
* Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 4
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] tool: perf tablet scan action
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 4
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 09 Apr 2019 23:19:10 +0000
Gerrit-HasComments: No
[kudu-CR] tool: perf tablet scan action
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/12945/1/src/kudu/tools/tool_action_perf.cc
File src/kudu/tools/tool_action_perf.cc:
http://gerrit.cloudera.org:8080/#/c/12945/1/src/kudu/tools/tool_action_perf.cc@727
PS1, Line 727: MiniTabletServer mts(fs_root, HostPort("127.0.0.1", 0));
:
: // Don't bother heartbeating; we didn't start any masters.
: mts.options()->master_addresses.clear();
:
: RETURN_NOT_OK(mts.Start());
: RETURN_NOT_OK(mts.WaitStarted());
: scoped_refptr<TabletReplica> replica;
: RETURN_NOT_OK(mts.server()->tablet_manager()->GetTabletReplica(tablet_id, &replica));
Do we need the entire tserver? Could we get by just opening the FsManager and TabletMetadata, and constructing just the Tablet? Then you could actually use this on a real server.
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 1
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Sat, 06 Apr 2019 00:40:57 +0000
Gerrit-HasComments: Yes
[kudu-CR] tool: perf tablet scan action
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/12945/2/src/kudu/tools/tool_action_perf.cc
File src/kudu/tools/tool_action_perf.cc:
http://gerrit.cloudera.org:8080/#/c/12945/2/src/kudu/tools/tool_action_perf.cc@744
PS2, Line 744: FsManagerOpts());
> Nope; I think bootstrapping may rewrite WAL segments, the cmeta, the tmeta,
Double checked, here's a non-exhaustive list of the destructive things that bootstrapping may do:
- Rename the tablet's WAL segment directory.
- Create a new WAL segment for the tablet.
- Flush the tablet's superblock. (AlterSchema)
- Flush the tablet itself. (AlterSchema)
http://gerrit.cloudera.org:8080/#/c/12945/2/src/kudu/tools/tool_action_perf.cc@756
PS2, Line 756: if (FLAGS_use_hybrid_clock) {
: clock.reset(new HybridClock());
: } else {
: clock.reset(LogicalClock::CreateStartingAt(Timestamp::kInitialTimestamp));
: }
> I meant to follow-up on this. Without looking at the details, I thought per
Checked; a logical clock here should be safe.
http://gerrit.cloudera.org:8080/#/c/12945/2/src/kudu/tools/tool_action_perf.cc@869
PS2, Line 869: "Scan the contents of a local tablet"
> OK.
Done
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 2
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 09 Apr 2019 20:45:01 +0000
Gerrit-HasComments: Yes
[kudu-CR] tool: perf tablet scan action
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/12945/2/src/kudu/tools/tool_action_perf.cc
File src/kudu/tools/tool_action_perf.cc:
http://gerrit.cloudera.org:8080/#/c/12945/2/src/kudu/tools/tool_action_perf.cc@744
PS2, Line 744: FsManagerOpts());
> BTW can this tool be run on a currently-running tablet server? Is it read-o
Nope; I think bootstrapping may rewrite WAL segments, the cmeta, the tmeta, or more stuff. Will double check and then add a comment.
http://gerrit.cloudera.org:8080/#/c/12945/2/src/kudu/tools/tool_action_perf.cc@756
PS2, Line 756: if (FLAGS_use_hybrid_clock) {
: clock.reset(new HybridClock());
: } else {
: clock.reset(LogicalClock::CreateStartingAt(Timestamp::kInitialTimestamp));
: }
> What does using the hybrid clock get us as far as scans are concerned? Coul
I meant to follow-up on this. Without looking at the details, I thought perhaps some clock readings make their way into either in-memory state or into the rewritten WAL segments. If not, then always using a logical clock makes sense. Will check.
http://gerrit.cloudera.org:8080/#/c/12945/2/src/kudu/tools/tool_action_perf.cc@869
PS2, Line 869: "Scan the contents of a local tablet"
> nit: probably worth noting that this doesn't actually output the rows, othe
OK.
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 2
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 09 Apr 2019 19:23:49 +0000
Gerrit-HasComments: Yes
[kudu-CR] tool: perf tablet scan action
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/12945/1/src/kudu/tools/tool_action_perf.cc
File src/kudu/tools/tool_action_perf.cc:
http://gerrit.cloudera.org:8080/#/c/12945/1/src/kudu/tools/tool_action_perf.cc@727
PS1, Line 727: MiniTabletServer mts(fs_root, HostPort("127.0.0.1", 0));
:
: // Don't bother heartbeating; we didn't start any masters.
: mts.options()->master_addresses.clear();
:
: RETURN_NOT_OK(mts.Start());
: RETURN_NOT_OK(mts.WaitStarted());
: scoped_refptr<TabletReplica> replica;
: RETURN_NOT_OK(mts.server()->tablet_manager()->GetTabletReplica(tablet_id, &replica));
> Do we need the entire tserver? Could we get by just opening the FsManager a
I wish that were the case, but I need to bootstrap the tablet in order to scan any data in the WALs. AFAICT that means standing up a TSTabletManager, which means bringing up a full TabletServer.
Seems like maybe I could get away with something like what tablet_bootstrap-test or tablet_replica-test do, but it's a lot of setup work. Is there an easier way that I'm missing?
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 1
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 08 Apr 2019 00:37:52 +0000
Gerrit-HasComments: Yes
[kudu-CR] tool: perf tablet scan action
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Patch Set 2: Verified+1
Overriding Jenkins, a super flaky Java test failed three times.
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 2
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 09 Apr 2019 05:09:42 +0000
Gerrit-HasComments: No
[kudu-CR] tool: perf tablet scan action
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Patch Set 3: Code-Review+2
Thanks for checking!
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 3
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 09 Apr 2019 20:49:23 +0000
Gerrit-HasComments: No
[kudu-CR] tool: perf tablet scan action
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Removed reviewer Kudu Jenkins with the following votes:
* Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 2
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] tool: perf tablet scan action
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Kudu Jenkins, Andrew Wong, Grant Henke, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12945
to look at the new patch set (#2).
Change subject: tool: perf tablet_scan action
......................................................................
tool: perf tablet_scan action
I found this action useful for benchmarking raw tserver scan performance
(i.e. by removing the client from the picture). Not sure it's generally
useful, but if it is, burying it in the 'perf' mode makes sense.
Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_perf.cc
2 files changed, 146 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/45/12945/2
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 2
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] tool: perf tablet scan action
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/12945/2/src/kudu/tools/tool_action_perf.cc
File src/kudu/tools/tool_action_perf.cc:
http://gerrit.cloudera.org:8080/#/c/12945/2/src/kudu/tools/tool_action_perf.cc@744
PS2, Line 744: FsManagerOpts());
BTW can this tool be run on a currently-running tablet server? Is it read-only enough?
http://gerrit.cloudera.org:8080/#/c/12945/2/src/kudu/tools/tool_action_perf.cc@756
PS2, Line 756: if (FLAGS_use_hybrid_clock) {
: clock.reset(new HybridClock());
: } else {
: clock.reset(LogicalClock::CreateStartingAt(Timestamp::kInitialTimestamp));
: }
What does using the hybrid clock get us as far as scans are concerned? Could we always just use the logical clock?
http://gerrit.cloudera.org:8080/#/c/12945/2/src/kudu/tools/tool_action_perf.cc@869
PS2, Line 869: "Scan the contents of a local tablet"
nit: probably worth noting that this doesn't actually output the rows, other than a count.
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 2
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 09 Apr 2019 19:10:26 +0000
Gerrit-HasComments: Yes
[kudu-CR] tool: perf tablet scan action
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/12945/1/src/kudu/tools/tool_action_perf.cc
File src/kudu/tools/tool_action_perf.cc:
http://gerrit.cloudera.org:8080/#/c/12945/1/src/kudu/tools/tool_action_perf.cc@727
PS1, Line 727: MiniTabletServer mts(fs_root, HostPort("127.0.0.1", 0));
:
: // Don't bother heartbeating; we didn't start any masters.
: mts.options()->master_addresses.clear();
:
: RETURN_NOT_OK(mts.Start());
: RETURN_NOT_OK(mts.WaitStarted());
: scoped_refptr<TabletReplica> replica;
: RETURN_NOT_OK(mts.server()->tablet_manager()->GetTabletReplica(tablet_id, &replica));
> Yeah, I don't think there's an easier path, though the setup doesn't seem t
FWIW, I used it on non-minicluster data (i.e. superblock, cmeta, WAL segments, and containers that came from a real cluster).
But, maybe the setup work won't be so bad. I'll give it a shot.
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 1
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 08 Apr 2019 23:51:21 +0000
Gerrit-HasComments: Yes
[kudu-CR] tool: perf tablet scan action
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/12945/1/src/kudu/tools/tool_action_perf.cc
File src/kudu/tools/tool_action_perf.cc:
http://gerrit.cloudera.org:8080/#/c/12945/1/src/kudu/tools/tool_action_perf.cc@727
PS1, Line 727: MiniTabletServer mts(fs_root, HostPort("127.0.0.1", 0));
:
: // Don't bother heartbeating; we didn't start any masters.
: mts.options()->master_addresses.clear();
:
: RETURN_NOT_OK(mts.Start());
: RETURN_NOT_OK(mts.WaitStarted());
: scoped_refptr<TabletReplica> replica;
: RETURN_NOT_OK(mts.server()->tablet_manager()->GetTabletReplica(tablet_id, &replica));
> I wish that were the case, but I need to bootstrap the tablet in order to s
Yeah, I don't think there's an easier path, though the setup doesn't seem too complex (hopefully no surprises though).
I would be weary of committing this since it seems like it'd only really useful for mini-clusters, and it might be confusing to have it around as a documented tool if that's the case. If you think it's not worth the fuss, this could just remain a "useful but not worth the polish so it sits in gerrit" tool.
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 1
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 08 Apr 2019 05:48:55 +0000
Gerrit-HasComments: Yes
[kudu-CR] tool: perf tablet scan action
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12945 )
Change subject: tool: perf tablet_scan action
......................................................................
tool: perf tablet_scan action
I found this action useful for benchmarking raw tserver scan performance
(i.e. by removing the client from the picture). Not sure it's generally
useful, but if it is, burying it in the 'perf' mode makes sense.
Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Reviewed-on: http://gerrit.cloudera.org:8080/12945
Tested-by: Adar Dembo <ad...@cloudera.com>
Reviewed-by: Andrew Wong <aw...@cloudera.com>
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_perf.cc
2 files changed, 144 insertions(+), 9 deletions(-)
Approvals:
Adar Dembo: Verified
Andrew Wong: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/12945
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Gerrit-Change-Number: 12945
Gerrit-PatchSet: 5
Gerrit-Owner: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>