You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Yingchun Lai (Code Review)" <ge...@cloudera.org> on 2021/11/24 08:56:38 UTC
[kudu-CR] WIP Speedup tablet bootstrap
Yingchun Lai has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18053
Change subject: WIP Speedup tablet bootstrap
......................................................................
WIP Speedup tablet bootstrap
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
4 files changed, 95 insertions(+), 61 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/1
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#15).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 173 insertions(+), 83 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/15
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 15
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
Patch Set 19:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/18053/8//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18053/8//COMMIT_MSG@18
PS8, Line 18: --num_tablets_to_open_simultaneously from default 1 to 8
> Can you show the actual numbers? Did you use the same num_tablets_to_open_s
Commit message updated.
Yes I tested both cases, but before this patch, --num_tablets_to_open_simultaneously is not used for _table metadata load_ procedure, the test is meanless.
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tablet/tablet_bootstrap.cc
File src/kudu/tablet/tablet_bootstrap.cc:
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tablet/tablet_bootstrap.cc@826
PS4, Line 826: if (b.term() == a.term() &&
> I don't think this makes the code clearer.
Done
http://gerrit.cloudera.org:8080/#/c/18053/8/src/kudu/tserver/ts_tablet_manager-test.cc
File src/kudu/tserver/ts_tablet_manager-test.cc:
http://gerrit.cloudera.org:8080/#/c/18053/8/src/kudu/tserver/ts_tablet_manager-test.cc@416
PS8, Line 416: const int64_t kTabletCount = FLAGS_startup_benchmark_tablet_count_for_testing;
> Why 8 threads specifically?
I‘ve removed this thread pool, the pool is meanless here.
http://gerrit.cloudera.org:8080/#/c/18053/8/src/kudu/tserver/ts_tablet_manager-test.cc@436
PS8, Line 436: FLAGS_minloglevel = 2;
> Why do we skip opening the tablet? Shouldn't we measure the time it takes t
Because this patch and this benchmark is only for tablet _metadata_ load procedure, the whole tablet bootstrap include tablet metadata load, open, replay wals, consensus, etc. The other procedures will cost much time, but I don't care for this benchmark itself, so I want to skip them.
http://gerrit.cloudera.org:8080/#/c/18053/8/src/kudu/tserver/ts_tablet_manager-test.cc@437
PS8, Line 437: }
> Why are we injecting latency in a benchmark?
I'm going to simulate a bad disk IO latency, to prove this patch's optimization works indeed.
http://gerrit.cloudera.org:8080/#/c/18053/15/src/kudu/tserver/ts_tablet_manager.cc
File src/kudu/tserver/ts_tablet_manager.cc:
http://gerrit.cloudera.org:8080/#/c/18053/15/src/kudu/tserver/ts_tablet_manager.cc@499
PS15, Line 499: for (int i = 0; i < tablet_ids.s
> nit: since 'tablet_ids' is already populated when 'metas' is constructed, i
Done
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 19
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Fri, 03 Dec 2021 07:58:58 +0000
Gerrit-HasComments: Yes
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#10).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 182 insertions(+), 82 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/10
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 10
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#12).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 173 insertions(+), 82 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/12
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 12
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
Patch Set 5:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/18053/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18053/4//COMMIT_MSG@11
PS4, Line 11: - short circuit return when found an active memory store
> nit: wrap these at 80
Done
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/fs/fs_manager.cc
File src/kudu/fs/fs_manager.cc:
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/fs/fs_manager.cc@719
PS4, Line 719: emplace_back(std
> nit: could use the move semantics for 'child', moving strings from one vect
Done
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tablet/tablet_bootstrap.cc
File src/kudu/tablet/tablet_bootstrap.cc:
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tablet/tablet_bootstrap.cc@654
PS4, Line 654: ock
> nit: the prior version was grammatically correct and easy to understand Eng
Done
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc
File src/kudu/tserver/ts_tablet_manager.cc:
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@438
PS4, Line 438:
> nit: an extra line
In fact I want to seperate each thread pool creating code block.
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@489
PS4, Line 489: uccess_loaded_co
> What's difference between a fatal and a non-fatal one? Please add a commen
Done
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@494
PS4, Line 494: seen_error) {
> nit here and below: for STL atomics, it's not necessary to use load() or ot
Almost OK, but not for reference in Substitute(), an error is threw:
error: invalid initialization of reference of type ‘const strings::internal::SubstituteArg&’ from expression of type ‘std::atomic<int>’
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@511
PS4, Line 511: s = s.CloneAndPrepend(Substitute("could not open tablet metadata: $0", tablet_id));
> Does KLOG_EVERY_N/KLOG_EVERY_N_SECS not work here?
they work, I'll use KLOG_EVERY_N_SECS.
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@519
PS4, Line 519:
> Missed '$0'?
Done
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@524
PS4, Line 524:
> Using atomics everywhere and then resorting to serialized/locking access lo
Good idea! I'll do it like that.
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@1416
PS4, Line 1416: deleter->Destroy();
> nit: this comment is too vague -- I'm not sure anybody (even you yourself)
I'll remove it.
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 5
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Wed, 01 Dec 2021 17:06:35 +0000
Gerrit-HasComments: Yes
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#20).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 183 insertions(+), 91 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/20
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 20
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#18).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 183 insertions(+), 90 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/18
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 18
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#16).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 180 insertions(+), 89 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/16
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 16
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#11).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 183 insertions(+), 89 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/11
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 11
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#8).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously=8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 181 insertions(+), 82 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/8
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#13).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 174 insertions(+), 82 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/13
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 13
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#14).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 172 insertions(+), 83 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/14
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 14
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] WIP Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#2).
Change subject: WIP Speedup tablet bootstrap
......................................................................
WIP Speedup tablet bootstrap
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
4 files changed, 95 insertions(+), 61 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/2
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 2
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#6).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time.
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 178 insertions(+), 82 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/6
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
Patch Set 15:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/18053/15/src/kudu/tserver/ts_tablet_manager-test.cc
File src/kudu/tserver/ts_tablet_manager-test.cc:
http://gerrit.cloudera.org:8080/#/c/18053/15/src/kudu/tserver/ts_tablet_manager-test.cc@137
PS15, Line 137: tablet_manager_->WaitForAllBootstrapsToFinish()
What if WaitForAllBootstrapsToFinish() returns non-OK status: should the error be ignored intentionally or this needs to be wrapped into RETURN_NOT_OK? If the former, please add a comment to explain why.
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc
File src/kudu/tserver/ts_tablet_manager.cc:
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@438
PS4, Line 438: max_open_threads = fs_manager_->GetDataRootDirs().size();
> In fact I want to seperate each thread pool creating code block.
Ah, I see: sounds good to me then.
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@494
PS4, Line 494: ED_LOG_TIMING(INFO, Sub
> Almost OK, but not for reference in Substitute(), an error is threw:
Ah, interesting, thanks. Probably, some extra care should be taken for SubstituteArg, but it's a separate issue not related to this patch.
http://gerrit.cloudera.org:8080/#/c/18053/15/src/kudu/tserver/ts_tablet_manager.cc
File src/kudu/tserver/ts_tablet_manager.cc:
http://gerrit.cloudera.org:8080/#/c/18053/15/src/kudu/tserver/ts_tablet_manager.cc@499
PS15, Line 499: metas.resize(tablet_ids.size());
nit: since 'tablet_ids' is already populated when 'metas' is constructed, instead of this the constructor for 'metas' could be updated to avoid re-allocation (see
https://en.cppreference.com/w/cpp/container/vector/vector for details):
vector<scoped_refptr<TabletMetadata>> metas(tablets_ids.size());
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 15
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Thu, 02 Dec 2021 17:23:04 +0000
Gerrit-HasComments: Yes
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#5).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
5 files changed, 103 insertions(+), 65 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/5
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 5
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
Patch Set 21:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18053/8//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18053/8//COMMIT_MSG@18
PS8, Line 18: --num_tablets_to_open_simultaneously from default 1 to 8
> Ah, I guess the idea is that the benchmark is rather a synthetic one, so ac
Yes, the parameter is relevant only for the new implementation, and if set this parameter to 1, it will fall back to single thread like before.
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 21
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Tue, 07 Dec 2021 02:20:00 +0000
Gerrit-HasComments: Yes
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18053/8//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18053/8//COMMIT_MSG@18
PS8, Line 18: --num_tablets_to_open_simultaneously=8
> Yes, the parameter is relevant only for the new implementation, and if set
Thank you for the clarification.
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Tue, 07 Dec 2021 17:50:50 +0000
Gerrit-HasComments: Yes
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#9).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously=8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 171 insertions(+), 82 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/9
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 9
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#21).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 183 insertions(+), 91 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/21
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 21
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#19).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 184 insertions(+), 91 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/19
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 19
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
Patch Set 4:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/18053/4//COMMIT_MSG
Commit Message:
PS4:
Can you add some benchmarks before and after using a release build to the commit message?
http://gerrit.cloudera.org:8080/#/c/18053/4//COMMIT_MSG@11
PS4, Line 11: - short circuit return when found an active memory store in TabletBootstrap::AnalyzeActiveStores()
nit: wrap these at 80
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc
File src/kudu/tserver/ts_tablet_manager.cc:
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@511
PS4, Line 511: if (cur_loaded_count % 200 == 0) {
Does KLOG_EVERY_N/KLOG_EVERY_N_SECS not work here?
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 4
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 30 Nov 2021 23:46:24 +0000
Gerrit-HasComments: Yes
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18053/4//COMMIT_MSG
Commit Message:
PS4:
> +1
Done
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Thu, 02 Dec 2021 03:34:26 +0000
Gerrit-HasComments: Yes
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
Patch Set 8:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/18053/8//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18053/8//COMMIT_MSG@18
PS8, Line 18: --num_tablets_to_open_simultaneously=8
Can you show the actual numbers? Did you use the same num_tablets_to_open_simultaneously for both before and after your changes?
http://gerrit.cloudera.org:8080/#/c/18053/8/src/kudu/tserver/ts_tablet_manager-test.cc
File src/kudu/tserver/ts_tablet_manager-test.cc:
http://gerrit.cloudera.org:8080/#/c/18053/8/src/kudu/tserver/ts_tablet_manager-test.cc@416
PS8, Line 416: .set_max_threads(8)
Why 8 threads specifically?
http://gerrit.cloudera.org:8080/#/c/18053/8/src/kudu/tserver/ts_tablet_manager-test.cc@436
PS8, Line 436: FLAGS_tablet_bootstrap_skip_opening_tablet_for_testing = true;
Why do we skip opening the tablet? Shouldn't we measure the time it takes to fully open the tablets?
http://gerrit.cloudera.org:8080/#/c/18053/8/src/kudu/tserver/ts_tablet_manager-test.cc@437
PS8, Line 437: FLAGS_tablet_metadata_load_inject_latency_ms = 2;
Why are we injecting latency in a benchmark?
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Thu, 02 Dec 2021 08:28:37 +0000
Gerrit-HasComments: Yes
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
Patch Set 4:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/18053/4//COMMIT_MSG
Commit Message:
PS4:
> Can you add some benchmarks before and after using a release build to the c
+1
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/fs/fs_manager.cc
File src/kudu/fs/fs_manager.cc:
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/fs/fs_manager.cc@719
PS4, Line 719: push_back(child)
nit: could use the move semantics for 'child', moving strings from one vector to another to avoid extra memory allocations
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tablet/tablet_bootstrap.cc
File src/kudu/tablet/tablet_bootstrap.cc:
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tablet/tablet_bootstrap.cc@654
PS4, Line 654: and
nit: the prior version was grammatically correct and easy to understand English, changing 'or' to 'and' here brings more confusion than clarity. If you want to clarify, you could change to
... neither blocks nor log segments found ...
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tablet/tablet_bootstrap.cc@826
PS4, Line 826: return b.term() != a.term() || b.index() <= a.index() + 1;
I don't think this makes the code clearer.
As for the optimizations: wouldn't the compiler be able to do the right thing itself? I'm curious: did you compare the assembly to justify sacrificing the readability for any non-negligible performance gain?
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc
File src/kudu/tserver/ts_tablet_manager.cc:
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@438
PS4, Line 438:
nit: an extra line
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@489
PS4, Line 489: seen_fatal_error
What's difference between a fatal and a non-fatal one? Please add a comment to clarify or just rename 'xxx_fatal_error' --> 'xxx_error' below
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@494
PS4, Line 494: seen_fatal_error.load()
nit here and below: for STL atomics, it's not necessary to use load() or other explicit methods -- the implicit conversion operators are supposed to do the right thing
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@519
PS4, Line 519:
Missed '$0'?
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@524
PS4, Line 524: std::lock_guard<simple_spinlock> l(lock);
Using atomics everywhere and then resorting to serialized/locking access looks a bit strange. Instead, what if allocate 'metas' with empty/null elements and pass the index/pointer to the particular element to each thread so it could replace a dedicated element its particular meta?
http://gerrit.cloudera.org:8080/#/c/18053/4/src/kudu/tserver/ts_tablet_manager.cc@1416
PS4, Line 1416: // TODO(yingchun): use macro to simplify code
nit: this comment is too vague -- I'm not sure anybody (even you yourself) in a few weeks from now will be able to decipher this; so please either add more details for the prescribed action or remove this comment
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 4
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Wed, 01 Dec 2021 03:04:53 +0000
Gerrit-HasComments: Yes
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
Patch Set 21:
> LGTM: I think it's worth it getting an approval from Attila as well.
LGTM as well, thank you for the contribution Yingchun
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 21
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Mon, 06 Dec 2021 21:58:00 +0000
Gerrit-HasComments: No
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
Patch Set 21: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 21
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Mon, 06 Dec 2021 21:57:21 +0000
Gerrit-HasComments: No
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
Patch Set 21: Code-Review+2
(1 comment)
Thank you very much for the contribution!
LGTM: I think it's worth it getting an approval from Attila as well.
http://gerrit.cloudera.org:8080/#/c/18053/8//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18053/8//COMMIT_MSG@18
PS8, Line 18: --num_tablets_to_open_simultaneously from default 1 to 8
> Commit message updated.
Ah, I guess the idea is that the benchmark is rather a synthetic one, so actual numbers might be not so relevant here, right?
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 21
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Mon, 06 Dec 2021 21:38:34 +0000
Gerrit-HasComments: Yes
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#17).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 182 insertions(+), 89 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/17
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 17
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#7).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time.
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 181 insertions(+), 82 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/7
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 7
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18053 )
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store
in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE
on hot path
A simple startup benchmark has been added, it shows that this patch can
reduce about 85% tablets metadata load time when set
--num_tablets_to_open_simultaneously from default 1 to 8
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Reviewed-on: http://gerrit.cloudera.org:8080/18053
Tested-by: Kudu Jenkins
Reviewed-by: Alexey Serbin <as...@cloudera.com>
Reviewed-by: Attila Bukor <ab...@apache.org>
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tserver/ts_tablet_manager-test.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
7 files changed, 183 insertions(+), 91 deletions(-)
Approvals:
Kudu Jenkins: Verified
Alexey Serbin: Looks good to me, approved
Attila Bukor: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 22
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
[kudu-CR] WIP Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#3).
Change subject: WIP Speedup tablet bootstrap
......................................................................
WIP Speedup tablet bootstrap
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
4 files changed, 95 insertions(+), 65 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/3
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [bootstrap] Speedup tablet bootstrap
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18053
to look at the new patch set (#4).
Change subject: [bootstrap] Speedup tablet bootstrap
......................................................................
[bootstrap] Speedup tablet bootstrap
Speedup tablet bootstrap by
- using multi-threads to load tablets metadata
- short circuit return when found an active memory store in TabletBootstrap::AnalyzeActiveStores()
- improving branch prediction by adding more PREDICT_FALSE/TRUE on hot path
Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
---
M src/kudu/fs/fs_manager.cc
M src/kudu/tablet/tablet_bootstrap.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
4 files changed, 95 insertions(+), 65 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/53/18053/4
--
To view, visit http://gerrit.cloudera.org:8080/18053
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I816417d2d4c24014edb6b2a40c060f29e37ae219
Gerrit-Change-Number: 18053
Gerrit-PatchSet: 4
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)