You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Yan-Daojiang (Code Review)" <ge...@cloudera.org> on 2026/04/23 10:22:27 UTC
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Yan-Daojiang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24227
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
[server] aggregate sub-step progress for "Reading filesystem"
The /startup page's "Reading filesystem" parent step is driven by a
single Timer that is started before InitAndOpenBlockManager() and
stopped after it returns. Its rendered percentage therefore flips
straight from 0% to 100%, even though the underlying work (especially
LogBlockManager::Open()'s per-container LoadContainer() pass) can take
many minutes on tablet servers with hundreds of thousands of log block
containers. During that window the UI is misleading: a sibling sub-step
"Opening container files" already shows 100% (see Change-Id: Ia372ad5),
but the parent stays at 0%, suggesting nothing is happening.
This change derives an intermediate percentage for the parent step
from its sub-steps:
* "Reading instance metadata files" (small, fixed weight: 5%)
* "Reading data directories" (dominant weight: 95%, driven
by the existing containers_processed / containers_total counters
when the log block manager is in use)
The aggregate is clamped to [0, 99] while the parent Timer is still
running so that the parent only transitions to 100% when its own
Timer is stopped (matching the existing semantics for downstream
metrics like startup_progress_steps_remaining).
Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
---
M src/kudu/master/master-test.cc
M src/kudu/server/CMakeLists.txt
A src/kudu/server/startup_path_handler-test.cc
M src/kudu/server/startup_path_handler.cc
M src/kudu/server/startup_path_handler.h
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/util/timer.h
7 files changed, 235 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/24227/1
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 1
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Yan-Daojiang (Code Review)" <ge...@cloudera.org>.
Yan-Daojiang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24227 )
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
Patch Set 3:
(8 comments)
Thanks for the review! I've addressed the comments. Please take another look when you have time.
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/CMakeLists.txt
File src/kudu/server/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/CMakeLists.txt@93
PS2, Line 93: ADD_KUDU_TEST(rpc_server-test)
: ADD_KUDU_TEST(startup_path_handler-test)
: ADD_KUDU_TEST(webserver-test)
> nit: please re-order these alphabetically
Done
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler-test.cc
File src/kudu/server/startup_path_handler-test.cc:
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler-test.cc@74
PS2, Line 74: ReadFilesystemNotStarted) {
> nit for here and elsewhere: drop the 'Test' prefix, otherwise the full name
Done
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler-test.cc@75
PS2, Line 75: ASSERT_EQ
> Using EXPECT_XXX doesn't make much sense unless there is a need to continue
Done
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler-test.cc@108
PS2, Line 108: }
:
> What are these two lines for if there is line 107 already?
Removed the redundant checks.
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler-test.cc@125
PS2, Line 125: ASSERT_EQ(100, GetReadFilesystemStatus());
> Does it make sense to add a check to make sure it's 100 after the parent ti
Done
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler-test.cc@145
PS2, Line 145: ASSERT_EQ(99, GetReadFilesystemStatus());
> ditto: consider adding verification it's 100% after the parent timer stops.
Done
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler.h
File src/kudu/server/startup_path_handler.h:
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler.h@63
PS2, Line 63: the [0,
> nit: [0, 99] range
Done
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler.cc
File src/kudu/server/startup_path_handler.cc:
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler.cc@109
PS2, Line 109: ead_data_directories", read_
> here and elsewhere: I'd think that fetching the value with the most relaxed
Done
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 3
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
Gerrit-Comment-Date: Fri, 24 Apr 2026 09:48:44 +0000
Gerrit-HasComments: Yes
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Yan-Daojiang (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24227
to look at the new patch set (#2).
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
[server] aggregate sub-step progress for "Reading filesystem"
The /startup page's "Reading filesystem" parent step is driven by a
single Timer that is started before InitAndOpenBlockManager() and
stopped after it returns. Its rendered percentage therefore flips
straight from 0% to 100%, even though the underlying work (especially
LogBlockManager::Open()'s per-container LoadContainer() pass) can take
many minutes on tablet servers with hundreds of thousands of log block
containers. During that window the UI is misleading: a sibling sub-step
"Opening container files" already shows 100% (see Change-Id: Ia372ad5),
but the parent stays at 0%, suggesting nothing is happening.
This change derives an intermediate percentage for the parent step
from its sub-steps:
* "Reading instance metadata files" (small, fixed weight: 5%)
* "Reading data directories" (dominant weight: 95%, driven
by the existing containers_processed / containers_total counters
when the log block manager is in use)
The aggregate is clamped to [0, 99] while the parent Timer is still
running so that the parent only transitions to 100% when its own
Timer is stopped (matching the existing semantics for downstream
metrics like startup_progress_steps_remaining).
Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
---
M src/kudu/master/master-test.cc
M src/kudu/server/CMakeLists.txt
A src/kudu/server/startup_path_handler-test.cc
M src/kudu/server/startup_path_handler.cc
M src/kudu/server/startup_path_handler.h
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/util/timer.h
7 files changed, 235 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/24227/2
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 2
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24227 )
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
Patch Set 1: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/2509/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 1
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
Gerrit-Comment-Date: Thu, 23 Apr 2026 11:19:12 +0000
Gerrit-HasComments: No
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24227 )
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
Patch Set 3: Verified+1
Build Successful
http://jenkins.kudu.apache.org/job/pre_commit/2516/ : SUCCESS
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 3
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
Gerrit-Comment-Date: Fri, 24 Apr 2026 10:29:06 +0000
Gerrit-HasComments: No
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24227 )
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
Patch Set 2:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/2511/
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 2
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
Gerrit-Comment-Date: Thu, 23 Apr 2026 11:21:50 +0000
Gerrit-HasComments: No
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24227 )
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
[server] aggregate sub-step progress for "Reading filesystem"
The /startup page's "Reading filesystem" parent step is driven by a
single Timer that is started before InitAndOpenBlockManager() and
stopped after it returns. Its rendered percentage therefore flips
straight from 0% to 100%, even though the underlying work (especially
LogBlockManager::Open()'s per-container LoadContainer() pass) can take
many minutes on tablet servers with hundreds of thousands of log block
containers. During that window the UI is misleading: a sibling sub-step
"Opening container files" already shows 100% (see Change-Id: Ia372ad5),
but the parent stays at 0%, suggesting nothing is happening.
This change derives an intermediate percentage for the parent step
from its sub-steps:
* "Reading instance metadata files" (small, fixed weight: 5%)
* "Reading data directories" (dominant weight: 95%, driven
by the existing containers_processed / containers_total counters
when the log block manager is in use)
The aggregate is clamped to [0, 99] while the parent Timer is still
running so that the parent only transitions to 100% when its own
Timer is stopped (matching the existing semantics for downstream
metrics like startup_progress_steps_remaining).
Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Reviewed-on: http://gerrit.cloudera.org:8080/24227
Tested-by: Kudu Jenkins
Reviewed-by: Alexey Serbin <al...@apache.org>
---
M src/kudu/master/master-test.cc
M src/kudu/server/CMakeLists.txt
A src/kudu/server/startup_path_handler-test.cc
M src/kudu/server/startup_path_handler.cc
M src/kudu/server/startup_path_handler.h
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/util/timer.h
7 files changed, 250 insertions(+), 11 deletions(-)
Approvals:
Kudu Jenkins: Verified
Alexey Serbin: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 4
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24227 )
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
Patch Set 3:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/2516/
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 3
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
Gerrit-Comment-Date: Fri, 24 Apr 2026 09:35:14 +0000
Gerrit-HasComments: No
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/24227 )
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
Patch Set 1: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24227/1/src/kudu/server/startup_path_handler.cc
File src/kudu/server/startup_path_handler.cc:
http://gerrit.cloudera.org:8080/#/c/24227/1/src/kudu/server/startup_path_handler.cc@201
PS1, Line 201: // FBM has no per-container signal; leave at 0 until the sub-step stops.
The comment "FBM has no per-container signal; leave at 0 until the sub-step stops."
is inside the LBM branch (`else if (... && is_using_lbm_)`), but it describes the FBM
fall-through (why we don't enter the branch). Should this be repositioned or rephrased to
describe what this branch actually does, e.g. "Use per-container counters to interpolate
LBM progress."?
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 1
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
Gerrit-Comment-Date: Thu, 23 Apr 2026 11:10:28 +0000
Gerrit-HasComments: Yes
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/24227 )
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
Patch Set 2:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/CMakeLists.txt
File src/kudu/server/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/CMakeLists.txt@93
PS2, Line 93: ADD_KUDU_TEST(rpc_server-test)
: ADD_KUDU_TEST(webserver-test)
: ADD_KUDU_TEST(startup_path_handler-test)
nit: please re-order these alphabetically
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler-test.cc
File src/kudu/server/startup_path_handler-test.cc:
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler-test.cc@74
PS2, Line 74: TestReadFilesystemNotStarted
nit for here and elsewhere: drop the 'Test' prefix, otherwise the full name of the scenario is already StartupPathHandlerTest.TestReadFilesystemNotStarted
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler-test.cc@75
PS2, Line 75: EXPECT_EQ
Using EXPECT_XXX doesn't make much sense unless there is a need to continue the test even if some of the EXPECT fail. If it's a single check in a scope, then it not different from ASSERT_XXX.
Consider switching to ASSERT_XXX here and elsewhere.
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler-test.cc@108
PS2, Line 108: EXPECT_GT(status, 0);
: EXPECT_LT(status, 100);
What are these two lines for if there is line 107 already?
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler-test.cc@125
PS2, Line 125: }
Does it make sense to add a check to make sure it's 100 after the parent timer stops?
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler-test.cc@145
PS2, Line 145: }
ditto: consider adding verification it's 100% after the parent timer stops.
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler.h
File src/kudu/server/startup_path_handler.h:
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler.h@63
PS2, Line 63: [0, 99]
nit: [0, 99] range
?
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler.cc
File src/kudu/server/startup_path_handler.cc:
http://gerrit.cloudera.org:8080/#/c/24227/2/src/kudu/server/startup_path_handler.cc@109
PS2, Line 109: containers_processed_.load()
here and elsewhere: I'd think that fetching the value with the most relaxed memory ordering would be enough here
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 2
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
Gerrit-Comment-Date: Fri, 24 Apr 2026 08:25:57 +0000
Gerrit-HasComments: Yes
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24227 )
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
Patch Set 1:
Build Started http://jenkins.kudu.apache.org/job/pre_commit/2509/
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 1
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 23 Apr 2026 10:22:35 +0000
Gerrit-HasComments: No
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24227 )
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
Patch Set 2: Verified-1
Build Failed
http://jenkins.kudu.apache.org/job/pre_commit/2511/ : FAILURE
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 2
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
Gerrit-Comment-Date: Thu, 23 Apr 2026 12:12:18 +0000
Gerrit-HasComments: No
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Yan-Daojiang (Code Review)" <ge...@cloudera.org>.
Yan-Daojiang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24227 )
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
Patch Set 2:
(1 comment)
Good catch, done.
http://gerrit.cloudera.org:8080/#/c/24227/1/src/kudu/server/startup_path_handler.cc
File src/kudu/server/startup_path_handler.cc:
http://gerrit.cloudera.org:8080/#/c/24227/1/src/kudu/server/startup_path_handler.cc@201
PS1, Line 201: // Interpolate LBM progress from the per-container counters.
> The comment "FBM has no per-container signal; leave at 0 until the sub-step
Done
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 2
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
Gerrit-Comment-Date: Thu, 23 Apr 2026 11:23:40 +0000
Gerrit-HasComments: Yes
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Yan-Daojiang (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24227
to look at the new patch set (#3).
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
[server] aggregate sub-step progress for "Reading filesystem"
The /startup page's "Reading filesystem" parent step is driven by a
single Timer that is started before InitAndOpenBlockManager() and
stopped after it returns. Its rendered percentage therefore flips
straight from 0% to 100%, even though the underlying work (especially
LogBlockManager::Open()'s per-container LoadContainer() pass) can take
many minutes on tablet servers with hundreds of thousands of log block
containers. During that window the UI is misleading: a sibling sub-step
"Opening container files" already shows 100% (see Change-Id: Ia372ad5),
but the parent stays at 0%, suggesting nothing is happening.
This change derives an intermediate percentage for the parent step
from its sub-steps:
* "Reading instance metadata files" (small, fixed weight: 5%)
* "Reading data directories" (dominant weight: 95%, driven
by the existing containers_processed / containers_total counters
when the log block manager is in use)
The aggregate is clamped to [0, 99] while the parent Timer is still
running so that the parent only transitions to 100% when its own
Timer is stopped (matching the existing semantics for downstream
metrics like startup_progress_steps_remaining).
Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
---
M src/kudu/master/master-test.cc
M src/kudu/server/CMakeLists.txt
A src/kudu/server/startup_path_handler-test.cc
M src/kudu/server/startup_path_handler.cc
M src/kudu/server/startup_path_handler.h
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/util/timer.h
7 files changed, 250 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/24227/3
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 3
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
[kudu-CR] [server] aggregate sub-step progress for "Reading filesystem"
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/24227 )
Change subject: [server] aggregate sub-step progress for "Reading filesystem"
......................................................................
Patch Set 3: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/24227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id658c9da1399c8a05f28a8a0e7d28a2c81a205c1
Gerrit-Change-Number: 24227
Gerrit-PatchSet: 3
Gerrit-Owner: Yan-Daojiang <ya...@163.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Yan-Daojiang <ya...@163.com>
Gerrit-Comment-Date: Sat, 25 Apr 2026 01:11:07 +0000
Gerrit-HasComments: No