You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Abhishek Chennaka (Code Review)" <ge...@cloudera.org> on 2021/10/19 04:10:27 UTC

[kudu-CR] KUDU-1959 - Implement startup progress metrics related to containers and tablets 

Abhishek Chennaka has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17947


Change subject: KUDU-1959 - Implement startup progress metrics related to containers and tablets
......................................................................

KUDU-1959 - Implement startup progress metrics related to containers and tablets

This patch implements the metrics related to the server startup.
* In case of log block manager, we expose:
  - log_block_manager_total_containers_startup : total containers present,
  - log_block_manager_processed_containers_startup : count of containers
    opened/processed until the requested instant of time and
  - log_block_manager_containers_processing_time_startup : time elapsed
    for opening the containers. If the containers are not yet opened, we
    provide the time elapsed so far.
* In case of tablet server, we expose:
  - tablets_num_total_startup : total tablets present,
  - tablets_num_opened_startup : count of tablets opening/processed until
    the requested instant of time and
  - tablets_opening_time_startup : time elapsed for opening the tablets.
    If the tablets are not yet opened, we provide the time elapsed so
    far.

All the times are in milliseconds and the time metrics are in debug
level.

Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
4 files changed, 66 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/47/17947/1
-- 
To view, visit http://gerrit.cloudera.org:8080/17947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
Gerrit-Change-Number: 17947
Gerrit-PatchSet: 1
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>

[kudu-CR] KUDU-1959 - Implement startup progress metrics related to containers and tablets

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17947 )

Change subject: KUDU-1959 - Implement startup progress metrics related to containers and tablets
......................................................................


Patch Set 4: Verified+1 Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/17947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
Gerrit-Change-Number: 17947
Gerrit-PatchSet: 4
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 03 Nov 2021 06:57:33 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-1959 - Implement startup progress metrics related to containers and tablets

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17947 )

Change subject: KUDU-1959 - Implement startup progress metrics related to containers and tablets
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/17947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
Gerrit-Change-Number: 17947
Gerrit-PatchSet: 3
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 02 Nov 2021 20:40:30 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-1959 - Implement startup progress metrics related to containers and tablets

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17947 )

Change subject: KUDU-1959 - Implement startup progress metrics related to containers and tablets
......................................................................

KUDU-1959 - Implement startup progress metrics related to containers and tablets

This patch implements the metrics related to the server startup.
* In case of log block manager, we expose:
  - log_block_manager_total_containers_startup : total containers present,
  - log_block_manager_processed_containers_startup : count of containers
    opened/processed until the requested instant of time and
  - log_block_manager_containers_processing_time_startup : time elapsed
    for opening the containers. If the containers are not yet opened, we
    provide the time elapsed so far.
* In case of tablet server, we expose:
  - tablets_num_total_startup : total tablets present,
  - tablets_num_opened_startup : count of tablets opened/processed until
    the requested instant of time and
  - tablets_opening_time_startup : time elapsed for opening the tablets.
    If the tablets are not yet opened, we provide the time elapsed so
    far.

All the times are in milliseconds and the time metrics are in debug
level.

Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
Reviewed-on: http://gerrit.cloudera.org:8080/17947
Reviewed-by: Andrew Wong <aw...@cloudera.com>
Tested-by: Andrew Wong <aw...@cloudera.com>
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
4 files changed, 67 insertions(+), 0 deletions(-)

Approvals:
  Andrew Wong: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/17947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
Gerrit-Change-Number: 17947
Gerrit-PatchSet: 5
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] KUDU-1959 - Implement startup progress metrics related to containers and tablets

Posted by "Abhishek Chennaka (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Andrew Wong, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/17947

to look at the new patch set (#2).

Change subject: KUDU-1959 - Implement startup progress metrics related to containers and tablets
......................................................................

KUDU-1959 - Implement startup progress metrics related to containers and tablets

This patch implements the metrics related to the server startup.
* In case of log block manager, we expose:
  - log_block_manager_total_containers_startup : total containers present,
  - log_block_manager_processed_containers_startup : count of containers
    opened/processed until the requested instant of time and
  - log_block_manager_containers_processing_time_startup : time elapsed
    for opening the containers. If the containers are not yet opened, we
    provide the time elapsed so far.
* In case of tablet server, we expose:
  - tablets_num_total_startup : total tablets present,
  - tablets_num_opened_startup : count of tablets opened/processed until
    the requested instant of time and
  - tablets_opening_time_startup : time elapsed for opening the tablets.
    If the tablets are not yet opened, we provide the time elapsed so
    far.

All the times are in milliseconds and the time metrics are in debug
level.

Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
---
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
4 files changed, 67 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/47/17947/2
-- 
To view, visit http://gerrit.cloudera.org:8080/17947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
Gerrit-Change-Number: 17947
Gerrit-PatchSet: 2
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] KUDU-1959 - Implement startup progress metrics related to containers and tablets

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has removed a vote on this change.

Change subject: KUDU-1959 - Implement startup progress metrics related to containers and tablets
......................................................................


Removed Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/17947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
Gerrit-Change-Number: 17947
Gerrit-PatchSet: 4
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] KUDU-1959 - Implement startup progress metrics related to containers and tablets

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17947 )

Change subject: KUDU-1959 - Implement startup progress metrics related to containers and tablets
......................................................................


Patch Set 3: Verified+1

The test failure looks unrelated.

TxnCommitITest.TestLoadTxnStatusManagerWhenNoMasters


-- 
To view, visit http://gerrit.cloudera.org:8080/17947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
Gerrit-Change-Number: 17947
Gerrit-PatchSet: 3
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 02 Nov 2021 20:38:28 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-1959 - Implement startup progress metrics related to containers and tablets

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17947 )

Change subject: KUDU-1959 - Implement startup progress metrics related to containers and tablets
......................................................................


Patch Set 4:

I'll look at the test failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/17947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
Gerrit-Change-Number: 17947
Gerrit-PatchSet: 4
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 03 Nov 2021 06:57:42 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-1959 - Implement startup progress metrics related to containers and tablets

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17947 )

Change subject: KUDU-1959 - Implement startup progress metrics related to containers and tablets
......................................................................


Patch Set 1: Code-Review+2

(1 comment)

LGTM, barring one nit, assuming tests will come later (or at least that you've manually verified these).

http://gerrit.cloudera.org:8080/#/c/17947/1/src/kudu/fs/log_block_manager.cc
File src/kudu/fs/log_block_manager.cc:

http://gerrit.cloudera.org:8080/#/c/17947/1/src/kudu/fs/log_block_manager.cc@163
PS1, Line 163: s"
It's probably worth mentioning that this is value is only for containers that exist when the LBM is first starting up.



-- 
To view, visit http://gerrit.cloudera.org:8080/17947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
Gerrit-Change-Number: 17947
Gerrit-PatchSet: 1
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 29 Oct 2021 04:13:46 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1959 - Implement startup progress metrics related to containers and tablets

Posted by "Abhishek Chennaka (Code Review)" <ge...@cloudera.org>.
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/17947 )

Change subject: KUDU-1959 - Implement startup progress metrics related to containers and tablets
......................................................................


Patch Set 1:

(1 comment)

Yes, the tests patch for tablet servers is going to have a test for these metrics

http://gerrit.cloudera.org:8080/#/c/17947/1/src/kudu/fs/log_block_manager.cc
File src/kudu/fs/log_block_manager.cc:

http://gerrit.cloudera.org:8080/#/c/17947/1/src/kudu/fs/log_block_manager.cc@163
PS1, Line 163: s"
> It's probably worth mentioning that this is value is only for containers th
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/17947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
Gerrit-Change-Number: 17947
Gerrit-PatchSet: 1
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 02 Nov 2021 03:38:01 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1959 - Implement startup progress metrics related to containers and tablets

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has removed a vote on this change.

Change subject: KUDU-1959 - Implement startup progress metrics related to containers and tablets
......................................................................


Removed Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/17947
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I9d1aa85b0585214475a6bdb8c0e5d7343c5bc3c9
Gerrit-Change-Number: 17947
Gerrit-PatchSet: 3
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)